From patchwork Sun Jun 6 12:30:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sandor Bodo-Merle X-Patchwork-Id: 12302017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59BD0C47096 for ; Sun, 6 Jun 2021 12:32:51 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2759C6127C for ; Sun, 6 Jun 2021 12:32:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2759C6127C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=Yj6BH8N5GiLkX6xyuPrRQHCnsdplv0kO5amL6z2gIqM=; b=WUS/iosYHGSpTS 0hkwR2UYhYk5C+5j+DmbT5YnjMOAouS6Vd6AgeVUyLzeJ/R2x3d7pvT5HgwFXodzMran9/UPUs4rP 6kbZ+xBdoYwaEYmqHNicF6fGQGJ28kjMcXiN9hxprsDhWMKn8vyx3oep+CAfoLTC7UdCo7/ybD6Tm bUQj/C1z5sJSYmoWlVuLsZ3O3W5bbWcN07rUNdiT0wAcaoNmi9bD4AYn4KyRa1TfFjvmrJt4YpD1E vG6arqBn8DrEqoYNrhU9k3ExLM/70DwCYKD7mwS3UWgB/SgpaL77Tgq3YdO0p3dO2nlnHFoYv4BHD hHF3MuLR8UJfZ4s88Trg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lprw0-000Hev-Td; Sun, 06 Jun 2021 12:31:21 +0000 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lprvv-000HeS-Ts for linux-arm-kernel@lists.infradead.org; Sun, 06 Jun 2021 12:31:18 +0000 Received: by mail-wr1-x436.google.com with SMTP id i94so9165600wri.4 for ; Sun, 06 Jun 2021 05:31:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ngEb9x/2utKfD/oQZnoIJU7hrPHYhp9F52zqoVWgx2Q=; b=ln3LRqC6tLdxL+1LU08oPx52dfBqDds6fq3/rr4qQA4cUSbe0SzLnE6+4MMNXd2bH7 Qnro11dvjRucyBvvWclLmMRQVoy5WDhwtt/OqzhoEIY1dEsH7smnEtn0f7pTXf1EDelr bdriH02MC6Ksuck9Hfj44uZDAXf8mAdYqbyzzOpKYh/S43xhCKVpdxSH4P+BrRBS/6Do enX4qeCZCOpfw5ix6q/5rv2yY2WZlr3CXEOXC11OtYhhL7+vI8MWGvF8REk2R/g9ZXRX aWAaSSr2v8Wdop1Tq0Wxs6MwasZZMDBN82qPVc2E1pUHTPAueVOZeCKz1XOTzUMSO/YR Frrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ngEb9x/2utKfD/oQZnoIJU7hrPHYhp9F52zqoVWgx2Q=; b=smOVlbUcrCZ2Ty2Iyd0js2c8+X6DMTHBDTIUfa1yOJ4xpAEUGgPNw0u0bjGxHhOg8W GJEAWuqPR6T9JiWQmXsKVrNSG/geT38ShrUBKZ0D0oqxB8dZAJ8ht0+5hBrqLfkM4bNm J/rgbQNDgY/5ij9RMVKRiX85ovsaqVGfu50ykLcbXItPG6GW8qbH+1QYAUx7P/yrIROy VRimD0jT1TH5A7hJzs7hGX5zhHCmCB8KO7FaYoswLG1YQmLX5Gykd+D7oyg/FiRFuPvQ Gfmfs/sSGnn2IyeaINE3CWf4i7q9/K43JINSFVAy7pa5yeN9NCeV/pYMU91oX+ZjWWoX /KOg== X-Gm-Message-State: AOAM530wkbinAhm0oKbpP0Jw/2FniropGMJPlAXJHHaA6TWRA/O37gVV qcI15zD/bY4FIMUo6jCfQ90= X-Google-Smtp-Source: ABdhPJxWiQPwTbCyI7h2PehlUVgYVecZGrRQnXC+VdJYC3TS78WhRModc4mvqS8dbMGWjdUi2qYDKw== X-Received: by 2002:adf:fd90:: with SMTP id d16mr12536436wrr.35.1622982674405; Sun, 06 Jun 2021 05:31:14 -0700 (PDT) Received: from snuff.lan (178-164-181-11.pool.digikabel.hu. [178.164.181.11]) by smtp.gmail.com with ESMTPSA id p5sm12922023wrd.25.2021.06.06.05.31.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Jun 2021 05:31:14 -0700 (PDT) From: Sandor Bodo-Merle To: Lorenzo Pieralisi , Rob Herring , =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= , Bjorn Helgaas , Ray Jui , Scott Branden , bcm-kernel-feedback-list@broadcom.com, Sandor Bodo-Merle , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: =?utf-8?q?Pali_Roh=C3=A1r?= Subject: [PATCH 1/2] PCI: iproc: fix the base vector number allocation for multi-MSI Date: Sun, 6 Jun 2021 14:30:43 +0200 Message-Id: <20210606123044.31250-1-sbodomerle@gmail.com> X-Mailer: git-send-email 2.31.0 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210606_053115_987256_83E1FB78 X-CRM114-Status: GOOD ( 14.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Commit fc54bae28818 ("PCI: iproc: Allow allocation of multiple MSIs") introduced multi-MSI support with a broken allocation mechanism (it failed to reserve the proper number of bits from the inner domain). Natural alignment of the base vector number was also not guaranteed. Fixes: fc54bae28818 ("PCI: iproc: Allow allocation of multiple MSIs") Reported-by: Pali Rohár Signed-off-by: Sandor Bodo-Merle Acked-by: Marc Zyngier Acked-by: Pali Rohár Acked-by: Ray Jui --- drivers/pci/controller/pcie-iproc-msi.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/pci/controller/pcie-iproc-msi.c b/drivers/pci/controller/pcie-iproc-msi.c index eede4e8f3f75..557d93dcb3bc 100644 --- a/drivers/pci/controller/pcie-iproc-msi.c +++ b/drivers/pci/controller/pcie-iproc-msi.c @@ -252,18 +252,18 @@ static int iproc_msi_irq_domain_alloc(struct irq_domain *domain, mutex_lock(&msi->bitmap_lock); - /* Allocate 'nr_cpus' number of MSI vectors each time */ - hwirq = bitmap_find_next_zero_area(msi->bitmap, msi->nr_msi_vecs, 0, - msi->nr_cpus, 0); - if (hwirq < msi->nr_msi_vecs) { - bitmap_set(msi->bitmap, hwirq, msi->nr_cpus); - } else { - mutex_unlock(&msi->bitmap_lock); - return -ENOSPC; - } + /* + * Allocate 'nr_irqs' multiplied by 'nr_cpus' number of MSI vectors + * each time + */ + hwirq = bitmap_find_free_region(msi->bitmap, msi->nr_msi_vecs, + order_base_2(msi->nr_cpus * nr_irqs)); mutex_unlock(&msi->bitmap_lock); + if (hwirq < 0) + return -ENOSPC; + for (i = 0; i < nr_irqs; i++) { irq_domain_set_info(domain, virq + i, hwirq + i, &iproc_msi_bottom_irq_chip, @@ -284,7 +284,8 @@ static void iproc_msi_irq_domain_free(struct irq_domain *domain, mutex_lock(&msi->bitmap_lock); hwirq = hwirq_to_canonical_hwirq(msi, data->hwirq); - bitmap_clear(msi->bitmap, hwirq, msi->nr_cpus); + bitmap_release_region(msi->bitmap, hwirq, + order_base_2(msi->nr_cpus * nr_irqs)); mutex_unlock(&msi->bitmap_lock); From patchwork Sun Jun 6 12:30:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sandor Bodo-Merle X-Patchwork-Id: 12302019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C913EC47096 for ; Sun, 6 Jun 2021 12:33:00 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 96D9C6127C for ; Sun, 6 Jun 2021 12:33:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 96D9C6127C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Sk2NQ0jGoWtvgFDeUnoNfDkGyhkyLNajlaiT4uK1bC0=; b=CKIOekbHs0kmpX VIYit6xenVlwhIq0Yko2Vt2VXFHTpcZOsVTbUQBbHIj5Bai7WXw8k2uudTqkf1BYnKW7czdMXNBoh wRxkazb12n0GRrRe56uTbU7gIQfruntAdYls69HGK6dfOnFLt4hDj12FSoc0B8qQ9DMyyzbyfcedx KU3rm8x0tVQDYesMBRRDXe9JNYMnLaBnnLfi0iTnqrBRg+v3o8kcWXZ6s4t6VBdpJ7iyPgSTTbLXX 3Qq8mw1JQFIcQzL/eFHwUlVtvILE7oQkmrbp5Fdz9eh6fYBDDXAbO1/jc3MPpEDOoSTaNIova6w4k aVB1waOkSTsz6bNk6xIw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lprwB-000Hfh-Df; Sun, 06 Jun 2021 12:31:31 +0000 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lprw4-000Hf0-5N for linux-arm-kernel@lists.infradead.org; Sun, 06 Jun 2021 12:31:25 +0000 Received: by mail-wr1-x42b.google.com with SMTP id y7so9569903wrh.7 for ; Sun, 06 Jun 2021 05:31:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kYwRQbli1I4hRkELxP1eiNXBdqKWRZGGQHjz4e56pPA=; b=HxfDQx8n4eFxXVY5t1YWxq4nYBT+yLIpZ821I/YZFbkADzUxOTIkwYVGSQI5knuaVc rNnTzxvSo0yL2eJCNKLHNYFkdtd2DC0H20X5tLu1BYsgmHNeDD8jHt8xbjk+/kWNH/uV ounfRlhyqCjkLf1CXcza8Hif36Q8Lcu/6DfTgDAjrVG4LHmlhoH8eWlxwwc7YzvkOkrP yCVhwJJyiQ46Ifq+51uQnwRP1bAI6v3b8er7U0B+44SZmmhttZAG2F0fbV3GPpSPzGxR NeSwbBnmvCLmwqK8fFTZDM/iYT0mXn0GFE7No59K9+PNWxcqJx64XmOPmL7rCO0U+koC LTRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kYwRQbli1I4hRkELxP1eiNXBdqKWRZGGQHjz4e56pPA=; b=cFEoDSO0ZBewhbFQqbvCf79miQ9LJ2uxmrHlmcLYscqQ8M+yZAB3AynQLTVBUKQtb2 UHv4zdkdtJb7w5juZpp8OAOVDPTHLIRN75zWSgHqbine4eqNXAHekYjYI/mUqrS/i34u NRRnn4Rmfh2e2pRny1hC1Pstke0/Jo8yCnbxHeqqJ8A1MR2o8RPqi2h5pwcSIIQ3YpvE +1Fd+CFeqs+O+FdNU+P5d9/7B63DIUj5nnkihEvjcbPQ80dRK+K/o5nsdonCJfI/RpX1 nS4Q4qja7ieuPgmTHzcdwjEgCCz27M90hP7FK64qCqccgZ5vhUvQFFNQQA1OiMkEJvFN vdZQ== X-Gm-Message-State: AOAM5311oZMNwbOUiSXIWrnYw5N/eKdv9yVmhDXNbkegUE4BNw8kcCJs du8CVJRlMZN7YzDZZQvEYus= X-Google-Smtp-Source: ABdhPJzQh10L6Nw6bzQ+bW4SV/yydOU0a0bTrlgXmmGQbGj5xwgMbTUwDqKzhDgn5JL89gNYgRwY2Q== X-Received: by 2002:adf:f98e:: with SMTP id f14mr12652140wrr.408.1622982682947; Sun, 06 Jun 2021 05:31:22 -0700 (PDT) Received: from snuff.lan (178-164-181-11.pool.digikabel.hu. [178.164.181.11]) by smtp.gmail.com with ESMTPSA id p5sm12922023wrd.25.2021.06.06.05.31.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Jun 2021 05:31:22 -0700 (PDT) From: Sandor Bodo-Merle To: Lorenzo Pieralisi , Rob Herring , =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= , Bjorn Helgaas , Ray Jui , Scott Branden , bcm-kernel-feedback-list@broadcom.com, Sandor Bodo-Merle , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Marc Zyngier Subject: [PATCH 2/2] PCI: iproc: Support multi-MSI only on uniprocessor kernel Date: Sun, 6 Jun 2021 14:30:44 +0200 Message-Id: <20210606123044.31250-2-sbodomerle@gmail.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210606123044.31250-1-sbodomerle@gmail.com> References: <20210606123044.31250-1-sbodomerle@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210606_053124_224579_16A27BBA X-CRM114-Status: GOOD ( 14.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The interrupt affinity scheme used by this driver is incompatible with multi-MSI as it implies moving the doorbell address to that of another MSI group. This isn't possible for multi-MSI, as all the MSIs must have the same doorbell address. As such it is restricted to systems with a single CPU. Fixes: fc54bae28818 ("PCI: iproc: Allow allocation of multiple MSIs") Reported-by: Marc Zyngier Signed-off-by: Sandor Bodo-Merle Acked-by: Marc Zyngier Acked-by: Ray Jui Acked-by: Pali Rohár --- drivers/pci/controller/pcie-iproc-msi.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/pci/controller/pcie-iproc-msi.c b/drivers/pci/controller/pcie-iproc-msi.c index 557d93dcb3bc..81b4effeb130 100644 --- a/drivers/pci/controller/pcie-iproc-msi.c +++ b/drivers/pci/controller/pcie-iproc-msi.c @@ -171,7 +171,7 @@ static struct irq_chip iproc_msi_irq_chip = { static struct msi_domain_info iproc_msi_domain_info = { .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | - MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX, + MSI_FLAG_PCI_MSIX, .chip = &iproc_msi_irq_chip, }; @@ -250,6 +250,9 @@ static int iproc_msi_irq_domain_alloc(struct irq_domain *domain, struct iproc_msi *msi = domain->host_data; int hwirq, i; + if (msi->nr_cpus > 1 && nr_irqs > 1) + return -EINVAL; + mutex_lock(&msi->bitmap_lock); /* @@ -540,6 +543,9 @@ int iproc_msi_init(struct iproc_pcie *pcie, struct device_node *node) mutex_init(&msi->bitmap_lock); msi->nr_cpus = num_possible_cpus(); + if (msi->nr_cpus == 1) + iproc_msi_domain_info.flags |= MSI_FLAG_MULTI_PCI_MSI; + msi->nr_irqs = of_irq_count(node); if (!msi->nr_irqs) { dev_err(pcie->dev, "found no MSI GIC interrupt\n");