From patchwork Tue Jun 22 15:26:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sandor Bodo-Merle X-Patchwork-Id: 12337857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74E39C49EA2 for ; Tue, 22 Jun 2021 15:28:32 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4A960611BF for ; Tue, 22 Jun 2021 15:28:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4A960611BF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+QQ2RpKJSYzVT4AQiSNig6s0QJ0Ka3mZzTpD6lez+QA=; b=yKAipME5RgGM/Z eJFY1Mqyb/Wy4wpyht81v+wkMkhf7tTBhdoITpT2wPH14kNGb9bhjzgaVsPRRF4jHvPbxP1EDm68U 4WLbRGXn8gr1DkQRJYyfJBFXp6PNaDx1LqnwAgnB4E4YFbDZBJQsqs8F/i04xNOsysE0NHmuECWkS /5WMgGIUGjHICMCxSTiwSM0kX9aQf9en9gSPFddiYifZPZ7uWc0JD2qItEZ/saVxDxohU80kUeKyC f7b1KYpg65H4zk7Dy7ogaFCaxQhrs2SfzQQJBxz/v0uKHNJuONPVgY0UowK9pDKnlV7oguRRv778O WGqMZ2yQ/RAJ1lKtSNRg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lviIl-007ggY-Jt; Tue, 22 Jun 2021 15:26:59 +0000 Received: from mail-wm1-x32d.google.com ([2a00:1450:4864:20::32d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lviIg-007ged-IB for linux-arm-kernel@lists.infradead.org; Tue, 22 Jun 2021 15:26:55 +0000 Received: by mail-wm1-x32d.google.com with SMTP id p8-20020a7bcc880000b02901dbb595a9f1so2530706wma.2 for ; Tue, 22 Jun 2021 08:26:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iREvRYd3Jw/uqUSWQKuYJucIkeBg2924CRlJ1YiHk5w=; b=R8uvF2EI8NCwXb8QpW12T3b4wTO8IcQ8jgbHHBiKgF9LGnVUhQJfJtGaN2bAiyglWH F7oTpfRXCp0IStdz/s/K39V/GnL00Fq430cFpwRfBQXeyYqlxV00YJlWZdkQn9HLn3NB R/woi01xmS3k81eF0UJHNYjQbA4xTCcE7hpI0cKTg0DQB066bATaFULTrohoAHUr17GI iKBEdRbgiZ5zbV0Z+PE1xlB3qzcKVsi02oskhaArCjvbsxFTy2rXY1NpdMLC3UJdCvR5 y39d45SpvUvizxRdmARP/8QoYRH6WN9D+FSabI5RwXJ/Egx1cxdKXuvUOljCeAqbWF+o Rl5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iREvRYd3Jw/uqUSWQKuYJucIkeBg2924CRlJ1YiHk5w=; b=DehgHAbRqq5/Km05KLULHI3LpLRT6GUmllH7w7cNOAY6WwNiTNo1Gtx1uV2xqhFHHp 4TQd+czcO+Hd8Pzvj/2ZrcYsQyFPtjVVBKTrugDJ5WnRYrm+Yi9pMHytyryN5ktrPcB6 cKj1HaSmlc1ug7k1CJfKzMRFJMuMAj74fw7K3HJzibOdeWCbJivcXKYP3Cmkn6Z5nk7B SCcImZIgleGD/TM8C945HBH8NElun9CIJOyhECQNG5pZ2EsjxAfRARTnK9i6RNQyy21d idiSZAUaCtHknqmbx6Duun7wW7Md08JaTq60OD7aV4OnrrrAz+F4uwwFZn1OlwLN/HCF bAhA== X-Gm-Message-State: AOAM531fsUt0DpkCNQDunbrdXaB3AugsHcbcOZ9weMF97Wpl5a87lp7h ttIWZTeWvA3CdXebFG1qsdk= X-Google-Smtp-Source: ABdhPJzUklaWfijjyr/QbZe9TxRCC6Rdn+RrwCKY4rd7cIy/g0zxVcLA8G7PDAcZMjIvtxpOJoA6Jw== X-Received: by 2002:a1c:2584:: with SMTP id l126mr5069924wml.83.1624375612387; Tue, 22 Jun 2021 08:26:52 -0700 (PDT) Received: from snuff.lan (84-236-11-56.pool.digikabel.hu. [84.236.11.56]) by smtp.gmail.com with ESMTPSA id l15sm11469907wrt.47.2021.06.22.08.26.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Jun 2021 08:26:51 -0700 (PDT) From: Sandor Bodo-Merle To: Lorenzo Pieralisi , Rob Herring , =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= , Bjorn Helgaas , Ray Jui , Scott Branden , bcm-kernel-feedback-list@broadcom.com, Sandor Bodo-Merle , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: =?utf-8?q?Pali_Roh=C3=A1r?= , Marc Zyngier , Ray Jui Subject: [PATCH v2 1/2] PCI: iproc: Fix multi-MSI base vector number allocation Date: Tue, 22 Jun 2021 17:26:29 +0200 Message-Id: <20210622152630.40842-1-sbodomerle@gmail.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210621144702.GD27516@lpieralisi> References: <20210621144702.GD27516@lpieralisi> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210622_082654_650472_AEF37C82 X-CRM114-Status: GOOD ( 15.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Commit fc54bae28818 ("PCI: iproc: Allow allocation of multiple MSIs") introduced multi-MSI support with a broken allocation mechanism (it failed to reserve the proper number of bits from the inner domain). Natural alignment of the base vector number was also not guaranteed. Fixes: fc54bae28818 ("PCI: iproc: Allow allocation of multiple MSIs") Reported-by: Pali Rohár Signed-off-by: Sandor Bodo-Merle Acked-by: Marc Zyngier Acked-by: Pali Rohár Acked-by: Ray Jui --- drivers/pci/controller/pcie-iproc-msi.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/pci/controller/pcie-iproc-msi.c b/drivers/pci/controller/pcie-iproc-msi.c index eede4e8f3f75..557d93dcb3bc 100644 --- a/drivers/pci/controller/pcie-iproc-msi.c +++ b/drivers/pci/controller/pcie-iproc-msi.c @@ -252,18 +252,18 @@ static int iproc_msi_irq_domain_alloc(struct irq_domain *domain, mutex_lock(&msi->bitmap_lock); - /* Allocate 'nr_cpus' number of MSI vectors each time */ - hwirq = bitmap_find_next_zero_area(msi->bitmap, msi->nr_msi_vecs, 0, - msi->nr_cpus, 0); - if (hwirq < msi->nr_msi_vecs) { - bitmap_set(msi->bitmap, hwirq, msi->nr_cpus); - } else { - mutex_unlock(&msi->bitmap_lock); - return -ENOSPC; - } + /* + * Allocate 'nr_irqs' multiplied by 'nr_cpus' number of MSI vectors + * each time + */ + hwirq = bitmap_find_free_region(msi->bitmap, msi->nr_msi_vecs, + order_base_2(msi->nr_cpus * nr_irqs)); mutex_unlock(&msi->bitmap_lock); + if (hwirq < 0) + return -ENOSPC; + for (i = 0; i < nr_irqs; i++) { irq_domain_set_info(domain, virq + i, hwirq + i, &iproc_msi_bottom_irq_chip, @@ -284,7 +284,8 @@ static void iproc_msi_irq_domain_free(struct irq_domain *domain, mutex_lock(&msi->bitmap_lock); hwirq = hwirq_to_canonical_hwirq(msi, data->hwirq); - bitmap_clear(msi->bitmap, hwirq, msi->nr_cpus); + bitmap_release_region(msi->bitmap, hwirq, + order_base_2(msi->nr_cpus * nr_irqs)); mutex_unlock(&msi->bitmap_lock); From patchwork Tue Jun 22 15:26:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sandor Bodo-Merle X-Patchwork-Id: 12337859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73B35C48BDF for ; Tue, 22 Jun 2021 15:29:12 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4554D60FEB for ; Tue, 22 Jun 2021 15:29:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4554D60FEB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=M0oR13NCYtc0o02C1eK2faqLBoEtBpfGpcD74h0PbvI=; b=WsecNAQqif6dNd haExD8A0YdBDYYI6BIXxei6ORC8JVMt+MnH8w8svfktsowXRmus3hAl9Daz8PFXX981vUiRuAtD1h FSwcRJ1hmHTTphlPltuDMHQmQicWF/0cXcuYC0W310ibxEtBlb+AI0tE0MlHyZ8+UJ+ahRvEv3gXT g+QCOamBwegDl5Z1QKoGveuAlljKSjLvHrTdowHFMFSoZNvE3njI0Ge6Mc07+ZLWGTJIi02LKM+fm WXpDLE9+3rfUeHljwXpEBBeOiQsCMPRmAwS8AE5YQte4gzBzypH5JXiDFxM6w+Qxi05DK/hlBR/fJ yaxfkSKltgHh/nBPoE6Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lviJ2-007gkq-87; Tue, 22 Jun 2021 15:27:16 +0000 Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lviIo-007giO-Pu for linux-arm-kernel@lists.infradead.org; Tue, 22 Jun 2021 15:27:04 +0000 Received: by mail-wm1-x334.google.com with SMTP id c84so13045805wme.5 for ; Tue, 22 Jun 2021 08:27:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=H0fRRUN9hmaR59XL6pJP1fOrRa3SKmuQ9EeViv8hJUQ=; b=bb88ouU3crmfqRKxU+pJjoZiy3AST7BfANO3yReF/O6pWOA7iBqjUsWYoT8ICvzP8r IIjp3xtUiQUpIbWbwXfZLciETKWnqXfdsvEXyQIOt3AL+Vv4KYu+Lsa/6HOnLXk7uL/R BP+5htCixe6hHRmQvIGxl5WuFojOJnUC5aro2E58Jvw2GuwvKmtVdj8ZZIj5TmZzCiAR ai7b5c8YJTWu952PjSVbuJ/kUAyBOkuiMbpmHVYpt7TRTV6zE16DIEvG7G8ssOMR+zbZ HOeZEMd7LmUwEvJ3ezUVJX7O716Yc8Y5Ci9j0draS4b6MkAXcsHXZcw4PltYtSQmrF/e FtlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H0fRRUN9hmaR59XL6pJP1fOrRa3SKmuQ9EeViv8hJUQ=; b=nQ+AsSXxr1mMpOY+ZKwxRJndem1tFrbkoR41Y+VQQf9Nd7Q7Rcb4hyGVdjxNzdRL1c IqH21BFKMV1sSj1LLMCo5Bt3oi8G8ffzdpxYP4BNvXHZYgGpTAbuRDKyy8DFupprb8Nj hQ3dL6pjd8Uwf7k5vVZG2KRVK0wcc26/uEpy1E9o6yqoiHjHMgl4U8AMMsZKly5QECSa IZAuVwL4+jmJH9F4fJwris6riGXdr5jvMBSocp7W1z9Audo2RiEj7rhn+gq6YNuNIoTJ hVB06blW0y8iyw8q8Tyg/1vldnAfnZrmbOfD+R07lqyX6YoucR6anGOBZkg1bOcTX1si t8WA== X-Gm-Message-State: AOAM530UtgMhrLIaVumjMshScEX4oTGXsx6lXS559Gj3+6u1Bq191h9x bB0ott89eHYt2nPn+cEKT4k= X-Google-Smtp-Source: ABdhPJy33SSeYQeH48XF1N0otWdKyEt0GndZA8MdlT5lYAscQku+HNc9OSLWOiH0pqnVbuUIQ+omOg== X-Received: by 2002:a1c:7706:: with SMTP id t6mr5237921wmi.62.1624375621362; Tue, 22 Jun 2021 08:27:01 -0700 (PDT) Received: from snuff.lan (84-236-11-56.pool.digikabel.hu. [84.236.11.56]) by smtp.gmail.com with ESMTPSA id l15sm11469907wrt.47.2021.06.22.08.26.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Jun 2021 08:27:01 -0700 (PDT) From: Sandor Bodo-Merle To: Lorenzo Pieralisi , Rob Herring , =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= , Bjorn Helgaas , Ray Jui , Scott Branden , bcm-kernel-feedback-list@broadcom.com, Sandor Bodo-Merle , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Marc Zyngier , =?utf-8?q?Pali_Roh=C3=A1r?= , Ray Jui Subject: [PATCH v2 2/2] PCI: iproc: Support multi-MSI only on uniprocessor kernel Date: Tue, 22 Jun 2021 17:26:30 +0200 Message-Id: <20210622152630.40842-2-sbodomerle@gmail.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210621144702.GD27516@lpieralisi> References: <20210621144702.GD27516@lpieralisi> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210622_082702_915812_365D34CE X-CRM114-Status: GOOD ( 14.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The interrupt affinity scheme used by this driver is incompatible with multi-MSI as it implies moving the doorbell address to that of another MSI group. This isn't possible for multi-MSI, as all the MSIs must have the same doorbell address. As such it is restricted to systems with a single CPU. Fixes: fc54bae28818 ("PCI: iproc: Allow allocation of multiple MSIs") Reported-by: Marc Zyngier Signed-off-by: Sandor Bodo-Merle Acked-by: Marc Zyngier Acked-by: Pali Rohár Acked-by: Ray Jui --- drivers/pci/controller/pcie-iproc-msi.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/pci/controller/pcie-iproc-msi.c b/drivers/pci/controller/pcie-iproc-msi.c index 557d93dcb3bc..81b4effeb130 100644 --- a/drivers/pci/controller/pcie-iproc-msi.c +++ b/drivers/pci/controller/pcie-iproc-msi.c @@ -171,7 +171,7 @@ static struct irq_chip iproc_msi_irq_chip = { static struct msi_domain_info iproc_msi_domain_info = { .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | - MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX, + MSI_FLAG_PCI_MSIX, .chip = &iproc_msi_irq_chip, }; @@ -250,6 +250,9 @@ static int iproc_msi_irq_domain_alloc(struct irq_domain *domain, struct iproc_msi *msi = domain->host_data; int hwirq, i; + if (msi->nr_cpus > 1 && nr_irqs > 1) + return -EINVAL; + mutex_lock(&msi->bitmap_lock); /* @@ -540,6 +543,9 @@ int iproc_msi_init(struct iproc_pcie *pcie, struct device_node *node) mutex_init(&msi->bitmap_lock); msi->nr_cpus = num_possible_cpus(); + if (msi->nr_cpus == 1) + iproc_msi_domain_info.flags |= MSI_FLAG_MULTI_PCI_MSI; + msi->nr_irqs = of_irq_count(node); if (!msi->nr_irqs) { dev_err(pcie->dev, "found no MSI GIC interrupt\n");