From patchwork Wed Nov 6 17:51:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 13865215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 747B3D59F56 for ; Wed, 6 Nov 2024 17:52:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+2SMCe7Ytd0Dl5sP11f7D2ZEXpMITHR1bV/mX30c/0g=; b=XdN1JJIYoMA+og CnlgX2u/y1YtPLRIuc6UzECd6Ea6j5/WlHxcVWKbZ+AIYoKBTztk6zRAbozXDBQyqhxsHyv9fjk7Y 8aKcGxNY9e1SFHd675Rd9nQ+QEWuYOV96NzEv8dE4BRl9Jkf5CuVKgN3I4/yIDU1iUz1bYigV0lmT Sbn45uiGwWCrV1YvieOdpcmQMe8ULG2osRj9jSb5sRDqrF7p3SH46gKb2e55BG2EtSjv0DAtQNvHJ 7gbCopyy2F+XSz1kuPOer5Bb4gh5FxBqL4mzkeZr/hNZqFrDRAE2xpppTkcG/020OkHonPCTXblHJ j7G4EDpWvjOWmT4s8jjQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t8kCX-00000004FPh-1wza; Wed, 06 Nov 2024 17:52:17 +0000 Received: from mail-ej1-x62b.google.com ([2a00:1450:4864:20::62b]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t8kBP-00000004F52-3tUR for linux-riscv@lists.infradead.org; Wed, 06 Nov 2024 17:51:10 +0000 Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-a9acafdb745so25593966b.0 for ; Wed, 06 Nov 2024 09:51:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1730915466; x=1731520266; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g96IwKhGkFLWIqbkT7NwpHd+U4DZ81t2GIHo2pf1780=; b=GHX8zJVJjJf1tdzw5+0qSgslDy0OoogGliJqwODFTWGgZ4IAJQGuDEi944NtrKk2Au VqeIL4ifUdR43RUU9EdQqR8+QK4cqPFVpwI+4VEMemVW6W9fvkWd0+oElFY1IZISj0Sj 4fI5K54wqxwkABKzuBqzzHr+qij+P/0QRrbbX4Vpn5mLH6eZ9ttQjnFqe8eQkClyGgqf haZRoXqLLHNLsi/EK52zZIreyY6c35cL5Pr5qU1OILSA9Ya3xXH75bBaPAJ68fAumhBJ gHAzbXkeb+M1jSMJPBx2oYMIiUwTcgsgdweThz7rz8Q4398v06BbtrpNOYZ3tC7Utyhq PbDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730915466; x=1731520266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g96IwKhGkFLWIqbkT7NwpHd+U4DZ81t2GIHo2pf1780=; b=j15alA6ix3owlUsg1Fox2wntE/x+a+MHnv3+tX0RxWiaNz+4Shf5hcBhaRAd8VgJkr 9bE4HSStDDqiNH4QEOT1ctLE3YJmRIabCbxew5LB66uibGXxcRSwNqnKgv4PXStG6c5J 2xR/8nTzG5E1vZ35oMHEqwwZqjrKXJe4Jd3fYi0S3y/+RQEpcXDXkreVulp2iGsCRGAL fKWYvaAuYhhqjP1+dhOH03avz2HpUhWwOghbyZOPm3P0+YAAdPBMKjW1sn36HSzB+/SG UvD2GVZ8BjMXPB9WLpUdxCnOM2VLVDZ3a3c1Ln8/6fQ6xd+u0Fh4i+2f0D5aAdqa1FuW fh/A== X-Forwarded-Encrypted: i=1; AJvYcCUnXBO5eJPkMXXsSip+nJKa+lnT96Si87AgdjS7WWesev9GEhUDSTDspn0pySTtZvtWG9gnaSnxnifWgg==@lists.infradead.org X-Gm-Message-State: AOJu0Yz5hWa0LOOfrMNMTtWoiqL6SACr9ppbwnT1Ix1ksbOij9zyZf62 l+U+2EUeE/uFjKGmXt5ir8Ozv5RArvoQrnA38Opv2s9Q+sZw/fb2h8iM+h7Q2Mk= X-Google-Smtp-Source: AGHT+IEFqB7EqZ9/qt6Tv+l9s3WePIJJrBkQGmuE+wise2peEn4wuYWmsni9+gmk+GDjL44CfjLWRg== X-Received: by 2002:a17:907:948d:b0:a9a:597:8cc9 with SMTP id a640c23a62f3a-a9ed50cb67dmr4422366b.12.1730915465830; Wed, 06 Nov 2024 09:51:05 -0800 (PST) Received: from localhost (2001-1ae9-1c2-4c00-20f-c6b4-1e57-7965.ip6.tmcz.cz. [2001:1ae9:1c2:4c00:20f:c6b4:1e57:7965]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9eb16d4d6bsm309041166b.44.2024.11.06.09.51.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Nov 2024 09:51:05 -0800 (PST) From: Andrew Jones To: iommu@lists.linux.dev, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Cc: tjeznach@rivosinc.com, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu Subject: [PATCH 1/2] iommu/riscv: Free irq vectors on pci remove Date: Wed, 6 Nov 2024 18:51:04 +0100 Message-ID: <20241106175102.219923-5-ajones@ventanamicro.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241106175102.219923-4-ajones@ventanamicro.com> References: <20241106175102.219923-4-ajones@ventanamicro.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241106_095108_014681_D6EF3D9D X-CRM114-Status: UNSURE ( 8.71 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org riscv_iommu_pci_probe() calls pci_alloc_irq_vectors() which states pci_free_irq_vectors() must be called on cleanup. Fixes: 68682e9578fb ("iommu/riscv: Add RISC-V IOMMU PCIe device driver") Signed-off-by: Andrew Jones --- drivers/iommu/riscv/iommu-pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/iommu/riscv/iommu-pci.c b/drivers/iommu/riscv/iommu-pci.c index c7a89143014c..25a27e627a0e 100644 --- a/drivers/iommu/riscv/iommu-pci.c +++ b/drivers/iommu/riscv/iommu-pci.c @@ -99,6 +99,7 @@ static void riscv_iommu_pci_remove(struct pci_dev *pdev) struct riscv_iommu_device *iommu = dev_get_drvdata(&pdev->dev); riscv_iommu_remove(iommu); + pci_free_irq_vectors(pdev); } static const struct pci_device_id riscv_iommu_pci_tbl[] = { From patchwork Wed Nov 6 17:51:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 13865216 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 683BCD59F5A for ; Wed, 6 Nov 2024 17:52:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=sNybegm0HTOuUIBRCLeJAJRo9BP8elbsUkBTcBvwzxM=; b=kcyoFG6VHGRdV6 +dVnUue9ayJFVmoxYF0GzHcNbBHzt8qaAQfUUDPoUo00xG0CBGWaayqpj06ht+Ssqge9P9ZnqlMU7 7/xIqGHSBIG1dex3ZixA2Z0himbhtNZvK82QbWNF/sUi9ulsFaZIyBTTrEbUPU/UEw7S0OinfydJb ZiVvKiraXoKDBlPoYEc8U7AZHa5xdpA3a0ZdRJCNZMG70CfBCdtpUDfegbeFsNtLUfZXbiVTn1DmU UsCrz8hNfY8gh/mPHXN3COpq+YkOJ4S66Oa6jROIvFLSXoscRn+NRDxAy2nOEXjtSn9A4yDPmkQDD Ek9oWjelsZNz0UhHqSMQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t8kCY-00000004FPn-0Mho; Wed, 06 Nov 2024 17:52:18 +0000 Received: from mail-ej1-x62a.google.com ([2a00:1450:4864:20::62a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t8kBR-00000004F5e-0ORt for linux-riscv@lists.infradead.org; Wed, 06 Nov 2024 17:51:10 +0000 Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-a9a68480164so723466b.3 for ; Wed, 06 Nov 2024 09:51:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1730915467; x=1731520267; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=49hG3lsHGrJySpAEzyxRduAfSvP7r1BXPqy63OvAnoQ=; b=HyCka+V9LExHAu3iQaDLn/MLH/OBj2bWz6tW72QHGUKfd8+lYb4Lv+6QeeZ2QVnVwL mAiYr1FiZLFAI1cIBQfHjeLISfT5UtEKgh1m1bEqW5mHmVJjnzgU/+3LEhGKQ19nMHo1 /6w2HKG48gdUJVgSRiNyCsTmeLif63Pv5JqecrnXYSkZaDW67650CTMCTN5/+R+wmH1L FS0swwYnYQ5J6eHXdGsNNAsSpTprnsPxfUK5QEDNLtOMeJ/ddm8+UGeSyqIuEpk3+uL8 x0Vi+ETtK2yCM7PxNnr0mz4mwVNAEq2Eaz6mzLNT1onGldkEJYmNO54nf6Z4AmRcdJvA 4cYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730915467; x=1731520267; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=49hG3lsHGrJySpAEzyxRduAfSvP7r1BXPqy63OvAnoQ=; b=QKGwfu+8dOnHuwQrwWFqBCdpzl/HYty1mklvz2lX6HJ5bIWGwi544UtrWMD0jb8aQY nUU3d9oYTOHDAJjiCf+0A+wkvxZRck0DnmM5o8Hddb28XNP2/feSqkl+HgRrDT7+ckAL vfpq6gtky0ahomqp0KyN8ySZ0eIc9I9zxD4CDywswv++zfrHQ3RBLuAhGgpShxuHTofw SxJ5hi2KusnWw77FHTbav2DPVMjRqcx3WlFKzyQtYLIQzYyH2W1mqcrYkzzGPb6a3ywF aIrUI01+QjXRrYIWubq2oj0FdYCwHWtApujSkgEDGveV8/VNOo4n5TN4REsVR7z/x8Vl U+oQ== X-Forwarded-Encrypted: i=1; AJvYcCWqOdVgQdl7TQ2p7ZeR8rN/m2M6t0aOIClD8j0hlr3umZpsRunmeOLYKCz2koafKEHsUDaWxW+wkdpYXw==@lists.infradead.org X-Gm-Message-State: AOJu0Yztd2esKCPRzfP+ssJQAaTnWiZMxkcv1FYeyp/5Svxr2K8gscGc lAuTLXzhLs3mndwqAtt4dfJADFRDbyPDRuLj4SotnUDXKY+fIkUuwpzvOV4TXo4= X-Google-Smtp-Source: AGHT+IFfCwyaClKnReIWxUZd0iEfNRrNS6oO6x2m/KaQtRbS25PxoEe6F/CuTCRBg5p3XO9EVv7f2A== X-Received: by 2002:a17:907:7d93:b0:a9a:8a4:e079 with SMTP id a640c23a62f3a-a9de5edea84mr4205271366b.31.1730915467354; Wed, 06 Nov 2024 09:51:07 -0800 (PST) Received: from localhost (2001-1ae9-1c2-4c00-20f-c6b4-1e57-7965.ip6.tmcz.cz. [2001:1ae9:1c2:4c00:20f:c6b4:1e57:7965]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9eb17d7328sm312443566b.129.2024.11.06.09.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Nov 2024 09:51:06 -0800 (PST) From: Andrew Jones To: iommu@lists.linux.dev, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Cc: tjeznach@rivosinc.com, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu Subject: [PATCH 2/2] iommu/riscv: Add support for platform msi Date: Wed, 6 Nov 2024 18:51:05 +0100 Message-ID: <20241106175102.219923-6-ajones@ventanamicro.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241106175102.219923-4-ajones@ventanamicro.com> References: <20241106175102.219923-4-ajones@ventanamicro.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241106_095109_264280_648AC044 X-CRM114-Status: GOOD ( 19.72 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Apply platform_device_msi_init_and_alloc_irqs() to add support for MSIs when the IOMMU is a platform device. Signed-off-by: Andrew Jones --- drivers/iommu/riscv/iommu-platform.c | 102 ++++++++++++++++++++++----- 1 file changed, 84 insertions(+), 18 deletions(-) diff --git a/drivers/iommu/riscv/iommu-platform.c b/drivers/iommu/riscv/iommu-platform.c index da336863f152..89aa622bcbde 100644 --- a/drivers/iommu/riscv/iommu-platform.c +++ b/drivers/iommu/riscv/iommu-platform.c @@ -11,18 +11,41 @@ */ #include +#include #include #include +#include #include "iommu-bits.h" #include "iommu.h" +static void riscv_iommu_write_msi_msg(struct msi_desc *desc, struct msi_msg *msg) +{ + struct device *dev = msi_desc_to_dev(desc); + struct riscv_iommu_device *iommu = dev_get_drvdata(dev); + u16 idx = desc->msi_index; + u64 addr; + + addr = ((u64)msg->address_hi << 32) | msg->address_lo; + + if (addr != (addr & RISCV_IOMMU_MSI_CFG_TBL_ADDR)) + pr_warn_once("uh oh, the IOMMU can't send MSIs to 0x%llx, sending to 0x%llx instead\n", + addr, addr & RISCV_IOMMU_MSI_CFG_TBL_ADDR); + + addr &= RISCV_IOMMU_MSI_CFG_TBL_ADDR; + + riscv_iommu_writeq(iommu, RISCV_IOMMU_REG_MSI_CFG_TBL_ADDR(idx), addr); + riscv_iommu_writel(iommu, RISCV_IOMMU_REG_MSI_CFG_TBL_DATA(idx), msg->data); + riscv_iommu_writel(iommu, RISCV_IOMMU_REG_MSI_CFG_TBL_CTRL(idx), 0); +} + static int riscv_iommu_platform_probe(struct platform_device *pdev) { + enum riscv_iommu_igs_settings igs; struct device *dev = &pdev->dev; struct riscv_iommu_device *iommu = NULL; struct resource *res = NULL; - int vec; + int vec, ret; iommu = devm_kzalloc(dev, sizeof(*iommu), GFP_KERNEL); if (!iommu) @@ -40,16 +63,6 @@ static int riscv_iommu_platform_probe(struct platform_device *pdev) iommu->caps = riscv_iommu_readq(iommu, RISCV_IOMMU_REG_CAPABILITIES); iommu->fctl = riscv_iommu_readl(iommu, RISCV_IOMMU_REG_FCTL); - /* For now we only support WSI */ - switch (FIELD_GET(RISCV_IOMMU_CAPABILITIES_IGS, iommu->caps)) { - case RISCV_IOMMU_CAPABILITIES_IGS_WSI: - case RISCV_IOMMU_CAPABILITIES_IGS_BOTH: - break; - default: - return dev_err_probe(dev, -ENODEV, - "unable to use wire-signaled interrupts\n"); - } - iommu->irqs_count = platform_irq_count(pdev); if (iommu->irqs_count <= 0) return dev_err_probe(dev, -ENODEV, @@ -57,13 +70,60 @@ static int riscv_iommu_platform_probe(struct platform_device *pdev) if (iommu->irqs_count > RISCV_IOMMU_INTR_COUNT) iommu->irqs_count = RISCV_IOMMU_INTR_COUNT; - for (vec = 0; vec < iommu->irqs_count; vec++) - iommu->irqs[vec] = platform_get_irq(pdev, vec); + igs = FIELD_GET(RISCV_IOMMU_CAPABILITIES_IGS, iommu->caps); + switch (igs) { + case RISCV_IOMMU_CAPABILITIES_IGS_BOTH: + case RISCV_IOMMU_CAPABILITIES_IGS_MSI: + if (is_of_node(dev->fwnode)) + of_msi_configure(dev, to_of_node(dev->fwnode)); + + if (!dev_get_msi_domain(dev)) { + dev_warn(dev, "failed to find an MSI domain"); + goto msi_fail; + } + + ret = platform_device_msi_init_and_alloc_irqs(dev, iommu->irqs_count, + riscv_iommu_write_msi_msg); + if (ret) { + dev_warn(dev, "failed to allocate MSIs"); + goto msi_fail; + } + + for (vec = 0; vec < iommu->irqs_count; vec++) + iommu->irqs[vec] = msi_get_virq(dev, vec); + + /* Enable message-signaled interrupts, fctl.WSI */ + if (iommu->fctl & RISCV_IOMMU_FCTL_WSI) { + iommu->fctl ^= RISCV_IOMMU_FCTL_WSI; + riscv_iommu_writel(iommu, RISCV_IOMMU_REG_FCTL, iommu->fctl); + } + + dev_info(dev, "using MSIs\n"); + break; + +msi_fail: + if (igs != RISCV_IOMMU_CAPABILITIES_IGS_BOTH) { + dev_warn(dev, "\n"); + return dev_err_probe(dev, -ENODEV, + "unable to use wire-signaled interrupts\n"); + } + + dev_warn(dev, " - falling back to wired irqs\n"); + fallthrough; - /* Enable wire-signaled interrupts, fctl.WSI */ - if (!(iommu->fctl & RISCV_IOMMU_FCTL_WSI)) { - iommu->fctl |= RISCV_IOMMU_FCTL_WSI; - riscv_iommu_writel(iommu, RISCV_IOMMU_REG_FCTL, iommu->fctl); + case RISCV_IOMMU_CAPABILITIES_IGS_WSI: + for (vec = 0; vec < iommu->irqs_count; vec++) + iommu->irqs[vec] = platform_get_irq(pdev, vec); + + /* Enable wire-signaled interrupts, fctl.WSI */ + if (!(iommu->fctl & RISCV_IOMMU_FCTL_WSI)) { + iommu->fctl |= RISCV_IOMMU_FCTL_WSI; + riscv_iommu_writel(iommu, RISCV_IOMMU_REG_FCTL, iommu->fctl); + } + dev_info(dev, "using wire-signaled interrupts\n"); + break; + default: + return dev_err_probe(dev, -ENODEV, "invalid IGS\n"); } return riscv_iommu_init(iommu); @@ -71,7 +131,13 @@ static int riscv_iommu_platform_probe(struct platform_device *pdev) static void riscv_iommu_platform_remove(struct platform_device *pdev) { - riscv_iommu_remove(dev_get_drvdata(&pdev->dev)); + struct riscv_iommu_device *iommu = dev_get_drvdata(&pdev->dev); + bool msi = !(iommu->fctl & RISCV_IOMMU_FCTL_WSI); + + riscv_iommu_remove(iommu); + + if (msi) + platform_device_msi_free_irqs_all(&pdev->dev); }; static const struct of_device_id riscv_iommu_of_match[] = {