From patchwork Tue Mar 2 09:26:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12111309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46DF9C433E0 for ; Tue, 2 Mar 2021 09:37:14 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D7E3264F0A for ; Tue, 2 Mar 2021 09:37:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D7E3264F0A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Oq354OllTDI5IF09ALFUY7grH2+pn8gBoMfJq2XHqNI=; b=GS+roIbCRfa+HhvESMvhd+ZO2 ODLcmwCi4UKopIyhkVzHBfgx/PTtKvRUKWqKsdAuOn3NG0b4ct+q03CuAdWjIqYdG58C/Ee7EgiBd OKuMH02GXTBskAF0wMBX7fS7iUPMCrayCMneWuOiMWlWm/ovy6nUEXFtYmAF70mPvUD5KAraad6Kq MbFaYdgPobR/B7WHiLDPKYX7XtAQNjlfUxW62l0ru3RFvHLswRrt5PehaHiemnCCqQWOFUHiJY6NV kPKJKKtHExxd/8h3Ot9zaaNstycifDbGwJDY/tLbYrfhAeLlXUBbgbjvrCt18f2ZC4FtAQLRJB1UM SzKs6TH0A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RS-0003Dp-Ph; Tue, 02 Mar 2021 09:35:46 +0000 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RG-00038J-DO for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 09:35:37 +0000 Received: by mail-wr1-x436.google.com with SMTP id e10so18850665wro.12 for ; Tue, 02 Mar 2021 01:35:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q1dBU0HCqaN2oBJe9EmSK9n4bOZOtCnnbOc1d9k2Lwk=; b=TjiGWaeH01qGoIEQwfZkE7Z5JsNayNoEh2k+bi13tJXonskCNhseJfQymh+UJSVBGY tQhTKXwiAJn4JRbC70qRUg+BSzt/tKIZ8YkjJks/LvKxcsxcuHVlkitDHLBeh1PxJrXj c5ASycut8M9NUDlB3StAbfCtaLJROK1bLKvUDQ8ryVV6skmVutS2tX0BSi6dRM1JAkMv sOl1UtbRJV55dRe5eIEov7WsPcStFJaoR7bTretXUc/lbGCZo11gafb+W2so/0XiRYJn HSZQEGVLwYQupOMz68Urbcq9FOu3bfPvFhIUo4eZRraVMDxVR7y1Nf4tpC+Sz8OpsrKh 8c3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q1dBU0HCqaN2oBJe9EmSK9n4bOZOtCnnbOc1d9k2Lwk=; b=ZbOlss+FHRxcpEdj6aSGnUHZCypDwAVgxumAIRNYxfzHgAvP0NVLLOGgEFb3J32j5C 8rSVFjFm/mObcQ7QtFU5Gq8/TwGmwsEHrhvWGJnvkgu+LZLy84YDe8jJHLoLOupF1Vqu jElr3+TBS9wUBDhh/VhEdOrUegt0AR23rNR5IXjSNhxJa6JfZAafFfSa8czL459V1g5U frjklEWpfZFgbZJUOLi7Kgcb5WygzNyKLElpm9Rw50xRizPg4Ut0P8HQE2GArJMzQ5Ch fsHcOPEmXUZXQk+piooo3G09i3ehSd1SxputsnM1gCZdrcbh4KBpogj1BCgCrNAqmm2i JavA== X-Gm-Message-State: AOAM53301HbTKsq4DwGMx+1/zWqCW+OaZpw1YYKtXZ39FZhHosqJukZh JJRrmnhcvzpFsBL9CU+HX5VFEw== X-Google-Smtp-Source: ABdhPJw0xjkmPSp0mtZPb6X4Nd58+owZ0dVYxXm8u5GKEjL3scy/QYMFqL4xdWND/ojmSq+WV9cIEA== X-Received: by 2002:a5d:6103:: with SMTP id v3mr10571422wrt.375.1614677731470; Tue, 02 Mar 2021 01:35:31 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w131sm1963818wmb.8.2021.03.02.01.35.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 01:35:31 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Subject: [PATCH v13 01/10] iommu: Fix comment for struct iommu_fwspec Date: Tue, 2 Mar 2021 10:26:37 +0100 Message-Id: <20210302092644.2553014-2-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210302092644.2553014-1-jean-philippe@linaro.org> References: <20210302092644.2553014-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_043534_491225_A3188FCA X-CRM114-Status: GOOD ( 13.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: vivek.gautam@arm.com, guohanjun@huawei.com, zhukeqian1@huawei.com, Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, wangzhou1@hisilicon.com, linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Commit 986d5ecc5699 ("iommu: Move fwspec->iommu_priv to struct dev_iommu") removed iommu_priv from fwspec and commit 5702ee24182f ("ACPI/IORT: Check ATS capability in root complex nodes") added @flags. Update the struct doc. Acked-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker Acked-by: Will Deacon --- include/linux/iommu.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 5e7fe519430a..1d422bf722a1 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -571,7 +571,7 @@ struct iommu_group *fsl_mc_device_group(struct device *dev); * struct iommu_fwspec - per-device IOMMU instance data * @ops: ops for this device's IOMMU * @iommu_fwnode: firmware handle for this device's IOMMU - * @iommu_priv: IOMMU driver private data for this device + * @flags: IOMMU_FWSPEC_* flags * @num_pasid_bits: number of PASID bits supported by this device * @num_ids: number of associated device IDs * @ids: IDs which this device may present to the IOMMU From patchwork Tue Mar 2 09:26:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12111305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6C69C433DB for ; Tue, 2 Mar 2021 09:37:11 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 815AB64F09 for ; Tue, 2 Mar 2021 09:37:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 815AB64F09 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=9eaAC3W4sgr06UKFI59NFzpAK3TN/KMD3qEXhkYY4bc=; b=M3MlawQHsPiDUeW4flhiE4IPd lTMGJ9soVSEr2aTNzq0fp7DgwmdlAqZz4ZSQ+G8tiHIIo01yZ9Qlc+XOlQM58nY4oW4t5RNECKyvo 40I35Y/MHDKtWjVGi07J82uS4lcm03cXz0Z2h/nYUJwpgmWF+CHhCebKAO3qN4t2DZLgBRZs9W0hu 5HVxU55nwxcuqciW39F5YF0Jzv60eRizbRiC2yPmcyE2mUBi/rZUfQpPjBeIMEKgrtM30FMf+f/Mn b13hi6uui6MkxKU6smcJCLFUSHd4vnjzPtu50AlQiitouBE1L04+7GJSBojkNUGMtlbb8nhF9PP/0 tpZDNqINQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RO-0003Ca-KR; Tue, 02 Mar 2021 09:35:42 +0000 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RF-00038P-V9 for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 09:35:37 +0000 Received: by mail-wr1-x431.google.com with SMTP id v15so19102502wrx.4 for ; Tue, 02 Mar 2021 01:35:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YldRYT03xwkqqahy3TqrWZQZDsJFb2h5IF4zYpZYWmw=; b=w+rdPO0JHTT2L2/3tw009CCOXkBgpshsEaSpLWqdi7dX1sjMwtm1fFiJW6+EEJQZ0/ oTCzvgEeKGnexhDYqAt+R0PaePsYyfx4xa4qp1EbQjyfp1WdpU2KfIh1Hv21rjyBxEK5 uVaxBEa5yUxPB6jniFP+zmwSkbcxCCn1zRO+dHx2VNs3Rw/mNyK6Z+UmWHb3LRNfxIxo DYyRLXTQM+UWRBu5/+17pOKch8qRZj4/f15wdaLKYkI4zxvA1jPlzRczVQd0ZmKt3tFM Lc3GCSZdScNztMRGazdmWgK+fDLXvxC8e0QBiOYXZoxo1WqmLJ3Oe5ZhjjcNX9rFDos/ Ob2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YldRYT03xwkqqahy3TqrWZQZDsJFb2h5IF4zYpZYWmw=; b=ncXCucHUwlJI038Kw+Bw2iz4QsTuOuNwoUnv81KECMh/1qmGD69yUpVHPbjvjm7CMI /MwG0N9s7eKB1BaC4qhpmeNwWV50Wgcgn4F40ejDZyuSZ5BYLF42XjyksLuZ3Be64QWz IYi6kmTjxrCcdEdt6+8fffeGryGDZHZ+THs4yGBNMKfZQFYWsaOWEU6c9NowICatIExZ Jlaq2aZ64XPK9/Ek4hBlvX8a3EDjDuuJNVeXAn+ZtRUGDT+BEqwF7foPI40E4QT/2Teh Ai3XzSVlooA0sjoTcyEm0+USiWxmCk/hmFwfOqKjykjCL3JjRmyJ1b3fQmhpGxHIHRL1 illQ== X-Gm-Message-State: AOAM533cNJPh8aTTblaPBmbOra13SgNddETigIvfYYcT1siM7M+RNDGA pCfZW92YR/UmTv75QKO8Sr+nsUqnxNsFeQ== X-Google-Smtp-Source: ABdhPJziUR7m/6sWv8w2q6YqohX5h02MqXZ7Bt3V5/veYoCEwom/Pgu7ispfA5hf1cn38y3xw7HAGg== X-Received: by 2002:a5d:6a86:: with SMTP id s6mr7586863wru.307.1614677732520; Tue, 02 Mar 2021 01:35:32 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w131sm1963818wmb.8.2021.03.02.01.35.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 01:35:32 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Subject: [PATCH v13 02/10] iommu/arm-smmu-v3: Use device properties for pasid-num-bits Date: Tue, 2 Mar 2021 10:26:38 +0100 Message-Id: <20210302092644.2553014-3-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210302092644.2553014-1-jean-philippe@linaro.org> References: <20210302092644.2553014-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_043534_060599_883FE6AA X-CRM114-Status: GOOD ( 16.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: vivek.gautam@arm.com, guohanjun@huawei.com, zhukeqian1@huawei.com, Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, wangzhou1@hisilicon.com, linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The pasid-num-bits property shouldn't need a dedicated fwspec field, it's a job for device properties. Add properties for IORT, and access the number of PASID bits using device_property_read_u32(). Suggested-by: Robin Murphy Acked-by: Jonathan Cameron Reviewed-by: Eric Auger Signed-off-by: Jean-Philippe Brucker Acked-by: Will Deacon --- include/linux/iommu.h | 2 -- drivers/acpi/arm64/iort.c | 13 +++++++------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 ++- drivers/iommu/of_iommu.c | 5 ----- 4 files changed, 9 insertions(+), 14 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 1d422bf722a1..16ce75693d83 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -572,7 +572,6 @@ struct iommu_group *fsl_mc_device_group(struct device *dev); * @ops: ops for this device's IOMMU * @iommu_fwnode: firmware handle for this device's IOMMU * @flags: IOMMU_FWSPEC_* flags - * @num_pasid_bits: number of PASID bits supported by this device * @num_ids: number of associated device IDs * @ids: IDs which this device may present to the IOMMU */ @@ -580,7 +579,6 @@ struct iommu_fwspec { const struct iommu_ops *ops; struct fwnode_handle *iommu_fwnode; u32 flags; - u32 num_pasid_bits; unsigned int num_ids; u32 ids[]; }; diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index 2494138a6905..3912a1f6058e 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -968,15 +968,16 @@ static int iort_pci_iommu_init(struct pci_dev *pdev, u16 alias, void *data) static void iort_named_component_init(struct device *dev, struct acpi_iort_node *node) { + struct property_entry props[2] = {}; struct acpi_iort_named_component *nc; - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - - if (!fwspec) - return; nc = (struct acpi_iort_named_component *)node->node_data; - fwspec->num_pasid_bits = FIELD_GET(ACPI_IORT_NC_PASID_BITS, - nc->node_flags); + props[0] = PROPERTY_ENTRY_U32("pasid-num-bits", + FIELD_GET(ACPI_IORT_NC_PASID_BITS, + nc->node_flags)); + + if (device_add_properties(dev, props)) + dev_warn(dev, "Could not add device properties\n"); } static int iort_nc_iommu_map(struct device *dev, struct acpi_iort_node *node) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 8594b4a83043..7edce914c45e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2392,7 +2392,8 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) } } - master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits); + device_property_read_u32(dev, "pasid-num-bits", &master->ssid_bits); + master->ssid_bits = min(smmu->ssid_bits, master->ssid_bits); /* * Note that PASID must be enabled before, and disabled after ATS: diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c index e505b9130a1c..a9d2df001149 100644 --- a/drivers/iommu/of_iommu.c +++ b/drivers/iommu/of_iommu.c @@ -210,11 +210,6 @@ const struct iommu_ops *of_iommu_configure(struct device *dev, of_pci_iommu_init, &info); } else { err = of_iommu_configure_device(master_np, dev, id); - - fwspec = dev_iommu_fwspec_get(dev); - if (!err && fwspec) - of_property_read_u32(master_np, "pasid-num-bits", - &fwspec->num_pasid_bits); } /* From patchwork Tue Mar 2 09:26:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12111311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E0EFC433E0 for ; Tue, 2 Mar 2021 09:37:17 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D35AA64EF1 for ; Tue, 2 Mar 2021 09:37:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D35AA64EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/k1MtEbFoN9Q8hwEOHgx3fr55d9D8u9qgoU7g6Pgnik=; b=QRaGiANskTo1woqBKUjEV6DSF 5JqBS5nsnqV2/R4NDFFITRqjHogTmm/ueuOJ8+dcL8YYYqvbUkMjh+WT3mV16Em/0E1pVb82X27RL OWBoABUI1IgXgwPZ4t0/dTLz1/KDdk5ucw7iUpysSehYu0vb4XRzGflj71nM+ng/ZzSxE+pZfKNwC i2kg+lQHLrrV1lg4NPBt3WtizqDcb9rbYswpyrGtWkTj0SnoyF2kBYV2r8ht22QIfmMmRDg2h6u0l NG/KtkD10556mo+m7M0nIeXd/kHXF9lZ7+UqufMxgB6KEmSRMIsIhfFtWkP6C2toZSE5GmNlB1afo fegGzH/+Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RY-0003Fu-05; Tue, 02 Mar 2021 09:35:52 +0000 Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RH-00038a-2w for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 09:35:37 +0000 Received: by mail-wm1-x335.google.com with SMTP id w203-20020a1c49d40000b029010c706d0642so759360wma.0 for ; Tue, 02 Mar 2021 01:35:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UwcG/UbYxOwu/Rok4cktInfyw7lfwNLJYyuoJI7sKTg=; b=vGsFqDqP3CHWV+Yt1menOPl3yoMGtirJWj2Zr9+MyVkCV7uKFvnywi7OQlvSvGy3RW rUqU7p3fe+muXm5qv95Ab6t47/zFtZqZQIrBhkgmESkzZ7VmtiyQgtLPd6ipfJkcWGiR 6MkIb1JCZWTNjtNCGUd1S45kmqCIKsbwDrmq0ZvtyLP3ED42TK1HYt1RRpJRZF68XtQm RqMax/jWgba5YRRKgtrUhISzlUhHNjRfbKMPZFVUUDgfT/5A9X7VLoYv3mF5t2phBix6 tE6f3/KB8GbwdBSzsU3+lmF0HXaqA9X/RzkPFERfSO8BJtcypuVS0MR0MOQfhVhr+Irt DjsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UwcG/UbYxOwu/Rok4cktInfyw7lfwNLJYyuoJI7sKTg=; b=Uye29pEveyeSxnMp5wvW2T9Ti0NZSvwkyb4mvNZwG16cAghQnFP+Q3BKHh85h6DvPd ugdw9K/HNvG7HiOo0a9S+dArWR6xm91W/P31MI4jHOzRDB/cNe7I/IAa43IAKjWdgg2K AY9fgLv5JaSRckQ02CNSBpgjmfNk7EncPupwUZARc3cogr4zsfwVdD/8kz6jV67yyMDW /xcBb963mx8/WIwUJwuRDJ+g0H3LTDmPG9VAQQpWKiBrtqOAoLscIwxIsWv01h4qS4Iz h2tm1nUtYvp1+aPycnXEt8QlNmF5EWqSc/UaEM8REo5g/nIODfNmpDf+7ddZhYZ3v2oC Hxww== X-Gm-Message-State: AOAM531zHNlK9nfnIQ0FnpCfxjsiimvK3h/xsTRKVlRBL/bu9QOVkAsl c+ls5GrXi9OlE9K0GJYvIIO+kg== X-Google-Smtp-Source: ABdhPJx3IarwrxvGVNUDDDTxKQdBz1ESv5vuEdngsYCCDXK+5PM6f0H2Y/ifeQTqAjmXHfnwbdzQBQ== X-Received: by 2002:a1c:ac86:: with SMTP id v128mr2993828wme.175.1614677733641; Tue, 02 Mar 2021 01:35:33 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w131sm1963818wmb.8.2021.03.02.01.35.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 01:35:33 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Subject: [PATCH v13 03/10] iommu: Separate IOMMU_DEV_FEAT_IOPF from IOMMU_DEV_FEAT_SVA Date: Tue, 2 Mar 2021 10:26:39 +0100 Message-Id: <20210302092644.2553014-4-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210302092644.2553014-1-jean-philippe@linaro.org> References: <20210302092644.2553014-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_043535_162120_E65BB55B X-CRM114-Status: GOOD ( 16.99 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Greg Kroah-Hartman , vivek.gautam@arm.com, guohanjun@huawei.com, zhukeqian1@huawei.com, Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, wangzhou1@hisilicon.com, linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, Arnd Bergmann , eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, David Woodhouse , rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Some devices manage I/O Page Faults (IOPF) themselves instead of relying on PCIe PRI or Arm SMMU stall. Allow their drivers to enable SVA without mandating IOMMU-managed IOPF. The other device drivers now need to first enable IOMMU_DEV_FEAT_IOPF before enabling IOMMU_DEV_FEAT_SVA. Enabling IOMMU_DEV_FEAT_IOPF on its own doesn't have any effect visible to the device driver, it is used in combination with other features. Reviewed-by: Eric Auger Signed-off-by: Jean-Philippe Brucker Reviewed-by: Lu Baolu --- Cc: Arnd Bergmann Cc: David Woodhouse Cc: Greg Kroah-Hartman Cc: Joerg Roedel Cc: Lu Baolu Cc: Will Deacon Cc: Zhangfei Gao Cc: Zhou Wang --- include/linux/iommu.h | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 16ce75693d83..45c4eb372f56 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -156,10 +156,24 @@ struct iommu_resv_region { enum iommu_resv_type type; }; -/* Per device IOMMU features */ +/** + * enum iommu_dev_features - Per device IOMMU features + * @IOMMU_DEV_FEAT_AUX: Auxiliary domain feature + * @IOMMU_DEV_FEAT_SVA: Shared Virtual Addresses + * @IOMMU_DEV_FEAT_IOPF: I/O Page Faults such as PRI or Stall. Generally + * enabling %IOMMU_DEV_FEAT_SVA requires + * %IOMMU_DEV_FEAT_IOPF, but some devices manage I/O Page + * Faults themselves instead of relying on the IOMMU. When + * supported, this feature must be enabled before and + * disabled after %IOMMU_DEV_FEAT_SVA. + * + * Device drivers query whether a feature is supported using + * iommu_dev_has_feature(), and enable it using iommu_dev_enable_feature(). + */ enum iommu_dev_features { - IOMMU_DEV_FEAT_AUX, /* Aux-domain feature */ - IOMMU_DEV_FEAT_SVA, /* Shared Virtual Addresses */ + IOMMU_DEV_FEAT_AUX, + IOMMU_DEV_FEAT_SVA, + IOMMU_DEV_FEAT_IOPF, }; #define IOMMU_PASID_INVALID (-1U) From patchwork Tue Mar 2 09:26:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12111313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5128EC433DB for ; Tue, 2 Mar 2021 09:37:24 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E274164EFC for ; Tue, 2 Mar 2021 09:37:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E274164EFC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=nmsjltq03PJVxxugiRofdHR9YmDtvCWbdgKX256gslw=; b=smZgJ9fhF6BcM7oOAYjwV3VE7 ynD7ei+pHJk8LSXxb0ZD1K0lLGwx3DzZKaCMB69VAUvJFpzjRg9nwYphPFzAUARfN52TqsODlnmKE L3KF8qiwDExMzvUJPl9QE4gII8hgmhhMqnUybtXW703cwHi7VKQzOLPfEA1lpnZc+z4hzRGyjPD4f DtOa1QduG+oPZ9uQfRkGUevkB9W1LufHC6gGDZ3ofILxjkQYQ8tgCk3nvyDWrlIma2A5O1XrpSpRt 1Rd4wBs3M9MQqY436VrOzje+BQBq7I6av2XsgXk23+VbSHdQdowIBVYRqZBUY2CcmEZMGbRKdUauL e9KgmWnOQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1Rd-0003HH-VV; Tue, 02 Mar 2021 09:35:58 +0000 Received: from mail-wm1-x331.google.com ([2a00:1450:4864:20::331]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RI-00039G-9Z for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 09:35:39 +0000 Received: by mail-wm1-x331.google.com with SMTP id o2so1443526wme.5 for ; Tue, 02 Mar 2021 01:35:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fAwFmceeI8PT69Qzv7BbL234cbzlwFiSG33CnDCXBuk=; b=pVUo6Je1O9n09zMy7XzMnOeoxCf2gwvJSFjHsUpFlcN2ASmVQotVn/P3GJaA2vMsPG 5Oa73C/bPbSORaAchXIQza2ekIiLHAMzhiBZSlMfN6WbI9De+lb3VRTl3rQ2yQVlngcr V2RHEDDpBUjaRbWMvhc/3ySgIrurqPttQruKNTQF2GytX5XdWxpKNvyRtIS3TzMszIvc JBZB4CBm0UCJiSKiLxjvSyjWHHMuuOuEHyTWoD+ba6Xp8wX8IFgRMCwyw68DBIQ5z2iC eWdpOaKbSD5yQITmiRYw1xTk99A7N0QY50S3SPObJAaXzoV7J2H4ZrpqEenmZmu1CZJM pCiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fAwFmceeI8PT69Qzv7BbL234cbzlwFiSG33CnDCXBuk=; b=DwprmUac+dPXFg7yycFRL2qO72cYj/QkIGdxr1fGumCKBoAiFRRVhtYOODrWm5xEl+ tkP8cfRX/4PG/DqRXMVE7qbgaiB4BEaB4Ln9wrE6pZI6zgXB+kVgrEV4jZNxjrx5iEtj LrtpTdDXB31I3HTs3QJDqGPhVWx8MUqUoLQ5xS8QDiywInga6mEsDGBy4GXJ50PF9Lie kzv/jbOMSW1na7F7ujF4Y73+abmRPWpQc7aNgXSP4kqSV3F6J3AtDWCmn04GEPoMVoMt Uj38fL/CY9BQV20IMMZXYQKnj25KpRym0/4sGvbcJA5RYeqJtRmRTPZbt8uGevXs52Tp I8Yw== X-Gm-Message-State: AOAM531hEh7F0cM41Z1nTU6n4F98B7F4YuxdyDaYHovPgwyYjbLkQSUr MPXYpGMOCq/NaiPstv4RaEwM5w== X-Google-Smtp-Source: ABdhPJwrpaQozpbifmdBMiCZvNYDUDCjs8CDLKRQmFMc/2Y3eNN20yvyuRxJXyxFUTgbOPqB1BFNww== X-Received: by 2002:a1c:a504:: with SMTP id o4mr3053405wme.174.1614677734793; Tue, 02 Mar 2021 01:35:34 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w131sm1963818wmb.8.2021.03.02.01.35.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 01:35:34 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Subject: [PATCH v13 04/10] iommu/vt-d: Support IOMMU_DEV_FEAT_IOPF Date: Tue, 2 Mar 2021 10:26:40 +0100 Message-Id: <20210302092644.2553014-5-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210302092644.2553014-1-jean-philippe@linaro.org> References: <20210302092644.2553014-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_043536_420960_BEC8B89F X-CRM114-Status: GOOD ( 13.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: vivek.gautam@arm.com, guohanjun@huawei.com, zhukeqian1@huawei.com, Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, wangzhou1@hisilicon.com, linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, David Woodhouse , rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow drivers to query and enable IOMMU_DEV_FEAT_IOPF, which amounts to checking whether PRI is enabled. Reviewed-by: Lu Baolu Signed-off-by: Jean-Philippe Brucker --- Cc: David Woodhouse Cc: Lu Baolu --- drivers/iommu/intel/iommu.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index ee0932307d64..c5c5fd444779 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -5343,6 +5343,8 @@ static int siov_find_pci_dvsec(struct pci_dev *pdev) static bool intel_iommu_dev_has_feat(struct device *dev, enum iommu_dev_features feat) { + struct device_domain_info *info = get_domain_info(dev); + if (feat == IOMMU_DEV_FEAT_AUX) { int ret; @@ -5357,13 +5359,13 @@ intel_iommu_dev_has_feat(struct device *dev, enum iommu_dev_features feat) return !!siov_find_pci_dvsec(to_pci_dev(dev)); } - if (feat == IOMMU_DEV_FEAT_SVA) { - struct device_domain_info *info = get_domain_info(dev); + if (feat == IOMMU_DEV_FEAT_IOPF) + return info && info->pri_supported; + if (feat == IOMMU_DEV_FEAT_SVA) return info && (info->iommu->flags & VTD_FLAG_SVM_CAPABLE) && info->pasid_supported && info->pri_supported && info->ats_supported; - } return false; } @@ -5374,6 +5376,9 @@ intel_iommu_dev_enable_feat(struct device *dev, enum iommu_dev_features feat) if (feat == IOMMU_DEV_FEAT_AUX) return intel_iommu_enable_auxd(dev); + if (feat == IOMMU_DEV_FEAT_IOPF) + return intel_iommu_dev_has_feat(dev, feat) ? 0 : -ENODEV; + if (feat == IOMMU_DEV_FEAT_SVA) { struct device_domain_info *info = get_domain_info(dev); From patchwork Tue Mar 2 09:26:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12111315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED680C433E0 for ; Tue, 2 Mar 2021 09:37:32 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8AA6464EF1 for ; Tue, 2 Mar 2021 09:37:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8AA6464EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=YcxA5jIpCn901j8O6W1E1cpyW6ysb9TOnxEbfa2eVsc=; b=Fa+pmXA2cO5WCADg60hkaKsHD bPpzmXVlFezClpZK3O+YizK1ODp1q7TLw9mHzWdv6bG13s5N7vuFK7nTVurCOz0T6hCvezXPlB5US caUe28cvVx7IhborVvOtYb/YsXjc6X8VS268oraEK5Kb7APmD5GJsBIgFAfCguy1tOa1TBkpl3k/E ZAH5KPViRnnnQzt5PahCfBM33Xq3YGwbshF6q4vnptP/xhVWcVebdBgldGegqDEQlnGa6LxITXcqR 8tDma9tkKFdtX+O9RlFZ5BYV63WZwJ4f3w49g+Y6o5dGwRyrfGGzHuBn/HY5usTPgcyi7Qn3g3lpq 52xZAEIbg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1Rm-0003LP-CS; Tue, 02 Mar 2021 09:36:06 +0000 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RJ-00039l-Cc for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 09:35:40 +0000 Received: by mail-wr1-x42c.google.com with SMTP id v15so19102700wrx.4 for ; Tue, 02 Mar 2021 01:35:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=N+2wRT0d2D2HxeGkEvggn2nlXqB54onyEwHP8db/7QY=; b=mur5/SaKYOVuZNj0OFEQEGwWcgveYIT+mr0Ko/887w+dI+wFoEZzBahHqx7a7+I0Jl VkPi0t4z6gtLLqU3Z6W1vbzMAAndT1MZ9KrbRNHk1vEuUOIpi0ZYDRbuvoEWdg87tGOP 5O4RpoYLKqHZXE3FQGWwN4CFbgtpArsvLI3oto3kHdPgIZOeSj1f4T9MAZit05DDSbcB l+UQie0L3aLh2mpteK0Ee3T2cIyH0k8stxV1g5YBP8hEmJ1fa8HwNjAJVDtSafrD4aPA o1og0ZnwYyUP8sWnCDCx2tzuBQAf9KmLkjLBnuLA1tg4Btab5VZJxWZlKs4Bu+NGgA6u HqKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N+2wRT0d2D2HxeGkEvggn2nlXqB54onyEwHP8db/7QY=; b=SLWcQPLGGbDjDX9NifhR7rEX1UR4O7NvYyi7ME9qSKTBzR/lG1tiRaKi5pt2yVMR+r rSZraLX8X3BLKVHiG8ZiWOM9aVNCl3vkqzmKdBL0mYJjQJObmabhn2drFOMwucgW28f0 OB8K/RLT2+ppMTfTSTW3Av9MY46H/py14fv96uarLc5SSnDSPm78FbrgFe60WUlHaig7 ctToj3mcepzWgym42a65oZQRB5/4umv3op1GegAjAuOhPRD3Wq0XHsKnakNWngu2uNl1 Dz/McAakYcQm69pBnm9WnWqVy7FaAm3yWfbBIFdY9dYLCXb8I1IdfTsdhA63NkreqXyB Gdew== X-Gm-Message-State: AOAM530umFEwfQyv/jaYtn5FCIMltry22MtySfT9NEt8IwNqvaQmhepD qpwSnMihIB2X9VfrVNtV4JfDkQ== X-Google-Smtp-Source: ABdhPJwF9FbBYZEYEJcEnxvX5Mh5YZ2ftfwCrSmxifnxS9pSwMMvYK2TmNSRxV7W9useydfpOfqpig== X-Received: by 2002:a05:6000:1104:: with SMTP id z4mr11973344wrw.10.1614677735910; Tue, 02 Mar 2021 01:35:35 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w131sm1963818wmb.8.2021.03.02.01.35.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 01:35:35 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Subject: [PATCH v13 05/10] uacce: Enable IOMMU_DEV_FEAT_IOPF Date: Tue, 2 Mar 2021 10:26:41 +0100 Message-Id: <20210302092644.2553014-6-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210302092644.2553014-1-jean-philippe@linaro.org> References: <20210302092644.2553014-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_043537_531240_EC0284E4 X-CRM114-Status: GOOD ( 17.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: vivek.gautam@arm.com, guohanjun@huawei.com, zhukeqian1@huawei.com, Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, wangzhou1@hisilicon.com, linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, Arnd Bergmann , eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman , rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The IOPF (I/O Page Fault) feature is now enabled independently from the SVA feature, because some IOPF implementations are device-specific and do not require IOMMU support for PCIe PRI or Arm SMMU stall. Enable IOPF unconditionally when enabling SVA for now. In the future, if a device driver implementing a uacce interface doesn't need IOPF support, it will need to tell the uacce module, for example with a new flag. Acked-by: Zhangfei Gao Signed-off-by: Jean-Philippe Brucker --- Cc: Arnd Bergmann Cc: Greg Kroah-Hartman Cc: Zhangfei Gao Cc: Zhou Wang --- drivers/misc/uacce/uacce.c | 39 +++++++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/drivers/misc/uacce/uacce.c b/drivers/misc/uacce/uacce.c index d07af4edfcac..6db7a98486ec 100644 --- a/drivers/misc/uacce/uacce.c +++ b/drivers/misc/uacce/uacce.c @@ -385,6 +385,33 @@ static void uacce_release(struct device *dev) kfree(uacce); } +static unsigned int uacce_enable_sva(struct device *parent, unsigned int flags) +{ + if (!(flags & UACCE_DEV_SVA)) + return flags; + + flags &= ~UACCE_DEV_SVA; + + if (iommu_dev_enable_feature(parent, IOMMU_DEV_FEAT_IOPF)) + return flags; + + if (iommu_dev_enable_feature(parent, IOMMU_DEV_FEAT_SVA)) { + iommu_dev_disable_feature(parent, IOMMU_DEV_FEAT_IOPF); + return flags; + } + + return flags | UACCE_DEV_SVA; +} + +static void uacce_disable_sva(struct uacce_device *uacce) +{ + if (!(uacce->flags & UACCE_DEV_SVA)) + return; + + iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_SVA); + iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_IOPF); +} + /** * uacce_alloc() - alloc an accelerator * @parent: pointer of uacce parent device @@ -404,11 +431,7 @@ struct uacce_device *uacce_alloc(struct device *parent, if (!uacce) return ERR_PTR(-ENOMEM); - if (flags & UACCE_DEV_SVA) { - ret = iommu_dev_enable_feature(parent, IOMMU_DEV_FEAT_SVA); - if (ret) - flags &= ~UACCE_DEV_SVA; - } + flags = uacce_enable_sva(parent, flags); uacce->parent = parent; uacce->flags = flags; @@ -432,8 +455,7 @@ struct uacce_device *uacce_alloc(struct device *parent, return uacce; err_with_uacce: - if (flags & UACCE_DEV_SVA) - iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_SVA); + uacce_disable_sva(uacce); kfree(uacce); return ERR_PTR(ret); } @@ -487,8 +509,7 @@ void uacce_remove(struct uacce_device *uacce) mutex_unlock(&uacce->queues_lock); /* disable sva now since no opened queues */ - if (uacce->flags & UACCE_DEV_SVA) - iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_SVA); + uacce_disable_sva(uacce); if (uacce->cdev) cdev_device_del(uacce->cdev, &uacce->dev); From patchwork Tue Mar 2 09:26:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12111323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12B82C433DB for ; Tue, 2 Mar 2021 09:37:45 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9996864EEB for ; Tue, 2 Mar 2021 09:37:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9996864EEB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Vof+9xdcRquNaOw9K8KfVS4F4xQbq3TPZbV1yT8fos0=; b=TUXoPMBCmEzQKIIhsmbbMue67 kdjxgizx4o+EPx6g+uoJ3FM+BIn0nrdKCJqtmM+XJSWptX+USBzPXJ/qTXyzCY8Wc4PvQ0/tyyAKI 2Uu2XURZ9kOVknpODIksCxkmTjlPUl6Ij7hYBmti1IezFeHl/8i0Ds++7RWavlUAWigvOoakDSt8T sTj6y/WtjDpZ+wTo4HIAHRoJsohPRFca19M4XltZMyPzRoC57Qt0vUMvORYhdXm+pisp61PqBXbrw m2SEpzCsR9Jn3GAduCaKZXnYKRgzGx7ewbfh4ARiC2KyFAnVxOWPlJPBIanXNLnacTLRvDyNH89Cj 9GO+Y7v3g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1Ry-0003Qz-DE; Tue, 02 Mar 2021 09:36:18 +0000 Received: from mail-wm1-x32d.google.com ([2a00:1450:4864:20::32d]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RK-0003Aa-LC for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 09:35:48 +0000 Received: by mail-wm1-x32d.google.com with SMTP id o16so1952052wmh.0 for ; Tue, 02 Mar 2021 01:35:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dd5eZlaorGsMdq9hPH/APeezGKxvte3e7dxk0HQSce0=; b=ELhwkX4lf8r8POQZWx0zCZp5Ld/OBshlAi2DGsgP1XHcM2sOVeqgc8yQYcHGTSVuHs /TCZwzGozc3x0Txmw49MhNby6PvjZUG6l9fHFqBb8MVY++xaCeQEINDFjMUBrvoHTD1c rVeGXoJdtui7sEvvj1fYsfvq8mwhz3wxrYNtk1uxuboV7egXkKX2j6V17W1Xi3CIMRYh YWhnBseovoJbCOi80mIJziTbkevdEnZHgqjpyDgKvwyKMUzf06HPtl0p0aKC1EhT5ASm XLyPIk5suLDvwNpPD8gkXYrpWVfgCDplVkaGW1jJJuM7yWIeA0aVefvJOAZonvIPlooT R6Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dd5eZlaorGsMdq9hPH/APeezGKxvte3e7dxk0HQSce0=; b=eoBMfGEtRTlsaqnH92k50l62YfbUu80A66xW93OV3kshonlHCcQUd0/JjJd+7/oB7d QrbM70HypvDOVMU8C52lYttIjiq02ddlzByPHxIwePg9XhlWksArhKT6hXg4yL5jlgkO PxU1xZqdvK1QQHCQcKooVCuzbpXXwIo32hZOy2OBNI1yG/65Kt7MD7w1bKxHfpG9Tw3t Bwb2ndiAZ4oOZ6eMTatB5A5bmRWmVVjtWRArFpTA02k0lkmB95Q3ev25QXX1J3I1R8uy +D+ZmFq0dFpu34UqOlQU7wQfr7Z89ZcVHK8HjPykUNKpCItTgjkFpJQF1PnYDtB5gaFl Cd7g== X-Gm-Message-State: AOAM532+70qiKNVUkt5RfbmS4PlxAQSdmfkE4113PjsoYTPL3Ve0aKEl ZXNxYaJK08xoVpBCG8dTb9e25g== X-Google-Smtp-Source: ABdhPJyFdVwaIXo/jwXwQNjdzHU8mAS4DCkwRBCrqT/zPo7wXCfSVfaeWNrLBtjuWx6NPW+T7SvAYA== X-Received: by 2002:a1c:e208:: with SMTP id z8mr2958024wmg.111.1614677737065; Tue, 02 Mar 2021 01:35:37 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w131sm1963818wmb.8.2021.03.02.01.35.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 01:35:36 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Subject: [PATCH v13 06/10] iommu: Add a page fault handler Date: Tue, 2 Mar 2021 10:26:42 +0100 Message-Id: <20210302092644.2553014-7-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210302092644.2553014-1-jean-philippe@linaro.org> References: <20210302092644.2553014-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_043538_922483_7D18C85A X-CRM114-Status: GOOD ( 33.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: vivek.gautam@arm.com, guohanjun@huawei.com, zhukeqian1@huawei.com, Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, wangzhou1@hisilicon.com, linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Some systems allow devices to handle I/O Page Faults in the core mm. For example systems implementing the PCIe PRI extension or Arm SMMU stall model. Infrastructure for reporting these recoverable page faults was added to the IOMMU core by commit 0c830e6b3282 ("iommu: Introduce device fault report API"). Add a page fault handler for host SVA. IOMMU driver can now instantiate several fault workqueues and link them to IOPF-capable devices. Drivers can choose between a single global workqueue, one per IOMMU device, one per low-level fault queue, one per domain, etc. When it receives a fault event, most commonly in an IRQ handler, the IOMMU driver reports the fault using iommu_report_device_fault(), which calls the registered handler. The page fault handler then calls the mm fault handler, and reports either success or failure with iommu_page_response(). After the handler succeeds, the hardware retries the access. The iopf_param pointer could be embedded into iommu_fault_param. But putting iopf_param into the iommu_param structure allows us not to care about ordering between calls to iopf_queue_add_device() and iommu_register_device_fault_handler(). Reviewed-by: Eric Auger Reviewed-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker Reviewed-by:Jacob Pan Reviewed-by: Lu Baolu Tested-by: Lu Baolu --- drivers/iommu/Makefile | 1 + drivers/iommu/iommu-sva-lib.h | 53 ++++ include/linux/iommu.h | 2 + drivers/iommu/io-pgfault.c | 461 ++++++++++++++++++++++++++++++++++ 4 files changed, 517 insertions(+) create mode 100644 drivers/iommu/io-pgfault.c diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 61bd30cd8369..60fafc23dee6 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -28,3 +28,4 @@ obj-$(CONFIG_S390_IOMMU) += s390-iommu.o obj-$(CONFIG_HYPERV_IOMMU) += hyperv-iommu.o obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o obj-$(CONFIG_IOMMU_SVA_LIB) += iommu-sva-lib.o +obj-$(CONFIG_IOMMU_SVA_LIB) += io-pgfault.o diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index b40990aef3fd..031155010ca8 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -12,4 +12,57 @@ int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max); void iommu_sva_free_pasid(struct mm_struct *mm); struct mm_struct *iommu_sva_find(ioasid_t pasid); +/* I/O Page fault */ +struct device; +struct iommu_fault; +struct iopf_queue; + +#ifdef CONFIG_IOMMU_SVA_LIB +int iommu_queue_iopf(struct iommu_fault *fault, void *cookie); + +int iopf_queue_add_device(struct iopf_queue *queue, struct device *dev); +int iopf_queue_remove_device(struct iopf_queue *queue, + struct device *dev); +int iopf_queue_flush_dev(struct device *dev); +struct iopf_queue *iopf_queue_alloc(const char *name); +void iopf_queue_free(struct iopf_queue *queue); +int iopf_queue_discard_partial(struct iopf_queue *queue); + +#else /* CONFIG_IOMMU_SVA_LIB */ +static inline int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) +{ + return -ENODEV; +} + +static inline int iopf_queue_add_device(struct iopf_queue *queue, + struct device *dev) +{ + return -ENODEV; +} + +static inline int iopf_queue_remove_device(struct iopf_queue *queue, + struct device *dev) +{ + return -ENODEV; +} + +static inline int iopf_queue_flush_dev(struct device *dev) +{ + return -ENODEV; +} + +static inline struct iopf_queue *iopf_queue_alloc(const char *name) +{ + return NULL; +} + +static inline void iopf_queue_free(struct iopf_queue *queue) +{ +} + +static inline int iopf_queue_discard_partial(struct iopf_queue *queue) +{ + return -ENODEV; +} +#endif /* CONFIG_IOMMU_SVA_LIB */ #endif /* _IOMMU_SVA_LIB_H */ diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 45c4eb372f56..86d688c4418f 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -367,6 +367,7 @@ struct iommu_fault_param { * struct dev_iommu - Collection of per-device IOMMU data * * @fault_param: IOMMU detected device fault reporting data + * @iopf_param: I/O Page Fault queue and data * @fwspec: IOMMU fwspec data * @iommu_dev: IOMMU device this device is linked to * @priv: IOMMU Driver private data @@ -377,6 +378,7 @@ struct iommu_fault_param { struct dev_iommu { struct mutex lock; struct iommu_fault_param *fault_param; + struct iopf_device_param *iopf_param; struct iommu_fwspec *fwspec; struct iommu_device *iommu_dev; void *priv; diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c new file mode 100644 index 000000000000..1df8c1dcae77 --- /dev/null +++ b/drivers/iommu/io-pgfault.c @@ -0,0 +1,461 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Handle device page faults + * + * Copyright (C) 2020 ARM Ltd. + */ + +#include +#include +#include +#include +#include + +#include "iommu-sva-lib.h" + +/** + * struct iopf_queue - IO Page Fault queue + * @wq: the fault workqueue + * @devices: devices attached to this queue + * @lock: protects the device list + */ +struct iopf_queue { + struct workqueue_struct *wq; + struct list_head devices; + struct mutex lock; +}; + +/** + * struct iopf_device_param - IO Page Fault data attached to a device + * @dev: the device that owns this param + * @queue: IOPF queue + * @queue_list: index into queue->devices + * @partial: faults that are part of a Page Request Group for which the last + * request hasn't been submitted yet. + */ +struct iopf_device_param { + struct device *dev; + struct iopf_queue *queue; + struct list_head queue_list; + struct list_head partial; +}; + +struct iopf_fault { + struct iommu_fault fault; + struct list_head list; +}; + +struct iopf_group { + struct iopf_fault last_fault; + struct list_head faults; + struct work_struct work; + struct device *dev; +}; + +static int iopf_complete_group(struct device *dev, struct iopf_fault *iopf, + enum iommu_page_response_code status) +{ + struct iommu_page_response resp = { + .version = IOMMU_PAGE_RESP_VERSION_1, + .pasid = iopf->fault.prm.pasid, + .grpid = iopf->fault.prm.grpid, + .code = status, + }; + + if ((iopf->fault.prm.flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID) && + (iopf->fault.prm.flags & IOMMU_FAULT_PAGE_RESPONSE_NEEDS_PASID)) + resp.flags = IOMMU_PAGE_RESP_PASID_VALID; + + return iommu_page_response(dev, &resp); +} + +static enum iommu_page_response_code +iopf_handle_single(struct iopf_fault *iopf) +{ + vm_fault_t ret; + struct mm_struct *mm; + struct vm_area_struct *vma; + unsigned int access_flags = 0; + unsigned int fault_flags = FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm = &iopf->fault.prm; + enum iommu_page_response_code status = IOMMU_PAGE_RESP_INVALID; + + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + mm = iommu_sva_find(prm->pasid); + if (IS_ERR_OR_NULL(mm)) + return status; + + mmap_read_lock(mm); + + vma = find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |= VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |= VM_WRITE; + fault_flags |= FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |= VM_EXEC; + fault_flags |= FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |= FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret = handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status = ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; +} + +static void iopf_handle_group(struct work_struct *work) +{ + struct iopf_group *group; + struct iopf_fault *iopf, *next; + enum iommu_page_response_code status = IOMMU_PAGE_RESP_SUCCESS; + + group = container_of(work, struct iopf_group, work); + + list_for_each_entry_safe(iopf, next, &group->faults, list) { + /* + * For the moment, errors are sticky: don't handle subsequent + * faults in the group if there is an error. + */ + if (status == IOMMU_PAGE_RESP_SUCCESS) + status = iopf_handle_single(iopf); + + if (!(iopf->fault.prm.flags & + IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) + kfree(iopf); + } + + iopf_complete_group(group->dev, &group->last_fault, status); + kfree(group); +} + +/** + * iommu_queue_iopf - IO Page Fault handler + * @fault: fault event + * @cookie: struct device, passed to iommu_register_device_fault_handler. + * + * Add a fault to the device workqueue, to be handled by mm. + * + * This module doesn't handle PCI PASID Stop Marker; IOMMU drivers must discard + * them before reporting faults. A PASID Stop Marker (LRW = 0b100) doesn't + * expect a response. It may be generated when disabling a PASID (issuing a + * PASID stop request) by some PCI devices. + * + * The PASID stop request is issued by the device driver before unbind(). Once + * it completes, no page request is generated for this PASID anymore and + * outstanding ones have been pushed to the IOMMU (as per PCIe 4.0r1.0 - 6.20.1 + * and 10.4.1.2 - Managing PASID TLP Prefix Usage). Some PCI devices will wait + * for all outstanding page requests to come back with a response before + * completing the PASID stop request. Others do not wait for page responses, and + * instead issue this Stop Marker that tells us when the PASID can be + * reallocated. + * + * It is safe to discard the Stop Marker because it is an optimization. + * a. Page requests, which are posted requests, have been flushed to the IOMMU + * when the stop request completes. + * b. The IOMMU driver flushes all fault queues on unbind() before freeing the + * PASID. + * + * So even though the Stop Marker might be issued by the device *after* the stop + * request completes, outstanding faults will have been dealt with by the time + * the PASID is freed. + * + * Return: 0 on success and <0 on error. + */ +int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) +{ + int ret; + struct iopf_group *group; + struct iopf_fault *iopf, *next; + struct iopf_device_param *iopf_param; + + struct device *dev = cookie; + struct dev_iommu *param = dev->iommu; + + lockdep_assert_held(¶m->lock); + + if (fault->type != IOMMU_FAULT_PAGE_REQ) + /* Not a recoverable page fault */ + return -EOPNOTSUPP; + + /* + * As long as we're holding param->lock, the queue can't be unlinked + * from the device and therefore cannot disappear. + */ + iopf_param = param->iopf_param; + if (!iopf_param) + return -ENODEV; + + if (!(fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) { + iopf = kzalloc(sizeof(*iopf), GFP_KERNEL); + if (!iopf) + return -ENOMEM; + + iopf->fault = *fault; + + /* Non-last request of a group. Postpone until the last one */ + list_add(&iopf->list, &iopf_param->partial); + + return 0; + } + + group = kzalloc(sizeof(*group), GFP_KERNEL); + if (!group) { + /* + * The caller will send a response to the hardware. But we do + * need to clean up before leaving, otherwise partial faults + * will be stuck. + */ + ret = -ENOMEM; + goto cleanup_partial; + } + + group->dev = dev; + group->last_fault.fault = *fault; + INIT_LIST_HEAD(&group->faults); + list_add(&group->last_fault.list, &group->faults); + INIT_WORK(&group->work, iopf_handle_group); + + /* See if we have partial faults for this group */ + list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) { + if (iopf->fault.prm.grpid == fault->prm.grpid) + /* Insert *before* the last fault */ + list_move(&iopf->list, &group->faults); + } + + queue_work(iopf_param->queue->wq, &group->work); + return 0; + +cleanup_partial: + list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) { + if (iopf->fault.prm.grpid == fault->prm.grpid) { + list_del(&iopf->list); + kfree(iopf); + } + } + return ret; +} +EXPORT_SYMBOL_GPL(iommu_queue_iopf); + +/** + * iopf_queue_flush_dev - Ensure that all queued faults have been processed + * @dev: the endpoint whose faults need to be flushed. + * + * The IOMMU driver calls this before releasing a PASID, to ensure that all + * pending faults for this PASID have been handled, and won't hit the address + * space of the next process that uses this PASID. The driver must make sure + * that no new fault is added to the queue. In particular it must flush its + * low-level queue before calling this function. + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_flush_dev(struct device *dev) +{ + int ret = 0; + struct iopf_device_param *iopf_param; + struct dev_iommu *param = dev->iommu; + + if (!param) + return -ENODEV; + + mutex_lock(¶m->lock); + iopf_param = param->iopf_param; + if (iopf_param) + flush_workqueue(iopf_param->queue->wq); + else + ret = -ENODEV; + mutex_unlock(¶m->lock); + + return ret; +} +EXPORT_SYMBOL_GPL(iopf_queue_flush_dev); + +/** + * iopf_queue_discard_partial - Remove all pending partial fault + * @queue: the queue whose partial faults need to be discarded + * + * When the hardware queue overflows, last page faults in a group may have been + * lost and the IOMMU driver calls this to discard all partial faults. The + * driver shouldn't be adding new faults to this queue concurrently. + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_discard_partial(struct iopf_queue *queue) +{ + struct iopf_fault *iopf, *next; + struct iopf_device_param *iopf_param; + + if (!queue) + return -EINVAL; + + mutex_lock(&queue->lock); + list_for_each_entry(iopf_param, &queue->devices, queue_list) { + list_for_each_entry_safe(iopf, next, &iopf_param->partial, + list) { + list_del(&iopf->list); + kfree(iopf); + } + } + mutex_unlock(&queue->lock); + return 0; +} +EXPORT_SYMBOL_GPL(iopf_queue_discard_partial); + +/** + * iopf_queue_add_device - Add producer to the fault queue + * @queue: IOPF queue + * @dev: device to add + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_add_device(struct iopf_queue *queue, struct device *dev) +{ + int ret = -EBUSY; + struct iopf_device_param *iopf_param; + struct dev_iommu *param = dev->iommu; + + if (!param) + return -ENODEV; + + iopf_param = kzalloc(sizeof(*iopf_param), GFP_KERNEL); + if (!iopf_param) + return -ENOMEM; + + INIT_LIST_HEAD(&iopf_param->partial); + iopf_param->queue = queue; + iopf_param->dev = dev; + + mutex_lock(&queue->lock); + mutex_lock(¶m->lock); + if (!param->iopf_param) { + list_add(&iopf_param->queue_list, &queue->devices); + param->iopf_param = iopf_param; + ret = 0; + } + mutex_unlock(¶m->lock); + mutex_unlock(&queue->lock); + + if (ret) + kfree(iopf_param); + + return ret; +} +EXPORT_SYMBOL_GPL(iopf_queue_add_device); + +/** + * iopf_queue_remove_device - Remove producer from fault queue + * @queue: IOPF queue + * @dev: device to remove + * + * Caller makes sure that no more faults are reported for this device. + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_remove_device(struct iopf_queue *queue, struct device *dev) +{ + int ret = -EINVAL; + struct iopf_fault *iopf, *next; + struct iopf_device_param *iopf_param; + struct dev_iommu *param = dev->iommu; + + if (!param || !queue) + return -EINVAL; + + mutex_lock(&queue->lock); + mutex_lock(¶m->lock); + iopf_param = param->iopf_param; + if (iopf_param && iopf_param->queue == queue) { + list_del(&iopf_param->queue_list); + param->iopf_param = NULL; + ret = 0; + } + mutex_unlock(¶m->lock); + mutex_unlock(&queue->lock); + if (ret) + return ret; + + /* Just in case some faults are still stuck */ + list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) + kfree(iopf); + + kfree(iopf_param); + + return 0; +} +EXPORT_SYMBOL_GPL(iopf_queue_remove_device); + +/** + * iopf_queue_alloc - Allocate and initialize a fault queue + * @name: a unique string identifying the queue (for workqueue) + * + * Return: the queue on success and NULL on error. + */ +struct iopf_queue *iopf_queue_alloc(const char *name) +{ + struct iopf_queue *queue; + + queue = kzalloc(sizeof(*queue), GFP_KERNEL); + if (!queue) + return NULL; + + /* + * The WQ is unordered because the low-level handler enqueues faults by + * group. PRI requests within a group have to be ordered, but once + * that's dealt with, the high-level function can handle groups out of + * order. + */ + queue->wq = alloc_workqueue("iopf_queue/%s", WQ_UNBOUND, 0, name); + if (!queue->wq) { + kfree(queue); + return NULL; + } + + INIT_LIST_HEAD(&queue->devices); + mutex_init(&queue->lock); + + return queue; +} +EXPORT_SYMBOL_GPL(iopf_queue_alloc); + +/** + * iopf_queue_free - Free IOPF queue + * @queue: queue to free + * + * Counterpart to iopf_queue_alloc(). The driver must not be queuing faults or + * adding/removing devices on this queue anymore. + */ +void iopf_queue_free(struct iopf_queue *queue) +{ + struct iopf_device_param *iopf_param, *next; + + if (!queue) + return; + + list_for_each_entry_safe(iopf_param, next, &queue->devices, queue_list) + iopf_queue_remove_device(queue, iopf_param->dev); + + destroy_workqueue(queue->wq); + kfree(queue); +} +EXPORT_SYMBOL_GPL(iopf_queue_free); From patchwork Tue Mar 2 09:26:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12111319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5973C433E6 for ; Tue, 2 Mar 2021 09:37:39 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 77EE264F09 for ; Tue, 2 Mar 2021 09:37:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 77EE264F09 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kixVa7SSouDHmcMfMZ2HnO6COpUc/3wHHjOnSIIoOvQ=; b=rthKSZMwI1+YeCbvXPthWw0qL F4xSe8z5PGe2EFMJ8SxMwQ2c5ENDvRzIleIMmGQyRn0RaHgYs1EXb6Ba12CQKjVtJdW3VxDmge2cx Kcy+7b+LMSjvdP2k2QZqoU3x51kheAUA4HDOV8UDoi6luB5wfqdQ4a9j7h8GP0b+x2bMt/qZijupq l9t6yeN/4i6X4NEWWW7+MDMzFl0FwFseG+5ovii4u99R2jX5mdbEYUhcU3Ai/8dNKNvbIaKurzGgy vqAfqgunr0psfjKsCXvErsztfyHKLAUXItUuEAa/WNRqVcxTYrXzckVodG/BbuIpGml7H8Zby04xY Ssjg7DaRA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1Rv-0003Pb-75; Tue, 02 Mar 2021 09:36:15 +0000 Received: from mail-wm1-x331.google.com ([2a00:1450:4864:20::331]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RL-0003Au-1e for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 09:35:45 +0000 Received: by mail-wm1-x331.google.com with SMTP id o16so1952089wmh.0 for ; Tue, 02 Mar 2021 01:35:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3TtdFOCenqYX/119xjy/PrrLZWKgY2fUwAevbhJepwI=; b=zfu0A9Ny7ledBeBa93KOT4QZv5DMfxQKAJFp1O5VY0W4OZyhF1LFyqA843M5noop99 eoagFsOb23G53tnAbYInygMwAgJhjH09LAfIizP2g7sHQyexGNUmzHa+rFC8Q6Txx8cc grXZDsdRS7dpHaMA4WGlkS/LODxYVsb2EjU19r+oXgXr4ALyJ+O3j71DrdU+FzoxK51S rEP4r03x7z/oiXDwO/Wi7DvPfxOV9ADaOnqL5ANppu2SwnAAX6BmPq7118eXVaG/qJAq v2wfYozsSoJnziTXQHTwYDxMGbz7YhdNu/elQcW74zMjkhPcoDNvgJct09li3hbDcXEv mBNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3TtdFOCenqYX/119xjy/PrrLZWKgY2fUwAevbhJepwI=; b=UWpnVSjVJlPvmGUrqEbyyVP1nqqtuTV8L96ic0z0bB8qeSoFjflVKHKq1IdmGKxZBO +vFM6KFGmUzbKCQhF/6jbOQKivoz8BTKB4rBdtpkcHU8aAubYc4DY2Z63LZDXeYzXHeF yD8vaHjsQCE2DwZquzSMWuJJ2Wcen7fsW+YGkmv07HaBJ3B4mNr8Emi348r8JeivgB67 hUVLqcK5vbtHTiS9TmeW071AYrnANTCI/id/TMP+ij4iF+/IztNVu+FdpnEP6jBOwP3h EenPYJSLsS25IBrJKIF/7aZoqMrRq+gwnDob3t2hnUmTt4WrBxA/k1iF1JnI+3A4GV0j wvmA== X-Gm-Message-State: AOAM5319kHCqRBRCamu8wa9Tq+5yfuIXTUjybBAvBmZXWC25hnX4FS0F GwD64ZpUKqEjdMaCBRx8+VPTvg== X-Google-Smtp-Source: ABdhPJzY7S8gk936peaMII9wgaYVxlhJLUsmzmQwNhf/VQ7yfv6hHpi5c06vdDJpT6cWcVDWCum9yw== X-Received: by 2002:a1c:6a12:: with SMTP id f18mr3164429wmc.31.1614677738149; Tue, 02 Mar 2021 01:35:38 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w131sm1963818wmb.8.2021.03.02.01.35.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 01:35:37 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Subject: [PATCH v13 07/10] iommu/arm-smmu-v3: Maintain a SID->device structure Date: Tue, 2 Mar 2021 10:26:43 +0100 Message-Id: <20210302092644.2553014-8-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210302092644.2553014-1-jean-philippe@linaro.org> References: <20210302092644.2553014-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_043539_225550_AB8A145A X-CRM114-Status: GOOD ( 24.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: vivek.gautam@arm.com, guohanjun@huawei.com, zhukeqian1@huawei.com, Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, wangzhou1@hisilicon.com, linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When handling faults from the event or PRI queue, we need to find the struct device associated with a SID. Add a rb_tree to keep track of SIDs. Acked-by: Jonathan Cameron Reviewed-by: Eric Auger Signed-off-by: Jean-Philippe Brucker Reviewed-by: Keqian Zhu --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 13 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 157 ++++++++++++++++---- 2 files changed, 140 insertions(+), 30 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index f985817c967a..7b15b7580c6e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -639,6 +639,15 @@ struct arm_smmu_device { /* IOMMU core code handle */ struct iommu_device iommu; + + struct rb_root streams; + struct mutex streams_mutex; +}; + +struct arm_smmu_stream { + u32 id; + struct arm_smmu_master *master; + struct rb_node node; }; /* SMMU private data for each master */ @@ -647,8 +656,8 @@ struct arm_smmu_master { struct device *dev; struct arm_smmu_domain *domain; struct list_head domain_head; - u32 *sids; - unsigned int num_sids; + struct arm_smmu_stream *streams; + unsigned int num_streams; bool ats_enabled; bool sva_enabled; struct list_head bonds; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 7edce914c45e..d148bb6d4289 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -909,8 +909,8 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_for_each_entry(master, &smmu_domain->devices, domain_head) { - for (i = 0; i < master->num_sids; i++) { - cmd.cfgi.sid = master->sids[i]; + for (i = 0; i < master->num_streams; i++) { + cmd.cfgi.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } } @@ -1355,6 +1355,28 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) return 0; } +/* smmu->streams_mutex must be held */ +__maybe_unused +static struct arm_smmu_master * +arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) +{ + struct rb_node *node; + struct arm_smmu_stream *stream; + + node = smmu->streams.rb_node; + while (node) { + stream = rb_entry(node, struct arm_smmu_stream, node); + if (stream->id < sid) + node = node->rb_right; + else if (stream->id > sid) + node = node->rb_left; + else + return stream->master; + } + + return NULL; +} + /* IRQ and event handlers */ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) { @@ -1588,8 +1610,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - for (i = 0; i < master->num_sids; i++) { - cmd.atc.sid = master->sids[i]; + for (i = 0; i < master->num_streams; i++) { + cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_issue_cmd(master->smmu, &cmd); } @@ -1632,8 +1654,8 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!master->ats_enabled) continue; - for (i = 0; i < master->num_sids; i++) { - cmd.atc.sid = master->sids[i]; + for (i = 0; i < master->num_streams; i++) { + cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); } } @@ -2065,13 +2087,13 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) int i, j; struct arm_smmu_device *smmu = master->smmu; - for (i = 0; i < master->num_sids; ++i) { - u32 sid = master->sids[i]; + for (i = 0; i < master->num_streams; ++i) { + u32 sid = master->streams[i].id; __le64 *step = arm_smmu_get_step_for_sid(smmu, sid); /* Bridged PCI devices may end up with duplicated IDs */ for (j = 0; j < i; j++) - if (master->sids[j] == sid) + if (master->streams[j].id == sid) break; if (j < i) continue; @@ -2345,11 +2367,101 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid) return sid < limit; } +static int arm_smmu_insert_master(struct arm_smmu_device *smmu, + struct arm_smmu_master *master) +{ + int i; + int ret = 0; + struct arm_smmu_stream *new_stream, *cur_stream; + struct rb_node **new_node, *parent_node = NULL; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(master->dev); + + master->streams = kcalloc(fwspec->num_ids, sizeof(*master->streams), + GFP_KERNEL); + if (!master->streams) + return -ENOMEM; + master->num_streams = fwspec->num_ids; + + mutex_lock(&smmu->streams_mutex); + for (i = 0; i < fwspec->num_ids; i++) { + u32 sid = fwspec->ids[i]; + + new_stream = &master->streams[i]; + new_stream->id = sid; + new_stream->master = master; + + /* + * Check the SIDs are in range of the SMMU and our stream table + */ + if (!arm_smmu_sid_in_range(smmu, sid)) { + ret = -ERANGE; + break; + } + + /* Ensure l2 strtab is initialised */ + if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) { + ret = arm_smmu_init_l2_strtab(smmu, sid); + if (ret) + break; + } + + /* Insert into SID tree */ + new_node = &(smmu->streams.rb_node); + while (*new_node) { + cur_stream = rb_entry(*new_node, struct arm_smmu_stream, + node); + parent_node = *new_node; + if (cur_stream->id > new_stream->id) { + new_node = &((*new_node)->rb_left); + } else if (cur_stream->id < new_stream->id) { + new_node = &((*new_node)->rb_right); + } else { + dev_warn(master->dev, + "stream %u already in tree\n", + cur_stream->id); + ret = -EINVAL; + break; + } + } + if (ret) + break; + + rb_link_node(&new_stream->node, parent_node, new_node); + rb_insert_color(&new_stream->node, &smmu->streams); + } + + if (ret) { + for (i--; i >= 0; i--) + rb_erase(&master->streams[i].node, &smmu->streams); + kfree(master->streams); + } + mutex_unlock(&smmu->streams_mutex); + + return ret; +} + +static void arm_smmu_remove_master(struct arm_smmu_master *master) +{ + int i; + struct arm_smmu_device *smmu = master->smmu; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(master->dev); + + if (!smmu || !master->streams) + return; + + mutex_lock(&smmu->streams_mutex); + for (i = 0; i < fwspec->num_ids; i++) + rb_erase(&master->streams[i].node, &smmu->streams); + mutex_unlock(&smmu->streams_mutex); + + kfree(master->streams); +} + static struct iommu_ops arm_smmu_ops; static struct iommu_device *arm_smmu_probe_device(struct device *dev) { - int i, ret; + int ret; struct arm_smmu_device *smmu; struct arm_smmu_master *master; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); @@ -2370,27 +2482,12 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; - master->sids = fwspec->ids; - master->num_sids = fwspec->num_ids; INIT_LIST_HEAD(&master->bonds); dev_iommu_priv_set(dev, master); - /* Check the SIDs are in range of the SMMU and our stream table */ - for (i = 0; i < master->num_sids; i++) { - u32 sid = master->sids[i]; - - if (!arm_smmu_sid_in_range(smmu, sid)) { - ret = -ERANGE; - goto err_free_master; - } - - /* Ensure l2 strtab is initialised */ - if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) { - ret = arm_smmu_init_l2_strtab(smmu, sid); - if (ret) - goto err_free_master; - } - } + ret = arm_smmu_insert_master(smmu, master); + if (ret) + goto err_free_master; device_property_read_u32(dev, "pasid-num-bits", &master->ssid_bits); master->ssid_bits = min(smmu->ssid_bits, master->ssid_bits); @@ -2429,6 +2526,7 @@ static void arm_smmu_release_device(struct device *dev) WARN_ON(arm_smmu_master_sva_enabled(master)); arm_smmu_detach_dev(master); arm_smmu_disable_pasid(master); + arm_smmu_remove_master(master); kfree(master); iommu_fwspec_free(dev); } @@ -2852,6 +2950,9 @@ static int arm_smmu_init_structures(struct arm_smmu_device *smmu) { int ret; + mutex_init(&smmu->streams_mutex); + smmu->streams = RB_ROOT; + ret = arm_smmu_init_queues(smmu); if (ret) return ret; From patchwork Tue Mar 2 09:26:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12111321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_NONE,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12B67C433E0 for ; Tue, 2 Mar 2021 09:37:39 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A2DB864F09 for ; Tue, 2 Mar 2021 09:37:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A2DB864F09 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=I3I3IeHCh/WivwYCtirmGMM3oWv6YYOsI/ihb4s0FJ4=; b=ykQIWg0P4KFUETUi2RvKCZyKa 59/6g8vljcLjV8mFSb38aC+Pm4/eMzlEIfPeHZUBsUUuyo+jatUGa1Pob+yxAwigq+POrKs9+EJzY pUNdoqmuh2qh0Pb9+4hxXMyvSl2fzf8vBvgW6R5WLy1N1SSEb1lVptvmdYaceJmCl/aOACoFUDPdl 0EVoClfCifr18nPIPa8o18JhdcPaGxYrfQvlq5j6q+NIWZqSwkdEmOYE7FqmgXT8NCj51QQtEmCXC yPAePHm4Y3RhSbYt57P9p2doSOey3+rrqyIxMhUDdeiYUAB2HW0jUhB/6FhiCEDWlUeqFo0HU5WLF VWfxM/Frg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1Rr-0003Nk-0y; Tue, 02 Mar 2021 09:36:11 +0000 Received: from mail-wm1-x331.google.com ([2a00:1450:4864:20::331]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RM-0003BV-13 for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 09:35:44 +0000 Received: by mail-wm1-x331.google.com with SMTP id m1so1944754wml.2 for ; Tue, 02 Mar 2021 01:35:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Pu6R7mxpT/d03S11wEchY1gMtD/oeS7GHk/CBhJ7EpE=; b=YbLVZfDwQ9GXIPCbEkUwwkRMvR9HrAsO5n0LI/djTCnV4nzBzSVle5iAGKlXTtMtiw eJHkGdaffqr6fcdMWyE/3WX0bGnq5e4HMkSM+Arqh0al6DVOIOumkUXDUrFHxWMmTAIU yTHkxAFLPIlogxWh2jIeXKWWwQMNznjxHFkXDciz8uSvSbZrFZ5654/tP4Y9w+csyJ3K 8zCuF50cetppan8iAK3X2gRwZu5Sim69K/o5vW7IM0u7Dsl3REevyhsWAtRcmw4PNdUs DZmz3vyX8O3JZ05RG3gzWa6Bu0ZHEW+0oNIXUyxi+Is1EL3zHBEwwvKxZRQFSL8503PP yKsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Pu6R7mxpT/d03S11wEchY1gMtD/oeS7GHk/CBhJ7EpE=; b=KTnFon8xcbUhlQUIqSEEG07853qs/bN033g2cDioCF+u1Q1aXOxX9eVuRao5DL9THh AvMzYakuu/1UcR5dZ+0fhIYrbcR2oTgM9od4MyaLSWtwoTTHZUadPq2mkMfMndVm6p2x nMdX9Azl/Q/oARq/p3IChlOHFDlDCs897c5MbQ9gMhRDDOhEzQL2RLmATjknCAEWHvwi UouR7InmNHnBVNWToWjXZbeyapY2r/WYDsF0cZPSXUzLnxNIG8GjYJ3e3uRfW1EKxgh3 eT4tsSIu6vPjiDvhrNmVLHheCiJfpCl1vx14Msu3ffrk+gYy80icPPFYeahRS/M/E0lo WRFg== X-Gm-Message-State: AOAM532YCcJzvPtVirU8EeToE5HaTdLtwK5I834DRHFxKS83Ja8uWOgP va4b8bkDbr8AMg9m4XS+yFj4kw== X-Google-Smtp-Source: ABdhPJzvB901sQ6CVC/aE5VynwlXzB5fbWdQphQ817qOPVn07dFbENigZBrBQ39ius3dHHfobkewXg== X-Received: by 2002:a05:600c:350c:: with SMTP id h12mr2990865wmq.39.1614677739171; Tue, 02 Mar 2021 01:35:39 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w131sm1963818wmb.8.2021.03.02.01.35.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 01:35:38 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Subject: [PATCH v13 08/10] dt-bindings: document stall property for IOMMU masters Date: Tue, 2 Mar 2021 10:26:44 +0100 Message-Id: <20210302092644.2553014-9-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210302092644.2553014-1-jean-philippe@linaro.org> References: <20210302092644.2553014-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_043540_139734_AE6DD646 X-CRM114-Status: GOOD ( 13.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: vivek.gautam@arm.com, guohanjun@huawei.com, zhukeqian1@huawei.com, Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, wangzhou1@hisilicon.com, linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, Rob Herring , eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On ARM systems, some platform devices behind an IOMMU may support stall, which is the ability to recover from page faults. Let the firmware tell us when a device supports stall. Reviewed-by: Eric Auger Reviewed-by: Rob Herring Signed-off-by: Jean-Philippe Brucker --- .../devicetree/bindings/iommu/iommu.txt | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/Documentation/devicetree/bindings/iommu/iommu.txt b/Documentation/devicetree/bindings/iommu/iommu.txt index 3c36334e4f94..26ba9e530f13 100644 --- a/Documentation/devicetree/bindings/iommu/iommu.txt +++ b/Documentation/devicetree/bindings/iommu/iommu.txt @@ -92,6 +92,24 @@ Optional properties: tagging DMA transactions with an address space identifier. By default, this is 0, which means that the device only has one address space. +- dma-can-stall: When present, the master can wait for a transaction to + complete for an indefinite amount of time. Upon translation fault some + IOMMUs, instead of aborting the translation immediately, may first + notify the driver and keep the transaction in flight. This allows the OS + to inspect the fault and, for example, make physical pages resident + before updating the mappings and completing the transaction. Such IOMMU + accepts a limited number of simultaneous stalled transactions before + having to either put back-pressure on the master, or abort new faulting + transactions. + + Firmware has to opt-in stalling, because most buses and masters don't + support it. In particular it isn't compatible with PCI, where + transactions have to complete before a time limit. More generally it + won't work in systems and masters that haven't been designed for + stalling. For example the OS, in order to handle a stalled transaction, + may attempt to retrieve pages from secondary storage in a stalled + domain, leading to a deadlock. + Notes: ====== From patchwork Tue Mar 2 09:26:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12111317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63784C433DB for ; Tue, 2 Mar 2021 09:37:34 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 057E464EF1 for ; Tue, 2 Mar 2021 09:37:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 057E464EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=T+FpqgIGNz3WJhz4XEnvHTmaBNnz6td349kBv2m87zA=; b=BRvx8kPNytKRbNPhUjAnWL/hV JQKgD0mZlkOce9cnw8aSz5GywiySbljbZkhxSKEVkxFZqdLIK2cdzi+qgIOOC4WiHEayhX6XDePi7 eaiaFZNsKjm/mbPyC14f1iyfAgWRkVEfLMwpXIBbB/GZ5EaPfdq19C2MajpucJVkQflySk9jD17Xm QstKvzflyJPFs+/sgrFdUbTGsC75RvD+xcORH+hK4IrYRi6ixDkZxWjIz9YQVLQIEc92TrHVZuU2W vcGt6Y3ewqsnNM9UnFR1AA98I7LOQWxCjpZ6vHcGL0L4xOqClVi3IqdkoqghrA1BNAsP3NmHp/d48 WAloOn9Bg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1Rs-0003Od-U6; Tue, 02 Mar 2021 09:36:12 +0000 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RN-0003C6-3R for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 09:35:44 +0000 Received: by mail-wr1-x436.google.com with SMTP id u14so19100060wri.3 for ; Tue, 02 Mar 2021 01:35:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Tljaa6sLGzqVEkWTy8WFZ8+1JVN/U6naVu2IpICY8qc=; b=CUqHhZIYRju55gc+SgJJfVnY562fuTxugAA10rym5tTkb8g8m152o+DT++duK/z+Fp 5E+eDlGNVXS5r3h4+KvyMUE9P96vilcFmZT+EnHNNa9MJl+MZ1zyKriTkgySFqJ8NoVM 6Rt2AA3J3Kk2WMRXeGu9VXSp9tJHuj7/jS077RIVONKPfbBxA6kJHgZVfAhhszUAkLLJ gYael+2uxEimdDjb5FL2gB2UfzfmgCSRrAELmBRIepAtkUEQimQU/LFw6NJEfqsvhei5 bWJl4B+z4JSM+OExnA+1C6+FYFU2kn+LWS556rFB6/W/5FQVf2A38rvwhRVQXW+Vc1Hb FOdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Tljaa6sLGzqVEkWTy8WFZ8+1JVN/U6naVu2IpICY8qc=; b=OPi6ONEl1pcGr3z/01lzMAV94HjY0fE5ZejNqZ+osEZJI3eKFGHEhCP5LyeKCaG0Jz KDuz3foBl1A8q52jhIF3GNpKZH9xpW5MKLGiO1j21KnaSgN1eCtInkM1ZaEv6HUcuX7P x81UycUJqwH9VdX/ap3zM0gmiiIuKLWx0JUJX4WUyEkjleniSh82O3prnPAQ8ns/wTQA 7t0GEID39/geQjwFW3wO4Pm0pR6nZnmKdoQTL7J9yqNVOP6jFoREzhx1Ipzfn74QE7fW vAKioDM5MWJV92AMiumhExM65n/nV/xeIVQIs54FNNFASStvnJqpuZ+lX+0LesAc18pR IDXg== X-Gm-Message-State: AOAM5335Wv8krBO0zzK0cAfQBsOX/JvGwAhnFt9ZEyMza73TBKKpV4UT 19Pu30y4bjxXfhIavVv+AFDdew== X-Google-Smtp-Source: ABdhPJy5shG4cAWZu1Stz095i3ckx38acnf/wOYqacuVRhrdrmVSW4ctRbes8RS7Kd/oDkBHRXqASA== X-Received: by 2002:adf:f8cd:: with SMTP id f13mr20377429wrq.27.1614677740193; Tue, 02 Mar 2021 01:35:40 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w131sm1963818wmb.8.2021.03.02.01.35.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 01:35:39 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Subject: [PATCH v13 09/10] ACPI/IORT: Enable stall support for platform devices Date: Tue, 2 Mar 2021 10:26:45 +0100 Message-Id: <20210302092644.2553014-10-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210302092644.2553014-1-jean-philippe@linaro.org> References: <20210302092644.2553014-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_043541_264293_5BF4B70C X-CRM114-Status: GOOD ( 13.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: vivek.gautam@arm.com, guohanjun@huawei.com, zhukeqian1@huawei.com, Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, wangzhou1@hisilicon.com, linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Copy the "Stall supported" bit, that tells whether a named component supports stall, into the dma-can-stall device property. Acked-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- drivers/acpi/arm64/iort.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index 3912a1f6058e..0828f70cb782 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -968,13 +968,15 @@ static int iort_pci_iommu_init(struct pci_dev *pdev, u16 alias, void *data) static void iort_named_component_init(struct device *dev, struct acpi_iort_node *node) { - struct property_entry props[2] = {}; + struct property_entry props[3] = {}; struct acpi_iort_named_component *nc; nc = (struct acpi_iort_named_component *)node->node_data; props[0] = PROPERTY_ENTRY_U32("pasid-num-bits", FIELD_GET(ACPI_IORT_NC_PASID_BITS, nc->node_flags)); + if (nc->node_flags & ACPI_IORT_NC_STALL_SUPPORTED) + props[1] = PROPERTY_ENTRY_BOOL("dma-can-stall"); if (device_add_properties(dev, props)) dev_warn(dev, "Could not add device properties\n"); From patchwork Tue Mar 2 09:26:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12111325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD1D8C433DB for ; Tue, 2 Mar 2021 09:37:55 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5BCDF64EEB for ; Tue, 2 Mar 2021 09:37:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5BCDF64EEB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/0TkYgoCiyYiL++U1IDtOuj/pqty0pBZmhsVTDumIZI=; b=W1YUBhayohwNVUohby1zQYN4q zgHSVbuBRaKyj5JwYD0tyIVwrvAAPJHQ3qeDJPjSe3vwxdqSReVgSS5JfQ4B9GvRxqoBWjUh0BfVS 1+WnEoRVtUPSVWk8leh9LEpmv+8aSwu/0bPcR+27NtJu55fhW4aSb5YphS0JgCCIiI+S3586PEtVk AHzta+XJGAiQN1dVCNsbt4eMuhwBxZKN2QYIzliQYzpjDkYyotpQ5eHd8eR8c7JXkONTi3swEtZgF RxN2H0RyAXk6dKdOYZv2bIqKfIQQr013kp+sD4NBxFo0fXCOA/ckNodaG/3zehZ0vLZNmWWH5zanH qWwd1eNVg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1SA-0003XC-Bm; Tue, 02 Mar 2021 09:36:30 +0000 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lH1RO-0003Cc-GQ for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 09:35:53 +0000 Received: by mail-wr1-x42b.google.com with SMTP id v15so19103011wrx.4 for ; Tue, 02 Mar 2021 01:35:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=orm2KtjVcaI4UVNOov/3Vndf1AzScWJ8h9dkL+Zlq3s=; b=t3/c3jOIlxKnSlIIGZ6mdOGLzQtkr8R64HzKOdlXjroeBijNnTUggu9LxBIyU8K6qP WaPq86LwTKL1D5yRqLxLGAHYlrjXv3xyQqavYPxzPqrmvRFkdI5sNSgTErSZyR8r3HYU bpVX1qDRrsfof3iHZ4Ms7I9R7AE6p6o8JeN7rIsP+QbFAhHodiRWSmTaNo3iXaLOEsru w3vYYQFQLkG/MRm0lMqEFH17RgMiVoLQdtW9M3VX1Rr7ScIo/oAjcrhE/GfD7foattvk /uul2Jp9PFK7TY6ce4W0PWZotIYtn3nVVi1BtdGtV40718RlOX9rpAwNgiTeKm6iAW13 r10A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=orm2KtjVcaI4UVNOov/3Vndf1AzScWJ8h9dkL+Zlq3s=; b=Nq2iUvSs4o8ST+UCwcycSmueT+0wGSaSSdTGiYtpCt24bFefSDHltT9hlF0EKv0jKH o3eBXgt1rbihVCX+1IuiMuVZ+moaalTF9lbWsckr9EBObb+R8ZNnEOd5uz+0QhLU92aq qbJuIEHK0vOhoHPE+3GjHHiBFSvNDQazTLT3mGY0v+aZ1Cz0ilCX+f5v9x5Oh+1Qx3C+ fAPto3Nz3TpTph9akAANBiJnATSkn2QkeMXa1mJxaN1Q1T3KjdCtn9NET12jyLONF4IC /ctSbnrxoyJZ0RS07cszkv+XaE2vXaMgtIMu1ae6Qc7PLXTgm7UYYlk5C4kTXy3jdGG4 xdPg== X-Gm-Message-State: AOAM530Lsm4P7qNKtVHcqZ1hXk/N2EcTg1N3bO1sISanYjtSb4mHBu0k jdytONg9l6V93U+HA0TSxWFRQg== X-Google-Smtp-Source: ABdhPJzjH4KjE/WoaosBvgFojVXHkSUvDDZ2kiXDTAUKGSvVy3CtY0icr2lC29HoR11uC8JmRHU3Gg== X-Received: by 2002:a5d:526b:: with SMTP id l11mr20961775wrc.385.1614677741366; Tue, 02 Mar 2021 01:35:41 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w131sm1963818wmb.8.2021.03.02.01.35.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 01:35:40 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Subject: [PATCH v13 10/10] iommu/arm-smmu-v3: Add stall support for platform devices Date: Tue, 2 Mar 2021 10:26:46 +0100 Message-Id: <20210302092644.2553014-11-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210302092644.2553014-1-jean-philippe@linaro.org> References: <20210302092644.2553014-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_043542_659585_E0C485CC X-CRM114-Status: GOOD ( 27.74 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: vivek.gautam@arm.com, guohanjun@huawei.com, zhukeqian1@huawei.com, Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, wangzhou1@hisilicon.com, linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The SMMU provides a Stall model for handling page faults in platform devices. It is similar to PCIe PRI, but doesn't require devices to have their own translation cache. Instead, faulting transactions are parked and the OS is given a chance to fix the page tables and retry the transaction. Enable stall for devices that support it (opt-in by firmware). When an event corresponds to a translation error, call the IOMMU fault handler. If the fault is recoverable, it will call us back to terminate or continue the stall. To use stall device drivers need to enable IOMMU_DEV_FEAT_IOPF, which initializes the fault queue for the device. Tested-by: Zhangfei Gao Reviewed-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker Reviewed-by: Eric Auger Reviewed-by: Eric Auger --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 43 ++++ .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 59 +++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 196 +++++++++++++++++- 3 files changed, 283 insertions(+), 15 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 7b15b7580c6e..59af0bbd2f7b 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -354,6 +354,13 @@ #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) #define CMDQ_PRI_1_RESP GENMASK_ULL(13, 12) +#define CMDQ_RESUME_0_RESP_TERM 0UL +#define CMDQ_RESUME_0_RESP_RETRY 1UL +#define CMDQ_RESUME_0_RESP_ABORT 2UL +#define CMDQ_RESUME_0_RESP GENMASK_ULL(13, 12) +#define CMDQ_RESUME_0_SID GENMASK_ULL(63, 32) +#define CMDQ_RESUME_1_STAG GENMASK_ULL(15, 0) + #define CMDQ_SYNC_0_CS GENMASK_ULL(13, 12) #define CMDQ_SYNC_0_CS_NONE 0 #define CMDQ_SYNC_0_CS_IRQ 1 @@ -370,6 +377,25 @@ #define EVTQ_0_ID GENMASK_ULL(7, 0) +#define EVT_ID_TRANSLATION_FAULT 0x10 +#define EVT_ID_ADDR_SIZE_FAULT 0x11 +#define EVT_ID_ACCESS_FAULT 0x12 +#define EVT_ID_PERMISSION_FAULT 0x13 + +#define EVTQ_0_SSV (1UL << 11) +#define EVTQ_0_SSID GENMASK_ULL(31, 12) +#define EVTQ_0_SID GENMASK_ULL(63, 32) +#define EVTQ_1_STAG GENMASK_ULL(15, 0) +#define EVTQ_1_STALL (1UL << 31) +#define EVTQ_1_PnU (1UL << 33) +#define EVTQ_1_InD (1UL << 34) +#define EVTQ_1_RnW (1UL << 35) +#define EVTQ_1_S2 (1UL << 39) +#define EVTQ_1_CLASS GENMASK_ULL(41, 40) +#define EVTQ_1_TT_READ (1UL << 44) +#define EVTQ_2_ADDR GENMASK_ULL(63, 0) +#define EVTQ_3_IPA GENMASK_ULL(51, 12) + /* PRI queue */ #define PRIQ_ENT_SZ_SHIFT 4 #define PRIQ_ENT_DWORDS ((1 << PRIQ_ENT_SZ_SHIFT) >> 3) @@ -464,6 +490,13 @@ struct arm_smmu_cmdq_ent { enum pri_resp resp; } pri; + #define CMDQ_OP_RESUME 0x44 + struct { + u32 sid; + u16 stag; + u8 resp; + } resume; + #define CMDQ_OP_CMD_SYNC 0x46 struct { u64 msiaddr; @@ -522,6 +555,7 @@ struct arm_smmu_cmdq_batch { struct arm_smmu_evtq { struct arm_smmu_queue q; + struct iopf_queue *iopf; u32 max_stalls; }; @@ -659,7 +693,9 @@ struct arm_smmu_master { struct arm_smmu_stream *streams; unsigned int num_streams; bool ats_enabled; + bool stall_enabled; bool sva_enabled; + bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; }; @@ -678,6 +714,7 @@ struct arm_smmu_domain { struct io_pgtable_ops *pgtbl_ops; bool non_strict; + bool stall_enabled; atomic_t nr_ats_masters; enum arm_smmu_domain_stage stage; @@ -719,6 +756,7 @@ bool arm_smmu_master_sva_supported(struct arm_smmu_master *master); bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); +bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata); void arm_smmu_sva_unbind(struct iommu_sva *handle); @@ -750,6 +788,11 @@ static inline int arm_smmu_master_disable_sva(struct arm_smmu_master *master) return -ENODEV; } +static inline bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master) +{ + return false; +} + static inline struct iommu_sva * arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index bb251cab61f3..ee66d1f4cb81 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -435,9 +435,13 @@ bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) return true; } -static bool arm_smmu_iopf_supported(struct arm_smmu_master *master) +bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master) { - return false; + /* We're not keeping track of SIDs in fault events */ + if (master->num_streams != 1) + return false; + + return master->stall_enabled; } bool arm_smmu_master_sva_supported(struct arm_smmu_master *master) @@ -445,8 +449,8 @@ bool arm_smmu_master_sva_supported(struct arm_smmu_master *master) if (!(master->smmu->features & ARM_SMMU_FEAT_SVA)) return false; - /* SSID and IOPF support are mandatory for the moment */ - return master->ssid_bits && arm_smmu_iopf_supported(master); + /* SSID support is mandatory for the moment */ + return master->ssid_bits; } bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master) @@ -459,13 +463,55 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master) return enabled; } +static int arm_smmu_master_sva_enable_iopf(struct arm_smmu_master *master) +{ + int ret; + struct device *dev = master->dev; + + /* + * Drivers for devices supporting PRI or stall should enable IOPF first. + * Others have device-specific fault handlers and don't need IOPF. + */ + if (!arm_smmu_master_iopf_supported(master)) + return 0; + + if (!master->iopf_enabled) + return -EINVAL; + + ret = iopf_queue_add_device(master->smmu->evtq.iopf, dev); + if (ret) + return ret; + + ret = iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev); + if (ret) { + iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + return ret; + } + return 0; +} + +static void arm_smmu_master_sva_disable_iopf(struct arm_smmu_master *master) +{ + struct device *dev = master->dev; + + if (!master->iopf_enabled) + return; + + iommu_unregister_device_fault_handler(dev); + iopf_queue_remove_device(master->smmu->evtq.iopf, dev); +} + int arm_smmu_master_enable_sva(struct arm_smmu_master *master) { + int ret; + mutex_lock(&sva_lock); - master->sva_enabled = true; + ret = arm_smmu_master_sva_enable_iopf(master); + if (!ret) + master->sva_enabled = true; mutex_unlock(&sva_lock); - return 0; + return ret; } int arm_smmu_master_disable_sva(struct arm_smmu_master *master) @@ -476,6 +522,7 @@ int arm_smmu_master_disable_sva(struct arm_smmu_master *master) mutex_unlock(&sva_lock); return -EBUSY; } + arm_smmu_master_sva_disable_iopf(master); master->sva_enabled = false; mutex_unlock(&sva_lock); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index d148bb6d4289..363744df8d51 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -32,6 +32,7 @@ #include #include "arm-smmu-v3.h" +#include "../../iommu-sva-lib.h" static bool disable_bypass = true; module_param(disable_bypass, bool, 0444); @@ -315,6 +316,11 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) } cmd[1] |= FIELD_PREP(CMDQ_PRI_1_RESP, ent->pri.resp); break; + case CMDQ_OP_RESUME: + cmd[0] |= FIELD_PREP(CMDQ_RESUME_0_SID, ent->resume.sid); + cmd[0] |= FIELD_PREP(CMDQ_RESUME_0_RESP, ent->resume.resp); + cmd[1] |= FIELD_PREP(CMDQ_RESUME_1_STAG, ent->resume.stag); + break; case CMDQ_OP_CMD_SYNC: if (ent->sync.msiaddr) { cmd[0] |= FIELD_PREP(CMDQ_SYNC_0_CS, CMDQ_SYNC_0_CS_IRQ); @@ -878,6 +884,44 @@ static int arm_smmu_cmdq_batch_submit(struct arm_smmu_device *smmu, return arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmds, cmds->num, true); } +static int arm_smmu_page_response(struct device *dev, + struct iommu_fault_event *unused, + struct iommu_page_response *resp) +{ + struct arm_smmu_cmdq_ent cmd = {0}; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + int sid = master->streams[0].id; + + if (master->stall_enabled) { + cmd.opcode = CMDQ_OP_RESUME; + cmd.resume.sid = sid; + cmd.resume.stag = resp->grpid; + switch (resp->code) { + case IOMMU_PAGE_RESP_INVALID: + case IOMMU_PAGE_RESP_FAILURE: + cmd.resume.resp = CMDQ_RESUME_0_RESP_ABORT; + break; + case IOMMU_PAGE_RESP_SUCCESS: + cmd.resume.resp = CMDQ_RESUME_0_RESP_RETRY; + break; + default: + return -EINVAL; + } + } else { + return -ENODEV; + } + + arm_smmu_cmdq_issue_cmd(master->smmu, &cmd); + /* + * Don't send a SYNC, it doesn't do anything for RESUME or PRI_RESP. + * RESUME consumption guarantees that the stalled transaction will be + * terminated... at some point in the future. PRI_RESP is fire and + * forget. + */ + + return 0; +} + /* Context descriptor manipulation functions */ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) { @@ -988,7 +1032,6 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, u64 val; bool cd_live; __le64 *cdptr; - struct arm_smmu_device *smmu = smmu_domain->smmu; if (WARN_ON(ssid >= (1 << smmu_domain->s1_cfg.s1cdmax))) return -E2BIG; @@ -1033,8 +1076,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; - /* STALL_MODEL==0b10 && CD.S==0 is ILLEGAL */ - if (smmu->features & ARM_SMMU_FEAT_STALL_FORCE) + if (smmu_domain->stall_enabled) val |= CTXDESC_CD_0_S; } @@ -1278,7 +1320,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, FIELD_PREP(STRTAB_STE_1_STRW, strw)); if (smmu->features & ARM_SMMU_FEAT_STALLS && - !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) + !master->stall_enabled) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); val |= (s1_cfg->cdcfg.cdtab_dma & STRTAB_STE_0_S1CTXPTR_MASK) | @@ -1356,7 +1398,6 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) } /* smmu->streams_mutex must be held */ -__maybe_unused static struct arm_smmu_master * arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) { @@ -1378,9 +1419,103 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) } /* IRQ and event handlers */ +static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) +{ + int ret; + u32 reason; + u32 perm = 0; + struct arm_smmu_master *master; + bool ssid_valid = evt[0] & EVTQ_0_SSV; + u32 sid = FIELD_GET(EVTQ_0_SID, evt[0]); + struct iommu_fault_event fault_evt = { }; + struct iommu_fault *flt = &fault_evt.fault; + + switch (FIELD_GET(EVTQ_0_ID, evt[0])) { + case EVT_ID_TRANSLATION_FAULT: + reason = IOMMU_FAULT_REASON_PTE_FETCH; + break; + case EVT_ID_ADDR_SIZE_FAULT: + reason = IOMMU_FAULT_REASON_OOR_ADDRESS; + break; + case EVT_ID_ACCESS_FAULT: + reason = IOMMU_FAULT_REASON_ACCESS; + break; + case EVT_ID_PERMISSION_FAULT: + reason = IOMMU_FAULT_REASON_PERMISSION; + break; + default: + return -EOPNOTSUPP; + } + + /* Stage-2 is always pinned at the moment */ + if (evt[1] & EVTQ_1_S2) + return -EFAULT; + + if (evt[1] & EVTQ_1_RnW) + perm |= IOMMU_FAULT_PERM_READ; + else + perm |= IOMMU_FAULT_PERM_WRITE; + + if (evt[1] & EVTQ_1_InD) + perm |= IOMMU_FAULT_PERM_EXEC; + + if (evt[1] & EVTQ_1_PnU) + perm |= IOMMU_FAULT_PERM_PRIV; + + if (evt[1] & EVTQ_1_STALL) { + flt->type = IOMMU_FAULT_PAGE_REQ; + flt->prm = (struct iommu_fault_page_request) { + .flags = IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE, + .grpid = FIELD_GET(EVTQ_1_STAG, evt[1]), + .perm = perm, + .addr = FIELD_GET(EVTQ_2_ADDR, evt[2]), + }; + + if (ssid_valid) { + flt->prm.flags |= IOMMU_FAULT_PAGE_REQUEST_PASID_VALID; + flt->prm.pasid = FIELD_GET(EVTQ_0_SSID, evt[0]); + } + } else { + flt->type = IOMMU_FAULT_DMA_UNRECOV; + flt->event = (struct iommu_fault_unrecoverable) { + .reason = reason, + .flags = IOMMU_FAULT_UNRECOV_ADDR_VALID, + .perm = perm, + .addr = FIELD_GET(EVTQ_2_ADDR, evt[2]), + }; + + if (ssid_valid) { + flt->event.flags |= IOMMU_FAULT_UNRECOV_PASID_VALID; + flt->event.pasid = FIELD_GET(EVTQ_0_SSID, evt[0]); + } + } + + mutex_lock(&smmu->streams_mutex); + master = arm_smmu_find_master(smmu, sid); + if (!master) { + ret = -EINVAL; + goto out_unlock; + } + + ret = iommu_report_device_fault(master->dev, &fault_evt); + if (ret && flt->type == IOMMU_FAULT_PAGE_REQ) { + /* Nobody cared, abort the access */ + struct iommu_page_response resp = { + .pasid = flt->prm.pasid, + .grpid = flt->prm.grpid, + .code = IOMMU_PAGE_RESP_FAILURE, + }; + arm_smmu_page_response(master->dev, &fault_evt, &resp); + } + +out_unlock: + mutex_unlock(&smmu->streams_mutex); + return ret; +} + static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) { - int i; + int i, ret; struct arm_smmu_device *smmu = dev; struct arm_smmu_queue *q = &smmu->evtq.q; struct arm_smmu_ll_queue *llq = &q->llq; @@ -1390,6 +1525,10 @@ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) while (!queue_remove_raw(q, evt)) { u8 id = FIELD_GET(EVTQ_0_ID, evt[0]); + ret = arm_smmu_handle_evt(smmu, evt); + if (!ret) + continue; + dev_info(smmu->dev, "event 0x%02x received:\n", id); for (i = 0; i < ARRAY_SIZE(evt); ++i) dev_info(smmu->dev, "\t0x%016llx\n", @@ -1924,6 +2063,8 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, cfg->s1cdmax = master->ssid_bits; + smmu_domain->stall_enabled = master->stall_enabled; + ret = arm_smmu_alloc_cd_tables(smmu_domain); if (ret) goto out_free_asid; @@ -2271,6 +2412,12 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) smmu_domain->s1_cfg.s1cdmax, master->ssid_bits); ret = -EINVAL; goto out_unlock; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && + smmu_domain->stall_enabled != master->stall_enabled) { + dev_err(dev, "cannot attach to stall-%s domain\n", + smmu_domain->stall_enabled ? "enabled" : "disabled"); + ret = -EINVAL; + goto out_unlock; } master->domain = smmu_domain; @@ -2506,6 +2653,11 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->ssid_bits = min_t(u8, master->ssid_bits, CTXDESC_LINEAR_CDMAX); + if ((smmu->features & ARM_SMMU_FEAT_STALLS && + device_property_read_bool(dev, "dma-can-stall")) || + smmu->features & ARM_SMMU_FEAT_STALL_FORCE) + master->stall_enabled = true; + return &smmu->iommu; err_free_master: @@ -2523,7 +2675,8 @@ static void arm_smmu_release_device(struct device *dev) return; master = dev_iommu_priv_get(dev); - WARN_ON(arm_smmu_master_sva_enabled(master)); + if (WARN_ON(arm_smmu_master_sva_enabled(master))) + iopf_queue_remove_device(master->smmu->evtq.iopf, dev); arm_smmu_detach_dev(master); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); @@ -2651,6 +2804,8 @@ static bool arm_smmu_dev_has_feature(struct device *dev, return false; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + return arm_smmu_master_iopf_supported(master); case IOMMU_DEV_FEAT_SVA: return arm_smmu_master_sva_supported(master); default: @@ -2667,6 +2822,8 @@ static bool arm_smmu_dev_feature_enabled(struct device *dev, return false; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + return master->iopf_enabled; case IOMMU_DEV_FEAT_SVA: return arm_smmu_master_sva_enabled(master); default: @@ -2677,6 +2834,8 @@ static bool arm_smmu_dev_feature_enabled(struct device *dev, static int arm_smmu_dev_enable_feature(struct device *dev, enum iommu_dev_features feat) { + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + if (!arm_smmu_dev_has_feature(dev, feat)) return -ENODEV; @@ -2684,8 +2843,11 @@ static int arm_smmu_dev_enable_feature(struct device *dev, return -EBUSY; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + master->iopf_enabled = true; + return 0; case IOMMU_DEV_FEAT_SVA: - return arm_smmu_master_enable_sva(dev_iommu_priv_get(dev)); + return arm_smmu_master_enable_sva(master); default: return -EINVAL; } @@ -2694,12 +2856,19 @@ static int arm_smmu_dev_enable_feature(struct device *dev, static int arm_smmu_dev_disable_feature(struct device *dev, enum iommu_dev_features feat) { + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + if (!arm_smmu_dev_feature_enabled(dev, feat)) return -EINVAL; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + if (master->sva_enabled) + return -EBUSY; + master->iopf_enabled = false; + return 0; case IOMMU_DEV_FEAT_SVA: - return arm_smmu_master_disable_sva(dev_iommu_priv_get(dev)); + return arm_smmu_master_disable_sva(master); default: return -EINVAL; } @@ -2730,6 +2899,7 @@ static struct iommu_ops arm_smmu_ops = { .sva_bind = arm_smmu_sva_bind, .sva_unbind = arm_smmu_sva_unbind, .sva_get_pasid = arm_smmu_sva_get_pasid, + .page_response = arm_smmu_page_response, .pgsize_bitmap = -1UL, /* Restricted during device attach */ }; @@ -2827,6 +2997,13 @@ static int arm_smmu_init_queues(struct arm_smmu_device *smmu) if (ret) return ret; + if ((smmu->features & ARM_SMMU_FEAT_SVA) && + (smmu->features & ARM_SMMU_FEAT_STALLS)) { + smmu->evtq.iopf = iopf_queue_alloc(dev_name(smmu->dev)); + if (!smmu->evtq.iopf) + return -ENOMEM; + } + /* priq */ if (!(smmu->features & ARM_SMMU_FEAT_PRI)) return 0; @@ -3742,6 +3919,7 @@ static int arm_smmu_device_remove(struct platform_device *pdev) iommu_device_unregister(&smmu->iommu); iommu_device_sysfs_remove(&smmu->iommu); arm_smmu_device_disable(smmu); + iopf_queue_free(smmu->evtq.iopf); return 0; }