From patchwork Mon May 1 20:03:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13228253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6388FC7EE21 for ; Mon, 1 May 2023 20:03:52 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.528260.821176 (Exim 4.92) (envelope-from ) id 1ptZkC-0005IF-Bg; Mon, 01 May 2023 20:03:32 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 528260.821176; Mon, 01 May 2023 20:03:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZkC-0005I8-8d; Mon, 01 May 2023 20:03:32 +0000 Received: by outflank-mailman (input) for mailman id 528260; Mon, 01 May 2023 20:03:30 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZkA-000525-Hn for xen-devel@lists.xenproject.org; Mon, 01 May 2023 20:03:30 +0000 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on20601.outbound.protection.outlook.com [2a01:111:f400:7e8a::601]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 3d35e810-e85b-11ed-b225-6b7b168915f2; Mon, 01 May 2023 22:03:29 +0200 (CEST) Received: from MW4PR03CA0268.namprd03.prod.outlook.com (2603:10b6:303:b4::33) by PH0PR12MB5646.namprd12.prod.outlook.com (2603:10b6:510:143::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.26; Mon, 1 May 2023 20:03:26 +0000 Received: from CO1NAM11FT032.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b4:cafe::2) by MW4PR03CA0268.outlook.office365.com (2603:10b6:303:b4::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.31 via Frontend Transport; Mon, 1 May 2023 20:03:25 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT032.mail.protection.outlook.com (10.13.174.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.20 via Frontend Transport; Mon, 1 May 2023 20:03:25 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 1 May 2023 15:03:24 -0500 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 1 May 2023 15:03:24 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Mon, 1 May 2023 15:03:23 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 3d35e810-e85b-11ed-b225-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AqZ/4rh6XbSxYd/ji14LZGcdeDQm7k1Zf5I2jMgWISTCJQ6bOv/XM8vfQah/68DnRhSTvLxCc7W3vokeXrigee51mGOsPEshKNjiwG+eAeoey/MPhBLz6ZBtqUgGBkBLvpIrnuAznADUZuIBvsZsLgCBzaetN7XN7aNi1jUzpFJ6ohzNCVegBqXs6jklO0ZtD4KiG66P03HDUGzpfsGkTW4oQ4P7xJLsEzewlmzmbLSkFLcif/QLNXW273pwYQcFC/ACw7Flxk5sQBioJ3JgfnbpZAfUulSnIsP5XQKICWp7Rm09vfmfAD20JePEEadsPUdpxNm/4Kfw4f7g6AlCUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EAeerVBnHarRbBFzP8eA8mfPIPjOdFnbNXDV9Rw/NiY=; b=QKTFOr2+L6TT9LmB24VAx/aWHmqldaT3JSeJXYWsdZHO7R69uV27VtM2MUpQg+8eZr9yvGW2ZshF2E9SGG1VegUn/Bw4JV7FGHshBhWw71a11ts884Q1urNRq+ZMBHuXUpExRbAeLNzN6H+oJG+bK6oeNc16g+46Y0ab4Wc0WZnDLAweC2nrnLezZir7PxbAt5Coei5rwT7lqlGT3s74XN//E29UzWltuIsfw6YebLMPtknMXMkbik9lBB6P87scy9LdBYjtskVBLzjE2QtGQ/us7gb1IMsuZZ2RGioJY9kjZcNwLGpVClxAC5+3ituUgsHFZPBtTbPtKs2Zt81FBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EAeerVBnHarRbBFzP8eA8mfPIPjOdFnbNXDV9Rw/NiY=; b=J+gcXbHaoFZ2ByEWu0VdephSDtAPkF9GFpgvRlAYX+/c1laSPCC0xs0skpQ25VZZgpZpI9BEm1AbB2XJrgfWVRQ55arblnYE/Qrqx/6JwaAP34aZDx7JtOfbqJ2QYUaeY4/wr1SQY+tGnX7Zcx1veKlHWy4GcBD/o5eiRMZLDWs= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Rahul Singh , Stewart Hildebrand Subject: [PATCH v1 1/6] xen/arm: Move is_protected flag to struct device Date: Mon, 1 May 2023 16:03:00 -0400 Message-ID: <20230501200305.168058-2-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230501200305.168058-1-stewart.hildebrand@amd.com> References: <20230501200305.168058-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT032:EE_|PH0PR12MB5646:EE_ X-MS-Office365-Filtering-Correlation-Id: 893b28b4-cda2-406b-784b-08db4a7f1f8b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xdwpARfOl4Si5U24LX0KjYnqGjOBU6slMV5yPGtgU+00+WO8ucRcbsYJEzqWnOBWOzn2GtuwgKgqz82us3QSlrcrT7EQ68Ju2iYj427m2kVjTHekKlTDBsGb9zrRIggi3SSyCLaHYPX6O0Q4CB9xrnvIpEo4x9UI5TNyNa12bIPnknuqIiim94vREsajHTBP11NN8uHnp5YE2IZrXRajtGrrem7g9Fv6rVYKJn3aCi/Fx8tsvE29MvS/Ym07YWuWntsiuS5zkZCutGBBDzrsE4kDocgXXt2R1MR0K3dh0V3VpVDlsIV+I1fVMBtwu3B1NGDIY6GKSlk7FvynXEP+OwODXS8uaXfAlD3Rt0XQ3pFg3kma9oiNX00l1udfv0fUGgbyTlZo51Q2ZFXnEQ3Lu3eJ4Uy2LVtdTNMt87zIMuX0T+oMl+XAurCqySEJVmgH5BCld7/qB2efZiLXUEByPAXejZF9Z5cMWMeXa717K22QlFnDWY6BVVqJZsQ3aUhNtUZ4aEYw6i5sNiztEHYyS0hOl/6ZPWZN+GUbH6oePU9jxDiJVu8plJdE0DOFfl7hsQktn8wHy+LTGZeEFVqvoQkFc1B4laWI0ZOSpfe5rULMGetqeU2nblAiDBgkH6e+09/A6EJif3mxUUmOusE5NiX7y7WWCe6npL5FHO/EKrkApUNNfmGgC8f0lCzGHUqi2mQ7uReD/tHruIXmAbLDMz0Pm6SVGCBgzq4alRGjH7Y= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(396003)(39860400002)(376002)(136003)(451199021)(46966006)(40470700004)(36840700001)(70586007)(2906002)(70206006)(44832011)(2616005)(336012)(86362001)(5660300002)(82310400005)(4326008)(8936002)(40460700003)(36756003)(6666004)(41300700001)(6916009)(54906003)(478600001)(316002)(8676002)(40480700001)(356005)(26005)(81166007)(1076003)(966005)(82740400003)(47076005)(426003)(186003)(83380400001)(36860700001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2023 20:03:25.2187 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 893b28b4-cda2-406b-784b-08db4a7f1f8b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT032.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB5646 From: Oleksandr Tyshchenko This flag will be re-used for PCI devices by the subsequent patches. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand --- downstream->v1: * rebase * s/dev_node->is_protected/dev_node->dev.is_protected/ in smmu.c * s/dt_device_set_protected(dev_to_dt(dev))/device_set_protected(dev)/ in smmu-v3.c * remove redundant device_is_protected checks in smmu-v3.c/ipmmu-vmsa.c (cherry picked from commit 59753aac77528a584d3950936b853ebf264b68e7 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/arch/arm/domain_build.c | 4 ++-- xen/arch/arm/include/asm/device.h | 13 +++++++++++++ xen/common/device_tree.c | 2 +- xen/drivers/passthrough/arm/ipmmu-vmsa.c | 8 +------- xen/drivers/passthrough/arm/smmu-v3.c | 7 +------ xen/drivers/passthrough/arm/smmu.c | 2 +- xen/drivers/passthrough/device_tree.c | 15 +++++++++------ xen/include/xen/device_tree.h | 13 ------------- 8 files changed, 28 insertions(+), 36 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index f80fdd1af206..106f92c65a61 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -2507,7 +2507,7 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev, return res; } - if ( dt_device_is_protected(dev) ) + if ( device_is_protected(dt_to_dev(dev)) ) { dt_dprintk("%s setup iommu\n", dt_node_full_name(dev)); res = iommu_assign_dt_device(d, dev); @@ -3007,7 +3007,7 @@ static int __init handle_passthrough_prop(struct kernel_info *kinfo, return res; /* If xen_force, we allow assignment of devices without IOMMU protection. */ - if ( xen_force && !dt_device_is_protected(node) ) + if ( xen_force && !device_is_protected(dt_to_dev(node)) ) return 0; return iommu_assign_dt_device(kinfo->d, node); diff --git a/xen/arch/arm/include/asm/device.h b/xen/arch/arm/include/asm/device.h index b5d451e08776..086dde13eb6b 100644 --- a/xen/arch/arm/include/asm/device.h +++ b/xen/arch/arm/include/asm/device.h @@ -1,6 +1,8 @@ #ifndef __ASM_ARM_DEVICE_H #define __ASM_ARM_DEVICE_H +#include + enum device_type { DEV_DT, @@ -20,6 +22,7 @@ struct device #endif struct dev_archdata archdata; struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */ + bool is_protected; /* Shows that device is protected by IOMMU */ }; typedef struct device device_t; @@ -94,6 +97,16 @@ int device_init(struct dt_device_node *dev, enum device_class class, */ enum device_class device_get_class(const struct dt_device_node *dev); +static inline void device_set_protected(struct device *device) +{ + device->is_protected = true; +} + +static inline bool device_is_protected(const struct device *device) +{ + return device->is_protected; +} + #define DT_DEVICE_START(_name, _namestr, _class) \ static const struct device_desc __dev_desc_##_name __used \ __section(".dev.info") = { \ diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c index 6c9712ab7bda..1d5d7cb5f01b 100644 --- a/xen/common/device_tree.c +++ b/xen/common/device_tree.c @@ -1874,7 +1874,7 @@ static unsigned long __init unflatten_dt_node(const void *fdt, /* By default dom0 owns the device */ np->used_by = 0; /* By default the device is not protected */ - np->is_protected = false; + np->dev.is_protected = false; INIT_LIST_HEAD(&np->domain_list); if ( new_format ) diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthrough/arm/ipmmu-vmsa.c index 091f09b21752..039212a3a990 100644 --- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -1288,14 +1288,8 @@ static int ipmmu_add_device(u8 devfn, struct device *dev) if ( !to_ipmmu(dev) ) return -ENODEV; - if ( dt_device_is_protected(dev_to_dt(dev)) ) - { - dev_err(dev, "Already added to IPMMU\n"); - return -EEXIST; - } - /* Let Xen know that the master device is protected by an IOMMU. */ - dt_device_set_protected(dev_to_dt(dev)); + device_set_protected(dev); dev_info(dev, "Added master device (IPMMU %s micro-TLBs %u)\n", dev_name(fwspec->iommu_dev), fwspec->num_ids); diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c index bfdb62b395ad..4b452e6fdd00 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -1521,13 +1521,8 @@ static int arm_smmu_add_device(u8 devfn, struct device *dev) */ arm_smmu_enable_pasid(master); - if (dt_device_is_protected(dev_to_dt(dev))) { - dev_err(dev, "Already added to SMMUv3\n"); - return -EEXIST; - } - /* Let Xen know that the master device is protected by an IOMMU. */ - dt_device_set_protected(dev_to_dt(dev)); + device_set_protected(dev); dev_info(dev, "Added master device (SMMUv3 %s StreamIds %u)\n", dev_name(fwspec->iommu_dev), fwspec->num_ids); diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c index 0a514821b336..5b6024d579a8 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -838,7 +838,7 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu, master->of_node = dev_node; /* Xen: Let Xen know that the device is protected by an SMMU */ - dt_device_set_protected(dev_node); + device_set_protected(dev); for (i = 0; i < fwspec->num_ids; ++i) { if (!(smmu->features & ARM_SMMU_FEAT_STREAM_MATCH) && diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index 1c32d7b50cce..b5bd13393b56 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -34,7 +34,7 @@ int iommu_assign_dt_device(struct domain *d, struct dt_device_node *dev) if ( !is_iommu_enabled(d) ) return -EINVAL; - if ( !dt_device_is_protected(dev) ) + if ( !device_is_protected(dt_to_dev(dev)) ) return -EINVAL; spin_lock(&dtdevs_lock); @@ -65,7 +65,7 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev) if ( !is_iommu_enabled(d) ) return -EINVAL; - if ( !dt_device_is_protected(dev) ) + if ( !device_is_protected(dt_to_dev(dev)) ) return -EINVAL; spin_lock(&dtdevs_lock); @@ -87,7 +87,7 @@ static bool_t iommu_dt_device_is_assigned(const struct dt_device_node *dev) { bool_t assigned = 0; - if ( !dt_device_is_protected(dev) ) + if ( !device_is_protected(dt_to_dev(dev)) ) return 0; spin_lock(&dtdevs_lock); @@ -141,12 +141,15 @@ int iommu_add_dt_device(struct dt_device_node *np) return -EINVAL; /* - * The device may already have been registered. As there is no harm in - * it just return success early. + * This is needed in case a device has both the iommus property and + * also appears in the mmu-masters list. */ - if ( dev_iommu_fwspec_get(dev) ) + if ( device_is_protected(dev) ) return 0; + if ( dev_iommu_fwspec_get(dev) ) + return -EEXIST; + /* * According to the Documentation/devicetree/bindings/iommu/iommu.txt * from Linux. diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h index 19a74909cece..c1e4751a581f 100644 --- a/xen/include/xen/device_tree.h +++ b/xen/include/xen/device_tree.h @@ -90,9 +90,6 @@ struct dt_device_node { struct dt_device_node *next; /* TODO: Remove it. Only use to know the last children */ struct dt_device_node *allnext; - /* IOMMU specific fields */ - bool is_protected; - /* HACK: Remove this if there is a need of space */ bool_t static_evtchn_created; @@ -302,16 +299,6 @@ static inline domid_t dt_device_used_by(const struct dt_device_node *device) return device->used_by; } -static inline void dt_device_set_protected(struct dt_device_node *device) -{ - device->is_protected = true; -} - -static inline bool dt_device_is_protected(const struct dt_device_node *device) -{ - return device->is_protected; -} - static inline bool_t dt_property_name_is_equal(const struct dt_property *pp, const char *name) { From patchwork Mon May 1 20:03:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13228252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 32DA9C77B73 for ; Mon, 1 May 2023 20:03:52 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.528261.821187 (Exim 4.92) (envelope-from ) id 1ptZkJ-0005cr-Pz; Mon, 01 May 2023 20:03:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 528261.821187; Mon, 01 May 2023 20:03:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZkJ-0005ci-MW; Mon, 01 May 2023 20:03:39 +0000 Received: by outflank-mailman (input) for mailman id 528261; Mon, 01 May 2023 20:03:38 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZkI-000525-PU for xen-devel@lists.xenproject.org; Mon, 01 May 2023 20:03:38 +0000 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2062f.outbound.protection.outlook.com [2a01:111:f400:fe5a::62f]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 41eb799f-e85b-11ed-b225-6b7b168915f2; Mon, 01 May 2023 22:03:37 +0200 (CEST) Received: from BN8PR16CA0024.namprd16.prod.outlook.com (2603:10b6:408:4c::37) by CO6PR12MB5409.namprd12.prod.outlook.com (2603:10b6:5:357::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.30; Mon, 1 May 2023 20:03:34 +0000 Received: from BN8NAM11FT091.eop-nam11.prod.protection.outlook.com (2603:10b6:408:4c:cafe::db) by BN8PR16CA0024.outlook.office365.com (2603:10b6:408:4c::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.30 via Frontend Transport; Mon, 1 May 2023 20:03:34 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by BN8NAM11FT091.mail.protection.outlook.com (10.13.176.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.20 via Frontend Transport; Mon, 1 May 2023 20:03:33 +0000 Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 1 May 2023 15:03:33 -0500 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB08.amd.com (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 1 May 2023 13:03:33 -0700 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Mon, 1 May 2023 15:03:32 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 41eb799f-e85b-11ed-b225-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LV0L+cf9/+6ujJZdLT8zIve3cMVi3MpH41QJrGA7CHEm4Q4bUgznShMOcJt80isbi9oScwHO1tQ1yyN3pIDsqUe+E7XujEyBxWcPnls5z+p9VQWlKlB70yRnCYYzwqn871TmJIxOy5oJAD66NMnTqB9khf4gHY9KBMLyvXp4XL5Ee1wlPhyptlXCJShRqGTkbLEf8GdJDo9mTysBEVa+lLk38YgZS1ro4EHmMkSO9NBNVvo7m1Fm50VriNcN/N166twvnQqR8dfTvRfX+bv71eowKXkprixLgMQDcnAqoFplhmJD96R7otYw+YqwGqEZyPyOyN5DRxs/iuZLJQGgOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FH3cbOq7w4Ry4gUimwrCXWO3YUo9mU/EEVtFtBj29+A=; b=Twdih1oH9w/ylQfYLwykufELuND7MvGz+L/nnqynK0t/AHIlrRbSbnnDENCuAAxSoEjoypXDVhFIttZhiDZLBETdIcTw6nO0TnVwRpa9jkKPWZW4DCKwl1wjZMpKibXe7M8eAM+MJKUk1FYPGQNs6JksM9ikY26W3FebZPogpwDfAtZgYYAkeNnJDHykR6pIUIaaEmyCr49zLoQ0t5MVim3EKe4TrYcvlOo73xznYS6t7HDiMWvpJb4wOde5rfFMv6YiQOsEX7EIAeXoOhCvGsfAzXmo1aEhxF58RSkQ0ndqxKfkdAy6P9UBNxOqc3BfqeBiRos0SyPNOZEToalGKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FH3cbOq7w4Ry4gUimwrCXWO3YUo9mU/EEVtFtBj29+A=; b=gb1DePfI6A2wEm5sog39kHFHnEzCTWnFg/y79Tfmwd7O9JC+dexz5WD0EsN0L/u8sVvH8KofezLyqcwpLE8qO91o7KxMBBkDCsIyLBGw2t5HgjJm+krxhwy1KoPEjXQAaBhMp3tWIBHwnj45KRfvobpb7zEt3gXCtG99yI2ZRys= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Stewart Hildebrand Subject: [PATCH v1 2/6] iommu/arm: Add iommu_dt_xlate() Date: Mon, 1 May 2023 16:03:01 -0400 Message-ID: <20230501200305.168058-3-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230501200305.168058-1-stewart.hildebrand@amd.com> References: <20230501200305.168058-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT091:EE_|CO6PR12MB5409:EE_ X-MS-Office365-Filtering-Correlation-Id: 8cebc486-355f-4c39-d618-08db4a7f24ac X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: haDaEyABepE2+Gd97f0WweYm957fdnjDqu1y9Tqk61FEjA6lT5xKt7LwNaydMWmDZ7I77xuy/Eucn1nzrilsG0x2MRPlZ6GUNpGtsMeapI+R0MoHXR3v4yg5xJqQueT06qoj833NWBAwI62BrhrxtNdOHfto+V82EP0/L/v2ZuytWl7rlVhtXb5byFjLVkqE/KxDbi7zuysm+xt8CvtwkHcoMTWcoypxKsz4vv7NeDQHM5vqtM/+pOwZ7AsqhX3Ny6GugI6iHPgOmNAqZVSCB7x2Z2HFeDhqkfmYy19JhxAcWpRxvc0xIgmPLQhAu88/hP8/U0ThVo73FAtloxfAO5lwm0JcyJCwHnehxPrnAdqyEKkeLbdgG26ng5sro0C/Yt4490vxFu9bsyRG1dhnPYZM+7iG/lB6WcAHLMI2VnJHIZkR9nLJWlqCAkP4a11l3Y/blD/UUAP6avM1NlzZp3hldQIKacjtEzL5HCfdqCV3XztBx2fogf5qh2q+Rq+OEU9nl0jh1fJNf5UOJQi6/3y2crM5H9gQYArHlJlHZynt/H9se1LT72kJtXhog26qf1OZoCGRwLC1whvSvosNTBqOenFL7jlzrr8PY6KFwBbrMJbFV2QrlssW9NUmz/GY5kr/8IaE+0BJuDAzBdRWxPlCk5DmC/vcNQ1jRuHbXBVeLPQhh9Hi6B3i4oTNk231YaSN2ECcCmHuILieZvkqmzW1Fj9Sk6Ec301LZWjMIIo= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(396003)(39860400002)(136003)(346002)(376002)(451199021)(46966006)(36840700001)(40470700004)(40480700001)(82310400005)(40460700003)(316002)(2616005)(4326008)(1076003)(336012)(426003)(70586007)(41300700001)(47076005)(83380400001)(36860700001)(26005)(478600001)(54906003)(70206006)(6916009)(6666004)(966005)(186003)(356005)(82740400003)(2906002)(36756003)(81166007)(86362001)(44832011)(8936002)(8676002)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2023 20:03:33.9197 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8cebc486-355f-4c39-d618-08db4a7f24ac X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT091.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5409 From: Oleksandr Tyshchenko Move code for processing DT IOMMU specifier to a separate helper. This helper will be re-used for adding PCI devices by the subsequent patches as we will need exact the same actions for processing DT PCI-IOMMU specifier. While at it introduce NO_IOMMU to avoid magic "1". Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand # rename --- downstream->v1: * trivial rebase * s/dt_iommu_xlate/iommu_dt_xlate/ (cherry picked from commit c26bab0415ca303df86aba1d06ef8edc713734d3 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/drivers/passthrough/device_tree.c | 42 +++++++++++++++++---------- 1 file changed, 27 insertions(+), 15 deletions(-) diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index b5bd13393b56..1b50f4670944 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -127,15 +127,39 @@ int iommu_release_dt_devices(struct domain *d) return 0; } +/* This correlation must not be altered */ +#define NO_IOMMU 1 + +static int iommu_dt_xlate(struct device *dev, + struct dt_phandle_args *iommu_spec) +{ + const struct iommu_ops *ops = iommu_get_ops(); + int rc; + + if ( !dt_device_is_available(iommu_spec->np) ) + return NO_IOMMU; + + rc = iommu_fwspec_init(dev, &iommu_spec->np->dev); + if ( rc ) + return rc; + + /* + * Provide DT IOMMU specifier which describes the IOMMU master + * interfaces of that device (device IDs, etc) to the driver. + * The driver is responsible to decide how to interpret them. + */ + return ops->dt_xlate(dev, iommu_spec); +} + int iommu_add_dt_device(struct dt_device_node *np) { const struct iommu_ops *ops = iommu_get_ops(); struct dt_phandle_args iommu_spec; struct device *dev = dt_to_dev(np); - int rc = 1, index = 0; + int rc = NO_IOMMU, index = 0; if ( !iommu_enabled ) - return 1; + return NO_IOMMU; if ( !ops ) return -EINVAL; @@ -164,19 +188,7 @@ int iommu_add_dt_device(struct dt_device_node *np) if ( !ops->add_device || !ops->dt_xlate ) return -EINVAL; - if ( !dt_device_is_available(iommu_spec.np) ) - break; - - rc = iommu_fwspec_init(dev, &iommu_spec.np->dev); - if ( rc ) - break; - - /* - * Provide DT IOMMU specifier which describes the IOMMU master - * interfaces of that device (device IDs, etc) to the driver. - * The driver is responsible to decide how to interpret them. - */ - rc = ops->dt_xlate(dev, &iommu_spec); + rc = iommu_dt_xlate(dev, &iommu_spec); if ( rc ) break; From patchwork Mon May 1 20:03:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13228254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 11976C77B73 for ; Mon, 1 May 2023 20:04:05 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.528264.821197 (Exim 4.92) (envelope-from ) id 1ptZkX-0006HD-2d; Mon, 01 May 2023 20:03:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 528264.821197; Mon, 01 May 2023 20:03:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZkW-0006H5-W9; Mon, 01 May 2023 20:03:52 +0000 Received: by outflank-mailman (input) for mailman id 528264; Mon, 01 May 2023 20:03:51 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZkV-000525-Du for xen-devel@lists.xenproject.org; Mon, 01 May 2023 20:03:51 +0000 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on20627.outbound.protection.outlook.com [2a01:111:f400:7e8a::627]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 492ecb6f-e85b-11ed-b225-6b7b168915f2; Mon, 01 May 2023 22:03:49 +0200 (CEST) Received: from MW4P223CA0027.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:80::32) by DM4PR12MB8497.namprd12.prod.outlook.com (2603:10b6:8:180::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.30; Mon, 1 May 2023 20:03:46 +0000 Received: from CO1NAM11FT049.eop-nam11.prod.protection.outlook.com (2603:10b6:303:80:cafe::d8) by MW4P223CA0027.outlook.office365.com (2603:10b6:303:80::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.30 via Frontend Transport; Mon, 1 May 2023 20:03:45 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT049.mail.protection.outlook.com (10.13.175.50) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.20 via Frontend Transport; Mon, 1 May 2023 20:03:44 +0000 Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 1 May 2023 15:03:43 -0500 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB08.amd.com (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 1 May 2023 13:03:43 -0700 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Mon, 1 May 2023 15:03:42 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 492ecb6f-e85b-11ed-b225-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=itY6LV+fQEhdFUOO4+G6drQvRa474rhgPkD3dElQ0n/CBoSn8ViLrJrFWXL0XthXLcqkYQa4m/3lQeMxjBTXOFk45z2VajOni1UCp732CiKiGaTby8K7KeWqVO2xHu+krlZz+goTIIyTXhNBAcrGp0BQAHLb+InkVvU0efz/ydHGhM4UlwII2d2ZwPSMzNqzKjI+SK422/dSJ9qiJR7EKNCWNUNACoDZsH76dL8ABNjQj94+Gw6gOWAMexnLLfFaqWaX0J7untUGwARtqAz0HAJJwfmYRaAIn77UFHJjg9BGWaP8c2EnagZ5yM8FT0VL/l12GEyQp40dIcJ0WmhaAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tPTXU+0KLiFp2//TpisX/eYI6MvOPIIjwmGHW7rytQA=; b=SCNFZJ+Em/zIOeUhRJtm8k7cfah0bZqbMPBJuS0C8mxSAZydfHaDhI5FTp02ltEQ9QZuR5ijMBtTrTYNeZhn6wo5tuPFmHDFS4ogn9ELo8Dl7qs5qV0dow/bpkMG/LwYgi7bTARS+fhF9QZQCNAF7qufAxusRw9W3lNBk54cWHJrj2S5Z3TCys6Mj8q0xCWJvRA2hdRZMf0IHW0Yz3dPuj9wNiHTUdS6qlLCgigpInM/QAp49sF/vTMG8DW5jDKzfXDCwomhNeL6TyGX0OunrJtUPLAKHUSbNt2Jn8Htz1TgrZGpGjubApSmx7Z5SV4F5MtPzDN/FoBRri1x8RMQKw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tPTXU+0KLiFp2//TpisX/eYI6MvOPIIjwmGHW7rytQA=; b=jNewda7fLrpIonms0NAPfIMB+Z/unDJRI/jFgpuewQo+7kr+Wxp+YpQVJhBtUKHry0KPItf/CDe3kdm6JcTvwq8uLRwTRxba1hvi01bgtncVEnUj2h3S0XSULc8bfTz7NDDxldtMPNx9FFe+Ztvt50Npabv6+E23nvRJhvMJLdc= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Jan Beulich , Paul Durrant , =?utf-8?q?Rog?= =?utf-8?q?er_Pau_Monn=C3=A9?= , "Stewart Hildebrand" Subject: [PATCH v1 3/6] iommu/arm: Introduce iommu_add_dt_pci_device API Date: Mon, 1 May 2023 16:03:02 -0400 Message-ID: <20230501200305.168058-4-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230501200305.168058-1-stewart.hildebrand@amd.com> References: <20230501200305.168058-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT049:EE_|DM4PR12MB8497:EE_ X-MS-Office365-Filtering-Correlation-Id: 47adc05a-10e5-4517-7712-08db4a7f2b49 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zETcY6KrC2KwwoH7dFvA7/l0iOb8nsC9CPbbncqRqnkpP6ZYaumC5eRjNFJsaBg+7QwQBFb3n8gHlmr/VJRW2zHNeYWHDJjauwprvXR9kW59I31UxauinjSppIWrZqdvpXCwyXs99UGC98/0XjvJMHMf9OWTl9DyyQKKwLmKxh8QiZ46WgXI10Xo4kdV+KBukgyuXgMj2D4+CFOpeJNbGyUbmRgCwqvLG6fuphV7D5MYHzLw9biDZNTvvR0sKTz2aCZELbgYSHHUuxIFxHlgL66/61mhQD+swFhY9mDXWNuehZkQJIn5zsL3ZQWmyrrR63DrsJD+xH7KCpXJ9Eky88x6s/Kv+mzZl0XwfKmGnJ6Ys80RxFRm7W+Qv6mIgFjkSbAOhQS3jE+L3AJDhu6w/QnPlJmBxkJs+O5PKyeDhOxTaxPfx/c3wy+LmPjmWex5mIVFqFYqeXfgOQyJq31McKkQs2tnw0PIcojYlPVegi61BKuA3tJ6mRqtYYzGd+hTy9g20Z/e191wRRQPw0AI79O1ckbQ/d0mi+i0JpfMmjnMY8Ys4U/B3BsTAzw/eYCQR2NfLQcALrnnxrm5Q9MlSnd0B6VINEF9oSwsByv68zE9yEHxvromlyH+ty8GcxPzDg7BuFnMMdtaXKZtKThOLQywjQter5XpjUlh1YnvGTNAN3uoGMQ7ScYtOQV+1/LO0b9UeGinCtMg3+56aRaNPEBP66TI+6X6rOcGNgsNM4Kr4WMCU/JQHhYZPRPA2zA8RNVUX5Lc75jf/3WoefU3qsYbWHUO0EpQQ/C3LaaLJkQ= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(39860400002)(396003)(346002)(136003)(451199021)(40470700004)(36840700001)(46966006)(356005)(81166007)(8936002)(82310400005)(478600001)(8676002)(82740400003)(6916009)(86362001)(40480700001)(4326008)(70586007)(70206006)(41300700001)(36756003)(54906003)(316002)(40460700003)(36860700001)(2906002)(83380400001)(47076005)(44832011)(966005)(186003)(1076003)(26005)(426003)(336012)(2616005)(5660300002)(6666004)(21314003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2023 20:03:44.9031 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 47adc05a-10e5-4517-7712-08db4a7f2b49 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT049.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB8497 From: Oleksandr Tyshchenko The main purpose of this patch is to add a way to register PCI device (which is behind the IOMMU) using the generic PCI-IOMMU DT bindings [1] before assigning that device to a domain. This behaves in almost the same way as existing iommu_add_dt_device API, the difference is in devices to handle and DT bindings to use. The function of_map_id to translate an ID through a downstream mapping (which is also suitable for mapping Requester ID) was borrowed from Linux (v5.10-rc6) and updated according to the Xen code base. XXX: I don't port pci_for_each_dma_alias from Linux which is a part of PCI-IOMMU bindings infrastucture as I don't have a good understanding for how it is expected to work in Xen environment. Also it is not completely clear whether we need to distinguish between different PCI types here (DEV_TYPE_PCI, DEV_TYPE_PCI_HOST_BRIDGE, etc). For example, how we should behave here if the host bridge doesn't have a stream ID (so not described in iommu-map property) just simple fail or bypasses translation? [1] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/pci-iommu.txt Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand --- downstream->v1: * rebase * add const qualifier to struct dt_device_node *np arg in dt_map_id() * add const qualifier to struct dt_device_node *np declaration in iommu_add_pci_device() * use stdint.h types instead of u8/u32/etc... * rename functions: s/dt_iommu_xlate/iommu_dt_xlate/ s/dt_map_id/iommu_dt_pci_map_id/ s/iommu_add_pci_device/iommu_add_dt_pci_device/ * add device_is_protected check in iommu_add_dt_pci_device * wrap prototypes in CONFIG_HAS_PCI (cherry picked from commit 734e3bf6ee77e7947667ab8fa96c25b349c2e1da from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/drivers/passthrough/device_tree.c | 145 ++++++++++++++++++++++++++ xen/include/xen/device_tree.h | 25 +++++ xen/include/xen/iommu.h | 6 +- 3 files changed, 175 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index 1b50f4670944..ef98f343eef2 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -151,6 +151,151 @@ static int iommu_dt_xlate(struct device *dev, return ops->dt_xlate(dev, iommu_spec); } +#ifdef CONFIG_HAS_PCI +int iommu_dt_pci_map_id(const struct dt_device_node *np, uint32_t id, + const char *map_name, const char *map_mask_name, + struct dt_device_node **target, uint32_t *id_out) +{ + uint32_t map_mask, masked_id, map_len; + const __be32 *map = NULL; + + if ( !np || !map_name || (!target && !id_out) ) + return -EINVAL; + + map = dt_get_property(np, map_name, &map_len); + if ( !map ) + { + if ( target ) + return -ENODEV; + /* Otherwise, no map implies no translation */ + *id_out = id; + return 0; + } + + if ( !map_len || map_len % (4 * sizeof(*map)) ) + { + printk(XENLOG_ERR "%pOF: Error: Bad %s length: %d\n", np, + map_name, map_len); + return -EINVAL; + } + + /* The default is to select all bits. */ + map_mask = 0xffffffff; + + /* + * Can be overridden by "{iommu,msi}-map-mask" property. + * If of_property_read_u32() fails, the default is used. + */ + if ( map_mask_name ) + dt_property_read_u32(np, map_mask_name, &map_mask); + + masked_id = map_mask & id; + for ( ; (int)map_len > 0; map_len -= 4 * sizeof(*map), map += 4 ) + { + struct dt_device_node *phandle_node; + uint32_t id_base = be32_to_cpup(map + 0); + uint32_t phandle = be32_to_cpup(map + 1); + uint32_t out_base = be32_to_cpup(map + 2); + uint32_t id_len = be32_to_cpup(map + 3); + + if ( id_base & ~map_mask ) + { + printk(XENLOG_ERR "%pOF: Invalid %s translation - %s-mask (0x%x) ignores id-base (0x%x)\n", + np, map_name, map_name, map_mask, id_base); + return -EFAULT; + } + + if ( masked_id < id_base || masked_id >= id_base + id_len ) + continue; + + phandle_node = dt_find_node_by_phandle(phandle); + if ( !phandle_node ) + return -ENODEV; + + if ( target ) + { + if ( !*target ) + *target = phandle_node; + + if ( *target != phandle_node ) + continue; + } + + if ( id_out ) + *id_out = masked_id - id_base + out_base; + + printk(XENLOG_DEBUG "%pOF: %s, using mask %08x, id-base: %08x, out-base: %08x, length: %08x, id: %08x -> %08x\n", + np, map_name, map_mask, id_base, out_base, id_len, id, + masked_id - id_base + out_base); + return 0; + } + + printk(XENLOG_ERR "%pOF: no %s translation for id 0x%x on %pOF\n", + np, map_name, id, target && *target ? *target : NULL); + + /* + * NOTE: Linux bypasses translation without returning an error here, + * but should we behave in the same way on Xen? Restrict for now. + */ + return -EFAULT; +} + +int iommu_add_dt_pci_device(uint8_t devfn, struct pci_dev *pdev) +{ + const struct iommu_ops *ops = iommu_get_ops(); + struct dt_phandle_args iommu_spec = { .args_count = 1 }; + struct device *dev = pci_to_dev(pdev); + const struct dt_device_node *np; + int rc = NO_IOMMU; + + if ( !iommu_enabled ) + return NO_IOMMU; + + if ( !ops ) + return -EINVAL; + + if ( device_is_protected(dev) ) + return 0; + + if ( dev_iommu_fwspec_get(dev) ) + return -EEXIST; + + np = pci_find_host_bridge_node(pdev); + if ( !np ) + return -ENODEV; + + /* + * According to the Documentation/devicetree/bindings/pci/pci-iommu.txt + * from Linux. + */ + rc = iommu_dt_pci_map_id(np, PCI_BDF(pdev->bus, devfn), "iommu-map", + "iommu-map-mask", &iommu_spec.np, iommu_spec.args); + if ( rc ) + return rc == -ENODEV ? NO_IOMMU : rc; + + /* + * The driver which supports generic PCI-IOMMU DT bindings must have + * these callback implemented. + */ + if ( !ops->add_device || !ops->dt_xlate ) + return -EINVAL; + + rc = iommu_dt_xlate(dev, &iommu_spec); + + /* + * Add master device to the IOMMU if latter is present and available. + * The driver is responsible to mark that device as protected. + */ + if ( !rc ) + rc = ops->add_device(devfn, dev); + + if ( rc < 0 ) + iommu_fwspec_free(dev); + + return rc; +} +#endif /* CONFIG_HAS_PCI */ + int iommu_add_dt_device(struct dt_device_node *np) { const struct iommu_ops *ops = iommu_get_ops(); diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h index c1e4751a581f..dc40fdfb9231 100644 --- a/xen/include/xen/device_tree.h +++ b/xen/include/xen/device_tree.h @@ -852,6 +852,31 @@ int dt_count_phandle_with_args(const struct dt_device_node *np, */ int dt_get_pci_domain_nr(struct dt_device_node *node); +#ifdef CONFIG_HAS_PCI +/** + * iommu_dt_pci_map_id - Translate an ID through a downstream mapping. + * @np: root complex device node. + * @id: device ID to map. + * @map_name: property name of the map to use. + * @map_mask_name: optional property name of the mask to use. + * @target: optional pointer to a target device node. + * @id_out: optional pointer to receive the translated ID. + * + * Given a device ID, look up the appropriate implementation-defined + * platform ID and/or the target device which receives transactions on that + * ID, as per the "iommu-map" and "msi-map" bindings. Either of @target or + * @id_out may be NULL if only the other is required. If @target points to + * a non-NULL device node pointer, only entries targeting that node will be + * matched; if it points to a NULL value, it will receive the device node of + * the first matching target phandle, with a reference held. + * + * Return: 0 on success or a standard error code on failure. + */ +int iommu_dt_pci_map_id(const struct dt_device_node *np, uint32_t id, + const char *map_name, const char *map_mask_name, + struct dt_device_node **target, uint32_t *id_out); +#endif /* CONFIG_HAS_PCI */ + struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle); #ifdef CONFIG_DEVICE_TREE_DEBUG diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h index 405db59971c5..d1b91ec13056 100644 --- a/xen/include/xen/iommu.h +++ b/xen/include/xen/iommu.h @@ -219,7 +219,8 @@ int iommu_dt_domain_init(struct domain *d); int iommu_release_dt_devices(struct domain *d); /* - * Helper to add master device to the IOMMU using generic IOMMU DT bindings. + * Helpers to add master device to the IOMMU using generic (PCI-)IOMMU + * DT bindings. * * Return values: * 0 : device is protected by an IOMMU @@ -228,6 +229,9 @@ int iommu_release_dt_devices(struct domain *d); * (IOMMU is not enabled/present or device is not connected to it). */ int iommu_add_dt_device(struct dt_device_node *np); +#ifdef CONFIG_HAS_PCI +int iommu_add_dt_pci_device(uint8_t devfn, struct pci_dev *pdev); +#endif int iommu_do_dt_domctl(struct xen_domctl *, struct domain *, XEN_GUEST_HANDLE_PARAM(xen_domctl_t)); From patchwork Mon May 1 20:03:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13228255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2F72DC77B7C for ; Mon, 1 May 2023 20:04:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.528267.821206 (Exim 4.92) (envelope-from ) id 1ptZkd-0006ht-BY; Mon, 01 May 2023 20:03:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 528267.821206; Mon, 01 May 2023 20:03:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZkd-0006hf-8Y; Mon, 01 May 2023 20:03:59 +0000 Received: by outflank-mailman (input) for mailman id 528267; Mon, 01 May 2023 20:03:57 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZkb-000525-TZ for xen-devel@lists.xenproject.org; Mon, 01 May 2023 20:03:57 +0000 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on20621.outbound.protection.outlook.com [2a01:111:f400:fe5b::621]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 4da851d7-e85b-11ed-b225-6b7b168915f2; Mon, 01 May 2023 22:03:57 +0200 (CEST) Received: from MW4PR03CA0347.namprd03.prod.outlook.com (2603:10b6:303:dc::22) by PH7PR12MB6762.namprd12.prod.outlook.com (2603:10b6:510:1ac::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.30; Mon, 1 May 2023 20:03:54 +0000 Received: from CO1NAM11FT009.eop-nam11.prod.protection.outlook.com (2603:10b6:303:dc:cafe::6f) by MW4PR03CA0347.outlook.office365.com (2603:10b6:303:dc::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.30 via Frontend Transport; Mon, 1 May 2023 20:03:53 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT009.mail.protection.outlook.com (10.13.175.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.20 via Frontend Transport; Mon, 1 May 2023 20:03:53 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 1 May 2023 15:03:52 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Mon, 1 May 2023 15:03:51 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4da851d7-e85b-11ed-b225-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kROiZiIA4cLY5yWmMHra6ZecDER05QnXBno6CzGvKXzhwDgCyKrGVlEPPgn/rNBwAgR1jrCnRxA2hQQOWonC+FtQCrJGCHlGXs6pznkhTTM5jiY6TtIwXK7w8uC8PgowZKMEchEkURt7km7cZs/lDfuZh3PMUiq1VO6/XkxD5HiirYAqh1GN3NDD72RnW7WkhZ4uEnAVKu2FYBl1TavXZNNGLskiOks0LbTYnevVJzklDh6np18XRLv0qm4Xj2Xzavk8pFWVgd4/h5s0et2N6Umxn2gML0nGqzxKxdk/9SkOMwPViUZNlj962I4soEyAcO0DquBlHo5IxPvUcuVz9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eCX9DqGe9db1MXdGpEVHFk+gaS1vUShIki9NWo1vZU8=; b=XO7Yk9gqBVOU/nHgCjzj2TFvEhNxvXbSdpfvjGNIgyuCp0TF/Z7ULuNkGAZqor4CUMI9bAt0QR0C4EAL30I5vDnD6adLd+P2NZCW1X0mQaD5NW/7xzab9Lsscv3JcRVEk22CgbmM6rzrPvTOo3juMicDJKU4slGdTMcsQSE/3KtQ+BxLCpEF+MW7MFJUg1KxXhivuwbezGodE/UIpM07M10SuQ4xKE3f1FvCpP6XHyKJNor3AVrfQ+qk5B2LbmK0bO2NrD9IUtSkX4yyigj9EKxyyOQp+DvYSzajyrj+UuRkdk7G5Eweg+f6N5WKfvetMzbo2O7gSWn08PtDelfsVA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eCX9DqGe9db1MXdGpEVHFk+gaS1vUShIki9NWo1vZU8=; b=u6LVLwI4G8an8UAnAZJc5KrGq37sRtf2keRD7tgB68K13JORbuPahCiJNZ6Ad9RhA0Feglb28tXME2tbuPOJy3ypGAdzflHQx8fWEilft0to8nrEo+78umjhBK8W7CKthmZeAFjagIGilBKXkpZEwnIacYug4vOBoxGGKgpJLnE= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Jan Beulich , Paul Durrant , =?utf-8?q?Rog?= =?utf-8?q?er_Pau_Monn=C3=A9?= , "Stewart Hildebrand" Subject: [PATCH v1 4/6] pci/arm: Use iommu_add_dt_pci_device() instead of arch hook Date: Mon, 1 May 2023 16:03:03 -0400 Message-ID: <20230501200305.168058-5-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230501200305.168058-1-stewart.hildebrand@amd.com> References: <20230501200305.168058-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT009:EE_|PH7PR12MB6762:EE_ X-MS-Office365-Filtering-Correlation-Id: 0ca6b476-853d-4ff2-67e4-08db4a7f304c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7g/OblPUtzx8aUQCo+wrnlELHWK8d/3yT/7x8z4fxDDsK6vZilApLcJcT1ONO5Xd36v6KASu6wpfvxrxTHAxnKr8jr99rlNe0xqiOOIexd1PYYUU80stkiJL+eevWuQCTTRCt/2qeVIB0VA+DpSvdxfNiCXkxx64MpjaJ/JwO3tNXNOQmk07zDkS0wdNpv/T2u45o3YnX/2BelJFF/DmVQzCYk43H4ygqUGshEmtRax/1fA5DPXXQ+4MShljL6k8ze+OZDwYb25jJH5n/wEfXZC3Tje2q5/LfKahoH31IXJkg7EnAj907GcBe8gTgpDP7bYSZhmn++zi1ksBozYDaGfyfPHDxpc3MPdzj/7DdbnuHVcfBQ9ZFY+GWbxA6A8a8Vcgjla27SC4iLaMDJflgEsV9jowhF9eC3wcWLBJ9cu+77OojR2MsCE6mtEfF9cqXDuDVDS+dEnueYdLUyjvzyt+VWaMxZIR/omB9qIrvAhO2b/Rvvnlc+yxFSHCprelCo2BCIE033BQkrsECbMPew88JsshxhThwu5yb0FMUmfIO8sH8TgxXQ5jrC8OdXsMAtmazRH+8Ho2M+zY25Gkew2slxPNyVXPQfRwC4a7zVPqeCIp9rY/TWV8yOZfJ+riT0ehFfgbmywyk2YSdQo1VsvRiNAHMuXKSrPx202NDqU+5eRRNUwI3zYKwsIrhMMpX5oRCZKWSF1WL25N8LAnlZq+4WvGtnE80/xeXHi92Zk= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(346002)(136003)(396003)(39860400002)(451199021)(46966006)(36840700001)(40470700004)(86362001)(36756003)(81166007)(82740400003)(356005)(8676002)(5660300002)(8936002)(41300700001)(4326008)(70206006)(316002)(6916009)(44832011)(2906002)(40480700001)(70586007)(82310400005)(336012)(2616005)(47076005)(426003)(40460700003)(36860700001)(6666004)(966005)(54906003)(478600001)(186003)(83380400001)(1076003)(26005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2023 20:03:53.3266 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0ca6b476-853d-4ff2-67e4-08db4a7f304c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT009.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6762 From: Oleksandr Tyshchenko On Arm we need to parse DT PCI-IOMMU specifier and provide it to the driver (for describing the relationship between PCI devices and IOMMUs) before adding a device to it. Also clarify the check of the return value as iommu_add_pci_device can return >0 if a device doesn't need to be protected by the IOMMU and print a warning if iommu_add_pci_device failed. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand --- downstream->v1: * rebase * add __maybe_unused attribute to const struct domain_iommu *hd; * Rename: s/iommu_add_pci_device/iommu_add_dt_pci_device/ * guard iommu_add_dt_pci_device call with CONFIG_HAS_DEVICE_TREE instead of CONFIG_ARM (cherry picked from commit 2b9d26badab8b24b5a80d028c4499a5022817213 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/drivers/passthrough/pci.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index b42acb8d7c09..ed5a6ede7847 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -1305,7 +1305,7 @@ __initcall(setup_dump_pcidevs); static int iommu_add_device(struct pci_dev *pdev) { - const struct domain_iommu *hd; + const struct domain_iommu *hd __maybe_unused; int rc; unsigned int devfn = pdev->devfn; @@ -1318,17 +1318,30 @@ static int iommu_add_device(struct pci_dev *pdev) if ( !is_iommu_enabled(pdev->domain) ) return 0; +#ifdef CONFIG_HAS_DEVICE_TREE + rc = iommu_add_dt_pci_device(devfn, pdev); +#else rc = iommu_call(hd->platform_ops, add_device, devfn, pci_to_dev(pdev)); - if ( rc || !pdev->phantom_stride ) +#endif + if ( rc < 0 || !pdev->phantom_stride ) + { + if ( rc < 0 ) + printk(XENLOG_WARNING "IOMMU: add %pp failed (%d)\n", + &pdev->sbdf, rc); return rc; + } for ( ; ; ) { devfn += pdev->phantom_stride; if ( PCI_SLOT(devfn) != PCI_SLOT(pdev->devfn) ) return 0; +#ifdef CONFIG_HAS_DEVICE_TREE + rc = iommu_add_dt_pci_device(devfn, pdev); +#else rc = iommu_call(hd->platform_ops, add_device, devfn, pci_to_dev(pdev)); - if ( rc ) +#endif + if ( rc < 0 ) printk(XENLOG_WARNING "IOMMU: add %pp failed (%d)\n", &PCI_SBDF(pdev->seg, pdev->bus, devfn), rc); } From patchwork Mon May 1 20:03:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13228256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59A22C77B7C for ; Mon, 1 May 2023 20:04:19 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.528269.821217 (Exim 4.92) (envelope-from ) id 1ptZkl-0007EO-R2; Mon, 01 May 2023 20:04:07 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 528269.821217; Mon, 01 May 2023 20:04:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZkl-0007EB-Ni; Mon, 01 May 2023 20:04:07 +0000 Received: by outflank-mailman (input) for mailman id 528269; Mon, 01 May 2023 20:04:07 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZkl-000525-2K for xen-devel@lists.xenproject.org; Mon, 01 May 2023 20:04:07 +0000 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2060a.outbound.protection.outlook.com [2a01:111:f400:7eaa::60a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 52e8df08-e85b-11ed-b225-6b7b168915f2; Mon, 01 May 2023 22:04:06 +0200 (CEST) Received: from MW4PR04CA0113.namprd04.prod.outlook.com (2603:10b6:303:83::28) by MW4PR12MB8611.namprd12.prod.outlook.com (2603:10b6:303:1ed::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.30; Mon, 1 May 2023 20:04:02 +0000 Received: from CO1NAM11FT094.eop-nam11.prod.protection.outlook.com (2603:10b6:303:83:cafe::eb) by MW4PR04CA0113.outlook.office365.com (2603:10b6:303:83::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.30 via Frontend Transport; Mon, 1 May 2023 20:04:02 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT094.mail.protection.outlook.com (10.13.174.161) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.20 via Frontend Transport; Mon, 1 May 2023 20:04:02 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 1 May 2023 15:04:01 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Mon, 1 May 2023 15:04:00 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 52e8df08-e85b-11ed-b225-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZOVMIRE2X7DPxKGPPVVzf7wJDjSP53mki8qz8xlrnp5JOFFthx4rcshJAdot3F2v8sraPYe6ZTyzYNF+DXw7GyeEth/skLrL3MSTQhLi4P1ByfBifMQdyMEM0orS0DCijZcriPoSooiyp7c8dAYxiAfvF+KobmA22oVi3PY7dhmzjoibJip1ITuFpRWGqd2a0UiOfG7+89fQa86ZUctZX8DX/kvfc1a7dlnnq7mJjssbnnzzPiOUsSeoPKpwr8lqOk1QvX5N3iHuaRBPgeBDx7u6W8lo5G9dH2yR5jRxNEJ/DR3Ya5CQjiPfsBJ+y6CwEIuWu33EjsvQVQarYxRCGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kw1Pv3wUj5s9KFrgoBEyVCO4rA7mTJkBr2g4lQl2H8w=; b=EYGB6wIAiZ9jLGggDl6UEyNcXjvDUmK119KnWwRtAeFoYhlRd3kdoxJGliaklqmMabLqrk3Ks4MQKN+WYzcBpXdf0PToK7Q50kA/4/lV7kc302Z/2nnYJfA0RG+yThRWtlkL3ZPwR2wnUXn/RbrQjdEVjR6MTtO9s0Ym6/8O+ziANJtf+fhQJQOABAKANcWApT6wLsnXotcjVKwvCH/aIW7h3f1wk7fW+N+6az8pAuIbGX6w4qDjck04HGrP6dDeXnjws5Oo1Yg+drdvfhDVnUiOMuBapkvWYHYrmOARp0nyRJblCqK2yPzs1j5L+F6DseDCTFIsoIJLhJhGnQKG5A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kw1Pv3wUj5s9KFrgoBEyVCO4rA7mTJkBr2g4lQl2H8w=; b=I38jmhxkLb05puhi8QlR6wZNDFyOaw+N1ZD7/elqOyzmNG/cWCNJEXk27AM45lHr7boYHJ14PwP2IJUDgio1r6e2YRUqbZLhIqEDmFcU+KcUf6Ly4nA7g/yAAfWpnwcUW1e1cD1tNGnZ1szY7Vvlemau4UPx3gYZmjFvW3SAEXg= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Andrushchenko , Julien Grall , Rahul Singh , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk , Oleksandr Tyshchenko , Stewart Hildebrand Subject: [PATCH v1 5/6] xen/arm: smmuv2: Add PCI devices support for SMMUv2 Date: Mon, 1 May 2023 16:03:04 -0400 Message-ID: <20230501200305.168058-6-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230501200305.168058-1-stewart.hildebrand@amd.com> References: <20230501200305.168058-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT094:EE_|MW4PR12MB8611:EE_ X-MS-Office365-Filtering-Correlation-Id: ed2ed5ad-8a9a-413d-094a-08db4a7f35af X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +ZUzcqKSKb3/OI8rYGlrEmYKUZY8tcNjGtQFCxHTOLyAGpS6qYwFdkOuL5xEmiDaIC2/OBSuQq8Eo+olxWxjMg5Z/qETCyJi9SoCY/ggxZuCty/Eq8/JziUP97iIPffxLASYoGSachstTALfjB3ZLEoAqgFk11dgWIxLjWayvKpn3u3vmhulbijKxOdeWN2jYaDDJ9PT8KDHc0pfNf9kshGQL3jy+fru5srpfrhlUxKEukpkJfneyl9XUcMr8gEsXVgptqDkn712vFmhdTUJecfLc7HAd/HzX/dnQu8W9pGm7IMq6mSph5gCMbxealJ3jit53ZrAGzbTDtGTgS4y+A/c9om+OG34XGg4K/wPP9foXxE9//4mPvVTe2xf1ShDgFQWAjXlXhuURUvXftraPkYvWJlM0/kAyB9M3t2MMyBUxRLCJj6BZzURVmAMDr8gX+w01V5+HwWX8clRlfoljYBQDMnYWYOz37TkRTLci6O+aMDjnoyx7uetEY1hlTcVc+xRi8lauClr2wXpqf5oE87CM1tyUQm+vqACIoGyPTljpu+1GMgyGF9vHtvKc1OHdLa9U9luW3X3jSLqErARM8l0F0iDdS40QupU7YIZPML3n5XYCKJVq7fwYr5Lx+7zyhPEUr53pthe1RqkGY/cyt/odC2ihj2K5JjZySs/pNMv74yshT5czVNSram4dcDnAXXJMJvQVrW+k3oQiZeqeLwH3toGERurUYlXDbQVwqIMttN/jTwPfdASIQvYfHe3acku1MMWSH/VxEvbS0VCs34pyJtTYcO85n6BdgXWv/E= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(396003)(346002)(136003)(39860400002)(376002)(451199021)(36840700001)(46966006)(40470700004)(4326008)(6916009)(82740400003)(186003)(5660300002)(36756003)(316002)(70586007)(70206006)(2906002)(41300700001)(356005)(81166007)(966005)(86362001)(83380400001)(47076005)(478600001)(426003)(336012)(54906003)(2616005)(1076003)(26005)(8936002)(82310400005)(36860700001)(40460700003)(8676002)(40480700001)(44832011)(21314003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2023 20:04:02.3460 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ed2ed5ad-8a9a-413d-094a-08db4a7f35af X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT094.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB8611 From: Oleksandr Andrushchenko Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Stewart Hildebrand --- downstream->v1: * wrap unused function in #ifdef 0 * remove the remove_device() stub since it was submitted separately to the list [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops https://lists.xenproject.org/archives/html/xen-devel/2023-04/msg00432.html * arm_smmu_(de)assign_dev: return error instead of crashing system * update condition in arm_smmu_reassign_dev * style fixup * add && !is_hardware_domain(d) into condition in arm_smmu_assign_dev() (cherry picked from commit 0c11a7f65f044c26d87d1e27ac6283ef1f9cfb7a from the downstream branch spider-master from https://github.com/xen-troops/xen.git) --- This is a file imported from Linux with modifications for Xen. What should be the coding style for Xen modifications? --- xen/drivers/passthrough/arm/smmu.c | 107 +++++++++++++++++++++++------ 1 file changed, 86 insertions(+), 21 deletions(-) diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c index 5b6024d579a8..c33f583f424a 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -134,8 +134,20 @@ typedef enum irqreturn irqreturn_t; /* Device logger functions * TODO: Handle PCI */ -#define dev_print(dev, lvl, fmt, ...) \ - printk(lvl "smmu: %s: " fmt, dt_node_full_name(dev_to_dt(dev)), ## __VA_ARGS__) +#ifndef CONFIG_HAS_PCI +#define dev_print(dev, lvl, fmt, ...) \ + printk(lvl "smmu: %s: " fmt, dev_name(dev), ## __VA_ARGS__) +#else +#define dev_print(dev, lvl, fmt, ...) ({ \ + if ( !dev_is_pci((dev)) ) \ + printk(lvl "smmu: %s: " fmt, dev_name((dev)), ## __VA_ARGS__); \ + else \ + { \ + struct pci_dev *pdev = dev_to_pci((dev)); \ + printk(lvl "smmu: %pp: " fmt, &pdev->sbdf, ## __VA_ARGS__); \ + } \ +}) +#endif #define dev_dbg(dev, fmt, ...) dev_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__) #define dev_notice(dev, fmt, ...) dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__) @@ -187,6 +199,7 @@ static void __iomem *devm_ioremap_resource(struct device *dev, * Xen: PCI functions * TODO: It should be implemented when PCI will be supported */ +#if 0 /* unused */ #define to_pci_dev(dev) (NULL) static inline int pci_for_each_dma_alias(struct pci_dev *pdev, int (*fn) (struct pci_dev *pdev, @@ -196,6 +209,7 @@ static inline int pci_for_each_dma_alias(struct pci_dev *pdev, BUG(); return 0; } +#endif /* Xen: misc */ #define PHYS_MASK_SHIFT PADDR_BITS @@ -632,7 +646,7 @@ struct arm_smmu_master_cfg { for (i = 0; idx = cfg->smendx[i], i < num; ++i) struct arm_smmu_master { - struct device_node *of_node; + struct device *dev; struct rb_node node; struct arm_smmu_master_cfg cfg; }; @@ -724,7 +738,7 @@ arm_smmu_get_fwspec(struct arm_smmu_master_cfg *cfg) { struct arm_smmu_master *master = container_of(cfg, struct arm_smmu_master, cfg); - return dev_iommu_fwspec_get(&master->of_node->dev); + return dev_iommu_fwspec_get(master->dev); } static void parse_driver_options(struct arm_smmu_device *smmu) @@ -757,7 +771,7 @@ static struct device_node *dev_get_dev_node(struct device *dev) } static struct arm_smmu_master *find_smmu_master(struct arm_smmu_device *smmu, - struct device_node *dev_node) + struct device *dev) { struct rb_node *node = smmu->masters.rb_node; @@ -766,9 +780,9 @@ static struct arm_smmu_master *find_smmu_master(struct arm_smmu_device *smmu, master = container_of(node, struct arm_smmu_master, node); - if (dev_node < master->of_node) + if (dev < master->dev) node = node->rb_left; - else if (dev_node > master->of_node) + else if (dev > master->dev) node = node->rb_right; else return master; @@ -803,9 +817,9 @@ static int insert_smmu_master(struct arm_smmu_device *smmu, = container_of(*new, struct arm_smmu_master, node); parent = *new; - if (master->of_node < this->of_node) + if (master->dev < this->dev) new = &((*new)->rb_left); - else if (master->of_node > this->of_node) + else if (master->dev > this->dev) new = &((*new)->rb_right); else return -EEXIST; @@ -824,18 +838,18 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu, struct arm_smmu_master *master; struct device_node *dev_node = dev_get_dev_node(dev); - master = find_smmu_master(smmu, dev_node); + master = find_smmu_master(smmu, dev); if (master) { dev_err(dev, "rejecting multiple registrations for master device %s\n", - dev_node->name); + dev_node ? dev_node->name : ""); return -EBUSY; } master = devm_kzalloc(dev, sizeof(*master), GFP_KERNEL); if (!master) return -ENOMEM; - master->of_node = dev_node; + master->dev = dev; /* Xen: Let Xen know that the device is protected by an SMMU */ device_set_protected(dev); @@ -845,7 +859,7 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu, (fwspec->ids[i] >= smmu->num_mapping_groups)) { dev_err(dev, "stream ID for master device %s greater than maximum allowed (%d)\n", - dev_node->name, smmu->num_mapping_groups); + dev_node ? dev_node->name : "", smmu->num_mapping_groups); return -ERANGE; } master->cfg.smendx[i] = INVALID_SMENDX; @@ -912,11 +926,10 @@ static struct arm_smmu_device *find_smmu_for_device(struct device *dev) { struct arm_smmu_device *smmu; struct arm_smmu_master *master = NULL; - struct device_node *dev_node = dev_get_dev_node(dev); spin_lock(&arm_smmu_devices_lock); list_for_each_entry(smmu, &arm_smmu_devices, list) { - master = find_smmu_master(smmu, dev_node); + master = find_smmu_master(smmu, dev); if (master) break; } @@ -2006,6 +2019,7 @@ static bool arm_smmu_capable(enum iommu_cap cap) } #endif +#if 0 /* Not used */ static int __arm_smmu_get_pci_sid(struct pci_dev *pdev, u16 alias, void *data) { *((u16 *)data) = alias; @@ -2016,6 +2030,7 @@ static void __arm_smmu_release_pci_iommudata(void *data) { kfree(data); } +#endif static int arm_smmu_add_device(struct device *dev) { @@ -2023,12 +2038,13 @@ static int arm_smmu_add_device(struct device *dev) struct arm_smmu_master_cfg *cfg; struct iommu_group *group; void (*releasefn)(void *) = NULL; - int ret; smmu = find_smmu_for_device(dev); if (!smmu) return -ENODEV; + /* There is no need to distinguish here, thanks to PCI-IOMMU DT bindings */ +#if 0 if (dev_is_pci(dev)) { struct pci_dev *pdev = to_pci_dev(dev); struct iommu_fwspec *fwspec; @@ -2053,10 +2069,12 @@ static int arm_smmu_add_device(struct device *dev) &fwspec->ids[0]); releasefn = __arm_smmu_release_pci_iommudata; cfg->smmu = smmu; - } else { + } else +#endif + { struct arm_smmu_master *master; - master = find_smmu_master(smmu, dev->of_node); + master = find_smmu_master(smmu, dev); if (!master) { return -ENODEV; } @@ -2724,6 +2742,31 @@ static int arm_smmu_assign_dev(struct domain *d, u8 devfn, return -ENOMEM; } +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) && !is_hardware_domain(d) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Assigning device %04x:%02x:%02x.%u to dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), PCI_FUNC(devfn), + d->domain_id); + + /* + * XXX What would be the proper behavior? This could happen if + * pdev->phantom_stride > 0 + */ + if ( devfn != pdev->devfn ) + return -EOPNOTSUPP; + + list_move(&pdev->domain_list, &d->pdev_list); + pdev->domain = d; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + if (!dev_iommu_group(dev)) { ret = arm_smmu_add_device(dev); if (ret) @@ -2773,11 +2816,33 @@ out: return ret; } -static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) +static int arm_smmu_deassign_dev(struct domain *d, u8 devfn, struct device *dev) { struct iommu_domain *domain = dev_iommu_domain(dev); struct arm_smmu_xen_domain *xen_domain; +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Deassigning device %04x:%02x:%02x.%u from dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), PCI_FUNC(devfn), + d->domain_id); + + /* + * XXX What would be the proper behavior? This could happen if + * pdev->phantom_stride > 0 + */ + if ( devfn != pdev->devfn ) + return -EOPNOTSUPP; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + xen_domain = dom_iommu(d)->arch.priv; if (!domain || domain->priv->cfg.domain != d) { @@ -2805,13 +2870,13 @@ static int arm_smmu_reassign_dev(struct domain *s, struct domain *t, int ret = 0; /* Don't allow remapping on other domain than hwdom */ - if ( t && !is_hardware_domain(t) ) + if ( t && !is_hardware_domain(t) && t != dom_io ) return -EPERM; if (t == s) return 0; - ret = arm_smmu_deassign_dev(s, dev); + ret = arm_smmu_deassign_dev(s, devfn, dev); if (ret) return ret; From patchwork Mon May 1 20:09:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13228265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5AEA9C77B73 for ; Mon, 1 May 2023 20:10:10 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.528280.821237 (Exim 4.92) (envelope-from ) id 1ptZqM-0000PC-O1; Mon, 01 May 2023 20:09:54 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 528280.821237; Mon, 01 May 2023 20:09:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZqM-0000P5-LS; Mon, 01 May 2023 20:09:54 +0000 Received: by outflank-mailman (input) for mailman id 528280; Mon, 01 May 2023 20:09:53 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ptZqL-0000Of-TV for xen-devel@lists.xenproject.org; Mon, 01 May 2023 20:09:53 +0000 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2061e.outbound.protection.outlook.com [2a01:111:f400:7e8a::61e]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 21b20385-e85c-11ed-b225-6b7b168915f2; Mon, 01 May 2023 22:09:52 +0200 (CEST) Received: from MW4PR04CA0323.namprd04.prod.outlook.com (2603:10b6:303:82::28) by DM4PR12MB6038.namprd12.prod.outlook.com (2603:10b6:8:ab::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.30; Mon, 1 May 2023 20:09:50 +0000 Received: from CO1NAM11FT067.eop-nam11.prod.protection.outlook.com (2603:10b6:303:82:cafe::44) by MW4PR04CA0323.outlook.office365.com (2603:10b6:303:82::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.30 via Frontend Transport; Mon, 1 May 2023 20:09:49 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT067.mail.protection.outlook.com (10.13.174.212) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.19 via Frontend Transport; Mon, 1 May 2023 20:09:49 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 1 May 2023 15:09:49 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 1 May 2023 15:09:48 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Mon, 1 May 2023 15:09:47 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 21b20385-e85c-11ed-b225-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RD3OwduzSOZw9pVYmJqz5vhD8oFwIil30HgUTqOGpE/EV9XjgSsHVo2Eyi0yJtfyyGzEiJod49SLv+8sKRZETj8Ahf9MPf3KijWKjK0lvgnDugGQSHbSUf3ikJH62BXbEQrkKQ/jkxmGYA+YzgihUNFbf4ha1TlaKnBMiHaRWbonuhDfh/5nhv8Szk977fsDjrbWDaGEz7DnoG8xuA/2GVKOtc2aE/In72CT97uLMW4l6j8mkf7ptC/hcgEGkCjAkACMhhiyEEAVm1ZkE4qX0fHrMZfvSsFrlZZawjZSST7N/hYybeZ/XnTQNUizUSYg4nOOr65auJZu5CRfqLDmEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0XJhPVciLUzEu7aYbf6zQpM/bkxm9q36IA+36Y75kWQ=; b=loBGtN6HImwO0TelplfgEoHq25dQ+HuRjGjWDUhFTIL2EvvCkcL5SH15jJRJQOejm0nx5eb81snfNK7MxlAV9Rxu+JVpEqfOYNeqgPc8vAZ9Ue+BCaUgcVTgHuF7+BqZSSIVOVXQkXm84q9CteSl3qG3JH24VJhfSF++WifF6mKatVKG+rwQ4UgpEqE56+7dB6svbEBGAjxKTXEcQIWEdbJd/XvE7djf0od+kai7Zecsb0enLMYsGsemkdfzm7IsliLS4IQ28k2m+08E5MkAGjTBJHbZMSu/R/xRBSw+1duO7GU7jb+ozjHSEas8Hy9LY0lszj1VzL8fVNy5YWeqfQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0XJhPVciLUzEu7aYbf6zQpM/bkxm9q36IA+36Y75kWQ=; b=0slI8cl+LqjDqadGUBE5m95+adcNqO6T9TVhZfCNoplQ6uN+DrJiKm0wyGb4PrLopego0mOmxDTFyAvLfEw399H9uXQMdTIAQ9Q4IUjRj1NKUmgoUhPiURvgI80BCAoKw6ELLQY5YhHYmZQGd0IvMMwYwHTR/F/Cu9OIp2+vrnI= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Rahul Singh , Bertrand Marquis , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Stewart Hildebrand Subject: [PATCH v1 6/6] xen/arm: smmuv3: Add PCI devices support for SMMUv3 Date: Mon, 1 May 2023 16:09:42 -0400 Message-ID: <20230501200942.168105-1-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230501200305.168058-1-stewart.hildebrand@amd.com> References: <20230501200305.168058-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT067:EE_|DM4PR12MB6038:EE_ X-MS-Office365-Filtering-Correlation-Id: 3a3272d9-79e4-4a75-ca3b-08db4a800482 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AYbOBjwjju28o+61rCE+w9k0H0/ZRJqJopCvwEl4QGMqHtNwsWGsDzPWCZcmr3cRCFHzCxOy5RGRthnSLLEzjlSPhg2IXy1qAhPBYDES5PQemCtNDwHYC06FwXclLRIBABAPoSLijsC9TbjZYafmwNbDc7O87Qc+2x7niTjjJbGcr6lsJk3CzIV+YazJUboqO3lvJvN10LKbqDYTJxUu9ok8ZaCO6z6j6XmkZe5qsqpmOBV3L3oK3LFRzNTj3FWg2wVv4IjYwGISy4GevbyLTbN8yM2fBw6O/Wjnjk83CZjvP5lD/z27Mn436uK+F3fE88BloLZ3473dn8gUzzXrLZ9v2KnNnoNr6tqJKRJXuqISudAUudZZMoz6jib1vxs6aGE3ZkgrGEgpgULlXBjasoH15vslF+RCLER/1zTQPL2xEKQAJ7L0PJx6oN5Y49PA/l8rEVRI7OylvSMpg8tp2x/Ka85tiRoNAphprAFJ+BAR1owYULnIiIuPEOfgGK7Qw7d+Gz1cl8pQ/YOymeezkyH+rWwcT7HTb8EypernyyScKj93xMvJ7V5/LB+gJpOCITAYSi5bmaGfIqVmLeMIviURG7xUUboaKaBWdC3v++7UDAxfVCQ5Gh3h90MIny6wcNEYEew9dqdx/awKIOaz5miCdUcEVOAK8sFds1ph4rdSJ0/RkBFko4uAVjfYrgxsuz56Ev8q8A/peJ3zZfvJqrYiX4aMAdZaq89Gmihv9RrHIJ8L3RJnVN7NlMVvXfcs X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(39860400002)(396003)(136003)(346002)(451199021)(40470700004)(36840700001)(46966006)(5660300002)(82310400005)(44832011)(86362001)(336012)(2616005)(83380400001)(1076003)(47076005)(36860700001)(82740400003)(186003)(356005)(966005)(81166007)(426003)(26005)(40460700003)(6666004)(8936002)(478600001)(316002)(54906003)(8676002)(36756003)(40480700001)(6916009)(41300700001)(4326008)(70206006)(70586007)(2906002)(21314003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2023 20:09:49.3600 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3a3272d9-79e4-4a75-ca3b-08db4a800482 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT067.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6038 From: Rahul Singh Signed-off-by: Rahul Singh Signed-off-by: Stewart Hildebrand --- downstream->v1: * rebase * move 2 replacements of s/dt_device_set_protected(dev_to_dt(dev))/device_set_protected(dev)/ from this commit to ("xen/arm: Move is_protected flag to struct device") so as to not break ability to bisect * adjust patch title (remove stray space) * arm_smmu_(de)assign_dev: return error instead of crashing system * remove arm_smmu_remove_device() stub * update condition in arm_smmu_reassign_dev * style fixup (cherry picked from commit 7ed6c3ab250d899fe6e893a514278e406a2893e8 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- This is a file imported from Linux with modifications for Xen. What should be the coding style used for Xen modifications? --- xen/drivers/passthrough/arm/smmu-v3.c | 66 +++++++++++++++++++++++++-- 1 file changed, 63 insertions(+), 3 deletions(-) diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c index 4b452e6fdd00..481a35a8b8d4 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -1469,6 +1469,8 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid) } /* Forward declaration */ static struct arm_smmu_device *arm_smmu_get_by_dev(struct device *dev); +static int arm_smmu_assign_dev(struct domain *d, u8 devfn, + struct device *dev, u32 flag); static int arm_smmu_add_device(u8 devfn, struct device *dev) { @@ -1527,6 +1529,17 @@ static int arm_smmu_add_device(u8 devfn, struct device *dev) dev_info(dev, "Added master device (SMMUv3 %s StreamIds %u)\n", dev_name(fwspec->iommu_dev), fwspec->num_ids); +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + ret = arm_smmu_assign_dev(pdev->domain, devfn, dev, 0); + if (ret) + goto err_free_master; + } +#endif + return 0; err_free_master: @@ -2607,6 +2620,31 @@ static int arm_smmu_assign_dev(struct domain *d, u8 devfn, struct arm_smmu_domain *smmu_domain; struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv; +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) && !is_hardware_domain(d) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Assigning device %04x:%02x:%02x.%u to dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), + PCI_FUNC(devfn), d->domain_id); + + /* + * XXX What would be the proper behavior? This could happen if + * pdev->phantom_stride > 0 + */ + if ( devfn != pdev->devfn ) + return -EOPNOTSUPP; + + list_move(&pdev->domain_list, &d->pdev_list); + pdev->domain = d; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + spin_lock(&xen_domain->lock); /* @@ -2640,7 +2678,7 @@ out: return ret; } -static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) +static int arm_smmu_deassign_dev(struct domain *d, uint8_t devfn, struct device *dev) { struct iommu_domain *io_domain = arm_smmu_get_domain(d, dev); struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv; @@ -2652,6 +2690,28 @@ static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) return -ESRCH; } +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Deassigning device %04x:%02x:%02x.%u from dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), + PCI_FUNC(devfn), d->domain_id); + + /* + * XXX What would be the proper behavior? This could happen if + * pdev->phantom_stride > 0 + */ + if ( devfn != pdev->devfn ) + return -EOPNOTSUPP; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + spin_lock(&xen_domain->lock); arm_smmu_detach_dev(master); @@ -2671,13 +2731,13 @@ static int arm_smmu_reassign_dev(struct domain *s, struct domain *t, int ret = 0; /* Don't allow remapping on other domain than hwdom */ - if ( t && !is_hardware_domain(t) ) + if ( t && !is_hardware_domain(t) && (t != dom_io) ) return -EPERM; if (t == s) return 0; - ret = arm_smmu_deassign_dev(s, dev); + ret = arm_smmu_deassign_dev(s, devfn, dev); if (ret) return ret;