From patchwork Thu May 11 19:16:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13238413 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 02C68C7EE24 for ; Thu, 11 May 2023 19:17:59 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.533542.830311 (Exim 4.92) (envelope-from ) id 1pxBnQ-0004W8-Mz; Thu, 11 May 2023 19:17:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 533542.830311; Thu, 11 May 2023 19:17:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBnQ-0004Vz-K9; Thu, 11 May 2023 19:17:48 +0000 Received: by outflank-mailman (input) for mailman id 533542; Thu, 11 May 2023 19:17:47 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBnP-00042a-7d for xen-devel@lists.xenproject.org; Thu, 11 May 2023 19:17:47 +0000 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on20608.outbound.protection.outlook.com [2a01:111:f400:7e8c::608]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 80e443ec-f030-11ed-8611-37d641c3527e; Thu, 11 May 2023 21:17:45 +0200 (CEST) Received: from DM6PR17CA0026.namprd17.prod.outlook.com (2603:10b6:5:1b3::39) by IA1PR12MB6652.namprd12.prod.outlook.com (2603:10b6:208:38a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.20; Thu, 11 May 2023 19:17:40 +0000 Received: from DM6NAM11FT103.eop-nam11.prod.protection.outlook.com (2603:10b6:5:1b3:cafe::76) by DM6PR17CA0026.outlook.office365.com (2603:10b6:5:1b3::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:17:40 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by DM6NAM11FT103.mail.protection.outlook.com (10.13.172.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:17:39 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 14:17:39 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Thu, 11 May 2023 14:17:37 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 80e443ec-f030-11ed-8611-37d641c3527e ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZNrkR38SIlI+XXPwGihe6pUHsD1bNE9EaSEqIxbT/y+OtHrGB7jF/D8Mevww3YOcap6YKHhsGHQA2p2qrPFER5RFWwYqXRJNbhD27UoArf7cYPnH/ZtIezTPFGQxpkjw4WKidvxuvd2RSL9Bi9ZVbzE8sCdqPxi/HvcAp4r9BXIEXC64Ij74xEX01FKbyNESzb9dbcAwrQqd45H0W5OW769p5t4xFWJv7f3Mauo4TrDOvhBPcb1xiu+MX0OAiXAC+09SQL8Q1GMOI3T55585c8Gl0esbGSie+TwM/iX0TeCijxAtEq70Z1Rs0b4juTDUQ48aDzfi5lLnzxJU0ZvENA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uxhzk6CbTYHjfXnu2i9bDdgPDhMQRJVaJmh+gENOEuM=; b=a77Rca2x9UA+3CSDdu49JuSFKTi1vdeu101eYX9bIJsMI+QizQ6d6TfY8xUhGljlZQzTrfbPY5ohxsZnBsPriDVK6WsV1NMp4dyJ8zJ/jjJMyA92K4KKtMAd/ZlHWsfwueCpT+wz1ERv/K5PQ+/CTEENysvFtUkdUXwhh7X9sP2iM47hCrO+eatcW3LUKE/nH/u8zoNEIbUhADdtDfUcR+2vfsZE0p/vv1kAVdy+5WaPuXH93o2hgI/Tu6VutRFZqGGNlTmvDIwjidPyJA5bAAivmn5jnmFF2YuG6rGhLs2+KdPTwhEytHzGIvaMpomUPyZhcoSpkYEHeNo656byUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uxhzk6CbTYHjfXnu2i9bDdgPDhMQRJVaJmh+gENOEuM=; b=nut1PzLZRW6qo16muzo+WvE7+Hy3zSU0rEByWvc2zzsTWracuU2+ar9FOYgKH0lu0BNul8GR7fHJdtk5mymstmFb6zsPwRUzu+4hNdWFfRFM1DQaYPTKpnCfnpsOVpLdnGtniLC3QUOahc4v8HeDJGdbT8izeraapExdQHf/CyY= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Rahul Singh , Stewart Hildebrand Subject: [PATCH v2 1/8] xen/arm: Move is_protected flag to struct device Date: Thu, 11 May 2023 15:16:47 -0400 Message-ID: <20230511191654.400720-2-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230511191654.400720-1-stewart.hildebrand@amd.com> References: <20230511191654.400720-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT103:EE_|IA1PR12MB6652:EE_ X-MS-Office365-Filtering-Correlation-Id: 9a6e51ab-3b66-43a3-f43f-08db52546342 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: R+XxmJ41CVTP5/g0zZwmIoj1hVO9HgQCnfTLdA+5jzWAHOE0IQiCs17W14HVNLv40h6vJMWI/1+PmVpVHzemEWY5Ln0AUkOc9I/F2kqrQVguBGUYVQCklovLX5C1DAJkUNudnGCYNWPltPFhNa4b5oWFhKtO02llLVq9YLeDER0jMl3RoRSlok8geLdh3y2Dm+awDOxKIImmz8xpKOXarfgPGoiO8AI2RCLm7ZRKhKwqdsJ6EcgFebvvbkFPU379+ft4c1vTxwFjzht2hyczQHvA/PZSjDCnhhm9x9nP5uQ3rVox0d7a6ir8KWR+S0O/CxltWlGmQ3zwmm3KDOtnoRdzvoQ7d0dRNUF2UnG6zqqDIfAxN72J9cdr4iX9Pu6Zh16JCTzXj8xAiJ//1lch85ivcNEaSx0qtIoRT1hwo/AUsNQ3tIQa1ePF5ko+GWXB2pQeK994G3oVVuKSGYmvBITOsY65xdg/2bV6fDsRAkcpK0S+wB932Ej5klzl1V38IgbGVPvsH/tMaXKorz7zW627A7FqysqF7cOB1m8pu6ENJ9m9rvPGmArtRqbZZ+vua6VvII8mkTf5nj++5AHkeF7yf+b3civpDk8IC3rbtH/2sHBKAE0Gx/1o9p5OcY42aAuI6e/xIwI8bcSSlqQbuD7tMMDOY3A8WcI7luX2JErq/Pm8hPROuZ47nE3OvaO2UClBhmlaarxFFhtkrgCSj+y1CS/8Y2mrTKjd4857qBjvcQXxz4AniOU8JXEPWLQC X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(396003)(136003)(376002)(39860400002)(451199021)(46966006)(36840700001)(40470700004)(6666004)(8936002)(40460700003)(966005)(54906003)(70586007)(36756003)(70206006)(82740400003)(2906002)(4326008)(44832011)(6916009)(356005)(41300700001)(8676002)(5660300002)(316002)(81166007)(82310400005)(86362001)(40480700001)(478600001)(426003)(36860700001)(186003)(336012)(2616005)(47076005)(1076003)(26005)(83380400001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2023 19:17:39.8338 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9a6e51ab-3b66-43a3-f43f-08db52546342 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT103.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6652 From: Oleksandr Tyshchenko This flag will be re-used for PCI devices by the subsequent patches. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand --- v1->v2: * no change downstream->v1: * rebase * s/dev_node->is_protected/dev_node->dev.is_protected/ in smmu.c * s/dt_device_set_protected(dev_to_dt(dev))/device_set_protected(dev)/ in smmu-v3.c * remove redundant device_is_protected checks in smmu-v3.c/ipmmu-vmsa.c (cherry picked from commit 59753aac77528a584d3950936b853ebf264b68e7 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/arch/arm/domain_build.c | 4 ++-- xen/arch/arm/include/asm/device.h | 13 +++++++++++++ xen/common/device_tree.c | 2 +- xen/drivers/passthrough/arm/ipmmu-vmsa.c | 8 +------- xen/drivers/passthrough/arm/smmu-v3.c | 7 +------ xen/drivers/passthrough/arm/smmu.c | 2 +- xen/drivers/passthrough/device_tree.c | 15 +++++++++------ xen/include/xen/device_tree.h | 13 ------------- 8 files changed, 28 insertions(+), 36 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index f80fdd1af206..106f92c65a61 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -2507,7 +2507,7 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev, return res; } - if ( dt_device_is_protected(dev) ) + if ( device_is_protected(dt_to_dev(dev)) ) { dt_dprintk("%s setup iommu\n", dt_node_full_name(dev)); res = iommu_assign_dt_device(d, dev); @@ -3007,7 +3007,7 @@ static int __init handle_passthrough_prop(struct kernel_info *kinfo, return res; /* If xen_force, we allow assignment of devices without IOMMU protection. */ - if ( xen_force && !dt_device_is_protected(node) ) + if ( xen_force && !device_is_protected(dt_to_dev(node)) ) return 0; return iommu_assign_dt_device(kinfo->d, node); diff --git a/xen/arch/arm/include/asm/device.h b/xen/arch/arm/include/asm/device.h index b5d451e08776..086dde13eb6b 100644 --- a/xen/arch/arm/include/asm/device.h +++ b/xen/arch/arm/include/asm/device.h @@ -1,6 +1,8 @@ #ifndef __ASM_ARM_DEVICE_H #define __ASM_ARM_DEVICE_H +#include + enum device_type { DEV_DT, @@ -20,6 +22,7 @@ struct device #endif struct dev_archdata archdata; struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */ + bool is_protected; /* Shows that device is protected by IOMMU */ }; typedef struct device device_t; @@ -94,6 +97,16 @@ int device_init(struct dt_device_node *dev, enum device_class class, */ enum device_class device_get_class(const struct dt_device_node *dev); +static inline void device_set_protected(struct device *device) +{ + device->is_protected = true; +} + +static inline bool device_is_protected(const struct device *device) +{ + return device->is_protected; +} + #define DT_DEVICE_START(_name, _namestr, _class) \ static const struct device_desc __dev_desc_##_name __used \ __section(".dev.info") = { \ diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c index 6c9712ab7bda..1d5d7cb5f01b 100644 --- a/xen/common/device_tree.c +++ b/xen/common/device_tree.c @@ -1874,7 +1874,7 @@ static unsigned long __init unflatten_dt_node(const void *fdt, /* By default dom0 owns the device */ np->used_by = 0; /* By default the device is not protected */ - np->is_protected = false; + np->dev.is_protected = false; INIT_LIST_HEAD(&np->domain_list); if ( new_format ) diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthrough/arm/ipmmu-vmsa.c index 091f09b21752..039212a3a990 100644 --- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -1288,14 +1288,8 @@ static int ipmmu_add_device(u8 devfn, struct device *dev) if ( !to_ipmmu(dev) ) return -ENODEV; - if ( dt_device_is_protected(dev_to_dt(dev)) ) - { - dev_err(dev, "Already added to IPMMU\n"); - return -EEXIST; - } - /* Let Xen know that the master device is protected by an IOMMU. */ - dt_device_set_protected(dev_to_dt(dev)); + device_set_protected(dev); dev_info(dev, "Added master device (IPMMU %s micro-TLBs %u)\n", dev_name(fwspec->iommu_dev), fwspec->num_ids); diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c index bfdb62b395ad..4b452e6fdd00 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -1521,13 +1521,8 @@ static int arm_smmu_add_device(u8 devfn, struct device *dev) */ arm_smmu_enable_pasid(master); - if (dt_device_is_protected(dev_to_dt(dev))) { - dev_err(dev, "Already added to SMMUv3\n"); - return -EEXIST; - } - /* Let Xen know that the master device is protected by an IOMMU. */ - dt_device_set_protected(dev_to_dt(dev)); + device_set_protected(dev); dev_info(dev, "Added master device (SMMUv3 %s StreamIds %u)\n", dev_name(fwspec->iommu_dev), fwspec->num_ids); diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c index 0a514821b336..5b6024d579a8 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -838,7 +838,7 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu, master->of_node = dev_node; /* Xen: Let Xen know that the device is protected by an SMMU */ - dt_device_set_protected(dev_node); + device_set_protected(dev); for (i = 0; i < fwspec->num_ids; ++i) { if (!(smmu->features & ARM_SMMU_FEAT_STREAM_MATCH) && diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index 1c32d7b50cce..b5bd13393b56 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -34,7 +34,7 @@ int iommu_assign_dt_device(struct domain *d, struct dt_device_node *dev) if ( !is_iommu_enabled(d) ) return -EINVAL; - if ( !dt_device_is_protected(dev) ) + if ( !device_is_protected(dt_to_dev(dev)) ) return -EINVAL; spin_lock(&dtdevs_lock); @@ -65,7 +65,7 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev) if ( !is_iommu_enabled(d) ) return -EINVAL; - if ( !dt_device_is_protected(dev) ) + if ( !device_is_protected(dt_to_dev(dev)) ) return -EINVAL; spin_lock(&dtdevs_lock); @@ -87,7 +87,7 @@ static bool_t iommu_dt_device_is_assigned(const struct dt_device_node *dev) { bool_t assigned = 0; - if ( !dt_device_is_protected(dev) ) + if ( !device_is_protected(dt_to_dev(dev)) ) return 0; spin_lock(&dtdevs_lock); @@ -141,12 +141,15 @@ int iommu_add_dt_device(struct dt_device_node *np) return -EINVAL; /* - * The device may already have been registered. As there is no harm in - * it just return success early. + * This is needed in case a device has both the iommus property and + * also appears in the mmu-masters list. */ - if ( dev_iommu_fwspec_get(dev) ) + if ( device_is_protected(dev) ) return 0; + if ( dev_iommu_fwspec_get(dev) ) + return -EEXIST; + /* * According to the Documentation/devicetree/bindings/iommu/iommu.txt * from Linux. diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h index 19a74909cece..c1e4751a581f 100644 --- a/xen/include/xen/device_tree.h +++ b/xen/include/xen/device_tree.h @@ -90,9 +90,6 @@ struct dt_device_node { struct dt_device_node *next; /* TODO: Remove it. Only use to know the last children */ struct dt_device_node *allnext; - /* IOMMU specific fields */ - bool is_protected; - /* HACK: Remove this if there is a need of space */ bool_t static_evtchn_created; @@ -302,16 +299,6 @@ static inline domid_t dt_device_used_by(const struct dt_device_node *device) return device->used_by; } -static inline void dt_device_set_protected(struct dt_device_node *device) -{ - device->is_protected = true; -} - -static inline bool dt_device_is_protected(const struct dt_device_node *device) -{ - return device->is_protected; -} - static inline bool_t dt_property_name_is_equal(const struct dt_property *pp, const char *name) { From patchwork Thu May 11 19:16:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13238414 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CCCFBC77B7C for ; Thu, 11 May 2023 19:18:18 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.533547.830321 (Exim 4.92) (envelope-from ) id 1pxBnm-00053o-0Y; Thu, 11 May 2023 19:18:10 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 533547.830321; Thu, 11 May 2023 19:18:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBnl-00053e-Tv; Thu, 11 May 2023 19:18:09 +0000 Received: by outflank-mailman (input) for mailman id 533547; Thu, 11 May 2023 19:18:08 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBnk-00042a-8s for xen-devel@lists.xenproject.org; Thu, 11 May 2023 19:18:08 +0000 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2061b.outbound.protection.outlook.com [2a01:111:f400:fe59::61b]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8e1778b3-f030-11ed-8611-37d641c3527e; Thu, 11 May 2023 21:18:06 +0200 (CEST) Received: from DS7PR03CA0107.namprd03.prod.outlook.com (2603:10b6:5:3b7::22) by PH7PR12MB7115.namprd12.prod.outlook.com (2603:10b6:510:1ee::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.22; Thu, 11 May 2023 19:18:03 +0000 Received: from DM6NAM11FT059.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3b7:cafe::72) by DS7PR03CA0107.outlook.office365.com (2603:10b6:5:3b7::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.20 via Frontend Transport; Thu, 11 May 2023 19:18:02 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by DM6NAM11FT059.mail.protection.outlook.com (10.13.172.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:18:02 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 14:18:01 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Thu, 11 May 2023 14:18:00 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8e1778b3-f030-11ed-8611-37d641c3527e ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=elqkgSRE/3k6CmmWliTBXBsU6XVz/7uZYbFq3LjTYIIad101OpT8y4Fjzss7/+ie1UW//fXrYHlj5gjN5O6AXDf2T2GuXYuQCs46rvMSm3NStsW4N3eUe9PAfd9bbKpH9I8Dp/UcoKFttFKArIg1gZiMDc/zwxZ5JicofWlDsIwhDWHVpt1MnadgFu7LYAZyCVJmQOtdc65Bu5T0ryYBV7XeE+LoG03T7Q9Liv/r+i4yJD0xaDfyva7gLeRiqwpXgc8K0MP18Vrx8HRNwrQKIYD3DzBIn1PRwQX4lzOJPj7BnOPOZBQzQG7L2qBBR8k7IDyzhB2N8NclqkGMjUkIxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=G7y7/iwGlnJBfhykkW+cVx9ujgWVYtHTxAqrWt9z87M=; b=c4f5M1B2230ElonxrsdaRz8CKbXBqAcXsKfoet9KEjlb+Z+TcUuU92j6WbPaOPIX33EhiG6HonPvI6fkMSRINLXLYXJoH5hkSJ13hN06BBxdcAr2NLXS4CnXbOrRfDKrXC5nUdNQx70XYzsDWVYQIkpjR2xH7RTrTMORhTqRcMB174vDafePd7v71ujszBWFmxsKsi8NXVZE3yybBzxu0CrMyupSIcbD61UHwlN2WE71ViWmNMFduH4jC/WDqSYIdPzTEZfCSxYLuQgdjcbMz4ZpNL5xPhEmyjN0GU6zupEBbcMtcQCeCELCBe7Zm6MVzPW+zztGBsQBy+YCf3BC4g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=G7y7/iwGlnJBfhykkW+cVx9ujgWVYtHTxAqrWt9z87M=; b=apdorjHW8giUT/PLJNxI6xPQZyuLMipk3UNfuu2epSJBZQazA6eiLq5ziLAOFXWpjULNYvl+Fp+CxAnUnboJR4ign4Jh2XrtZiSZ4uOfNrR6EaCoGPsgi0u4vIUfd0sBVJn2jAu3GS33LHiOWcnaXLxSYFpuPVMugt1vE4Yw54w= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Stewart Hildebrand Subject: [PATCH v2 2/8] iommu/arm: Add iommu_dt_xlate() Date: Thu, 11 May 2023 15:16:48 -0400 Message-ID: <20230511191654.400720-3-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230511191654.400720-1-stewart.hildebrand@amd.com> References: <20230511191654.400720-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT059:EE_|PH7PR12MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 2d98ad87-1de2-499b-be23-08db525470b8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lYQCq3Fzr77Vspqg8KBdd+4/068rVH7qPe5cdSfX2VzLSRExEf1u2W2CjhkLtO8FZLkBzboerUPLaloglKIDBLKl7m2RqeIK9FJvcaoko6zytAH/IMgjCe/s+eEq6XemitUOAW1cap0NZQo+sI/1UDLtPrQzrk3g+Aa7APck1SFnTxKqSBEPgSrLO+Ucbf/vELnQpXx+VlOiXFR1zv6cP4B2Y3tW5FStaiqa+ftMd4uJ8K++j7HI2y9Oua4nR9yNrx9dBS0PNB7QvmKf9kvNgqgCxtWW3rsungXJs2Dm7ea4yD8oDbBiJ0AviIQuwiaKiezeS6jtwydkLnmwbrswotY5oFUNabSKOGTqc+HtzifH/Pci04BMqq+Cdw6rOdZBOI40jhYqqTDe+Ls2hQMcE+C+apfx7nuKqzIAYaACiiq77LJsZdltamcI3Cab1KJBlEcBDq71YdcaCom5e7nJweZzNoVMDIbybnXP6Cyxts+HRmRVFbc409FFZ0esXEZ1P4MIHQIcyzYYbR/EPcGAXS+pBaFxBAdpBu3dkLRh14m89EVNOUpGsyrwPxCWrreji/pbDrw3G2J+z9vWk1b1GalBQF0OIPOUa12mbSfEJXptBeWi9dG2ebBwbCOid1em//oNqRqrskV+Jhz55KNMyd0HO1DDqmAiJCLxKWownEUyoz3LnkNQI/top1QseWuLL/2RFC1XkSHhnPChi8DP0Q== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(396003)(136003)(39860400002)(376002)(451199021)(36840700001)(46966006)(40470700004)(356005)(316002)(47076005)(82740400003)(478600001)(81166007)(8676002)(2906002)(426003)(41300700001)(336012)(6666004)(82310400005)(86362001)(5660300002)(70586007)(83380400001)(36860700001)(70206006)(8936002)(4326008)(966005)(6916009)(44832011)(54906003)(26005)(36756003)(1076003)(2616005)(40460700003)(40480700001)(186003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2023 19:18:02.4005 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2d98ad87-1de2-499b-be23-08db525470b8 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT059.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7115 From: Oleksandr Tyshchenko Move code for processing DT IOMMU specifier to a separate helper. This helper will be re-used for adding PCI devices by the subsequent patches as we will need exact the same actions for processing DT PCI-IOMMU specifier. While at it introduce NO_IOMMU to avoid magic "1". Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand # rename --- v1->v2: * no change downstream->v1: * trivial rebase * s/dt_iommu_xlate/iommu_dt_xlate/ (cherry picked from commit c26bab0415ca303df86aba1d06ef8edc713734d3 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/drivers/passthrough/device_tree.c | 42 +++++++++++++++++---------- 1 file changed, 27 insertions(+), 15 deletions(-) diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index b5bd13393b56..1b50f4670944 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -127,15 +127,39 @@ int iommu_release_dt_devices(struct domain *d) return 0; } +/* This correlation must not be altered */ +#define NO_IOMMU 1 + +static int iommu_dt_xlate(struct device *dev, + struct dt_phandle_args *iommu_spec) +{ + const struct iommu_ops *ops = iommu_get_ops(); + int rc; + + if ( !dt_device_is_available(iommu_spec->np) ) + return NO_IOMMU; + + rc = iommu_fwspec_init(dev, &iommu_spec->np->dev); + if ( rc ) + return rc; + + /* + * Provide DT IOMMU specifier which describes the IOMMU master + * interfaces of that device (device IDs, etc) to the driver. + * The driver is responsible to decide how to interpret them. + */ + return ops->dt_xlate(dev, iommu_spec); +} + int iommu_add_dt_device(struct dt_device_node *np) { const struct iommu_ops *ops = iommu_get_ops(); struct dt_phandle_args iommu_spec; struct device *dev = dt_to_dev(np); - int rc = 1, index = 0; + int rc = NO_IOMMU, index = 0; if ( !iommu_enabled ) - return 1; + return NO_IOMMU; if ( !ops ) return -EINVAL; @@ -164,19 +188,7 @@ int iommu_add_dt_device(struct dt_device_node *np) if ( !ops->add_device || !ops->dt_xlate ) return -EINVAL; - if ( !dt_device_is_available(iommu_spec.np) ) - break; - - rc = iommu_fwspec_init(dev, &iommu_spec.np->dev); - if ( rc ) - break; - - /* - * Provide DT IOMMU specifier which describes the IOMMU master - * interfaces of that device (device IDs, etc) to the driver. - * The driver is responsible to decide how to interpret them. - */ - rc = ops->dt_xlate(dev, &iommu_spec); + rc = iommu_dt_xlate(dev, &iommu_spec); if ( rc ) break; From patchwork Thu May 11 19:16:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13238415 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0B5B5C77B7C for ; Thu, 11 May 2023 19:18:47 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.533553.830330 (Exim 4.92) (envelope-from ) id 1pxBoD-0005i5-9k; Thu, 11 May 2023 19:18:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 533553.830330; Thu, 11 May 2023 19:18:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBoD-0005hy-6w; Thu, 11 May 2023 19:18:37 +0000 Received: by outflank-mailman (input) for mailman id 533553; Thu, 11 May 2023 19:18:36 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBoB-0005hc-Rh for xen-devel@lists.xenproject.org; Thu, 11 May 2023 19:18:35 +0000 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on20620.outbound.protection.outlook.com [2a01:111:f400:7e89::620]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9e71650a-f030-11ed-b229-6b7b168915f2; Thu, 11 May 2023 21:18:34 +0200 (CEST) Received: from DM6PR02CA0098.namprd02.prod.outlook.com (2603:10b6:5:1f4::39) by PH8PR12MB6745.namprd12.prod.outlook.com (2603:10b6:510:1c0::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.18; Thu, 11 May 2023 19:18:30 +0000 Received: from DM6NAM11FT055.eop-nam11.prod.protection.outlook.com (2603:10b6:5:1f4:cafe::38) by DM6PR02CA0098.outlook.office365.com (2603:10b6:5:1f4::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.21 via Frontend Transport; Thu, 11 May 2023 19:18:30 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by DM6NAM11FT055.mail.protection.outlook.com (10.13.173.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:18:29 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 14:18:29 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 14:18:28 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Thu, 11 May 2023 14:18:27 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9e71650a-f030-11ed-b229-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kZylhLI3sSmFoFIcwsHD+R4LLsKF9jsiZkbSg36DgtQU2VMipJTF37SWZo+nUMDjs/TsjN8W61dngspvRE8gGQofJc7+4IXoa0e4uKT0JTx8E8k3JMa/v5yop+wl3S43Nc9mib1c8KxjXLz5/5K1ZSKrENdw2t6zWD8oaCC5ywdx8mygKLvhv8s+jznce5iFyzug+DF/wIFmisMr3N2KjEJ0qKZaAEeZtXekRCRIHhsm+W0HjAUnOUTR0/hc2TCkxqxdY5hIDun4wxHBgNtwePNN7KQ2wCXf5kpcHJp72jWsFk2wX7MqDIBLKpmLPqNb4d4C3nO9JocMtfjx/sCeoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YjmoY8fsHSqzorBGqSHDpm3AnWulRPSKxDcliH64UiM=; b=B1Xkajjn6BYLu6gi4dlwdebJX8Dj5FvxQ4iMcjgTSL7k7lDq+UW2EE9PpspaxD/F5BsQGrh5ixvCBpNTjNZUNJg3jDI+nNCR4lMA0NpM7qqrIqRZ4pnJGgMZO51JXMBZ/HWSJ+2Ow3L/DBudftTYOhYDY6i6MqWc3MpomO6YnCfB4gIzNjpBgljAjQ4RgSArwl4YXdDYkNnIGYYOJfk0LQQjQlA+PbEX2G9rM2vJqw0RH/kl7sm7ffnnGxcZkDe9JRt5gYVV8G8jFxSrFWWiBD7NUXBPBJ0PXN8TQludC5gE7aeKhDTCeaOdmQT4znZltq/MA0GrlA9E7BkfrZ2Xng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YjmoY8fsHSqzorBGqSHDpm3AnWulRPSKxDcliH64UiM=; b=WTa8ObgbqZ+3Btd+BKoGQmZNmM5nT+648LBW3ZFlyo1v82VxiMF2J1g52QGi+Mf1l16qt1pAxyUQ5ZNNFn0iTeiU3JZRBp59ImpGtSUaTNcK0Il/bVhG3atZSYEV56cgomrpPuSRQ/cHyvhcu9AdE5h6WCugVBLVt8fjVZnHql8= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Jan Beulich , Paul Durrant , =?utf-8?q?Rog?= =?utf-8?q?er_Pau_Monn=C3=A9?= , "Stewart Hildebrand" Subject: [PATCH v2 3/8] iommu/arm: Introduce iommu_add_dt_pci_device API Date: Thu, 11 May 2023 15:16:49 -0400 Message-ID: <20230511191654.400720-4-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230511191654.400720-1-stewart.hildebrand@amd.com> References: <20230511191654.400720-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT055:EE_|PH8PR12MB6745:EE_ X-MS-Office365-Filtering-Correlation-Id: 8850e3da-9fe3-4b02-d190-08db52548103 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5LoKySNp3TsAYAH4AYg1lfWfUV+HSJ3y8Zy5kvyWQqviKFHsKY72LYNR8Ji6dcr8r2RYHhayMHg72zB3jz18zS5cDYf1VcJCYNVR1gCAR3zse2zyOlxYSZl5QAdMbX+cFaCDZCFa19CYReTuvcc/ZzzjC92QWocuM1oB4FU2YCazXl/i3td+swGbbNFdGjd8KQhRztEypZoWHsNhrGnyQW6vEBYzwWpAX4ABTBASzXBPC7y0AN7yF8zWhSlm0+wR3dHHDomjggOsfJaCmGAHDEncq23xvHxpPSvstciBoghaefLigBZ/SFn2WtBmSHtbfYMp80D7LgvA9yvTOco3Hz+ToJN9NV45e7HC9H+5wn45G23z1wj7WnhSTi/FV2p+HjKWOQO573qJHpN6SbvEQIIC01I3xtL+5vf540uH2L0dDHYKS62ZrjnK2rOdvhXZDE09dgiS24IZkK84RR9sKpH6AxT3huHE6cLk5y5BoatQXGe/NEEX/jhyJBixpAm8+sBDyrElKiBJOkfeHgM0I6ColMND2fvYEKhnSlijMfJ8TlSZpBEj5FJuOnfKfUWqlUekEK8F0GkH+erhx31ocieQWhZY+L96gOAOHlcIQx/nBAYRVRhtJZEBxe9ZayMWcIkbEfK9EEdiwCvvJADd0Hob1pgSm+kBw8a3nhTrfePOgTaufrrIhJs8UFj9DP8s4SwvSp3qfGTw0X58HCJ0Wip8erT3AVuM/OoQjKMqxTJ8NZndhlCTkCH+DYtdEkSpc9JXi1POdMVxl2Rmy+Neg1haZNeGYdNgZjimeIDm0nIujo+HqsEylnnGxEbvCo1U9duei52ub4DCMkRiCBcABA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(39860400002)(396003)(346002)(376002)(136003)(451199021)(46966006)(36840700001)(40470700004)(2616005)(186003)(8676002)(8936002)(40460700003)(6666004)(82310400005)(26005)(40480700001)(36756003)(5660300002)(1076003)(2906002)(356005)(44832011)(316002)(82740400003)(81166007)(478600001)(54906003)(70206006)(47076005)(83380400001)(6916009)(70586007)(966005)(4326008)(36860700001)(86362001)(336012)(426003)(41300700001)(21314003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2023 19:18:29.7517 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8850e3da-9fe3-4b02-d190-08db52548103 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT055.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB6745 From: Oleksandr Tyshchenko The main purpose of this patch is to add a way to register PCI device (which is behind the IOMMU) using the generic PCI-IOMMU DT bindings [1] before assigning that device to a domain. This behaves in almost the same way as existing iommu_add_dt_device API, the difference is in devices to handle and DT bindings to use. The function of_map_id to translate an ID through a downstream mapping (which is also suitable for mapping Requester ID) was borrowed from Linux (v5.10-rc6) and updated according to the Xen code base. XXX: I don't port pci_for_each_dma_alias from Linux which is a part of PCI-IOMMU bindings infrastucture as I don't have a good understanding for how it is expected to work in Xen environment. Also it is not completely clear whether we need to distinguish between different PCI types here (DEV_TYPE_PCI, DEV_TYPE_PCI_HOST_BRIDGE, etc). For example, how we should behave here if the host bridge doesn't have a stream ID (so not described in iommu-map property) just simple fail or bypasses translation? [1] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/pci-iommu.txt Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand --- v1->v2: * remove extra devfn parameter since pdev fully describes the device * remove ops->add_device() call from iommu_add_dt_pci_device(). Instead, rely on the existing iommu call in iommu_add_device(). * move the ops->add_device and ops->dt_xlate checks earlier downstream->v1: * rebase * add const qualifier to struct dt_device_node *np arg in dt_map_id() * add const qualifier to struct dt_device_node *np declaration in iommu_add_pci_device() * use stdint.h types instead of u8/u32/etc... * rename functions: s/dt_iommu_xlate/iommu_dt_xlate/ s/dt_map_id/iommu_dt_pci_map_id/ s/iommu_add_pci_device/iommu_add_dt_pci_device/ * add device_is_protected check in iommu_add_dt_pci_device * wrap prototypes in CONFIG_HAS_PCI (cherry picked from commit 734e3bf6ee77e7947667ab8fa96c25b349c2e1da from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/drivers/passthrough/device_tree.c | 140 ++++++++++++++++++++++++++ xen/include/xen/device_tree.h | 25 +++++ xen/include/xen/iommu.h | 6 +- 3 files changed, 170 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index 1b50f4670944..5e462e5c2ca8 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -151,6 +151,146 @@ static int iommu_dt_xlate(struct device *dev, return ops->dt_xlate(dev, iommu_spec); } +#ifdef CONFIG_HAS_PCI +int iommu_dt_pci_map_id(const struct dt_device_node *np, uint32_t id, + const char *map_name, const char *map_mask_name, + struct dt_device_node **target, uint32_t *id_out) +{ + uint32_t map_mask, masked_id, map_len; + const __be32 *map = NULL; + + if ( !np || !map_name || (!target && !id_out) ) + return -EINVAL; + + map = dt_get_property(np, map_name, &map_len); + if ( !map ) + { + if ( target ) + return -ENODEV; + /* Otherwise, no map implies no translation */ + *id_out = id; + return 0; + } + + if ( !map_len || map_len % (4 * sizeof(*map)) ) + { + printk(XENLOG_ERR "%pOF: Error: Bad %s length: %d\n", np, + map_name, map_len); + return -EINVAL; + } + + /* The default is to select all bits. */ + map_mask = 0xffffffff; + + /* + * Can be overridden by "{iommu,msi}-map-mask" property. + * If of_property_read_u32() fails, the default is used. + */ + if ( map_mask_name ) + dt_property_read_u32(np, map_mask_name, &map_mask); + + masked_id = map_mask & id; + for ( ; (int)map_len > 0; map_len -= 4 * sizeof(*map), map += 4 ) + { + struct dt_device_node *phandle_node; + uint32_t id_base = be32_to_cpup(map + 0); + uint32_t phandle = be32_to_cpup(map + 1); + uint32_t out_base = be32_to_cpup(map + 2); + uint32_t id_len = be32_to_cpup(map + 3); + + if ( id_base & ~map_mask ) + { + printk(XENLOG_ERR "%pOF: Invalid %s translation - %s-mask (0x%x) ignores id-base (0x%x)\n", + np, map_name, map_name, map_mask, id_base); + return -EFAULT; + } + + if ( masked_id < id_base || masked_id >= id_base + id_len ) + continue; + + phandle_node = dt_find_node_by_phandle(phandle); + if ( !phandle_node ) + return -ENODEV; + + if ( target ) + { + if ( !*target ) + *target = phandle_node; + + if ( *target != phandle_node ) + continue; + } + + if ( id_out ) + *id_out = masked_id - id_base + out_base; + + printk(XENLOG_DEBUG "%pOF: %s, using mask %08x, id-base: %08x, out-base: %08x, length: %08x, id: %08x -> %08x\n", + np, map_name, map_mask, id_base, out_base, id_len, id, + masked_id - id_base + out_base); + return 0; + } + + printk(XENLOG_ERR "%pOF: no %s translation for id 0x%x on %pOF\n", + np, map_name, id, target && *target ? *target : NULL); + + /* + * NOTE: Linux bypasses translation without returning an error here, + * but should we behave in the same way on Xen? Restrict for now. + */ + return -EFAULT; +} + +int iommu_add_dt_pci_device(struct pci_dev *pdev) +{ + const struct iommu_ops *ops = iommu_get_ops(); + struct dt_phandle_args iommu_spec = { .args_count = 1 }; + struct device *dev = pci_to_dev(pdev); + const struct dt_device_node *np; + int rc = NO_IOMMU; + + if ( !iommu_enabled ) + return NO_IOMMU; + + if ( !ops ) + return -EINVAL; + + if ( device_is_protected(dev) ) + return 0; + + if ( dev_iommu_fwspec_get(dev) ) + return -EEXIST; + + np = pci_find_host_bridge_node(pdev); + if ( !np ) + return -ENODEV; + + /* + * The driver which supports generic PCI-IOMMU DT bindings must have + * these callback implemented. + */ + if ( !ops->add_device || !ops->dt_xlate ) + return -EINVAL; + + /* + * According to the Documentation/devicetree/bindings/pci/pci-iommu.txt + * from Linux. + */ + rc = iommu_dt_pci_map_id(np, PCI_BDF(pdev->bus, pdev->devfn), "iommu-map", + "iommu-map-mask", &iommu_spec.np, iommu_spec.args); + if ( rc ) + return rc == -ENODEV ? NO_IOMMU : rc; + + rc = iommu_dt_xlate(dev, &iommu_spec); + if ( rc < 0 ) + { + iommu_fwspec_free(pci_to_dev(pdev)); + return -EINVAL; + } + + return rc; +} +#endif /* CONFIG_HAS_PCI */ + int iommu_add_dt_device(struct dt_device_node *np) { const struct iommu_ops *ops = iommu_get_ops(); diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h index c1e4751a581f..dc40fdfb9231 100644 --- a/xen/include/xen/device_tree.h +++ b/xen/include/xen/device_tree.h @@ -852,6 +852,31 @@ int dt_count_phandle_with_args(const struct dt_device_node *np, */ int dt_get_pci_domain_nr(struct dt_device_node *node); +#ifdef CONFIG_HAS_PCI +/** + * iommu_dt_pci_map_id - Translate an ID through a downstream mapping. + * @np: root complex device node. + * @id: device ID to map. + * @map_name: property name of the map to use. + * @map_mask_name: optional property name of the mask to use. + * @target: optional pointer to a target device node. + * @id_out: optional pointer to receive the translated ID. + * + * Given a device ID, look up the appropriate implementation-defined + * platform ID and/or the target device which receives transactions on that + * ID, as per the "iommu-map" and "msi-map" bindings. Either of @target or + * @id_out may be NULL if only the other is required. If @target points to + * a non-NULL device node pointer, only entries targeting that node will be + * matched; if it points to a NULL value, it will receive the device node of + * the first matching target phandle, with a reference held. + * + * Return: 0 on success or a standard error code on failure. + */ +int iommu_dt_pci_map_id(const struct dt_device_node *np, uint32_t id, + const char *map_name, const char *map_mask_name, + struct dt_device_node **target, uint32_t *id_out); +#endif /* CONFIG_HAS_PCI */ + struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle); #ifdef CONFIG_DEVICE_TREE_DEBUG diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h index 405db59971c5..7cb4d2aa5511 100644 --- a/xen/include/xen/iommu.h +++ b/xen/include/xen/iommu.h @@ -219,7 +219,8 @@ int iommu_dt_domain_init(struct domain *d); int iommu_release_dt_devices(struct domain *d); /* - * Helper to add master device to the IOMMU using generic IOMMU DT bindings. + * Helpers to add master device to the IOMMU using generic (PCI-)IOMMU + * DT bindings. * * Return values: * 0 : device is protected by an IOMMU @@ -228,6 +229,9 @@ int iommu_release_dt_devices(struct domain *d); * (IOMMU is not enabled/present or device is not connected to it). */ int iommu_add_dt_device(struct dt_device_node *np); +#ifdef CONFIG_HAS_PCI +int iommu_add_dt_pci_device(struct pci_dev *pdev); +#endif int iommu_do_dt_domctl(struct xen_domctl *, struct domain *, XEN_GUEST_HANDLE_PARAM(xen_domctl_t)); From patchwork Thu May 11 19:16:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13238416 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9BE07C77B7C for ; Thu, 11 May 2023 19:19:29 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.533560.830341 (Exim 4.92) (envelope-from ) id 1pxBos-0006P0-On; Thu, 11 May 2023 19:19:18 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 533560.830341; Thu, 11 May 2023 19:19:18 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBos-0006Ot-Lj; Thu, 11 May 2023 19:19:18 +0000 Received: by outflank-mailman (input) for mailman id 533560; Thu, 11 May 2023 19:19:17 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBor-0005hc-Er for xen-devel@lists.xenproject.org; Thu, 11 May 2023 19:19:17 +0000 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on20604.outbound.protection.outlook.com [2a01:111:f400:7e88::604]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id b83c3e05-f030-11ed-b229-6b7b168915f2; Thu, 11 May 2023 21:19:16 +0200 (CEST) Received: from DS7PR05CA0036.namprd05.prod.outlook.com (2603:10b6:8:2f::25) by DS0PR12MB7876.namprd12.prod.outlook.com (2603:10b6:8:148::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.18; Thu, 11 May 2023 19:19:14 +0000 Received: from DM6NAM11FT101.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2f:cafe::a1) by DS7PR05CA0036.outlook.office365.com (2603:10b6:8:2f::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.20 via Frontend Transport; Thu, 11 May 2023 19:19:13 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT101.mail.protection.outlook.com (10.13.172.208) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:19:13 +0000 Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 14:19:13 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 12:19:12 -0700 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Thu, 11 May 2023 14:19:11 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b83c3e05-f030-11ed-b229-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cHlBC3twaD0QcwAZzctAdvpgiJLTsz63RaOEVJ9msGHr8Y7j2zoLjVFuny0/5ogQz5rxJ1Q+NRSoufixqQCeNkpimpZkIgSnXM+DW6asFCLMt3NTAnSrj/zIIZzDF4SDUnnpnIQmdZhreh6fBzsq5UUaNJH+marOXj3Rh4VIPCOoaM4Er1HxwAQ4WA/mAd8e7gqWCy7igpxVN7DmiECKvErtNXC4AvuXQzOWZPDVHy31qAFbEQwbX9cg/ngqYz+6vZ9GyVd2pzE+w5CjK7moXejaEDWF6dvPNgbTDPDm47so+ox2K8d5JN3+5TDP5X6SFKK5Y5Jv9H9QAsua1ccshg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Inv85I3n2ejvqoWpRiGzQrvDhgSa+TdZRQ6ClAmebH8=; b=lvpyvywFyUV9sB/e1wCiHUb2rF9rs/1Dbcm/RSG+CROWgjBzD6OqdMKTbdGfwQzu63HecwTAokLVKuaNNm0+BlYsO8tniNiFiRXh5QkBWwxKY3UK0bBX5z9B2b9bzA+g3KXbsQpnOBdakYjMZF7bOVk8u/BhsgsgYYY19FVhP1j5FChuvVOKg7SwvWUFETOJC7L2xAmkLuALR10y5L/REupwNYyG/wK1rbjx/BjU8+Z+IwjCwRTN3SaSy7/NQwY+9x/cbnV6sZMZfKH4sagXUy8j5MV15h0kDnbe9Elpxrd3GVLaSIdynjDb7kq58V2YZYPNLc43WmGBh8zGYCcwZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Inv85I3n2ejvqoWpRiGzQrvDhgSa+TdZRQ6ClAmebH8=; b=rXphUoHFH8OuKLb8YQ9JRF1clLyR1HiDbY5zUpbeYDDfkyyYPrDRtIzyUGU0APGXstcKp+GcMylqIr2q2mWjU3+o3pNH/xdOlNfC6fom4WnN4Ufuy8TrtNfKacvrCwvHeDzxPsRpwAdGvRBy2/A6AN6RXZsTVlqpD9rRD1AAKeA= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Stewart Hildebrand , Stefano Stabellini , Julien Grall , Jan Beulich Subject: [PATCH v2 4/8] iommu/arm: iommu_add_dt_pci_device phantom handling Date: Thu, 11 May 2023 15:16:50 -0400 Message-ID: <20230511191654.400720-5-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230511191654.400720-1-stewart.hildebrand@amd.com> References: <20230511191654.400720-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT101:EE_|DS0PR12MB7876:EE_ X-MS-Office365-Filtering-Correlation-Id: 64e53b38-bdff-4ffc-663e-08db52549b27 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hN4ihwctcp6vilATTmpnC9zylTk3X/7PDQiM6B0SHx5O93vqoFB0yPa2+soU8K5kbOhorNaF6ZOPvY46X55yrlZHP8fsxlC+PidT0sUWS0pV/cuaW9doAjAP8vGo/NMOcfKKg7cA1kLEeRz9Y7mq42IJo5tEwmIKRcRljiuAq5j7og4sZ22aP83FBhUZRTOFx6Wt4TnTe5G5TCWLYaM+7FOcEQYSS2bGGHRgKF0VPcoo3ARLbOIjTJJcddeOHmLwncCcuGMOPdDdNYXZnf3fdRZDKjdJSGe6RxzDHk+NCsBgjNCE3JGFxJsO7P+XKfyIfnwSugAi2bjeXr15SynAUvCLcIb+7vEd0NJ6FBBxalmKGGuk8WAcMCAocjYAaaVWAGcCTPxD8uWxvcwkiAd0ebPYDT8gexXVhAPuyR67IVJl5DBCBwUhAvF/TeVVWdOt5krNM6PFqjZtQSdDjo6NgNj2t7PZeIpA4vwR2NoyuWLdAFDgEvu/eCF/cac4Rh5qn9GCdlBTvaKvEfDjTmzL2E9OY3xkf6BIB912ls8Ciw3Xmakb6+V8rGN+Z6h78IU6uNhmGXyPwis13LtvFTMQU8e8/Tzw6PoorUBu98Qz85u/QCOc4+ZA/5s/ReaqtkA926s/o7fLOoRmxGRhJWsFbCmdgbOtdgGW6llTcKBZHmzt1F508Ov8ZCuLjPaoMFbmOPjl92oY/yFCXLRu4AgSQulznURIpD06wrd9ZaaAHKbfOrZh0X37AODzPQeG7TAl X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(39860400002)(376002)(136003)(396003)(451199021)(36840700001)(46966006)(40470700004)(47076005)(81166007)(54906003)(40480700001)(41300700001)(316002)(83380400001)(356005)(82740400003)(1076003)(26005)(82310400005)(8936002)(336012)(426003)(70586007)(6916009)(4326008)(70206006)(36860700001)(5660300002)(8676002)(86362001)(44832011)(186003)(966005)(478600001)(36756003)(6666004)(40460700003)(2616005)(2906002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2023 19:19:13.5900 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 64e53b38-bdff-4ffc-663e-08db52549b27 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT101.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7876 Handle phantom functions in iommu_add_dt_pci_device(). Each phantom function will have a unique requestor ID (RID)/BDF. On ARM, we need to map/translate the RID/BDF to an AXI stream ID for each phantom function according to the pci-iommu device tree mapping [1]. The RID/BDF -> AXI stream ID mapping in DT could allow phantom devices (i.e. devices with phantom functions) to use different AXI stream IDs based on the (phantom) function. [1] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/pci-iommu.txt Signed-off-by: Stewart Hildebrand --- v1->v2: * new patch --- xen/drivers/passthrough/device_tree.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index 5e462e5c2ca8..ced911f4fb31 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -247,6 +247,7 @@ int iommu_add_dt_pci_device(struct pci_dev *pdev) struct device *dev = pci_to_dev(pdev); const struct dt_device_node *np; int rc = NO_IOMMU; + unsigned int devfn = pdev->devfn; if ( !iommu_enabled ) return NO_IOMMU; @@ -275,7 +276,7 @@ int iommu_add_dt_pci_device(struct pci_dev *pdev) * According to the Documentation/devicetree/bindings/pci/pci-iommu.txt * from Linux. */ - rc = iommu_dt_pci_map_id(np, PCI_BDF(pdev->bus, pdev->devfn), "iommu-map", + rc = iommu_dt_pci_map_id(np, PCI_BDF(pdev->bus, devfn), "iommu-map", "iommu-map-mask", &iommu_spec.np, iommu_spec.args); if ( rc ) return rc == -ENODEV ? NO_IOMMU : rc; @@ -286,6 +287,26 @@ int iommu_add_dt_pci_device(struct pci_dev *pdev) iommu_fwspec_free(pci_to_dev(pdev)); return -EINVAL; } + for ( ; pdev->phantom_stride ; ) + { + devfn += pdev->phantom_stride; + if ( PCI_SLOT(devfn) != PCI_SLOT(pdev->devfn) ) + break; + rc = iommu_dt_pci_map_id(np, PCI_BDF(pdev->bus, devfn), "iommu-map", + "iommu-map-mask", &iommu_spec.np, iommu_spec.args); + if ( rc ) + { + printk(XENLOG_WARNING "IOMMU: add %pp failed (%d)\n", + &PCI_SBDF(pdev->seg, pdev->bus, devfn), rc); + return rc == -ENODEV ? NO_IOMMU : rc; + } + rc = iommu_dt_xlate(dev, &iommu_spec); + if ( rc < 0 ) + { + iommu_fwspec_free(pci_to_dev(pdev)); + return -EINVAL; + } + } return rc; } From patchwork Thu May 11 19:16:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13238428 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 72CE3C77B7F for ; Thu, 11 May 2023 19:27:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.533578.830380 (Exim 4.92) (envelope-from ) id 1pxBws-0001PX-E9; Thu, 11 May 2023 19:27:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 533578.830380; Thu, 11 May 2023 19:27:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBws-0001PQ-BK; Thu, 11 May 2023 19:27:34 +0000 Received: by outflank-mailman (input) for mailman id 533578; Thu, 11 May 2023 19:27:33 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBpB-00042a-D8 for xen-devel@lists.xenproject.org; Thu, 11 May 2023 19:19:37 +0000 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on20629.outbound.protection.outlook.com [2a01:111:f400:7eab::629]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c30d4630-f030-11ed-8611-37d641c3527e; Thu, 11 May 2023 21:19:35 +0200 (CEST) Received: from DS7P222CA0023.NAMP222.PROD.OUTLOOK.COM (2603:10b6:8:2e::9) by DS7PR12MB5911.namprd12.prod.outlook.com (2603:10b6:8:7c::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.18; Thu, 11 May 2023 19:19:31 +0000 Received: from DM6NAM11FT093.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2e:cafe::9e) by DS7P222CA0023.outlook.office365.com (2603:10b6:8:2e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:19:31 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT093.mail.protection.outlook.com (10.13.172.235) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:19:31 +0000 Received: from SATLEXMB07.amd.com (10.181.41.45) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 14:19:30 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB07.amd.com (10.181.41.45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 12:19:30 -0700 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Thu, 11 May 2023 14:19:28 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c30d4630-f030-11ed-8611-37d641c3527e ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z62fgB6WMELun9BSFmliivebJ5ZvIqlHQuq5HKMM+7NuUREA8YK69GgS1H+Icfa7jriO+hm/znpTDCYgiWLRnm4rPeVG7/uyg+Ld35p66cRCGE8znaD6zdX1ULGZKBUWJhhmYMDfcGF///QIUM0V4UGx+/91mTbzhwFuhCP5tRAyDCnl3rRynzlEu0GiouQACMPbC5mMhwNLNNUfeadwzeP0CnvZF4tt+raXjqGMw5kOuOG6BTRM7XujSDtmdDvuWKM/BkmDO3OA59NTpgY4Gkf7pdqALB2puYkalPFYKZUVlDuHzQpAejuWRibBoJQUHVXzRM0OJrx8y00slHd9ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bw0rFieP9TglgEJiPQkHGTzJjkGcjQqChjjGQBEv5dQ=; b=C+ZrViLwvKc3xnHec0guBvhmK0C8aJaIg5ARM4oghkIeY5At1hqYvhFM1++VgyfDAwy3A1eBnd1xrnPyvRUaey69xoYnhEpMGFz8hUo39+jsu0pF7yz/lmcsVRdhlxtiSh2kicwVRGDntt2rcKfWtzJYZUyQKvn1Pwv3YvtK5qDYAnmmq5ofhXrrrfcl87Kb1X2U3424vGKWmrnGcMri1mED0uineT3rVrJvaaqtABr00V45CStyRJe6UCgRR3rfFCFqs1FmHMRNlXGKrlm+8L6sg9BW7kS8QL6WNxIeRUO/gFX+AJRWc0aRKQ0g2ZEiOwwNbmjFuNNWcFCahnOzIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bw0rFieP9TglgEJiPQkHGTzJjkGcjQqChjjGQBEv5dQ=; b=4ie/rqrNqJOuaGk+X0TXvsO7gIVhKgqH4AZIGH67kU7nrY49OO1C4K+ldJ1dtfciTt/YPrzNCHj8KoLeipA4zTHBqE0G8BOkad1a/RnkDRN1pbSki2HM3d/qnhZnLKCswE7ExAnSXGH4CqOogddj0SmBeRiraQ84ZaGEAnyZujc= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Jan Beulich , Paul Durrant , =?utf-8?q?Rog?= =?utf-8?q?er_Pau_Monn=C3=A9?= , Julien Grall , Rahul Singh , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk , Stewart Hildebrand Subject: [PATCH v2 5/8] pci/arm: Use iommu_add_dt_pci_device() Date: Thu, 11 May 2023 15:16:51 -0400 Message-ID: <20230511191654.400720-6-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230511191654.400720-1-stewart.hildebrand@amd.com> References: <20230511191654.400720-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT093:EE_|DS7PR12MB5911:EE_ X-MS-Office365-Filtering-Correlation-Id: aedad1e9-6edc-487b-3cca-08db5254a5bd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /JCPZQF+b+gEQ7Aq5Z2HwxiuUbrh7bQR2eRmoN4NuvXTxiIukSybeSfgepwVVoAZ7pQ5LnNIrNsHJgzWq+dko+4KsW6Om/o/WJNMYQZjaciPQM+womfyzZ8T0J0f6q/gW/byBgC0r1YdC+sEU98NT0UWt65WNe3BBcEfY/gLbNTyEuJcspJ97swiiQf6MvYVl+FpIaurGUddLx8oib/libFWq1xyz3yU3P/NCfisU4z8dij0pwI585wSi/6eb+Z+N/1niE/jCCHIjz0+srQpJ8+IG6Y/ipvPa30hH2lyPHS9ieQMJltg6RDO2IqrO//9ChlwWljO9+r7XFh73/heAAFvb9xak55cCz9aMmc+LTZJRZ36eCgrVnOPVAhKOFcwq3KNvBYqtffWt03WXsc+84W9QsedDalDMW8RySk/PAkdZXTq5S6qEc3R33c/8EssaXnPFezaF9YfewLNAfO0o+a4I/l1xn8Rwqvh5PD1KxNHirCcMWfcViEbHjrWsoCEUvXSP3PidaAGkeiWVaanaOiwdh18btELwEfWxMivJahntk1dJKqViGXwKUdkSZyl2VYCmEvqv1gaAGB8k0sTr1+yRskLkANi1tjSTbI61Jr8epxi1uhjXhgVZGgR1uneZtSaS0lAqQlkbhknhbjRPOKWdVQDNFkdyNsak6xepvH0DbwgNWAq3MMGDAATROsQTz/MPwVgrudLXfRz+rj83tPFuy03yQf/KN48g6IgupOijicMirmg4ULctbbIMkIZhm/lh204Q7GZJqvP59gqrw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(39860400002)(396003)(376002)(346002)(136003)(451199021)(46966006)(40470700004)(36840700001)(40480700001)(81166007)(1076003)(316002)(186003)(6916009)(4326008)(86362001)(26005)(70206006)(70586007)(40460700003)(82310400005)(54906003)(356005)(478600001)(36756003)(44832011)(47076005)(5660300002)(6666004)(966005)(7416002)(2906002)(36860700001)(8936002)(41300700001)(8676002)(82740400003)(336012)(426003)(2616005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2023 19:19:31.3648 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: aedad1e9-6edc-487b-3cca-08db5254a5bd X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT093.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5911 From: Oleksandr Tyshchenko On Arm we need to parse DT PCI-IOMMU specifier and provide it to the driver (for describing the relationship between PCI devices and IOMMUs) before adding a device to it. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand --- v1->v2: * new patch title (was: "pci/arm: Use iommu_add_dt_pci_device() instead of arch hook") * move iommu_add_dt_pci_device() call (and associated #ifdef) to pci_add_device() * use existing call to iommu_add_device() downstream->v1: * rebase * add __maybe_unused attribute to const struct domain_iommu *hd; * Rename: s/iommu_add_pci_device/iommu_add_dt_pci_device/ * guard iommu_add_dt_pci_device call with CONFIG_HAS_DEVICE_TREE instead of CONFIG_ARM (cherry picked from commit 2b9d26badab8b24b5a80d028c4499a5022817213 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/drivers/passthrough/pci.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index b42acb8d7c09..6dbaae682773 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -34,6 +34,11 @@ #include #include #include + +#ifdef CONFIG_HAS_DEVICE_TREE +#include +#endif + #include "ats.h" struct pci_seg { @@ -762,9 +767,20 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, pdev->domain = NULL; goto out; } +#ifdef CONFIG_HAS_DEVICE_TREE + ret = iommu_add_dt_pci_device(pdev); + if ( ret < 0 ) + { + printk(XENLOG_ERR "pci-iommu translation failed: %d\n", ret); + goto out; + } +#endif ret = iommu_add_device(pdev); if ( ret ) { +#ifdef CONFIG_HAS_DEVICE_TREE + iommu_fwspec_free(pci_to_dev(pdev)); +#endif vpci_remove_device(pdev); list_del(&pdev->domain_list); pdev->domain = NULL; From patchwork Thu May 11 19:16:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13238421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 33C66C77B7C for ; Thu, 11 May 2023 19:21:29 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.533563.830351 (Exim 4.92) (envelope-from ) id 1pxBqg-0007sp-4G; Thu, 11 May 2023 19:21:10 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 533563.830351; Thu, 11 May 2023 19:21:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBqg-0007si-1S; Thu, 11 May 2023 19:21:10 +0000 Received: by outflank-mailman (input) for mailman id 533563; Thu, 11 May 2023 19:21:09 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBqf-0007sN-33 for xen-devel@lists.xenproject.org; Thu, 11 May 2023 19:21:09 +0000 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on20625.outbound.protection.outlook.com [2a01:111:f400:7e88::625]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id fa77328d-f030-11ed-b229-6b7b168915f2; Thu, 11 May 2023 21:21:08 +0200 (CEST) Received: from DS7PR03CA0128.namprd03.prod.outlook.com (2603:10b6:5:3b4::13) by SJ2PR12MB8884.namprd12.prod.outlook.com (2603:10b6:a03:547::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.20; Thu, 11 May 2023 19:21:01 +0000 Received: from DM6NAM11FT109.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3b4:cafe::e) by DS7PR03CA0128.outlook.office365.com (2603:10b6:5:3b4::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:21:01 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT109.mail.protection.outlook.com (10.13.173.178) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:21:01 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 14:21:00 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Thu, 11 May 2023 14:20:58 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: fa77328d-f030-11ed-b229-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZQAw7w0st9xKb6p8aeneQt9eobiV7QsJIVBkAMup4rxrPMIehdZSrz4jKmTkHHuMw7nRfACVKXjwC7BAz4Gd3MbiI/phHO+K7tw0tbWE9h9WP7mPxpzKzmnVT0ZeTLkg8Dk1gEiqwaWw8rfv1ccgdqMZn2LcX0F/3ZM/fzZrLna5ZfbeWuiCsglANvkBTXeIdWSJCM6mEFL9eB189hwxY21+MjQZEWjqo0/C7WhKjHUGWIZzkf9C3lpqn82IIKZrpf6paQzSbPUiEgnTjC5t7HR1NYrdsjNSRXKiwkJgIQK2LMhpwAePKG1oQcc2vTMl2fBgP//mEiGgM7e7h6KFoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GjAtPrq7Zr2LNV7JB6qbwRaGGVcDQHrRJHlX0JDBPbY=; b=mcP4s4F3g55OB3UvRSMRgfTjj1CuPGatFt3t5L5B2hSsR9cFyPoQmHp54Ngma4t3MzY1Vxt3pw9xHyhce0AVjsSNioEurVg2QoyD424VeMyRvPyb//wvaLnDWACn32HaT3/udMeFYhBYvIUgid/d470IOmFMpW9ZuN7yf2k1AGJAedL0OSa0bs6pcI1EjnPeImFHgBrchpA/XU7rK4itP3i4Y9S0A94WmPWTYr4tEqRVZuwaPMa9ia4TGp7O9Ykrwu6b7rTJIw++KJA8fLsUEjgV8x+ZGL5dKjve2WqEnSrzWlBLHH9BVHcM8PgKRjKy8KGr5shCRnkPg9zOJKb8uw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GjAtPrq7Zr2LNV7JB6qbwRaGGVcDQHrRJHlX0JDBPbY=; b=Wgq07Ka0qPjcMhcQXk04uu/i+6q+C+xpj/lQgAQh77IOJ/iWVtAzwLGj9krMS4DfTW0wJQOzThxFO8XB+zb7iRnLfFxpD7Mm2pQep6ovJYAASzZ3wKzB8rppR3DA2JNL4IHXv9nmvFJZ/55a7x967feqW2jvOY1EJD2EquVTwhU= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Stewart Hildebrand , Jan Beulich , Paul Durrant , =?utf-8?q?Rog?= =?utf-8?q?er_Pau_Monn=C3=A9?= , Julien Grall , Rahul Singh , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk Subject: [RFC PATCH v2 6/8] pci/arm: don't do iommu call for phantom functions Date: Thu, 11 May 2023 15:16:52 -0400 Message-ID: <20230511191654.400720-7-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230511191654.400720-1-stewart.hildebrand@amd.com> References: <20230511191654.400720-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT109:EE_|SJ2PR12MB8884:EE_ X-MS-Office365-Filtering-Correlation-Id: ff383d8c-2656-4bb3-7375-08db5254db40 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XEYm88fTWcuzOiKkobw4GSBwO2PuLjnE7oNBdzPsN8fxEnbpAsswhruMw+xQjn1YpIRlLY5M7uhq5WUzm+cgKip2CEyCST/uQsvrBF+9N8JohRFagw2rj1mRTh3hFGBlWqGS7vg9PcC3WW4Dwq5AhomUyPEMQ71H6mtbNY6OEzr4X4kfFaT1wJmgSyeKNi0LuQpK1qCQglKkJTMIdutP0689SxKMQ9WcxT69/mSVB2AV5a0j0m6Sn2b9awDi1drJxLXpjOq0Kc/ROePQhx7HOL8DAUTHIZU/FNV0xucHUBmilVfhY4SnDwxLEldmBaJqjk6b22Zfyv0KfvI5N68CagtkmnkASY0buvnszvpO/+6L1PzYvQROK+NiY0jRRAEq4FJPSC6WqLA1DnyV612j2KjyFB/D8a6DcYHYm855pPZL1qLzlXMKjB4hLZGgbMSLNHLAdZgPLhWVsIKGjqvi/HaRZEQDj6iKgaOU8W36ZazL7ij865i6DF1onY22OczlCFWNdXppIMcOYeRysUO8YvHAL/Zb0Htzc4D+xCd8bOgcp6hxtSzV1d0520B4zggRDD6pB16hmTxxa0wLh0yZ5bnnnhhT4h5HQtbEgpTzMpYiroanK8GR123Rew1d1hGlf57IkCmh0DcfY6EQ7XdfJwx3IRipCw7wNFgbAqq20Wyevz6H78RfIwSyJJsXa6KGLOfe65ntek4RRhrwj8CKIOm4oCV3w9vvg8X0wUez9JCMjpNquEhrKYprtGrhaJ7QDnQDdL8Cn8R9paMFkOdoAA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(396003)(346002)(376002)(136003)(39860400002)(451199021)(46966006)(40470700004)(36840700001)(186003)(26005)(1076003)(356005)(81166007)(2906002)(41300700001)(4326008)(316002)(6916009)(82740400003)(47076005)(83380400001)(36860700001)(36756003)(86362001)(8676002)(2616005)(44832011)(336012)(426003)(8936002)(5660300002)(40480700001)(54906003)(478600001)(82310400005)(6666004)(40460700003)(70206006)(70586007)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2023 19:21:01.1440 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ff383d8c-2656-4bb3-7375-08db5254db40 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT109.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8884 It's not necessary to add/remove/assign/deassign pci phantom functions for the ARM SMMU drivers. All associated AXI stream IDs are added during the iommu call for the base PCI device/function. However, the ARM SMMU drivers can cope with the extra/unnecessary calls just fine, so this patch is RFC as it's not strictly required. Signed-off-by: Stewart Hildebrand --- I'm aware the indentation is wrong. I just wanted to keep the diffstat small while this particular patch is RFC. v1->v2: * new patch --- xen/drivers/passthrough/pci.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 6dbaae682773..3823edf096eb 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -871,6 +871,7 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, else target = hardware_domain; + if ( !IS_ENABLED(CONFIG_HAS_DEVICE_TREE) ) while ( pdev->phantom_stride ) { devfn += pdev->phantom_stride; @@ -1335,7 +1336,7 @@ static int iommu_add_device(struct pci_dev *pdev) return 0; rc = iommu_call(hd->platform_ops, add_device, devfn, pci_to_dev(pdev)); - if ( rc || !pdev->phantom_stride ) + if ( rc || !pdev->phantom_stride || IS_ENABLED(CONFIG_HAS_DEVICE_TREE) ) return rc; for ( ; ; ) @@ -1379,6 +1380,7 @@ static int iommu_remove_device(struct pci_dev *pdev) if ( !is_iommu_enabled(pdev->domain) ) return 0; + if ( !IS_ENABLED(CONFIG_HAS_DEVICE_TREE) ) for ( devfn = pdev->devfn ; pdev->phantom_stride; ) { int rc; @@ -1464,6 +1466,7 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) pci_to_dev(pdev), flag)) ) goto done; + if ( !IS_ENABLED(CONFIG_HAS_DEVICE_TREE) ) for ( ; pdev->phantom_stride; rc = 0 ) { devfn += pdev->phantom_stride; From patchwork Thu May 11 19:16:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13238422 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C7E3DC77B7C for ; Thu, 11 May 2023 19:22:29 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.533567.830361 (Exim 4.92) (envelope-from ) id 1pxBrd-0008Qf-F4; Thu, 11 May 2023 19:22:09 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 533567.830361; Thu, 11 May 2023 19:22:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBrd-0008QY-Bl; Thu, 11 May 2023 19:22:09 +0000 Received: by outflank-mailman (input) for mailman id 533567; Thu, 11 May 2023 19:22:07 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBrb-0008QA-G5 for xen-devel@lists.xenproject.org; Thu, 11 May 2023 19:22:07 +0000 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on20620.outbound.protection.outlook.com [2a01:111:f400:7e89::620]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1bda4b36-f031-11ed-8611-37d641c3527e; Thu, 11 May 2023 21:22:04 +0200 (CEST) Received: from BY3PR03CA0009.namprd03.prod.outlook.com (2603:10b6:a03:39a::14) by MN0PR12MB6368.namprd12.prod.outlook.com (2603:10b6:208:3d2::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.20; Thu, 11 May 2023 19:22:00 +0000 Received: from DM6NAM11FT029.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:39a:cafe::67) by BY3PR03CA0009.outlook.office365.com (2603:10b6:a03:39a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.20 via Frontend Transport; Thu, 11 May 2023 19:21:59 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT029.mail.protection.outlook.com (10.13.173.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:21:59 +0000 Received: from SATLEXMB07.amd.com (10.181.41.45) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 14:21:59 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB07.amd.com (10.181.41.45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 12:21:58 -0700 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Thu, 11 May 2023 14:21:56 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1bda4b36-f031-11ed-8611-37d641c3527e ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JvMfM6sm2eQUSXF6J+K09dyMMFcKESUeF9HoASPMtNVKB4xPxheK4xAxOmofVP2/lFZCNfzvGhCYEXj7ZhpPclm8jNiCyT4j+MPsy3ekvPWAtpQCP5BiH0lGwwjPnieBIrpd/C4yfY6SK3zhXx0/9xHktUzIynCQFf8OOyWzrWkaagOB40efi3x5lRJkxsIj7CGI91VzKQjCndaWTCaQi2VumYvfYs/CWkg0aBTabOSlSmENmPw3f7ELwdW2jNzhDGv/13MLxegcHI5jjsneckxVVFDfXnV1lCzRm+LeuVF3jvDIHvcFMCACbJxwFuDRjUskYpQjosiVg6QSeD7iBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oJZqJ3OSmTgBvHf2UzJUuvXPD1qwSpScJxGorP0Ak6c=; b=dDTRKnBZh0Z84C5TflKobnntbh3kf6+WnpeZiKLs4wD6qCxzbE7SeLfBNA7N//upLmcSYbrK+Hzp/cYX4EdIL/isLdDxCv37f+998LAKICi5KDyN+R5LwWE1o5vvQVZxwvVy3PqOWtAimvRsHYvav7Q+0Jmqor7XZcrDNplE8a9oQaFWZGniD6rdne9colf1NdCVbexC8tiiBNEq88nefWrkXAHG3F1UGmJlWEpJP9cnhFVLhNNsPljs2eR0fifVKecfi4+4Se6IZBctBb9TZPcbDDLi+5D0Tq6KNKQe0vDR5OoxfFAzI3GjsYc8dGYHMnYQEH3rqucgmBVNl3NRkA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oJZqJ3OSmTgBvHf2UzJUuvXPD1qwSpScJxGorP0Ak6c=; b=HJA+rY3eLG5bcfwDIqAz8BFSnwIU6mIqtaGBtgBIvLyrRGQ7eKJ9Bg9sDO5DpZ9rUuLfCHFxLRTKxh3wQ35QJzJQa9Q9Dplrj3Dl9Ew9SW/VZ8nYUzUpxAbAHj+dQXDshJuWHuX1Sd6DeMd8LVsLElYlbIFsxg6B6VLXpsKxhh8= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Andrushchenko , Julien Grall , Rahul Singh , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk , Oleksandr Tyshchenko , Stewart Hildebrand Subject: [PATCH v2 7/8] xen/arm: smmuv2: Add PCI devices support for SMMUv2 Date: Thu, 11 May 2023 15:16:53 -0400 Message-ID: <20230511191654.400720-8-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230511191654.400720-1-stewart.hildebrand@amd.com> References: <20230511191654.400720-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT029:EE_|MN0PR12MB6368:EE_ X-MS-Office365-Filtering-Correlation-Id: 87a366ff-48a8-4867-74cc-08db5254fe05 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: er6pSDsuUmE9S6VPddE5L9MyS3uJI8AcwGZYrQs2XFjOWiQYghEDUCNzlOxQJU6mek7kihlxFjgYdrdBAH8v9RO5otqafUb3FtznBLJq7CSBAb3W24CYgAEd/2ngkxhlUa6WvMPtbiIDuLtabneQ4YgHQpXNQ5A+FcWAmB3DoVgCMGsP5av7VMXASSs2yfTbVQHrj4ng4eQCh1lxF7FsVpzUHYt/5JL4IKcdwfvZUS+MDFiAT+M5+MyIzJ4uQ0E0mjG14frhvdijSMVgKJfcJsRcK5WOml/6A6m3JNqVJeaI4i6d7YHNCyiJ/QgF2xuwpRJOp3QousGLGLd3PwIUaUxcd6aAYito8JulTvWRx92RYpt6jUcKnRSSx/+6WIBoHSMTdB19K2/70SzTBKSBy1vAAtF+p1LqYwFSncL2ahXpWY1QfwPB0ZyIgwdvoPOinH0qwIbHqQHcrAnHH4GnrFBMPfLJKPQJSfFuOyc+DRa4vmJPfdyO6PUtTlkb32Fpb598mCrUNanbbyYFwVRidR0v37+xyV/Jk6m3QGY/sJNdlSqbYQlQEc7NIbYfcPqTj0OpQobv1FtioBSHyoiPgyp4iPrl2qEsrOqjnBjRPyy7gGDaRPrlZ4H9U82K8J2x+uaQagBYUOPvAbppBfO7XbQ6Omo41qumfd5MBpWQ/0k0hiIAknbSTxavF2OQzyWdJ+QdKnBzZr63sqQAMrmtaesPpYgq/BvpWdZE4med/LD+sYh96PPatSsFsrG7JHwDfZfbscRyauAY4t9okfZnZFOf3iMWMgeWXqRneZz16tI= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(396003)(376002)(136003)(39860400002)(346002)(451199021)(40470700004)(36840700001)(46966006)(5660300002)(44832011)(54906003)(478600001)(6666004)(41300700001)(966005)(8936002)(8676002)(316002)(1076003)(26005)(4326008)(70206006)(6916009)(70586007)(2906002)(83380400001)(336012)(47076005)(40460700003)(426003)(2616005)(186003)(356005)(36860700001)(82740400003)(81166007)(40480700001)(86362001)(82310400005)(36756003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2023 19:21:59.4620 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 87a366ff-48a8-4867-74cc-08db5254fe05 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT029.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6368 From: Oleksandr Andrushchenko Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Stewart Hildebrand --- v1->v2: * ignore add_device/assign_device/reassign_device calls for phantom functions (i.e. devfn != pdev->devfn) downstream->v1: * wrap unused function in #ifdef 0 * remove the remove_device() stub since it was submitted separately to the list [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops https://lists.xenproject.org/archives/html/xen-devel/2023-04/msg00432.html * arm_smmu_(de)assign_dev: return error instead of crashing system * update condition in arm_smmu_reassign_dev * style fixup * add && !is_hardware_domain(d) into condition in arm_smmu_assign_dev() (cherry picked from commit 0c11a7f65f044c26d87d1e27ac6283ef1f9cfb7a from the downstream branch spider-master from https://github.com/xen-troops/xen.git) --- This is a file imported from Linux with modifications for Xen. What should be the coding style for Xen modifications? --- xen/drivers/passthrough/arm/smmu.c | 108 +++++++++++++++++++++++------ 1 file changed, 87 insertions(+), 21 deletions(-) diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c index 5b6024d579a8..a8476a22b096 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -134,8 +134,20 @@ typedef enum irqreturn irqreturn_t; /* Device logger functions * TODO: Handle PCI */ -#define dev_print(dev, lvl, fmt, ...) \ - printk(lvl "smmu: %s: " fmt, dt_node_full_name(dev_to_dt(dev)), ## __VA_ARGS__) +#ifndef CONFIG_HAS_PCI +#define dev_print(dev, lvl, fmt, ...) \ + printk(lvl "smmu: %s: " fmt, dev_name(dev), ## __VA_ARGS__) +#else +#define dev_print(dev, lvl, fmt, ...) ({ \ + if ( !dev_is_pci((dev)) ) \ + printk(lvl "smmu: %s: " fmt, dev_name((dev)), ## __VA_ARGS__); \ + else \ + { \ + struct pci_dev *pdev = dev_to_pci((dev)); \ + printk(lvl "smmu: %pp: " fmt, &pdev->sbdf, ## __VA_ARGS__); \ + } \ +}) +#endif #define dev_dbg(dev, fmt, ...) dev_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__) #define dev_notice(dev, fmt, ...) dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__) @@ -187,6 +199,7 @@ static void __iomem *devm_ioremap_resource(struct device *dev, * Xen: PCI functions * TODO: It should be implemented when PCI will be supported */ +#if 0 /* unused */ #define to_pci_dev(dev) (NULL) static inline int pci_for_each_dma_alias(struct pci_dev *pdev, int (*fn) (struct pci_dev *pdev, @@ -196,6 +209,7 @@ static inline int pci_for_each_dma_alias(struct pci_dev *pdev, BUG(); return 0; } +#endif /* Xen: misc */ #define PHYS_MASK_SHIFT PADDR_BITS @@ -632,7 +646,7 @@ struct arm_smmu_master_cfg { for (i = 0; idx = cfg->smendx[i], i < num; ++i) struct arm_smmu_master { - struct device_node *of_node; + struct device *dev; struct rb_node node; struct arm_smmu_master_cfg cfg; }; @@ -724,7 +738,7 @@ arm_smmu_get_fwspec(struct arm_smmu_master_cfg *cfg) { struct arm_smmu_master *master = container_of(cfg, struct arm_smmu_master, cfg); - return dev_iommu_fwspec_get(&master->of_node->dev); + return dev_iommu_fwspec_get(master->dev); } static void parse_driver_options(struct arm_smmu_device *smmu) @@ -757,7 +771,7 @@ static struct device_node *dev_get_dev_node(struct device *dev) } static struct arm_smmu_master *find_smmu_master(struct arm_smmu_device *smmu, - struct device_node *dev_node) + struct device *dev) { struct rb_node *node = smmu->masters.rb_node; @@ -766,9 +780,9 @@ static struct arm_smmu_master *find_smmu_master(struct arm_smmu_device *smmu, master = container_of(node, struct arm_smmu_master, node); - if (dev_node < master->of_node) + if (dev < master->dev) node = node->rb_left; - else if (dev_node > master->of_node) + else if (dev > master->dev) node = node->rb_right; else return master; @@ -803,9 +817,9 @@ static int insert_smmu_master(struct arm_smmu_device *smmu, = container_of(*new, struct arm_smmu_master, node); parent = *new; - if (master->of_node < this->of_node) + if (master->dev < this->dev) new = &((*new)->rb_left); - else if (master->of_node > this->of_node) + else if (master->dev > this->dev) new = &((*new)->rb_right); else return -EEXIST; @@ -824,18 +838,18 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu, struct arm_smmu_master *master; struct device_node *dev_node = dev_get_dev_node(dev); - master = find_smmu_master(smmu, dev_node); + master = find_smmu_master(smmu, dev); if (master) { dev_err(dev, "rejecting multiple registrations for master device %s\n", - dev_node->name); + dev_node ? dev_node->name : ""); return -EBUSY; } master = devm_kzalloc(dev, sizeof(*master), GFP_KERNEL); if (!master) return -ENOMEM; - master->of_node = dev_node; + master->dev = dev; /* Xen: Let Xen know that the device is protected by an SMMU */ device_set_protected(dev); @@ -845,7 +859,7 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu, (fwspec->ids[i] >= smmu->num_mapping_groups)) { dev_err(dev, "stream ID for master device %s greater than maximum allowed (%d)\n", - dev_node->name, smmu->num_mapping_groups); + dev_node ? dev_node->name : "", smmu->num_mapping_groups); return -ERANGE; } master->cfg.smendx[i] = INVALID_SMENDX; @@ -889,6 +903,15 @@ static int arm_smmu_dt_add_device_generic(u8 devfn, struct device *dev) if (smmu == NULL) return -ENXIO; +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + if ( devfn != pdev->devfn ) + return 0; + } +#endif + return arm_smmu_dt_add_device_legacy(smmu, dev, fwspec); } @@ -912,11 +935,10 @@ static struct arm_smmu_device *find_smmu_for_device(struct device *dev) { struct arm_smmu_device *smmu; struct arm_smmu_master *master = NULL; - struct device_node *dev_node = dev_get_dev_node(dev); spin_lock(&arm_smmu_devices_lock); list_for_each_entry(smmu, &arm_smmu_devices, list) { - master = find_smmu_master(smmu, dev_node); + master = find_smmu_master(smmu, dev); if (master) break; } @@ -2006,6 +2028,7 @@ static bool arm_smmu_capable(enum iommu_cap cap) } #endif +#if 0 /* Not used */ static int __arm_smmu_get_pci_sid(struct pci_dev *pdev, u16 alias, void *data) { *((u16 *)data) = alias; @@ -2016,6 +2039,7 @@ static void __arm_smmu_release_pci_iommudata(void *data) { kfree(data); } +#endif static int arm_smmu_add_device(struct device *dev) { @@ -2023,12 +2047,13 @@ static int arm_smmu_add_device(struct device *dev) struct arm_smmu_master_cfg *cfg; struct iommu_group *group; void (*releasefn)(void *) = NULL; - int ret; smmu = find_smmu_for_device(dev); if (!smmu) return -ENODEV; + /* There is no need to distinguish here, thanks to PCI-IOMMU DT bindings */ +#if 0 if (dev_is_pci(dev)) { struct pci_dev *pdev = to_pci_dev(dev); struct iommu_fwspec *fwspec; @@ -2053,10 +2078,12 @@ static int arm_smmu_add_device(struct device *dev) &fwspec->ids[0]); releasefn = __arm_smmu_release_pci_iommudata; cfg->smmu = smmu; - } else { + } else +#endif + { struct arm_smmu_master *master; - master = find_smmu_master(smmu, dev->of_node); + master = find_smmu_master(smmu, dev); if (!master) { return -ENODEV; } @@ -2724,6 +2751,27 @@ static int arm_smmu_assign_dev(struct domain *d, u8 devfn, return -ENOMEM; } +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) && !is_hardware_domain(d) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Assigning device %04x:%02x:%02x.%u to dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), PCI_FUNC(devfn), + d->domain_id); + + if ( devfn != pdev->devfn || pdev->domain == d ) + return 0; + + list_move(&pdev->domain_list, &d->pdev_list); + pdev->domain = d; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + if (!dev_iommu_group(dev)) { ret = arm_smmu_add_device(dev); if (ret) @@ -2773,11 +2821,29 @@ out: return ret; } -static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) +static int arm_smmu_deassign_dev(struct domain *d, u8 devfn, struct device *dev) { struct iommu_domain *domain = dev_iommu_domain(dev); struct arm_smmu_xen_domain *xen_domain; +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Deassigning device %04x:%02x:%02x.%u from dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), PCI_FUNC(devfn), + d->domain_id); + + if ( devfn != pdev->devfn ) + return 0; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + xen_domain = dom_iommu(d)->arch.priv; if (!domain || domain->priv->cfg.domain != d) { @@ -2805,13 +2871,13 @@ static int arm_smmu_reassign_dev(struct domain *s, struct domain *t, int ret = 0; /* Don't allow remapping on other domain than hwdom */ - if ( t && !is_hardware_domain(t) ) + if ( t && !is_hardware_domain(t) && t != dom_io ) return -EPERM; if (t == s) return 0; - ret = arm_smmu_deassign_dev(s, dev); + ret = arm_smmu_deassign_dev(s, devfn, dev); if (ret) return ret; From patchwork Thu May 11 19:16:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13238423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AF1C9C77B7F for ; Thu, 11 May 2023 19:22:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.533571.830371 (Exim 4.92) (envelope-from ) id 1pxBrz-0000Xh-Rw; Thu, 11 May 2023 19:22:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 533571.830371; Thu, 11 May 2023 19:22:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBrz-0000Xa-Oh; Thu, 11 May 2023 19:22:31 +0000 Received: by outflank-mailman (input) for mailman id 533571; Thu, 11 May 2023 19:22:31 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pxBrz-0008QA-DH for xen-devel@lists.xenproject.org; Thu, 11 May 2023 19:22:31 +0000 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on20631.outbound.protection.outlook.com [2a01:111:f400:7e89::631]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 2ad39f39-f031-11ed-8611-37d641c3527e; Thu, 11 May 2023 21:22:29 +0200 (CEST) Received: from DS7P222CA0011.NAMP222.PROD.OUTLOOK.COM (2603:10b6:8:2e::18) by DM6PR12MB4171.namprd12.prod.outlook.com (2603:10b6:5:21f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.22; Thu, 11 May 2023 19:22:24 +0000 Received: from DM6NAM11FT085.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2e:cafe::24) by DS7P222CA0011.outlook.office365.com (2603:10b6:8:2e::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.22 via Frontend Transport; Thu, 11 May 2023 19:22:24 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT085.mail.protection.outlook.com (10.13.172.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.20 via Frontend Transport; Thu, 11 May 2023 19:22:24 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 14:22:23 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 11 May 2023 14:22:22 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Thu, 11 May 2023 14:22:21 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2ad39f39-f031-11ed-8611-37d641c3527e ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bak56nci9K5jjkkIkXvkbiaY+3qyRk9Y/L9C3w3i5L8DqGHjZQqoLxpjtSs+9Z7wLfvCAinnXUnKiA30MxXnXwomtoxjIDGEkjCXKmOmkT/Ymxowvei3Yi9PsISYTALPuP3LTb+nh7MDPwc61xDRsZ8GwdytdJi/I2X9tzUwkB6rMzThDWNCVAiux5I39wKRtNG4tTDdI7UiU6M3z/Wu7Sifui/Y4VrL8D7Sbkq54AhY8bkzrlg4siiYR7wQoO0+piCQnXgY6AfhAPyjdsJj21QK65keRuZUfRAHUGnchdezRzeeU7TxPQqKNadNrIdUYsGFzTA5wfa2Fge9kcGFRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SO3THuJ+/ik7n2bM+aWq9Ulbj6rZ/FkssTS50Og6lnI=; b=DcVPho3m/knVFB70n+uJdsR+l1LRyMEVYqut78+7RKqzAGXvc4v1b9+EJifgHOzUcSoZkHS64HeYASE8jloHotNwv5o1STGH1/qDDpAdbTnnSHivsr5jHDiQ+2p3Umsd9aDQnemmnCOEluOP08s0mog6BqvWs0TVUVSv9LOaTomnM+I3llOzfMJSGHQAM3Zufyqf+QXMVKzftqFG71+fp/6M0r6J5ueQJUx9G/LEA9Y+vsmLInpsZB6NipiaTZk8m1dv070z0R7ONBBWLZdWTDIOQnGNFz+jwVs/5eHrEFZ2FEuH0pCd+64kgdhVneldA4YNSemYOtepZ8Q+rE74lg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SO3THuJ+/ik7n2bM+aWq9Ulbj6rZ/FkssTS50Og6lnI=; b=4A5tPR31dR/4f0o4ukSFqtuV8joZhJa17RwUd9cWru37kLlJaiZso5W7W6iP0wjin7oZBXCCyZcjYlHqgjziPI/zZG/SYFtZ+lwVez7VgBktz3euXhezzif1CX/eG84daUWlPafePXTY1+gS6vdNzQ6fYJySXaZuH/LBCLaOp/I= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Rahul Singh , Bertrand Marquis , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Stewart Hildebrand Subject: [PATCH v2 8/8] xen/arm: smmuv3: Add PCI devices support for SMMUv3 Date: Thu, 11 May 2023 15:16:54 -0400 Message-ID: <20230511191654.400720-9-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230511191654.400720-1-stewart.hildebrand@amd.com> References: <20230511191654.400720-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT085:EE_|DM6PR12MB4171:EE_ X-MS-Office365-Filtering-Correlation-Id: 686471a2-ae31-4dc7-22f2-08db52550cb2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3sG1t+oyvh+DFdbRQmSOotQNDk8aoyCD9Gi82mgtcjL/LvUeDP0PUS4nfl7gXyjug8wk987CNowXe/bLDix0H46cqeBNRI3i8s58zVy5REQ3S4KTGV74WCkltexXERhLTxqhCNxSWOeuu/1S2PypELaqI33L8qF7goba0/XkXIl5NK7g6FfB3oC7sg6yPXg+HuuRYCrPPPrpxRuofOBqbaqFsjVJY1sF8V+sZZWggSKhQuhpClxcwRSFt/HVLgptrGaLF0swGTFAWUOv+V7dg6k0umUyPPOPn+7zKozSNp0gzvG/IAJm2m0Ej9UNrDh6G7s9UjXEEWXl46EIUV4ky65F3MZ/y5SeWEBAC0AGIxy75aCGkRBku7yMICPW2fv7ya8KuDaBrzx+E1JnjialBJ2pY/aGGhLGl1btBB24Dwyn1d18JfVQo88K8AL+QY7xg1eX5RjAAwVfUtaIRTXh8yMD5N2WHhCxF8ztg0WR3+FHlmXIZJyaAXgMVnpkdX4kY2JR+g7JeHnoVwmmJFuaU4Q7B2zwXAl/BoOftglE0z/U1reN5rK/D46dlvlKLQ1rN58jHqzKgPbHmAFFmldHE8B92J66upaWuGuj/PJ83DLc6uVc+wfOLIwEndDgAgm8h+vzPn6JxN3fWaBP/Un49hB366dB8oB0orAztl7lmXeh5exnsU4pz7yquItV3pWzbgX9AdUa8uxMkSuZuZ+/IEgGL2sPBsEtrDf62wOoyRoyxuGgcXHj6C57fFprkZvFsYZdM7AsZ4UvDEzoOQJtcA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(39860400002)(346002)(376002)(396003)(136003)(451199021)(40470700004)(36840700001)(46966006)(36756003)(86362001)(54906003)(316002)(6916009)(4326008)(70586007)(70206006)(966005)(478600001)(6666004)(40480700001)(82310400005)(8676002)(8936002)(41300700001)(2906002)(5660300002)(44832011)(81166007)(82740400003)(356005)(186003)(26005)(1076003)(336012)(36860700001)(426003)(83380400001)(47076005)(2616005)(40460700003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2023 19:22:24.0974 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 686471a2-ae31-4dc7-22f2-08db52550cb2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT085.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4171 From: Rahul Singh Signed-off-by: Rahul Singh Signed-off-by: Stewart Hildebrand --- v1->v2: * ignore add_device/assign_device/reassign_device calls for phantom functions (i.e. devfn != pdev->devfn) downstream->v1: * rebase * move 2 replacements of s/dt_device_set_protected(dev_to_dt(dev))/device_set_protected(dev)/ from this commit to ("xen/arm: Move is_protected flag to struct device") so as to not break ability to bisect * adjust patch title (remove stray space) * arm_smmu_(de)assign_dev: return error instead of crashing system * remove arm_smmu_remove_device() stub * update condition in arm_smmu_reassign_dev * style fixup (cherry picked from commit 7ed6c3ab250d899fe6e893a514278e406a2893e8 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- This is a file imported from Linux with modifications for Xen. What should be the coding style used for Xen modifications? --- xen/drivers/passthrough/arm/smmu-v3.c | 67 +++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 3 deletions(-) diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c index 4b452e6fdd00..807cfe575345 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -1469,6 +1469,8 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid) } /* Forward declaration */ static struct arm_smmu_device *arm_smmu_get_by_dev(struct device *dev); +static int arm_smmu_assign_dev(struct domain *d, u8 devfn, + struct device *dev, u32 flag); static int arm_smmu_add_device(u8 devfn, struct device *dev) { @@ -1484,6 +1486,15 @@ static int arm_smmu_add_device(u8 devfn, struct device *dev) if (!smmu) return -ENODEV; +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + if ( devfn != pdev->devfn ) + return 0; + } +#endif + master = xzalloc(struct arm_smmu_master); if (!master) return -ENOMEM; @@ -1527,6 +1538,17 @@ static int arm_smmu_add_device(u8 devfn, struct device *dev) dev_info(dev, "Added master device (SMMUv3 %s StreamIds %u)\n", dev_name(fwspec->iommu_dev), fwspec->num_ids); +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + ret = arm_smmu_assign_dev(pdev->domain, devfn, dev, 0); + if (ret) + goto err_free_master; + } +#endif + return 0; err_free_master: @@ -2607,6 +2629,27 @@ static int arm_smmu_assign_dev(struct domain *d, u8 devfn, struct arm_smmu_domain *smmu_domain; struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv; +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) && !is_hardware_domain(d) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Assigning device %04x:%02x:%02x.%u to dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), + PCI_FUNC(devfn), d->domain_id); + + if ( devfn != pdev->devfn || pdev->domain == d ) + return 0; + + list_move(&pdev->domain_list, &d->pdev_list); + pdev->domain = d; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + spin_lock(&xen_domain->lock); /* @@ -2640,7 +2683,7 @@ out: return ret; } -static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) +static int arm_smmu_deassign_dev(struct domain *d, uint8_t devfn, struct device *dev) { struct iommu_domain *io_domain = arm_smmu_get_domain(d, dev); struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv; @@ -2652,6 +2695,24 @@ static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) return -ESRCH; } +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Deassigning device %04x:%02x:%02x.%u from dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), + PCI_FUNC(devfn), d->domain_id); + + if ( devfn != pdev->devfn ) + return 0; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + spin_lock(&xen_domain->lock); arm_smmu_detach_dev(master); @@ -2671,13 +2732,13 @@ static int arm_smmu_reassign_dev(struct domain *s, struct domain *t, int ret = 0; /* Don't allow remapping on other domain than hwdom */ - if ( t && !is_hardware_domain(t) ) + if ( t && !is_hardware_domain(t) && (t != dom_io) ) return -EPERM; if (t == s) return 0; - ret = arm_smmu_deassign_dev(s, dev); + ret = arm_smmu_deassign_dev(s, devfn, dev); if (ret) return ret;