From patchwork Wed Jun 7 03:02:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13269951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70F52C7EE2F for ; Wed, 7 Jun 2023 03:05:54 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.544291.849969 (Exim 4.92) (envelope-from ) id 1q6jUM-0002Os-TA; Wed, 07 Jun 2023 03:05:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 544291.849969; Wed, 07 Jun 2023 03:05:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jUM-0002Ol-Q9; Wed, 07 Jun 2023 03:05:34 +0000 Received: by outflank-mailman (input) for mailman id 544291; Wed, 07 Jun 2023 03:05:33 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jUL-00029M-3z for xen-devel@lists.xenproject.org; Wed, 07 Jun 2023 03:05:33 +0000 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20601.outbound.protection.outlook.com [2a01:111:f400:7eae::601]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 29760b33-04e0-11ee-b232-6b7b168915f2; Wed, 07 Jun 2023 05:05:32 +0200 (CEST) Received: from BYAPR07CA0090.namprd07.prod.outlook.com (2603:10b6:a03:12b::31) by MW4PR12MB5625.namprd12.prod.outlook.com (2603:10b6:303:168::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Wed, 7 Jun 2023 03:05:28 +0000 Received: from CO1NAM11FT003.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:12b:cafe::f6) by BYAPR07CA0090.outlook.office365.com (2603:10b6:a03:12b::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33 via Frontend Transport; Wed, 7 Jun 2023 03:05:27 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1NAM11FT003.mail.protection.outlook.com (10.13.175.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.36 via Frontend Transport; Wed, 7 Jun 2023 03:05:26 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 22:05:25 -0500 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 22:05:24 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 22:05:23 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 29760b33-04e0-11ee-b232-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=A31Zozbf4U0MlcxvxzzzVvE4IGw2EC3S72yZrehb3FaYhv4gPWixfRk56tPS8bry9AXH9wCagNWUoAZB/NqZDomr/0Z7B7Y8444YhVgtBTPC3EyIH/IPu8jUlkvwaUYW6rjJlMaiInqXDhPlf90g7ocGV5hxUQzMT2zSA25oL0uX+njVNNGVrtx+4L1nCtZQPKxKvHDh9mow7W9vMiZalrF+Btp4sD+AzIgKFwP6wyYyP5+WXxWEvZWS32FvkIZ23vS196JrCXFpOI3lUlUeOBKWKtqvMkzFKjmtaleKtyAe54qDttwGyWdMId8Kgr0bKQfJYlrMk5DuE+5/qSOZmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IR23MxhVQUAbCp/F/DH8joMby5YUmz7+v0X+yjQZ5Cw=; b=kuoGyO3tcw21UfhiSctHkgd4FEEsCYNVSLhgP/066BJ+AiyPbOYWshwbmiEsiRrQpPxpxejmwrNN7VfROqEpTeKg9HcL6q6WkgiYBeSI8OUwml7+XjwTG+5R0kxbOtFcdsfqvR62Nl1/uAbWKaw7txW6gFtJtVwqMCwrijjesTAKCaO8uz2TA51YjUq7Pz1vF4tlAVyx84LOs/n28bW75NzuHLvujwuNkDp3+5Dt9dMCkx4et+uuHLIHFkP75J7b0ZAatytcVpesp55YDOYHuWLIIDpAN0snN0OKdUL/2MvbLUEIJMj0aApuCPvKdqW7M5lEEbnbBGIbkOu6De1iMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IR23MxhVQUAbCp/F/DH8joMby5YUmz7+v0X+yjQZ5Cw=; b=19/bz30kSO09H3yJp76Ez4lWRUfoxYU5c9WhFWMunIwKRdh92zgNsqB3ZINgKpZOeoWfJb5YcDbTTRYn3jERqwoI6WOdodlwqLicnxRiAx0XZO6ZTDFW/NE1WJ4339yAjZe2gkDqCVxMeSnQquqtJyY6tOx5ua/x1P3kWtg+q70= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Rahul Singh , Bertrand Marquis , "Stewart Hildebrand" Subject: [PATCH v4 1/7] xen/arm: Improve readability of check for registered devices Date: Tue, 6 Jun 2023 23:02:14 -0400 Message-ID: <20230607030220.22698-2-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230607030220.22698-1-stewart.hildebrand@amd.com> References: <20230607030220.22698-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT003:EE_|MW4PR12MB5625:EE_ X-MS-Office365-Filtering-Correlation-Id: 098a4bd8-1043-4374-25a2-08db67040b4d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: k+jlx5Akjo+zGyu7iVck76upxSHq5rS8qkW0Me8T2NGylBVEkGXZsfmA8fB/doL7g51nABr0dTn/QZwFUdO0ea2PRBYY4siZX0zeyoyMmGNlBMYUyqTv8k1yPrpdnvTaRT7YMwuRqgdq/V9zQ54cyFU6vXukcBhU3B5wn2assI3wqFHJ1f+w1/dvQPZ9jpgMsbeMDPWgEewDeaiPzQ1MWMiAp792Zb00AayewCABbZzdfRbrST0UsN8xjGMir9+uiPQk4c08kpvKFWra1uLevQ35bUQOaK8YdeEMT4auaLTwc1Zqawk7AizHI55KkNfgXgQNaZsR2qBtOhohttagMPiq58g8hRtQMyEAejCW/+k7MC3kZ9mwE1xuAmtY/nHy+4QSwqqlA6FpuRhVWQkfdZ3zn3gLI7ARvwTmB+EA1E194uv9N1g6a+oxrdw/RHD3arQzIPte1FIg+1nJeL5zQADG0qJaS7tD6pO1e2+pJH9t27r9J15DOXdvC+B54jOImCGttQZ9sNz2br6/CQ2zPVK93z1ha3q+flrY6UPDiueIVFqwMDdBmEnnJ87K4D4FGGTvBmVAM/N1Y5+xFusMO0IEisPnO8yO48Wrm8SFPfrQ8/tFjON+CB9KN2X49DAyEoYLjJtami/YjaI6kgO4OQOKFSn6BvLJVusfncydIuc2uh9gF6D6OU2vdoDLMi2RyYt4Lwj4nOUkj8KdTVKGEhQ9SBxHyOzBw4UauLSYikVn2r/dnX/dGAZ33G46hVgAv7GyOgQZu3S2TW3HNjEizA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(396003)(39860400002)(136003)(376002)(346002)(451199021)(40470700004)(46966006)(36840700001)(36756003)(82310400005)(44832011)(5660300002)(86362001)(2906002)(40480700001)(47076005)(6666004)(186003)(426003)(336012)(36860700001)(83380400001)(1076003)(26005)(356005)(81166007)(82740400003)(478600001)(40460700003)(54906003)(2616005)(316002)(41300700001)(6916009)(70206006)(70586007)(4326008)(8676002)(8936002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 03:05:26.8630 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 098a4bd8-1043-4374-25a2-08db67040b4d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT003.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB5625 From: Oleksandr Tyshchenko Improve readability of check for devices already registered with the SMMU with legacy mmu-masters DT bindings by using is_protected. There are 2 device tree bindings for registering a device with the SMMU: * mmu-masters (legacy, SMMUv1/2 only) * iommus A device tree may include both mmu-masters and iommus properties (although it is unnecessary to do so). When a device appears in the mmu-masters list, np->is_protected and dev->iommu_fwspec both get set by the SMMUv1/2 driver. The function iommu_add_dt_device() is subsequently invoked for devices that have an iommus specification. The check as it was before this patch: if ( dev_iommu_fwspec_get(dev) ) return 0; and the new check: if ( dt_device_is_protected(np) ) return 0; are guarding against the same corner case: when a device has both mmu-masters and iommus specifications in the device tree. The is_protected naming is more descriptive. If np->is_protected is not set (i.e. false), but dev->iommu_fwspec is set, it is an error condition, so return an error in this case. Expand the comment to further clarify the corner case. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand --- v3->v4: * new patch: this change was split from ("xen/arm: Move is_protected flag to struct device") --- xen/drivers/passthrough/device_tree.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index 1c32d7b50cce..d9b63da7260a 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -141,12 +141,17 @@ int iommu_add_dt_device(struct dt_device_node *np) return -EINVAL; /* - * The device may already have been registered. As there is no harm in - * it just return success early. + * Devices that appear in the legacy mmu-masters list may have already been + * registered with the SMMU. In case a device has both a mmu-masters entry + * and iommus property, there is no need to register it again. In this case + * simply return success early. */ - if ( dev_iommu_fwspec_get(dev) ) + if ( dt_device_is_protected(np) ) return 0; + if ( dev_iommu_fwspec_get(dev) ) + return -EEXIST; + /* * According to the Documentation/devicetree/bindings/iommu/iommu.txt * from Linux. From patchwork Wed Jun 7 03:02:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13269952 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DB352C77B7A for ; Wed, 7 Jun 2023 03:06:10 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.544292.849980 (Exim 4.92) (envelope-from ) id 1q6jUl-0002tw-71; Wed, 07 Jun 2023 03:05:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 544292.849980; Wed, 07 Jun 2023 03:05:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jUl-0002tp-48; Wed, 07 Jun 2023 03:05:59 +0000 Received: by outflank-mailman (input) for mailman id 544292; Wed, 07 Jun 2023 03:05:57 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jUj-0002sk-Jw for xen-devel@lists.xenproject.org; Wed, 07 Jun 2023 03:05:57 +0000 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20601.outbound.protection.outlook.com [2a01:111:f400:7eae::601]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 36ad68e3-04e0-11ee-8611-37d641c3527e; Wed, 07 Jun 2023 05:05:53 +0200 (CEST) Received: from MW4PR04CA0115.namprd04.prod.outlook.com (2603:10b6:303:83::30) by MW4PR12MB7287.namprd12.prod.outlook.com (2603:10b6:303:22c::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Wed, 7 Jun 2023 03:05:48 +0000 Received: from CO1NAM11FT048.eop-nam11.prod.protection.outlook.com (2603:10b6:303:83:cafe::7a) by MW4PR04CA0115.outlook.office365.com (2603:10b6:303:83::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.19 via Frontend Transport; Wed, 7 Jun 2023 03:05:48 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1NAM11FT048.mail.protection.outlook.com (10.13.175.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.33 via Frontend Transport; Wed, 7 Jun 2023 03:05:47 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 22:05:46 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 22:05:44 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 36ad68e3-04e0-11ee-8611-37d641c3527e ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=A7G5pFcTCYze0cZ0bsUJjfSoUvROVE74ZSeu1ld7Mk7P7IVpeV4eGV90eQFF+BXc23Gh9ITPTI3rVpTLjWpSRzOiz3nEYUqVEwL2LAheTTmIlDUsOynwidH19d6cNen9cqWEZ9C9XbVixjQMWgt55VdVFRpysOgqTlvD8KcL1VRRhehIRNAVzIwpLzzbm+wOc7qKSmfSwArgnxxemb7QWV2S4Q2vXheRC5cHpN24iSXTfU33vd4Q7GfQy8DcSnuph71j3edHNRWrww7p6jEJz6sX/5cTZiQj2co2OrSh5qpZvbi0KzE7OnXol93kDLug++QBmHSdvR8IRPlU+67KDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0vhE+gJtWzDt5s54scLJj3hXZ6GdKzqeUWBduIkmLOg=; b=Fwi2v4W7YmbM+IdQS/NQwX6aXOrdj6aIYt9JG7w+8eVxDkmd7uhpBOItF9Ka5a6UbNsVMoyJGGi9smECU9GMxpQi6BrUgXY3TpFTLw3hfxuCOFOk46+NA/SrK6DulUu3128AAV/Qrr5w3AivxNs6FpZTe6EIhsQjeNdVMqnSeMyxVyV6btN5SE6tiDNJOkKa0VTzv6c0kOiKuENoUgfU4yj+zCLMi46eAdoewhOayzb628XuYt+qCkDJQpC8SOgnYklDkDeucmKoaM2wdWjNDETOE4El9Cy/e4Ce5jePYcjkT2uRBjIF9R1HPJIMDPQ92ZSSwo+cU48bNX2cfLM7kg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0vhE+gJtWzDt5s54scLJj3hXZ6GdKzqeUWBduIkmLOg=; b=E5k6akw0JUKf1h5KD5YvuEGF3CMw+fepVmVLmU7NgLFulBrs4GQQCQZ1JIhgsmmoFSREFDqACoZLQlO3NNIml5q0hhUY/1U6dFcIF6lVgGqldcbLVjRrycN3QLGdpWEYrV8pcFRI9MSn8oKm1AHqWvEm3z8BwK/5lGTMRp0wQEk= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Rahul Singh , Stewart Hildebrand Subject: [PATCH v4 2/7] xen/arm: Move is_protected flag to struct device Date: Tue, 6 Jun 2023 23:02:15 -0400 Message-ID: <20230607030220.22698-3-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230607030220.22698-1-stewart.hildebrand@amd.com> References: <20230607030220.22698-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT048:EE_|MW4PR12MB7287:EE_ X-MS-Office365-Filtering-Correlation-Id: d1d727e3-66da-4cff-3e5c-08db67041763 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: O6o+jMgG5hX5uAfbG0L3M3Osa2Qg7zKxevIeyF5p0+Gl5YFXCIfHYZksAJXfb7wR041l9+glNNj/8ATN8W4P64/jESdSoJnrvKFeaxwzXpOqWtMV3zTbM7sP/RY4nD2qqqiuDFJGJBDGH0SfwZDgkgGmg+6QyikLIwOUIGHHRWA0D0qb+FOG84347SgZNF9LvO2dmjnhFdOgzfqD74dZ9cZSDDDbkHBQzi7PZ79cvHfa9OumT0FFNFT+VLWH1EI7WYvfnMtxsl/CztHzHGsc/jkLEULCrSf/lytmzQRb5nBTovQa6Kt/XiLe8kW8U6Ary6sxD2HFJB7jifYdK/XiEPJaPWVExCJ5du/5pMAswaoK0naIOufHm52AhtHjC9tPYUX1DKP422Tg9mM/sk9Rtshdrg/jWTOqGupLS7lEwH/EAUK7m+9553d0mxLdT5n6TTsPxqNofCJYwzCVK+0eTJ76oovtqdIPIcYAnRSDC0Hd5lKcnFerdxz4uwpSxv00L8q7aWVytwxfnXi0dnnQvNQNWORgn4Dk0xnTjLsi41JP//y3+R9GpQwD1W1rZh2cRIrBhSenWVLPVW4vsKUMSE+9lj7WXI7qsA4MgelFZjuxvnsPxxdmNsufwrcYDjhw+DUsFSW8sB3IhxeSKxcIqlLwsPfa7evXVPcLbVhlpw0mLJi/mzIomwtLxtAYoFcncLum+RQP9N7FLohaQNSix/uf9QeVVHA7BYxQsszu2ydrnSNPpes1MCR5Nsq1HbSWEtVqONdIwrvq4oojUNQfCg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(136003)(396003)(39860400002)(376002)(346002)(451199021)(40470700004)(46966006)(36840700001)(81166007)(54906003)(478600001)(5660300002)(44832011)(8936002)(36756003)(2906002)(86362001)(8676002)(82310400005)(6916009)(70586007)(4326008)(70206006)(316002)(82740400003)(356005)(40480700001)(40460700003)(83380400001)(47076005)(41300700001)(1076003)(26005)(2616005)(186003)(426003)(966005)(336012)(36860700001)(6666004)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 03:05:47.1561 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d1d727e3-66da-4cff-3e5c-08db67041763 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT048.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7287 From: Oleksandr Tyshchenko This flag will be re-used for PCI devices by the subsequent patches. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand --- v3->v4: * move is_protected flag within struct device to reduce padding * re-add device_is_protected checks in add_device hooks in smmu-v3.c/ipmmu-vmsa.c * split mmu-masters check into separate patch v2->v3: * no change v1->v2: * no change downstream->v1: * rebase * s/dev_node->is_protected/dev_node->dev.is_protected/ in smmu.c * s/dt_device_set_protected(dev_to_dt(dev))/device_set_protected(dev)/ in smmu-v3.c * remove redundant device_is_protected checks in smmu-v3.c/ipmmu-vmsa.c (cherry picked from commit 59753aac77528a584d3950936b853ebf264b68e7 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/arch/arm/domain_build.c | 4 ++-- xen/arch/arm/include/asm/device.h | 14 ++++++++++++++ xen/common/device_tree.c | 2 +- xen/drivers/passthrough/arm/ipmmu-vmsa.c | 4 ++-- xen/drivers/passthrough/arm/smmu-v3.c | 5 +++-- xen/drivers/passthrough/arm/smmu.c | 2 +- xen/drivers/passthrough/device_tree.c | 8 ++++---- xen/include/xen/device_tree.h | 13 ------------- 8 files changed, 27 insertions(+), 25 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 3f4558ade67f..b229bfaae712 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -2524,7 +2524,7 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev, return res; } - if ( dt_device_is_protected(dev) ) + if ( device_is_protected(dt_to_dev(dev)) ) { dt_dprintk("%s setup iommu\n", dt_node_full_name(dev)); res = iommu_assign_dt_device(d, dev); @@ -3024,7 +3024,7 @@ static int __init handle_passthrough_prop(struct kernel_info *kinfo, return res; /* If xen_force, we allow assignment of devices without IOMMU protection. */ - if ( xen_force && !dt_device_is_protected(node) ) + if ( xen_force && !device_is_protected(dt_to_dev(node)) ) return 0; return iommu_assign_dt_device(kinfo->d, node); diff --git a/xen/arch/arm/include/asm/device.h b/xen/arch/arm/include/asm/device.h index b5d451e08776..8ac807482737 100644 --- a/xen/arch/arm/include/asm/device.h +++ b/xen/arch/arm/include/asm/device.h @@ -1,6 +1,8 @@ #ifndef __ASM_ARM_DEVICE_H #define __ASM_ARM_DEVICE_H +#include + enum device_type { DEV_DT, @@ -15,6 +17,8 @@ struct dev_archdata { struct device { enum device_type type; + bool is_protected; /* Shows that device is protected by IOMMU */ + uint8_t _pad[3]; #ifdef CONFIG_HAS_DEVICE_TREE struct dt_device_node *of_node; /* Used by drivers imported from Linux */ #endif @@ -94,6 +98,16 @@ int device_init(struct dt_device_node *dev, enum device_class class, */ enum device_class device_get_class(const struct dt_device_node *dev); +static inline void device_set_protected(struct device *device) +{ + device->is_protected = true; +} + +static inline bool device_is_protected(const struct device *device) +{ + return device->is_protected; +} + #define DT_DEVICE_START(_name, _namestr, _class) \ static const struct device_desc __dev_desc_##_name __used \ __section(".dev.info") = { \ diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c index 8da105291184..7444da3e0aa5 100644 --- a/xen/common/device_tree.c +++ b/xen/common/device_tree.c @@ -1910,7 +1910,7 @@ static unsigned long __init unflatten_dt_node(const void *fdt, /* By default dom0 owns the device */ np->used_by = 0; /* By default the device is not protected */ - np->is_protected = false; + np->dev.is_protected = false; INIT_LIST_HEAD(&np->domain_list); if ( new_format ) diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthrough/arm/ipmmu-vmsa.c index 611d9eeba5c3..a71fd76d89a3 100644 --- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -1288,14 +1288,14 @@ static int ipmmu_add_device(u8 devfn, struct device *dev) if ( !to_ipmmu(dev) ) return -ENODEV; - if ( dt_device_is_protected(dev_to_dt(dev)) ) + if ( device_is_protected(dev) ) { dev_err(dev, "Already added to IPMMU\n"); return -EEXIST; } /* Let Xen know that the master device is protected by an IOMMU. */ - dt_device_set_protected(dev_to_dt(dev)); + device_set_protected(dev); dev_info(dev, "Added master device (IPMMU %s micro-TLBs %u)\n", dev_name(fwspec->iommu_dev), fwspec->num_ids); diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c index 720aa69ff23e..8842db1ec07e 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -1521,13 +1521,14 @@ static int arm_smmu_add_device(u8 devfn, struct device *dev) */ arm_smmu_enable_pasid(master); - if (dt_device_is_protected(dev_to_dt(dev))) { + if ( device_is_protected(dev) ) + { dev_err(dev, "Already added to SMMUv3\n"); return -EEXIST; } /* Let Xen know that the master device is protected by an IOMMU. */ - dt_device_set_protected(dev_to_dt(dev)); + device_set_protected(dev); dev_info(dev, "Added master device (SMMUv3 %s StreamIds %u)\n", dev_name(fwspec->iommu_dev), fwspec->num_ids); diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c index c37fa9af1366..d874417958b5 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -837,7 +837,7 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu, master->of_node = dev_node; /* Xen: Let Xen know that the device is protected by an SMMU */ - dt_device_set_protected(dev_node); + device_set_protected(dev); for (i = 0; i < fwspec->num_ids; ++i) { if (!(smmu->features & ARM_SMMU_FEAT_STREAM_MATCH) && diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index d9b63da7260a..c60e78eaf556 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -34,7 +34,7 @@ int iommu_assign_dt_device(struct domain *d, struct dt_device_node *dev) if ( !is_iommu_enabled(d) ) return -EINVAL; - if ( !dt_device_is_protected(dev) ) + if ( !device_is_protected(dt_to_dev(dev)) ) return -EINVAL; spin_lock(&dtdevs_lock); @@ -65,7 +65,7 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev) if ( !is_iommu_enabled(d) ) return -EINVAL; - if ( !dt_device_is_protected(dev) ) + if ( !device_is_protected(dt_to_dev(dev)) ) return -EINVAL; spin_lock(&dtdevs_lock); @@ -87,7 +87,7 @@ static bool_t iommu_dt_device_is_assigned(const struct dt_device_node *dev) { bool_t assigned = 0; - if ( !dt_device_is_protected(dev) ) + if ( !device_is_protected(dt_to_dev(dev)) ) return 0; spin_lock(&dtdevs_lock); @@ -146,7 +146,7 @@ int iommu_add_dt_device(struct dt_device_node *np) * and iommus property, there is no need to register it again. In this case * simply return success early. */ - if ( dt_device_is_protected(np) ) + if ( device_is_protected(dev) ) return 0; if ( dev_iommu_fwspec_get(dev) ) diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h index c2eada748915..c2f315140560 100644 --- a/xen/include/xen/device_tree.h +++ b/xen/include/xen/device_tree.h @@ -90,9 +90,6 @@ struct dt_device_node { struct dt_device_node *next; /* TODO: Remove it. Only use to know the last children */ struct dt_device_node *allnext; - /* IOMMU specific fields */ - bool is_protected; - /* HACK: Remove this if there is a need of space */ bool_t static_evtchn_created; @@ -329,16 +326,6 @@ static inline domid_t dt_device_used_by(const struct dt_device_node *device) return device->used_by; } -static inline void dt_device_set_protected(struct dt_device_node *device) -{ - device->is_protected = true; -} - -static inline bool dt_device_is_protected(const struct dt_device_node *device) -{ - return device->is_protected; -} - static inline bool_t dt_property_name_is_equal(const struct dt_property *pp, const char *name) { From patchwork Wed Jun 7 03:02:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13269953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6E105C77B7A for ; Wed, 7 Jun 2023 03:06:39 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.544302.849990 (Exim 4.92) (envelope-from ) id 1q6jVE-0003bi-Jq; Wed, 07 Jun 2023 03:06:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 544302.849990; Wed, 07 Jun 2023 03:06:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jVE-0003bb-Gf; Wed, 07 Jun 2023 03:06:28 +0000 Received: by outflank-mailman (input) for mailman id 544302; Wed, 07 Jun 2023 03:06:27 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jVD-0002sk-7Q for xen-devel@lists.xenproject.org; Wed, 07 Jun 2023 03:06:27 +0000 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2062c.outbound.protection.outlook.com [2a01:111:f400:fe5a::62c]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 48f07ddc-04e0-11ee-8611-37d641c3527e; Wed, 07 Jun 2023 05:06:25 +0200 (CEST) Received: from BN9PR03CA0116.namprd03.prod.outlook.com (2603:10b6:408:fd::31) by SA1PR12MB8919.namprd12.prod.outlook.com (2603:10b6:806:38e::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.32; Wed, 7 Jun 2023 03:06:21 +0000 Received: from BN8NAM11FT066.eop-nam11.prod.protection.outlook.com (2603:10b6:408:fd:cafe::e9) by BN9PR03CA0116.outlook.office365.com (2603:10b6:408:fd::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.19 via Frontend Transport; Wed, 7 Jun 2023 03:06:21 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT066.mail.protection.outlook.com (10.13.177.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.36 via Frontend Transport; Wed, 7 Jun 2023 03:06:21 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 22:06:20 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 22:06:19 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 48f07ddc-04e0-11ee-8611-37d641c3527e ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xxow2M9MOd3Z2b8h5M5Axcohjie9xXdjE+JYfBRq51g1p14lgqA3zEB5p7w/vfnd3OIZw04/I0qGjCVTHUGK9hDwHl81Ex62qYNceYz70CV+xnan+ZzAlUg7soHBSt6iVz4091JpiOD0k+U7dHX48N6dJbbnC31VvJwPV66sZKtMfLEzH5wfu+xYxibKn64BStrSVfQzQ6fThRR3PDAvUG1a3/0a2w4VhZj5gsV92IUSxbbsZOYQi7be5O2/xVYeOsSfF4EyruWP2mUxxTKVMHIDgMCn/u5OEkQ4ZmNCLfj+gQKWp7r+lT8/Na1yux6N5cOUKHUB2XPwZXI1X2cMNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dCjhvhQeWLTKE7ucXbp+AwYN9GScAyicPVw/L8OSrEY=; b=JKHoGJLD8HUkdDIBldahwyia1XJetPhUU9GmPkMCx18RfIV+pvS+URABHzl3gfXjY0IVz1m8FX3nHOi1rjxXxJqZt7kxUZTURdjCaF1fK1BC+VBsBFj6cWO5hVP33FykUjYED2ZxdVJgUO+GZgFA8+h+grtSY2yyC/B2Sg4YWaPZhmZX4se5KZuNLzcMijK48oGZwl5ILOwVLfZ55fEMHeDku4xorW1gXDIIpu1dJHDr/yFbuZ/5LozhuQdlk1NytPTHCue3x1UEIMN/XtImSx0VX4xZXiMUf+jXeGqPYchke6Q1Oo8vOxrmtvQWYgyPh1FwvC3mVM44jjYPnZPbcw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dCjhvhQeWLTKE7ucXbp+AwYN9GScAyicPVw/L8OSrEY=; b=nSleHD2ycFd4+mL2Ff66xWgt7wvSf9ZhaC/nWE+Hp/mwu0OjXbwDvkeJBGM2dlIQpwJbvrtRxBPFJXv1k4yhVOnYerR4SXEeWU2go1Ed9suiasw33tQKe99ivz8XpBSYrgok1HlHqsYof8NBPBRQfYjR6CdjX+14vg/hZEl/up8= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Rahul Singh , Bertrand Marquis , "Stewart Hildebrand" Subject: [PATCH v4 3/7] iommu/arm: Add iommu_dt_xlate() Date: Tue, 6 Jun 2023 23:02:16 -0400 Message-ID: <20230607030220.22698-4-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230607030220.22698-1-stewart.hildebrand@amd.com> References: <20230607030220.22698-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT066:EE_|SA1PR12MB8919:EE_ X-MS-Office365-Filtering-Correlation-Id: 6a031bd7-e07a-435a-f8a6-08db67042ba1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kTYd6IvCdjHxaL/hRo+VRTS28aPrqQ6MplLR5RlMMlUNqk1nA7mocbSXE7U45BDn4gZ3iNACdqyg0dxJAGmOG6ET64TcKmU8ItaUvy/VHOuGdtopc0sjlsGmUUSnF9HknbA3uhCasoBJkq3xCXI8EAfr/qefevZZgBC2DrAnGuAmU2cr6AEGdYyc11HSg6LLO5h/NmQVpGDKnDRo2v3ue7rIgHMV+fR9G19iQjx9UpnuWGwrlQdWkerDoqPy76OuZgmtLkwywqDQhb6u8s6N9F0TRQB/yw+VZfYYxG9dxw2gPLyy0HFGG1fOyZwwcMC+E5fwf0HtHl8EODlEEFpsNcEQ7mNjQvV+arHLyp3YLgXYZ8TUSaruAVfxrS9ujeWyKbUrSDxJJHEDKaymxId1KU/Fisr80F2nOTWGtziFQ5tx+sglExPj+SFzrOJ3nMkPnLLBK95woqO4xDl5Lhi3AMV+0lpAJBotRYCe4oMHiaEhfo4ftR8rSY3QTwacE8CHGAsBpJ+GKSOXXSFg2Vqd69io+2jaiHzWck9QF+pjXXz2ElNW0Uv9CuXX5SESu5IJgyTY+pmfd/+l/j7Xym2MJgnGyNLjrw5bWWOlsLM5rWxwgaHqMd7PkG6ongTi5jyq6INXqaiLxmsd+JJ0Homo7VyvpMthBOZs/FcDKndPzf1dytI6U2gjbL7NrmySZsPTATfDW+0eBIvkBbot1NA8xYEtUH6CPrMbr31FBGw4pkf485TFUjIuzQN9Nb7Vvl7F X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(396003)(376002)(39860400002)(136003)(451199021)(46966006)(40470700004)(36840700001)(36756003)(54906003)(82310400005)(2906002)(40460700003)(478600001)(8936002)(86362001)(41300700001)(5660300002)(8676002)(82740400003)(316002)(44832011)(81166007)(356005)(4326008)(70206006)(6916009)(70586007)(83380400001)(40480700001)(47076005)(1076003)(26005)(2616005)(186003)(36860700001)(426003)(336012)(966005)(6666004)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 03:06:21.2153 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6a031bd7-e07a-435a-f8a6-08db67042ba1 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT066.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8919 From: Oleksandr Tyshchenko Move code for processing DT IOMMU specifier to a separate helper. This helper will be re-used for adding PCI devices by the subsequent patches as we will need exact the same actions for processing DT PCI-IOMMU specifier. While at it introduce NO_IOMMU to avoid magic "1". Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand --- v3->v4: * make dt_phandle_args *iommu_spec const * move !ops->add_device check to helper v2->v3: * no change v1->v2: * no change downstream->v1: * trivial rebase * s/dt_iommu_xlate/iommu_dt_xlate/ (cherry picked from commit c26bab0415ca303df86aba1d06ef8edc713734d3 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/drivers/passthrough/device_tree.c | 47 ++++++++++++++++++--------- 1 file changed, 31 insertions(+), 16 deletions(-) diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index c60e78eaf556..ff9e66ebf92a 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -127,15 +127,42 @@ int iommu_release_dt_devices(struct domain *d) return 0; } +/* This correlation must not be altered */ +#define NO_IOMMU 1 + +static int iommu_dt_xlate(struct device *dev, + const struct dt_phandle_args *iommu_spec) +{ + const struct iommu_ops *ops = iommu_get_ops(); + int rc; + + if ( !ops->dt_xlate ) + return -EINVAL; + + if ( !dt_device_is_available(iommu_spec->np) ) + return NO_IOMMU; + + rc = iommu_fwspec_init(dev, &iommu_spec->np->dev); + if ( rc ) + return rc; + + /* + * Provide DT IOMMU specifier which describes the IOMMU master + * interfaces of that device (device IDs, etc) to the driver. + * The driver is responsible to decide how to interpret them. + */ + return ops->dt_xlate(dev, iommu_spec); +} + int iommu_add_dt_device(struct dt_device_node *np) { const struct iommu_ops *ops = iommu_get_ops(); struct dt_phandle_args iommu_spec; struct device *dev = dt_to_dev(np); - int rc = 1, index = 0; + int rc = NO_IOMMU, index = 0; if ( !iommu_enabled ) - return 1; + return NO_IOMMU; if ( !ops ) return -EINVAL; @@ -163,22 +190,10 @@ int iommu_add_dt_device(struct dt_device_node *np) * The driver which supports generic IOMMU DT bindings must have * these callback implemented. */ - if ( !ops->add_device || !ops->dt_xlate ) + if ( !ops->add_device ) return -EINVAL; - if ( !dt_device_is_available(iommu_spec.np) ) - break; - - rc = iommu_fwspec_init(dev, &iommu_spec.np->dev); - if ( rc ) - break; - - /* - * Provide DT IOMMU specifier which describes the IOMMU master - * interfaces of that device (device IDs, etc) to the driver. - * The driver is responsible to decide how to interpret them. - */ - rc = ops->dt_xlate(dev, &iommu_spec); + rc = iommu_dt_xlate(dev, &iommu_spec); if ( rc ) break; From patchwork Wed Jun 7 03:02:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13269954 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 103FBC7EE2F for ; Wed, 7 Jun 2023 03:07:23 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.544311.850000 (Exim 4.92) (envelope-from ) id 1q6jVu-0004I3-Ut; Wed, 07 Jun 2023 03:07:10 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 544311.850000; Wed, 07 Jun 2023 03:07:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jVu-0004Hw-Q2; Wed, 07 Jun 2023 03:07:10 +0000 Received: by outflank-mailman (input) for mailman id 544311; Wed, 07 Jun 2023 03:07:10 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jVu-0002sk-0A for xen-devel@lists.xenproject.org; Wed, 07 Jun 2023 03:07:10 +0000 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2060a.outbound.protection.outlook.com [2a01:111:f400:7e88::60a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 6273928d-04e0-11ee-8611-37d641c3527e; Wed, 07 Jun 2023 05:07:07 +0200 (CEST) Received: from MW2PR2101CA0019.namprd21.prod.outlook.com (2603:10b6:302:1::32) by CYYPR12MB8732.namprd12.prod.outlook.com (2603:10b6:930:c8::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Wed, 7 Jun 2023 03:07:04 +0000 Received: from CO1NAM11FT065.eop-nam11.prod.protection.outlook.com (2603:10b6:302:1:cafe::5c) by MW2PR2101CA0019.outlook.office365.com (2603:10b6:302:1::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6500.7 via Frontend Transport; Wed, 7 Jun 2023 03:07:04 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1NAM11FT065.mail.protection.outlook.com (10.13.174.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.33 via Frontend Transport; Wed, 7 Jun 2023 03:07:03 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 22:07:03 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 22:07:01 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6273928d-04e0-11ee-8611-37d641c3527e ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Itoey9ysYWthzmBqtfMYlz9VQ1D3dqmu3PRXAZNsXVYNxXy2D6QJQIfzHDabyl7GhzX42YeRaHV4kYhrnQOYVjdCvKohlZwTZHFz5r16t0geZNagYycd/x9TpDgqGjdmJ09I6HScuoFzcHjCk176gy00/O0sm5MBA4C5KOx3v3Jl6nQym1kmi2fe0BRbt30MGmBz9ZLDKZ8GhFa0JLkI41CU6/lXD7PMqYsM54c7EXtV6TN7jiummfUqSYgfrZ+dyRMM2Ca9Z7Kp4etwgukOMH7HuxSUh/FCG2e5EuYY8r/Uu3e/IbAvhCiAQDvoFL/0YBir3S5Dubym77PcUJDM5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wJAdGtIHyvrVYngxULF6IRmPI2fTcN9vqrIymUQEzeM=; b=ZluNPUL9su66cGN8W+78Z6qR3zUPH8NF/v5pa/1bMfnhinQBV1XRVroCohHoJmy1iTi0QH4CDqIQ4V0phgLz/nnzZ/KBxl2oZBQ8tFZHXg1Se37QRnGTVTOGPRWe7Zso7PLCTuLlaUGQgFaDP/qcqhVwDJT0+PUWQnXbOXpCl3yQALPOi5S7FwNjg7cQc96g303j2t3mJMAP2VcbTWLLJ+yClkipKcCQKe3LxCKhGExJ0kW1RTpjydiM/a+LEWdNNoZP2ioMf7rllv/3dX9bQ3lt0s9exj/acfjkAQCfZnbpS9T4qynlKSgF/NKO8xMA1CUd+BmVsSkBA0PmDBz7zQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wJAdGtIHyvrVYngxULF6IRmPI2fTcN9vqrIymUQEzeM=; b=DTPLTwTfm6SkF8JFSOD+Y6CPTuBHgqB799Ibl2GCI68+LtioM53ZD2uYFP0tkamVFdFtlJp1hu5wbtn50BVVoM1FB/ukRK7iyCuWrAcC/bTRw7qeJuKVhkRIOMm2gDLeQjHWWco8CEo673JdShBLYP0jz4KsNBeAbTyOk8WxstU= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Jan Beulich , Paul Durrant , =?utf-8?q?Rog?= =?utf-8?q?er_Pau_Monn=C3=A9?= , Rahul Singh , Bertrand Marquis , "Stewart Hildebrand" Subject: [PATCH v4 4/7] iommu/arm: Introduce iommu_add_dt_pci_sideband_ids API Date: Tue, 6 Jun 2023 23:02:17 -0400 Message-ID: <20230607030220.22698-5-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230607030220.22698-1-stewart.hildebrand@amd.com> References: <20230607030220.22698-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT065:EE_|CYYPR12MB8732:EE_ X-MS-Office365-Filtering-Correlation-Id: 0362084f-5415-4f42-e466-08db67044508 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eYOrRG4OkiPBcBXgqmpyRu9TG1nhLfq0IyOwS/KyaSrHUL/MItTlPv9tdQo8ky/OzZoGek1wlvz0TOgqbcoazFr1FU8rSvQx9wrM3y+1HmW0QqQk97nZwyXoGKcjtApWIN0T4PTTHArKHq6taE4OaqSgIHsYRJFH9uzNdCdfF8X1eQn8HeV17tlmRNdHciSfCcsr4dtlEb0uPUCpdhFxVYT0MxYgYnV1lkeZJZx+3JBVO9RxVMRt0sfTQ+ZZjZinmx0VHUHNSVfPX90UvmNexucquunTfAbBf218IdLG3vXB15yzfHJ5oT57cUMW9iatEZ9tgAkf6aUJfQVtMIHYf7/Rpb0Pgept1uz6qVTRVkQUC4CfPhwmAg+KQ0cLfNR87SQQnV02zyarYGzmrjuY3RwXkAKB8XPEpiPcs0wSzmS5+7JX8mBan3sDhQIOMOHuQ37lSXN3gXfWk1rciHUbvslfF95zGfFllSP4EXNxNeFz+fE+phrm45F3a9gJ8OZn4jAuRrLQyXi13wHyJ3tlGmwhFYYVO1+ql0YugH9Q4Mr7XTAfXMD8fiazywWbQN6JOs/4wWBkBKLbOZOthsIc5IZisqEgDotZX76PnjKOuopHG+5cx1riq92zuZSeGUZsAELWTw6lxUiPc/k6dX/wsmxVHYKflNBNs5YpbMOqcZtOow7iO2T6aYbipRJgWXsvLIfyZ2Dy/lpxiCsTNs+eXVWBGb2gClNEygHmw/RPt+EqTw3Z2iWpkEbDXdBRGHJyaMRd8FCKe1ToN4lmUH6Ec/2lzJ0rRT3+ImbWxgEVCIA= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(136003)(396003)(376002)(346002)(39860400002)(451199021)(36840700001)(46966006)(40470700004)(8676002)(8936002)(478600001)(54906003)(966005)(6666004)(5660300002)(316002)(41300700001)(1076003)(26005)(6916009)(44832011)(186003)(4326008)(70206006)(70586007)(336012)(2616005)(83380400001)(40460700003)(2906002)(47076005)(426003)(82740400003)(81166007)(40480700001)(356005)(36860700001)(86362001)(82310400005)(36756003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 03:07:03.7391 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0362084f-5415-4f42-e466-08db67044508 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT065.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8732 From: Oleksandr Tyshchenko The main purpose of this patch is to add a way to register PCI device (which is behind the IOMMU) using the generic PCI-IOMMU DT bindings [1] before assigning that device to a domain. This behaves similarly to the existing iommu_add_dt_device API, except it handles PCI devices, and it is to be invoked from the add_device hook in the SMMU driver. The function of_map_id to translate an ID through a downstream mapping (which is also suitable for mapping Requester ID) was borrowed from Linux (v5.10-rc6) and updated according to the Xen code base. [1] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/pci-iommu.txt Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Stewart Hildebrand --- v3->v4: * wrap #include and if ( acpi_disabled ) in #ifdef CONFIG_ACPI * fix Michal's remarks about style, parenthesis, and print formats * remove !ops->dt_xlate check since it is already in iommu_dt_xlate helper * rename s/iommu_dt_pci_map_id/dt_map_id/ because it is generic, not specific to iommu * update commit description v2->v3: * new patch title (was: iommu/arm: Introduce iommu_add_dt_pci_device API) * renamed function from: iommu_add_dt_pci_device to: iommu_add_dt_pci_sideband_ids * removed stale ops->add_device check * iommu.h: add empty stub iommu_add_dt_pci_sideband_ids for !HAS_DEVICE_TREE * iommu.h: add iommu_add_pci_sideband_ids helper * iommu.h: don't wrap prototype in #ifdef CONFIG_HAS_PCI * s/iommu_fwspec_free(pci_to_dev(pdev))/iommu_fwspec_free(dev)/ v1->v2: * remove extra devfn parameter since pdev fully describes the device * remove ops->add_device() call from iommu_add_dt_pci_device(). Instead, rely on the existing iommu call in iommu_add_device(). * move the ops->add_device and ops->dt_xlate checks earlier downstream->v1: * rebase * add const qualifier to struct dt_device_node *np arg in dt_map_id() * add const qualifier to struct dt_device_node *np declaration in iommu_add_pci_device() * use stdint.h types instead of u8/u32/etc... * rename functions: s/dt_iommu_xlate/iommu_dt_xlate/ s/dt_map_id/iommu_dt_pci_map_id/ s/iommu_add_pci_device/iommu_add_dt_pci_device/ * add device_is_protected check in iommu_add_dt_pci_device * wrap prototypes in CONFIG_HAS_PCI (cherry picked from commit 734e3bf6ee77e7947667ab8fa96c25b349c2e1da from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- xen/drivers/passthrough/device_tree.c | 134 ++++++++++++++++++++++++++ xen/include/xen/device_tree.h | 25 +++++ xen/include/xen/iommu.h | 22 ++++- 3 files changed, 180 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index ff9e66ebf92a..bd0aed5df651 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -154,6 +154,140 @@ static int iommu_dt_xlate(struct device *dev, return ops->dt_xlate(dev, iommu_spec); } +#ifdef CONFIG_HAS_PCI +int dt_map_id(const struct dt_device_node *np, uint32_t id, + const char *map_name, const char *map_mask_name, + struct dt_device_node **target, uint32_t *id_out) +{ + uint32_t map_mask, masked_id, map_len; + const __be32 *map = NULL; + + if ( !np || !map_name || (!target && !id_out) ) + return -EINVAL; + + map = dt_get_property(np, map_name, &map_len); + if ( !map ) + { + if ( target ) + return -ENODEV; + + /* Otherwise, no map implies no translation */ + *id_out = id; + return 0; + } + + if ( !map_len || (map_len % (4 * sizeof(*map))) ) + { + printk(XENLOG_ERR "%s: Error: Bad %s length: %u\n", np->full_name, + map_name, map_len); + return -EINVAL; + } + + /* The default is to select all bits. */ + map_mask = 0xffffffff; + + /* + * Can be overridden by "{iommu,msi}-map-mask" property. + * If df_property_read_u32() fails, the default is used. + */ + if ( map_mask_name ) + dt_property_read_u32(np, map_mask_name, &map_mask); + + masked_id = map_mask & id; + for ( ; (int)map_len > 0; map_len -= 4 * sizeof(*map), map += 4 ) + { + struct dt_device_node *phandle_node; + uint32_t id_base = be32_to_cpup(map + 0); + uint32_t phandle = be32_to_cpup(map + 1); + uint32_t out_base = be32_to_cpup(map + 2); + uint32_t id_len = be32_to_cpup(map + 3); + + if ( id_base & ~map_mask ) + { + printk(XENLOG_ERR "%s: Invalid %s translation - %s-mask (0x%"PRIx32") ignores id-base (0x%"PRIx32")\n", + np->full_name, map_name, map_name, map_mask, id_base); + return -EFAULT; + } + + if ( (masked_id < id_base) || (masked_id >= (id_base + id_len)) ) + continue; + + phandle_node = dt_find_node_by_phandle(phandle); + if ( !phandle_node ) + return -ENODEV; + + if ( target ) + { + if ( !*target ) + *target = phandle_node; + + if ( *target != phandle_node ) + continue; + } + + if ( id_out ) + *id_out = masked_id - id_base + out_base; + + printk(XENLOG_DEBUG "%s: %s, using mask %08"PRIx32", id-base: %08"PRIx32", out-base: %08"PRIx32", length: %08"PRIx32", id: %08"PRIx32" -> %08"PRIx32"\n", + np->full_name, map_name, map_mask, id_base, out_base, id_len, id, + masked_id - id_base + out_base); + return 0; + } + + printk(XENLOG_ERR "%s: no %s translation for id 0x%"PRIx32" on %s\n", + np->full_name, map_name, id, (target && *target) ? (*target)->full_name : NULL); + + /* + * NOTE: Linux bypasses translation without returning an error here, + * but should we behave in the same way on Xen? Restrict for now. + */ + return -EFAULT; +} + +int iommu_add_dt_pci_sideband_ids(struct pci_dev *pdev) +{ + const struct iommu_ops *ops = iommu_get_ops(); + struct dt_phandle_args iommu_spec = { .args_count = 1 }; + struct device *dev = pci_to_dev(pdev); + const struct dt_device_node *np; + int rc = NO_IOMMU; + + if ( !iommu_enabled ) + return NO_IOMMU; + + if ( !ops ) + return -EINVAL; + + if ( device_is_protected(dev) ) + return 0; + + if ( dev_iommu_fwspec_get(dev) ) + return -EEXIST; + + np = pci_find_host_bridge_node(pdev); + if ( !np ) + return -ENODEV; + + /* + * According to the Documentation/devicetree/bindings/pci/pci-iommu.txt + * from Linux. + */ + rc = dt_map_id(np, PCI_BDF(pdev->bus, pdev->devfn), "iommu-map", + "iommu-map-mask", &iommu_spec.np, iommu_spec.args); + if ( rc ) + return (rc == -ENODEV) ? NO_IOMMU : rc; + + rc = iommu_dt_xlate(dev, &iommu_spec); + if ( rc < 0 ) + { + iommu_fwspec_free(dev); + return -EINVAL; + } + + return rc; +} +#endif /* CONFIG_HAS_PCI */ + int iommu_add_dt_device(struct dt_device_node *np) { const struct iommu_ops *ops = iommu_get_ops(); diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h index c2f315140560..8385cd538a58 100644 --- a/xen/include/xen/device_tree.h +++ b/xen/include/xen/device_tree.h @@ -892,6 +892,31 @@ int dt_count_phandle_with_args(const struct dt_device_node *np, */ int dt_get_pci_domain_nr(struct dt_device_node *node); +#ifdef CONFIG_HAS_PCI +/** + * dt_map_id - Translate an ID through a downstream mapping. + * @np: root complex device node. + * @id: device ID to map. + * @map_name: property name of the map to use. + * @map_mask_name: optional property name of the mask to use. + * @target: optional pointer to a target device node. + * @id_out: optional pointer to receive the translated ID. + * + * Given a device ID, look up the appropriate implementation-defined + * platform ID and/or the target device which receives transactions on that + * ID, as per the "iommu-map" and "msi-map" bindings. Either of @target or + * @id_out may be NULL if only the other is required. If @target points to + * a non-NULL device node pointer, only entries targeting that node will be + * matched; if it points to a NULL value, it will receive the device node of + * the first matching target phandle, with a reference held. + * + * Return: 0 on success or a standard error code on failure. + */ +int dt_map_id(const struct dt_device_node *np, uint32_t id, + const char *map_name, const char *map_mask_name, + struct dt_device_node **target, uint32_t *id_out); +#endif /* CONFIG_HAS_PCI */ + struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle); #ifdef CONFIG_DEVICE_TREE_DEBUG diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h index 405db59971c5..3cac177840f7 100644 --- a/xen/include/xen/iommu.h +++ b/xen/include/xen/iommu.h @@ -26,6 +26,9 @@ #include #include #include +#ifdef CONFIG_ACPI +#include +#endif #include TYPE_SAFE(uint64_t, dfn); @@ -219,7 +222,8 @@ int iommu_dt_domain_init(struct domain *d); int iommu_release_dt_devices(struct domain *d); /* - * Helper to add master device to the IOMMU using generic IOMMU DT bindings. + * Helpers to add master device to the IOMMU using generic (PCI-)IOMMU + * DT bindings. * * Return values: * 0 : device is protected by an IOMMU @@ -228,12 +232,28 @@ int iommu_release_dt_devices(struct domain *d); * (IOMMU is not enabled/present or device is not connected to it). */ int iommu_add_dt_device(struct dt_device_node *np); +int iommu_add_dt_pci_sideband_ids(struct pci_dev *pdev); int iommu_do_dt_domctl(struct xen_domctl *, struct domain *, XEN_GUEST_HANDLE_PARAM(xen_domctl_t)); +#else /* !HAS_DEVICE_TREE */ +static inline int iommu_add_dt_pci_sideband_ids(struct pci_dev *pdev) +{ + return 0; +} #endif /* HAS_DEVICE_TREE */ +static inline int iommu_add_pci_sideband_ids(struct pci_dev *pdev) +{ + int ret = 0; +#ifdef CONFIG_ACPI + if ( acpi_disabled ) +#endif + ret = iommu_add_dt_pci_sideband_ids(pdev); + return ret; +} + struct page_info; /* From patchwork Wed Jun 7 03:02:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13269955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5C803C7EE2F for ; Wed, 7 Jun 2023 03:07:44 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.544313.850010 (Exim 4.92) (envelope-from ) id 1q6jWK-0004nD-7F; Wed, 07 Jun 2023 03:07:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 544313.850010; Wed, 07 Jun 2023 03:07:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jWK-0004n6-3Z; Wed, 07 Jun 2023 03:07:36 +0000 Received: by outflank-mailman (input) for mailman id 544313; Wed, 07 Jun 2023 03:07:35 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jWI-0004mr-V1 for xen-devel@lists.xenproject.org; Wed, 07 Jun 2023 03:07:34 +0000 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2060b.outbound.protection.outlook.com [2a01:111:f400:7eaa::60b]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 71f46c71-04e0-11ee-b232-6b7b168915f2; Wed, 07 Jun 2023 05:07:33 +0200 (CEST) Received: from BYAPR07CA0022.namprd07.prod.outlook.com (2603:10b6:a02:bc::35) by SJ0PR12MB5421.namprd12.prod.outlook.com (2603:10b6:a03:3bb::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.32; Wed, 7 Jun 2023 03:07:30 +0000 Received: from CO1PEPF000042AA.namprd03.prod.outlook.com (2603:10b6:a02:bc:cafe::89) by BYAPR07CA0022.outlook.office365.com (2603:10b6:a02:bc::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.19 via Frontend Transport; Wed, 7 Jun 2023 03:07:30 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000042AA.mail.protection.outlook.com (10.167.243.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6477.13 via Frontend Transport; Wed, 7 Jun 2023 03:07:28 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 22:07:20 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 22:07:19 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 71f46c71-04e0-11ee-b232-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Fdf+zUekkZmRelrKbPqGS3pksN9fYa+PebeE8IB2GxW0UofxcWx8bNFyPod3wvwd04fI3JKjsHb8fEUTY0+A14qfT15NbfevHrYFWg03MXQjUFaO4zllPfppiSAurCeYI3ddel2lWC6pVAvT4XT10YG+xXjFqNNerywqeVNen8MAXPdhDQypSuH6cTQog0gbqyfKhkneSZCDWQIdwltlgB6IvT56IgHx+hsktDAMN2VCNTB3u4mRi+kxSQlCO5Yj9W4c24l+IscUFEz7xruELJ6EBsaDAsmlmDZUpRzULie3IxM3srR+0f8ew2RBQOeVZ1KX7bAwMdxeUBxj92FJpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cDZk7iF4AhnQAMe+ofo4jPmpjRfddhm3KfqYwi15ldw=; b=akKi5u0L1ZBcJh/ECRWclCyrZmYAx21f6ircuReLgd9fNt03Rjz/F0j533rpACp8N/k9PfRcXXXZzx00z/WGEyCp1JWfKWQYoUwwTD8qJ5PEJhhL4BLWGHADiDwthl+p3gX0qr8qusK5lw8qrsWMPKEz0m6PJAifCm6L2i8CQy7+vJ2ErKzpveA4fadR3E303bdptAHTSJN3anjXRRuUxy7mvHLDzYsG6R4McV5NWNu8/MuR0b41rPYnwPD8gCuj2hZbARd/a4jR+77Gpl0irHnW596yQyudPmYY9L/+JALqhsxduESHXcAQwO4YOPGFOj0PLAIrEkafOacbR1/XXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cDZk7iF4AhnQAMe+ofo4jPmpjRfddhm3KfqYwi15ldw=; b=cUBmYc05PXU/vL32juDZiTWvhlIzEoDVz3wVMPyQ6XtK7vt3wN/MqBlbzpqeSd8+7nNahzNVW94+P93SNdZrPNWKtODoLavysQ8AAAax00QesFIPUDkF8baJvOzpLl6VhL5/6qonCHIa/d3iL3PC81LnXFlDW/rQwDtj0P1LJVs= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Stewart Hildebrand , Stefano Stabellini , Julien Grall , Rahul Singh , Bertrand Marquis , "Jan Beulich" Subject: [PATCH v4 5/7] iommu/arm: iommu_add_dt_pci_sideband_ids phantom handling Date: Tue, 6 Jun 2023 23:02:18 -0400 Message-ID: <20230607030220.22698-6-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230607030220.22698-1-stewart.hildebrand@amd.com> References: <20230607030220.22698-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000042AA:EE_|SJ0PR12MB5421:EE_ X-MS-Office365-Filtering-Correlation-Id: ed1fadd7-69b4-44ef-2c03-08db6704540b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mc0mavjz+MTVAOWtsyJ+3KcXmZ/R5RF2rrtkjjHjftJcFP8BLxEYaj6VPVMp0ijrRAXHGaO7WccgwmdVCJjUsW8hPquZKRvOb0TRwZOUF/7LBOUEvrGvTFmExCHzpz4e+KQNdmZTV4xH3mDBMAmd8ZmxkHdUor4kkM7dmdEf33I0JnQpEHutfcofJeYcV3lHXbGA0yRUtPKJ3CUdS1mprNgU6/64pbKMLFzdmwQfxA5cNoHyDmpN+vqEKWolcs8NF9GR768G26kXcnwVwcALWlDhkS/WHGWvzwimERHC+wdEb8nUvI2NPNIuD4iTfdcPJ9jL1AZ4qY4YNV0AvKSv+XRMc1fCL7BO3FARmL72egQ4Nu1aLAPWmBy43dHQ3FZDbeoAdvim1C0yz5XtQqjUhMeO3KCPGcnXv1tkJ5fGAGwMuuC2Z2nbqOtqrKU9RJ7r4l2n/fywGGAiVqOmcx9gP5LT6KDT7JqxyG3ymf0hVi51Xx8h5x8ImcwNKs95BmphRyk7GUgFqr/g8deox02aNDbnS1PGaCKFouEcApNaN/47wB40arjCAlr5dbH4jPqGHSr8xkM65WgyKRSjL07NO/Oq6jo60hNxTtIs1VH1GZ2HZug5PFbPhxyPH4Lx9kdsT38y1F4GVfvNIZbnTFib/XRQPjB7hHOvwsgNt19zHxHOGOpVx3LgMg1MWYSsuHkCpRDih3KWWRb4WtcFuKVZc0sMEczi5JYO2FU8V1B5hXJoVCBROgJWu4b/yHbL7mI/ZzlTwg7w8pC/JcRLIHCaQA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(136003)(39860400002)(346002)(396003)(376002)(451199021)(36840700001)(46966006)(40470700004)(54906003)(40460700003)(478600001)(40480700001)(8936002)(5660300002)(44832011)(8676002)(36756003)(86362001)(2906002)(70206006)(6916009)(81166007)(82740400003)(70586007)(316002)(82310400005)(356005)(4326008)(41300700001)(2616005)(1076003)(36860700001)(47076005)(26005)(966005)(83380400001)(186003)(336012)(426003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 03:07:28.9043 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ed1fadd7-69b4-44ef-2c03-08db6704540b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000042AA.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB5421 Handle phantom functions in iommu_add_dt_pci_sideband_ids(). Each phantom function will have a unique requestor ID (RID)/BDF. On ARM, we need to map/translate the RID/BDF to an AXI stream ID for each phantom function according to the pci-iommu device tree mapping [1]. The RID/BDF -> AXI stream ID mapping in DT could allow phantom devices (i.e. devices with phantom functions) to use different AXI stream IDs based on the (phantom) function. [1] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/pci-iommu.txt Signed-off-by: Stewart Hildebrand --- v3->v4: * s/iommu_dt_pci_map_id/dt_map_id/ v2->v3: * new patch title (was: iommu/arm: iommu_add_dt_pci_device phantom handling) * rework loop to reduce duplication * s/iommu_fwspec_free(pci_to_dev(pdev))/iommu_fwspec_free(dev)/ v1->v2: * new patch --- xen/drivers/passthrough/device_tree.c | 33 ++++++++++++++++----------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index bd0aed5df651..b7de0175ec7e 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -251,6 +251,7 @@ int iommu_add_dt_pci_sideband_ids(struct pci_dev *pdev) struct device *dev = pci_to_dev(pdev); const struct dt_device_node *np; int rc = NO_IOMMU; + unsigned int devfn = pdev->devfn; if ( !iommu_enabled ) return NO_IOMMU; @@ -268,21 +269,27 @@ int iommu_add_dt_pci_sideband_ids(struct pci_dev *pdev) if ( !np ) return -ENODEV; - /* - * According to the Documentation/devicetree/bindings/pci/pci-iommu.txt - * from Linux. - */ - rc = dt_map_id(np, PCI_BDF(pdev->bus, pdev->devfn), "iommu-map", - "iommu-map-mask", &iommu_spec.np, iommu_spec.args); - if ( rc ) - return (rc == -ENODEV) ? NO_IOMMU : rc; + do { + /* + * According to the Documentation/devicetree/bindings/pci/pci-iommu.txt + * from Linux. + */ + rc = dt_map_id(np, PCI_BDF(pdev->bus, devfn), "iommu-map", + "iommu-map-mask", &iommu_spec.np, iommu_spec.args); + if ( rc ) + return (rc == -ENODEV) ? NO_IOMMU : rc; - rc = iommu_dt_xlate(dev, &iommu_spec); - if ( rc < 0 ) - { - iommu_fwspec_free(dev); - return -EINVAL; + rc = iommu_dt_xlate(dev, &iommu_spec); + if ( rc < 0 ) + { + iommu_fwspec_free(dev); + return -EINVAL; + } + + devfn += pdev->phantom_stride; } + while ( (devfn != pdev->devfn) && + (PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn)) ); return rc; } From patchwork Wed Jun 7 03:02:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13269956 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 61D70C77B7A for ; Wed, 7 Jun 2023 03:08:02 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.544319.850019 (Exim 4.92) (envelope-from ) id 1q6jWb-0005IP-LH; Wed, 07 Jun 2023 03:07:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 544319.850019; Wed, 07 Jun 2023 03:07:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jWb-0005II-If; Wed, 07 Jun 2023 03:07:53 +0000 Received: by outflank-mailman (input) for mailman id 544319; Wed, 07 Jun 2023 03:07:52 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jWa-0004mr-7R for xen-devel@lists.xenproject.org; Wed, 07 Jun 2023 03:07:52 +0000 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on20608.outbound.protection.outlook.com [2a01:111:f400:7eaa::608]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 7c7edec4-04e0-11ee-b232-6b7b168915f2; Wed, 07 Jun 2023 05:07:51 +0200 (CEST) Received: from MW4P223CA0029.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:80::34) by DM6PR12MB4188.namprd12.prod.outlook.com (2603:10b6:5:215::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Wed, 7 Jun 2023 03:07:48 +0000 Received: from CO1NAM11FT069.eop-nam11.prod.protection.outlook.com (2603:10b6:303:80:cafe::4f) by MW4P223CA0029.outlook.office365.com (2603:10b6:303:80::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33 via Frontend Transport; Wed, 7 Jun 2023 03:07:48 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1NAM11FT069.mail.protection.outlook.com (10.13.174.129) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.37 via Frontend Transport; Wed, 7 Jun 2023 03:07:47 +0000 Received: from SATLEXMB07.amd.com (10.181.41.45) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 22:07:46 -0500 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB07.amd.com (10.181.41.45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 20:07:46 -0700 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 22:07:44 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 7c7edec4-04e0-11ee-b232-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VX2I7YGOqbtsbXzDXz/xl4yLpE3DLW5h5EVLkIggwXtSmHIWfNGNqOWWZjAR0mLD+BzuaZSFXJFNUkCqVrfsrKCwgC90452oeiV8FCkrEfDA0bX9HnNZ+cXXjJ9c8qPLMxpK4UCFL5Ixpv4j1h2aCKk73BUVWJilKqBJjSVvTHd3uF5Ak65GIBdaSniH8o4ZqGcKRmT4hstYiQYCXNhSdrGVYelkO+3IkFqeIou6Q0aCEQ1EgOghAgIyQVJnAL33tnrTr1ncKwZbwSWjvCvjXtNi/4QLExNOyNBC28ODdthoKtgHgz1LIUoWalnGGYquQh1twlbHzNKGoCoW0LrDkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1NwxgsRLXVRgyixeYcAGGIjd7v3ZGkKgyTYitjgcDg4=; b=hZ2k0Vw9X1hXM1wrhePy4GCOPvggWXv6/w57PEP4KjZxPjdheGd4mwEx9jk2+Q6dDqK2WEdAPxrNbNoOkgee3jcmZiy71StTP1xabjpLsptQkKkOIyqVLe0+Fw9FW1Hgs+XuEjrT+5W1et03X29Iv3qVrLvjZc6/bJvg5FOOCZJKOb+OAvk8ISAdmdLZOcPtKd6WTR46GLRm7dQgfWLf4C1eisYS9mg7T74HTgv/9xLW7xBW+M/6FkIfvW7Dj7VkukaCCAx7eoPxsSiEYaFtBsCZzyCrQPvPLQzzk6kOceRAtWiQaWMy4slLe4M+ty/Vgyg5ypMgoSudlZybOE292w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1NwxgsRLXVRgyixeYcAGGIjd7v3ZGkKgyTYitjgcDg4=; b=jyrc7rRWOpJJdQFK+pchTu6H/vPzc9auD7tRPK2AIIKGtleUiPa+2UhAT9hXWe+SD+aP+nTMcf4tstn+JXJTCzzAHx3AfY9U/I7q1j0Vqohs2erdDwCqX5mRT5OPAzyOzg1vEclqGnfgcRVW9AAPyIAcaFgHlIaChYrsAOd0/oM= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Stewart Hildebrand To: CC: Oleksandr Andrushchenko , Julien Grall , Rahul Singh , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk , Oleksandr Tyshchenko , Stewart Hildebrand Subject: [PATCH v4 6/7] xen/arm: smmuv2: Add PCI devices support for SMMUv2 Date: Tue, 6 Jun 2023 23:02:19 -0400 Message-ID: <20230607030220.22698-7-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230607030220.22698-1-stewart.hildebrand@amd.com> References: <20230607030220.22698-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT069:EE_|DM6PR12MB4188:EE_ X-MS-Office365-Filtering-Correlation-Id: 8697a878-05b0-470a-18c8-08db67045f30 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8izQtLXcoNj0GTZSOUcmBnwjpZWTCArrv1Vz4My8n5vtJCCgxnih5scLWyNzLOmwVBVGDH/4B1O/COK+DPAOSKNvxr2/zt9JiGJ1dm8RreC8eTtLxq5Y6ou1zWVqlkZVu4KZ8LYSnWiDy5eITt+hFJjAneNJsbCtsjcaKbX/pEmYtiN7fUazqDSr+dWZjyPHStxk/PoMv/fvvapLhF/iTrOia7sguE4rFfEHC1LIdwuEsuuMFW1Jzx0kBUb/2LHM1jM16Z6hfOStVDinEASaNP7kzhObLxIHY2XtMlswS+aQg6JWUxDLhg/EDI9fMO2Db35qSV0yLXdoe1+DNpIzftwC9zdFA6eRz57YhI1BEPAkiJ236+vGKdzCFj5rC8kWv1P+O2ZQLyr5GHo2XDCYD3oJPriVhHyZJuJpcKzLCk02ijYWRoj/nqc+Gw1p732IvYyVjAUjnboqCz5IDxvZkz3nXvEnbmn84o/NYDstDMkGpff915Jwx3wDRLwT2XBPC3f7mnmEUfltPczkJH93ZNCehT+Oj0RSvx07ZSKjHCKESq6MJsBejufg9l+lyKXMhbgJA8w9krAUZGaiW41IQ51hqGDGOmyOk9Lbc9f2inUtAQCuW0+zkcNdos0IYodKEy0pW3qPFTjQz3D1TPsMDctpNWHQsfVseIdA5ds1DUWjEc+0A0+H0OHmKXw7eBCOgja6XevEGWrpZ+zp5QxPoBx2b1AbuH1BWp8M5n2zyB71u8Q8hbq2BfXOBiw34PHssyW3to/1DWT3bdVhyfdswuNmXWnV7OlFKlWbzFzCBwY= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(396003)(376002)(136003)(39860400002)(451199021)(40470700004)(36840700001)(46966006)(40460700003)(966005)(6666004)(83380400001)(36860700001)(186003)(336012)(36756003)(426003)(2616005)(86362001)(82310400005)(81166007)(356005)(47076005)(82740400003)(1076003)(26005)(40480700001)(316002)(54906003)(70586007)(70206006)(8676002)(8936002)(6916009)(4326008)(5660300002)(44832011)(41300700001)(2906002)(478600001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 03:07:47.6168 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8697a878-05b0-470a-18c8-08db67045f30 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT069.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4188 From: Oleksandr Andrushchenko Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Stewart Hildebrand --- v3->v4: * add new device_is_protected check in add_device hook to match SMMUv3 and IPMMU-VMSA drivers v2->v3: * invoke iommu_add_pci_sideband_ids() from add_device hook v1->v2: * ignore add_device/assign_device/reassign_device calls for phantom functions (i.e. devfn != pdev->devfn) downstream->v1: * wrap unused function in #ifdef 0 * remove the remove_device() stub since it was submitted separately to the list [XEN][PATCH v6 12/19] xen/smmu: Add remove_device callback for smmu_iommu ops https://lists.xenproject.org/archives/html/xen-devel/2023-05/msg00204.html * arm_smmu_(de)assign_dev: return error instead of crashing system * update condition in arm_smmu_reassign_dev * style fixup * add && !is_hardware_domain(d) into condition in arm_smmu_assign_dev() (cherry picked from commit 0c11a7f65f044c26d87d1e27ac6283ef1f9cfb7a from the downstream branch spider-master from https://github.com/xen-troops/xen.git) --- This is a file imported from Linux with modifications for Xen. What should be the coding style for Xen modifications? --- xen/drivers/passthrough/arm/smmu.c | 120 ++++++++++++++++++++++++----- 1 file changed, 99 insertions(+), 21 deletions(-) diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c index d874417958b5..edb5345fb1cd 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -134,8 +134,20 @@ typedef enum irqreturn irqreturn_t; /* Device logger functions * TODO: Handle PCI */ -#define dev_print(dev, lvl, fmt, ...) \ - printk(lvl "smmu: %s: " fmt, dt_node_full_name(dev_to_dt(dev)), ## __VA_ARGS__) +#ifndef CONFIG_HAS_PCI +#define dev_print(dev, lvl, fmt, ...) \ + printk(lvl "smmu: %s: " fmt, dev_name(dev), ## __VA_ARGS__) +#else +#define dev_print(dev, lvl, fmt, ...) ({ \ + if ( !dev_is_pci((dev)) ) \ + printk(lvl "smmu: %s: " fmt, dev_name((dev)), ## __VA_ARGS__); \ + else \ + { \ + struct pci_dev *pdev = dev_to_pci((dev)); \ + printk(lvl "smmu: %pp: " fmt, &pdev->sbdf, ## __VA_ARGS__); \ + } \ +}) +#endif #define dev_dbg(dev, fmt, ...) dev_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__) #define dev_notice(dev, fmt, ...) dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__) @@ -187,6 +199,7 @@ static void __iomem *devm_ioremap_resource(struct device *dev, * Xen: PCI functions * TODO: It should be implemented when PCI will be supported */ +#if 0 /* unused */ #define to_pci_dev(dev) (NULL) static inline int pci_for_each_dma_alias(struct pci_dev *pdev, int (*fn) (struct pci_dev *pdev, @@ -196,6 +209,7 @@ static inline int pci_for_each_dma_alias(struct pci_dev *pdev, BUG(); return 0; } +#endif /* Xen: misc */ #define PHYS_MASK_SHIFT PADDR_BITS @@ -631,7 +645,7 @@ struct arm_smmu_master_cfg { for (i = 0; idx = cfg->smendx[i], i < num; ++i) struct arm_smmu_master { - struct device_node *of_node; + struct device *dev; struct rb_node node; struct arm_smmu_master_cfg cfg; }; @@ -723,7 +737,7 @@ arm_smmu_get_fwspec(struct arm_smmu_master_cfg *cfg) { struct arm_smmu_master *master = container_of(cfg, struct arm_smmu_master, cfg); - return dev_iommu_fwspec_get(&master->of_node->dev); + return dev_iommu_fwspec_get(master->dev); } static void parse_driver_options(struct arm_smmu_device *smmu) @@ -756,7 +770,7 @@ static struct device_node *dev_get_dev_node(struct device *dev) } static struct arm_smmu_master *find_smmu_master(struct arm_smmu_device *smmu, - struct device_node *dev_node) + struct device *dev) { struct rb_node *node = smmu->masters.rb_node; @@ -765,9 +779,9 @@ static struct arm_smmu_master *find_smmu_master(struct arm_smmu_device *smmu, master = container_of(node, struct arm_smmu_master, node); - if (dev_node < master->of_node) + if (dev < master->dev) node = node->rb_left; - else if (dev_node > master->of_node) + else if (dev > master->dev) node = node->rb_right; else return master; @@ -802,9 +816,9 @@ static int insert_smmu_master(struct arm_smmu_device *smmu, = container_of(*new, struct arm_smmu_master, node); parent = *new; - if (master->of_node < this->of_node) + if (master->dev < this->dev) new = &((*new)->rb_left); - else if (master->of_node > this->of_node) + else if (master->dev > this->dev) new = &((*new)->rb_right); else return -EEXIST; @@ -823,18 +837,24 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu, struct arm_smmu_master *master; struct device_node *dev_node = dev_get_dev_node(dev); - master = find_smmu_master(smmu, dev_node); + master = find_smmu_master(smmu, dev); if (master) { dev_err(dev, "rejecting multiple registrations for master device %s\n", - dev_node->name); + dev_node ? dev_node->name : ""); return -EBUSY; } master = devm_kzalloc(dev, sizeof(*master), GFP_KERNEL); if (!master) return -ENOMEM; - master->of_node = dev_node; + master->dev = dev; + + if ( device_is_protected(dev) ) + { + dev_err(dev, "Already added to SMMU\n"); + return -EEXIST; + } /* Xen: Let Xen know that the device is protected by an SMMU */ device_set_protected(dev); @@ -844,7 +864,7 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu, (fwspec->ids[i] >= smmu->num_mapping_groups)) { dev_err(dev, "stream ID for master device %s greater than maximum allowed (%d)\n", - dev_node->name, smmu->num_mapping_groups); + dev_node ? dev_node->name : "", smmu->num_mapping_groups); return -ERANGE; } master->cfg.smendx[i] = INVALID_SMENDX; @@ -880,6 +900,21 @@ static int arm_smmu_dt_add_device_generic(u8 devfn, struct device *dev) struct arm_smmu_device *smmu; struct iommu_fwspec *fwspec; +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + int ret; + + if ( devfn != pdev->devfn ) + return 0; + + ret = iommu_add_pci_sideband_ids(pdev); + if ( ret < 0 ) + iommu_fwspec_free(dev); + } +#endif + fwspec = dev_iommu_fwspec_get(dev); if (fwspec == NULL) return -ENXIO; @@ -911,11 +946,10 @@ static struct arm_smmu_device *find_smmu_for_device(struct device *dev) { struct arm_smmu_device *smmu; struct arm_smmu_master *master = NULL; - struct device_node *dev_node = dev_get_dev_node(dev); spin_lock(&arm_smmu_devices_lock); list_for_each_entry(smmu, &arm_smmu_devices, list) { - master = find_smmu_master(smmu, dev_node); + master = find_smmu_master(smmu, dev); if (master) break; } @@ -2007,6 +2041,7 @@ static bool arm_smmu_capable(enum iommu_cap cap) } #endif +#if 0 /* Not used */ static int __arm_smmu_get_pci_sid(struct pci_dev *pdev, u16 alias, void *data) { *((u16 *)data) = alias; @@ -2017,6 +2052,7 @@ static void __arm_smmu_release_pci_iommudata(void *data) { kfree(data); } +#endif static int arm_smmu_add_device(struct device *dev) { @@ -2024,12 +2060,13 @@ static int arm_smmu_add_device(struct device *dev) struct arm_smmu_master_cfg *cfg; struct iommu_group *group; void (*releasefn)(void *) = NULL; - int ret; smmu = find_smmu_for_device(dev); if (!smmu) return -ENODEV; + /* There is no need to distinguish here, thanks to PCI-IOMMU DT bindings */ +#if 0 if (dev_is_pci(dev)) { struct pci_dev *pdev = to_pci_dev(dev); struct iommu_fwspec *fwspec; @@ -2054,10 +2091,12 @@ static int arm_smmu_add_device(struct device *dev) &fwspec->ids[0]); releasefn = __arm_smmu_release_pci_iommudata; cfg->smmu = smmu; - } else { + } else +#endif + { struct arm_smmu_master *master; - master = find_smmu_master(smmu, dev->of_node); + master = find_smmu_master(smmu, dev); if (!master) { return -ENODEV; } @@ -2725,6 +2764,27 @@ static int arm_smmu_assign_dev(struct domain *d, u8 devfn, return -ENOMEM; } +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) && !is_hardware_domain(d) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Assigning device %04x:%02x:%02x.%u to dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), PCI_FUNC(devfn), + d->domain_id); + + if ( devfn != pdev->devfn || pdev->domain == d ) + return 0; + + list_move(&pdev->domain_list, &d->pdev_list); + pdev->domain = d; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + if (!dev_iommu_group(dev)) { ret = arm_smmu_add_device(dev); if (ret) @@ -2774,11 +2834,29 @@ out: return ret; } -static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) +static int arm_smmu_deassign_dev(struct domain *d, u8 devfn, struct device *dev) { struct iommu_domain *domain = dev_iommu_domain(dev); struct arm_smmu_xen_domain *xen_domain; +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Deassigning device %04x:%02x:%02x.%u from dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), PCI_FUNC(devfn), + d->domain_id); + + if ( devfn != pdev->devfn ) + return 0; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + xen_domain = dom_iommu(d)->arch.priv; if (!domain || domain->priv->cfg.domain != d) { @@ -2806,13 +2884,13 @@ static int arm_smmu_reassign_dev(struct domain *s, struct domain *t, int ret = 0; /* Don't allow remapping on other domain than hwdom */ - if ( t && !is_hardware_domain(t) ) + if ( t && !is_hardware_domain(t) && t != dom_io ) return -EPERM; if (t == s) return 0; - ret = arm_smmu_deassign_dev(s, dev); + ret = arm_smmu_deassign_dev(s, devfn, dev); if (ret) return ret; From patchwork Wed Jun 7 03:02:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13269957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D086CC77B7A for ; Wed, 7 Jun 2023 03:08:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.544326.850030 (Exim 4.92) (envelope-from ) id 1q6jX5-0005wS-07; Wed, 07 Jun 2023 03:08:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 544326.850030; Wed, 07 Jun 2023 03:08:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jX4-0005wJ-T2; Wed, 07 Jun 2023 03:08:22 +0000 Received: by outflank-mailman (input) for mailman id 544326; Wed, 07 Jun 2023 03:08:21 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q6jX3-0005Dg-JF for xen-devel@lists.xenproject.org; Wed, 07 Jun 2023 03:08:21 +0000 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on20615.outbound.protection.outlook.com [2a01:111:f400:7e83::615]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8d5d8bd0-04e0-11ee-8611-37d641c3527e; Wed, 07 Jun 2023 05:08:19 +0200 (CEST) Received: from SJ0PR05CA0035.namprd05.prod.outlook.com (2603:10b6:a03:33f::10) by IA0PR12MB8280.namprd12.prod.outlook.com (2603:10b6:208:3df::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.32; Wed, 7 Jun 2023 03:08:16 +0000 Received: from CO1PEPF000042AB.namprd03.prod.outlook.com (2603:10b6:a03:33f:cafe::df) by SJ0PR05CA0035.outlook.office365.com (2603:10b6:a03:33f::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.19 via Frontend Transport; Wed, 7 Jun 2023 03:08:16 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000042AB.mail.protection.outlook.com (10.167.243.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6477.13 via Frontend Transport; Wed, 7 Jun 2023 03:08:15 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 22:08:14 -0500 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 22:08:12 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8d5d8bd0-04e0-11ee-8611-37d641c3527e ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XbiBz0kecpBhF2DYbPmGd/kLtbfs6Ca8+4/xGF44TLE43g1wMSonzZjHcFFWTZbIFHXRpwDNV/cDMTSZvYliYg1t30LqUzqgbqseVRXdKT3sosFNdUBs+ScX2yiHKLPjKZ69qjoIL7PtEqGLfbjzMR9ytoReA0FXvfJ10A3NYYBN7sNOeIafGBtsOnDWmB8TM181gaYnhUJrsp91sU0wZTKF6zzlXzOX1O5q6lsHSr/Fqjv7rr30CVI8CJmsivZzP3HUbKxGEZhEyu9yeTkTZmGFsS34SDBXJHGbR/81UhObqrqFtGYg/hGf1V2xf4RxTsy8f9B7x7lT9BcLRlDh3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wRrQWmRSzu/GIUmuJ2fEbC87ERNVvfs7J7zPQvkONhA=; b=CSYcLwQAQD4HXLEzDUGCCQSTUMN57BX8GsuIlki6kdeUsidwsGx5nRenFJ/xnLpDFCaQI0+yGjqUZt2yHXtzVZfES6GVjepeeG0LOLYC3kg+KJFMeWr/GcohWSJ2YvgqA4KOFJ3Hn4WgN6M3yyTvZtnc/BArUHNpNGJjgAQuOlgeI86ghBpTJ6zUMfEa6T/6FTnFd8I+Lpab2Savz5Q1GHniGTcmX2DaBOvTRoWcwJ0THi5GIHizS1s/OW7/EuPtjxEpLPFI+HmPWyar11gYuHyATPUPpLj03VXd6gTUNmt11Hib8J0mfYStGxp5c2JYMy4ZY2FURLBIJJyzv7F5qg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wRrQWmRSzu/GIUmuJ2fEbC87ERNVvfs7J7zPQvkONhA=; b=Lk/gF79vHhHHUMVSDVYBPx9GMbPmjMrDnZTmCf6TAtVn5v1vR9H1T4qDgc9juPdPbfgvsGKRefYpVUArPCPExm9x56z/yi7OYVtnpRm3FGWdZsNTQ6s/+AhXGbcdeqvvUfxOWljsUNIuvJRfIA/TA5GCIQiRoEgCW+nc9B0L5/o= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Rahul Singh , Bertrand Marquis , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Stewart Hildebrand Subject: [PATCH v4 7/7] xen/arm: smmuv3: Add PCI devices support for SMMUv3 Date: Tue, 6 Jun 2023 23:02:20 -0400 Message-ID: <20230607030220.22698-8-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230607030220.22698-1-stewart.hildebrand@amd.com> References: <20230607030220.22698-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000042AB:EE_|IA0PR12MB8280:EE_ X-MS-Office365-Filtering-Correlation-Id: 250657cf-1483-4a7b-4f46-08db67046fae X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: J+ykEm8fma+CPdpj2dIoU0v2cSX82VX2RObO6QAzu6d0H9oeYsMaVYJz35sSn9diEhle8Lf6NcRJ3wp3GAP3IfVgb64qWXqAuPjh/XwsGspWFpQe80es3BQmqfvTMOL2o7MK43rI3soRzhaIHbSbP/Zv73qpuQfEYOctsjjrWkX31LHMqXtMmUayXO7kUVW0nIlQ6drosNcohc36Z2Q1rGGeWrMc8QJRCkWfBG1Ag6snbGDmgjxLRmxORpSI6QrfE+nGUS4knSzXhbIM/kI3/luS0oVKAhYSN+zZmJCmxA41cOltmDtwwtrSrSo5ZburiwofntPu04/yy+1ssbGxZfPW8SVdxwzQgZujzjK+oRWUQbmlvFsAfwFN47kjVYegscfsXW3eS6H2Uax0Ic68ICcsCyoa5VNEFPScE64f3orNozobN+2ICp+fSWIdQ619V1LI5gAPzLWr5P63aJsZ+Ie3uy8gnoCgfewOLhJGsUS0PMBX55JnUNR/PfWeOgGcnpD5Y1M//JnQbFFvZf0mAwCZQoC16aXs1vYnn5BKNiev1Dynv9qSN1qIFWTNwGFVXPkMNZapMDRqhgksUVfa2mGlvSXXeDGMpqw5QM5Vb3r35GlMwHc340etOhvoMJv9tKYX4HMp+sPKYdMtVAMOFDyOkxTi3W+E5+O5+PclqNWVhIgO5eAdNeSdJBOATjj5gTtBSTpthXEoi2yytCQE+TYlgm+jGDfX5lVhR4u3/2N59CdNrbf9sc4u2gIpOiI8OVpFizheVC9DSxDqKNoUUQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(346002)(136003)(39860400002)(396003)(451199021)(40470700004)(36840700001)(46966006)(36756003)(54906003)(40460700003)(2906002)(82310400005)(478600001)(86362001)(41300700001)(5660300002)(8676002)(8936002)(82740400003)(316002)(44832011)(356005)(81166007)(40480700001)(6916009)(4326008)(70586007)(70206006)(83380400001)(47076005)(1076003)(2616005)(26005)(186003)(426003)(336012)(36860700001)(966005)(6666004)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 03:08:15.2719 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 250657cf-1483-4a7b-4f46-08db67046fae X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000042AB.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8280 From: Rahul Singh Signed-off-by: Rahul Singh Signed-off-by: Stewart Hildebrand --- v3->v4: * no change v2->v3: * rebase * invoke iommu_add_pci_sideband_ids() from add_device hook v1->v2: * ignore add_device/assign_device/reassign_device calls for phantom functions (i.e. devfn != pdev->devfn) downstream->v1: * rebase * move 2 replacements of s/dt_device_set_protected(dev_to_dt(dev))/device_set_protected(dev)/ from this commit to ("xen/arm: Move is_protected flag to struct device") so as to not break ability to bisect * adjust patch title (remove stray space) * arm_smmu_(de)assign_dev: return error instead of crashing system * remove arm_smmu_remove_device() stub * update condition in arm_smmu_reassign_dev * style fixup (cherry picked from commit 7ed6c3ab250d899fe6e893a514278e406a2893e8 from the downstream branch poc/pci-passthrough from https://gitlab.com/xen-project/people/bmarquis/xen-arm-poc.git) --- This is a file imported from Linux with modifications for Xen. What should be the coding style used for Xen modifications? --- xen/drivers/passthrough/arm/smmu-v3.c | 76 +++++++++++++++++++++++++-- 1 file changed, 72 insertions(+), 4 deletions(-) diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c index 8842db1ec07e..427946e68f9d 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -1469,14 +1469,32 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid) } /* Forward declaration */ static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev); +static int arm_smmu_assign_dev(struct domain *d, u8 devfn, + struct device *dev, u32 flag); static int arm_smmu_add_device(u8 devfn, struct device *dev) { int i, ret; struct arm_smmu_device *smmu; struct arm_smmu_master *master; - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct iommu_fwspec *fwspec; + +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + int ret; + + if ( devfn != pdev->devfn ) + return 0; + + ret = iommu_add_pci_sideband_ids(pdev); + if ( ret < 0 ) + iommu_fwspec_free(dev); + } +#endif + fwspec = dev_iommu_fwspec_get(dev); if (!fwspec) return -ENODEV; @@ -1533,6 +1551,17 @@ static int arm_smmu_add_device(u8 devfn, struct device *dev) dev_info(dev, "Added master device (SMMUv3 %s StreamIds %u)\n", dev_name(fwspec->iommu_dev), fwspec->num_ids); +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + ret = arm_smmu_assign_dev(pdev->domain, devfn, dev, 0); + if (ret) + goto err_free_master; + } +#endif + return 0; err_free_master: @@ -2622,6 +2651,27 @@ static int arm_smmu_assign_dev(struct domain *d, u8 devfn, struct arm_smmu_domain *smmu_domain; struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv; +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) && !is_hardware_domain(d) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Assigning device %04x:%02x:%02x.%u to dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), + PCI_FUNC(devfn), d->domain_id); + + if ( devfn != pdev->devfn || pdev->domain == d ) + return 0; + + list_move(&pdev->domain_list, &d->pdev_list); + pdev->domain = d; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + spin_lock(&xen_domain->lock); /* @@ -2655,7 +2705,7 @@ out: return ret; } -static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) +static int arm_smmu_deassign_dev(struct domain *d, uint8_t devfn, struct device *dev) { struct iommu_domain *io_domain = arm_smmu_get_domain(d, dev); struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv; @@ -2667,6 +2717,24 @@ static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) return -ESRCH; } +#ifdef CONFIG_HAS_PCI + if ( dev_is_pci(dev) ) + { + struct pci_dev *pdev = dev_to_pci(dev); + + printk(XENLOG_INFO "Deassigning device %04x:%02x:%02x.%u from dom%d\n", + pdev->seg, pdev->bus, PCI_SLOT(devfn), + PCI_FUNC(devfn), d->domain_id); + + if ( devfn != pdev->devfn ) + return 0; + + /* dom_io is used as a sentinel for quarantined devices */ + if ( d == dom_io ) + return 0; + } +#endif + spin_lock(&xen_domain->lock); arm_smmu_detach_dev(master); @@ -2686,13 +2754,13 @@ static int arm_smmu_reassign_dev(struct domain *s, struct domain *t, int ret = 0; /* Don't allow remapping on other domain than hwdom */ - if ( t && !is_hardware_domain(t) ) + if ( t && !is_hardware_domain(t) && (t != dom_io) ) return -EPERM; if (t == s) return 0; - ret = arm_smmu_deassign_dev(s, dev); + ret = arm_smmu_deassign_dev(s, devfn, dev); if (ret) return ret;