From patchwork Wed Aug 23 16:47:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13362850 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A858AC79F65 for ; Wed, 23 Aug 2023 16:47:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237662AbjHWQrx (ORCPT ); Wed, 23 Aug 2023 12:47:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231454AbjHWQrw (ORCPT ); Wed, 23 Aug 2023 12:47:52 -0400 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2065.outbound.protection.outlook.com [40.107.237.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87FA61AD; Wed, 23 Aug 2023 09:47:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=P5uIAJmPEFlWO8hDj2gBBW4TyGETQhHE8mu52gBstUm9x6+6PEvgo2kd1sq+XyiWEbc8efuiFQhR9qF68eeFBBTTS/jv/1LAwnr9HyumurDBGXjsbxruwInawhs713HkkNPUNyHCbs4txN7yiqZmrLC+drEzFrHkajGIxAF1QLSU94RlS03Cn7DR6Pda6w2iQ9ZSagUClfSaj6uAiGTlkIZOzOY0Ab/pJbkMrgyyzHum9lru9hzZlq0XnLybATcbmIf3AKwuJvo1ScS4gvJdGlFyaCeDTUIO+4Tyg/Xoav7QuclkQA7B27IxwMo3C3sVoMsWCcRPlwl2n8RNVwaomw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=t6M9n/VjC1eTk+y9VT9i+RBTlxuxYFMfXiM2LaimdaA=; b=WNswjAkaV7uNomhLEwT6sE/4WSbG12QyCcKWEd5JBLOJ6mlSvq2ZGKQvbwJC7ZM88NruFiLhf+1m+VbrFMkQu3T4/V+b43WeU6ZfvAcRh59OlnWfmr39AftSkLHgXoywvpYBu7At3Fn7aqpmI/unL3aBJ+PcyLNrCYpRrz+wUiinb+EPvsRY8F8JjdyxQX94Mp2aCPIlqOU3TUU3sB/d6UPRkT7sJKBcHzTvmhfpAXB9m59r9HzhaXJ7ScrapVhFxgC4YBm/PAL5aYywv6e632fGiJAyhVslYsPX6VxmJG2vCLPtOPkDMk8JP0sjSDPEDjWkN9sqG+4T0ndG3CbaHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=t6M9n/VjC1eTk+y9VT9i+RBTlxuxYFMfXiM2LaimdaA=; b=s4Dt09ifkGr9NO1YkVzKrevMSCp0frrVDKHDBA48e7C0o8JgrGCV+/mosPZgBI2hM4E/tzY15TSZmErsNkiRrJ3xcn83UgspIec/jM6+wGnWtPc0g3AR26C6cvbo1cFyK5lUisjkj5024P1pnYlZgZgTT83TerqsMaNQAaDKrdgBph15ztnk2z5L7knCZ84+esJFg+01XJP7jwVqOQqoPVGaBsiGpNtkMS1OVMcJipbQIjMOV4ph/CaDlk7bBXyR3vj5q2yag3stAh1p/886lZ+Z4CzzvygGK3UbYWWthVc6+zjEj3vbLca90jZ1eo46F4OZLmh1YpwJa3u2qDmR8w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by CY5PR12MB6346.namprd12.prod.outlook.com (2603:10b6:930:21::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.24; Wed, 23 Aug 2023 16:47:47 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1%6]) with mapi id 15.20.6699.020; Wed, 23 Aug 2023 16:47:46 +0000 From: Jason Gunthorpe To: Andy Gross , Alim Akhtar , Bjorn Andersson , AngeloGioacchino Del Regno , Baolin Wang , Christophe Leroy , Gerald Schaefer , Heiko Stuebner , iommu@lists.linux.dev, Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Kevin Tian , Konrad Dybcio , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-arm-msm@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-s390@vger.kernel.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Russell King , linuxppc-dev@lists.ozlabs.org, Matthias Brugger , Matthew Rosato , Michael Ellerman , Nicholas Piggin , Orson Zhai , Rob Clark , Robin Murphy , Samuel Holland , Thierry Reding , Krishna Reddy , Chen-Yu Tsai , Will Deacon , Yong Wu , Chunyan Zhang Cc: Lu Baolu , Dmitry Osipenko , Marek Szyprowski , Nicolin Chen , Niklas Schnelle , Steven Price , Thierry Reding Subject: [PATCH v7 05/24] iommu/fsl_pamu: Implement a PLATFORM domain Date: Wed, 23 Aug 2023 13:47:19 -0300 Message-ID: <5-v7-de04a3217c48+15055-iommu_all_defdom_jgg@nvidia.com> In-Reply-To: <0-v7-de04a3217c48+15055-iommu_all_defdom_jgg@nvidia.com> References: X-ClientProxiedBy: CH0PR03CA0347.namprd03.prod.outlook.com (2603:10b6:610:11a::26) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|CY5PR12MB6346:EE_ X-MS-Office365-Filtering-Correlation-Id: 95187253-e506-46dc-80e5-08dba3f8ab4e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: j/lmfd0Wy6FDMKu37PmaIwaphdEfiLFlUsoFoFcUiZBo8PKJVsjCMBZpg37Q0/k/hGlMRSzYcEcEUpVMhpsqO9fWgZDxhuJIdhGSxlgJEY38rUE6Ev3bTpKmW5SRd+G19TEokEb+pzvGy2guAkwQjkk1LeLA8KZwShGkwZCUh5GTGyuvOCdcD2IO1PnW0glc8dN2ZTYog1TUvPJOvsVOWUa4KZyIuSRScANqYGjxeNPgymmXEskzffD0nEBf1kZOTOlHPjXyZlpjPm142QGcUn20WSnnrYRQ8PjPcFe5UPiAWsM8pro9V1v0h/oOJnvTRxRBf+v7uIZ+q3ylZcr0hytnsb2AEGVks40U52oy3G+qROLupmjgqJYZV6liJecqArjIl2qdGxj4NVzHZpW+lsd2EQblFGOQo+aU7Nn79PX6EOmBmr0K01nfq6o2O/iJP54Wl5xt85sUjxC8etBInFefG1ekfb2eyFKQ0GwYxvMRkxYldcs3Ch2p9yJ83u5Ecd6LIAb845J2L+BdfJBi+d/6EgCa34/1U5V3hMIc/e7UdNZFm61+QOVSS7BeFa1AD6P1t+nRhNmVZTnFyIbABtEwMDgsvBBm2EfF4gcoLI07OZ0Vj5A2qb1FnnKD2m0u X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(396003)(136003)(346002)(366004)(39860400002)(186009)(1800799009)(451199024)(54906003)(66476007)(66946007)(6512007)(316002)(66556008)(110136005)(8676002)(8936002)(107886003)(2616005)(4326008)(36756003)(41300700001)(921005)(478600001)(6666004)(6506007)(38100700002)(6486002)(83380400001)(7416002)(2906002)(7406005)(86362001)(5660300002)(26005)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: s/tFuWUF70RgeCfabqkq/n7tt2KOCASJl1KABd+fmUDAdT6Ty2vipmFcyO7aVpxAwa3a5i86G84fGvPJHPVf+W0UJI4DQKloGcMuyYZQ/Tn/NoiAcGAElOl+RnngKEGCwhioKckrfrgwutVjdi5/u9ghzW750twGsLPGpzrkNgGrM7Z2ekVGxy4XFCzpjMUctHGsNuGWe0TtJBR8L2ArjrMo3oWoWwkdTPcuvo19hebIBwV/m83PiYA28vhtw0L0IEP0/T/6E3hO+Cu7JB4aVjTqA7OfvQmARmVyZj+DBAHsuXfqIXly4nhsUSqd+SxEU+L/npGMsRU/IX6K6S13Fk45GjfboSbSziWcXF7PqynbwKLY8+Hnz9hrGO5ah9IHyv52yFW0b6BczB//meXwCeRVnYghRGLCw6WaVV8saS1CAMDDz8NXBSPcU7yULq3Am6Lmg90e/IazKc+f4Os7w1TnXn13Kr0278k07UiuVP6q8oB0QfEPwFrVyWfuwglyFbVrEhcXNHsdtxcVNSchav0/w3/5AramKfhcAsTtVkQtYs+JDMV+vimz03cWfj+UT+k/r0gx6g9ehHJYSCsejJ6KNRvymaEQ8K/wfwFGo7V7j4yPG01UUQO0eMFzOqVqGsCpqJelBLwSkHgsgM4hFqIt3d4/O2vXIA/CbaqJRvh10IuqPjIOcaQwHoLCAeGRVLgAMd61hbHMqIvJ8G05u36RHAA9i3V84FH9S+UaWWCaDcM2PQNxs1ZgC6bLKcADDJVv1CgG16hqz4WXnZ5oniJpCUbCQKaKgDeBRrYc09NEVNLwqtwAZi1eKj0sMr+ivubrz3KGlFEYOC6eNBs+VbGpBz+sFdQ5X/FYiXA5HvgDpaDK0M2EslB4vmZNej84hNT48FcAxPfUA8VPsQlHeGOAWVBmfymB8dtKQVGOI2aLh2zqlhxpyquH1iakE2vve16yiWoc+f+TKm/jyiurcZSfdIHmcHVxXymJBp+xjLuZUhgWEQDOo8Ob30GtCZIA1fjk2h7VYK7ueiVcyOootAdBhXEnJnDaGNVYcpKHOJ1f5k/6ykiSy/bBHNj7E5DsKTo6Du1uPLiqJHtY4tQH/8xKXc2EGrzMRO+xKZ6DHqmh9XjPCXedRANnQ8y9kmfeY8Coj9MGQ6K+cT67TasblasPmcV8fQPVZWwvC/e8PM+lSpa9ajnFTMyW6knsy0fJRPJIuCfaCTBT4NwOHl5cU4ssTSg/6AyaE3HIZlldUDjZAidyz5e1Ofxo5BYhtFWFHV3zyMTBrPgcrGS12GE2oiZdzHBbn091v32fx5ll8f4OUEWqm+XeatB23dTnfGygfrZvBuLKnc+PYIq7D0+9MUT/bzdq6Oa/AjIyyIQdeMzQ79MUCIcaC3k3g277St7zNDinvgR5rPuLV8KbubLucX3WgxYbhQyBNoMk5hOH+IQKHSnwgyWn4en14B+2YThMMQuFBRZslyg9/7en3NCNZYerVpCnNn7pUBLKAXtKKSfqnCu6iuB0eIwTN+h4xVakCnOuYvepka27FQ9TWrwNHwCBctx2fg11MOEAPn22/fGk+j+ULGRgnXZubxnzrhqz X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 95187253-e506-46dc-80e5-08dba3f8ab4e X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Aug 2023 16:47:42.7931 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: R7MrcuwEjDq+RDZtWmfgVqkqH1PDu6CQbDc93paJKYcctz7uUEnUvB3tPcV6W6M2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6346 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org This driver is nonsensical. To not block migrating the core API away from NULL default_domains give it a hacky of a PLATFORM domain that keeps it working exactly as it always did. Leave some comments around to warn away any future people looking at this. Reviewed-by: Lu Baolu Signed-off-by: Jason Gunthorpe Reviewed-by: Jerry Snitselaar --- drivers/iommu/fsl_pamu_domain.c | 41 ++++++++++++++++++++++++++++++--- 1 file changed, 38 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/fsl_pamu_domain.c b/drivers/iommu/fsl_pamu_domain.c index 4ac0e247ec2b51..e9d2bff4659b7c 100644 --- a/drivers/iommu/fsl_pamu_domain.c +++ b/drivers/iommu/fsl_pamu_domain.c @@ -196,6 +196,13 @@ static struct iommu_domain *fsl_pamu_domain_alloc(unsigned type) { struct fsl_dma_domain *dma_domain; + /* + * FIXME: This isn't creating an unmanaged domain since the + * default_domain_ops do not have any map/unmap function it doesn't meet + * the requirements for __IOMMU_DOMAIN_PAGING. The only purpose seems to + * allow drivers/soc/fsl/qbman/qman_portal.c to do + * fsl_pamu_configure_l1_stash() + */ if (type != IOMMU_DOMAIN_UNMANAGED) return NULL; @@ -283,15 +290,33 @@ static int fsl_pamu_attach_device(struct iommu_domain *domain, return ret; } -static void fsl_pamu_set_platform_dma(struct device *dev) +/* + * FIXME: fsl/pamu is completely broken in terms of how it works with the iommu + * API. Immediately after probe the HW is left in an IDENTITY translation and + * the driver provides a non-working UNMANAGED domain that it can switch over + * to. However it cannot switch back to an IDENTITY translation, instead it + * switches to what looks like BLOCKING. + */ +static int fsl_pamu_platform_attach(struct iommu_domain *platform_domain, + struct device *dev) { struct iommu_domain *domain = iommu_get_domain_for_dev(dev); - struct fsl_dma_domain *dma_domain = to_fsl_dma_domain(domain); + struct fsl_dma_domain *dma_domain; const u32 *prop; int len; struct pci_dev *pdev = NULL; struct pci_controller *pci_ctl; + /* + * Hack to keep things working as they always have, only leaving an + * UNMANAGED domain makes it BLOCKING. + */ + if (domain == platform_domain || !domain || + domain->type != IOMMU_DOMAIN_UNMANAGED) + return 0; + + dma_domain = to_fsl_dma_domain(domain); + /* * Use LIODN of the PCI controller while detaching a * PCI device. @@ -312,8 +337,18 @@ static void fsl_pamu_set_platform_dma(struct device *dev) detach_device(dev, dma_domain); else pr_debug("missing fsl,liodn property at %pOF\n", dev->of_node); + return 0; } +static struct iommu_domain_ops fsl_pamu_platform_ops = { + .attach_dev = fsl_pamu_platform_attach, +}; + +static struct iommu_domain fsl_pamu_platform_domain = { + .type = IOMMU_DOMAIN_PLATFORM, + .ops = &fsl_pamu_platform_ops, +}; + /* Set the domain stash attribute */ int fsl_pamu_configure_l1_stash(struct iommu_domain *domain, u32 cpu) { @@ -395,11 +430,11 @@ static struct iommu_device *fsl_pamu_probe_device(struct device *dev) } static const struct iommu_ops fsl_pamu_ops = { + .default_domain = &fsl_pamu_platform_domain, .capable = fsl_pamu_capable, .domain_alloc = fsl_pamu_domain_alloc, .probe_device = fsl_pamu_probe_device, .device_group = fsl_pamu_device_group, - .set_platform_dma_ops = fsl_pamu_set_platform_dma, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = fsl_pamu_attach_device, .iova_to_phys = fsl_pamu_iova_to_phys,