From patchwork Tue Feb 4 18:34:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13959644 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 03D77C02193 for ; Tue, 4 Feb 2025 18:57:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=GaLVkctvhEv1FkYriYm5WR/smy/3nE7yXIwuZDLwAyQ=; b=qd82Jsuf5Qvoy0OCHg7PXLdGL8 8ItFDv2OV1Mcq7kPPoAiYGuEWYuPTgncsPjXZArRtPBm/7vwz3PYjRNdz10krEzl8Vwz/GtYkgXlh 5H6dCGmDRGhYVIHnVzuEFXY+Oc394NcCKve8hPgpKYInRLuk4tx6xf9y2hu0wb25hWRSOgzgQpL/k tODJPo0oYSQrul06ww/MEuHxEF2ymk1DF/DwTHRVHJsvuSKZixKG7jeq27YufdFBASeqMpTAVz9yF bExMpjvcfk6M/HST0DDqfSxOSdsXZVFvQq0qgvDOT1EmeauXvfEw0A0mC/+ja+VbH2IjVi55dDRFm kKuUSDaw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tfO6R-00000001I3P-0uO2; Tue, 04 Feb 2025 18:56:55 +0000 Received: from mail-bn1nam02on20610.outbound.protection.outlook.com ([2a01:111:f403:2407::610] helo=NAM02-BN1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tfNla-00000001ELM-27G0; Tue, 04 Feb 2025 18:35:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XzUz2HkKwcYNQTMyNX8bX2nN4upqHDA0JpEOv2+R8Pfy9Ui4yjiP9aWKg9C+2+HFPkEV6S6e/vYNCgsSLpYpUP1cn4WsZgGzOvb8n6RCl9rkF3OdETZJR5l8LYfrstCpfxBJyAI6/aUnT0M8S140MOeNswK4ZcVUuZT72SQRssPXZrqdeRJ+pcojjU7tBxtyfScb6x8iwvxs6CQhqrPVR8k0zriQ/YRnaEu9Yks4iFDW+XDrdhtoHqHNCW9cO6dCvuHRB4PwVEIO7YVirDM92YoSzeHUR8sH/gEWlO5+YsdSNoy03Jy0wvncZDhjbbiuh65V6RQyGZnBtdPcMMYLyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GaLVkctvhEv1FkYriYm5WR/smy/3nE7yXIwuZDLwAyQ=; b=NmRNiioOTU/grZ6HdxUnB0k778UZVRZTvpE2e4Nt9oCZ4kPszocR3SgfVm0xp9+dBf2ItY1+8T8ubjq80/KhWi6Jqw/emAxGHDRJFRLOVpzckf0cMKy7Ai+YGZ5Xq3I5++TTGs5mcBh1e056tHRJ/99CroQDQ0Xdq2lu9s0cSG5HmNrsJBQClp8Dg7L7db/lJWKXg5y8P9D6Ku9A2H1tvY31x1+fmWfHMs8/qzh09NI89mFNRgU3GC7VnR0NzdAE9zrvEXwEHU/CGqhBNawziB90O/6r6h+607VF4kp0Btix8pK8P7QI753NKwjLTPX16rjrbLb48BkdJKIpBlazAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GaLVkctvhEv1FkYriYm5WR/smy/3nE7yXIwuZDLwAyQ=; b=h+H3c7YrPkSL3egb0cdasVkvoSCfIb3o+nGIyqWSWIfjT/BfOWRaRPFK9SMeX59uO+7fGz2GxwVkqrjPbWAJapGjGNVqfWkrJK30MwlNxENaXiqvFF2nuFJpB0mDx5SFELTe9O/8g2ei80uHuAoTNAJZ22QaNHDQ8QU26qVCMrfIx5gdjl6Gb8U5Th9dyBubzKZapbUOC2rCjKvJMZ7pT3AJoY3SexsXFEynW7AD+BNszBXVdCiiMlbRF+9p5GvOnl3hGH8G2nwTbjnh+PQa8WMcflkUSrW+stzdaPVKpTQUgDG837aO8BJ/ZhHGdq4mUjwGb3eCFP0wDbdkh7WPdQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by CY5PR12MB6573.namprd12.prod.outlook.com (2603:10b6:930:43::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.24; Tue, 4 Feb 2025 18:35:08 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%6]) with mapi id 15.20.8398.025; Tue, 4 Feb 2025 18:35:08 +0000 From: Jason Gunthorpe To: Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, Lu Baolu , David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Hector Martin , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Bagas Sanjaya , Joerg Roedel , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Matthew Wilcox Subject: [PATCH 10/19] iommu/amd: Convert to use struct iommu_pages_list Date: Tue, 4 Feb 2025 14:34:51 -0400 Message-ID: <10-v1-416f64558c7c+2a5-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v1-416f64558c7c+2a5-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0383.namprd13.prod.outlook.com (2603:10b6:208:2c0::28) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|CY5PR12MB6573:EE_ X-MS-Office365-Filtering-Correlation-Id: 521d221a-f226-4c9e-c514-08dd454aa40c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016|921020; X-Microsoft-Antispam-Message-Info: wBaeO/s9tSo+UIUdbDW1mj14Z1TIzEauZhefZOyieCpNcyxBIDJpbz5UGVngZtFhexvZe4mYze0Kg2RwuNPSKzL0aqOYAyQ9pcgX7I9Ko1DXFoEiR8WEDFV7rqwuEGBmt4LLZp8Y5/AIAWqglV0vqyirBd6SBtgPLc6NogAHVQV+B6JUemmw3tiksWO7LiyXIPkCtaT4U8a5KJ0LgsbTFiIa34jVcNdmeTsUdS75qOZVKuv6gXQNY3cf2ksoMcthzkuUO59mLNGblMI/Ru+hIZ2sM71F0TcxkfenFCf/Cg68R16ffa0H9LiLWo4iTAcTxZBqM0UuBzTWNrf3LfZZ/HECPNxmoSwx1QItIO2YqwqSdFqk6e7+Bkg8hFPkxcOevpuNh2JlywdohucWXjtA2727exbDHzm7J41CAvO7D1X/wuA/X9v9v5jRw3yla9de2Q/zEOCZqMxmCy62ZRiv4IWdgsyPv3Xl5prQDJxMVI+H1amx7U9Tl98tnptF0ieFXb1pBeuF05yUr6AMSlAQWs+1GTY9ASBmEzQP5z3jahi3KysSg3FIyHrd28VEakJggSOa1eSdtf2xgghumqlfAvGPX0rs9TQVk2ERdgffnCkqs3r5P+dPMU5+T6ue4NB58ouSJDXxX+J5HH7QVmvDkwPmGodZceCjv3A3TKt+cHl/SIj8e3KQMypv+uMCD/48kVN9lyPhcn9BHvNWDjRyS4sn0c7Sq5SaFzIuksXuhSNoBnXPhvBgLMWI4KQuxCAwAMd9uKJ8pM23pqpJ+XTHkpB+0b7R2GPOZfrVGjXw80vRF3ip8wFr5uxMxX9zNLliJlb0LYrnsqTZFEVU/AOeXoOumv/c0ZHVhbfCBn1axV8Hkf0UOim3cvhOdUSc47OmyXybCznSzer3w1pbcLcGM9zFGgR4uCkLHdBHUxsB2ublLFma1mmbhIi0oUz1gYaavHMrZt+eSwnToznQVMPRO2TljowT7vmiWHv6ag60m1weLIcFT07q5IbNzi76MbYsFD8qb2q4cBKjAmUen55vImrYmICMcJYVkss88cgJsEE56hdnpP1mH1C9rG6oQPTU3tW2XQJmaRtHw6COWsvDmAjyqhujvHYJVKYW3MkflAbk+woj/nlqJNxFQksV9mJWCKjmiljIz6NKMQJwPl2BWBRDP5Dgek/+8kI0raGIWkO2R8FWuADB5QWhkBLbJCn2XMFb5l873NjnszR+9F8peVR61ydmYjLQgP9FPKcLJsZYTQL//4ANJcCuCuqu42h5YPxmPiIo66b2Js6nHvlyq5Vq7s/CXJvUK01DUK9y6Tm4hczXv/dyFT91ec+ED4MWc7zHPeoxIrynCNR7yXWrYWyY0ir0Zt0o7BMVlBM4OIggQaGqKHWf2rPTtkl79YeYjWd8eO1RL316qJO4eydAuMEyBQ0qa2qJh/9u0uII/MQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: AgsRQzky7nfpzPX3CyGHPz+G2m0s/gq3m0FJcbA+oa8mB/jEqhj6A/ZdOPb0TM1NhmjC4K+2obE433mFzlhWlgmMI7Mhfr9AH+OCW/MyvLQ4kbkU9F0TYqXIvqTR3UNkx9lQIJ2xUbFqEeqf5vfPjrNP2G0IDLh81WhBRBHmNc0zSvW3lBF86zu87acCya0DKJbToIx7y7ACazpHfvej/KZMmsAC7FqHAVvJLXObIC5OnZPbIQZc3JogkUppyk2nnyg1KDaEq0z0byVNodffCHFkeFmiogADyczPoXSiZrXm6GFxTF5/egGGqDnBImwG0eBsQtbfQvTtksymG12SLTVFtVVmk48qnBytTC5MCE04h6MBx/eqOHXsuUG5rrTyfV7WybUfQsZR3jhpiByccQnNRjOljhQKz/ooMhY5QD27ffNUxb3D198ni5ZFWF0hpQP22+Khl1JLIcn8/dbvKVn+3eENwvxh9RPeZObyvludNRwffuei/Dk9guY7B9eA7IlGOgVHUQHWLzmmiWYq9cpGD1CwkbGFMvQqYz8fwN/+hYjM5WLAv9n5wp0ltaxH4gYItgcoCJga0NgA1To3axh1XRjbldWZiDrxPZmCbFn1cdWHjNHbuHDqvqWOC54seVBlMOjQh1TIvYRViyngb1iPcqFYGCIUxjWRjEuYHR7+ZTrAFzU8GBRNkos2bUniNnmlDd8XRQBl/+huZxrRwkDRBLNFHyO6qt3h5OhaHqi997NizcJA0i+PecnCVTS31JsMRjUvFe43dp8Sa55/EJIZurEm1MtzWi0XB9nNPFlc5K9pdcW9z2qpH5VlOUmymtFZPF9A4G6kLdsBzMCu/s8ThXQUr3UE1aRRRnesEiRcD2yHPziC7DYHBcncnU731L7V2BBwiRwK5pY6qcaavxCZ0ENG4pUzP1MNakK+GEPaqaOmYY+19mkF8wP/w34PM22pX3wkzqOddJ2U1r+Ig83e8GI5k5PTdhts1QBzQ24RHlNrUGrnd+wD9lEjC5wPbTv2sNDlF7AzjGNqPmtea3HNi+iXVN1mRsdw7Qdg0fJvt3whAWuBBODT89Y7pfKO+Iuc3fhSP5jMY+DsldG71d3KNidnjr9z7j3DJVH+IuKVFgfomcOLvcuPhRgwP2tQpuu9vdGMFHt9nipoko16Qs4Z8aB5HAgw9EGz481uI9yRn7yZUSbN80nYGuvymuvEHF/jruJYJpu4DekPZHpq/f4big8SiKR5rj4BPE5sIXECh/BxsMrUo5Evu/7k4aAQfOO4/w7rf2+nye/S7XEbV6pXqo+TUpDOqaRAAKw6OT91kqTpJkhBRUWIqwj3EJVFYkjbVoqMrrKPIfIettoxHy2VfCtFgakq7JWgVkJd2ezur5Xuoymt/yxZU6aYmSdpr1gpCiNd1bVzSLL/wWImBK8KZu/hOXaRtRMd85quOgg3y3npudzbw5/19YXeyygBMTFq5FfQG7qtodRsZ3pWxRYZGCislTx9UFYP61Xe+IE+RJ4GXID6wy2ZR9KzzS6zPJk0z4a9FJ0fVgZkEq3+kjzUFvFbqoinwQlHPsYGbUI= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 521d221a-f226-4c9e-c514-08dd454aa40c X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Feb 2025 18:35:04.0800 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IlofT3Yyv3Wc46eGEq/XayevLFJ/7papp2/wxZvP+ue78bAbq4i1WUmDoSi4Afnu X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6573 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250204_103522_554581_8988665B X-CRM114-Status: GOOD ( 12.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Change the internal freelist to use struct iommu_pages_list. AMD uses the freelist to batch free the entire table during domain destruction, and to replace table levels with leafs during map. Signed-off-by: Jason Gunthorpe --- drivers/iommu/amd/io_pgtable.c | 26 ++++++++++---------------- 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/drivers/iommu/amd/io_pgtable.c b/drivers/iommu/amd/io_pgtable.c index f3399087859fd1..e375ddefdb2cde 100644 --- a/drivers/iommu/amd/io_pgtable.c +++ b/drivers/iommu/amd/io_pgtable.c @@ -54,14 +54,7 @@ static u64 *first_pte_l7(u64 *pte, unsigned long *page_size, * ****************************************************************************/ -static void free_pt_page(u64 *pt, struct list_head *freelist) -{ - struct page *p = virt_to_page(pt); - - list_add_tail(&p->lru, freelist); -} - -static void free_pt_lvl(u64 *pt, struct list_head *freelist, int lvl) +static void free_pt_lvl(u64 *pt, struct iommu_pages_list *freelist, int lvl) { u64 *p; int i; @@ -84,20 +77,20 @@ static void free_pt_lvl(u64 *pt, struct list_head *freelist, int lvl) if (lvl > 2) free_pt_lvl(p, freelist, lvl - 1); else - free_pt_page(p, freelist); + iommu_pages_list_add(freelist, p); } - free_pt_page(pt, freelist); + iommu_pages_list_add(freelist, pt); } -static void free_sub_pt(u64 *root, int mode, struct list_head *freelist) +static void free_sub_pt(u64 *root, int mode, struct iommu_pages_list *freelist) { switch (mode) { case PAGE_MODE_NONE: case PAGE_MODE_7_LEVEL: break; case PAGE_MODE_1_LEVEL: - free_pt_page(root, freelist); + iommu_pages_list_add(freelist, root); break; case PAGE_MODE_2_LEVEL: case PAGE_MODE_3_LEVEL: @@ -306,7 +299,8 @@ static u64 *fetch_pte(struct amd_io_pgtable *pgtable, return pte; } -static void free_clear_pte(u64 *pte, u64 pteval, struct list_head *freelist) +static void free_clear_pte(u64 *pte, u64 pteval, + struct iommu_pages_list *freelist) { u64 *pt; int mode; @@ -335,7 +329,7 @@ static int iommu_v1_map_pages(struct io_pgtable_ops *ops, unsigned long iova, int prot, gfp_t gfp, size_t *mapped) { struct amd_io_pgtable *pgtable = io_pgtable_ops_to_data(ops); - LIST_HEAD(freelist); + struct iommu_pages_list freelist = IOMMU_PAGES_LIST_INIT(freelist); bool updated = false; u64 __pte, *pte; int ret, i, count; @@ -360,7 +354,7 @@ static int iommu_v1_map_pages(struct io_pgtable_ops *ops, unsigned long iova, for (i = 0; i < count; ++i) free_clear_pte(&pte[i], pte[i], &freelist); - if (!list_empty(&freelist)) + if (!iommu_pages_list_empty(&freelist)) updated = true; if (count > 1) { @@ -531,7 +525,7 @@ static int iommu_v1_read_and_clear_dirty(struct io_pgtable_ops *ops, static void v1_free_pgtable(struct io_pgtable *iop) { struct amd_io_pgtable *pgtable = container_of(iop, struct amd_io_pgtable, pgtbl); - LIST_HEAD(freelist); + struct iommu_pages_list freelist = IOMMU_PAGES_LIST_INIT(freelist); if (pgtable->mode == PAGE_MODE_NONE) return;