From patchwork Tue Apr 8 16:53:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 14043440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BF2BAC369A2 for ; Tue, 8 Apr 2025 17:12:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LNg4ZD94IDIitY8vj+IFABlEolj+AplmGFJxU40zyyI=; b=wqF0NNFiZ0FXUfyEcBJK11RBC/ FJDxbUMGpX4vq9+iaoYkDxkT/OGTN91TNgbFwXJw6qqGjcCyY8Mm0DmMWBIDurBqF7eRu0Tdtanih aUwjAGjB8gNSo2pkjJrMYPOI03WMSVYZijT8aek3PPVpzUGm78rts0Nkm9jrVXozJPcCZHb1goosa 3d6A9QWgf7DFWDIIG0htS7EbFxdqk6YJqeDzYQZMe2LHnR20J/UT0h/812mrUK+YGrpIU00M5P1KL 4tu0YHSLIQblolYL3M3KraZN1znSghFMfOeqqvJRjk7/QONafVRENiCIlhdNuB4qIVr+1xaikhkyb ltGRjk/w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1u2CUu-00000004vuq-2HQt; Tue, 08 Apr 2025 17:12:28 +0000 Received: from mail-bn7nam10on2062c.outbound.protection.outlook.com ([2a01:111:f403:2009::62c] helo=NAM10-BN7-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1u2CDU-00000004sBS-12uR; Tue, 08 Apr 2025 16:54:32 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UkZkKkLl53iKPT4BGz1yleXqWBtyDHsiGhIGfBgvwnKY6w9XbblbTNRJrv3GBkCUH8sGFsxpaRkUJTyngjIQZKd/QCjZOfZKsjROXtC+C8ckLEgb5RxGv8bNb9nUY7y/FIJHUR8lqsJKl3TDn6jbZj1/eg2ykMIqkXZ0u4va/7kazpZZMYzq1ajE9uXaT/I/Q5Ionf+y6KbTW0FGYk67A/eXN0WOLSCXMbeLyDFq/KetsDYeMED3WQN0lLGWhSBfg8IZO3ZuALsSEHWPuBLUL1a4qlFLqewnWQdSpPhJiUq7aHDgwfs+s8zciVmAYiNcCM8KiID05nt+vqx2pgp9Dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LNg4ZD94IDIitY8vj+IFABlEolj+AplmGFJxU40zyyI=; b=oBtPpvteMU/359RUzwtDzT0N9cSv0DBKsR4Xd9HKw5ezHDIf3ADV58dD1b6YzytSrzGCVMKmxR/Idq6UJMgi9TUMhSBGthu6wY+4cWcXFYv+xhcTBTS+lUoo93e0y7ohhYOFXwpQqNTst/+Wx+N+f1qxebB+CCWEd65mWR8jpf533TLUEGfMQEcD2ZJ+WCHjxriKy0Ae1X6GBgFzuWXlHQ+p+zTSZRaCYMsexL92BOvDrVmZvpOMnDPU7DRaObPsCVr0upgw8HhdVUQu52tRwhQGa1VLGeyMxro/CIKGYLRE4MP6HkhUyjGYZAMadAutqC+sEiD0cNxCHfmiF1Mppg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LNg4ZD94IDIitY8vj+IFABlEolj+AplmGFJxU40zyyI=; b=oOxCsiYxxlXaYtXJ5D12E9c+d7ogD+pDbwMSUGqyqsPHgfZxCy+w3XrZT4GcBRUAcP4adGiXeICN3pBV8fMeAmp/8W1gCH03pO1EtJYKpO28rkLaui6ycXw/7YJqdu3Q7r2+fCkaFhD8gFYsHngkr1wn10qe/yO2cNPFjrfvE489h2a7STQ03VgR/67qCsvp1NoOMRo4QX2WnqJM5D3gATPcVlphmCz6hSfrZdpnTlJcS3pa2gL5BISfn0LbVe9nyfROQQhK2OzgoHvaGOh7qTMHTkqauuZSstnDu/VQ91OunxwKvQfnhXrYUlEzhKgSldhRodsM1E6B9//rv0sHMA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by BL1PR12MB5971.namprd12.prod.outlook.com (2603:10b6:208:39a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 16:54:19 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 16:54:19 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 04/23] iommu/pages: Make iommu_put_pages_list() work with high order allocations Date: Tue, 8 Apr 2025 13:53:52 -0300 Message-ID: <4-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: BN8PR12CA0032.namprd12.prod.outlook.com (2603:10b6:408:60::45) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|BL1PR12MB5971:EE_ X-MS-Office365-Filtering-Correlation-Id: 3b9b012b-0bb4-4b2f-2859-08dd76bdfe70 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016|921020; X-Microsoft-Antispam-Message-Info: 2HxTvJmuy7/6Hw1SomN0C3dUc2945goVY4aBoJ2lArlOusxyzUCdlwBTWh8hnDiBMGT2rxjml7ls8XC1BnltTdIQADDmJBqYjZNlDyRXP+RbkbgZGLwLPE4Yb3iHbIWkftp6CuL4rospeyxv7jM9eb+48+F1WHEDlcthaifve5JK6Lz99wm0bEaEqJj25Eid7d4A2QGxnf3vAy022Ag+AvgIby/XPWHJ0Dqu+b9+yF9HJtKtNZ958GXmD/fwXRdPnhus0Z4EO0YmoSytJctR6CdPfs/oJ7XYlFQjXEKuvJtjjW3NnbXB/FcZB+rpvToyKD5lIeI2qABl59YszMvIeu4qYWxoHT3df+CgwzpftHwOWXd7vqVXlw9D6t8LwHBEVN0fDvJukjAurxB3hcqMiqitu91ja4Wx4vwl/HJuyODXMvTpC4ghIZ/BVDWOTOIwWGOcVIi+s+AaDt7334B788ZOYnyFgy19ErdpKFMMC7VbB82YeDkaVWS85jzwxp01g2T/etd+qL4Zl0YpcMYlQ35kipMMn6fedNnQyCX72brRe7EYiM8Heyr86vH/Cyz0YoxiKIY6EiZ3ZkKwzWnf4yQh0AhKE8F+4IDmQS8fN/CdDWISFQt/6+riNBo2pnpYhFC+4IFupER3dG9l9d92dXV5aEjjwEbQGDoL4oBnfIBl8AuucmEHLdyZwh0LUI4ZAJR8TaKXFzE2wX9Ovk08ITzSl/cE6hHhT2lCIh7PVQ0+IXyKb7H1HxltsJlffbjefXd9lMFVUBdWIM+W9acilH9Melaje6C5FyABbuUvMP4R2AioOGg2h3Pl6pcxVXlwtUW0yrDudBdkNo1v25XxI+UnmETPEUk6zRUPUS8NByWbOKxfhS5z9idgfbld2nNGMU0QU5opklez536PxROevyvz4CkAneFjsIq/M6OovSka8sOIRMTGmWUlK4vv2PZ5sRU6SkVSO3UPEq1Gs2E0U00t0M0X6Ool5C135vY9TAOV/jLPu31AGvFJU1aR/Qxc5NMfnOFNIWnAbLlVbaqnl+LMo2KmO2aXhaPSIoJvyJgmgUTYb5cpSf5DNN83UGo2ES6a5R2aWyjXG695Vu2V6Cod88IEoT9c5xpjdZqu/XgMs0Pag4M1bzl217GOebG6OhLixcN08Mdj+mxzMEvTVmYh4fMzriZE5QFc3i30eAjlygUhzRKeQwiyzKAtQy0NpAKlb7e9iu8zdx4jjis8r+tPSbOMmWz3M9q/Y2M2b7y/5PNKvXMlcX6Pb+eZRDz+thzWEv0u5jGPM2eSwHczfZXcRKxJKoJgjmxjc/oNims4MhI6YTWMy+hSEFMUoyTE9u6kOfm/AqsY7KNoO8MLR/4Xvd/82hlcGIto0/Xeh9R+nTzEzcFyj0wUcVlp8PA1/KRlpyzSKKbpcb05Q6LwFKy9v9T7RrPpGJDuXFzu4/oBvCSDczIVDSqDWKsFdLMoq/lRiDeTdU6QxipNXKQNSQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: cxGmjKo0oPxu4yh6IZTDg0X4VYQSbcw3IShFghUicche71QK813ZU1jkJ0TBTvVTg9OsEFevHnFgxOmRrfCXme0vR/qp5ZGTZrJMVZfvUdqtPFlp3weBcezIOTpc1J91NB4CnAms1WgZqgs7h9ng0wlKW0ms0cAxjHCZF8QB+CZPlMcGaKThAy/f03WjsON44MP6KtQIibb0jQ6ZggoDlB2VALhz1fb+aVOGMrkKeFXOUL6bgY/Al47Yvmr/9V4klmLWMB5JN3l94lmFbAphxwF4x7Ho1ff9vMF4rEhzFbHwqnGjdR+ePbTopi+jcgOqdgPuSLjtnYu7AmfHLOiljbud7c/juPSBT4CShKmYhVq7Q0xYtSZxyIqvng3o44hX6xjQXDirYwAfyruwa8nUBlJ3SH4fSqtLWDJwsTSL8/DLmiF2Ct6NnR6L0jVlzXyeuaDmMjxXjUp8gyBZejx7GPLTCBTm+80NY7IQ7wKazjOuKLGgnHxjyqeDS++oWsZDt/G/NWKbM925M/dHygODgoO++Okw5OYGVDVn0e9KadCFUX0pAbExErX0LjB05KFzH36htecZMndJqY8CTS3JA9T45E98CVomxNYSY5n0DIY+GGdGRzFdzP9vsYqMiOU0IzvhZM0wjGy2bHChh/yi3wlL74/A8KgItR6iv8CfniPTO2dFQ4tzpMNouaUyiQU5I7RWY6uIw9HNDRaUEjKgpQNyPBHJm1RQdUmFl+MRx3tbvSCSwEFjLvR2pU8jio6dt3eM3L3UBFHPUiT0zWoyhbHSlb+4UX/y1OC1xAR27z75nw+fQ4XY3v3lKKBmCPAuRoGLh4T1SowzsMKzp5TcptD1jH0/4G+BF8DuotlKb3dNJeKcZ8fVnA5V7IT7rrKaX+2ULkHrZC3bJlLRmYPYs5P8noMn07ptwluz682NDWpXaP1opEFBDiz97jSelx5NV0Rw4T2zbwS2ibZheO/98DIUMygBfpMVdKRemsEc5KgBBDZPuOznOwSsqWm7hCGcii/MXqaGcG0hhxyE930ylfXLhsjLPteZri2EmrLIfbpVbhdkBoNEiDSXz6a+CPNzbDYex0XJlGJYUhEjLW+KHWKbgyGI0UgJsO4W/nnVp5mPHKuZRmd/mIFzfymyq74pG72yk1cgI6/Qm2xfZMOWRfMEPnN7SQHlU4up7ZUC+0hVSRTvoPlVSFWeia9Z41DAh+UfkAtowdXCWrgqiIdGpiNS/Qcw4igFU2cRqT0+xv9ZyxowsTMVFGlKWu01jWwuPL24S6rer66aUqenoD4FJikgCA/7xpPi7m919NZIeHcgT8ILBcJeINvY0Idj4+dEgzLDNURNYGYK62i6qQyPhfF9fcSJZCNoWZ4C7P2aHdM8l6WbGk2VdiL5EhTC6wYRWxERnPOTOqWdilNuX7XA09nzZTFuI+cz9NrUpVHnRIdxWlRVRUCbES4a/1PBR0tvPPqky3jWwwMEUBTwJcewlVKuFPNxcHQQc8pFn0a2Gu+BuoH5iAu9L7KWqYQTBrIRvZRuUV7lznH1d4guXKU9bJ/t93F395yNHa+czYmpsnaRMIDPGLE+Ysh2HhccdmiL X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3b9b012b-0bb4-4b2f-2859-08dd76bdfe70 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 16:54:15.2725 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: c5XeVOHeZ7Fn7hCd5qDQSaurQ3F5IXKMOwGtct1NXoArlLgOjDEJNa0v67ZO6BOG X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5971 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250408_095428_291546_BCD03712 X-CRM114-Status: GOOD ( 14.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org alloc_pages_node(, order) needs to be paired with __free_pages(, order) to free all the allocated pages. For order != 0 the return from alloc_pages_node() is just a page list, it hasn't been formed into a folio. However iommu_put_pages_list() just calls put_page() on the head page of an allocation, which will end up leaking the tail pages if order != 0. Fix this by using __GFP_COMP to create a high order folio and then always use put_page() to free the full high order folio. __iommu_free_account() can get the order of the allocation via folio_order(), which corrects the accounting of high order allocations in iommu_put_pages_list(). This is the same technique slub uses. As far as I can tell, none of the places using high order allocations are also using the free list, so this not a current bug. Fixes: 06c375053cef ("iommu/vt-d: add wrapper functions for page allocations") Reviewed-by: Lu Baolu Tested-by: Nicolin Chen Tested-by: Alejandro Jimenez Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommu-pages.h | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/iommu-pages.h b/drivers/iommu/iommu-pages.h index 0ca2437989a0e1..26b91940bdc146 100644 --- a/drivers/iommu/iommu-pages.h +++ b/drivers/iommu/iommu-pages.h @@ -38,8 +38,9 @@ static inline void __iommu_alloc_account(struct page *page, int order) * @page: head struct page of the page. * @order: order of the page */ -static inline void __iommu_free_account(struct page *page, int order) +static inline void __iommu_free_account(struct page *page) { + unsigned int order = folio_order(page_folio(page)); const long pgcnt = 1l << order; mod_node_page_state(page_pgdat(page), NR_IOMMU_PAGES, -pgcnt); @@ -57,7 +58,8 @@ static inline void __iommu_free_account(struct page *page, int order) */ static inline void *iommu_alloc_pages_node(int nid, gfp_t gfp, int order) { - struct page *page = alloc_pages_node(nid, gfp | __GFP_ZERO, order); + struct page *page = + alloc_pages_node(nid, gfp | __GFP_ZERO | __GFP_COMP, order); if (unlikely(!page)) return NULL; @@ -115,8 +117,8 @@ static inline void iommu_free_pages(void *virt, int order) return; page = virt_to_page(virt); - __iommu_free_account(page, order); - __free_pages(page, order); + __iommu_free_account(page); + put_page(page); } /** @@ -143,7 +145,7 @@ static inline void iommu_put_pages_list(struct list_head *page) struct page *p = list_entry(page->prev, struct page, lru); list_del(&p->lru); - __iommu_free_account(p, 0); + __iommu_free_account(p); put_page(p); } }