From patchwork Tue Jul 18 00:49:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 13316651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAB93EB64DC for ; Tue, 18 Jul 2023 00:50:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3327D6B0071; Mon, 17 Jul 2023 20:50:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E1EE6B0074; Mon, 17 Jul 2023 20:50:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 135128D0001; Mon, 17 Jul 2023 20:50:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 021CC6B0071 for ; Mon, 17 Jul 2023 20:50:20 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CA2CAA065B for ; Tue, 18 Jul 2023 00:50:20 +0000 (UTC) X-FDA: 81022901400.11.B10EF54 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf15.hostedemail.com (Postfix) with ESMTP id 4ED33A0012 for ; Tue, 18 Jul 2023 00:50:17 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=q1HFfJbr; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=BH7ASMRS; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf15.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689641417; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tT5JR7OwlptdMOMpxbuk0zZhbMiMcLEjli2VWIuDKaQ=; b=rpWrY6WyKIja5fTI3oqRdCTXImeV1XwCCznDFP1OPSU8A9w9bwkkjAE9gW41oJt8pAn3Dj Lva2PF44GabHKUbl/DhAQtRF9xyWs3r50Q/swCwgcVAs5ZWu+YsQUkd5uINwWer4XJykJW GG5LaF9KhqYjh6jj0sBTaAyL9QcEAKI= ARC-Authentication-Results: i=2; imf15.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=q1HFfJbr; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=BH7ASMRS; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf15.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1689641417; a=rsa-sha256; cv=pass; b=qXEQr/93Sl3LhSDbPmF5sddUOXaiBEfgLbAfjJtlx3eyczS3AwVnRDWfqqh1T04a5jfLtu /Tp8PBMAFwQHQ6v6lskTnJ/4RGGgDxbUDKfgwAr6jND4n6JXLsAOtZhpvtouTqQo5eJ7PQ l6n3ifvCNfeDAmJlsVZQZH9A5OXuNsg= Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 36HKOMtW026841; Tue, 18 Jul 2023 00:49:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=tT5JR7OwlptdMOMpxbuk0zZhbMiMcLEjli2VWIuDKaQ=; b=q1HFfJbrz/FdCFR/hRyyA5D1ZnH3yOohzEc02qkN/pbmTyBjQrzIMKVlKr3O5aeaEQDI J33WlTilHn/1pFXkLDPvWWwLgCb1ALy+Quy0f1Wc70YyuxEWUroPoqlUjVk0J0+5IkfQ EVO1g6dWJGlrk5VRjYDKFnWL5w8i4PaYIK9xSwnYqovQuhkCjmVLiPwrdHA7SUX2Cyxn GaoZbvAlIkEBa/567zLL6tNvxo1qhk0Y8OB9YXnLqYxpF2jW9QvB739CHwKUtRgd58Gs oim76LEJEgsuHdypNUIt7OT0ePeEtUQnPbdOriNwUuxjDCrqwPgfsu9wlnoYsB1ObNDx Hw== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3run783w7n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jul 2023 00:49:54 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 36HNt2OH000934; Tue, 18 Jul 2023 00:49:53 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2103.outbound.protection.outlook.com [104.47.58.103]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3ruhw4dbfs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jul 2023 00:49:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VmucW9KiYab4lI6MzLMuvVs9v2IWamfFkHC0EJU4aoAVmOTSi7u3TlLZ3xu9Kn+duL3JbS1R8cLjSKawvxGdrqWaq+e8NQrbj8Iq0EPlljaQVMUfTwL7PhZ2srtWjoa8NKCPiElzvsjBOPrQ0jGGOjMnLwxBR3XzqZOGZAno3KtQk2WrHEcqwAztBF9gB+qjeVqJyVJFvhrxYlbz2XSC9fqVaWogjy6IobwqZ9mfKIXqZ9FdNhPOV2X3fphPytL5RJInKjskGYXBXx2yylGciXUxthVYbvD58JQWDkxeklRKnhePggBl58xXeO4d7rRqEoBkqt/EtIPd+80Flnt6Mg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tT5JR7OwlptdMOMpxbuk0zZhbMiMcLEjli2VWIuDKaQ=; b=naW7CoTnVyR+Xo9qNoKZTWKQlUGP2h1L5x/nnAqDMJTZT3DDAQ8Z0opDMaZbZ8zpZaGBttldcGxnD9uy6M4WW+w6/mmbpgJ+Bvlxlh7FRhwLjwUlZz2geoUXnJL21aXCfo7AlWQjKcMfE2oRICpt2TDmzahrCcEMIx8QwxSXNkt3WP+Q3g+PvoM8urc5pQec9zrMpVlKrO8g3ugFLXi93kR5CUxkZQHpRKQkg9vOsZk/9V5B4xK46+HaY0A7xhWWSbEP7LjuqoSl+fIBwl/Zaqvex6hPY+10Qjh5xylL3Sfh9ZsACBulpJp7vlcyse8jMK8kbV5eLTzJh+OSTFphxA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tT5JR7OwlptdMOMpxbuk0zZhbMiMcLEjli2VWIuDKaQ=; b=BH7ASMRSSaDemXQv1jWEpjyIs974Nja0vu8yZEtsNGghUzpVB45kBz71HYBdXG0pr5tHDwNq0K6ERyrQIBCK5PeprCXsuqJDIfFLiFBziSGINY3lPVL56xAypjmc4o4E4cmHeI1FIUF524jSaHnrJ/bGxlwsO8/vhMh6IjKwJOA= Received: from BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) by IA1PR10MB6193.namprd10.prod.outlook.com (2603:10b6:208:3a7::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6588.31; Tue, 18 Jul 2023 00:49:50 +0000 Received: from BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::4a17:13b0:2876:97f2]) by BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::4a17:13b0:2876:97f2%7]) with mapi id 15.20.6588.017; Tue, 18 Jul 2023 00:49:50 +0000 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Jiaqi Yan , Naoya Horiguchi , Muchun Song , Miaohe Lin , Axel Rasmussen , James Houghton , Michal Hocko , Andrew Morton , Mike Kravetz , stable@vger.kernel.org Subject: [PATCH v2 1/2] hugetlb: Do not clear hugetlb dtor until allocating vmemmap Date: Mon, 17 Jul 2023 17:49:41 -0700 Message-ID: <20230718004942.113174-2-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230718004942.113174-1-mike.kravetz@oracle.com> References: <20230718004942.113174-1-mike.kravetz@oracle.com> X-ClientProxiedBy: MW4PR04CA0120.namprd04.prod.outlook.com (2603:10b6:303:83::35) To BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR10MB4196:EE_|IA1PR10MB6193:EE_ X-MS-Office365-Filtering-Correlation-Id: 7ab88ceb-2a52-4419-6e5c-08db8728e490 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GBg9FEGhgBAXxC4Gx9Jrynj0thSCmA6UW75IgmhDfsIbthBsxGONxHa5VffS+w57Q2qDY1bPUKmgGM4/AILbNBLV2ycPGftl1f/M8VZu+0UGkh4S94Im4oboDHFQHUcLWuHL6YneNK4udAwiDRD77G+ILfwDN2mW5FKUqzpGexcDfIkNI3x8vk6zw62IL4QwtGNiixUNh8n6CBG7RF48dvypam28IHA6e8IDQWxoolbDQ+4J5RjO/6n0D/PzfyB1RyyGgAtVL4LKpO6jakT80LdiSscGNwg38PlFUk6ynu0HpNO24+9r2agIMiPAipxrpGqzjZk0hb6HtnPdpT46NvNgdouDMyz+MIxYlKoureuN8ikzZLnlzY/X3ngPQjV38NGTDZonCcHGWLUPBr5JqTmHo+BzHhfOx5egyl87aFzLHuOWzRgavAj4tbcLl8ZvEn9g6ROBHLBlJ0k5icvZKNJM2OWZV+T5pGh9QjFYoAC43nLQP+vXd6KXneAzgFJmghXeptpBVHyZScPIyZYp3DGYKgTkFfvHBLyDhYzW+ridH+Wl9rHCe/ERlXV2AxT2 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR10MB4196.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(376002)(396003)(136003)(366004)(39860400002)(346002)(451199021)(54906003)(6486002)(38100700002)(6666004)(41300700001)(8676002)(8936002)(5660300002)(66946007)(66556008)(66476007)(316002)(478600001)(4326008)(186003)(83380400001)(6512007)(2616005)(26005)(6506007)(1076003)(86362001)(36756003)(44832011)(2906002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Xm5PQryyaKrFgg/4O9Bt4azqAxh4xChnJrX36auv8d/ryjsNaeJgp5wpiERYYWj0KzLx0+v7g3nGT9a5f1MpZ8Gjmx6FFhyuOPucVbQfdFMPY9TWF7J7qBuGw1etDv/Kyxo3fU7hKx5gtP359SGU/EriSAlCWre7pTn82pTSmE4OqU3+bXqCTP4wbYRCrk+DWENxTMAMEtDfYCiC1/T60V8AP3hsQQKcSDfGTcDRke+9ppRgeM499wZOqjtxrzQERSWi2otF2aSh+7hFBgokanKsRr/6PRARU4sE61mMt4qiyKGw93umIhmTWmIN8MgADjMhLePZ/vkK2BuaL+y6o8ZFDG6boCWViTYq5CnqB+SyPCogUvRbZgCRgcZmZ3ZJZgVWPoZUkCu9sdWttVGeFvEG1HBJbH+a2/EsJ7c8+2T/PGf3hVWvPbA0NtzoctT4wtsMdRuKfiN0ArahmsoOTLyhQB3El0WSWfMueWStBNiqB0Fz9yk8AnJmWHusxJHuF0UZm3zwBZH337ZqpR6xck9aJMDvD8779i9n+1Hnha8lygm/eUYMxKLZJnFrkoUmwRL/futlGImJR80pHirwwmLyjshgliOWpf+IJ3CSb5kEZxbE24TeBLS4BrIup3e2U4WFNAA8aoydgPkX4fskfZfuqZiJ69j92w6JncHZWsPKI8jKWgfAQEl36vJA+4yqGpmfxjUUA/cEgZO/8QyKfiwo8Z56SMA3V73h4favqPG/SyNCXcTXf+dlwz03+PHmzt8D8VQroZSvRZp3RkYa6Gt8r7GUv8FGHWLhIK9Lx44LEzAO3aQolYvjhdZZ0y3bB9VHRbY7wZHyGXAQSb/WYxYR0S3XnmCmnqcCb/33E2WDIpKDRbU24kLOKAL3cxjqNUGwPYJWHfEYLznss/WCli7AbKtk8uu0VLdiRTegUliu3uWfDblas/Qz0Wq4RLuf7jpbk1Co6P8i0plI/l0oC9t+/hLmZyLvHIZNlZAg28ryw8AArNtWn0jA49HWF53S5SY1wqehpXHsENxLYg3Dz+AOCGfYlGJqIE4JVXF/dUjevKs0jCMmWfFwcXAhbAwMXPt1Pak9/wgEYXgueDVcfaVr8MZ/AGTCa5GDz3bsX0ufU/tt34u4R6J/4zp6XWcFLbPgiKALd8lK/jC2A1dB4YYmW1balNp++Ll1UvkR4TKcAYoj+yv2UllQAf2fuOcwhNZvqGPBTA0Jog97ipTh9jlU1G5vldS9vi2cYKVrM/QN/Pz1jr052PaRLNM5fCAwaj+w1awlv3R1rgGEaDI6Qc7UGJPnlJY/kmaI5z03/OvHUO0ePFC7IgHInUUktcFEn+PF/rRs2vmPjQL5b+2hfuiHoTQhFZKe/KtOXdk9A3RPXhoAxryL6XI8Mznm2fHOYchjLam83RmSRQuJdPaUG8p7RHLpBNsrzcYAz00Our13maBmktTc2xaobcxjNCPagduDL+4qu/SCu/zBIXmvSDfk9WOgWUYLiO1f6WRZNshWtCP2j9jFN33oriuGz9byvMQxGfP87i6SZ1+obKwjkqkHwUzWa5g4sVTMbMLW3mAqSg55q9jyPogazBIhh7qq6xnvgixR1Q67KVWm+TRfKQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 3NYU9MoVc34rAiv8s32eq0zZyn8IMLfAdJG8IEY+Bx5cQpbW9Udq3OxRHz2hW+hewLT7cxsU/g5of5cJ/GK8/iByC+I+bAr/N9J71z+VF8Ls6O6T2oceoh8PJs5YwiF4S9uaN7VBcQVLXxqBs3ci1hnFXE2Hpr3DuN7Oiud8ikIquA49hNsdNTpIl5cNOGeGxz01jkttcxl4L+wgJ4SNcQ47tOsmdE9GJcwVbVAmpYd+WVyoYj13DlprOzJX7FYoR32Nq8YYjEp8nVVLkcFBAFzi74PK/o6SXDpOdQNlDzM5LiH71HX/1keFgx51vkWxF47UBRP++R+DdT+eew48ytrCDjcOYnh7Pkj5xcw/PNhMaImJyjW9KTJsaNIh9kZ2LoGhThBCe+eg8b7huP7oK/Q3lTEEczLWHEwZU/muKb4TOy3G0c9xiXzIOPupcY9lIG7/OQ0VrGFWyEMVVm78itgTxk5gyc3W6u4uGEVliQF/wgj9byoCVO6rRGOjIneQLwsOI1bG+/doS4z3oBcLMx+KUDWgNYjwQ5z1Wcq9vMeuFLxNLgbw9WWk9eelqpweW/OYvyUl0ExnYfHatZCDZrgSg6rlVMppAvhEWUr4nJRapjeuCSJdzpnYrrDWTwO+EdwGFeGgcTTRLgebhiJzSuyHc91B5VlRE61oV9PXrK5TUUvLMY0teaMHRtIu1cZ1HACaMGZPXTX00cN8umAU+adVC1AGS7Uc135cqbWYsV8bEjkoTrDJkeKHsgxJxv3fiesB7TzSH5AuVbdwX+La2Q5JLhtVCZq0dqJtLH6jh9VqAtS8zul6kuqnKmuHSSjyQ5YXXUOJDkvJ10KtT1mtxjSWmN3lA12KkH+2FC1YwH9jvj4pNnX+veen7K0LMUZu2taWRcMsiKBJM1IVqDEGF7B2fFpnuYqiviqnjTd3cucs21QyWuWc+4LwK9CQnYywtCr1/3J02od284xqF5QL3zneg0hT3ba0IGReyVnTlpQ4BONDJgb4msxAf18yi+6l X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7ab88ceb-2a52-4419-6e5c-08db8728e490 X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jul 2023 00:49:50.7363 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: DwkMwT5NJ9vQHz0x5AALAC36tVXZLu4eZ2wQA9WkGAD1pmUh2uW//GmYdSCUtmK6dX2fvB98kZLyOXwVdqKXxA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR10MB6193 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-07-17_15,2023-07-13_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 bulkscore=0 mlxscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2307180006 X-Proofpoint-ORIG-GUID: 5CrmmDV1sUbe8Y5OZeWmg-5FqrMKGG0J X-Proofpoint-GUID: 5CrmmDV1sUbe8Y5OZeWmg-5FqrMKGG0J X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 4ED33A0012 X-Stat-Signature: zg15wxgm1q3bb6xd69i5rnjqwqcqk5cp X-Rspam-User: X-HE-Tag: 1689641417-822230 X-HE-Meta: U2FsdGVkX1+5+JJmFckMH4GaXXq9BbB3Rfp3qSzKF3ekQIxpsEqJjaA+f9j0EwcnXPuAIuExjbuw9cl55D01wX2IJNcbl9scSj+vU0JqczLfrOcYPqT3prjgsWWrDzFJCGmHaU8us+5WaDJ0HRhqEVI+z+Wmx/ZkMsUA2tgYTCwP4GwvusTeIp9m6vVV1rKI1rmOmIvqOl6kIOtw6K+5mVPj7hB6o2hFSBoFRqFIlkJ9OVhHJ5zu5j4Ar58x+7VqK9grTw9J0fzs03yCwt5aKVsSUc+FvWhHX8dbebE6cgA1vwb6BXxg+Y5b+YYjZD4piTgdVM5POeHE3zqU5Zgb7rpIHjbe0viKaVtQeX/UsuN+7sFeOaXXePrPSyCcHZ97ypX0s0UuVvpRZItzUSHJzhubkryrjpbcSFx+h4mo2vhyqScacbLnyKpvdXaXnORMM2AvosWnxo/L0xOTjJICvJ0Nu6LGVITaKH5czN7rAmGIf4HvjWnecMc0yuHVt9R5YOBK64fYYFDVkJxMGyxfx9EktNbweDuo9rKJNmK6Q9WXZ5aqcwLJrVyCiN7TgHYu0EEV2dcbFq3YmhcE8jp82heUjf73QHlqir3VudtdrGSckHMYHg+Tk5y16uzfDlOI4rQIlhVPUyCzVZy/dP+pLbJonNCxIi3gOcHrnh2LnCT48uHHlnyaw1ZPrvUeM8xqmZiPfEbnjI6ztkUx0w4z9gTy0Zc7GrFs7XvmmNxznOMNBzqcb73Ca16bpRMHD//hUZFStTpn6lTrQlsTOLfGYdUdthixiS65RHWndgt7xJvb+xzly+0sHBzsWGajmMmcexpd2g24vYwTANdLlkB7/7G7BrkUGmmxmbW+gBCNaAx/7kfdoW7NPhLZxIb+dqY3JwTm3lysLh/Gh+kn/5v4zFSIlOFsRmH8sSc04EFIu4Iu9uX+Hwjw2BzE4wtXgwzX75iFODEvKfmf99K9ain wTA7yOP4 Mfs6JuYTw9CyN53shOHw0e6xYP/avy6IhVG9CN3eTQKVcRlggUkSSvJF1Co6BSLMXFyzSt/qWrK7tTzxXRPdRp6pa8xhmeSSwSAV5WaL+qbV+cRqtTFs8hOstIQE3JWTzfQ7HiyCl8G1dav9qN90D54JdbzhHMgrLvqZ+tvNro8611wW5iQWGwyf0QmSCl+AgVnwqw5NvsCzOxUyvtdIxc2Ff+ik5DC3lH4YwzpGSQmAi7se0VUQUctb8JKLdcaw9/s8m1GNkvthP3VA4fa8chGf2G+0nooEGtfTQAZGrbon3pFvJgVchiB+nle0y0EBfw295vHzj9Zapp/EMKf0lpqBGvcrCXdyTvJaEl3EWJZFnY32R9IS8bbhZDdFYr62qC5ZxLVdX/VDpQTxIqb8m2Ibt+6VH4r2/Kt2MehdJZ519lY20huE8XJ7fgQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Freeing a hugetlb page and releasing base pages back to the underlying allocator such as buddy or cma is performed in two steps: - remove_hugetlb_folio() is called to remove the folio from hugetlb lists, get a ref on the page and remove hugetlb destructor. This all must be done under the hugetlb lock. After this call, the page can be treated as a normal compound page or a collection of base size pages. - update_and_free_hugetlb_folio() is called to allocate vmemmap if needed and the free routine of the underlying allocator is called on the resulting page. We can not hold the hugetlb lock here. One issue with this scheme is that a memory error could occur between these two steps. In this case, the memory error handling code treats the old hugetlb page as a normal compound page or collection of base pages. It will then try to SetPageHWPoison(page) on the page with an error. If the page with error is a tail page without vmemmap, a write error will occur when trying to set the flag. Address this issue by modifying remove_hugetlb_folio() and update_and_free_hugetlb_folio() such that the hugetlb destructor is not cleared until after allocating vmemmap. Since clearing the destructor requires holding the hugetlb lock, the clearing is done in remove_hugetlb_folio() if the vmemmap is present. This saves a lock/unlock cycle. Otherwise, destructor is cleared in update_and_free_hugetlb_folio() after allocating vmemmap. Note that this will leave hugetlb pages in a state where they are marked free (by hugetlb specific page flag) and have a ref count. This is not a normal state. The only code that would notice is the memory error code, and it is set up to retry in such a case. A subsequent patch will create a routine to do bulk processing of vmemmap allocation. This will eliminate a lock/unlock cycle for each hugetlb page in the case where we are freeing a large number of pages. Fixes: ad2fa3717b74 ("mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page") Cc: Signed-off-by: Mike Kravetz Acked-by: James Houghton Reviewed-by: Muchun Song Tested-by: Naoya Horiguchi --- mm/hugetlb.c | 90 ++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 66 insertions(+), 24 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 64a3239b6407..4a910121a647 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1579,9 +1579,37 @@ static inline void destroy_compound_gigantic_folio(struct folio *folio, unsigned int order) { } #endif +static inline void __clear_hugetlb_destructor(struct hstate *h, + struct folio *folio) +{ + lockdep_assert_held(&hugetlb_lock); + + /* + * Very subtle + * + * For non-gigantic pages set the destructor to the normal compound + * page dtor. This is needed in case someone takes an additional + * temporary ref to the page, and freeing is delayed until they drop + * their reference. + * + * For gigantic pages set the destructor to the null dtor. This + * destructor will never be called. Before freeing the gigantic + * page destroy_compound_gigantic_folio will turn the folio into a + * simple group of pages. After this the destructor does not + * apply. + * + */ + if (hstate_is_gigantic(h)) + folio_set_compound_dtor(folio, NULL_COMPOUND_DTOR); + else + folio_set_compound_dtor(folio, COMPOUND_PAGE_DTOR); +} + /* - * Remove hugetlb folio from lists, and update dtor so that the folio appears - * as just a compound page. + * Remove hugetlb folio from lists. + * If vmemmap exists for the folio, update dtor so that the folio appears + * as just a compound page. Otherwise, wait until after allocating vmemmap + * to update dtor. * * A reference is held on the folio, except in the case of demote. * @@ -1612,31 +1640,19 @@ static void __remove_hugetlb_folio(struct hstate *h, struct folio *folio, } /* - * Very subtle - * - * For non-gigantic pages set the destructor to the normal compound - * page dtor. This is needed in case someone takes an additional - * temporary ref to the page, and freeing is delayed until they drop - * their reference. - * - * For gigantic pages set the destructor to the null dtor. This - * destructor will never be called. Before freeing the gigantic - * page destroy_compound_gigantic_folio will turn the folio into a - * simple group of pages. After this the destructor does not - * apply. - * - * This handles the case where more than one ref is held when and - * after update_and_free_hugetlb_folio is called. - * - * In the case of demote we do not ref count the page as it will soon - * be turned into a page of smaller size. + * We can only clear the hugetlb destructor after allocating vmemmap + * pages. Otherwise, someone (memory error handling) may try to write + * to tail struct pages. + */ + if (!folio_test_hugetlb_vmemmap_optimized(folio)) + __clear_hugetlb_destructor(h, folio); + + /* + * In the case of demote we do not ref count the page as it will soon + * be turned into a page of smaller size. */ if (!demote) folio_ref_unfreeze(folio, 1); - if (hstate_is_gigantic(h)) - folio_set_compound_dtor(folio, NULL_COMPOUND_DTOR); - else - folio_set_compound_dtor(folio, COMPOUND_PAGE_DTOR); h->nr_huge_pages--; h->nr_huge_pages_node[nid]--; @@ -1728,6 +1744,19 @@ static void __update_and_free_hugetlb_folio(struct hstate *h, return; } + /* + * If needed, clear hugetlb destructor under the hugetlb lock. + * This must be done AFTER allocating vmemmap pages in case there is an + * attempt to write to tail struct pages as in memory poison. + * It must be done BEFORE PageHWPoison handling so that any subsequent + * memory errors poison individual pages instead of head. + */ + if (folio_test_hugetlb(folio)) { + spin_lock_irq(&hugetlb_lock); + __clear_hugetlb_destructor(h, folio); + spin_unlock_irq(&hugetlb_lock); + } + /* * Move PageHWPoison flag from head page to the raw error pages, * which makes any healthy subpages reusable. @@ -3604,6 +3633,19 @@ static int demote_free_hugetlb_folio(struct hstate *h, struct folio *folio) return rc; } + /* + * The hugetlb destructor could still be set for this folio if vmemmap + * was actually allocated above. The ref count on all pages is 0. + * Therefore, nobody should attempt access. However, before destroying + * compound page below, clear the destructor. Unfortunately, this + * requires a lock/unlock cycle. + */ + if (folio_test_hugetlb(folio)) { + spin_lock_irq(&hugetlb_lock); + __clear_hugetlb_destructor(h, folio); + spin_unlock_irq(&hugetlb_lock); + } + /* * Use destroy_compound_hugetlb_folio_for_demote for all huge page * sizes as it will not ref count folios. From patchwork Tue Jul 18 00:49:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 13316653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2E8BC0015E for ; Tue, 18 Jul 2023 00:50:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 41E286B0075; Mon, 17 Jul 2023 20:50:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3CD418D0001; Mon, 17 Jul 2023 20:50:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 220066B007B; Mon, 17 Jul 2023 20:50:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 143CE6B0075 for ; Mon, 17 Jul 2023 20:50:28 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id CA57812065E for ; Tue, 18 Jul 2023 00:50:27 +0000 (UTC) X-FDA: 81022901694.24.FB9DBFE Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf30.hostedemail.com (Postfix) with ESMTP id 6BFEC80014 for ; Tue, 18 Jul 2023 00:50:24 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=MlWCxkzX; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=ONQlmkrq; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf30.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689641424; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZgV7qJ5Wq9AaKiuOJNgLUYIz8E1B+1MkUqffdohg7SM=; b=ByzyobhbW0odRL6FuONrzLC8Nx+BqQdbDHhZrIBtqIpm6czAHVAVG6G+dnGmn1KbGlgeGs bh6zaVTWfXDnLIA1f7n0J/UhcQgMg70OYdgoXXX6ThLOWDv5iM+4bE3e2M3X5YTIymgWcK EJMduzwgpuO4HPjx80jXhONFU+E5pVc= ARC-Authentication-Results: i=2; imf30.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=MlWCxkzX; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=ONQlmkrq; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf30.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1689641424; a=rsa-sha256; cv=pass; b=EmUHvyBADB/YEPa4ZdEuy6E72FeGWALZeAtcR+60F7YO7tzXbSzeuf1eHdIEmNo4G7RKBc yCvyGsGTBAf7xDMlgbOwRsC/ZiM2B5UTrmWS4cWUk8q86+OdB6yjtauJuBleCC87olx7cy HVsAfupobiTrvRtnveDCYivNW+Uesxo= Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 36HKOL3a023700; Tue, 18 Jul 2023 00:49:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=ZgV7qJ5Wq9AaKiuOJNgLUYIz8E1B+1MkUqffdohg7SM=; b=MlWCxkzX4YDQJmrpp7+xKoYrbHnz8jJ6+BvO57qEhd/1X+AwaP3EVACY+hFcb1RC/moM 1wggJqacg13/V9BY0W01wXK5woluj2C9gupKTd7MRSnUMoTqAGu2Qw4OOb2sJQcA0z7y aTyE2nLiUT2VsvCXOZ8IVSq+M7ShRLa8rphfWdLPVtjJqbtSZmOJlIXRR1GtszaMcMCG os66GlVCwV0/3YGfDLx/pJWU9Dhl2wiH3v/vUzIrmR5NJOLBFpqMzpYNsb+s2rhHsQ3g ujlh6MxfFP4UnercS1moL087wl94YsSjW1AiZTUooKCvasTITZgbXVu5Lr+5bq1BgFZ5 YQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3run88ky6g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jul 2023 00:49:56 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 36HN24fc007757; Tue, 18 Jul 2023 00:49:55 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2101.outbound.protection.outlook.com [104.47.58.101]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3ruhw45dqm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jul 2023 00:49:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eHKucMnXopTCDhuXXPr+XTd87+Kp0ayLXxDMDFI8Mh/QarplmsumP318CUKTB7ggcOjPOEGGB3KQLOTzX+MrARf2TF6f8UJFX08ZfNa5dmQid0AzDk4BAzRDZQNxBZmBcrReO7MdjvXfEMMv8fzfyIzK5djtDTxzuaOx9ztQhmfBlUEV+QCG1f15j+WU8H6nVGJDLHLpJsxvQCYZdAu4yTiVJi3PPfnJjXi8EaIo7f5JSbHc617uIJMd+m+VuI34bc3FzVtobBQ1zs79a77gX3J7dMfdiXDY/WvJ1uvPPMC/43rcDGe9ZqcfRxABH/UZEieWvWEi5JwM8VScJa9RmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZgV7qJ5Wq9AaKiuOJNgLUYIz8E1B+1MkUqffdohg7SM=; b=LFmHjmWFcQykli3VcAsf/UF7Jxa9qtaS4vTG8Fu6r/Pc7ogJ8NQSsEasLvuaoQd4NELWROjn6YBsi8hAvNWyIjqhi17dSvjrF9kYDN4v11tk71UCFl0EGB4ZHNrpUrcug4l4etk+aNlbAGnbsMhk5Jht/ZeV2bFEBY/+vxqT8Mr2EBwI2b0lNncmbRdernUl+DKX5YomFWV4b5Gjd8nI5AiJ9dTs94UEHshK60TKinnzSzBpUC/eoVpJBeEYZba3Kt29jw2XXtUXO4gULDtVjjpYorm3XxRY/jJJhJgqXm9PeiH9Zgk/v9Shyp/ryiKv619AkN3RrR3HkZX8XqpshA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZgV7qJ5Wq9AaKiuOJNgLUYIz8E1B+1MkUqffdohg7SM=; b=ONQlmkrq8Hsb49WH/qlMrRs0If4jjIDKq0lBHnFCRczwPoEO/AE6W1vCOzZdBdTYtLhQXvj9Fqz7k8+NiYI0hhHCXVj+uJOEqtaht0AyZC87KmvgHHTY6oDAzLtXXnyfQIFK4CQhuETEyI9ePWZLtDH/EqvFianBkjPB2bH3mXM= Received: from BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) by IA1PR10MB6193.namprd10.prod.outlook.com (2603:10b6:208:3a7::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6588.31; Tue, 18 Jul 2023 00:49:53 +0000 Received: from BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::4a17:13b0:2876:97f2]) by BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::4a17:13b0:2876:97f2%7]) with mapi id 15.20.6588.017; Tue, 18 Jul 2023 00:49:53 +0000 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Jiaqi Yan , Naoya Horiguchi , Muchun Song , Miaohe Lin , Axel Rasmussen , James Houghton , Michal Hocko , Andrew Morton , Mike Kravetz Subject: [PATCH v2 2/2] hugetlb: optimize update_and_free_pages_bulk to avoid lock cycles Date: Mon, 17 Jul 2023 17:49:42 -0700 Message-ID: <20230718004942.113174-3-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230718004942.113174-1-mike.kravetz@oracle.com> References: <20230718004942.113174-1-mike.kravetz@oracle.com> X-ClientProxiedBy: MW4PR04CA0117.namprd04.prod.outlook.com (2603:10b6:303:83::32) To BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR10MB4196:EE_|IA1PR10MB6193:EE_ X-MS-Office365-Filtering-Correlation-Id: 5b0a0528-001c-4cd6-4f73-08db8728e5e5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7muyrlVdGiGSoIwzPME2poQSpbS6wzNvMgQAw4qHdh6eKsrn6xNBzAKlbnot6RLfrrNalsc5fAYPToXGNsh9K3T1ReKFYDiVMsf2rKEwMkOLUKtgD4X4d8io25uzLppTLyBD865cD6XQ0CrwCMsXlinQa6HBa/+IL79LF9JSLIiih8XEGhEiy/H0nvA+D7cpjR+vV+bIaiet4NBEbxLw01/av5UTL/EhQj6YMHKPMp2kNURo/qs3e96GKToMqmrbnTcjnueIgysbX1vVlgHzLclwHaUKWcJCbzgXGjEl7ZEVy8YlqxbKa6cnWYDRFdAuba/pdDruWBfRAlLfZSYRPfXviO3SxiHWH5mF2Nh4jNqNwtHxn7369/eEL/Pm/7qKASomvKqEGosG2yuQ+AvMdIF59JV+E1DuUt1Fo5cjJamM8sAseDaskOIHSwEa4l0qlAHHfqI9BWDO+OmTsBxfPePw5DQ5URSimXalZ6IgyRuKr/8Mmrkx9cHIrst6KUZtSU0Fy0dgkglWIM5VOF9zcBZ40p8Oa3Om/pt6Gwq51wLVg0vKlURP5zmOkwWoCXKo X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR10MB4196.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(376002)(396003)(136003)(366004)(39860400002)(346002)(451199021)(54906003)(6486002)(38100700002)(6666004)(41300700001)(8676002)(8936002)(5660300002)(66946007)(66556008)(66476007)(316002)(478600001)(4326008)(186003)(83380400001)(6512007)(2616005)(26005)(6506007)(1076003)(107886003)(86362001)(36756003)(44832011)(15650500001)(2906002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: bMmnxjIDcqNnb/iYJoQ241b+5iEnY98WoX/AKOg0gPnYQM6U962+6DW644gYBVA/u7KEIsjWR3IgMkH0a96lfNC1Drr4ORuJrMVFnw4dXXJ0ZokgZtavh1XfKzei8mAUGzFcfXrO5N/2cohfC2skoihlVBsEt7ipwWkB+L75VGvKdckeqdFBBogb4JUTmqW976CivQDy3kLKinNnzFjS7tVzLRPabvQ0ZeudQJWtfTiWx+NoYjX29QD7CcaUyl0YToXjl20O1/gXWoIPJatDmK4A9uGRCaAURuRS2Neo7gr27qHdiEyHJs7j/drz1FzhU3IKBsw9FZzAU8bhldrdRp4fTCwZatoto8mlInpcY15+gJExdDJ/Y54DYtqwPwbnueK7Eg6zI9KOAcpb+0KzIqHMSmedKELSOla7jwUQ6vZjxAfw2MDEpS7M3xG2TcQs074QcIjTUaUtxKpGuHxRySk0G5MWxR1DcFnfuOBWep/oJNdpIxhw7fiy+CUBXL873h4TB5tjewuQUoLQtM/HDoUthrpiXoVpsfED1LrHQZdkoV+TFv4R9XLExz32z16Bd9TJUIxR6O0ujlhDCEOWroz2qh+SpE4BG0pXELa3Dit+zWycs34tJ5BLDKUsrWwmoM/BezqCc5AcGQDIpjau171Z81L2UUwi5UzkShHh+yDnag2DdAjiaA0+D+6n/ZCei36BR9EWVASR0j4Ud0s+/STZqp1EJdjD05Nht4kaKqvgydu2QWLc7ZkXKUB/lln4T6wg0/GPVnobHVKE2uYJF0Ps8US9xtduDbFZxTDo/l1A+Y9KloisNZJngw/r6ljQ69oQ63zfpqVLkuXKMxYTiHO5poD18sGOCYYcJDI6Eu4MSRVcQxnjLFRLDWh56m9SkZVv7YnxXZagmhuoLj6KXTT5ML/COLLUowhQUNxzx54XuBd4DlGFU/UMiTkitI0sXteIXQWVwFVv/9wtQQvEaSg+41VP/5ENfG/PvblSl3APAoVf9KzWQtrCP7+MXuSzCbT2rMtV3KDnNMvMtvNGffhKyr1AdfPysBPTf+2fBoNNKA1XzXoYQP3ECw4F1x6HHyXHdsrT0MR8khvsZtAonQghSuhZuyqyEs8oDyJc7p651Dt3VNasFV9FAElB8ckx5NEj/h4ycDluXfjmGttrfW9bN2buXyUDgNWJyo6UKQRq64Pnsxef7OBz6tLKk0hm0hZJ4lY4KorZRimWvUebFqjD44NvDyaGwpgsI5wIEDcaWCt+DUssb21OEjgE0n9FpL8fvS7mssy3rZmxbXcb/2c9gnCpvZapwIhXf0zVMEWKJ2T9ZCb4/rhu84yyGNe2V5x0nKUXQA1uT5WT7MtepC9FEyUF6IQUOq2bLupDhprwgmY2Foo4HZzQKlFA7JZgyhlo/9bftrq9QcOvF47j1oP3fo2kiNt5llZ89KG51/cZzcj5rcvEwY4mdHH+vBo6tRgRJIUOAQDnHbpmClqQj1YzGtDW/ssKAFXlMNuglwjvuJfd1hFBQKtvcLa/NYAdEHQ+V772Ac3DPL2zUVcgzT0lpmm/NLBgVzGY7MWazbWNWV+ZlEZJkIc755zVy++HIrikb9eAguece4gx4vfu4A== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: eXvGaZXsU6P8RGNnAq+SjTN7Ht4qX2IINgOLXa+slLloCtYEKDcCqOkY5yoBAvUEuvEVZQ3oPg67ldsXZq0Ah9XRhmugFLaMbH1dnhehmRVpWUx8hcA76DYiXoKhTq8rmwRMK9freKzDHIsgDFniDVICKYBYAYiGXVd8uCyJIB0nyWqXlM86QmSjlR8Sz7Gmlqo6FCAdnjMd/qPU5FKNk5IJE+h3XFt245mr8gsjewZK0C2tRjQms4kjafFsnmljuGVzyjR/FITt8XEht6y+mXygFijKFKB83DWz7efPg4/nX8lD5Xvd0jWCvwlCKpiBccj2RBLzxxXgZYde49NVWwKVmIgwEiqxiVAmGbuseznPQuYRvJz5riGaMoWJAj+i6Rs8vLZSe6SIjDhqC35/UG/O3htmvM82vjCMEl5yuTJYN5G7uj3N0N19qpRT8RJjLXivaQ+n3rEAUQx9M3HVCFKCw9C6VLuvxXs+JkKkrAwMhv3zS5V0YzNs+DKyrfzAgYaynm+zYYfX5EKsTmmJBktaZ2rWElrx9l02Q3mVrqItVVN2tv/mdQTCENoN+GKz7X6nrFN9EUSlb1nk6mStxBVwTxSczZ02EK+z+WD2N+Pmx9tG5nIFaTmlrpfynutM9zsriYmuS8Nq76eyUQvkKNZzhZrBBdSBe06GhplFZxtLbtFohXrJA8oXTvFIf/nOiWPVhPYOE45SffuRVdCcth13qgccp52SfX8sSISmryjuN8sKkZ6gU40t+3CPVfj1frVJWWRrIgMVzVQtddV3AG9XuJnF35aNOBT+PdzKFVA7WaI9dc7fnbXvsSqqjng3+KOPNsYJPbPEaEv7tEiRzA13T8oSlXPZe79TCSP+3jjDyzBC0JQuCfbC4FW3WCB+zB+Y2PuG5lR9jeYTBSB4f5NLW3OIMDciGcpQIZQ8AfYbnf2/uN61vn+Y+3EwyZoyLZftOMOcgtTcm88wgStA+Q1DKvy+L0bSB2E3pftqyNjxTEP71n0PvKLBSy1DX2LM X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5b0a0528-001c-4cd6-4f73-08db8728e5e5 X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jul 2023 00:49:52.9624 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CMJuAxBUdTfDB1y7JACY4im6hyYsSnJm9J9SCnBJfCl+3NrHOo2Xx8c/GaRYxHLKIB8VXMNfSymnH7KbzQfs7g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR10MB6193 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-07-17_15,2023-07-13_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 malwarescore=0 spamscore=0 phishscore=0 suspectscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2307180006 X-Proofpoint-GUID: -sYHlFC7LOPP7X9cvRVh3iPq52MmGGLK X-Proofpoint-ORIG-GUID: -sYHlFC7LOPP7X9cvRVh3iPq52MmGGLK X-Rspam-User: X-Stat-Signature: syni343y6we58h8fk745us4tyis3d8i5 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 6BFEC80014 X-HE-Tag: 1689641424-713699 X-HE-Meta: U2FsdGVkX18BeEFktGYbtV6EnncTO2LOxYdLZ8F7BieOcMCVOvy2VkTHsWS5y/7HUui9LKLxnt3EjPGdmvczkp1y+pnXMKRIAI5bXSkLFgD/yfzRxQ6Se3GiwzkD0qPnmP2EZq8sthEcEmeJX7JjHonYXmNwmNkPBvsAUT5OFPP6SNwxagCVgialtcARKH/rIDN0PtgnMO6Vr+tkjF2Ma6E0M0fv+mkoCnx419KyTv7kZClf5LlrhBOBxJm1L70VusK1QJB05yE2/nSSzhWeDti8BPyUxu1dQnF0xFVIx/WQnvPHvM7T5sf7/cMyi/3UrNS4ZTz/WOkbBVKECmTcsd5xtuTHjGx0HCvq8rAEtn2lXT1t5aQI6nBvx37qGiUsp9Gf71/CwblaEkBVCLcu8r46aVxlZC210zVGFDxuxQrAq71U7ysHcsOzq8SXM6H+TDJ0BmD3VNcSiNOigZEfXoW4xAFwHqo89Wf393Mu5qvjM0UrhuUVxVJcPue/sesMtdRfORlMOaKZ1xQVJ/n7jBzKlZ5AWqI6ZWwnfVlUQOX0R3LB6PI89SWQT5C4z3e3vPQKdGTqQ3MmxHzqMeo9MKo6jE20c+tOoa1tJdtxAllYf0hWdkxbvxujJyGUqyVLzi6eWQIOmgi6oL4ufJzkRcpvrNKscgJDP6xktXa/NIGcxKFwwF7nShwKvsb+U+YP3KWXUx61nUiBdhuVZs5uHYeHjctyAP8N1WtQxyYZ6SXuUO3QwdLoJ6//tqkLcm2B6oXsRY6T+wr/l2uNYWaltjgKsP4RzXAyQqt3PA/ZZHmzmz30vjuoCDGK3Sp1oI5jG4Z0sIjpJ+boSBj/k9ZEqI0SPBHQNiNaibiICtp3rfZxa0NffgsJiTztI4iK96ALUuNBwy/+QrfF2t3Au+5i2Ys2yIDGmhVRY2YtbYKeUgYM+xfPjr/pJHVOzryQttxOAgFUgQpPoQjw51R3m+4 SpLMrxuk WlaPB1Zx4NtYEIWtLHBan59Z/DuZQtymaYl/wmDAMh9urRHKQV1kqgGpz3+SZQHWrzqmz2WmQpU8RRnIupY2hn86mF3gzBqkIgSugV2ABBaH/vRLleBYRwyp2d7Gt4k0KdsIcAJDokHCabYp7kzRvB1ldaUS3+0m3/O1FqPvRK6s/BYQgAtPaGdQBFN0BXwmibMTDChHoGyqekKlxHlXaKJP0O6v4ZeEJ4pLV/MDBxqBCps5tsR5fEk3bqCEfBdLLoCd9d3mtAMO+8IQg880OcfT7lTSMj0CkrYEFan1UPNXWXqdTbrx0vP+GMD6n/Fdr/QzPisXBlGiTLpvIDRvAFixbZs4qKiyV/XMwtNkE+vVzGA/8gZyj3Q1Bvj4S/CgRbhch2NvazAKJ5yTYC8m1qf+ACthhxEw+6+EmJh57zTo8XOk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: update_and_free_pages_bulk is designed to free a list of hugetlb pages back to their associated lower level allocators. This may require allocating vmemmmap pages associated with each hugetlb page. The hugetlb page destructor must be changed before pages are freed to lower level allocators. However, the destructor must be changed under the hugetlb lock. This means there is potentially one lock cycle per page. Minimize the number of lock cycles in update_and_free_pages_bulk by: 1) allocating necessary vmemmap for all hugetlb pages on the list 2) take hugetlb lock and clear destructor for all pages on the list 3) free all pages on list back to low level allocators Signed-off-by: Mike Kravetz Reviewed-by: Muchun Song Acked-by: James Houghton --- mm/hugetlb.c | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4a910121a647..e6b780291539 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1856,13 +1856,43 @@ static void update_and_free_hugetlb_folio(struct hstate *h, struct folio *folio, static void update_and_free_pages_bulk(struct hstate *h, struct list_head *list) { struct page *page, *t_page; - struct folio *folio; + bool clear_dtor = false; + /* + * First allocate required vmemmmap for all pages on list. If vmemmap + * can not be allocated, we can not free page to lower level allocator, + * so add back as hugetlb surplus page. + */ list_for_each_entry_safe(page, t_page, list, lru) { - folio = page_folio(page); - update_and_free_hugetlb_folio(h, folio, false); - cond_resched(); + if (HPageVmemmapOptimized(page)) { + if (hugetlb_vmemmap_restore(h, page)) { + spin_lock_irq(&hugetlb_lock); + add_hugetlb_folio(h, page_folio(page), true); + spin_unlock_irq(&hugetlb_lock); + } else + clear_dtor = true; + cond_resched(); + } + } + + /* + * If vmemmmap allocation performed above, then take lock to clear + * destructor of all pages on list. + */ + if (clear_dtor) { + spin_lock_irq(&hugetlb_lock); + list_for_each_entry(page, list, lru) + __clear_hugetlb_destructor(h, page_folio(page)); + spin_unlock_irq(&hugetlb_lock); } + + /* + * Free pages back to low level allocators. vmemmap and destructors + * were taken care of above, so update_and_free_hugetlb_folio will + * not need to take hugetlb lock. + */ + list_for_each_entry_safe(page, t_page, list, lru) + update_and_free_hugetlb_folio(h, page_folio(page), false); } struct hstate *size_to_hstate(unsigned long size)