From patchwork Thu Oct 19 02:31:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 13428193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BC80CDB482 for ; Thu, 19 Oct 2023 02:32:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 363FE80080; Wed, 18 Oct 2023 22:32:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23C5980074; Wed, 18 Oct 2023 22:32:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 044FC8007F; Wed, 18 Oct 2023 22:32:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E433F80074 for ; Wed, 18 Oct 2023 22:32:09 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C6B831A108C for ; Thu, 19 Oct 2023 02:32:09 +0000 (UTC) X-FDA: 81360636378.12.30EDB9E Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf07.hostedemail.com (Postfix) with ESMTP id 8F67F40014 for ; Thu, 19 Oct 2023 02:32:06 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=e9YEpOuc; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=VcQCFtKW; dmarc=pass (policy=none) header.from=oracle.com; spf=pass (imf07.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697682726; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aj9XYBvc1umDA4dSHKq8Dgaza9nvmhLjC3NH0H6MDKA=; b=A09dJcBB6DVOKliw0M2Y618HweWPdsz/c361bUTap47y/+wewoZKiyJx3zmoMbsypOkjjT pP2XpiQviUrXa7Ls+b6hJjVjyGTFJaqTYZ8GqDklJEGEr24JcwxovEu+L3iecct7TeV7xG XrPzgay33U0He2OARO1zeQN8naWXAMk= ARC-Authentication-Results: i=2; imf07.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=e9YEpOuc; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=VcQCFtKW; dmarc=pass (policy=none) header.from=oracle.com; spf=pass (imf07.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1697682726; a=rsa-sha256; cv=pass; b=m6JoPymtSzZMeanD0aNLdze0PIgXpV/IQnnWskSkuUkfDkiWpeIsrdsBGCf6hZpMge5Jeq jdk2dLc59heJD7VD/3/nOmfxEB4H74b5e43orJ5RhC9XpoEZGyo3ZoLCq3JiBGoAgMLjIX 8IJb8bJXlSZdiAO/JP8F+SInkTbqR2E= Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 39IIn3Cn027685; Thu, 19 Oct 2023 02:31:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=aj9XYBvc1umDA4dSHKq8Dgaza9nvmhLjC3NH0H6MDKA=; b=e9YEpOuccwTjeZWuQ1t3MT5hXeGQMLPTfONh/qmXjzwXA1hqm1Jnq7L1VCznDFent1DN ZUNRmUSzeRzcn5Q2AZzm7wYLHIUokQoFIKIrWft7BkBVfDPXZqWKrlCHr/Z6l8i81En0 Sl5MAOVFrSpx9Z45rlwm5llvdiRMQXgeeEsMcYgxZgDWKdQz0CyjSEVrlrQlcDmfmxet aSwXS+jxELMuOsMqxb21FWzRTIMcnFcOmLTuwgzFtZkbbVkSs7DW7IOyEtwPEZwOpTdf trTZTu8eqYjGD96+Kc3QYyMj+3vqI0MY0s01tBjeJ6T5h+b773WhsNQW8TJf2/ozd6ir 6A== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3tqk1bs38e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Oct 2023 02:31:23 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 39J1r0c0040448; Thu, 19 Oct 2023 02:31:22 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2041.outbound.protection.outlook.com [104.47.66.41]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3trfypjxg9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Oct 2023 02:31:22 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V1KBugUHc2dqKT3skjw0iTR5kvxNa9vUvVOizhCJmK6tcfGuFXfs8oiSFnmUUgx4/mZN9rwKr8C+bw92CaWUeP7UT3rtZMcFfiV20PDpJpGUsNuSfY6GUPpnuj2/mNthLqg6eD1vpU4xqh9zFjbe/rQYPWdevSbV+nW2/wiO40Sw8yze33KuPykK2mwat16WCf9YlE7vFgHtSeIPZF4BFLY50DCeAVOaTLhTyh68sBoLWnqDEEja5SiT75iUoP66BDfJXptE3yRxA3xJtNn0/cYnwY+kzLymtJEhOb0xRyxAG6IeTfLmNlJhKUPqXXzT5eWrRK9XVIH7jL0w1i9eFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aj9XYBvc1umDA4dSHKq8Dgaza9nvmhLjC3NH0H6MDKA=; b=aUzUQGyg5EnH8vjtgyBJAwTFTKpP8JVQMdO3Yy3dSjqEZh3DZd7VrXnQMNqzhOkb76Noc6etKyvTgJR6foOYt0kPaMvhy3emBNzZ6+lPo2+lCQgFSYlBCAKCUkNyRwkKbTMuWUYXcAekuLOdSM/aU8dr+3Fy5P1Qchh5geaqa7SEWIOtG3YE/c1mSykPviionqdaqE6M0TrBf4UR+KtgGdpX0zplreWy6IJYAssQyLKsoXzf3DlxFpJOAUKspZLeYSB6oNIqV/dgdwc8t4CuNEprcImc5YoLIRd3V7825QrdDIy10f4w8iHM+aesk9ToNpMJkVF61OKztggrrJ3TjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aj9XYBvc1umDA4dSHKq8Dgaza9nvmhLjC3NH0H6MDKA=; b=VcQCFtKW4NsanTqwEm6ZfCfG9AI1qkuivUrX4HA7zuVm+qMrQ7zIwDNGJem+9ADm83qZC0slUz2fIlBp089N6G/yrTneP0N4c+78LSSJSF+RoRtzZvNtyisWVDl74LKk4zJ+IVb8kx4P47pQkfhe3nGFjfxTSMqfEVg6pYnNxQw= Received: from BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) by CY8PR10MB7123.namprd10.prod.outlook.com (2603:10b6:930:74::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.36; Thu, 19 Oct 2023 02:31:20 +0000 Received: from BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::59f7:ec45:eb41:d8c2]) by BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::59f7:ec45:eb41:d8c2%6]) with mapi id 15.20.6863.032; Thu, 19 Oct 2023 02:31:20 +0000 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Muchun Song , Joao Martins , Konrad Dybcio , Oscar Salvador , David Hildenbrand , Miaohe Lin , David Rientjes , Anshuman Khandual , Naoya Horiguchi , Barry Song <21cnbao@gmail.com>, Michal Hocko , Matthew Wilcox , Xiongchun Duan , Sergey Senozhatsky , Usama Arif , Andrew Morton , Mike Kravetz , James Houghton Subject: [PATCH v8 1/8] hugetlb: optimize update_and_free_pages_bulk to avoid lock cycles Date: Wed, 18 Oct 2023 19:31:03 -0700 Message-ID: <20231019023113.345257-2-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231019023113.345257-1-mike.kravetz@oracle.com> References: <20231019023113.345257-1-mike.kravetz@oracle.com> X-ClientProxiedBy: MW4PR03CA0095.namprd03.prod.outlook.com (2603:10b6:303:b7::10) To BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR10MB4196:EE_|CY8PR10MB7123:EE_ X-MS-Office365-Filtering-Correlation-Id: 3497091d-d510-4766-32d6-08dbd04b7a93 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sCBmrf6FWGXnnDYsdXozTxR5k4OK2ZxgIhBrJ3FW41L34aAa5ooKgOUAnrJ7StqROYXGkh13gqVVs+LYd92/kUpRkpTNdsu4/7F76TfA6+x/KeX47A1lFmqyPVAhAMskubQxKagE30MIzvavNrCbqNaMNy/7GCe1CQwkGT0TRYsKtH28SEoVyzR9dI1RRBbNt4Ah8nH81eV9O91w/E1nZ8BtXuFsTx48t/11qi0jhSF0ebbyBZ/MQ1FeQlCxrqi4+lWh0Qi/5trG6jabOUyLoKAzwVzyAs0WnR+1eVFQZMwy/70H0qQu6p6pgligr/ocPsijAh4pNLA9N/44qwxh1n3X1orTj1PDoNA9VHmUDcFD8P+KMaJa84tZX7tonForh689luxqWpzdf98ZquAb9w+vd8csxKKP/PTZMf/+CyNXUMMEKd3HFVTSfNMxLXb6os/ji33fZHQNKfQisyr5QBzh9I79NvN2k5q475a+TGiS3rd6T9kTSUAEXRnsvzwWNvkKH4Pybx+6nBDQPi75sQKdjhWBxO+1BKSlGzX8I29XY2OeKHCSMoMQWUrANjfy X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR10MB4196.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(136003)(346002)(396003)(366004)(230922051799003)(1800799009)(186009)(64100799003)(451199024)(478600001)(54906003)(316002)(66946007)(66476007)(6486002)(66556008)(6666004)(1076003)(86362001)(2616005)(26005)(6506007)(36756003)(6512007)(8936002)(2906002)(4326008)(15650500001)(8676002)(7416002)(44832011)(41300700001)(83380400001)(5660300002)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: fDX8WngwI5tItkRa0jsEXfz2RznQjt4vANh2q3dj8ewmTU7d/aj6aB9CNdRgmmCgenBkSxEY/elEnJh0faJBn/w1RUZkG+evHtHcLCPAcAJ/RZdAfOqctvUCEpg0wmUhY7ME/RocfUb0Y2KmPLfMODXQ9KvAPhhvxiIYrCk7tNxjWYPBarOkHMte00sBQQ7BAA+jfeHw7v8Siplr0hZ8rvWr8Vi9cjXPNdlkASrQPTv4uW+a8SHr6ooS2gw32EHP01lOTfdqBHIReiRdu1PKav6aBLd0jr92GR1b+I1W6+0roLkxDDaLWHcDqxGmggR1HAYUxpZOfBnxe71KAx6ZcvoYAdTH3DRKPFufaWNczcCBX8wApPi2yO5gfqprAy3Zq/VqMCJE6tRWwcYuxge/yKz/Q7Y3i0w/1xddRnWdRUFwWfUae+FViDStOzSYtouXeNyJC4CI09XRA+FZlZ5Ku/w2Lm+kH4vEUFoBFFqP6cliwc3iwF3rzulGPfkCTM6/DEdLpfrUGaQp5EJyhQqSvCzhJc+bVQ92+tm5d+ybFQmYqKtYaLN0ug3ekjgo9IhMn0FYIED2z1ud5JIvM96RVC2TCBA39xDcKB+pbTU7eK4Hd15mb4Wb1afSsRqIyVaKDlgNA/RYFffLFbkK/WtvpKq4WM91RsmyO7IbyVpXxzIRCJOS/esReAMayde0PnWY1iMo8q6VyBvukbaEt3XkC2o3C7uKZxdJHugVDt06EFMsMbBg7HsYrXHBNVfTowYd/XtkJHCNvTS7/3h0DG1N39X4OLP/4PHQBxAoCSinZklAR230mJaYv+90YSrQ4+dX6nKwwA0FbeOLPYsp/jQJTvoPfPRK9BR1nbLWNga7Y4MgJc9z4FUA4lz2kpliEs8Zg1rB9vqvPMXaDbLC6MmEXdHXMWR1YWUgk1/pqiQqthitTnYf+4YQz5OuOKn4j+xTRVyqO9uFP8B69pjp4ySJLdcyYaoU3KBCKABNx54Ay8VxastoGlwGHOhOgB9hAhI1+NZAXzSkShq28ujZCLhHZ/3DMOuYyknYjJVfyLVByjpBGh6fdpcZUmz9mbnP/hedF5s/D84yD6d/HG1awcPkX8nLCGhYl4EFCmJwu3gN1D/4B1Z4Tw+JNVubrwC43B5jqLnXmvelEU6tUgZaDAB5FuASdJnU1VzNXXxclkamL0chSuGJAKY6Dc19x2wNDfTUD+Vu4+X275yWIFlq/EdG/3flt5/D9mnXgKPPaO934adfyN1O5jZSxqfuXXn0bpzjHBKsur8gCsQwqs8xAy6OkAT6tJDajdWw18phNe0NsOT9KLSJYXfOYInlahIYw7F3xXYm8152xmE0qcW93hiLrIxAyjFGPlNUtT0lkHghMwNc1EVoVkh7vcZiaAjUeDbDP/zHsjJ2etT1fWxrMDQb4ZDdkhZf9YstCyvkDpbSknMUapHxkwbpUcjVatVKVjsI06x7RCOKwggdEty5yN1y79BlWac12CA8gXbKg7kq7GLk8ETUu+RPFKo03LfQMHMUCKlXEC6fBOFw3I/XQuf8KqlUQRIE46+MK41cQ08aa+AeNNGo2PUb5FUxg7KvzI/Wr0+HSX/xE1Ch8Y99QcIHnQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: u05OtFiN+b2Z/V5/d6o9tBg2EYRGdopbGya8sv3u8cowCnNv03RIqY8C2KDGkF7Sw52UHlFigtowWI56Em3znndJ8UKqbqPIzeHBkln8nX8BZAtpFfqezSRm5vWH6dgj7NskUmy8SZOA6fV0b9UMy7GCjBT8TLVwrvQb+iravV4df3eeAVyKFIMFcti+PYmTsQ2zwlgHiCm1rc/mLdq4887gDD1Srgxg/8HfZ7d+iKrSDIaWB739gIp+3J5eOdZBaPAp4/B0IJNGbb2RSyRoE7AZFvnMFmEpFsIZSiBYCvK20ERQAaV3oJ4p//EjbkYORNh1thw2/OYjev+WUIkStMAHILuwgZ0D8jfHNvSQHEkfjVs/KPDQmWjKBbUSZ5asg0d/FEmEzFBQX0Om6/AfkWCVjmRYroVPc59MhbFPHUC6YHaFRyVFfGd6EmdPSNqllRKi4KI8sHVq4IoX0sm6ePMbZoKkvvP6HbPMoT/TC2A/hYhzzcNIKd78pjZNLECabWQ/dXI+T4gpW34dS3lP7FmUP9DClrYeexkixlH8BgPuUfFCeSHCtNCEL/BDMXAW9lASYnT6Cg3VIPisIpFa9auAePZQYhL7qAayTSnHG3Pvjya9oH6BnMOBLpouONFTuabhcHrIC3EJlUfFev05RWBZu8jze7KOnYFmgfXWmYyTmZXDMMV8A2+d+4r+KYPp1PxEGjLPyq/brIDWMqrafgoKMdCX8r+lAzZFc0BnQaWCeaNQz3WqVnfttyBo5nH7v0hGrugTiVtdA/BFOUlvP6UwgpfVwCqIrI3Z0Iz1t7K5zunZhvdDIIDZ5jxoi6C+kb2Z4yN1lP480eNqx5N2W1BPR3Cw99OX32CSB+BQaJgYflXw+OzmiSYQ3a9qt61zNkd5+9z6oCaae5jQdBXDwcSFJQXLztg8SwpEQiyy2PBnp630JyjhMlUFIpxeVquVcnU5n9o+QB/ZyfgDiWgT34lKFtIwWDW+IhdYKiECqDauaq6+rAcBfSn3/8jtxxQQ4NsrF0xncITXzzDEGYcfdRRX769fjgVLnjRF198QqVO5Zqlsmfmseba0o0WM0nmZFy7FW39+8uwVFFs0isxpTXsBkWHx9b+At3qSMbf8vZHlC92q95qO1yvH2iPZ1nSq5GLYrhUIHPNAeqTKHzk26bqMS73uj3m01Pt5Tv1TcULYed8Gq3WMtyJ/juAYN09k X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3497091d-d510-4766-32d6-08dbd04b7a93 X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Oct 2023 02:31:20.1867 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: e0La7n9nhs6MV2VIaz83dp3QI3MlRcpJngx4sj4Et+DTJMf9+zb7beJJszoYD1PFOthBchx9vnrp7lARveidkg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB7123 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-10-19_02,2023-10-18_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 phishscore=0 spamscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2309180000 definitions=main-2310190019 X-Proofpoint-GUID: eVSFDYN1w36MvDUeeE3XEksew9_kiHYQ X-Proofpoint-ORIG-GUID: eVSFDYN1w36MvDUeeE3XEksew9_kiHYQ X-Rspamd-Queue-Id: 8F67F40014 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: wehrf9fm943gng5mmd5ewkj84n49b3nq X-HE-Tag: 1697682726-364137 X-HE-Meta: U2FsdGVkX18Ag9HBP8b1dQfkl2U0X1mj2b/fjc8RmfETSgpz1DmnH9gyEu433C41x6zupc1ZH1/xdkcrSNzAGw9Jl7QLFKnHlvcIq/s/qop9JMCz0vNBOCZMVPZkvJsA/pH6wCMK4CJRjN0BjUcrXI/OwWR4n4QpQ/M8CfsdSl4wgTwb2XmOSKldSlOXwljzNDkLP6MAmGXxh0iNnyXWiItwg9XZeUaV6DuMUCfi+Sa3uOZDhYFMdfbT24iJ5V6OCkGF/TfS/OnfGMBEVfzwHa6LlVqXkBmnptRMFf8MOY693U9X+pxV2DySqz9W3GYJQ8ytn2vqp+Eyzn/7iDnp3gj1coPB2eBqPopL1prcfqeN3NVRC+OTC0uiJvV2/drmk/AJFu6ODx1e8dx1oIg171M39yJiuxlt4fKPG3myCBEVQoK36rbC1OEU0qPrCtW0rc1wZbYau5X3MC/ZkzLq53+GgYiG4YC7D0bE50Kg1elmcXv4grLbSy1WbCkA4bBZ6Gkl0kqnObW4mUHnphDLeHgV3c1QOXIHf0/5fQ1h64XxnB0N5jIzoC/11K6yb0WZZQXXze2bIJKZW1zzw4PEbZAOoiHmQ5x+6cgl6E3Gahi1ETYzF+9rcg1aKOeHw9rlBuxcIbAIzwWloxcWwYJe7f/XEe6Fi3b8Ke+zKnzBeJ/7giZZtml/lugP52arYfu2QZMtLNQ2ZYgMXVC3veiHcrzOs7cQZzKpHGRaRVT7Aulgmd192sWzTAf1dwFznFyOB1p1h7v9FI8vJlm8HhxXVoRDRDX6LVKaTDQqIBt3rn6ERcw37zRLCq6FV++aACLumqtrLXxqsRkglTlWxHFhb9TfUhEF/FPg7c5efm9DCuMtoJE8SuWgCeCj735Nu9H29RnhrHyVBp4dAN6B+81A6yAVyr9m4fzPTICP0vEv8eUPSBrzaqcn1hDKPBPSA9QzU0eOYV/YzpNJz2RjFZV /D4jAnEx aNji+4SoVnN7BniiMLoY3nwJu0Yb2gV4WV5IWxzUOE748sX5txbhwY/4gxBXxwCsYmJG8tamtixW3FXJ6rMMRxOhtxJ9Hn+K8iJqrZF2zv1gOj2mqffrV7wqZ0jMxB/kbNlJ9yqko98Z3UqABHo0HIvQTYUKiWKoYgMysgtypFOb4W6JdmZfg7XZQ09eFu8y6BwzXiYZeKT2OXKxgaxQx7QueSyNl7rn/iFy34qlBuZmdfuCsIZESb9On6mR0ZOZCp+bex9k76cPsXXDeVXnsjFwy/z/Fh1aJSlIVQ2QSOWNegcRyQqZwC7zn4w6bF5zg12HXjt7p+vNr0ClpwWZBR3u/MJkDIOFA5+KB9FE8IjgCrJt+kDM0l89HO1UE/uQ7I+Y6M327YMc2VCGg6T0GAlGBactpFNFj3MkwaycQMrF2MXZO4FY3bnmm4aNSh85RuZYz1+fIZXEEuv8Lcg2XPIJ9fUaN4Cq3uojvgyFLiaGXGMtoNn76gcR+d3R0fes7lyqk1anuggxOQKtIEhF3okny0sfUQlGUi7g8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: update_and_free_pages_bulk is designed to free a list of hugetlb pages back to their associated lower level allocators. This may require allocating vmemmmap pages associated with each hugetlb page. The hugetlb page destructor must be changed before pages are freed to lower level allocators. However, the destructor must be changed under the hugetlb lock. This means there is potentially one lock cycle per page. Minimize the number of lock cycles in update_and_free_pages_bulk by: 1) allocating necessary vmemmap for all hugetlb pages on the list 2) take hugetlb lock and clear destructor for all pages on the list 3) free all pages on list back to low level allocators Signed-off-by: Mike Kravetz Reviewed-by: Muchun Song Acked-by: James Houghton --- mm/hugetlb.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index da6f85b7db88..b839080a2a6b 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1862,7 +1862,46 @@ static void update_and_free_hugetlb_folio(struct hstate *h, struct folio *folio, static void update_and_free_pages_bulk(struct hstate *h, struct list_head *list) { struct folio *folio, *t_folio; + bool clear_dtor = false; + /* + * First allocate required vmemmmap (if necessary) for all folios on + * list. If vmemmap can not be allocated, we can not free folio to + * lower level allocator, so add back as hugetlb surplus page. + * add_hugetlb_folio() removes the page from THIS list. + * Use clear_dtor to note if vmemmap was successfully allocated for + * ANY page on the list. + */ + list_for_each_entry_safe(folio, t_folio, list, lru) { + if (folio_test_hugetlb_vmemmap_optimized(folio)) { + if (hugetlb_vmemmap_restore(h, &folio->page)) { + spin_lock_irq(&hugetlb_lock); + add_hugetlb_folio(h, folio, true); + spin_unlock_irq(&hugetlb_lock); + } else + clear_dtor = true; + } + } + + /* + * If vmemmmap allocation was performed on any folio above, take lock + * to clear destructor of all folios on list. This avoids the need to + * lock/unlock for each individual folio. + * The assumption is vmemmap allocation was performed on all or none + * of the folios on the list. This is true expect in VERY rare cases. + */ + if (clear_dtor) { + spin_lock_irq(&hugetlb_lock); + list_for_each_entry(folio, list, lru) + __clear_hugetlb_destructor(h, folio); + spin_unlock_irq(&hugetlb_lock); + } + + /* + * Free folios back to low level allocators. vmemmap and destructors + * were taken care of above, so update_and_free_hugetlb_folio will + * not need to take hugetlb lock. + */ list_for_each_entry_safe(folio, t_folio, list, lru) { update_and_free_hugetlb_folio(h, folio, false); cond_resched();