From patchwork Mon Sep 25 00:39:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 13397118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 533F1CE7A8B for ; Mon, 25 Sep 2023 00:41:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BE956B0188; Sun, 24 Sep 2023 20:41:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 646A76B018A; Sun, 24 Sep 2023 20:41:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 388426B018B; Sun, 24 Sep 2023 20:41:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 217FD6B0188 for ; Sun, 24 Sep 2023 20:41:03 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E961C1CA31F for ; Mon, 25 Sep 2023 00:41:02 +0000 (UTC) X-FDA: 81273265164.20.CC20509 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf22.hostedemail.com (Postfix) with ESMTP id 8924DC000F for ; Mon, 25 Sep 2023 00:40:59 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=wtvFBpcP; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=mAc5Gxun; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf22.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695602459; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RRqnSj9V2V9kQEWWcxwgMfag+uHNl9i5WXsEUMojxeE=; b=5kAFurO9/znD9Isu+58FocxFcVuIvVk0gUQFZXEMMuuEFNA3iw19Ofg87vr8WaIboPzRnE 55dn/nwvTJdNHasGD42iGHh9oWF0Req/YYsNprgPJ6nIq3NOhjd+CwrfBpbg6AKcabpNPi P+vhgO6n6qZegp9wYBXNe4q3OLIKUHI= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1695602459; a=rsa-sha256; cv=pass; b=JQ1m9HhC2qKM2LlzlAO2yq3m+9ktA0Xql4EvjOaQJr6SYsy6Nw97IdxRZxbNCmDWoH7JYw qZ/GpL7CCh2IZnmSZQaASkLb/av5N9TU+YUwEeAzU5nKsPcRWu2LOa2pslG9vIG5vzZNIH 7WiK/So7XMzGgjGopJAzHqlYtT8IWxQ= ARC-Authentication-Results: i=2; imf22.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=wtvFBpcP; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=mAc5Gxun; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf22.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (policy=none) header.from=oracle.com Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 38OKqRiP013276; Mon, 25 Sep 2023 00:40:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=RRqnSj9V2V9kQEWWcxwgMfag+uHNl9i5WXsEUMojxeE=; b=wtvFBpcP7X+6b76qtOKdZt38nhZV684R5yUlpkX51gjUTJwU7cWTGL53QjaI0XiOSjUK +uzUq5YAbISYgIRU14NfKAGQ8mJOkuIP/guJLNZV9PWfs4BlsOC5kR7okVOKm5a/7ahe acAVo3WTbjWBnBK4YljZDjNX+og80nvocjbIbLIpaieCLo9Yr2tMVGDt2Yn+YHd0SyMk qOTWYtjx9fEA8orUd5qYuvQWbbSORaoENUKKO9UjMEi69oV1s4dy/1HbvYMvlD1vLY7m hkyC8YBoSI7c3I9VWjJhkhh7npctcoFkJLkTjnp1JTRUL5AdXY0Xc7V646UL2rJO5RUz bA== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3t9qwbadqb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Sep 2023 00:40:26 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 38ON0wxH030950; Mon, 25 Sep 2023 00:40:25 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2169.outbound.protection.outlook.com [104.47.55.169]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3t9pf9usgb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Sep 2023 00:40:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZhKuE1OpGeKbr3tTK7lrTkBEvHzzmVOnp4eb5vF/3QBh9/c0TBKh+W0o5h5hzwS/t+i4NSzi01V8xBm07rxHIBpDgMWbdM3gAQmPPUP/NS/1MoBMb6BF0ybyHcc0ijNGsXVo4bErCz1qLffphl8DSFO7BqwYC4wdgdO1G67oBYCEpjv9Sp1xWX+Zm4B7sk/yGU76Q7RoytS2RSPe1q1f+vNM1aFrKyf7wR9BsTchMuoMUdEmBGwuFkSOtAxzksvf9Bzs6fH1UvaFcLrPj9Zz/Yk74cenmju4jrXiCAEX1KDd5GlEL6Twr+PMCNQ0gof7z2P2ceoH8jzRHGkllagsxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RRqnSj9V2V9kQEWWcxwgMfag+uHNl9i5WXsEUMojxeE=; b=FgbIVfKYBpE+vCxE7lWhwjlsohd1TYXgg3FZYIKdvc7wNvE9vQRoa/TiawOlx/zDXy6Djjtm+cfqAzh3Gt9Uo3bGjdaGkEQeLUx+K1zXZRVUz9h9rgBRu4bCgOkKPfq95rm1cbfoh0wzJVRSOZP/teVNgsJ3AUYnmeIDXOJ/cjR3EKJmOwinDJSk0izmuPr3cirbyaIOWOqepAtbw8Wzo38IYmKcg1Zs94h8ND/Zee5HMclyWUzJ6Vjj2RXj1I8Hh10m13PPehbNL3lA5W1+AJo1w6YmihnYaIxRQ0Za1vovEANmpD9/YdHYct5E0AZQHLxLkVH6ireVjGUmTcnyRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RRqnSj9V2V9kQEWWcxwgMfag+uHNl9i5WXsEUMojxeE=; b=mAc5Gxuns37/uGSbKerPv0LLUqU8v1VGe99JPmcyr7GmKXQ4fvrJzLMWleWgKoNDlv7wOsOzaDFWH4ISA+DKIC5tKnJsOh472gi9b83RWxpkL5moyVU+zW1Xv5vhgqx+kKK+UpFStMKblMogE/bpJOZuEgVJjsrPJK8A+6/s3dg= Received: from BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) by DS0PR10MB7174.namprd10.prod.outlook.com (2603:10b6:8:df::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.28; Mon, 25 Sep 2023 00:40:22 +0000 Received: from BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::c621:12ca:ba40:9054]) by BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::c621:12ca:ba40:9054%5]) with mapi id 15.20.6813.027; Mon, 25 Sep 2023 00:40:22 +0000 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Muchun Song , Joao Martins , Oscar Salvador , David Hildenbrand , Miaohe Lin , David Rientjes , Anshuman Khandual , Naoya Horiguchi , Barry Song <21cnbao@gmail.com>, Michal Hocko , Matthew Wilcox , Xiongchun Duan , Andrew Morton , Mike Kravetz Subject: [PATCH v5 4/8] hugetlb: perform vmemmap restoration on a list of pages Date: Sun, 24 Sep 2023 17:39:48 -0700 Message-ID: <20230925003953.142620-5-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230925003953.142620-1-mike.kravetz@oracle.com> References: <20230925003953.142620-1-mike.kravetz@oracle.com> X-ClientProxiedBy: MW4PR03CA0346.namprd03.prod.outlook.com (2603:10b6:303:dc::21) To BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR10MB4196:EE_|DS0PR10MB7174:EE_ X-MS-Office365-Filtering-Correlation-Id: 9f83b1df-a9fb-4670-b7d3-08dbbd600086 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nldle0gaFOlSlBMEZtHmUqn8SWuHUezdGLIDMhK1YKcwKKOytRZf/CMIjQ+qKsZ3+EednyA8mvtfko8S+weBfsUthtRXG49o8Vlc7eSMSbZunbPPWa95vBgkG7+i2DiGcjHubR6c6+ZdWk7oKRo3IYtVMBhE/QPQZ5Mr1eM9ltiIii/H1BkXUz5ke6d6ZSY9w4CjShAmZTFTq49JB1fiY97gCeSg2w8IZBdgV2yuGgthk+DDE+jn6h4aN1KDmvzLB0sFS9mPqSEObcTbKaEQx7qpGMbeSjKOpVakRwu4bigHcxMWq/9A9XAdq/P6KInZrybfj8qOanGo3Wt72+L+/chdGWWHCEgBwGuNM3ASDMASzp3HqBwDfydztmlZ4GAKOk0JCBP8OLBzqGNUYS2VyVZN3bgvlIkme5eqirWN4D9Yu8Pzl+WWHpx/rZxbL7OSgsVxGsKuFj5e9zb7x7S14t9c4PtaWA25A7q03gICvjeorLliWs9ymL1epKfjIfNB9mbCLwCJGzZoZIcA/9xi5v2k+oLER+N4z3w2+9dgxFAdWIQTYlEm3/N2dl3wLmgH X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR10MB4196.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(366004)(396003)(136003)(230922051799003)(186009)(1800799009)(451199024)(6666004)(478600001)(6486002)(107886003)(2616005)(1076003)(26005)(83380400001)(36756003)(86362001)(38100700002)(5660300002)(6506007)(6512007)(66556008)(66476007)(54906003)(66946007)(316002)(7416002)(2906002)(44832011)(41300700001)(4326008)(8676002)(8936002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: uFjQq2jS7U2YH8zJM5Ssw1feLmf5lBhPeM6ySlWLFJNu/akuQPH46bQAPOM+A9xb7cGe+9FKHHIHLA0zLJ9VW2o5Hl26dYsZKbcbf2s1VdneEFuAMf5iyTMB5syROlxYVI51vRJATtpNfRNoEts0FG3tOjH251LR8tTl580hdBybRC2QPuz3BM4RBZXrHjLucAgiyoVYcReD64qU3o0q3tNtOkhUtwBK0mTEQDOnBqwwBD5+7e7IaU7LX/SQGjlI4RUx3iV/iBvYD4XG5V02P5PBW4uB3vDwqpkMawCK0oYFfffct/QIo8gfXEI/TOTyzULOggg6VnOSM3DqE+8ya5KhOC3Qscj5vwsYonkUKj2M9cVjyrOBx8ZzlxXQ7LpIqHp4puiZTsSDngwD/GHa0Pnf6Fri5PUWd79mUzaSlrikczrYkyb96N+WZKvI4KQsgKe/eOujjITW73Fg3z7Zpq8RFATfLc6KWi5IP5Wtb2vjlxS2fVAohY7Pp0vQRLj5S6Sn8XczVWLOL09P+lxekL7rGL8zTPcud0I3Vf4vvlQkFpzSQ2yfQAXLgEEXSdpVT7MAdNdJHZ3OtEaTXeCRZDEe+eCVcXK3ZfkE0J3ZfVi1FUfb9iVWk8GbSPyF2cU/ZRD8poRvnI6GfdBDP9vz6bJY1NHePzv9qNYAORKNH99GdSHHCx2bIeEk/i7+2ErUvYzAdLikDcprIkEDU+6DUz/U8IjlTlJzsWkxWO6zFCgpsCsUF9CJ0F0x4Sx/ZsQnAZQosSjCvujRH4NryeNOlS4KI+eG0uQwro+elLnUsx+09V0hB6iogr7b+Embm1KVEIk2rCoxNnJCyYn1nmAffATmQnxRIJ1XPKTcDLnfZXoLjhF5LZC9Ldh31gXKvFWQh1q4+xKjSjq7dyv03xKuw7S14W6WJQERmmSPy1XgYDS/1xaOGYiL9y7ZO1m1Em24lyv4xN3sAmJcH8sbEjoLIG4dSvj3GhymmQsxy/mDK4pFBtPkMEC4uCkMkITCNR/bQ7eE1Sit/PP9YlmlKy+BW+rRYdoubde5vF0aCGPMzplT5h5gVzYNAfXJ3glGsP9j37Fbw1d74XFCozr4AI1jdv54mFhYRYssXQGyDGhaSYlzT9mjeS3pgz5PTOrg/8ZAsqr7iVVKV3Pjde0liYxXy2TLqgXO9G86yEESg2fZXEWpHntaRy45t21nv1BhubQ6PGK+3TphLvQrWPzEpgbSFeQDAeOIShg4dJadw+zqijuhJQmoO131WTBmyorKTQLtP/j+ZSwphGTLqTvzcTi4dYjvFd4LNYfag5Jf0QAuRY7ELaihJgfkEkUVvj+Bf8MWpjT6TiHldb4DfRuN7nQrIeFRFn1SpsHgw59HeRjCL/DJDzzEi4+0cCX+02zZS3GYZPHbZoubl6nDQM+6AiHvZaGygl93l/NICwJ2YMhMBZPhuEv88Ay6q2hK2mJYJ3geOJdPBGrPlGLZleWt9MMWP5vAjSWrJNwSuRh83cgmFhmw0qUz51KYEd54r0RxopgZBtbCc0RkF77Ld1/H4AWFqVQyrP/XR6NLgd868suow0ldfyXeYEpJuFJikVQ1Dk8k X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: /gnYhUnuXgNiVlt1bj9WPxHb136/wpVHv/5uxoRVdmrqitxGlhgfDkR3K4Sck0T2SEIqUu8t3TrWT/OzfXk+XfAwVQaLiAwxCNjfJETmItSctw1kYjoxedLEqrEYumJy8rdCQdrQVBOj0axGEiQjsM+oFMx9Th/bnp2ZLUckZefmi6b7pFPILkYQWp4cNxntGGgUv2LWaeahxGTw8cdl2IAIZ2kgP7uorqmEkmP/U2oQt913m6jM95TzICTeoR2y+yZ8Y6xcow5q01touW7+2ZAAySM64ZR79RCf2sCEujc4Jxl6SU1iGw900OeQqqKpIzDyfxMKyKBhAmyUqXluwKotcQcqNqy4XzCHHDE+vZa7CdUxH3I0HszcWgPzt9dlujY+0WGy+2zAzQvNBz0gf6j6cC+30vMnP8jgrG21V7tZ5H3ZsvGKtBMdiUOGBg0vHIfj18xXXVmbGAtb5z/L/pVMriddRmeaBMhRQz21wTcr3c4kBRlMBgsmOyWJXvqqLzpnKFyJXZ/eo27lcBUGWi1RIpALXB0YjGcBIPLR4oc7bexuAttSjwGFGAKPzeEHBGUEgy3iMgyySo3In8QRaVe+INEKvuSVn+jhIxOcvPvReoXbOCs6zC7Z7JMY0nS55Zz5d6X7S+61RIwsgQ1g6VQ5DecHOQo3PWrwUQNg/6/NEgmbMd6RPZxiGNJw0oQbMESEK8a3WSLKqrBTb+ZtAwNZKYRpeosl6z/1Qq+CU/l+STa30CipBjfZYP6NZOliob4aeN3QaI52gROpNMh7eWihcblAwwFOXcmf2VPP9Ycsxd44jd1Z3kFOKOfCPgztJzYtVnGOZ4B0Pjy5DzUD5Q/7Wg1xyqsYk3LfKSMq5OCHeF7fLZ2fLKguEvomw8dO1xkChzbJYAesVpFJ4OlKXiVWMjvv9wYyoXs9AOwHMLOEXlL6N1LnJ2JsEzdhB+WQa2YNTJDxwQhcLNuJjpoHVNJUe+/IizNMfvxem+vp7bRctxL7Qx2sm3jmcSYYRj29NPvCau7bZhQcmXTPvabdmJtEvVY18zRS8z4r9QdavNLAnx4Il1XVJtqmN1w+LbSWZc2xAEyVLNj1xXBXW0btq7I/E2wRBQECx9JSSSLlWH3hpWSCHw9Jb/3BllfWq2zc X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9f83b1df-a9fb-4670-b7d3-08dbbd600086 X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2023 00:40:22.7395 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Dqm59EcUGTHSW0AKAIxjrin7LwuGtlEng0r7wmiW7BTQuhdE1cuC/HOLXsTOyKhD3KrfXn+tZWHqJ5PHilaiWQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR10MB7174 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-09-24_21,2023-09-21_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxscore=0 malwarescore=0 suspectscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2309180000 definitions=main-2309250000 X-Proofpoint-GUID: Eb5O6Y-A3RZ-PuDzE7UVSIBLQgGwsH7h X-Proofpoint-ORIG-GUID: Eb5O6Y-A3RZ-PuDzE7UVSIBLQgGwsH7h X-Stat-Signature: fr9r8weoowjitnfxf4kotyzdza96yh9k X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 8924DC000F X-Rspam-User: X-HE-Tag: 1695602459-170876 X-HE-Meta: U2FsdGVkX1+GAwGzBFT49YoBHLGq/pAVvUTp2QHXek5Yra+pKf2Na08k1j6xEQjCWMGGnCItVFqHSgST5jlJ4DaAdz4xv7l1pamCx/TJE+yfVxgoWBUy6b04IkJOe9tZUcz2gnnsPKT+jKv7uYlWAQYALGY6qZvx2ybjxIfeudIyzCYOz7Qggy+ir7mNjNWuC6487KPTgQ00zNxhUGXF1ado2JBLrBjM6JjoDIuPd2NsTCSxc25D0R/RiONsIE/oFXwLNXO5wpYdGpGanNhsDt497WwezU/W39BAs4iW4XYWqM+oq2wuS+FJyRGmw5GtAgJrfrxPkjHo2/ndvJRpWKE4RXS4YWaJdm0v6Hg9uz4Mf/EtzsppLEHVN+mdGl0ESDlRd9W2xP6l2xV5kaSQS7DIBGt6MyExJQIa1ndWXbYRHBsT+gVWdBMKOmVP1FmtpgbLqLo8sTRN6QNlM8b4rWnrekqWJB5Yg7jTYKexPC5Mk/z1TX4YRgYaqZsxioKsphEHEFdbGrB8WwNhYEY4wfU/8ecuEl+qrJPl0JyaHZsubGO9b1pQFeev7ibxiLMVfuuGr6o+GyDBqFFMfQS8uNwoyqf2RKR9b1PdJRqwE0tanAZLamsOelIBGJnE733vezzP4+o/yxACbmMF60eCI+s4RmzxX/Hi79M09rmwkXNGonlEOXvyneMbSpy0Nigt7+/wlOej++fmxDUlGSH+bwK/S5MoAA+EJXYg6inY3UKxuDDBpphqn2t6kdOdP3jnF7Jf2d9ThTXL4dqIXgn9aDTOq0+Y/tKGdMvItBKJaGlntdgMNpn29wV5YBl+vyFMBI+P2AmJ4mWydRHmgMXyMy7kp528NFlqHtQ4r63LKaTyqY8DZb2/XhNxBQ8JKP6F7spW0M/LjO4YxuznrH9se/HuPrVWi4UAw1OYdnUbQq52NnNozIkOpsEiDB8dmfOOTelrmRti35oTmtpVB8o jzOQ4LzR XqIIAZh1UA6Gy8PKFizlBKDklJJ1KP14WHYTg/8Bq+cnMPFd75i5l5Pdni0bxMHgB4Vm01sqU7zGnuFdJTXPJeFTk5A1NL8ebATimhpDaKe7+VexOVgprgicbgP6/+wg0L9A04dbGtx3r2szcsrkC3zosTqKTY3U4zlQyWSXXMxRetXBr3iAK2IZE489o7a/jt8+kIf03Ozja5/ymRm7mPhCpJDFCs0vw4S1ql3Epo+YFfd/SpGQuzmbN2WpqDAaTTP6Dfpc2fO5luF9MjNxUBi+NzM5U+ickVk+Y+pnZoQmlVZxSvEAX5Xc4+y7nh4f6iSg9Xf8RpAB2LlWWB0LOykuV3RVKTgecEcNqzoWC0hjKT+7iOJXvbqvyFQRCg18i0sJIS/hdvwShGGldOea4l1diWcHJqaRbj+XP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The routine update_and_free_pages_bulk already performs vmemmap restoration on the list of hugetlb pages in a separate step. In preparation for more functionality to be added in this step, create a new routine hugetlb_vmemmap_restore_folios() that will restore vmemmap for a list of folios. This new routine must provide sufficient feedback about errors and actual restoration performed so that update_and_free_pages_bulk can perform optimally. Special care must be taken when encountering an error from hugetlb_vmemmap_restore_folios. We want to continue making as much forward progress as possible. A new routine bulk_vmemmap_restore_error handles this specific situation. Signed-off-by: Mike Kravetz --- mm/hugetlb.c | 98 +++++++++++++++++++++++++++++++------------- mm/hugetlb_vmemmap.c | 38 +++++++++++++++++ mm/hugetlb_vmemmap.h | 10 +++++ 3 files changed, 118 insertions(+), 28 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index da0ebd370b5f..53df35fbc3f2 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1834,50 +1834,92 @@ static void update_and_free_hugetlb_folio(struct hstate *h, struct folio *folio, schedule_work(&free_hpage_work); } -static void update_and_free_pages_bulk(struct hstate *h, struct list_head *list) +static void bulk_vmemmap_restore_error(struct hstate *h, + struct list_head *folio_list, + struct list_head *non_hvo_folios) { struct folio *folio, *t_folio; - bool clear_dtor = false; - /* - * First allocate required vmemmmap (if necessary) for all folios on - * list. If vmemmap can not be allocated, we can not free folio to - * lower level allocator, so add back as hugetlb surplus page. - * add_hugetlb_folio() removes the page from THIS list. - * Use clear_dtor to note if vmemmap was successfully allocated for - * ANY page on the list. - */ - list_for_each_entry_safe(folio, t_folio, list, lru) { - if (folio_test_hugetlb_vmemmap_optimized(folio)) { + if (!list_empty(non_hvo_folios)) { + /* + * Free any restored hugetlb pages so that restore of the + * entire list can be retried. + * The idea is that in the common case of ENOMEM errors freeing + * hugetlb pages with vmemmap we will free up memory so that we + * can allocate vmemmap for more hugetlb pages. + */ + list_for_each_entry_safe(folio, t_folio, non_hvo_folios, lru) { + list_del(&folio->lru); + spin_lock_irq(&hugetlb_lock); + __clear_hugetlb_destructor(h, folio); + spin_unlock_irq(&hugetlb_lock); + update_and_free_hugetlb_folio(h, folio, false); + cond_resched(); + } + } else { + /* + * In the case where there are no folios which can be + * immediately freed, we loop through the list trying to restore + * vmemmap individually in the hope that someone elsewhere may + * have done something to cause success (such as freeing some + * memory). If unable to restore a hugetlb page, the hugetlb + * page is made a surplus page and removed from the list. + * If are able to restore vmemmap and free one hugetlb page, we + * quit processing the list to retry the bulk operation. + */ + list_for_each_entry_safe(folio, t_folio, folio_list, lru) if (hugetlb_vmemmap_restore(h, &folio->page)) { spin_lock_irq(&hugetlb_lock); add_hugetlb_folio(h, folio, true); spin_unlock_irq(&hugetlb_lock); - } else - clear_dtor = true; - } + } else { + list_del(&folio->lru); + spin_lock_irq(&hugetlb_lock); + __clear_hugetlb_destructor(h, folio); + spin_unlock_irq(&hugetlb_lock); + update_and_free_hugetlb_folio(h, folio, false); + cond_resched(); + break; + } } +} + +static void update_and_free_pages_bulk(struct hstate *h, + struct list_head *folio_list) +{ + long ret; + struct folio *folio, *t_folio; + LIST_HEAD(non_hvo_folios); /* - * If vmemmmap allocation was performed on any folio above, take lock - * to clear destructor of all folios on list. This avoids the need to - * lock/unlock for each individual folio. - * The assumption is vmemmap allocation was performed on all or none - * of the folios on the list. This is true expect in VERY rare cases. + * First allocate required vmemmmap (if necessary) for all folios. + * Carefully handle errors and free up any available hugetlb pages + * in an effort to make forward progress. */ - if (clear_dtor) { +retry: + ret = hugetlb_vmemmap_restore_folios(h, folio_list, &non_hvo_folios); + if (ret < 0) { + bulk_vmemmap_restore_error(h, folio_list, &non_hvo_folios); + goto retry; + } + + /* + * At this point, list should be empty, ret should be >= 0 and there + * should only be pages on the non_hvo_folios list. + * Do note that the non_hvo_folios list could be empty. + * Without HVO enabled, ret will be 0 and there is no need to call + * __clear_hugetlb_destructor as this was done previously. + */ + VM_WARN_ON(!list_empty(folio_list)); + VM_WARN_ON(ret < 0); + if (!list_empty(&non_hvo_folios) && ret) { spin_lock_irq(&hugetlb_lock); - list_for_each_entry(folio, list, lru) + list_for_each_entry(folio, &non_hvo_folios, lru) __clear_hugetlb_destructor(h, folio); spin_unlock_irq(&hugetlb_lock); } - /* - * Free folios back to low level allocators. vmemmap and destructors - * were taken care of above, so update_and_free_hugetlb_folio will - * not need to take hugetlb lock. - */ - list_for_each_entry_safe(folio, t_folio, list, lru) { + list_for_each_entry_safe(folio, t_folio, &non_hvo_folios, lru) { update_and_free_hugetlb_folio(h, folio, false); cond_resched(); } diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 4558b814ffab..77f44b81ff01 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -480,6 +480,44 @@ int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head) return ret; } +/** + * hugetlb_vmemmap_restore_folios - restore vmemmap for every folio on the list. + * @h: hstate. + * @folio_list: list of folios. + * @non_hvo_folios: Output list of folios for which vmemmap exists. + * + * Return: number of folios for which vmemmap was restored, or an error code + * if an error was encountered restoring vmemmap for a folio. + * Folios that have vmemmap are moved to the non_hvo_folios + * list. Processing of entries stops when the first error is + * encountered. The folio that experienced the error and all + * non-processed folios will remain on folio_list. + */ +long hugetlb_vmemmap_restore_folios(const struct hstate *h, + struct list_head *folio_list, + struct list_head *non_hvo_folios) +{ + struct folio *folio, *t_folio; + long restored = 0; + long ret = 0; + + list_for_each_entry_safe(folio, t_folio, folio_list, lru) { + if (folio_test_hugetlb_vmemmap_optimized(folio)) { + ret = hugetlb_vmemmap_restore(h, &folio->page); + if (ret) + break; + restored++; + } + + /* Add non-optimized folios to output list */ + list_move(&folio->lru, non_hvo_folios); + } + + if (!ret) + ret = restored; + return ret; +} + /* Return true iff a HugeTLB whose vmemmap should and can be optimized. */ static bool vmemmap_should_optimize(const struct hstate *h, const struct page *head) { diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h index c512e388dbb4..0b7710f90e38 100644 --- a/mm/hugetlb_vmemmap.h +++ b/mm/hugetlb_vmemmap.h @@ -19,6 +19,9 @@ #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head); +long hugetlb_vmemmap_restore_folios(const struct hstate *h, + struct list_head *folio_list, + struct list_head *non_hvo_folios); void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head); void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_list); @@ -45,6 +48,13 @@ static inline int hugetlb_vmemmap_restore(const struct hstate *h, struct page *h return 0; } +static long hugetlb_vmemmap_restore_folios(const struct hstate *h, + struct list_head *folio_list, + struct list_head *non_hvo_folios) +{ + return 0; +} + static inline void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head) { }