From patchwork Wed Aug 30 18:49:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13370629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AF6EC83F01 for ; Wed, 30 Aug 2023 18:50:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 52677440171; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 300A8440174; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0DE9440170; Wed, 30 Aug 2023 14:50:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CD892440171 for ; Wed, 30 Aug 2023 14:50:36 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 904B240388 for ; Wed, 30 Aug 2023 18:50:36 +0000 (UTC) X-FDA: 81181662072.02.81EDD2E Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf17.hostedemail.com (Postfix) with ESMTP id 3920A4000C for ; Wed, 30 Aug 2023 18:50:33 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=bU3ORyE+; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="XWE/Ilpx"; spf=pass (imf17.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1693421433; a=rsa-sha256; cv=pass; b=lWHq2kcrs0aCjPiPwQQxtPA6Y/sYalbdaY/guoJ4HFK6r7VB6tGaBjNMk0/GB8cgzn3Hmg Lrks4UmH0dCj6yYONsGuasDUCaO8BaK4KAw64na5Jf5HHaInj/nUAiMc0T61yZdQdjOlCc Cp1xJE3Okdyjgh3REF5kRrLLX5vHsxw= ARC-Authentication-Results: i=2; imf17.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=bU3ORyE+; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="XWE/Ilpx"; spf=pass (imf17.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693421433; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=V2qBt25bpg2D3yMTQCWte8BlIlS/6RXGVrigjE6CaKg=; b=cMCjGMwP+E58IFqdxk3TSsMjgwTfrN019H7+CMp1U9f8Pdfs0akb4tzTTy+lyKYfJ3oLom ouzUgbES1bpw7BI3DZT4i44emdBZaMrBR1HoC6mWjZyD9nFYFa/6KPdy38vJ5DQzKUyJSI Xd1i1AX4HRACcW1XR+sIV/AysOGOl+c= Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37UInfdj032402; Wed, 30 Aug 2023 18:50:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=V2qBt25bpg2D3yMTQCWte8BlIlS/6RXGVrigjE6CaKg=; b=bU3ORyE+KcJVO19uIH+FQhex8iHbNXXa5L5qX6DsfGpOb0a98DzGaoPsxfWz8hyBa5TT ZRxIFdX+oT4AkZetf5YhruZEuz53uRR0kJpr+1HEArV4zl1C5GOOSw0h3P1T9PwUIvPS ILuM3PmuPOk5+qffLuzhKQDy21VJDAyPiCUPj4QjBYalMZQBHcauY8QF3HZghSe6HrTn dNDwA/AC9kKib3K+8gi3DmWjCR/9OVQCPCAtFpv072+jTTWbUgkPj6vywlq0smh++D43 eJkZjNBzzpnD1cNi3ERUQh90Vxo3rXeHpyw8vy7tCgMSNnei0Em36DO/YxTjeax3OE/q /Q== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3sq9k686b2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:07 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 37UIDu8g032772; Wed, 30 Aug 2023 18:50:06 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2049.outbound.protection.outlook.com [104.47.66.49]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3sr6dqehd0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Y7VkF1CkHtnnlqNDVASGby2tY35cKWDuAMphkRQ8rasG7WQhoZz6iGTwXihZlSk6vW09fWKtmC5f484TMkz07ebQ3QMkL9hOjIr/8XrQiy5UiK+EZNMoilVKK4B2fp8nHfo/6AK2yDeIs+H+WUzkJg4ahk9e7DpRo5aHcXu0cQyIu/oDwMh3BX2FvwfmVllDJzDlHufRGH9NeTLuFgXg7kuLxJnjmvy6MIMBwC01NCPR8B1LUtbfNX1IYRzJ5DNjd5IRyjAyzQVwo+u+3bxvgh80jKUSCWEBxrN5j8ns5aqBBDvra2jcfWiEmToMeDqUzopVxDrchrU8MPYfTrneMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=V2qBt25bpg2D3yMTQCWte8BlIlS/6RXGVrigjE6CaKg=; b=R+5JCffPlraidlcQKJnxugIBlmnyMJTSiDpDdKuA21vJE3ECHxGdM88dy9q5Kr6YRZ3wt/ve8pgb9fGQhjE+Q+t9MdvPr3WIDfn+hvuYacoU1Ulu3zn0ccKLyzobDkV9+14qGVmfL5Exb1oP/uXpwPUmf6vTa3Pu5RVMrAqeRIyNzOJTvqWlRbCzpkDtkZBTZDIyfgFtWW6effvN+HuoPPymXtXu92GtEPP3e0d+XO8PGr7XKqEzJ7b1RrvJsZWgzkkcna86vuq4FmGLkVRvGvfiFV1qc0EnyG5cbyfmkP8PZYv2VdAkZPJCyhm+wOh4MyK0lnsdomdDp5CtS3v1rg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V2qBt25bpg2D3yMTQCWte8BlIlS/6RXGVrigjE6CaKg=; b=XWE/IlpxOPoXMb3XV3UMEJUFNdck1zWvzDCS61Z5VksJOf49MOx6dgFpl2wSY+17KQQsJZBo5CH48ZanX0F3CdLFfOdP97E9fhqKfIDTw7aL6d1BbJr1ZgJMHt3qVBcg2gvKSt6GdSUedE/ZC+ih0JKeg909SKnyWMT4+ncOwkM= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CYXPR10MB7897.namprd10.prod.outlook.com (2603:10b6:930:da::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.20; Wed, 30 Aug 2023 18:50:04 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415%4]) with mapi id 15.20.6699.035; Wed, 30 Aug 2023 18:50:04 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora , Linus Torvalds Subject: [PATCH v2 1/9] mm/clear_huge_page: allow arch override for clear_huge_page() Date: Wed, 30 Aug 2023 11:49:50 -0700 Message-Id: <20230830184958.2333078-2-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230830184958.2333078-1-ankur.a.arora@oracle.com> References: <20230830184958.2333078-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: SJ2PR07CA0015.namprd07.prod.outlook.com (2603:10b6:a03:505::28) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CYXPR10MB7897:EE_ X-MS-Office365-Filtering-Correlation-Id: 88e90712-dc34-4d15-c881-08dba989ec30 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /cOrjgqvaWjaaJy8zApf+RRQ7Z1B61halLQvLIIyugbtv1qHf7lo5xY6yweRF+i6TVqZlnGyKY6mAweZgiaF5Nkdn9VZKK/TRFByyHuXgU2NsFwzFp4bnLWt5AzHBF5Hgg5NdE6iS+7/3RTbbgyzwrkemmm/2utCYZyK0b4+zVpFkhFNeqG8qc7nAxg+SyZ913n3HRhRduwdcEpxpCQbpa2tPT2iw8xFC0NN+AL4/AYf72ZDkctpFxH3/DbFSXbI2Zy83rvZQWIU9g8I0X1DrvhZk/151dqugQWrimSJrh3HDNb3lGIqlyBHRVdrHdDj1XYXcSkDHioRM5PkT/ITsTDTepxRpUieWU1qnVPWJBiLTNZz8lYjWjTthN31EX+/fKzWq/burIvoiaaI1ZTOUIx9awSiLDmZIh2/OmAk+BeNY3bVq55PA03alBGtWiFfylAt6hJ5i/IcMZ+JSMPUn6nfQ/BlHgJajm1UjckvcHvBF+CFMu/Bnh73LCtAqbiVJSJ8fQ/gcxQv7YvGqlE8IKoyyUiEyy1pbrszCQHAXNtBM5MQpXe2+piGCuwGNYOpb9lgwt3Meqrk2kIaKfxSAz5bDWk0BenXmAQSgT81M8U= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(376002)(396003)(136003)(366004)(451199024)(186009)(1800799009)(478600001)(38100700002)(26005)(1076003)(36756003)(66946007)(41300700001)(6512007)(83380400001)(2616005)(54906003)(66476007)(66556008)(6486002)(6506007)(316002)(6666004)(7416002)(5660300002)(4744005)(4326008)(8676002)(103116003)(8936002)(86362001)(2906002)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: X1wyC/J05XQmV7EsgVIks1cK2v0bSV5lOJWMxfKEPc+z3SiS/df7ICAbDSjxCeTLtEjEfbjUhUb/r3md25xeu1iq5zIT+fLmpQrk8hiAxLASmGCi0ndRhWPYajxmmR0pekPI/cu+WNHMR2DuSZX/BbQ/lcimEI24Zy4XAmvLRQLbxrLK8q8T4tnAOyv1sJNW2qzVsos+5G62qLJTjj+uRlKeHPipo6BjooZ/h0bYkFQCgNGzngMMiZ40wT/MNx3cCFtyn7L8GJLQ8id7lNmmrIrReL38DX+WR2PcQKEhVbXpilP+46fbg/MIiHktJoP2JVsp4PWEfow4XuJwfSkFJzmmxJSuzzgmL8txhYsKqitGbRLLfLsxgEXbvJNBU2LeksfL083CI/5/FkLP58EJNp+AR71Uxgvzg0vJm8Cx8xpUTV7XF/JqFQaBJnGzChQpfj+ZVsToFWmasiva59dUns6xcYzf6V7B4LgI+bn7wjpiaDnTZ7pizxP1SOUnJzICLNcI7SqLzCp/fkdiCZLtYVfefGNujq+OUkG7uuO1f7Gp4ERHWWE/iRgX+p64E1wqwIJm0iR7Z1y+3up+NFcmoMdPb/C9L2s+F8aHmy1Wozg1EHR1ZZ6Bje9BiKN8ZX1C9ccLAnCiTpv4VSe2ZIG+zcd0MpC//4NSKmZL/RIzumgEZGBMz1Vm9j2aUQ2xa9hCTikhD6fhofK4wfku/Nl1lEu+hwU9EwHwWpUi+cX98HRvSIyimET7Lf2ABQoaT/NJEat7ZpU8bpzjGg1mNId8odfuoWLbDblimm36zzPViINDW0omKLjenqqwsgC35DmOzbimFclxOhSAouAy/G7tdfhH5qkqZuCyd7m+c+lxq4cYQN9gjpnpKlSCTsRnFZkZL37gxUDc3XQ02MzapJphR7YEUGYOMpr9YnECXS4zLTiys+nXN47pgSM8BFNi5FeudU7baHlBirrmrtKNSTrtwT2IGGXQ6IGTLiOYXlZK8AI+OupqbFsxvLTxyOvzQk1XHWuy8wxOm8HHKDi/Z8XQN74b/75eyBgHHQxSV78q59rLlkrDqP1t/shF5Q9Gg7FKyUMkkN8pi7tsjPsedT3BTMwwg8u8HG1tvlBIg66p7Q6udK8lt55ycLOIU5CIT8/ieC0oah2acgLituTmq/zLLEOG8U4U/sdSbya5mjXwrJGhTGw8qipBtwYaeN9/yaYo0XAPA2Kfnj1X55m0hAcPSXZUU/ypq8cFp1DUMupoj1QEEgSehNv5tYUrmUAqiZBug3Xa5vQ21GShOIxna/pSzHuaBSz+17pP5skIaExBzddq1biCAs+nu5USZn7sb+9ksGM60jGobnQ7HLf0s1lpeDTr/pu26rQJZQ5BpDuEGv3prlTpeQP1XYCB/OIZKaouxcaKCIarG2BGCIk/LYT5X6M+FUJyXQ+Kn4Xl/TncRnioR9fpKr0AR2EyOE0QjO/vSrouC3jSYoN+H+4mTrDthuiSqK23lus86PRSd0zf4yHFTRHOl8TDNxOEAXPLWzR8FsBsgvAbVpYoM5LSfcV5C52n8Jr3q12lxIAWf9QniIvh3YCw6/zr2lITe2MdpqfRiEPoUG6AnD0PuKR3d95kmA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: ixRomdRqZk7RzwTLspYovQbngCVoB77Yk4o4nI8zGN6a8ctr0a7qVXOreSdQHBWX9+4+fuI9Z5HG0tsquKV3rLrddCyY1wSyeBOhddXE8XBc9xu7kb53uD4krcf6A+m55i6A5RsgAiy27sILlD7yYkLv9IgckzSLD92GK9AN1krvcHDGHg07Gnn9d/5d5qIPEDDyEe47lbqi8QZ2Iz6OnAhXgChi47qtQPpwqAumI3rL5r8Bc2spT4z2SSIziOoPmzSIYtaa+0c5SNAJ3accaR4PA1nQuJ9hzWdf9FsIQ+H3gtKpA6ut2OQc9ZhMgpKnSgA3jsM51tUTvj8Vbwy3+KVgoy7WrCc3nhAfNyfUTzRlhk8zpPrGpCtl60P+M/j7ZWjcuTGAJSaAr9HuLi//eXwwq6HX6hF+0ldvcVAJNuY7XcNqwFzDdZir8IK9Rwr6CrBVPDUcy63KaqgVrd46g9ZY8m/5L+iF9J0koHVn+S4xGuDqBJYA0Z+suyCMvzPj8zUTIWeNhXkXCs7EY4TZ0k9kVR6ybWTeseqIQVciLh1wPGCR745y+GMYFsPyAupNepdTgLXrvVdW7fTRZkkkcKkdfLNFf9aAyAAg25j3rjNVBpyuZm/0dn2tYqGJFAUuf9FUxIZIBG8zRZFTKuLY2aR/ZvqruZq3/eeIZeFLTJ+I04xn8nwJdMEhJAzm0jtpIB4TX1U03xCNp5wMgW3MjSOOCiHHeOz5LMTdUZ8rXXd2sbajnK1K5gFg4u199X9RFHZNVkbU7BRF++VluQxJVRg3x39q9KAs3dfA6hZRk5gbFacVxWLwI5JT55eBOFhT2HtdS7fWnWz9LXa/GG4U60n4lICO1cevi0ht8n5dfPe3QLHTILvRcd9eXstNcC/qsbjUrkLBY6+yvW2Cbj86ONO6n179ENeEeRPqA7y1wVN5ZhexrhX2P3MlvZKFkLy0lZwHCwhU6IQMWBGuuZgzjpk9/xjHqtO4uPwdCTMZTD4K8wcjjXgPZdEhL4Yp9wfn7BMVN12AL54KQq69HAxaWNRiSbCq5Tw/hMJehsLd6v94exrx6MWB79IvjW65OpwFS6G60/pteoLuIMrxwjqe7CF/NbKBnD2q0ZDte87tpA0ECl88Y1L5PWLrLvp5fsezgPd1afTU6e9buSSUwnzRe1gv/LrGXn6KVQPqJxMuNP8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 88e90712-dc34-4d15-c881-08dba989ec30 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2023 18:50:04.4151 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0aIn4blNgYhZpYoip8ut2asFexK4l+AbHq7Wxv4B3GndIo9D0jIoiaFUdGL7M/IHkHecZhKkre6ONocfCEyTjfKFSHmK3MCT6XvcS2Q+O34= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR10MB7897 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-30_15,2023-08-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 suspectscore=0 adultscore=0 phishscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2308300170 X-Proofpoint-GUID: Gy16ycZ5fuBeDdzttyBSLeWGicyGRc8u X-Proofpoint-ORIG-GUID: Gy16ycZ5fuBeDdzttyBSLeWGicyGRc8u X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 3920A4000C X-Stat-Signature: 4a7x9szxad88e3h3o7f9qp8u5st5q6p5 X-Rspam-User: X-HE-Tag: 1693421433-566763 X-HE-Meta: U2FsdGVkX1/uw3tXCK+oLe+C3iorlfVTEYZaf3hXIbj+bXDfU4cdrjIvDwRk7O/GvAp7qqleVg2hbQNVQfAUFY38cjhZb+Jk+CMl4C41riryrH+q/4qyWysXwi0nBn5W93+qAD49wI48CdjMFRYBHecoI9Z2Z+mGJlgGOE8LiNP6hBNtANilng1ftU/LJmmKKuMsgQH5LIfusWvkcKNRtiXc21LQgPGNj5oBgLfpP92H/Z1Ua0QMSxwe0E2Y3sgck1wYX9vPP65ct8Kurx4MlFvJMCPirQb28LL3N08FNUW3y2c+5ZZNftylGNAYa/h1DTIJgJoRabr+bFJnORecdABY6zOthT06lz5OZbv3rSBdGv/8/haF2ISCLxcxNgn6iUyEaEK5KLVeY+/95O1F+YUTQZiCa4Wjff+jhZVLe0KG2OUwwsxFH9+4AKHjhCtNZy0nB8LFojKj+QdhYy2XBJDbXX/5nGdiMFFK/3Z9I0yHxpOWvaSfJTQQsgdJ7BpTPaTu2Q9BjbCt0YBQpk28HKaRQfkHu4NCEQ1g4bLdtpC/hOOHmk4FNfB+5FzNFsw68iFedB2FqxIbBdiL4KvddgHikd2pquVNYUbnOnTuaApSj3t3uRnuQ681+Z1/7La4gMA1KFV06qYC05WrTIEXm9phX4SMPsZXaB2QBjwq2DHHH4N8Gie7U9NK7xQOe9qb4fClfRTlbhWLwrr66pGVzyjdn7hV4hdb6DQaCmahS5LAjg47BFKeFsb4lGVvhAUXl2ZXB50rXL3/7cCgI/e2Ygt6wgAkEMBndipd9cbROZUOTZRxWZ6qcP5OznggX0Su08jFvSLf3dQXyvOYzsbIHQe2/aADGrWkyNouuR/JUoqssbrZu3By4gl0FhiNzxP5+PXOJlbo1P63S5f1r13sty44PKzrOpzBCRfta+L/sSoScx8S2l+Tj+SJvaJxUVdNWV5weNjJhy6KXicYSAp x7FVcAqZ M30jQijexsDw6UntZM6Yk2cRxMRDQKbCpwcoUalZnOQW1Kj7jGYcJSwwULSls1QgaEFRzY1WxM1P9zdMcsISwEooqUQSvQJPyF8vJkipMXimkWsmPfAotQvue9bm23VtzmwXt7+gtsbZ07ROO6wY+0Pd5ukXJrJSn50+IeqB5cVeYcV7QMqy6pXDa0kIzJBRk2fpTBWYr5rztT0JU7IhjikRo/F4kQ0t5Zki+7Q+7zt6GLT5QJszbigaq91d9Zu9GBf/LB96mZHeuWETmjti6DzGjndWM9e9WiWvKGUqM9OUXUg3o1edd4E8oqHS5Dw62wRL9cVFNpHX6jZbDK8xJXykawM5TUQU4h5MLT4VwXQZgmk7gevx341dKWXIwndwrafA7/+3Hs16nbzABAahuuM4m+zsp6Zya+5OdMYE/PumcaHagHgm0A5JUtORTgmwPARndBVyE9aPy6RrKpHkblvxHp2vOAdv54hVpgqpP/durMDsKhtqWl/VNf4V9wVzgRuhFAs3umQm7Ez0+FkVmXyICD0I1tJnWf/47FM15XVx4IQWdwkzrm4EfWsHu5YMHZrHhKB+v60JWL1yhiBtms7G7qvmdr1s4AD5SCyZ8MYlYqJo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: clear_huge_page() is constrained to process a page-at-a-time because it also handles the CONFIG_HIGHMEM case. Mark __weak to allow for arch specific optimizations. Suggested-by: Linus Torvalds Signed-off-by: Ankur Arora --- mm/memory.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 405a483d2fd1..3854f0b9b3a9 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5982,8 +5982,9 @@ static int clear_subpage(unsigned long addr, int idx, void *arg) return 0; } -void clear_huge_page(struct page *page, - unsigned long addr_hint, unsigned int pages_per_huge_page) +__weak void clear_huge_page(struct page *page, + unsigned long addr_hint, + unsigned int pages_per_huge_page) { unsigned long addr = addr_hint & ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); From patchwork Wed Aug 30 18:49:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13370627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FCE1C83F01 for ; Wed, 30 Aug 2023 18:50:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6FAB244016F; Wed, 30 Aug 2023 14:50:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6844F440170; Wed, 30 Aug 2023 14:50:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D99444016F; Wed, 30 Aug 2023 14:50:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 10676440161 for ; Wed, 30 Aug 2023 14:50:36 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C093914037F for ; Wed, 30 Aug 2023 18:50:35 +0000 (UTC) X-FDA: 81181662030.10.CC50C0C Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf03.hostedemail.com (Postfix) with ESMTP id 3F3D020019 for ; Wed, 30 Aug 2023 18:50:32 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=1wBBdaqQ; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="H0hRP/gR"; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf03.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693421432; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ddZ9ZaIYlX96ohbL+L/v67ViAco4TJ28+TNGKpwzYho=; b=faM1mNpUJ72l6yqI17WMnlAS9LZNuTeuv/IVOKHfTSD6g4nloLUkrQ7g2QwYeBRe4q3iMg //w6i1JKaZAHqVbqW5aYOiXo3iJzVYg48Ar0AS4f57zkOBAB3zSs2ZS3yOXNhtApiIRrst 3mieY82Sonf2aCw48RAphp+42UOINpA= ARC-Authentication-Results: i=2; imf03.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=1wBBdaqQ; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="H0hRP/gR"; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf03.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1693421432; a=rsa-sha256; cv=pass; b=Xu5zNMplC+KyiW4cRbD4eU8kfT+mDXhCJZmSvvVYA9uUtsvyW7EWppJbS2KY4OcLtSyjSa innNkqdAl2cjlb9+TgVXg9BAFSRt1o6gm7zYAPhu94x8HrWybKXs2PtkkLc2RiUOPGyGPI XQFML+isPqfgLk5ij3onwoiu9r6Xw5M= Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37UInbgN026408; Wed, 30 Aug 2023 18:50:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=ddZ9ZaIYlX96ohbL+L/v67ViAco4TJ28+TNGKpwzYho=; b=1wBBdaqQ5GFL5cad6b4FSqnuHoq7LDWxxhAdFA0xLN08Ix0cE05Qa/LycaHJEUCkPr/4 kkGZBFeRtK/jiB1wryS1nYLbnshB+8GUQaeKG+cSKwWO7VdIvYy0UpcoIfGSNvJ8lJpg yArtJ/7Wo6mcFZnPPZjBfZOdCpJeQQemY7oxrVr0aeZcNO3R1bELzPxOoBLUghZHG66t AyD+V9gAQMNu7YA43se0kzJuvsq61C5fQ+H4y4zGUVp8IKt5JUPX2yk1hCmno8jBaGg2 6eDys2CVLO+jjSArbsy2mgdPKuZnkXVhcc4nak4PHuOppycDtG1/mHWqcfJw5mrcu8b3 Bg== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3sq9xt86f6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:13 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 37UIJHaB009143; Wed, 30 Aug 2023 18:50:12 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2047.outbound.protection.outlook.com [104.47.66.47]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3ssepyapk3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CZ0Js+WxKxmUEtu7mp5B2tqGO88xK1aA6IGahKpAeE+i9Qi7MkD4WdD4KN/05av/1M6JbSDE60UBb/7+EwwnWaI2uv28mQF8SEssbuzek1KjWgHNs9UyrVMeA9dsB7ARR04/VXbxPv07NS1NOtRnaUWmUtceBGLcfXe2bd505yJ9xB5ZYReBJv9VvdUbg27uG3Q+hTw3tARNeVbH9RHQGJTwahf09YMMA2LVOcEypl5IcN1fB0t0fLHE32ENhLmi2lLp7nMjUH3qLqZFGr+ppaQN80/n8QHJomGVFv/lzmuHqAKK/iluszDz9AUzmMpChyJsm5iaLgx/k+DE6LFo3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ddZ9ZaIYlX96ohbL+L/v67ViAco4TJ28+TNGKpwzYho=; b=KbFpeiIgkb/FTja0gNDh9SkAijxpzPl33gZXlRKgdPt+e1/tNHT/ef79dozHvLHx1jlkvKpUbltQ4eoAfEtl4fY7IZfDe5gGE3vIsBsIulywdvTJ4evbx0GhctITK/DbeLYKH35pE1tUaUNRiZ9uLhYDPXk4zRh3BCpGpph4XGgnh+io+yZgq1Ht4+guOwUF3uQZqGXXVOrnrFsuCsZQ5LzqmRGShvVJgT0WtwqJKRReiv53VFGhY1IlVx6gNe+ZDql8dTY7EgeLLUiSBVU9pZo/eiHQ0Doi3wiEBKDau3upIZ89I8sH0ROaj3iIcgcLLTLK+Y54AwExKrRxjYPBqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ddZ9ZaIYlX96ohbL+L/v67ViAco4TJ28+TNGKpwzYho=; b=H0hRP/gR1/js6O3uoMs7LOXOkvTHHFPmuH2GNQesKEUeMBTh+Lg4puykUncycCbdQ9US8lqMvx1y7AB+eZywuUpITwVF3tiHeoo2Fcwws/ey4Ze08BEPzOoiGdz2/yC9Bg/d3fyM0H9jWQ5RiquzcTmPAVCEsMJyzqXOiAwfhkA= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CYXPR10MB7897.namprd10.prod.outlook.com (2603:10b6:930:da::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.20; Wed, 30 Aug 2023 18:50:07 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415%4]) with mapi id 15.20.6699.035; Wed, 30 Aug 2023 18:50:07 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora , Linus Torvalds Subject: [PATCH v2 2/9] mm/huge_page: separate clear_huge_page() and copy_huge_page() Date: Wed, 30 Aug 2023 11:49:51 -0700 Message-Id: <20230830184958.2333078-3-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230830184958.2333078-1-ankur.a.arora@oracle.com> References: <20230830184958.2333078-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: SJ2PR07CA0001.namprd07.prod.outlook.com (2603:10b6:a03:505::11) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CYXPR10MB7897:EE_ X-MS-Office365-Filtering-Correlation-Id: aedf43d9-4937-4626-f1ef-08dba989edd1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3dS8xJ/1R7CrPbD+dR6vZcgjWD8wA6bWGbSb2uBglyFyp+mvjvxGqq8boK3Ej9cV99fK0U3R3zr+WaCwKTwIpFuTx73pNp/zh6shFstUOvIKmKCGmKNrAmWGtT/MFxkS7qIrvnYtbfv2c/n9g3aC0GAAW+K0Uob2YETrkZ7+1rLSmp0XSw6OgSy34mnbkUvodNXUUsFeO/U+UCHjcchqTTve7v75rIiGbcfPwmeLZ7LkeYDVAmRY7v3rcBeX+nlZPeikMag795x/HG99kmWjZySPuZCDV9Qf+1oyUkEuV4qcM584edCo8pWFXab/oDXUuzybZDc7AicfTVPRkRdXAtBsNsG/6nLsputqkfOVIAt8Gj7wvb/hNbWmjBrJgreZ/LK/gb4FEp5mk8Ex8ZzMly1oPumFiS/O4osb02TQIyG1NU7y1Jk5v6n/jc2jQxJaN4jsWbByj2dNAAnx34udCgMIg6XV00yO9Wjzq5XEuRGlsnR5gdhwi6annh6oNBi5y0WqKiL5NNk61VM+dAHesTj2swToNTQCiLRH595AL8AX+0DpsmfsSg5pwYteh6Iq+jVHTE5SrpM/j/0UHtUxYi7RDB9WTWZNpTvp94oj7hE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(376002)(396003)(136003)(366004)(451199024)(186009)(1800799009)(478600001)(38100700002)(26005)(1076003)(36756003)(66946007)(41300700001)(6512007)(83380400001)(2616005)(54906003)(66476007)(66556008)(6486002)(6506007)(316002)(6666004)(7416002)(5660300002)(4326008)(8676002)(103116003)(8936002)(86362001)(2906002)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: cdBZC8AWzkNfx1uZVvbgG6AViaUhuYurjupZDuq56u6UIvG3lRELnSJd4zv/0zUGUcaaRErtfTsHTbDi4cARiHHmNz0/ZdZwkSn5ATfeREB7JHhS9R/qU+PXKTJV3Z1ALQKZgv+oTpYyh6ZOa5+ZoCdumtFdtlgJZnm3EUBxogw7HNRqoU5uXgw1UzeyDDEdDScoWGUP+qoSRuZHxJeNuMnBjG/S5xXB0lBrTCOsWTaRvVlP8cocyMBh5qNe0Z1oDWUR+0KqscAIjSZLgQiAaPFHf2y2RDv2i5cjL6RDKWfW5qGoxVkSYDj+dJjEtoosDAD4MqEu6lmRvTi/ttKR35CJQaEYbrNCjDg5AQ3Z0tP7+MYxKY2xKkUEaYYaK7XPT7AAbjUi3oVY8OUxxuyYy4s26ux8JLU8Cv3/yTd+3QxzdAuk2uDRkncp8IeHZ/5FFp/CSie76LSRCyY5FDkRi4Fg8gyDdCm3Aee4N8IfRNacz/7vYrHe+DWeH/TzLdD6l7i/nrk4ee5TfVvSfxOagDxn1H3OWh6WnloHWyV9GMF+qvc6fMZx6xsqQ4oDlASZoQJF9NCNFU+YZ+nDxuT/QMuhH2caO4YfslHZlc9IF7QIjwFCMFaMdssV+Izt/OhD3Nq4cz5JyzbhfyR8hrOaK3NPst490g7F6M7mVjcM7cd2bdztvPgtmrML5iZhXMwUECG2BoMxPHrPQD2Jhmb6lC49XsvUVbrsJa+f136Mcnyv2BIRxQ3GQk7hkrJuBXZHP+SNQ2Ot6/dv/aQJaOj5ShRt2lhD8T64/vFjR9gK522XSCIdBjGoPDRbGGKKCixQUONd3vfudWzJXWJVk6Nvag2fob1oA146RhNZwMOalDMobQxfBng4rAUklMcOKjbGzKpOaoVfrVK3NTM0nK0Jb0O9Yf66Vmse0+Z5BNj/A17+wZUIaDLE/FrbDpktNGRnOwl9nxefL0QxkKaaELK9MBkVUIT5PhTPMDsKe6TAO9umYCwN5LeGo/pOKiFjKmgmBHMH4KtJnKrhwhJTmew7ZM12oKkCvoqTWd/gAkVMR0hzrMqKYaV1JpPpn/BlVJcEug0fkYXIi1bDd0SJ6YMOo9orc1xY8XNBOiGtHnPmQXKjaYdRVck/rTzB+8Ly+OVLFBILLUj6EUL8ginnym4PknX0LMyaNxBOny3S7zz1wLAg1RrueynxB28KNXHIEMuEht/cuzSA+FKlwK1S2IZ2iIL70TELg2yEGqeELYZ6LRooP36zBMlIjHMrnm5/OTMeLZveb+mNgFCnBg49gD9PI/21z8aXwahn5S291aiQVHeqBLjKP3E1T/dvvPoH375r0xutcF+5ky1CqBlUBM+rekt8D+yCWyWVi4WHx9SRwM/XnSbE91gy7MOuc/fxkFCm5SlIpktFH8epj/1qFwIXRyJxYW+q/YITEYSmIGY3zzSkWlu1AY65udPOsn/gKc7ShFNte6/HQwhQaawEWWIiDPQjjZEo/xdGCM7X7NoWJ8fv3KJgDgDGicoDNiDyP32V9XPI49HJb4IZ/QrBdyhypNt+REOFkDNDYYrPu9XG8wLhP3FhOZdjj0mBBeP/0IX+GLl1cBvrUzgypk6hZMcvgg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Ussg6HHf/gl5Zp3u1IKZC5E8ZuIwnbTwKehXwriNa1kJuwDQ6D166GaTRhAI0/MOvexgjuB4FrMdG77QMliB5VuPsIzoBfJzThjnlOrmrbYCQHBdXjhLFk32duMfoAJ3DLRKY69iTmNZdYFJI+lrXLZyU2xzJ2tBrIqpFJKNpFJe/zoS5uoESqLQPXCKqlgYtFs7k3JkZ99u0pHs5k7xuACEQFz/aB3FmWklCqG38/TvQmqM0ptL6ERFvqDdwinz/h4boL/w8X8wZdKACdh7u1Kr3chIof5PLMc1YuMaYab4DpS1gWqK2FnVyCCvK4z2eg2vzP4i4iWCTjwIumjm3MW2oDjwVR2jeyc9pDgWO577C57DlmY0dgbOw1ZxxoKOuaJIKtjqs/sf95Ktoq5rkJqIngPlh1tN52y1XS0G8sp69ypI6JNPQWUcLZx1odYPTV9f0bBttQrcoT7i6PsmqxqZVqqTigwxgTSUEhRBYXJegUqqa12LOPWgllEk7rCB1yol2Omo9jzmTFHgeUUWVL2/57iwQxoWeLSp0lVjVkQNpfDvnGbq7dkUT4var22eamVoh1k7/jBK7uMHpssgpcQuU1IsoNvNWzDtaOn6ArSfHkDxc6yZrmewWAduGtfEUTr05oAGzwxhGMNJ1BL6ARaKc0lfe5lt827uhnJO8pK+Ybsi642728bU746OVStm7KPLhr35uVQoH5x35Lw//yuF3a8bJ1r0QeTj+l6/H695Oh0xbo4oJwfLlXudJfTUWGX3Uz8nTiS+Pnj1/QNAcY+iOyGN6tMEwCNVUq/63kKiH/YeAZdAHrwY3FjmjOvF9XB9OtjDE2yxvnuHP/u33++wH7F56u1uifaqjKTm0++Cj4qveIYbh6G4wzokfRsseeXjNfvnez0CEYLEQ+uY73p/M1mI9N9xaBaZg+SPVRn+F/35qWD68UhrgVJh3GByDi5luZmclMVimzvNyd7OTnpRp0vkvjcggcSOlQ7GlRXI0tvO1UMciQw+PAS0fU0A7nuTvQowDoJejO+5OTgGt1fsHBeixSyyWP+B66lq/jhUZ30xnGaZn+Bn3oAnAy2qpLig0nmovq+A6vUaGfV6MD9Le+6iVk0q7pnij960CggbOSxEqA57Ju/JFH9Mn7oDsD9B+hxWadW9jD6KxiyHchzH5MjC6AuyPsGua6E418k= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: aedf43d9-4937-4626-f1ef-08dba989edd1 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2023 18:50:07.1748 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Zn+b98msFQF+j124tNb/cX4Xbugrpn3EBgr4FhjLkgZrg2y957FHm9kA2DRM3/jlKIQ3Om83d1FR5KL1BTgFsDHD7Igym/LtM1dgRsTaZH8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR10MB7897 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-30_15,2023-08-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 malwarescore=0 spamscore=0 phishscore=0 suspectscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2308300170 X-Proofpoint-ORIG-GUID: mgOw-Ck1q6pVxnALtk4nl9gdAuKwZjuY X-Proofpoint-GUID: mgOw-Ck1q6pVxnALtk4nl9gdAuKwZjuY X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3F3D020019 X-Stat-Signature: 1yhaosd9x64bxxaeibq56tphtg195y57 X-HE-Tag: 1693421432-475462 X-HE-Meta: U2FsdGVkX19d3oSnTTqvhcMe9cXIIH8YQAFRch5a8ZFeEc8vnH0mIx01z2V7ZkcqjhBu7LzjBAFA4x24TiZfXCKB/a2+h9g+sWdYTGjpmSrtLMstKoYeO6SU7TF6AOUdMpBhB0QZa3l+6N2/EstNLk/+0DsAKT+27lJAVAozMgHaxgk/cURzs2MIpJhmz3x9hLO8IeYtITKs1jKktmsfiHofdmenNgY8amiFae1m4bnuopNi1HZiJdMSwxPieqaOp2pIRsNVqTk0wxXrmHyDH+wyNuxPrLQdiKKnwQLKUu6V6Zm9O1qiNLfnCQzNV+YYimoJOPOJJTbWmQdC5bI7GRg0od/V6MeKt2NCkGBfBHN+k8DMjpwGZ77E7m3dlymmbqF4DLwpBrrGLDGfIAAP98sKK/fPsG4v3gO6lBM+S1/l3006t5wDWtBNrBxIEHNhCIcwY0KNyel3A5dpKqLe3UUn9Xx2LsjEptGjHFAhBV9zSdUsB0qzsSllI1UqMQu6lOwx+YgnxaE2XSr/Ca2mCLEzcl+HtRyEvtvgnYOo7sbGQbJ6Btnvo3Dg8qkE7Uuq6zwG3o8LfTBqJYvh/Y2K/8NpuGOLmWM6/WUbhelRryZ10bdWyEJUjx6U7YxW/hUr0kuFCS4F4Aeji9ICto+ghHVungrw8hKWHfpD78Vk9PXp/eU1O0lv+8/iGwnBviobkDQ/apvCHXXG8xpFhk1/ARHCLaSsjvjzYBZFHnbflqV6HBMwsR66BTmkyy9u7gmqJz8CNS5yrvPtJZEp4tdJnBgFedEqvjShc90tPMSfaglEqKZD6JVbUpxkdee0XuaoeCgiqBHfDfMFl1pldCznAEbuoUnBUyfeOvCxNqYJfds3KQ32PgVotiB7Yf/Bm297UhKPOGMCEje+Ry0FdZvuyB14CL3pY9tKrBbuVh/bvcdTInKLjsArmCoha/y7EFIFaf8UaxYTcOupESf67yz 0QB+ftXH uzRcv06KZhd85MFe0d+2G6tOHYat0NNYDTafIsFFZKAEfNMsJdAKSossQDmPvcxmxbFFjpk03E8/U+gFpZ5O7CXJzrEcu9Dop6cYotlov+KSvmjWXu1mA31mowN/5jo61oFu87/87Ki+5UNvUuiYLVfEO400tch6S25ESXyWv4Cz7T1Sr27kt8wqAHo0kDZvXjQqGrQ9smFcZZwN/3b8qGNQ86SyYx3CB1Y++NnC0J1t62sDK2dwbzGjLcW4gHQ210IRlxlu6yHPQAjt8xeIk5R7JRrIz2goBdIIbtAUrAbkWBy9+TcC+BXzehAJkM2BqgesQs1z5G7386L4fDslJkZfxjI9YaXYJSq7JB6YFL0lXI8r6u9nNYjhWXRtZvdi9QSapaHmG2hQBMz7eLJFQ5sIE8WlJFwdDjw9XSOqUkV7fqDUMvWE38hEKi/eNhK9TFA3aRgG+5UFGXaxrDi6SFSjT9ZNBwH/8zMYrw0BqrZzVjn14GPaiKiJkSH4jScmDMmSdZM65o90l+9zPvPzv2sxao5+g4a91AF3S2JJtaIF2NxiItxuuDUKd2Jyg764zxIZCpBW9BkfoNaPqCCZxZVDx6VCQnunqLrSBY5KjTENZqFE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Clearing and copying of huge-pages is done via process_huge_page() which subsequently calls process_subpage(). This preserves the structural similarities while processing the pages but needs some ugly casting and even uglier indexing: for instance in the call for process_subpage(), we pass the indexed address, along with the index so the handler can also do the indexing. Additionally, both of these paths have diverged since this code was written: for instance clear_subpage() cannot fail but copy_subpage() can fail in copying. Note that there's no runtime cost in having this code be common, as most current compilers inline process_huge_page() (though some older, supported ones do not), but it's unnecessary complexity for something that only has two users. Accordingly, fold process_huge_page() back in its callers. Suggested-by: Linus Torvalds Signed-off-by: Ankur Arora --- mm/memory.c | 182 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 120 insertions(+), 62 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 3854f0b9b3a9..6e005b787608 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5899,66 +5899,6 @@ EXPORT_SYMBOL(__might_fault); #endif #if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_HUGETLBFS) -/* - * Process all subpages of the specified huge page with the specified - * operation. The target subpage will be processed last to keep its - * cache lines hot. - */ -static inline int process_huge_page( - unsigned long addr_hint, unsigned int pages_per_huge_page, - int (*process_subpage)(unsigned long addr, int idx, void *arg), - void *arg) -{ - int i, n, base, l, ret; - unsigned long addr = addr_hint & - ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); - - /* Process target subpage last to keep its cache lines hot */ - might_sleep(); - n = (addr_hint - addr) / PAGE_SIZE; - if (2 * n <= pages_per_huge_page) { - /* If target subpage in first half of huge page */ - base = 0; - l = n; - /* Process subpages at the end of huge page */ - for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { - cond_resched(); - ret = process_subpage(addr + i * PAGE_SIZE, i, arg); - if (ret) - return ret; - } - } else { - /* If target subpage in second half of huge page */ - base = pages_per_huge_page - 2 * (pages_per_huge_page - n); - l = pages_per_huge_page - n; - /* Process subpages at the begin of huge page */ - for (i = 0; i < base; i++) { - cond_resched(); - ret = process_subpage(addr + i * PAGE_SIZE, i, arg); - if (ret) - return ret; - } - } - /* - * Process remaining subpages in left-right-left-right pattern - * towards the target subpage - */ - for (i = 0; i < l; i++) { - int left_idx = base + i; - int right_idx = base + 2 * l - 1 - i; - - cond_resched(); - ret = process_subpage(addr + left_idx * PAGE_SIZE, left_idx, arg); - if (ret) - return ret; - cond_resched(); - ret = process_subpage(addr + right_idx * PAGE_SIZE, right_idx, arg); - if (ret) - return ret; - } - return 0; -} - static void clear_gigantic_page(struct page *page, unsigned long addr, unsigned int pages_per_huge_page) @@ -5982,6 +5922,65 @@ static int clear_subpage(unsigned long addr, int idx, void *arg) return 0; } +/* + * Clear subpages of the specified huge page. The target subpage will be + * processed last to keep its cache lines hot. + */ +static int __clear_huge_page( + unsigned long addr_hint, unsigned int pages_per_huge_page, + int (*process_subpage)(unsigned long addr, int idx, void *arg), + void *arg) +{ + int i, n, base, l, ret; + unsigned long addr = addr_hint & + ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); + + /* Process target subpage last to keep its cache lines hot */ + might_sleep(); + n = (addr_hint - addr) / PAGE_SIZE; + if (2 * n <= pages_per_huge_page) { + /* If target subpage in first half of huge page */ + base = 0; + l = n; + /* Process subpages at the end of huge page */ + for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { + cond_resched(); + ret = process_subpage(addr + i * PAGE_SIZE, i, arg); + if (ret) + return ret; + } + } else { + /* If target subpage in second half of huge page */ + base = pages_per_huge_page - 2 * (pages_per_huge_page - n); + l = pages_per_huge_page - n; + /* Process subpages at the begin of huge page */ + for (i = 0; i < base; i++) { + cond_resched(); + ret = process_subpage(addr + i * PAGE_SIZE, i, arg); + if (ret) + return ret; + } + } + /* + * Process remaining subpages in left-right-left-right pattern + * towards the target subpage + */ + for (i = 0; i < l; i++) { + int left_idx = base + i; + int right_idx = base + 2 * l - 1 - i; + + cond_resched(); + ret = process_subpage(addr + left_idx * PAGE_SIZE, left_idx, arg); + if (ret) + return ret; + cond_resched(); + ret = process_subpage(addr + right_idx * PAGE_SIZE, right_idx, arg); + if (ret) + return ret; + } + return 0; +} + __weak void clear_huge_page(struct page *page, unsigned long addr_hint, unsigned int pages_per_huge_page) @@ -5994,7 +5993,7 @@ __weak void clear_huge_page(struct page *page, return; } - process_huge_page(addr_hint, pages_per_huge_page, clear_subpage, page); + __clear_huge_page(addr_hint, pages_per_huge_page, clear_subpage, page); } static int copy_user_gigantic_page(struct folio *dst, struct folio *src, @@ -6038,6 +6037,65 @@ static int copy_subpage(unsigned long addr, int idx, void *arg) return 0; } +/* + * Copy subpages of the specified huge page. The target subpage will be + * processed last to keep its cache lines hot. + */ +static int __copy_huge_page( + unsigned long addr_hint, unsigned int pages_per_huge_page, + int (*process_subpage)(unsigned long addr, int idx, void *arg), + void *arg) +{ + int i, n, base, l, ret; + unsigned long addr = addr_hint & + ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); + + /* Process target subpage last to keep its cache lines hot */ + might_sleep(); + n = (addr_hint - addr) / PAGE_SIZE; + if (2 * n <= pages_per_huge_page) { + /* If target subpage in first half of huge page */ + base = 0; + l = n; + /* Process subpages at the end of huge page */ + for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { + cond_resched(); + ret = process_subpage(addr + i * PAGE_SIZE, i, arg); + if (ret) + return ret; + } + } else { + /* If target subpage in second half of huge page */ + base = pages_per_huge_page - 2 * (pages_per_huge_page - n); + l = pages_per_huge_page - n; + /* Process subpages at the begin of huge page */ + for (i = 0; i < base; i++) { + cond_resched(); + ret = process_subpage(addr + i * PAGE_SIZE, i, arg); + if (ret) + return ret; + } + } + /* + * Process remaining subpages in left-right-left-right pattern + * towards the target subpage + */ + for (i = 0; i < l; i++) { + int left_idx = base + i; + int right_idx = base + 2 * l - 1 - i; + + cond_resched(); + ret = process_subpage(addr + left_idx * PAGE_SIZE, left_idx, arg); + if (ret) + return ret; + cond_resched(); + ret = process_subpage(addr + right_idx * PAGE_SIZE, right_idx, arg); + if (ret) + return ret; + } + return 0; +} + int copy_user_large_folio(struct folio *dst, struct folio *src, unsigned long addr_hint, struct vm_area_struct *vma) { @@ -6054,7 +6112,7 @@ int copy_user_large_folio(struct folio *dst, struct folio *src, return copy_user_gigantic_page(dst, src, addr, vma, pages_per_huge_page); - return process_huge_page(addr_hint, pages_per_huge_page, copy_subpage, &arg); + return __copy_huge_page(addr_hint, pages_per_huge_page, copy_subpage, &arg); } long copy_folio_from_user(struct folio *dst_folio, From patchwork Wed Aug 30 18:49:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13370626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C31B6C6FA8F for ; Wed, 30 Aug 2023 18:50:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E768440161; Wed, 30 Aug 2023 14:50:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2656A440170; Wed, 30 Aug 2023 14:50:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0676444016F; Wed, 30 Aug 2023 14:50:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E1A07440161 for ; Wed, 30 Aug 2023 14:50:35 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B4C12B2861 for ; Wed, 30 Aug 2023 18:50:35 +0000 (UTC) X-FDA: 81181662030.10.2BEF463 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf14.hostedemail.com (Postfix) with ESMTP id 4D95810002B for ; Wed, 30 Aug 2023 18:50:32 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=SpCEOU6T; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=x2TmEqvp; spf=pass (imf14.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693421432; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hk33vgDddTabMGLcd8V8QAslNep2bYD+vzm1FUsH/n4=; b=7JQeLkiVfyv0DMjjXWmiU89FTZsr7wIoRvkDe8H4sF4WTCHTIY642C4+AQeZ+2InW9FquC 7tYSR/9ETmM9TRH4uan64ZUYHj3E3NiWhr9cBEIpju9Cx7im5fEZBMSiHzJAt2IG6JGLgW o10Q0eoc5eGJR2puSbz7tuB+rWEWzX0= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1693421432; a=rsa-sha256; cv=pass; b=Ijm/e1mwjRnHOYDdRbkco9wDbNPxSWd08D75spDSGIWOlo+nFbuZtzWP6vKauaEZTvZvui 0QwhG/GS2p/AFbwtTtAICIFfoED0NHXyuBoPP09HVXFLkLaH0GQOcPjJF7v+3tcipb5BWr KcZ0bqF+7Qs0/+/oPIz4TeuUUyA0up0= ARC-Authentication-Results: i=2; imf14.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=SpCEOU6T; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=x2TmEqvp; spf=pass (imf14.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37UInXRW032362; Wed, 30 Aug 2023 18:50:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=hk33vgDddTabMGLcd8V8QAslNep2bYD+vzm1FUsH/n4=; b=SpCEOU6TcEc269289HEcDZgbfCF0wHUliDxcFSHkDFlyd7Y5oYH3GCvMyESj5adOHxeY j1BG/6+XBhYIOzs555TmkkxrNE4D2FacYvKnjPXx6UkUtTC87KvPob1xYh2NY5yfDEMJ Jmit+GgkqcXnqY5gJcofivbjI4ldMa0OH8a/uFXvTeru5dtnkpmWqOvR/oK7AhJFj9lb oLVWmLoqiJY7PmLpQO1ciq0sitW4e9IeyIazC7QM3WVGh2i38F5U72Ffx/k8xwWWkv/k pLkhVMMcBfnK7GxFbyq8LVkq9c/CoMpejtz7ZinsL5d5TraD+SDxwgI7adyqe2BNj/Ok Jw== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3sq9k686b8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:14 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 37UIJHaC009143; Wed, 30 Aug 2023 18:50:13 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2047.outbound.protection.outlook.com [104.47.66.47]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3ssepyapk3-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:13 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q+6aOJP5dhQzl2BPX33TzPwwIAJintGXJgBxkWTDEV5XYkzY0DBGU1GaLPRwnEBE8egfuI2WecZ6XdWp9QLURnbFljetgNWb24lP2QDZTeYImyAnIoA4HERNHLjrxHo/wEJ84rr1iugvDiG9KN1v4PU6UcpjicxdxhRKlPZBkb3q4jfgCO4Kpe8rQGxhV50MOrexPgv4RqEigVYnuoqEmn9QnFnn54oataDhgpvLrv4eqzRkuMleSXmu23H1FFgltX9fqOmzzQ2LbiVk8aii95e5c3pXKP28/V91HhENUGxf1XrWfqqDDOwNDrp+8kxMVwdfgOUP3ilVVc4kDt6Jpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hk33vgDddTabMGLcd8V8QAslNep2bYD+vzm1FUsH/n4=; b=lCQLaB1h1fsZHcvJ1ASeshvGHOXknb34UftQbWrFcLx+bg1hu5hdwl34e6MXrCxhy9A2qQ5S3f9WrZHsVKmaslF8i8B4iUWi06p+2c2mofPzEgbN/AsNjHFVgG+ky1BbROq/BOR5mGKpIu52ow9h08Z3eeSszVE+Ut3m+lbGtc/q+Nqulb7ICqEGr1xYboZ58N2uOqTbCzgZ25hdzdUAuAwC4GTC0ouqOm30UCI+TquKNTtoEiMDkti3u86iGZWVWr5eYU9D9W+T0A0UmKbntXZNDdihdYhl4C+YHTN9qqoJLft0NUx0d0GXDxQag3myjCTA7HNJsDogUDU2i5long== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hk33vgDddTabMGLcd8V8QAslNep2bYD+vzm1FUsH/n4=; b=x2TmEqvpQfG5gThHqUWqHmUL1c9FNdzao25uNhKTZjZrzEDLG/KZHOSFpG5zqZDThgm3TdtqAnGE3scotGHzb4L0tnisXIx8WJi39DL3WMohRw30za0viCtX0pXXma7Bto2TzVet4k1RAOsffJ2K5FhvMqSBOFmiGqEEdDL3X3o= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CYXPR10MB7897.namprd10.prod.outlook.com (2603:10b6:930:da::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.20; Wed, 30 Aug 2023 18:50:10 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415%4]) with mapi id 15.20.6699.035; Wed, 30 Aug 2023 18:50:10 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH v2 3/9] mm/huge_page: cleanup clear_/copy_subpage() Date: Wed, 30 Aug 2023 11:49:52 -0700 Message-Id: <20230830184958.2333078-4-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230830184958.2333078-1-ankur.a.arora@oracle.com> References: <20230830184958.2333078-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: SJ2PR07CA0012.namprd07.prod.outlook.com (2603:10b6:a03:505::13) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CYXPR10MB7897:EE_ X-MS-Office365-Filtering-Correlation-Id: 7c370bb3-20f2-41f2-ca1d-08dba989efa2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: a8CoWdwka9GKhJuUJpgCFHLAEUgkPTGWZKcdHZvoe9LaHTSqTCjzt2r+exZ6zBCnZBMeQaizSduoDNgLv7bbBbNopT2Byr1z9NKjzhYWDEpFA3DxanpBO7HjtQwPtWVS84T8EpaXoqUkMsmEYvhQYrmLItF9bjaniDoxEqHj5lo8D5P+5DDa5CjR7mtXJIjAOrThrY/EOnBvxkMeo7O1aN15a9blR6Ohp3nQzXbpWHfmVPRlz6U91ZWddu+hR5/zVYz5lSvWcvoHJr62qZjWipJ+7RRbKsA5CjsWJ/rgJhVc1xZNQaKhzggRKKUHBqIeQQbGKshdlv4TEi2HbTMOp6c2Nd66iOEYjp6ormEmXAYchWhTeLbtS6Ivj0UT1pCt/BnBpynnpLuNTj/aRlawSyIAB6qPsSNZOj8ZXxMhxaoMggKi7G4XtpLRn63Fv++JX6XX7CzQ/E3jGNg90MlMDypmtHOu6ZUjkC6/xHHel5ph1AgIkCT3vfJc1G3TaikXGV8riwRPckAQugtDoErTImgQAa3QycuILPCjufoeD65qho6WsBqlkZNm3YqC9fLd4IrcGXGbCggLPq6TTdvsMvuGrCfFd2DreTlcbqFTZVc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(376002)(396003)(136003)(366004)(451199024)(186009)(1800799009)(478600001)(38100700002)(26005)(1076003)(36756003)(66946007)(41300700001)(6512007)(83380400001)(2616005)(107886003)(66476007)(66556008)(6486002)(6506007)(316002)(6666004)(7416002)(5660300002)(4326008)(8676002)(103116003)(8936002)(86362001)(2906002)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 3D3u1xCvM76a6dgzs/4GfgHr7ZTkz28EjRo6dySGNF1tfPBndiGsPYLiYvy1m9n639nfWgRCs4K2yhDJFKg0PJfxpmPVXMNBew0Vthw5HVqmAaOp7i3JJZNDRm5OLAdolre/QgouryquaYdaCQK0y5TJ78GwQc2NMVT/wVATNFdcOPY5gf34H0hYPWZJiOvNuMP8r96RLn08iZFpSegDwrEsZrSWCmofgRGc+Eq0OyCesKWb2cayiNKYLOUk50TX30fZ9z3aTWWRXtV/F0ip3BYCqbcwW2XM41XD6vqPvPRqwMdjIlXQDBTWDH4bT8G6yPz+6v+yvG+SQoPsMI2geBL7+q7GYCC4KMKksSz7u4i/9byzVWxjgtq2R8i/jW0L4Fjqufq+ONFSLszcd+RKWrFUS/kzpFwY/Z8vEiCo1QL2xqjurfVPkohQYPaCHW+uECKjU3c5BwkUYRqbcfd5PGq5wurKW9sc+e8+V50VNFx/gHySuJAlRtPgEMu0DpnqAMEYfJkOfjjRg0g6q3rQEku4E597RqNioP11YPWaskoPfI3KH/LSLHGj9CPMERhjbZ1vWkNzoDChXt+O/xF0Ag8/DRbs4Eb/keWIvK3x3EF/NQHa6Gg2PT+NiXztWSFBMkALHJseoimUlRayCc+Rpcs2N2D+2WI4hg9/SyEyFm3M5/GxVhG/ctRQvc9eANTjt7OOpaKRGlCSf2K3uVMl6asWsSNT69UbY95HuCqW5ExTNP9wi9LgRZwh6YK+km2UR2aKaa+9aSVL7nhuz0brrHWVMUX9/AargbOAz4qxwNjOTs3o4CWkJqt5FkmZQTPPIDKnM1ZAc6Hrwc2bV9EZQTgi/gdGDyw982XrEZVu361tAXtH1HtemtcDioOK3BVFXf91vbxIjlwg7jOaGed0YNMnsVbvVgsWdtrOfMa6230ItERfuKf2cRyMOVtUev7yGuoNznF45RaqBWeq3w4WHhvwK8COTOjLlP0490YRy0ANxWJo24AbF6n6SFSeL3dqyTJFKQGKzhuI5ZvA6eImGIZp4qZWCB8sE6mpxoVN0BtA2jzoNi7mDPKl9vt+Iv2AFt/gb3I5to3FS8uJ7+jjvRMCNNIqM9BLQUfHF+4GTKpfJ2WVWTFgkKxnZ5ugVMbsvTvdU+JXC0Rapc+RGSY2ZgeWR0cyYp7dYQsN1YImdy+QQ2whtmdiXGiYx/QdljQh4D3Ht2WeibX6ioggfPuTcjfDYJZN//KFZ1/YjO1+wXFNVTVHAv51zkeLOsdR3gIduz6UXbuNyO8imHT9ceZ6QOXWX8DdsIHOQhDEG8B8U3c8jrrKEmS0Px2cGLNLXmjcHdnDijbOSCXtOnWeAX7pl3+NQ6saa8qKVWrWHRsTAF50RFmQ44Lf8SIu+XO1fJDpxEztYlkVupPR4VsgnHr0wSHoh3z3knt8M/SEbPl9FOHE11yNYM3ZCrfZw/YJsqY1rB+PocGNKswH0SsBYUrj2xqgE22aUYjohxg8UeCQxBQZtWopjojOPhEaBb8xhOEp4RyNR7LJDiMHypKKEqvrgVN5Xoh6T5NrHd1ItMjbdFtsl0SrMBMasC/SQjdvKPCE/prRewPw1kELRmPn0NT+8g== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: NvKzZ7L4jWm73LZS93EiXtnkrGxcD264ImOID4rNPFPbLjxYIs1fqaEWXMnZyGDZ5v/11C/2CiRyZhcVG+5c5JY2D4lgT2zFsKdaeS/WQvg8s7YKwihYcLWcUy0Sg5rNqfjfqPAJTK9t1GL3aYzCafvVcKtAPKixdBjkm2zMwRYa37msAGGUl0OkXAYVUWatlQ5j0+geoTSlb/rO79lw9oHSFCo98jLcpffGF7oA6S4+BpE0fMj1piKBF/1YjVSU3YvZNCEh75PyGN26KISXa7ktotXRiZ9+umCX6jjBenDEwaqpU0t5VauneN80C/ZgG9NpeBPMPhxFnvjrLJ31yu57fAgwfz7/UC3qFhqPh1rvvNb3lxT2qUeyYU/FXFxBG+SJfwdHUmnKnaYuxzVaKrSbcEULUn63GOH+wz0ethYUs8/xGivCvkNzVNGsPwXOLolUWv9DcTlEajKqibW9MqTjzEROvI4W7Fcr2EHES32h2hIZJvWxTHURlOVaKbqFfri024x9HT8e6MEXvx8NEBFKPOml3ITTHINjWoQt45c57DaimZvt2sk5RgYIxm10eFKsNwG0YwRcHdn4y+l7srrKd4ND+kxo8MeCK6/matId9HWRu6eAGFoYMOKKL1zx1rwkhDletV6iE6y5JeoWb/ZRukOgOlwAte0HLV4sEYYQVOrV8ADG1WETo0aqpj3t84lgjOiSpoKezEbTZlncYTcBh7Kz6z1Zidjrka96PECjIvs3bY2Zs7G+E+WHxZALAO+p3OqNsv4tsGm74Dpfz8IOoEsRqTK+tA7uq15vQuVDqPVryFJ762ZVkcIzPMCYvJzvlj1RzT/o5gvoBsvjMe58QftMAlhx9QXhN8dPKNgbaKVe/VpuvMkuxIsvsdtpdLH9r/VaWLxF7w3nqHlsW0UurRZQyaC9W1xWqLvJJ+9c89v6lNZn0O7w8/he75y+n5q44XAaOG2eU3r33EQuo7qIKi721X36ZWaREJcBDOKJ5knAc7tfu9fqFAREaFB7IHeVGGUrUF58Qs0l6up2c3B473nhZZuyEtANlf1iGfW47SmCmD/l5W7vY0IheLKQhOczj9+sCSrg2XKbPeQUZic0Rnlm4RGr7wfamKnexHmgVYzoJUcADlnlhsP/0YKyu4Fum/brmyAMC13344kWVlW0aglbKjVILLoackoIMww= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7c370bb3-20f2-41f2-ca1d-08dba989efa2 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2023 18:50:10.2225 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YBjRkwE0VI9QKHvgvP4zzjSVXNqe9GBGBlJkEhr3zAy/tG79Owh+CgmVQRzKTzgtWPN3C+S+YGcPFjZiJBHplsXuIzprukCHWWNlCo4Y77Y= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR10MB7897 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-30_15,2023-08-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 malwarescore=0 spamscore=0 phishscore=0 suspectscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2308300170 X-Proofpoint-GUID: oNAlS3PA7YW6O0NhswrsjxO3qsP6U2eL X-Proofpoint-ORIG-GUID: oNAlS3PA7YW6O0NhswrsjxO3qsP6U2eL X-Rspamd-Queue-Id: 4D95810002B X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: f6dfd61ouxt1sq47ywpz8phny7jhxu4i X-HE-Tag: 1693421432-608123 X-HE-Meta: U2FsdGVkX18BwuPJ+oEa+8bvvlcfoYgLzA+FGuocyVJ1j4tNBVLFde2pDkMrW9FzB9LjWzP+AsWa6KriMo5Gm8Egl2apWQyfDmG0IjiaI6HqdMsPxL8193AF+6kD+K1CPgW5QlfBRJP651xqrfOuyErFG1jdwTqHupiuXM6h1yBa/NAnpYBfMFxr4yvK3wstADX2eNiHyQwx89oTJiiKsXYcDSDCsv1/Yey+I3BsDfuEMLODH21Ko3EISoMPSUSt4Wx1Yf++NOuU9gxmoQYBDZgtZo8BDDF4hkZS0vr+LkGeOYKoyCwU6agoC+TvzW0raLHpXePzaul2f1bvayLXRxEdPyCYEl6CBtuzThlep/0OnZkB6oy3CHYleD9RG4vioQ+oihQZnhQfTUwrbzbSdIz62rDqpnE24xGYTKVRov4tAwD9xPcAm2G7RdG4jkuKf9QKaviSWzrV6L0QXWgn7fskN4JobzZU1qaN/9wl9ktOId9OX1NWtFrfT7n1W+bm1Z4+731aHzbpLSTpbUEGcpc566zVNNbKBrNLdiR8G1a66S+nFH82EpoFcRaXSoyUKipnBW8CzBEFS7mR0HaZpgBSx+rellUnpvXt0bPf2pMqxfEfZenFJBctsa1YOr/TTvk4bWZgy2gyM+XR3vOE/yYe+IXCYkKaYkeHddcLkTpMC5pmF/PEVKMXz79xeeiRlAyzdDU0q1akw30IZTyYHFU9PzCpfMY0MACu2eosOHljgTtGxnEP6mfHDK4zMLbeZrh7xvypHMCRB5o0k5OcM2IfPoGAFhIyms1bTBCaZ2Ic7MFUb7KssxtFoE9LmasdwKtqDg6qTshUmTpsdHmyJqdUFJHv0G7IFR8JUb2WgG1qxqiOhV5KrWfJsZh6AhT8vz3Gc1dxS1/bx2grM+wm2D96s6XTWS8MXufH3sYDUmhNd6qr06OlGFTOmayzqQ+W7D/408ksSFuic1B74qa xVN3GmGN P6mgwG91Du/2C4UBIst1RGHlTVdgBihXoeMWyZxOfIKFUHUZ+jrrdDO3G90mHaXugnTNOrP2RxJgveonIncc2f0SA6Awpe7BmvuueqqpqFrWwmHBIgTK6v0nNuRdmL/x9OkiH+XF81kZ4Ew3OHZV3Z1zsojFhsb/Ku7nZ1xCOvYD9Ivam1CjnoF5yZ3OqknKjuFyFeVGxwNBpRvGbSyBKbuarXokAwBf/tpYD+lWCDBD3Cp/GLLafuc0ociaSNzmAlDqktdTZdUgjfljr/P6dZrAAyjHzN/NU+nGeV8HoEtfKnxcpb+1rvcbg5H5kme+ubBUPe5bpbihOyy6FQNBYJxIDAFP+egWfiUzaBUAM5DmYeEkn1acFGvMmx2pZDEGrvPza2sEDoP3/9CTTFT1BKrAR8BXUeEW5n6oRG4FgQUme60q8KcvxYUS9s2tHIPgr6ANo5iwUsst6dc1Z+4RNj/6oferGbu9au9tYMAunYal/JIVmmq592fzeT/SFJkHnhRAG/cXV626susyf/iRN6IJUxmsAS771zpU+IdYekU8hK2Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: clear_subpage(), copy_subpage(): static int clear_subpage(unsigned long addr, int idx, void *arg) static int copy_subpage(unsigned long addr, int idx, void *arg) take as parameters: an index, a post indexing virtual address, and a struct page * (clear_subpage()), or an opaque structure (copy_subpage()), both of which we would then resolve using the index. Fold clear_subpage() back into the caller and cleanup the indexing for copy_subpage(). Signed-off-by: Ankur Arora --- mm/memory.c | 58 +++++++++++++++++------------------------------------ 1 file changed, 18 insertions(+), 40 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 6e005b787608..4f1ce3ad0af5 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5914,24 +5914,14 @@ static void clear_gigantic_page(struct page *page, } } -static int clear_subpage(unsigned long addr, int idx, void *arg) -{ - struct page *page = arg; - - clear_user_highpage(page + idx, addr); - return 0; -} - /* * Clear subpages of the specified huge page. The target subpage will be * processed last to keep its cache lines hot. */ -static int __clear_huge_page( - unsigned long addr_hint, unsigned int pages_per_huge_page, - int (*process_subpage)(unsigned long addr, int idx, void *arg), - void *arg) +static void __clear_huge_page(struct page *page, + unsigned long addr_hint, unsigned int pages_per_huge_page) { - int i, n, base, l, ret; + int i, n, base, l; unsigned long addr = addr_hint & ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); @@ -5945,9 +5935,7 @@ static int __clear_huge_page( /* Process subpages at the end of huge page */ for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { cond_resched(); - ret = process_subpage(addr + i * PAGE_SIZE, i, arg); - if (ret) - return ret; + clear_user_highpage(page + i, addr + i * PAGE_SIZE); } } else { /* If target subpage in second half of huge page */ @@ -5956,9 +5944,7 @@ static int __clear_huge_page( /* Process subpages at the begin of huge page */ for (i = 0; i < base; i++) { cond_resched(); - ret = process_subpage(addr + i * PAGE_SIZE, i, arg); - if (ret) - return ret; + clear_user_highpage(page + i, addr + i * PAGE_SIZE); } } /* @@ -5970,15 +5956,11 @@ static int __clear_huge_page( int right_idx = base + 2 * l - 1 - i; cond_resched(); - ret = process_subpage(addr + left_idx * PAGE_SIZE, left_idx, arg); - if (ret) - return ret; + clear_user_highpage(page + left_idx, addr + left_idx * PAGE_SIZE); + cond_resched(); - ret = process_subpage(addr + right_idx * PAGE_SIZE, right_idx, arg); - if (ret) - return ret; + clear_user_highpage(page + right_idx, addr + right_idx * PAGE_SIZE); } - return 0; } __weak void clear_huge_page(struct page *page, @@ -5993,7 +5975,7 @@ __weak void clear_huge_page(struct page *page, return; } - __clear_huge_page(addr_hint, pages_per_huge_page, clear_subpage, page); + __clear_huge_page(page, addr_hint, pages_per_huge_page); } static int copy_user_gigantic_page(struct folio *dst, struct folio *src, @@ -6025,12 +6007,10 @@ struct copy_subpage_arg { struct vm_area_struct *vma; }; -static int copy_subpage(unsigned long addr, int idx, void *arg) +static int copy_subpage(struct copy_subpage_arg *copy_arg, unsigned long addr, int idx) { - struct copy_subpage_arg *copy_arg = arg; - if (copy_mc_user_highpage(copy_arg->dst + idx, copy_arg->src + idx, - addr, copy_arg->vma)) { + addr + idx * PAGE_SIZE, copy_arg->vma)) { memory_failure_queue(page_to_pfn(copy_arg->src + idx), 0); return -EHWPOISON; } @@ -6041,10 +6021,8 @@ static int copy_subpage(unsigned long addr, int idx, void *arg) * Copy subpages of the specified huge page. The target subpage will be * processed last to keep its cache lines hot. */ -static int __copy_huge_page( - unsigned long addr_hint, unsigned int pages_per_huge_page, - int (*process_subpage)(unsigned long addr, int idx, void *arg), - void *arg) +static int __copy_huge_page(struct copy_subpage_arg *arg, + unsigned long addr_hint, unsigned int pages_per_huge_page) { int i, n, base, l, ret; unsigned long addr = addr_hint & @@ -6060,7 +6038,7 @@ static int __copy_huge_page( /* Process subpages at the end of huge page */ for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { cond_resched(); - ret = process_subpage(addr + i * PAGE_SIZE, i, arg); + ret = copy_subpage(arg, addr, i); if (ret) return ret; } @@ -6071,7 +6049,7 @@ static int __copy_huge_page( /* Process subpages at the begin of huge page */ for (i = 0; i < base; i++) { cond_resched(); - ret = process_subpage(addr + i * PAGE_SIZE, i, arg); + ret = copy_subpage(arg, addr, i); if (ret) return ret; } @@ -6085,11 +6063,11 @@ static int __copy_huge_page( int right_idx = base + 2 * l - 1 - i; cond_resched(); - ret = process_subpage(addr + left_idx * PAGE_SIZE, left_idx, arg); + ret = copy_subpage(arg, addr, left_idx); if (ret) return ret; cond_resched(); - ret = process_subpage(addr + right_idx * PAGE_SIZE, right_idx, arg); + ret = copy_subpage(arg, addr, right_idx); if (ret) return ret; } @@ -6112,7 +6090,7 @@ int copy_user_large_folio(struct folio *dst, struct folio *src, return copy_user_gigantic_page(dst, src, addr, vma, pages_per_huge_page); - return __copy_huge_page(addr_hint, pages_per_huge_page, copy_subpage, &arg); + return __copy_huge_page(&arg, addr_hint, pages_per_huge_page); } long copy_folio_from_user(struct folio *dst_folio, From patchwork Wed Aug 30 18:49:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13370630 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DA2AC83F17 for ; Wed, 30 Aug 2023 18:50:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 85D6A440174; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 74DA3440179; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4AF4C440170; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1D515440173 for ; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E70E81A01C1 for ; Wed, 30 Aug 2023 18:50:36 +0000 (UTC) X-FDA: 81181662072.01.DA12BA5 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf10.hostedemail.com (Postfix) with ESMTP id 7972FC001C for ; Wed, 30 Aug 2023 18:50:33 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=utW3nYS2; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=szWPjUYG; dmarc=pass (policy=none) header.from=oracle.com; spf=pass (imf10.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693421433; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nJloHB0zDu8LwgV39mz1tQFxCkCRoHiIuVP6I6CUHrM=; b=0QLBmhF+ST5wwb8AsW2V/JQoDYcnuxcmDn7PNVfBxoZYRT4l2KkiyC4aNlPTPk/Xd+Xr6a 0qNRVBaMuX981x6RLTYR7ZmMu3A5C0/VM7dlGUK3KVH30D0wJPo6U3B8S5+0HS+mXFyvxw FRfbnpWrzRH88oBBz4Kl4VIy7+GJtnM= ARC-Authentication-Results: i=2; imf10.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=utW3nYS2; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=szWPjUYG; dmarc=pass (policy=none) header.from=oracle.com; spf=pass (imf10.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1693421433; a=rsa-sha256; cv=pass; b=doc+0XiZfD3hHQOzBseSUd2gZYXwtZ0ycD9RMnmt81XJWeHiFlOhpEoCqxeFtpCnR2XYok 96bEhnPdnS70levWzCSZLcvWJU9o1pSmQuwUWkI5MbTtcGnYcy/qZlBvR+nbIF9VidaEWw uDUS7jsLm9i3EFn2H/jJTp0wx+l0MfU= Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37UInrRw018041; Wed, 30 Aug 2023 18:50:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=nJloHB0zDu8LwgV39mz1tQFxCkCRoHiIuVP6I6CUHrM=; b=utW3nYS2w74/gf46FinDK3cutipHI2RgnpVqYOY+yr9oyWS5FMr4lLHsqCmAc/+jz2Pi 4Lk7psHGTVwrbJgKre76kWpUybe30K4hCZAxjMDp9lwozd9OOdpSYc/FhSLD5sIDqx0V 2Xebu6m255owgerAEkspCV2NO+V/r8Zoyb9YMqtbwYkZNZK1D4CDtru3+gi6yynVOC4F AGH742rkLmiSABi1k7fS2geLM3ThcxjvBj0ERZb5myB8NU900CtULRnJsJb3mmQCntAK xxTIzxSx56NB458CEqQDt2c3eVRqxSpaG0r79SOqlZG5+PRYLmeSRHPRUq+EZBnqoACX ew== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3sq9j4g75r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:17 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 37UI0boG000447; Wed, 30 Aug 2023 18:50:16 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2046.outbound.protection.outlook.com [104.47.66.46]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3sr6gcyw3h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:15 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RhFpn+d6a/o/HZX341TRrp8HGqXTvGbnMX7Tix2aLT49CleuzbDxvyptntIJo9bZJ/2lVAthMG0VZ30XWAeukOC2UeC6HzL5KmGjDlmPqhphuSgL/S+n3Y9BI/pfffrsaee8XN7f9gc6mo2hTLTIxDXreuFPm3pYNsCh5b/U4Z/4/3rJUS34CaFk+wc4f34etrPHQpLNNK+/gCa14aQ4lhp+aIO8+2F3yFIoGjDwz33FgcnMNgcpepJ5bUrGiyRUfYF9Of95rbujode8admDR3l4i2Blk1dVzuHVdq7V8stIN/FBY3mCfObWjgK8PdT96ucQyNeY2aGbRjlEjbxwWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nJloHB0zDu8LwgV39mz1tQFxCkCRoHiIuVP6I6CUHrM=; b=dvL1cBjpMoD/WdCK20r/ZBrBQXj200DfwuxgYXMBttrNeWbOLQ3liFDlsxTWe1Qv9DjLYytcC94WP7BiTtLVZOd6RZsdjC0OGeWw9YFbDU+KXZ0SP1f+givpeUFmLN0/o5MlLG6gZoAX/ntXyrqi5SccPe/LCKm0wwOScQ/kkF5nR0USnO4IKvHlUQ0+Xmy/adW3fHESlEmpUGNjgl2aQmJFBQcOqw+0gS+r9CMAUwWyG8XXitZi9e3RPyJ8KscTItQ+2+Yh0AOqGfe1gXC1NZM6UkitJDQE0IG0N2Dl1AH65dsqucGygPTVA4sNvP/me/JRekT3IwEHDb+N7QoTrA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nJloHB0zDu8LwgV39mz1tQFxCkCRoHiIuVP6I6CUHrM=; b=szWPjUYGjkD7KLZalDmBB/CiCIeU6aR5dw+f3SGPm4c0CWssdaDcM9h7Gwy44jrDbr5qgt9+7LmqOMH+KXplbguy+AKta2bEM7VzEvyREHYTAXp78qrm1XmehHSOJ9lQE5BfKkYsNQSUZA/EkSuGJhgp8WuiTmSOoFbIdThInjo= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CYXPR10MB7897.namprd10.prod.outlook.com (2603:10b6:930:da::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.20; Wed, 30 Aug 2023 18:50:13 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415%4]) with mapi id 15.20.6699.035; Wed, 30 Aug 2023 18:50:13 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH v2 4/9] x86/clear_page: extend clear_page*() for multi-page clearing Date: Wed, 30 Aug 2023 11:49:53 -0700 Message-Id: <20230830184958.2333078-5-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230830184958.2333078-1-ankur.a.arora@oracle.com> References: <20230830184958.2333078-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0025.namprd03.prod.outlook.com (2603:10b6:303:8f::30) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CYXPR10MB7897:EE_ X-MS-Office365-Filtering-Correlation-Id: f0d48c7a-c6ca-4080-3991-08dba989f190 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tWHw1Bzleq3VLt57kDBz6HOlx1Yj+hbmhix/327oZvg6ojP4A1BjeS90W+0IU+SmBdf4fRl29Who9GnxkL0oQDPEDSbAaOtvrjO6eY3195797WawwAk35zbAPVGCGtRJIZmbcqyPYtI0xpJwgVfISDF3oefcOE+MmgJmxJH9y7o2lGKfvI9i9oOCCBz83Gr/AgWftGolZQDv7ls9FhWmX74FFRiOnQ7f3cZhY5GR9q0+kjgkQ9xrmE90oCcSwAd+XjmJIe4ZRlaXHTBtbIbT8zriqgCKGkCDL4R+KxKThSXrxT7Q+qQrP2I8fEl/dt7kcvufTPIOPse+tpztACqvlvjJIcetj7T3xTwNMfnjbDLH2pyn3tbOZ1EAJByRlMClDFHSGNmdO+zpK/xXRmoBAAAjcRjI/kxKrUgxLhJ6YxWKo5iZCch/abKPm7YxQ1wew9uEmYyAmKB7bpErdTu19+jgppPXlFTHTiyf/BfvTdxNHjLLG0yq239CLfq6QtjTlT91hIbjZYQq8CKmOd3TGYhg7YocNbzNN2mvdmuQ/Lu6trhxWjlx8+cxQOh+FRM6 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(376002)(396003)(136003)(366004)(451199024)(186009)(1800799009)(478600001)(38100700002)(26005)(1076003)(36756003)(66946007)(41300700001)(6512007)(83380400001)(2616005)(107886003)(66476007)(66556008)(6486002)(6506007)(316002)(6666004)(7416002)(5660300002)(4326008)(8676002)(103116003)(8936002)(86362001)(2906002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: MNGn0hz5IREwEpfgI5QtLSKwZJgZ7WZbc6CtXAeKh9zn7jqpoLbCzXvxJIQuhEIGnSgKGNuK+fzoSUWVSipnV2OTX6GDVSdYbPjKmpJp/E0g+KJAgb/8rOxhNq5uxQehaGqDkmzPFy8y7/louzlS0NUFensDAE86xU1bmnhn1Qq7C4SxBfZOCuuofGmYzz35GQlYTrV101KvR/DN19o/CbHLroAeu5TDh2z+XmtbQNxbPpxL5o4cAPqZz15BRxEK7dnkbfk2UgrIJ3HxzkvbuGWPOOwbeydJtqXNBxFQ+orNOMISW9fv5X81lFsGX5BObpmjbHU3BNtAL8VFTCCXuMGwVbzQFH7rKroKSPtgMICJgI+2i3hnRPK+1ide2ChCorb5eY/+W/HaoOV6Q3jeflmtw++W9CBKyEKmNY4FXLwCeKSvHmi9VQ+/32eofGYGeZtjYMgKkIug0kADbYpEPQUUutKwZ2zGLkUvWL7AThKxV8vtTfhIr6GEXI1oztUkQBoGJCKmP1FFmtOwkx5AGLukbS1JM5bCe4Z62qcAhyfwAyWo76abt1zqp7fL0L2eCVvZj0LUBX6sajKL+XFEHt/B7zkOxzTA9MP6tN4/ycumffGpTsr1jg/GrZUARtLCMDOImqBuLVYOUIm2RnsgHnN76OmcYasda8XICYbiXdj/n0oNbIJwTxps2wmMRWbc/8WFBwgcc5DzwTGPNUygLYfi8ssSWp6canJgNGMK35Tkclf2NMevWgPnBW9cuXCqOjNpjF9/7/I+WmtaQLhkrMF5bQTC2XMJtxZHH23eW2cdsgUkwuB6RGunWg0v5sryb1rIabgQA1ggI19bBPYqbFQ8GXMg9WD4T+PzQ6renq5NtXDgr6xvdrt3/xZ5x1Ag3wsLUA/R4ddId+4FK0+MmGBxkVKGVV95ElJJwr+ThUB7wic7Axw/icMyYo5sCgG35MdDV4BO86FLeuHnyBPhuYh9QME7xcheUsRw4gIl4Q1X1JtA+Po3Ywk78Y6J3oFAcb1p+/oSJuM0wNZy9yERsqLtz0gXiLYlEwXEe5cc2dk4HzS/2TmIlgUli90tVMeixCWIJiLL6qUaSstekMJi2k9/D0SNbLdFySYjH0zTmi/4gO0czfh1VDD7Mrr6vfj+LFqckm9SFngqvpaIjRr5DzL5XTxfkud2yHPb6DjKp5sXoCqkuZHTxzS7PXZv+ala57oxc4Xnel9LL6Im5joH9xwqpQHkI+wvdPWXcwGIcQyMtgEbVX1ylqlpBvEHUHI+YTzV6fuQMWsLPVKyFdp8ZT7aTsMgj+gwr7KU8QpH/pYxXd/4r65/ZPZXiVTFy1V8NdFOC3gsnzHz8UQIug3hAosYaUludZyxr4aQKs2gkbFWYkeJvVvj6FC3ZcT5UkaZ7GfGqGLLWis7UqGbMEgHur44EGs1CydKvlpT7oKyzaXiRK85ZueNCgj/ys5t8OOmDplsUEbFQiq7fCd9IQBxvn788ebCqUzONoY3inONMJ/bV5sv+o3hrlsHK1xNhw0Tykmuat9VF5XyCNXX5iiGYUqouCqMPvDjOcXlXIwpe1GuNaxWJ9wmtUJIxEhIykWj/Dm3HzRpZl6d6d/Ft8qG0Q== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: lb1Lk5rZslOEUWrpZyof+mbkzXtSnlzf0/D4MozEDrvZSUKwg6xOPeT+XRfnkdDidQ4rd2JmGlCoiniopkywOuVowUjIUkVb6BsMxxKgDvHS4iqj3wjT2SLUbH5m5JHGJMVc2RkLYjw/VmlOJcGi95UOHDPFPlZJGtCYwb5I8DOw+OzDzOIzldapmiPIlo42zME6tbLwen9P5Q7EH6ycJ7FQ2zMnL2MpB8Z5+bamXTG0c0MLsh1S8T9Y4XOi/sezU/loHkEWNfthcd2zao4SLcekzXxJqlmVuRBVQVBoNgMGFa2FsDI725WrNUE40C21uc4AAlUzzkEFReQE8rTvl6DjgHm7rYxZ3AdLfVQxP5iZGMhdoN+fVJMmHglXmSokOJyEFGgcwyaFh1tyfEZ6CezIXSk+8qdDrEtHaDIKnDWW67TO/feAbbuJdconfA86Hrklzc5t9vhenox+zDhUzdo0qLlF7AW6JIUwrGAetxhuxVFn5BmYsGJpXrzhb1QVQ/n+DxepRgB5jWZwuvbnQ7SDSulvfBNcfLkjr9rHmh6SxojM4oTpql6bQ/VmFO2Q63ezG3GfQx3pIUQviZ9x1UNIrGOYz9Yh9vQTr5FCcEQEpcIudTJdsQWyDyW7r8cTAhjVsd0X72nAgsJUlLlYy6/sxUNQktNuNJbXwqtWJ/sbWLqL0Q/+zv1ve7R7P3QZ46vAgTRccZ75CjzMd6OXrxayp8PD455xWcvN3kGIqTsKidlz2MA85I5yAEY/eslCFA3Mi8BYfif3IaI61Yp0zyuwR+NbfhG7dI96X5iTRRsdt9Az/MAdp7HfwzMU/PdgpR8WeLKeJK9k2uA1WZoKsTO5TtprYgcD9mOX9rIjz93t+5LhuPRXcxze8UhyC5fA9FGk2WxfAaAYaLiEKLXX6xkXrQJUzNheXA6dqwb00PZtQrM/x7ojhVoDb5DtNWJ0T2QscCncXELyhhdw1vJnPGR+x69Gzg8oljzp0TUml2axrtZqyc1SXwC3GJ4CGtdGRx7tT8XoxCAk7V8uB8L8/VZHt7Ynocji4X0dkyRTqLv7hR/8SZSrDqNK4ir7PhquK9KneVSLyBHSz1th5tpQaFhFKF60PHpqo7QrsKc5z/mFOA9KagO50rV1EScxcWLCKtVEM0b7UKqg1r1jGREf2mo7BitqXzRqt0tYhkhLbkQ= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f0d48c7a-c6ca-4080-3991-08dba989f190 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2023 18:50:13.4385 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XCjLtkeS/3muNw6Sgm8Uy4PcauxCtioNilkUhMP9Ko0i24T1BXzGpDp5nxREY8oamPHbjZ5QZBLuvuMBZYAJNjJSirzMupFXjhcaC4YrHMI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR10MB7897 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-30_15,2023-08-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxscore=0 adultscore=0 bulkscore=0 spamscore=0 phishscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2308300170 X-Proofpoint-GUID: CPje_lIh5Rw1YeWA3jtGvcbAznD8HBQm X-Proofpoint-ORIG-GUID: CPje_lIh5Rw1YeWA3jtGvcbAznD8HBQm X-Rspamd-Queue-Id: 7972FC001C X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: c48ua7ccepciiepa6saqrmoyqeodx1uu X-HE-Tag: 1693421433-950059 X-HE-Meta: U2FsdGVkX1+11KhsuXwPHu/AyijZJGjln8Yy9q7eSU1JSYkulJmmKpK5/GGdONPEmWSu9IATr6SVohvdY0QpRShTga53S7VRBT31WPsiMkGdMj8sKrqjU0RUrOuzFbjHMI3Ouh9fBejr8ibYhdV1k9lNTOUaxaaeFEqUlK/xRyvKqptrnKl8uvk5XDNHYCF/L41cKnN0JIx+s/VMawrh1jgQPPwbAan1jg47Niba/+DKorJN1mcxawawWc2M26FoxQ0T7cEQ1nS6IG9E3Op5GokRXzuLJiy/ORDdQm5kzoPD2O2hezrXVRlStfceBTsSipwQKAvm9I8+2upczQkY9OJo37aEoyTDTgS3BqPep3se2gRvqxTieEVi/JQMcUIa77S7NDqBW/Dl3QeR4jo9XDd39amt3UpIhjHrPG7J0UxS9Kim7DnR8QEAYHVVyAaC43T2+Qb88ZIK35+Vvfsm2/s3akjBMRrAhVb7Bc7OdmNxQrcWUWAI7ske4LukHMKmQvHAYEtmUAoBmhb1URNa20F2xTYizm9oZQagRO2Rgg/4FQKIRMCjnuVrdENAR78nTA5ZBoPX7v+XPDUn2+5ZjGKGCnINLIGtfwOjLGNgWirAzX9JvBiBcYxA0K6uWxJMiBnHPoaBIEyDn9qF5wTs9yEDC4tpYKaBIzswwkUODkXp12ewIMfuJoN1tiXinH1zsuyVdfkgEX8JS7uXiB1QuErZMeAOJnXWdhjuzdEhFB+NXG8bjt5m/Mqe/v8VOZo/9HjLgPwVqQJDSS9knU9Bq1oTRXtYMBfqzcT3XNXz2O6k9PyV0zt/LAWrZN0rjdx3TILXmPx2eXko1U/OoNSz62+ytr0XJoMmrJ/dVC1wadLvXjc5w/u1KUXov4sZ172vjb6VYv2NkqQZEjPX12m0kvFvHQAf7GtkUu8/UfXwg3NQ7lmMIcBZ3/+8SeKO/w+F7uGE8e+AZcQC+KSrxLa QWaCbifG xLo0sG9Xj7nvfPq24UKtP/9a/hMB4abPN7YzoWnxHhtLB8i9QJZDmgX1QgsLprP9INjxQU4Cxaun/iOotsMLKSodVgqNNLqUHiBhVv0LbezK6xYTA/FaAI3qBxTncs8BsJfv76WFQ1xZo/1Sih0idhAzDa5E+S+PnBlKUZA227pyJ9eTWstBpC7t+x02b9ISnpkB046Bh3Kv1KNZdffTkLrdnrZbdBATgpGqrTjRxdym3sGUsbO9bYXE+nL7YvyqKb81w0FxvCHL3j6olCjj9KpaddYtlNIpUlALtJRrVDDXenGrMNgDDI3VCOJEJ+khHa86f4+8bV/pBMjoN91w+dMvj89k6+bFBBuFRqrw1ISz575cTV+vNAJo8oWmjXejrHI8qgrUPrbMbLM+/mlryJIqD9StCQxzDoEY3TTGnR+JezVjTNuvT5CxM6JOX3tuREBFheFj+ggdHHnpocK8Az9Ac+G4Z2J1SEP91TT2uOESEP2P94hW2CIYa7hSyCbNFv5ci3f7t3cLLr1Yk9eDBRf+0V/21h5mtrBDV6XemcY2idIU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: clear_page*() variants now take a page-aligned length parameter and clear the whole region. Rename to clear_pages*(). Signed-off-by: Ankur Arora --- arch/x86/include/asm/page_64.h | 20 +++++++------ arch/x86/lib/clear_page_64.S | 52 +++++++++++++++++++++++++--------- 2 files changed, 49 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index cc6b8e087192..56f9adf0e892 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -39,22 +39,24 @@ extern unsigned long __phys_addr_symbol(unsigned long); #define __phys_reloc_hide(x) (x) -void clear_page_orig(void *page); -void clear_page_rep(void *page); -void clear_page_erms(void *page); +void clear_pages_orig(void *page, unsigned int length); +void clear_pages_rep(void *page, unsigned int length); +void clear_pages_erms(void *page, unsigned int length); static inline void clear_page(void *page) { + unsigned int length = PAGE_SIZE; /* - * Clean up KMSAN metadata for the page being cleared. The assembly call + * Clean up KMSAN metadata for the pages being cleared. The assembly call * below clobbers @page, so we perform unpoisoning before it. */ - kmsan_unpoison_memory(page, PAGE_SIZE); - alternative_call_2(clear_page_orig, - clear_page_rep, X86_FEATURE_REP_GOOD, - clear_page_erms, X86_FEATURE_ERMS, + kmsan_unpoison_memory(page, length); + + alternative_call_2(clear_pages_orig, + clear_pages_rep, X86_FEATURE_REP_GOOD, + clear_pages_erms, X86_FEATURE_ERMS, "=D" (page), - "0" (page) + "0" (page), "S" (length) : "cc", "memory", "rax", "rcx"); } diff --git a/arch/x86/lib/clear_page_64.S b/arch/x86/lib/clear_page_64.S index f74a3e704a1c..58cb1d6355a7 100644 --- a/arch/x86/lib/clear_page_64.S +++ b/arch/x86/lib/clear_page_64.S @@ -11,20 +11,35 @@ */ /* - * Zero a page. - * %rdi - page + * Zero kernel page aligned region. + * + * Input: + * %rdi - destination + * %esi - length + * + * Clobbers: %rax, %rcx */ -SYM_FUNC_START(clear_page_rep) - movl $4096/8,%ecx +SYM_FUNC_START(clear_pages_rep) + movl %esi, %ecx xorl %eax,%eax + shrl $3,%ecx rep stosq RET -SYM_FUNC_END(clear_page_rep) -EXPORT_SYMBOL_GPL(clear_page_rep) +SYM_FUNC_END(clear_pages_rep) +EXPORT_SYMBOL_GPL(clear_pages_rep) -SYM_FUNC_START(clear_page_orig) +/* + * Original page zeroing loop. + * Input: + * %rdi - destination + * %esi - length + * + * Clobbers: %rax, %rcx, %rflags + */ +SYM_FUNC_START(clear_pages_orig) + movl %esi, %ecx xorl %eax,%eax - movl $4096/64,%ecx + shrl $6,%ecx .p2align 4 .Lloop: decl %ecx @@ -41,16 +56,25 @@ SYM_FUNC_START(clear_page_orig) jnz .Lloop nop RET -SYM_FUNC_END(clear_page_orig) -EXPORT_SYMBOL_GPL(clear_page_orig) +SYM_FUNC_END(clear_pages_orig) +EXPORT_SYMBOL_GPL(clear_pages_orig) -SYM_FUNC_START(clear_page_erms) - movl $4096,%ecx +/* + * Zero kernel page aligned region. + * + * Input: + * %rdi - destination + * %esi - length + * + * Clobbers: %rax, %rcx + */ +SYM_FUNC_START(clear_pages_erms) + movl %esi, %ecx xorl %eax,%eax rep stosb RET -SYM_FUNC_END(clear_page_erms) -EXPORT_SYMBOL_GPL(clear_page_erms) +SYM_FUNC_END(clear_pages_erms) +EXPORT_SYMBOL_GPL(clear_pages_erms) /* * Default clear user-space. From patchwork Wed Aug 30 18:49:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13370628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE982C83F17 for ; Wed, 30 Aug 2023 18:50:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 30626440175; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 151EE440171; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E23C3440173; Wed, 30 Aug 2023 14:50:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CC6C2440170 for ; Wed, 30 Aug 2023 14:50:36 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A38FD1A036B for ; Wed, 30 Aug 2023 18:50:36 +0000 (UTC) X-FDA: 81181662072.01.7CD74A1 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf25.hostedemail.com (Postfix) with ESMTP id 34FC2A001C for ; Wed, 30 Aug 2023 18:50:33 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=RK56wnOu; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=mQSij2sF; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf25.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693421433; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=f4OnxoLyqQLXegBCEBZgTprHs3FXRbEDL24PiKFwl6s=; b=t8iSZRC3mFBbmyyQRUB7Vmf5Dq8CuykVxmNY4mh+rRdm1r2SddhT9uoka3nFR42b+L0Wke 9HNmkceunLOHeZ4GPQ9CdXue20TrpuZU1JpJlEMIQlJP9UZb7KETMxHWdLjdaJrBrbt3P6 WZO2qWfOOKQ2dAg5GXPtsTZUBO0oO1Y= ARC-Authentication-Results: i=2; imf25.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=RK56wnOu; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=mQSij2sF; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf25.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1693421433; a=rsa-sha256; cv=pass; b=7k/E0cle6DMjM7xwks8iBmiDKg+INbW1B3gBTB4G7MAJ8KLOJpOaiQgG5dotye0QRiwCMf fs2dIJHNjWf03VjmpXPdM5rzz8b3Z6U8ljigbPbUn7ymj9e7awZDu1PAyEyNkp9OV0eCWn Pr9QYIJ/n/MhqH/a5M4iZCggc5JahRY= Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37UInWYr032351; Wed, 30 Aug 2023 18:50:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=f4OnxoLyqQLXegBCEBZgTprHs3FXRbEDL24PiKFwl6s=; b=RK56wnOu0QkZnqxXtEboZftEvj3JUv080fonk/+0gHTOOvTw7L/cMPx+AFJN6AdHcBil iZBWRJCt+1lJsvtm28C45xjyNhvEl6hGU//2VRSr6OP1IhkX+ZVQ5tW5p12nX4Yr60H1 WgWtAUU/om0pqn6uxoYD7qX799j0yA3SFCE31QntrW78sY1uo0kaEPvNNjRuvmjONUGv Mz9qX28sHojw6zHK9T+2wOem8eYsPufN+ZlyiGupl2ReB0/qNaggRaYBIInEYygBXGFZ 4ChqchpSHEuzwdaaNZT2bK4y0z+uH08VZPiciLbUReSjJt8CJCLzlGw+kY/pZ0N0kEE8 Sw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3sq9k686be-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:19 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 37UH8ZkU024310; Wed, 30 Aug 2023 18:50:18 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2042.outbound.protection.outlook.com [104.47.66.42]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3sr6dpxdpj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H8Xs569DfhBMDWIx4oYi2tvNYohrKLluzf5YBgQD6Zd8pZUJ4U7EbXLEo6Ujq8BMseL4it+qNQgluUOamBna3zsD8lMVsoXKJy5EosExOf+FT8OtFjRCcN6ut/ZdFzgua4bkDCd3suFnjhLGFjlOC95xOC7oJnqNqqELBNDnWugsVk7d/ZtrTmDAUKvk283UWPjpXcg9VKn+k0nhm+jGv97NWX0Dijn+93Wl3Y1sXIyuGPPIkpFSxguHkH+91pbFva0JrWdjmYDaBia8YKMyMkLuaJFVL9V0uOt90x67qrpAhkYdDoezUjB/N2CQ/79EBVvCXLJ3jCiKsnB1iAsoUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f4OnxoLyqQLXegBCEBZgTprHs3FXRbEDL24PiKFwl6s=; b=HCG0LwtYtWklSbdIm7GxY7e7b274OOIDlZXuNk4GeZUENMy6He0A6SsGl0ho3XwxmlzL3ms5Z64+2OFGW3fZv/lP36RqCUPj3Yb9AiSDUv5GodhlG+3BYXqc1rMzoYtn7yUTXJoGy2Z+hCDClp07Txc6JPJqHDmOutAQTY1uMLa6incS7PdMUmyQ+QJiNk3/7pW3lcoP4cvnT5hjHOiC2cDiwowRtTTBgwwaIop+A19F/EntWrXPQDoS7TQLl/1y5boJvH4KAEgMv/p9q07/ln4ode2baN0QtsohEweK2yDY0VtiiGqx0e4hh69+3AwdCjGqIytA/QFywzmxIcabqg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f4OnxoLyqQLXegBCEBZgTprHs3FXRbEDL24PiKFwl6s=; b=mQSij2sFzFH8OF49dSJHo/HEyZWcj9i4r/H3CJfmd/InDcvKDv+E6araT8qKmOdpxGzrtCYWwc0DGqidrLrU+E/UZGtmXq4JrKRUoAzSmUwwDm1Ha8ycoOzYV3uOdTF38l65K71DzU9CAgUKfiLkQetx89ytqmLwJS/ZPzNChFI= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CYXPR10MB7897.namprd10.prod.outlook.com (2603:10b6:930:da::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.20; Wed, 30 Aug 2023 18:50:16 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415%4]) with mapi id 15.20.6699.035; Wed, 30 Aug 2023 18:50:16 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH v2 5/9] x86/clear_page: add clear_pages() Date: Wed, 30 Aug 2023 11:49:54 -0700 Message-Id: <20230830184958.2333078-6-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230830184958.2333078-1-ankur.a.arora@oracle.com> References: <20230830184958.2333078-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0037.namprd03.prod.outlook.com (2603:10b6:303:8e::12) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CYXPR10MB7897:EE_ X-MS-Office365-Filtering-Correlation-Id: 15522295-d22b-44f4-8411-08dba989f376 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: USGohVyEfoXKqZ7N8gal4AAUWfEUluDAJW0m8qk38KHSH2bKnMBTZnyx0oq60UU8QRtun2Jj9c/YCc8I9Cu1kIgIZkcZ/X2YXNGHZpSYybk1C9EXxyu2qXXZ11wIQ/ZKEJvwR1DcR7Ed3CdM4/Lh67ckdA6XG5HraPBZdeUAP493zzB4kdZ/OIIakf9YNFLNFGcVxrLGQL/OO79lh5qPlaQdHmjt5LjosZEhR9QDDRP3t7UoqIrFcI5qR11HvVYfoCzPqM2ccxODnAAr+Z3SBDP6IabOqJ7AkFpI/LqVnUJ7qVr4Njiiksl08/ssdV8qoIeQwwITDZz7VA6eSsariHcRKasfi7ZUvtMpRNK/D8qDro5IfKjwxAj4o7ZfUlS14E+OH3YvA/jMnen96e0eRb5Q/1Vq3JRIfoDrrByeCioAW6OFAz92mKIbxZIzKs+M1OyWXLvqd5hdRxdY2ZQljrhsLqvficZFlv7DgtO5j6OaMbtCt1wv0lAUs/Z8/Bpy1YfRoFvXX2/w0QvI+su5lkkxrP43uxBYpO+ZdEFs+HatxJUjnHknpoOnQKstbAYS X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(376002)(396003)(136003)(366004)(451199024)(186009)(1800799009)(478600001)(38100700002)(26005)(1076003)(36756003)(66946007)(41300700001)(6512007)(83380400001)(2616005)(107886003)(66476007)(66556008)(6486002)(6506007)(316002)(6666004)(7416002)(5660300002)(4326008)(8676002)(103116003)(8936002)(86362001)(2906002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: uTu5kpS8sz1hUDLsjANxDWQSDL1AHRQ/Lrwzo6DdSKtBV2JVi7ynx7DVfukChaK4TFII3bjIYSQrcXYQPrejm+V2jLNbLg7hezF8EM+ireVXAB/lN78nveP/9Iqof+6cPPxxQLQqaKXggdkhjorhd6mdcCc39SRH9KeN9/XTHX7B2voVUHL23H+W69vvRkLCWLHCIGLyi+QglzK4e0EJ7yRrBnPc/nVLOZwbNxFUtZ+3UyXYaOq9hpLfvyJfNsT/aQLcnXl8XB/xo4r6x0wD4i7+JfAllEqVmtBa4Aws019Oahdxw4Vp6jY6AtxY9cNC2LQ4euekRezAQtskAssCfDgQde4rew1QVdtRqY1kxdPkRdJY18SYV+Lkd4sfkO57CrB42356TCZmnLw4sk9Q0BUIlZr1O73J5B1LISRhixMcNzueELnx9/fFIxj7Ty1wll6BcqZlK0Xp3jA5hR06d3WLJ4wTHX3zJOsrRzmvJGVP5o0cqA+ad2DbWyGfo1MWReO0VKYPiiPewyYOFkM3UxJX4OIcFjhMwPGoKGzygp98Yuq77TP6KCkZA1T1Ezldm40W9NnjDwgqzn7P/cHB+cOOEI19U6C62YfkUhFlOIxvO5x2NLxLpVaz/tkQQC703wBCJ9DymIQ7RD7ep4lC6Vt3/MjfZBfs2NUpoSKLZZaLO9778aPkrmn/9pPmcY+89RK6o/7Y3bpNFv8NOTcl2FAyssEj3Ub0qADaGuPKh+oCUmvE5i8u5fuhVWpwFr6tQbhUDx/MLMFJKa6aV0qPKuuZoj30chO7q5nW6pcu9/OYkrVmkI1rP+C4EAC8QBPGYBUJgrg2okLTZz3QYvK7DHb53NgB9LTsV4HcyFqhCWh34bFtnoGUGD8PpvKIPmJeTPlqEMmQ6KCyg7EDGAo6gmSBnkpU+V3ajqqvHW9ku0ilQNW4bJr94NfrWm0f5HBuOYz0FC34/rbv85R0RzS9kVcdvBxd5Sqb5sDAEZKxiGxWvuSqxjEmOrsk4kozTwpYfZCqU7TE3UtshBDAPUOWJ7Y9ZVIKH1yHkuvq+k/90yvfiY05yHS0DrlUjAoY2a7FGN7azisGjGowW7dICnAixq4EL4qvikqQYNDyekAAPXpVfAH8jPBw9zAS0Q/G+Bl4i9PvYDx8jfxoFN38XM3hNGg8LFU2HLULZlP6aBh86Ch96GNQYR8mTXxAGRmZJ+Qg7Optu4V8wK0MWejXw+L/SyaCyz4UcQGsGmbr4kqnA4Ru3FrR7/SqeqaCK/+Bu9mJHDgiHkPRPfa5snuobhjzZF46DkAbtlWNyIsaOjUO9jQtPb1ZSI1AwiYgV/0v40BTaa7vVvU7jXue8PS0JFO5f1ct6qc5tutJcLGDdfE2/mx8RYG2uNCyOlKKWC2Lnv2IzD8wLNjRLS25sXrVYNME6QKUBtK2mfHhxIEKRmrQ8hVsqKTaaRoA5xCfZiEcEVXYZo//yYg6/gzQyQYiPMscbCM/lrhITtmED4v0ly3ayu5jHHHf75BQotSPFggOMl96QSxUYxVpXgUPiXR/REOSxFl9z0U/kIW1vOJz/LLnTGkzTbc5gQwFmhOVUYJNSGeU8godSggM301DFivu6BTynw== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: ehboCqBSLkxqnQy+q3oarupS1pkV6soB3y/SxDVSzvuyROsGoRfahO6VDGmz8k8r8ISv6b7rIiU/zMq28uZ+5MmrHhMt40LeC3wlU0tfTVWea9bi3f12qVuhEXYqNq11Gox/txzKgIw4+M6lo0X6VJ+4SFmLfoP5elSy/ka+cFwdEo7uhKVGY4Vr1frvd6ZAGUL2o7ZRJDnrPrpnitT6AmobFDhy6NCAei/o8Tsf6qrrEODF2Jv2duQAjeWjb5qtXMeEUUAqjVONB0LtWW8hJ+evr7qZo5a1gESuCYUeMpX9NXINmN3HrUYNhhA8uaeSsGHA1lppA1E+Q7LKTVqvYgAHwgelshFzhISdncwdtKsbixVeiKfzBVp1ZXy0qn2SilIKUoLI1Lwd14n5FU0O/Bx6aa3j1lSU1YBk1Wfr++z6j/yiEnfbTYsPkfspgm3mb5i44nsBITV23AeD12xyk1vsOXUmeAGArjuEct9+/899gMAynCPnkUyfZUOesWqUpfFIlyhefH4F7iq9lC2MxCRX2bTqlI4D2/EtZ5PPqewcNGZOGaEbb5it2cLpMurR1oSG/TnF+zDopTvt3/MHsD4sLf8xFqUeLA/Pod3EXdmZ33zNeVWH3oR0sibjKR2bx2eXmVcU1oAKCEKyjYpgsqZmx7IO1NK6/rKOTglF+KsldIrchm5XX6W3GLAVKxTSkqBPKyfCTuImbjI9CdmUlH9pLf/gfjl9nBF7rNo6FWKSxj0BZ35+3V+ct+H+KrgRTP/SuI3CPOG+FbdBtmHpXqM+7oLYETKKSNIDM9FTRby5MJbFTR6sFPN4dF5lofK4OHzS79llS6bkUSCHfFq6+IRMSdhIBMmvjI3zZh77usblK3w4UOcFRPtkXrTXqPFnAw1Ou8RnuE9D7LBowhf/8rK0x2HZb4o+IfWZh43a7GQD+KWFGpeui99CbLUaoe905Ae4L9aPdIzlo3c6VMVgbgPjDmvRD9OvBiP435+p0eH5k9Iak/roCu8S+DJezkbMiJIPR4XOFTvb9eqx7+TbhxfjetmB/6xfChP7TVWHISZiAGra7kfmQuby+Ec5MBwGV8+dHFWBoig6PTDtBWfiLQK8Ony3t3HHKvke7dTYtpYxtLK6vGUhckd/xNsODEFc1C30La2eTNmCRJ72kVl3pDq8vwT76/0+auhaxA6XuVo= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 15522295-d22b-44f4-8411-08dba989f376 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2023 18:50:16.6142 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: V+i1tj9py8sU1bIso13RywBga3n8zHMg0IzyhH9xiDd2k4p1kidQEMFLFUn2Ox/b8NMg7z6xak7NeVxboIogunw420RRDm5kKUiOY+2dVmI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR10MB7897 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-30_15,2023-08-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 adultscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2308300170 X-Proofpoint-GUID: HYKnwjGXzyMIkziu8CDz8UyInc4rrSH5 X-Proofpoint-ORIG-GUID: HYKnwjGXzyMIkziu8CDz8UyInc4rrSH5 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 34FC2A001C X-Stat-Signature: q3u5b3rafgcbwutwwpnmf577tb1a9sg9 X-Rspam-User: X-HE-Tag: 1693421433-320425 X-HE-Meta: U2FsdGVkX1+8U0wPTeBfPBxkAeQ+wIEtZTEMty0ISKTOoS2uo//Ta7/3THD2t6Jz2lRt5SaA64jlIdhM26q2ZevcJ4CmMNLwaPJBECvcZa23NTSeUT40E9xaNPD001N9fs4bws/8C8NEHk0Ad85pE/jTZaA8DnP9fHF3Gv+SM3EYZi4VkF8LiUvT6zG2ez3njO2q/fTpoMKMstqqwAiwZUNj5ZbNH76kf3aZyFqJGD9QX4B5yVDcZI7JSHozCmPKLZwRlvPWVE6BdjsHRThPRFTc4a+gOkdCg9q7In48d1arVaP90/OzfEomxtY0zSZX/SxO8hbmK7fnnUPAo4RQbzSnPqu9AVgNW7g/5Iup7Av+K4PpG0U2sfdVNnORCC7XWovrIhK3mQupD9KHh24a5pzJrYSjunCeH7mujKpZp6mD+9w6dYzfimBi8mSVODBu8gb3cK2T+OQlDRtfcF3PRggyjzflU63geMz+Rr0FsFCh/BciWUcM49cgOfD1TdMhw8aijnk0a/xDRLytJ6Ku9RXkKNrGvARzR7pKdTBMR4UxI/nqNfJSmPdnf+yXVsg4OsJEA1kGxZqmGVX8uFC+32ZVbkMeWXYbxaCKg2uVVsf/43psH0EudEOWWdTUO3qjWhoPIvyQ1XYCSoq/JN6bg88lzvj/9v+LLZvmyuTEZqVksJAujcFyJihnDMJCSd4A7AzrqSTykYPoHBeR7VsL/S3b+RYU5G4LwgmbWP9VQdHTxnbX5/A0ovLW+zLBaoMo8P0/u9vXgcuZQJTaKHSrUKF1pKoC57VWSvCe32A9PGUwPVrwHIo3B2D0sHZ5+6nBxbT/pSO9kYd12Co7Wpq5h1OQVD1YS78uYMpd//oGXLyTsvjRXFmB6FPCUDKMNC6HZkbn3vQ7ryn+4dEXh5vR7xi9vDYSxwvCfcAh/YMq1aRORCbICvmCvgyUoKjTn44p9ffZlAEqcY23Sh9r1/Y vOaMdubS WQlS8q21niAMG7YR6QotAUyeYXGAigiKKyDMGs9fm3HkYUfFilzcNS8JPsBHKvMLVA2PgVTXyqqh9BwY4FBQ+qq2Aj1RYK1tWRWWMmnA5N6f8sRzQt2yRT/MVQiI5qpXWyP/OSYxM1GlV4Btob8tQoEXhZzAqmVeYkQzY3ShhW9M5T709nKNuQ4wUKVTsDGiisQTngOkvWRmW/i3nYsBxZjFlx7zaBsIRN/Sfb0OGTl3Yt6Z+QfNg+HxSqjXOPwHfWGeJ3ogBSB0d4RsYGTwTu8+TZJ254kGZ7ZH+LjrJtYb6Kn0sz97FyB5X18ab/DmOaoABEo/LcZTe9XuUuRzT4kBBHZmNU5hLm//2afEGPl3YAWEioFANPsMldONci4kpsDrtjqzeEdCi5ZV+r1dzyAdH2wG8JwXJKzjVL2B3O3+jZ3i3GkMXE30/QK30Tvf/+sq3DLqAT4KgyzxgZdZKA8Qe4plCsJtgYUL6x5yPdgN7RD7BbIznB+FTnERgfJQiFOF4+I1sWAdUz63BgK1JqxX9UsHVU8QnShUbRJdFV4tpj/k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Expose multi-page clearing via clear_pages(). Signed-off-by: Ankur Arora --- arch/x86/include/asm/page_64.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index 56f9adf0e892..1e71570ea987 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -43,9 +43,9 @@ void clear_pages_orig(void *page, unsigned int length); void clear_pages_rep(void *page, unsigned int length); void clear_pages_erms(void *page, unsigned int length); -static inline void clear_page(void *page) +static inline void clear_pages(void *page, unsigned int nsubpages) { - unsigned int length = PAGE_SIZE; + unsigned int length = nsubpages * PAGE_SIZE; /* * Clean up KMSAN metadata for the pages being cleared. The assembly call * below clobbers @page, so we perform unpoisoning before it. @@ -60,6 +60,11 @@ static inline void clear_page(void *page) : "cc", "memory", "rax", "rcx"); } +static inline void clear_page(void *page) +{ + clear_pages(page, 1); +} + void copy_page(void *to, void *from); #ifdef CONFIG_X86_5LEVEL From patchwork Wed Aug 30 18:49:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13370632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4A13C83F17 for ; Wed, 30 Aug 2023 18:50:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0FF8D440173; Wed, 30 Aug 2023 14:50:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E4954440179; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7F5B440173; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5AE41440178 for ; Wed, 30 Aug 2023 14:50:37 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 360BF1A01C1 for ; Wed, 30 Aug 2023 18:50:37 +0000 (UTC) X-FDA: 81181662114.28.3C26EF6 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf06.hostedemail.com (Postfix) with ESMTP id D1AF8180014 for ; Wed, 30 Aug 2023 18:50:33 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=kRkiQc+x; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=MCadMPk5; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf06.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693421434; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k3dumfrrLftC0rbMzucfNY2caXdgyJ52sKyZgZ2BJlk=; b=mLVYKeNkzzA8VfhCGN5reK4r/+DFs+cwjO9GGrZ/H/4xyuaF9abZ7LfcPvSYtaWw5yPE28 uSK4/jQlghuFdC9pV60/lZVa7eVTMmqUKyRbMHu5sdntYwLP07nlNl4h2UlnE1HXHabOd7 ONiV8ucW8cuHXmA4J8+Ne2Tg1gqgXaM= ARC-Authentication-Results: i=2; imf06.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=kRkiQc+x; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=MCadMPk5; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf06.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1693421434; a=rsa-sha256; cv=pass; b=uMHBhgRGjEX6BHKA9+DcxDrf+r9U/swH+N4FxO8n4SqwXaX5zxzX0Xu48vb4XLfl1mcPkN K66nvwITr4s5MGM/aJ2OPnJKErARtsqFVTEGTjrLqLyWpMQd7gDZOdNudlcHCDC+AABr+Q vPVgCsLh3h0nMSEAo1kVv/aBYlfGeTw= Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37UInXEB032355; Wed, 30 Aug 2023 18:50:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=k3dumfrrLftC0rbMzucfNY2caXdgyJ52sKyZgZ2BJlk=; b=kRkiQc+x0Y00SD9IonzHq1CPYPEIVCDZq6p2fUDPausThUPSy+o3D87R0kw1Z/kchNRH IuIX2PEpmQ5Mlb3EAuQvJcFlvN0BV8o2QJ1Lo5E4aLmfWDvEbgrmMXFwx+Eh/3K5wJIa FC3nqw+bdzWFAltBHI8QeBpS4j/keHzA3lWlD1tyg3Kv6y5Nd2amOMTM2Ux5HhWaapXI JK6GwUZcfPHkzRgLr8qXhWAfM7tKowT6ufX+A3x4vPdV9DjIDppunXjYEC/iJzGv4R21 Lm4lL5+Hd05w3jl1y3YvDihiGNVxfDeLk6wJSwtVmws/+pCrGdgW1dBTSAFI2D1KGjiT Vg== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3sq9k686bj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:22 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 37UIDLKC032817; Wed, 30 Aug 2023 18:50:22 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2049.outbound.protection.outlook.com [104.47.66.49]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3sr6dqehre-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:21 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GmDEIYeRanHsCPQYOL3EvPSmsSrNQ6OBsA7XY2K1HvQB+rJATqe2SK5nj14mQxOhLUua7tpf338DbS0lxIFWFR2EHuges3T4CTMnjGsk2A8gwr8ZDpvBI4OGW5VR6IdWdPoqjU5q2YzwD0J3cNmqHDjEMvNiBdpiPDPXfKz1xxGymR/1J/dP/G4kFN/9l7VRxWn8LCOkwt47a4vP1l1fNaZ+jwxbQubHUgMmlO601b8ejDvMiGSYWmANC+H6gPOVwN0bA8K/iLeFoM7Klu+J4/2YmIdqmSH4w7oSj+R7OnrlxIY9GgDBUt9QNkv8UjYAzd0lXwdBBumynx49COgMgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=k3dumfrrLftC0rbMzucfNY2caXdgyJ52sKyZgZ2BJlk=; b=A+sQ06I7i6ywJholBgVjvTsEujiFiePhbQERpH3WoFwN0fLfxi/CwAqcN+E0kOJlL3/Qz6MpQxBtpZ3OFCRGFHhShQooVFEte1N33BT0eYJ/gdvhROPbhyE8g813J3ZlgvSi/sgCaSwpktlrhyTj/c9rR2gh42HtjunRUIzqyVWaYih9KliXPdX6JAF59E7H1ksYJ8I2o//cSL58z5kOEmvQcm6kpW+wax+SnB9vbioAqtzm8IcPISwbabH/P5QpipU1uHOPTzr8b1qLNFOJ3SaLSna7L6lXD3tO2v8fBxfQb8KB7MewLusMD3ZeNZCO+49r0N2ZHILwQtDP/j4rwQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k3dumfrrLftC0rbMzucfNY2caXdgyJ52sKyZgZ2BJlk=; b=MCadMPk5Jw1+KGSUZOG64aA7sz7z5Jw++pZVpjBeUVKHlp9hJjGrc01Cd/pZMhxHbv+Gyg+e3RrrRd+gVeIwVKxQE7I6/yCoMF6kGIKUmDD3r62wXBGEamFliNUAonSgg5J0x92yrYWccCiF1YYW6BZhKoOIVntNPF4pifII4II= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CYXPR10MB7897.namprd10.prod.outlook.com (2603:10b6:930:da::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.20; Wed, 30 Aug 2023 18:50:19 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415%4]) with mapi id 15.20.6699.035; Wed, 30 Aug 2023 18:50:19 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH v2 6/9] x86/clear_huge_page: multi-page clearing Date: Wed, 30 Aug 2023 11:49:55 -0700 Message-Id: <20230830184958.2333078-7-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230830184958.2333078-1-ankur.a.arora@oracle.com> References: <20230830184958.2333078-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0318.namprd03.prod.outlook.com (2603:10b6:303:dd::23) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CYXPR10MB7897:EE_ X-MS-Office365-Filtering-Correlation-Id: b67a77f8-6007-4ad3-5c00-08dba989f547 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3QTrGcnRQXwG4GAP4xpIQb3YPeleqOAMiPJMKgNoAMXqLkvCw2RO7wCYVSaiI4HFx4hnCEzgOKo55F1n5hKH/q5WaKQ8vHQgU2THGowMw61b1FfB7v/N+Blcvw2H7hP1J57JoIQml7AvhGSwNY7TEicTTwMOod3/MVYerl7wxXrB4Z/QW6Fdr5rKBEL4BD5ndcU4I+z+yT/t2d+5jbStrNqJWjvtG/edR8g4EON299ygEtcyTqnFgus24xextAzIIexGWEfRnE//HiCCLA3tuHYryJxqOMhE/4m7iVgNKwT48qi+117xkivnEq4QRWqNsp1tq7uCJtREPrC4nVYHc7D2EbmcvJesfZkaVxZRg3cHStDAW00so+radWdaEpmUtVPZQVsW9Hw+QOuBqHCI3/N+TxkUL8L4+dnKsLnnW79zgOPFYBMqiz5zh7n3SJ9TXoABU93Oo+rBtfeFdKW94mL5lleRSS97iUW855bQV/0WrO00X/61VC8kTwIezGnNZSgXaE7Hml+pj9U0YEvbSWE2H3imNfuCsH4TMss2L6KRsMUCuSOoZWFoIUHLTxDQPfeTSQKPivWYCS4cJblFrPLQu1mSJ58U1mhxyHKdBmY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(376002)(396003)(136003)(366004)(451199024)(186009)(1800799009)(478600001)(38100700002)(26005)(1076003)(36756003)(66946007)(41300700001)(6512007)(83380400001)(2616005)(107886003)(66476007)(66556008)(6486002)(6506007)(316002)(6666004)(7416002)(5660300002)(4326008)(8676002)(103116003)(8936002)(86362001)(2906002)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: DA1kNghMTEqQZ00yWf5AxqqyuX62HKg2M6s+pTR8g9PBEaehhEm/TPHHwuhguN8NIA0VChvBm2wrVsMk0CWa8TIlZUzdEHqjECS5td6XwH86BTbQJ3m54/kP1fSLIrXdffLGZtYpn9krzQrSdAIhNHZfr7p/nKGGROUE6yngVbu/XKFp6xidqPGL5hoYdbseUX0NLgL0MWlIpJmiFFtFL5jJ62f8fjqWeWRBQAOLxObHSgQPLJwopg6GcIramk/YBQ3Tpm03gh1hYaNf3+lx5x0rD6jIxpvQgIfTYPxuvh2VcMKn20jASFZYwqiX13Pn9xhklNsklTw3ayCtcyGE/VZMKGnrN3u1Qw/sD5U/AwXUMC79sBGfZ7unl0WecrD2VeDPctqCb1UnyNVuY30edIY5J6eQLLkbt96Zo6C7R0FhVAHR4I4Ixl2EQ1RV7HysIL/h9ZQ1Hz1NQOBczJc2CMr4sdgOcXeenFkIT2/eZsCfAXIWUqi/NZk6mOOiYrqTy12dkcA0NBQ6B6laAEpRkGEGxreNuzgX/yTDf1qjL/MCFMuuKyARsfzfJGviF/x3wLkmhMvQr0R1EympBZ1RfqsmXBGg2UQUMUZbNhSCAlY4cXDduyYuItqob3DyJX/TywWmSOnjoqgz5zazxzLKEug/jJ5z7qucnAFpmC4Mn7yeNiu42mqNAarFqM1kHLB38BvBPfw9KyBkw4Y+7y5+/aZfbiQpVd2CJuNg1Ld1v8kpCmeMryWq/LpvQ+/cPdqpnvHnPP1DrsSYRVQHhDbw/e+BVkP7ONC/6ZCdut3hucdHCWSf5yI7goxBZhDe7mlgC0/8tPhcS1MEuv9yt9OESWIQ7mDJ9JpsnXIfCCKSjugJseQLw3n79DEeoj4KlKVY9EmFcHAxH/xpxnmHA/P7D2T4oKX8N4nruYmD9/c4GlaHufEmxeCEps+AxeQZyFeO0xGbKMnHeHq03Zacr8QBo0auEe3qnngEQCsatYex7C5VX2XkiyfYHZCmB7J6lDHNgX9SrCi4W4svVrhUphlqHSDm3KZHQYiDP8D/A4L5sRqdFrIs62F17s70bCDV1O7ggK1ggRClK33lnciZLsMUoX14OaZLdT4C69/ib1lPlzOLUkVfEyJyJzV+0KfzMzLgjxfRa4WxNAJwMI1JavPY6uZN1mQkmJHRtHo4fCgjmVZHcFiZFblLP8K6JEW8Sse4xyEo6fN6p1F3FIaB/SzSf6h+0hVq8OJjUsllF1pRkqZ4KHsXLYrO1zs/LzBNqz8RGuPPCptLUW8HEpkz8IezQ4VgEV03/OAt4e5lMX4STInH4b5WWt+cKcvKpfhNcT79T6dwtVi8An1ZBgUaaO7sl7SVcHTnaGkm9YJzFc5f6YtFNjtas7toFtW9VORmRgWSXHCA+xy+d1/sm2mgYDq8UNHvX+y39dZA6pP/eQwYKXDakyzWTi5uD0Sro6EwQA6b9pEzib6uNBpFam5KTWLGGXMS+74x3nAnKvBwbrsI1gBbTzrzpqNF9NiNMZvqo7+3Ew1Em1NO+/5+poj9BBWIvK+4VnHqZH8IDl5v3S6NdwirTZulafcVQvyOxW+OhpdrkGfEFsVlha/Tg4DMLuwmFA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: yZwHKS2LEHYv7B5KTcxUQEeL7xtQCvg/wlnOxyMBuEizNSUrdqNuxZ0hmNduK9Nj1YaFCns8wyxrgH9pQ6NZIEXCJH2F9mon3lF9J9x88Lmj7+SRrjDsta+by0Yy+l3mCCaqs17jxZ2qzO21otBcevWABeViawcWZsICJYWZvnbKtem/73tjqms5z265oH5EtDq5BulElgwPqMe/n+6JXway6IWWesRgy6dBBPNnH1U6IlqqCDA4/Hs9Difu8rCvWUw3AfnztifKA+8NIVTLcwfHw2eqCqHYsqcvEL8b0HkXZ6L2DSTFGJ1wgaLB38crM75Y6xcF1iW5xaoD46lhJHovXCy91ziwauDALMytFAjG6jfg4oCkjuDuWJ0v4aEW7tj1SISS+h9J+41r5QfkytqTSylUHUGUi88atxviKxPXtV+nguWKc+G1vczTBgMtB8dQcnspM0ppaX2l6/hclI4R3BYMkabaQXlFguuVLKfRUp7aBNAKrwr3/K2zNamVx7hGRt++X0mSI7Sri8wvmxgpzjs//J3fS6CxhxBfd3D1wFanxlzeNYkFGdNq6utxwf+xSrXDgdR9Vz6VP6LvmCgyeVH3va//DcX3zgicsxcyE7P7CPmF9fjBpO1YqUIHzAvo7cTJHyPy1RmOuUMp07vN6uvGBwHlkQZC6nTCXj3u6vnNnigKMs8vFbo9UNcZtsPyZD3moSwuC7fl+bK+yUvkFgk4rZxPBr0xJ3si1PGGX1vJ24roVrAP/OYhyRM3GCqRj7hF6EPX2uHrGvwSXOgS+yzXUINsG0JguuQ385QW7OvA36Yjg6XOYUYeNWpz0BUhBA/EdyK4tsG2x4DYyReqgw1y3vYRTt2N8Go2GFLIZuRo05hIkQ3TZ883cMfTYiLx2hzEsxC+kegkQXv6Ea2jWSFVUQ3asCTjcu038fE8nWXG0CWg1omXsOt0ls9v4IPfoh5dctDydpX60iJpWTLzACt9J8z/lb9U6rugUeIzx2t7TmB7oFBojcwv0it8QQAdbecQyxgIx5pkM/gBBxdhCswoJP2aAn9CiMWAbXBWf0s5T9QZqDi85yUiYrhuBM782lecRMjMOFo/BictvYGMOqvHJa/7FW7yWnj7GAiAaJDt8KIQ+oGRfD7HKwXGJfEnaHQB6VKH1SA6jwbSpLr+hxGd54iGz7L6nVs0aho= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b67a77f8-6007-4ad3-5c00-08dba989f547 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2023 18:50:19.6471 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7mmiD+O1KCRYJUxXTlN/6aitQCTNwcg2MN3nKN/vlXq06hYG2G3gE9YwMhzDiUb+Z2Ppho/MriTqBcKaa2DPpN1av9/Jb+td73gxevYq91w= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR10MB7897 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-30_15,2023-08-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 suspectscore=0 adultscore=0 phishscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2308300170 X-Proofpoint-GUID: 4__312qXJmNbYqMLkgPDDwLakmlzmzuL X-Proofpoint-ORIG-GUID: 4__312qXJmNbYqMLkgPDDwLakmlzmzuL X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D1AF8180014 X-Stat-Signature: ugn3o63ic5hp3hw9enbom9u6pqf3989y X-Rspam-User: X-HE-Tag: 1693421433-740896 X-HE-Meta: U2FsdGVkX18QbXcC+2OV83hkACCPGUSoAsmc2bMAv4cequ58vdJhMSgtpjkGptbIr+i/Skptnj8rJQBPFfXJGrJJ4eZUbM4pz6xLR8O7bt0s9GVtip/IVA/mry6KOgJ4G38yiknkjn37ZiiuQtc1i6pXNww9LbIHMMrpc/nzYJYk8v/BzAXdekoWrehVlNie64wkJzo1+dNnJMiZydrVHyOe06gqh3R5l7TByzbyIKg4+6y780Ye31fbHF7RxvSXjAMlpDlEGja/waX09KQvnBiaZ/3h0rLga5Nz0+fAe5pjglbWmyxdjVVLw8KebiyXIoG3UlPCSkKb3BaZ6pJZX2bi/TcpC0vY1nB8wD8HrburK31N0UDra1EIkMIn6RmpUAsH4fBYdhvb52VMQmuRRoTOOpXzFxWMMM9x6wy1lEAW2KDTt5GNwq4+sX9g52zJ9bIb13EsOb5LAGfhH0OV2Z6SEY9rUK50WCzzYzB7qPqMd+BKCmHAK44sngCg1tGirG0uYh2i5W4uuN0M1JECmxeH8Un0FLCraiOmMAj46dHLubmZBnznBAM+J71z0f8XhN8Pp2jZw13mxCLGq2VFAzllmMrVoB+0+WyEe29AUzkUNYUka9/LcklV5z4wZSE/eUG1EIkb7dYdAQ5vpH12dAkn5cRdnSgd1WAUNnF9pMtHNZ+paiMMdIiDA96ga/h2KiZDHVue+jTp5I8ouuQT/VEqlOf+KKJa6riGTwvpo/597VMcfzcDpnodHUycldOdTHY/feQXtxTrtIYyNeXYsucpjHIjOEjAC9DMm9vF/pnEH6n4UzpjERJzOXpwgvE7oFPB0BFciDsik7FwRUkjxlINwm/E/h2kP7blsWpJimBvA8EVMKORViRs9oq7J+1c8aMriAqYyEN/3mjTMyl2d3l+7kmhvdWfKw6QGbB8/SqdoHsTfiz2aAKnE/+JsIFlA5vsXkrZguluKZ2XFAf yInP46k9 EnPKZ3EB9fmvH2ZoeYCJEw7H2pSMgw+zwtCMuw1ZDEzuU8dbbfUTiCFbbsFlOenIZ4rSNzdbwgcAiCAhJkSq409MvoMsyqXtpuie1XYb0bv79u4WV+JdByuf6LmW3QJYYtQ1vB99mtJ1e6+3Z/SfE3oRSxEymojlXN/uIQjT0KVsW2txgVbNvrsopiL6mqN7v7o0BFEeaa40zqM6xozzOxibcDGxbnH+QFaUe9HE5ohbFoas/PVQV7HEP8fasbzWJF0Qhar1nozL76DGJMsAXOScmtb7sZYicWCPssJkSvwnfkxniWokTnVJ8tWrv6ee+Ol8+V6+68t1a6YKiNVL92IbowK0fEUWRRmlkScKQINRSi4TC/jGKNyp0TOVbBLw9MjZQTYaW2yd1kl8S838EidinIbWY5Yovui4GcAsIYdJu/T2yJd/1jyNhrP+ViEECqMjwvAEEEiGRPkCOb2f86cd8FA1CZz/dt+ILTuRnEUszO6tnJwnR/r0D0zJUlhiSPVPFE1HpUoHHdKrKii4G1lB3P6pjakKe6eyonAuEGeObwUCnuxK6aa9QGA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: clear_pages_rep(), clear_pages_erms() clear using string instructions. While clearing extents of more than a single page, we can use these more effectively by explicitly advertising the region-size to the processor. This can be used as a hint by the processor-uarch to optimize the clearing (ex. to avoid polluting one or more levels of the data-cache.) As a secondary benefit, string instructions are typically microcoded, and so it's a good idea to amortize the cost of the decode across larger regions. Accordingly, clear_huge_page() now does huge-page clearing in three parts: the neighbourhood of the faulting address, the left, and the right region of the neighbourhood. The local neighbourhood is cleared last to keep its cachelines hot. Performance == Use mmap(MAP_HUGETLB) to demand fault a 128GB region (on the local NUMA node): Milan (EPYC 7J13, boost=1): mm/clear_huge_page x86/clear_huge_page change (GB/s) (GB/s) pg-sz=2MB 14.55 19.29 +32.5% pg-sz=1GB 19.34 49.60 +156.4% Milan uses a threshold of LLC-size (~32MB) for eliding cacheline allocation, so we see a dropoff in cacheline-allocations for pg-sz=1GB: pg-sz=1GB: -23,088,001,347 cycles # 3.487 GHz ( +- 0.08% ) (35.68%) - 4,680,678,939 L1-dcache-loads # 706.831 M/sec ( +- 0.02% ) (35.74%) - 2,150,395,280 L1-dcache-load-misses # 45.93% of all L1-dcache accesses ( +- 0.01% ) (35.74%) + 8,983,798,764 cycles # 3.489 GHz ( +- 0.05% ) (35.59%) + 18,294,725 L1-dcache-loads # 7.104 M/sec ( +- 18.88% ) (35.78%) + 6,677,565 L1-dcache-load-misses # 30.48% of all L1-dcache accesses ( +- 20.72% ) (35.78%) That's not the case with pg-sz=2MB, where we perform better but the number of cacheline allocations remain the same: pg-sz=2MB: -31,087,683,852 cycles # 3.494 GHz ( +- 0.17% ) (35.72%) - 4,898,684,886 L1-dcache-loads # 550.627 M/sec ( +- 0.03% ) (35.71%) - 2,161,434,236 L1-dcache-load-misses # 44.11% of all L1-dcache accesses ( +- 0.01% ) (35.71%) +23,368,914,596 cycles # 3.480 GHz ( +- 0.27% ) (35.72%) + 4,481,808,430 L1-dcache-loads # 667.382 M/sec ( +- 0.03% ) (35.71%) + 2,170,453,309 L1-dcache-load-misses # 48.41% of all L1-dcache accesses ( +- 0.06% ) (35.71%) Icelakex (Platinum 8358, no_turbo=0): mm/clear_huge_page x86/clear_huge_page change (GB/s) (GB/s) pg-sz=2MB 9.19 12.94 +40.8% pg-sz=1GB 9.36 12.97 +38.5% For both page-sizes, Icelakex, behaves similarly to Milan pg-sz=2MB: we see a drop in cycles but there's no drop in cacheline allocation. Signed-off-by: Ankur Arora --- arch/x86/mm/hugetlbpage.c | 54 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c index 5804bbae4f01..0b9f7a6dad93 100644 --- a/arch/x86/mm/hugetlbpage.c +++ b/arch/x86/mm/hugetlbpage.c @@ -148,6 +148,60 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr, return hugetlb_get_unmapped_area_topdown(file, addr, len, pgoff, flags); } + +#ifndef CONFIG_HIGHMEM +static void clear_contig_region(struct page *page, unsigned int npages) +{ + clear_pages(page_address(page), npages); +} + +/* + * clear_huge_page(): multi-page clearing variant of clear_huge_page(). + * + * Taking inspiration from the common code variant, we split the zeroing in + * three parts: left of the fault, right of the fault, and up to 5 pages + * in the immediate neighbourhood of the target page. + * + * Cleared in that order to keep cache lines of the target region hot. + * + * For gigantic pages, there is no expectation of cache locality so we do a + * straight zeroing. + */ +void clear_huge_page(struct page *page, + unsigned long addr_hint, unsigned int pages_per_huge_page) +{ + unsigned long addr = addr_hint & + ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); + const long pgidx = (addr_hint - addr) / PAGE_SIZE; + const int first_pg = 0, last_pg = pages_per_huge_page - 1; + const int width = 2; /* pages cleared last on either side */ + int sidx[3], eidx[3]; + int i, n; + + if (pages_per_huge_page > MAX_ORDER_NR_PAGES) + return clear_contig_region(page, pages_per_huge_page); + + /* + * Neighbourhood of the fault. Cleared at the end to ensure + * it sticks around in the cache. + */ + n = 2; + sidx[n] = (pgidx - width) < first_pg ? first_pg : (pgidx - width); + eidx[n] = (pgidx + width) > last_pg ? last_pg : (pgidx + width); + + sidx[0] = first_pg; /* Region to the left of the fault */ + eidx[0] = sidx[n] - 1; + + sidx[1] = eidx[n] + 1; /* Region to the right of the fault */ + eidx[1] = last_pg; + + for (i = 0; i <= 2; i++) { + if (eidx[i] >= sidx[i]) + clear_contig_region(page + sidx[i], + eidx[i] - sidx[i] + 1); + } +} +#endif /* CONFIG_HIGHMEM */ #endif /* CONFIG_HUGETLB_PAGE */ #ifdef CONFIG_X86_64 From patchwork Wed Aug 30 18:49:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13370633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2A02C83F01 for ; Wed, 30 Aug 2023 18:50:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D14CD440179; Wed, 30 Aug 2023 14:50:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C9D31440178; Wed, 30 Aug 2023 14:50:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 990A3440179; Wed, 30 Aug 2023 14:50:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 844D4440178 for ; Wed, 30 Aug 2023 14:50:40 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4B874A039A for ; Wed, 30 Aug 2023 18:50:40 +0000 (UTC) X-FDA: 81181662240.30.3E36C7B Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf30.hostedemail.com (Postfix) with ESMTP id D0ADF80018 for ; Wed, 30 Aug 2023 18:50:36 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=H2ddvZEB; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=md5Q25oB; spf=pass (imf30.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693421437; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0CFuIMSxAiAQM8abfHcbcFv68C8S+dyPEtQxUd/f0nY=; b=0LnbEgTUHQmhG6aSP5gWghuuEMtEBAEYQDb7iRjumYpGmvFLb55lMYMRgn9QGgPNCFKNcH n/ctxMORvI9W83UTCrVWI0KhS5H5YdEyMLLZbLcHNYiIYAThvQmTWa2i2wiL/fH6haANZx zZKJu11PEQCshhA3tG79z1Sggz9Ca2c= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1693421437; a=rsa-sha256; cv=pass; b=sbZbPiWuwxr5L+7SR5vmalchRXwAk3fMWBUjfPtu/FPLbdR6PXDqZ7HB5Co34eQ6vU3yKt NMv5oirDQZKe+z1Pq+lSL76tXVnD85Q0P1ljoVEXvocHsWL0dgchVTcZRtqUV5fe60xxIY iPB+SCeh/2ryopM76flo77/67OvsxfM= ARC-Authentication-Results: i=2; imf30.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=H2ddvZEB; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=md5Q25oB; spf=pass (imf30.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37UInwQp013269; Wed, 30 Aug 2023 18:50:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=0CFuIMSxAiAQM8abfHcbcFv68C8S+dyPEtQxUd/f0nY=; b=H2ddvZEBlPaaRycR2QzhBh3qDFaS0XV/Veq8js5sITuJtXFJMv+tLR1Q4X1b0F1JxR0k XQi6T/Xz6daqY3Ypk7CGPrtx2byRBOPShj+6F/gXQX0jVHVB4tbirlMdFBWxyhB1ded9 4iSLNBKOBJf07szxbD3cqIbqtB25relhYXSfNNgVePpG+uQFP2/fOOjmrUrQoyLxBmON JsZPhSXHga0GohX1OnVT+SnFRMlmzYks6tL9/styKKb8JKlNpYFIhu6Bgu5FcpBRqkAg A5/9ghV+ZNZaGGq1rWB3fwB1OSXfFq6kSJtay35Nud6a0ZKDBDyduNkfMzdv+3yoi1xr rA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3sq9mcr5a3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:26 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 37UInd6m014527; Wed, 30 Aug 2023 18:50:25 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2042.outbound.protection.outlook.com [104.47.66.42]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3sr6hpx511-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:25 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KmyRVgTje48hquCSTrT9yJY3SQkDIayafsStvK/cn70EY89gcQpoQ3Qextl2EzLGC3bzdXvY95yBKRrGmgFKZiRIYaYBAG4evVnEGX1plSx+joGl5d73mrQHOcZURkj0Xh2Guyt5gW4ZLo7dGfin8Dgl0KUyG1vL4Ki3BGEBF3+dZO1pGH7CGfeT+wsp27qnyM7+I05I0KvxlL3h9BCcgdg/QZwhzlejK8twrMZgaIgRLLISPNyenn1dBYJfJ7VrdHuq+5tStWgmOaQ/xZoX/YQQ5Qfa/z+J2zt0yZfKojOcsaQ0/m/OD6KpoFx9EfBJhFLj/BDHc8rYPkg5kqCD7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0CFuIMSxAiAQM8abfHcbcFv68C8S+dyPEtQxUd/f0nY=; b=b6KKsV/ITFpnMG4lwCd4Ie2+4FwtaX6Qf+Du/0Zea9qxHtvFIrHgV4abON7w11PBJiUPT+NdgsLeQGOKmXhMH/IccpArqS2gmWcWIRPQ3rt3smXYkJiuKuXPKRrzWKwMAXQCXL8SmmLpGWKUkhOxpGNhhmdcQXZvQN6vqfOxhUdShieZaiOIHMaj/xVsLO03dqRogsdlNJEZFPdCZecDvR89Ly/WWQ2DXl6jj8il4P0QoKUtcHgj81OtYLuQn+sFeSHv1QeQ7D1zwaw1a2+7LlKBfhH4GVnNJa1me1AkKwCKVF48o+rr2mSm1El9RjLe/JiVBesFOGYnBD+GvFS+Yw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0CFuIMSxAiAQM8abfHcbcFv68C8S+dyPEtQxUd/f0nY=; b=md5Q25oBZoE75he6N18CNHPj01scRLe9cOwxe/nE/Itipe54yN0TY91U+sDn/X+TjFKSX4iqS6xgeIK3zfXMffU8/klbHv2l+1bkx/waEYAmpmZ6TnSTXXXpGGcDjwtp8JFpxbZ3BNub+zSn/dYpLjT1IbK4ZAdn8H8QCYEKTbA= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CYXPR10MB7897.namprd10.prod.outlook.com (2603:10b6:930:da::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.20; Wed, 30 Aug 2023 18:50:23 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415%4]) with mapi id 15.20.6699.035; Wed, 30 Aug 2023 18:50:23 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora , Linus Torvalds Subject: [PATCH v2 7/9] sched: define TIF_ALLOW_RESCHED Date: Wed, 30 Aug 2023 11:49:56 -0700 Message-Id: <20230830184958.2333078-8-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230830184958.2333078-1-ankur.a.arora@oracle.com> References: <20230830184958.2333078-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0373.namprd04.prod.outlook.com (2603:10b6:303:81::18) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CYXPR10MB7897:EE_ X-MS-Office365-Filtering-Correlation-Id: 40b8e4c2-af52-44f7-7b46-08dba989f745 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lRl70Qk1TFTOZnxYSHzq7yJ+IdJYBeBcEtMPBK6AssB5D53UsG2GFV1MSKuCyAal6BC8F1YUxMP4VBgnno/Do5tozGABGw49zqK67ApigMH7VOUc5QY+xpfLC/2TLMsDNbQPMfh2hBoVf34xSH1CBjY5GPPV0ahO8RvtRbwNyJcEmrxO1E4OmrrYo1bmc/K1I9Zj2T0rcRfm7j+2OiYt2/QQUZHFNMppIu+4FqXWZJ9633eYXFC4LZxxaTnBpasIZoWiW01PWFndtk1hO5dIMQy6886xC9zIAnkMyRCC8/W4yYmBDS7T20RIT5ryPRu9KtVdf21qxNKhOe1r+CRyjFckD+Vai9DcVeMW38VWgmAAQudJK4pg5JIli0vSyvMUI9rukI4p1y3zr6LUx/5eyvMV85js1x44/70dV072hVp2C5MiAt1hFBi48c9+4BOQCdBTs6BWVHhBXgDdgQuqUTB2x5vYnzZiU0PTeOZZB5Upk/G0B9YHbtlHXutNyNh339MHFJPzSsVPCtbJpV/3omFdUq8zpfzcLkVHSZmblTCNb1vBevLe63s1futzZwj7 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(376002)(396003)(136003)(366004)(451199024)(186009)(1800799009)(478600001)(38100700002)(26005)(1076003)(36756003)(66946007)(41300700001)(6512007)(83380400001)(2616005)(54906003)(66476007)(66556008)(6486002)(6506007)(316002)(6666004)(7416002)(5660300002)(4326008)(8676002)(103116003)(8936002)(86362001)(2906002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: nmdBHd6KbfrGvPQSPYgi6HsVB3YEwN84lduLRI9ClA/+PkzFzsGcsGjtucJAAqg13I1xCg5VR5t9hpd2LoD1eOJm5qykKAuB2kdpIv/yWINT+8e/PkrPMhc91C3zR0vQHg4zwy01VjG2DDAAIil8QFrxRhX4CtTfQiPCWOzZ681W5cIF3E9HTJEPO8heb/D65T+Xvyag0LNni7sc8nU5UkFDZReL4lL29ykz7LHEqXgjvKPfkUrUjadm5J9yt7fF7ZThpWy4nwol+m6JaXpepLVZORGIdcnUEdFUkSma9t7SFlsWxdA7zWV96wPPmNBjak0JtyR+zy5DzZEPbe4LKll+sYaumqbQ0idXlBpfS7l0o/Il4KiLk8b9Jy4DHEiftxz0/8BPXpg9e4hHiRekCixCb987ygGdnmdfbm3mZHq2tpRBGO+geZitjTD5HfnLA1rmJjeC9cmfbnSja4pH47bpKQQaldtu0UFNkrmPHl+ibGBIRdT4dqW1s/XK0rmH9aZtHqL+oTDN/SgHecxno2g3ieFCw6Sv6YAYazIxYIEQVbBMTWTHYa45/YTWfUthscHGQnnujD83n1uK3I0uv8TDf1gYCGWSyQ6vSVj44jbLAFtVW/kUKcsk8qz10dL6APokMPPtAOFznNR9rZSjJzu0AfXsHXXOj8VdMz+Cho8lzM/JDARS7fDR425rMRbNPqdZUtO3b9HeR63265FpZn9uGZ8Rjguu4x3Z6fJdyK9H3BN1zdCp1NGAfaLH/Yf5NwmvaUdHyvJ6oKFZBPx1JROTfn8J2LIoAfi1DuefrWee6q4hLYQOHjU+8t0Oxs1HCYjGVqCaCf1/aGfGVNv+DP6CsAMwP+Vmku+4LSo6a74IQy4slYKRJhRlKyGjKFPcLMK7I3EkqbYyFXac0MXw7bWvXjLoqC7h1vbYWuOU+smzEgyWG0rAldQZQwL2xFQp539OIVeIG3DlMBX4CPYtg+VuA7cPJi1SjdQgYVfbzzPcbNwtrOSIiN9Gl9Kf62QM+HNmB9R0N5WOg2lcmqHrgQIdH1R3NpFKI2pp7g+3lQosuHacxr9zBcLt0tNgwtTO9FL2Qpns8FTCNYg/QSLSj5sMp7khTFkb7eYKN5eYNb6K45KBda773jNeLASlh2DiR13G4jiTiqQEJGFbE9hHLPb4eu+8idLC1M1grJ6A4r4SqThztmLYlc0y29+643yTHkBICMrxGRvulkHM4pfapMWrWBqUoEEqPveAXjNnHvesRekaGq4mZZfjuMNkvbt/lY1mklMZ39WaVogBsBtQTJY5qT1081YOOerIMqDxsqeq9VyP5SMSA3dFrGdBIvhVgFAaoJDGmtu/njv6BL0Z9UpTDSAkb5nbOk/R2JOqHDxH7wPx9t/pVcFV0SH+U+0BPVtlEoNEkPpNNQ9y4msPjbJbNFyOPdy3pd5yLRIs9uajx+UzCkxun6owTBWYzj98I5nC9dxrCyI4Jte/1s0+OQFqIblKJx0FyaBSuW0c7+APIMh84L8LPSUVoT+3n0XPEm3Fo4TbYbkzEGz00Cvv1kv/PfhI1Ne1g6ktSeATCztN3i95NUA/jQoLapLopkfbagiziSjWtbTCkiDJXFpBMw== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: kg6rMERIKfjGlwQm52+bAhJ6uapRFWZ8SL3ULoO1xyHhMtj3WFldD7kBxGHlrh7f3d0C/oqPQv8EF46A73CY15xvXNjZRlq+amSGWCQ+srIy6kimGLNQpEMHfIUxaDm3cnRbbqK7CseR5XvVBECOi0ijVsd6PtDlmRt1vuxXnzdcxeLtE842B9sBMPgiQW63aTiIsfdQSX3/YTDJavkYwLgcKusxFjloTlCv7WDxbbi+yHla9x1LI8BYLoAcYxbcDKaSZLCwFh/oqEK9rikAU6ERHNaN8l5B44ZJeP4B3bfuI2XC5li0PNmFcJJSF+glmKfwFcbIig9JbFlQIRAkGF/8qZsQH43/el7CD29+J6vsHTfRILob4Ze6d7Yq6J7CCe4uY+4NZFtVamWxUxpmLTF0mKRrcaJVAGEtP5goSvKQiYB4RwGtgGPA6gBYl6wXplMGTYHVLIWkoptlgP2p1H1Bo72oXxneP6eIs+BqkgOm/ANede1LB7OGNUhsSEzOZ5+z+/rD+1De8KCKPshTI+1C3E8DJ3rQnkEVYgnztEccKi7yLtJs/giD4f7n56bXzt4SB4zlHy0dwBTYsfpM22tAMjMIHbwlnoA8TGgnTFq3T7RXWUoeIqP0mH3hKO+hfpr80KXAXRxTJPcREGf1H7KYVpTXuJ+3kmUc2fYYs4g/f3iVC+5xfqzzpzxRstPKdrHDdphUW4uU1s0eZaNLhCypE/QV4GYz+NcoVeWP030TJQj8UAE1R6+pLC0tU+pUV9BAxpXejaUzHyd1xo1XzccWyqBnCbU0ukraQVChZiwbQnfEHRw6CmO2tHHmsybjXSrdQzeHNf/36vEvOsKHkjEpMUjltoQ+QsDomSutUD3gwYbr7y+CXbWIavDzYqZsoXXXo0OGd9OeG24X8iT4jwgB3X2QqrpwARKxuWujmBdyvhWof2SNpc4/XVpyPLlTnHx7qulbHH+xZaxRxq4hSqOMw9qKJh246Gr+xxcc67I1YC5PvfjoEb4hp9ft4yrcvGoRt+cU8XANFW5cvpsVBqKqi2QUFHWrJMQDx5JzgO488RwOREi6qhASDJJkGi/O+4Ssv1CuOm5Pm18MkMZDm7kip8tAAIxnMxGu8qEIJLY7mENv3POGJAkhh1WHShmqXzAWJWAca0Ykczvj5a7twqt+dLS2BbAJZhXw26WCbZQ= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 40b8e4c2-af52-44f7-7b46-08dba989f745 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2023 18:50:23.0004 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UiA87hyir+IIf3x+gRxj3JGnMFruFf/KE7nHJNJZqoHhYusqt/gmozUg9wtnurEkc5CnjNXBYnCuU5mkrwC5VASwSoX+slkEb5Ei1u5uxtM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR10MB7897 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-30_15,2023-08-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 malwarescore=0 bulkscore=0 mlxlogscore=997 mlxscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2308300170 X-Proofpoint-ORIG-GUID: J0hZYB6_6RDZYMCLXaArRS4EvQSaBzX9 X-Proofpoint-GUID: J0hZYB6_6RDZYMCLXaArRS4EvQSaBzX9 X-Rspamd-Queue-Id: D0ADF80018 X-Rspam-User: X-Stat-Signature: tbacm57d4yxtu6kn9xh6zggh45ghegnr X-Rspamd-Server: rspam03 X-HE-Tag: 1693421436-509262 X-HE-Meta: U2FsdGVkX1/qY7+UykHxhRQpfBtk5SnzFKV/4HMltnCyaaFkgvwomEPMIkqXTn6gwvHzPnCw3g8hTb84ircwqpN/JII3yBvVakPT2UXVqy+aSYpJXjigMN4sh5yr/1B1qYhBkjnuYe4M8zwqOgoaeR91Mynqdf8sgJvTYoT7HuM5vtUbN7ObwDl/NQFuOxuFDzM0AxiUFGGPdRr25Uu12r5usYrEBEJe4rijzCKPNQs/Am3QN1mSGzMjjrYhp38h4jkRBSUKENMKG9C0A3H9qY7cWXL1eKiDrNSYM7QtaWPuO8F7SK3Ez74Tm4rfUtmuYrRhwWzULWMaU6bCZhLeN8F7TRsS7hQ+ffnzt6/zv+qLBSLt0/gHI1aEwk8vlATdr84XsSJDeGG51HHQDWdyQt5MBlIR/LHYR7bQeWtnKc6HmrBlvY58vDiqk3kUZEWo4J+1SnSKSigvnCM6XnP3J/0cHf7ibd0jFoIjwbGIESZkoaT9DoPhcPAb3KEq8tRLa2w9N2DU+SCnQg7HV4RtzdW5CPMJ61lyYwJDoIu6aVVjbL2UBuhQpsRBcXVpsaG1swgm8G4qWveq8kFJ2JG01xoZObeuKKlJQNSnQkMFngduzup0VJHbtrm7PgjrujQwZqascFms0sBd8oyAvAJqsdCgJzpgCxVZNrsHcQ170gv+GDjb4xBVxiMT7q9r2iXyj3+nMV9KQvYedvUC4X6WswhnCIvmCLW/WdlbBhRMkD4tXUz6QRNIDw0fV7CFGuzuvGm/z0Xi8cjGVNUFZJgILSiA0gFbD+4k7cnt1J6e6Dw90RaJ6HuVIkWFFMsy3Jy7ZJX5sUhWatob/T7l4YGzHOZJKk83XQbZxLJUgf9CEoCjKx8QLJw/tu2Xw7qGjGxCmsM6ABDlEGFUY1pt+YuJqzh4jYK3ZFynGUhJW7JyFykjQDStYDyN3AQMz6/K11y6mM34vp0AoyUXoqpZK5T gRYWg3GP yq80xQYNzPVm+GNuXLNIY1WmKPwI6OrQRNrkc/LzhYrV9vrtNJStjgt2QFHmRR9Na5q+Jfn+gA+2LayiKHvnSuUnzWl6enjuzoB2UtBE8CYciu3V2R/JaLeGPd155YYZQg03OgsZySStWw87f8ptwiZINa+T6n3b1q/32vxiMfGEkXJmLDtnKOPhxdiyhIYGV8idOArYTTD+pWiW0gAZLfN7+TrxvWwY53VJtxvOxMw0M8WvN9FkTQ+1+/YFCujH6pPqN2YvGgdYhDdoGAEIHPcZyHq9AoDfTDianVPxFKanI6QYnknvHEt1FjwE1Jd3XCfN1Hu1NfGMqCMOLG783YTgOa/Q6J7sg5BELth/U8HQ5TAAHbF9vBE/QUYlXSVdLRXTDtwdAq3MMVqHtpobyjoyja8tTYyeduGf+RzhqS66MXr4YWt8mRDyN6P3q99OoCHaX3Lp93GSJPklw8Nb272ijng0Tc2g06zgvvl2AP0SP+/SKzHAsczI3GAIxv6NmHtUDQZbqV6o/D5/7KlbWZQWPe0Uqr2Pea7vGc55iyoJk+5Gqf5nq75uaLpsKMVMToGvi8SmeBlouTF8OGwRpbLFjslErsZSc5FTbuGksmPcg+XE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On preempt_model_none() or preempt_model_voluntary() configurations rescheduling of kernel threads happens only when they allow it, and only at explicit preemption points, via calls to cond_resched() or similar. That leaves out contexts where it is not convenient to periodically call cond_resched() -- for instance when executing a potentially long running primitive (such as REP; STOSB.) This means that we either suffer high scheduling latency or avoid certain constructs. Define TIF_ALLOW_RESCHED to demarcate such sections. Suggested-by: Linus Torvalds Signed-off-by: Ankur Arora --- arch/x86/include/asm/thread_info.h | 2 ++ include/linux/sched.h | 30 ++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h index d63b02940747..fc6f4121b412 100644 --- a/arch/x86/include/asm/thread_info.h +++ b/arch/x86/include/asm/thread_info.h @@ -100,6 +100,7 @@ struct thread_info { #define TIF_BLOCKSTEP 25 /* set when we want DEBUGCTLMSR_BTF */ #define TIF_LAZY_MMU_UPDATES 27 /* task is updating the mmu lazily */ #define TIF_ADDR32 29 /* 32-bit address space on 64 bits */ +#define TIF_RESCHED_ALLOW 30 /* reschedule if needed */ #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) @@ -122,6 +123,7 @@ struct thread_info { #define _TIF_BLOCKSTEP (1 << TIF_BLOCKSTEP) #define _TIF_LAZY_MMU_UPDATES (1 << TIF_LAZY_MMU_UPDATES) #define _TIF_ADDR32 (1 << TIF_ADDR32) +#define _TIF_RESCHED_ALLOW (1 << TIF_RESCHED_ALLOW) /* flags to check in __switch_to() */ #define _TIF_WORK_CTXSW_BASE \ diff --git a/include/linux/sched.h b/include/linux/sched.h index 177b3f3676ef..4dd3d91d990f 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2245,6 +2245,36 @@ static __always_inline bool need_resched(void) return unlikely(tif_need_resched()); } +#ifdef TIF_RESCHED_ALLOW +/* + * allow_resched() .. disallow_resched() demarcate a preemptible section. + * + * Used around primitives where it might not be convenient to periodically + * call cond_resched(). + */ +static inline void allow_resched(void) +{ + might_sleep(); + set_tsk_thread_flag(current, TIF_RESCHED_ALLOW); +} + +static inline void disallow_resched(void) +{ + clear_tsk_thread_flag(current, TIF_RESCHED_ALLOW); +} + +static __always_inline bool resched_allowed(void) +{ + return unlikely(test_tsk_thread_flag(current, TIF_RESCHED_ALLOW)); +} + +#else +static __always_inline bool resched_allowed(void) +{ + return false; +} +#endif /* TIF_RESCHED_ALLOW */ + /* * Wrappers for p->thread_info->cpu access. No-op on UP. */ From patchwork Wed Aug 30 18:49:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13370634 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C727C6FA8F for ; Wed, 30 Aug 2023 18:50:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 070F244017B; Wed, 30 Aug 2023 14:50:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F3611440178; Wed, 30 Aug 2023 14:50:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D168744017B; Wed, 30 Aug 2023 14:50:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B2D74440178 for ; Wed, 30 Aug 2023 14:50:46 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 948E6140277 for ; Wed, 30 Aug 2023 18:50:46 +0000 (UTC) X-FDA: 81181662492.01.26CE484 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf28.hostedemail.com (Postfix) with ESMTP id 3CB47C000D for ; Wed, 30 Aug 2023 18:50:43 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=0INu6VVv; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=pxcH0wXl; spf=pass (imf28.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693421443; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZQUSDAXAq2hTKgg73tSO5DT1RApTyE+5aqfIw3lJuGs=; b=59O5f5I1mwOiNQgKK5arUkw4kkZk+qsT7y8qsmH47hFpUlux+uUy/Bh4hs2vV19PQUMw5K ZAwLcRQno1KFofSSFcuYtj7Jf/jvx+Oh99zdjDGg6UXe+irUoQwlbWQlDb26Yt73bfa3rJ fhajm5ZRjijSMPu9pNS+RjbT1LH2dBo= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1693421443; a=rsa-sha256; cv=pass; b=iqZXzEgEbmCozCujAr7tCqKn0JMrx9h05ggQF1heYouBuN5odGfFkqNo22RE0BHQdtQf9J UH2LtPEt+QVc0WvUHPTOVz2LguzcGsXDSFwBcFd8uWxMrNAUpUJ6zn/xqVoB7Bz1o4SbUE k2h5GrwZlYJ7+e3enMix3pGbv6Zz2bY= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=0INu6VVv; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=pxcH0wXl; spf=pass (imf28.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37UInqUt018036; Wed, 30 Aug 2023 18:50:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=ZQUSDAXAq2hTKgg73tSO5DT1RApTyE+5aqfIw3lJuGs=; b=0INu6VVv0GEMTTAIEQmj3IRUbaAkWGlnNLkcMfKUsq+nqi0t28qnnoXevRoJJxEtuSyx VB2XmzK/k5I0OCTCnA3Me57Cka7WdAJGRutZJnwW/WiR8ep/mJva6fh7AmhPWxxnDeqf 4R3sdHOYUOmQfS2fiOSmp3gdkO5wi1q231JzTqome0Sa8+Um27rFZS6J3XKjM4TkhAPQ ODR/HBUeCS96Px2WcW6kD49BkGHGEaG2IqX2TVjwC/VWWtPW/pFjFO6rDOAZAR3x1EyI IM9Mhc5QM9K+c8ptIjOAzUTpIJePzO4Xrko4OPfHPltWvxN7zyvo/SN0t3DAFcm9gLN0 TA== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3sq9j4g76d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:30 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 37UIVmJV002385; Wed, 30 Aug 2023 18:50:28 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2043.outbound.protection.outlook.com [104.47.66.43]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3ssyw43vmg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:28 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eeBUntXapP+PwJX3vGI8EfwjVIpabiZp/+hTtWwjj6YvhqCeIOKH9+EDdOFKNPZhvvhPyhLlqX/83URG+rNLJ6pPH5vwpzqmjKBxhNjZOJ9jR5oFDTNNiQqs8du2ar1ir1VXXCWVh/BEdTVTuQzY0wHTuGjDDsSaJS19+m4YtVdtbz4qxs3qhfIIvOnipjvGOncvOA05FP8MVVw6W3pF+wjzlRR/xQIVyWlQrmuePQE7TEWdWCiUHtTeKD3Vv883euo55SAqAcn90+rv6c7idsAr+Zq4Vv/27TxYTfcYJvdcSUBcErBjRg8/xpTkC4v9e3j/JIwVGKbiqvQz+TR40Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZQUSDAXAq2hTKgg73tSO5DT1RApTyE+5aqfIw3lJuGs=; b=cRU1wsqZKWUw89kkygDeCjAGloekB2TNxhYLexHSZ6+pReZGESmYXg8ZWZjAHsO2ne13cwWou3m72JKs/lgANEgemT5FtdbasxS6PGIbUO/XPQIppBfMKv/Kr6U8EItri41/ITJnUBVjJcI2Do/EnWen3OWIru/ErE18F2RYWF5EAaw67afC1OVOr+SmIFATYVcv3INTEDY78T8RDPCP77BG8cUNy9+VZRnVuB8OHSo0SgubPRMNl5crYJg6LqGMC2G7pPPVlLthDR0oxIeV2xP8f0uTHowt8CqTq2nfPMpvmPMjcEZ73WfA2f8z4rdGYDsBVcg7BLxqpcN+dAEy1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZQUSDAXAq2hTKgg73tSO5DT1RApTyE+5aqfIw3lJuGs=; b=pxcH0wXleHNC7zGYlqE6t5mRFwY6E+knl73bCte126V/QJptfWaico93rZR+/RNQy46X9ATAqCTvFFDRWFxxzydA3I7amqfiJRu7JIJ1lck9V+TO3wPD7ySx2F9blSsuCJzFZSRCMouLGRLVrWyhB2nCh24AvAfBULeThhkHs14= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CYXPR10MB7897.namprd10.prod.outlook.com (2603:10b6:930:da::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.20; Wed, 30 Aug 2023 18:50:26 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415%4]) with mapi id 15.20.6699.035; Wed, 30 Aug 2023 18:50:26 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH v2 8/9] irqentry: define irqentry_exit_allow_resched() Date: Wed, 30 Aug 2023 11:49:57 -0700 Message-Id: <20230830184958.2333078-9-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230830184958.2333078-1-ankur.a.arora@oracle.com> References: <20230830184958.2333078-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0231.namprd03.prod.outlook.com (2603:10b6:303:b9::26) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CYXPR10MB7897:EE_ X-MS-Office365-Filtering-Correlation-Id: 859026f9-65e8-452f-5177-08dba989f916 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9vhy3ZQGGhzt8VqtEpcbh/jak9MnZ6SaC9nzjmiZ8LO4Y/bzDt4Uq1mdXYePczh4yeAEQbE6MvOVSo7R2vVvOF/yqtkjp2hsM1dD94+KiDZoKQKmUDlfvcI2jDjBUdFobZ+V8ekRcCbLArx7D6R74j/YhkDHzfOKg1gUZheiQ4j65IFTceCsxvJ00JppKo5S6IkYEMARGBLYLcE4mLSJnYcRlW2/w7Cfh8oHmt3LCyoC9JBfogXSc2d/klZ9b7Xh/LanPXTw16wtt7brYpgT0nG2Pqu+aKMpqeUVeF3fZFCG9SCB9RLnmi6B1Zbj21jB6j+1hXatG7zV+4ETst9NDSN+oNmE3LpihSsKvyUKdaB5a4WiSJobxHiqC9y6HA5t8SGsvfQ6Mb+5mlrhD9uBJfyvwyIOcENh59AFdYSU/vx/YHwVpnEgYxxLOrXdyq+HfJaD24ecM6ya88oAFFxZkhtOKY4RruQZky4UQHwR0KM+Ny21yizEe8pK2yCU6xOJxvoAtimOOo2gJDG3Tq1i8oVQU55rBANbeQoAtJQFFmRg29ge9XL43BirwMGK71Qy X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(376002)(396003)(136003)(366004)(451199024)(186009)(1800799009)(478600001)(38100700002)(26005)(1076003)(36756003)(66946007)(41300700001)(6512007)(83380400001)(2616005)(107886003)(66476007)(66556008)(6486002)(6506007)(316002)(6666004)(7416002)(5660300002)(4326008)(8676002)(103116003)(8936002)(86362001)(2906002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: yUHmZYk5hcuhGYVVWykHTIFUSr0HeoGfE3wZnab1E5X64dXBkPaWmnO08hC4neSNrivKQ4W8WeGsa3VHAxKCXc68ht0iwP9qg/t7RDAMWIqGBTJpTCYFeWqX8is1G4G//4GqvRd7Xz4Rni5q3cBfRSceeJndISpHelO8GKQchjpbhFViLEQLmUCdNnKXznsHrYl3/2OuJ4s2y0tWpqUlBG9BPH9SkwyGokPIpnwATJx1wc6oIGtMnh4xyCNltQOZkAnXZJKLRcTQHrYQDF73WPwJcBQ2dpQ/h9+I+Qxd20UnyDnMefhtREoMk2egWjDIqGUIQfPdk5jPcUuGzD9pMXoZ0ijnAC0wPvyBl67xq7tVpYLl58r4tGb0VFShNuWNwLyTJYwkVdXRGbpdsThlkNkz66l1/uuOF6/WTwMbPMFl2gpxrm0X3pbIvb/PBmaIpUNwkdXA59R/Jml2QsQnQTiaoZB2F6seQjXXbGfR4KpbqFF/q82M8Z0ysyQ/cs3xKnh/Ym4nIpvHfFJDVIYRJs2nxVeNKZCIw0My4OhZh0qm2AnDZT8AewENwT/nMxbguxBeqSAgP5Ruxjl6lNiIRtzQw+EQuPAKxtenwYLqFnWmkwwqQ6Aw9X/1dfRf7VP7PiyaXQs3a1OGGbxQi0vvC5quS9qOWuavNrjET/UTlbGgnQNCuvQ+quBMm0voIrSUSElwPLAmdIa5PnzqBcqQH6/3oUzrnk0FA1idBfJ4S1wSRJqBBMk9nZhFbTQHLuiOU94pjWqbeiZZ+xhK5X61OEIUFPRgIxCiSHa+0pyzbvYSOlTDgVgm92c3ZE1yAc324byPhEhx/PyPDPbbRX4EaMgaQxdOk/MZbH6UKsjYwh/XgUGQ16p2MdPHIgdCNv02zdWnWJeoxrNcZgRgvfKNUxIawl26n6q6Fl/l7HbCpRGYzwVo4DNR7Cok5RHncGO0uu4LjbUeExE2pyLGSPA44r7tCV2uHVkkdVqp/DWUjONtHVBsX1knANcWjougHaP0FwWlOV3UHuPhXIa9kHMUq4KbG8uHaVf7JBJy0VsrRg6Ha7d45oEFj0cqMLAKrDZkQPqLjBQZpaSbuXaz/crV3RY1a4Lm7h8NlY3ePlffBy+BIS2YDS2SYUYOj699ljQfcx8CYJEwAJ2Ho9IjkMmIe1K1Uda1+eWk3rza1yEl1m8ZNztjfkcrt8n7+TsWs4cYfzDZh98k1yq3PXXmqFPTZplAEedfqUCU6ExYUyf9Dut8fa2pRrp0JQiqDXTXcG+6BOr1ORh92dHudFo6dM34utb6hEpS8W9DzDMz75CodRno4vW1FoUMz2TZ6lVHspxmMmH/FVop3HNYLwzmAb/rGm/vOFmBiKSUdRpcZ/Rf6pyhftZ+h4LnYoBuNvvZzTh8G/aSFqNckpDlSSPdljuNj5qGNq/DOXC937Tx6lU4V5V04ti+eqEpqaycz1i4Ov2B+9lP1znkRqXW5dzKHhnHQSQOPOB3R+XjVtbo/v+fUem49sqAFPcx8IszQeQ13o0twcF2lnG5F6X4W04c0JKtElWPE05pvx3F+k3VBUrYbj1IpmC0RDH17ap4ASqHrWPRGVD2mNO6rD4k29a4Zcjafg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 8S6aWMRaoiIqqx9xSb0a1wWIkKniSJiQeezEmMiZIutrGvTnZZiFP2wd2lnoHFJ1J47GjIORk2Wun9IRz0hbxOgazpsdJtKGCnq6obxwcAAQs0h4Qut/tPlatr0Y9An2w8TF1mfNLaIMmzk3RfyqqqpHnYALET90lK21PD9qABUXmjxT5Ioo5QGx626Eo7xAWtmr7mo1ziQOBNcm1vc72xIOWyZ1fSd5CfJko+K6v4ngakf7i6dcbBPvBmkKgdS1CSk3//aqYd5f0iECo2vRQ8ZSAMNHJD/SKt1/Oek15Er1oWlCnXMfaEVEh0ABxclbw1Jjm7Lmx4pJMW1EN2QYxvluWrFdwf7kiVrlgk7FDJuckh+y+PcmUiR4g299RZ65xMeBOsPuvwcDfQ+oAPPxs0IOBR7JVJK32L2eJAP54YjgLGhDw9H21+d1djLSk3rlI5f83fxT7wfcvYCwQwhFdYxEpVZ2gN9k8RVXXx0N8IRd0ANNgXJL5ioHFgDy6RbdMQQAVabm5GG3iT/CaAHGhvWj3b0/sSu/ic2+lYgKKrGYJUdboLkdQ2+9836dGz4u/heVY6e6zRbb8ZRpwwrKZZQ6X1zEZfVoghrBsNwpFLN2k5UPnHn8WpPPFMBC4up59RAI+4hKtEf+zIe/KwcKKKfBrZonJxZgPaIvsJ0eDZbHCef9lfospmFBhs59zYYH+MsNkQcNvVuV4ntxB5XCZS6UNguOQA6fRlL+NFWOrBwUBNL396QqUp1jj0ySgpEwHwYlFaXPP7ghQECDPdKPTosY5JhRjWyat1ydk0F26MqWNFQLYmCpWbguwUwDLhJ6UMoS41a2dN0Owiy9uRD6lSllSh8n8RcvmChJ0l4UUvym43WUvcZ7xfH2Nn5tsjWDBylXaEUhk6x+Vt92/2eZjOzD07jwzbYJjLhxHDcn4loy3F8wwS8hAQWX8mk0cswPYAEgr2W6fyFeUJtLFHoNDneAQVMhPO3unfZoP6zfWc9uZRO0f23d7yN2Fi33Vnmy7vbNYQIAw+nS+JaMIkMqaiLJhP7MUeXshVdxKF6AKKzgUkth0rNFaIPPQnAARVDQricMk+ucCLnSjIMx3Z0VKz/FZBk5IhjLrAU/FA1f3yVm8pLCJsgLeX22jwmo8U8sWOugbC04S+1OS+s/nh4xF6+ToZYZMsS6XMt1r7eOgCY= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 859026f9-65e8-452f-5177-08dba989f916 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2023 18:50:26.0338 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tlIgP7oBt4bn9kBHfx27nH4m1QBsyiqqPqvzxlkyXxrPgJo/M3YfWLTMudWUdNpf7TFhgOYfWbNjs8ZpJFvc8jUNklM2kIClMyGBnmFqRF8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR10MB7897 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-30_15,2023-08-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 suspectscore=0 phishscore=0 mlxscore=0 malwarescore=0 spamscore=0 mlxlogscore=482 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2308300170 X-Proofpoint-GUID: mb-hkt9pfv4cL6pGQ4HITQCjtSzH_TkP X-Proofpoint-ORIG-GUID: mb-hkt9pfv4cL6pGQ4HITQCjtSzH_TkP X-Rspamd-Queue-Id: 3CB47C000D X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 9cykf3kjebrnqhbnaferd6mi1pcpb1cm X-HE-Tag: 1693421443-757742 X-HE-Meta: U2FsdGVkX1/UsAmuij6nGBrXm1QgCf+u4uxYJG/sCNpqS8I+GNIxB7v3xVI7MT/RT1VuPydjWXJLT9jPOSe//TMJHIF9W94JxKaLNKahK1rL1aFdRVJOUcGCUPE52c2KIQ+ybkzeC4QChswtJQxPAeGwouw11pVcYw3C1vIeXQCTGnlHVsI7+eZphIZleE81xIOHY1PzB7hSKTPSoSoi5mFCxiryMK/Vhr3ce3duVmHrgkwxPZ0G/JG4HYzr+eWmDG3sr2/qbN0LKTj4T4O9cp1vkvJylkhjIORFrm9/YT6qPOAPFMRn57j+a4oGmqMqqc+PvAz0nF3rmMO2iJGgZYbo2BuWySaEvGGro44eGl2Nx4sdCw8DM+2wgu2omW6K19QsNkSL6F1V6eZGNJEZ50o1STGeHIu/h9yDOdaSfnPi4oTKSRV14I30YO/v5ieXRCOLmJq1BX2sQOZHXLd1IQqvNztuNdgVByUxVMOUW/szRRh9U/H5a073aVXCaSZlUbubWVtm/Oed2TpZ/IiILXXzoxuBq/TFCNvQ735479jXCDTV33mVs+bepnGl+yh3rIqUVLKH9EXR9KemQVKFjZbu2SNqjF0MAdlMh2xrPEKxrHu9piYfeNi5dIG+jRBTQjGDXn9Oq9k0tY3U0Fy9M/7EUGZdVdx3HFW94Y/dk4AADTxhl+0XPCYHEt5zs+88QdW8HoaqA43pijtz+201tDSK90szm++ZTi9GeNAr/yHbzcl+ax1K9J+fgAT+d+Ku9nwJhIKRU2GpRsg4ULbc7O5orNxffdp0ZHDcUtCyQn6fem89mPacae6DGGMgPLYlp71QpIaIS4lW1EuNt3j/cau3SZVrKf0CcFodJBaoYyuPwJTQ9eyX3PlbCKO9MMapTJFoaDfJAyX3Vm9XDR2+fZjjfakoEmVD738pWzcdoJEvE1XWJom8zeC426uNI77UfSLs7ehdSl8+bAtw+/h KyspDpN2 ZDG9lIe9oKw2uqHZikx6xuyPQbNGPnMWyHo+R5CllqeHGxi3EMEyLRA8m+x2KaT8+vpPLNjWI5djij6kTYDAZjF5GKkOEPYTW6KZcBdoLDL5JfNN7gBaBXF99itlHl/0ONwp7qZdILB2mIIWjmFq/hEgT/aEPYgz5INs1X2/oomlweKYtHWYWkFfnv695RnvKtidWZm2ZUI6kTdtDz7HnQpfY5TUBOiVr6KbKj9VSIFaXdBfsJWUviksDw0XrXpuJlceaibv0tRnZmduYIdW+7tXx7MCziiZgfvR8CC8EfuX1SuHUh7bdW6RmO2v47U5fIbTS4gajo47x7RZ44NRO3NBaRNJAXkxH5tEOaZHiN0vJgnLX3YWWBCkpLjqhslxXFD+x1gsa65qzviOsJfrK5J0gky8kiUr/XjA9plkXVNw7A0JtTH9fUrJSSqBBlMpPPhnQUFxhOlanb5v49hY2mWQHiLLL4VgJM4OeWupp1/c2ksMp7Cuv3PE4LyBR5Nnvbr8dvF0XK04YZSKY01ydXEV6Yav1It6Y1MM3Aoib22a7oSWwgAQtF07PScWWHtK57to9o3/HjOso3CpaOwwJcRvD61w19knXtsGma7L4Xs4xNGs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Threads marked with TIF_ALLOW_RESCHED are preemptible, but do not have explicit preemption points. Handle them as we do preempt_model_full(), by rescheduling in the irqentry_exit path by calling irqentry_exit_code_resched(). Co-developed-by: Peter Zijlstra Signed-off-by: Ankur Arora --- include/linux/entry-common.h | 13 +++++++++++++ kernel/entry/common.c | 13 ++++++++++++- kernel/sched/core.c | 32 +++++++++++++++++--------------- 3 files changed, 42 insertions(+), 16 deletions(-) diff --git a/include/linux/entry-common.h b/include/linux/entry-common.h index d95ab85f96ba..3716c223a703 100644 --- a/include/linux/entry-common.h +++ b/include/linux/entry-common.h @@ -415,10 +415,23 @@ irqentry_state_t noinstr irqentry_enter(struct pt_regs *regs); * Conditional reschedule with additional sanity checks. */ void raw_irqentry_exit_cond_resched(void); + +/** + * irqentry_exit_allow_resched - Conditionally reschedule on return from interrupt + * for tasks that are explicitly marked TIF_ALLOW_RESCHED. + * + * Enabled for both preempt_model_none() and preempt_model_voluntary(). + */ +void irqentry_exit_allow_resched(void); + #ifdef CONFIG_PREEMPT_DYNAMIC #if defined(CONFIG_HAVE_PREEMPT_DYNAMIC_CALL) #define irqentry_exit_cond_resched_dynamic_enabled raw_irqentry_exit_cond_resched +#ifdef TIF_RESCHED_ALLOW +#define irqentry_exit_cond_resched_dynamic_disabled irqentry_exit_allow_resched +#else #define irqentry_exit_cond_resched_dynamic_disabled NULL +#endif DECLARE_STATIC_CALL(irqentry_exit_cond_resched, raw_irqentry_exit_cond_resched); #define irqentry_exit_cond_resched() static_call(irqentry_exit_cond_resched)() #elif defined(CONFIG_HAVE_PREEMPT_DYNAMIC_KEY) diff --git a/kernel/entry/common.c b/kernel/entry/common.c index d7ee4bc3f2ba..b4cee897d6f6 100644 --- a/kernel/entry/common.c +++ b/kernel/entry/common.c @@ -389,6 +389,13 @@ void raw_irqentry_exit_cond_resched(void) preempt_schedule_irq(); } } + +void irqentry_exit_allow_resched(void) +{ + if (resched_allowed()) + raw_irqentry_exit_cond_resched(); +} + #ifdef CONFIG_PREEMPT_DYNAMIC #if defined(CONFIG_HAVE_PREEMPT_DYNAMIC_CALL) DEFINE_STATIC_CALL(irqentry_exit_cond_resched, raw_irqentry_exit_cond_resched); @@ -396,8 +403,10 @@ DEFINE_STATIC_CALL(irqentry_exit_cond_resched, raw_irqentry_exit_cond_resched); DEFINE_STATIC_KEY_TRUE(sk_dynamic_irqentry_exit_cond_resched); void dynamic_irqentry_exit_cond_resched(void) { - if (!static_branch_unlikely(&sk_dynamic_irqentry_exit_cond_resched)) + if (!static_branch_unlikely(&sk_dynamic_irqentry_exit_cond_resched)) { + irqentry_exit_allow_resched(); return; + } raw_irqentry_exit_cond_resched(); } #endif @@ -430,6 +439,8 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state) instrumentation_begin(); if (IS_ENABLED(CONFIG_PREEMPTION)) irqentry_exit_cond_resched(); + else + irqentry_exit_allow_resched(); /* Covers both tracing and lockdep */ trace_hardirqs_on(); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2299a5cfbfb9..3c1b1b0cd575 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6570,6 +6570,8 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) * - explicit schedule() call * - return from syscall or exception to user-space * - return from interrupt-handler to user-space + * - return from interrupt-handler for a task marked + * with allow_resched() * * WARNING: must be called with preemption disabled! */ @@ -8692,25 +8694,25 @@ EXPORT_SYMBOL(__cond_resched_rwlock_write); * * * NONE: - * cond_resched <- __cond_resched - * might_resched <- RET0 - * preempt_schedule <- NOP - * preempt_schedule_notrace <- NOP - * irqentry_exit_cond_resched <- NOP + * cond_resched <- __cond_resched + * might_resched <- RET0 + * preempt_schedule <- NOP + * preempt_schedule_notrace <- NOP + * irqentry_exit_cond_resched <- irqentry_exit_allow_resched * * VOLUNTARY: - * cond_resched <- __cond_resched - * might_resched <- __cond_resched - * preempt_schedule <- NOP - * preempt_schedule_notrace <- NOP - * irqentry_exit_cond_resched <- NOP + * cond_resched <- __cond_resched + * might_resched <- __cond_resched + * preempt_schedule <- NOP + * preempt_schedule_notrace <- NOP + * irqentry_exit_cond_resched <- irqentry_exit_allow_resched * * FULL: - * cond_resched <- RET0 - * might_resched <- RET0 - * preempt_schedule <- preempt_schedule - * preempt_schedule_notrace <- preempt_schedule_notrace - * irqentry_exit_cond_resched <- irqentry_exit_cond_resched + * cond_resched <- RET0 + * might_resched <- RET0 + * preempt_schedule <- preempt_schedule + * preempt_schedule_notrace <- preempt_schedule_notrace + * irqentry_exit_cond_resched <- irqentry_exit_cond_resched */ enum { From patchwork Wed Aug 30 18:49:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13370635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D47E8C83F17 for ; Wed, 30 Aug 2023 18:50:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A2F844017E; Wed, 30 Aug 2023 14:50:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 12E62440178; Wed, 30 Aug 2023 14:50:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFD5044017E; Wed, 30 Aug 2023 14:50:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C7378440178 for ; Wed, 30 Aug 2023 14:50:47 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 86504140385 for ; Wed, 30 Aug 2023 18:50:47 +0000 (UTC) X-FDA: 81181662534.19.35E0B6B Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf16.hostedemail.com (Postfix) with ESMTP id 259E5180024 for ; Wed, 30 Aug 2023 18:50:43 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=gaOyPPuh; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=jkal+VWN; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf16.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693421444; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Rgf7D4Qv7h/ZhjnYoNWuL0UL8IKa0QgHu+CezcwKZIQ=; b=tAs00kPwUg8MHXcRU9xF1E+UorG8HPWdhiXPKRUa/RkjnGnu6zpMxmaaYG4jCl5MkhY2JJ Omorjnxbueuca2DBqLx6Iab7/8ckoLspgLOOyXU4CMgQ2xZOP1Nvl9o+6qjCFw2MYLEFjh pZXRF8AxJBu0q0x9EuJsYzm/Geqpckk= ARC-Authentication-Results: i=2; imf16.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=gaOyPPuh; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=jkal+VWN; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf16.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1693421444; a=rsa-sha256; cv=pass; b=rV7rdYwBHAtdLvWDQxQENLqAKG1J5OHq/5h5XH1IRGXRJNJS5RZ20l3GjZpq49h2hTVWne 9zMlQi4DX1ASEvcOeL8bRqbCG9cYEajO0uz51ok7XiuB5sPSRb+NIKBx6XzMQjyJGAVnxi H01FSbpkk7MJjmOr24pqGgrGXi1pExE= Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37UInXRZ032362; Wed, 30 Aug 2023 18:50:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=Rgf7D4Qv7h/ZhjnYoNWuL0UL8IKa0QgHu+CezcwKZIQ=; b=gaOyPPuh5Id9LbMf+b/4Hp+ZJ3KfRwRQqR7h/P7/6Aw8rLoM529ZJs0gr+hMQnFgtB03 GWd4FESILOaIpEx4q55qjoM8kwVrmO8eH0UtqlenXSDsHBhdJq5D1MurhiDRdeuZ5ahq +p0ji8QwDb1SCUGp/ieWGDmrPNO65/TD7BRhG14E9MPsafI5PoBZee7AC8iMte+wL6S8 7bqluobe26FHcXkCf+60itDIB4nnoZ0uYlSE9OJrrsaTkvDGeU3OvIgjQbtxIIKavzUV MEnwtBy5qjPGQSsvHD3b7sdjY0xY5s1x2EHmzmb/c8cI/UGwXCcEG6/y+HPPi4zrEEZ1 Hg== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3sq9k686c4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:32 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 37UIFApZ032865; Wed, 30 Aug 2023 18:50:31 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2040.outbound.protection.outlook.com [104.47.66.40]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3sr6dqehx1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 Aug 2023 18:50:31 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gHfPsuzo1xpmPOjKpCDP1E1nwy8IG6ne/9OMT+idhGRt5DETdXmshkbAjQALZnMUOga/upkt5fXV9gRDKJLyjVyqRt2dYD8Nm9hY+Yt74ytsbY7eeHPhMtzTQQl7C68rH1hj3EU2igMig9ZSbykfYb0Nm3pflYTCXdrLCcOJj2MRJoItg4BhNhCs9mmZe+fLo4eDo12e7j0Zwx0UEx1XfwqPhI/IFI6wSGdegkUYaq4d2zz7UZ8U32ukjFkqkTQTL9AsbERu+G208HFCKKyEc9jn5gQ3fDVrcGS9BP0E7uWtEXivKdXMjefyjmF4+a+BriDIGHIgro+EtcuayDLOzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Rgf7D4Qv7h/ZhjnYoNWuL0UL8IKa0QgHu+CezcwKZIQ=; b=bXQGnROMD0XnmQU1cnv/5kTsBxy6m+CKzuomgSmOFtBfvAGJiBzzZ95y6AsgwhSNp3DakwTvDd9oy3AuA+ZHxXPeO3nvt0JwsaE/SsHKqFYeRl8P29PHiaO6YdPyAJEH6E8YtantJmypSmwX3JpkYiv+SKADKJuCa6egRd9N8kPX4iFNNEH3ELF+cbkClhW/K1E6fCZomZ/e+vyoLfFNEfM5SiUvoosX2Bg3HqNchz5XR/UEbKt+W51If1nymK0IU/ubmxqyZf9VKMOU3oerf8f3FxSRTG/KGtxAcW6+e1LvXVUWdFI+wT4Pg+tAz/owdG9xwhe/dQwe8yYXPPkpFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Rgf7D4Qv7h/ZhjnYoNWuL0UL8IKa0QgHu+CezcwKZIQ=; b=jkal+VWNR2kayRhoqHqwCWQTrJ2fj91kcWqdN+jH2qPIrKD9CvHUatIvHJrnaqgWyV3dsnrXWaHzw/Sp1zrwEHuiEVaJUqeC2g+hHPokCMVxOuGTUpsVBXCccFZADxB1LJo77YvsXX1ABpmp8zjSf3c6mTX0KrQc/md+wsiFDf8= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CYXPR10MB7897.namprd10.prod.outlook.com (2603:10b6:930:da::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.20; Wed, 30 Aug 2023 18:50:29 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::238e:5e86:cbd0:7415%4]) with mapi id 15.20.6699.035; Wed, 30 Aug 2023 18:50:29 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH v2 9/9] x86/clear_huge_page: make clear_contig_region() preemptible Date: Wed, 30 Aug 2023 11:49:58 -0700 Message-Id: <20230830184958.2333078-10-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230830184958.2333078-1-ankur.a.arora@oracle.com> References: <20230830184958.2333078-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0087.namprd04.prod.outlook.com (2603:10b6:303:6b::32) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CYXPR10MB7897:EE_ X-MS-Office365-Filtering-Correlation-Id: 4e2b1122-bff9-4520-d771-08dba989faf7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7bpP11w80OK83GggkGbTm7S6+CWSMxPJNVAUisHDmCpSS5g47M/4O/0jnSETpPhFXf2pnkeZXZaU/muO9JkG1KqsGCaYzULaeFodt+2zKQGPM0pYFXwcrzP08HC73al74VPevPBSvRVALIgnX8waaKEyHu3kSJSSUhPz5dmUawVWP4ma7Xu+k+zWmVTCKHOKbPTJuos+7sn5ZQQbn2GC09FVozqzGApBxLHZCeLZ31Xsr14FUEzuzuLF3jmUwPqXN/DaDfvL7Lc3+orZH2BwFpC2xb3PyWlRbIKS61196ioHIk4SEfaMkrtXV1ewpjmwTcW9MxAgTiIlGmisK2vx9/skWgtwea3AeHCXeehGubqoM8Zl3MfzBHW2zrcc/Q+QF7giMq1BrzyJT9jpWJUMOU9Jhk7hNbO1JvaKQ2HNszy0x/u2O/RmboQL1TRltD06LuaDO++q+QPF9VuNIPV6WG08+1R5mFyEYmaDoHKgGJTCzwLHvn8DS3/P2f/p38LWoWl2nIyG7ivbYeArtEiJG8iJrx/sTPZlrfX0Vt82cyZow91l8JqyjF4Krn1zVcy9+wO7ldAjYhzEJh9KAzQ+W6Uvfuuh6B17Ouw233g19u8= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(376002)(396003)(136003)(366004)(451199024)(186009)(1800799009)(478600001)(38100700002)(26005)(1076003)(36756003)(66946007)(41300700001)(6512007)(2616005)(107886003)(66476007)(66556008)(6486002)(6506007)(316002)(6666004)(7416002)(5660300002)(4744005)(4326008)(8676002)(103116003)(8936002)(86362001)(2906002)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: FA5hLm6enP0ZjokfOmXztjNjgTQCoopSfhfpFsicJlayeGXgYFjRx213wwN05hb90B0dAvYFFvD22GrhejjVODcOdmQjv9CN87uKeDAqUvTb3Itn5HQDALOtOuF18YYjoU6o7cy+YLU/j/pEs5/waFwGqfUPBIvvM5+BeOkkpHijvMP+NyWIuCL4bXx0oLWVS7vQaaExO+XjpNhJuT/QigFfWwUKXin0cwmIUhUgJODP8W2uVQKljqS0uEAiJ+F/x2eoXLup+++fcn54iSBBz7zieyU3fUkhykvec6e3Bw21zcjSfKDos/JXEY98AvwDbDPG6GgOK51oO2cFigk8MUc7DE9Hc9lvuq9hNhCny4Qbovv2jzUoB5badqhD48VIv7UxP+YN+qHTtoLfW/gDn33jRNRxSvafisuHbM9ZKa5sz2XeYrUZZpy9sekmNtbpsy+3uZmxvcqnAp3xnxVqKc87TvHaOyf9a7yzhYlhR1OPuXOjjmyIo4Yl61laWxHwxGUkfkkdmjdDPJAkEOLKQZV8BWXCcqZIxxjTO1LCaa9CAyi9jW6tf9luXiTqYv7c9MX3R7zZwdaB62GGUQB7seANmx8kbMiV0MVhMV0XdP1m1KIDjWvdpaYjSZj3p7y3l5cbydK4NVwEGZ5y7Uo04QQPbdJaigBh7v9/VDw3fvgUkdxw4TM8SnAV0SWBO5Shc3CwZajDIV/704GP37yC5oNGFPxFmo/V3Kjh2udCC+KGWEDapT/6w1PE0Pn5htfGKY8lASP2g9+x1QrABCjB8zVIXy05F7DHSyQV/gXPJnLLNRQuTaPl8no4jB4jzu3PoyxnMFGcDyjIQ1915HkIYKb0SJ8kvSb5EOCyGZcqL7h9ZPxrEaNH/A685ljSD2RWMjjsTDtYkYjhq+sNkDFYdaEgvkkj0XysjN53cBLIbj8pcsVOQxq3wYgBFIoicvTT7VVzHwzQV+XdYNTFUyrMo7TW37xTT9zciccV0XCV30Fqhv5KOPnhl14qPVTqRBC+Kp8FuE821wxKKTizes18wlAQyu1TxaJL9r4j+d5kNz4tWyNjiPc+9UDtv/Jtt9g7g6SiROLQR21Y+yP0PUQfeAgtUgv+CuQm07ceOdor5SkAnQkqp3vxrzvD1TMcbc9zwO1AJEz/OU9Z4qoYxJkG4VaYK9Yp3ooDBckp+qssys7qUdDZTkW7RUNP1pgm++H5JS1EknHyHGOPgB+tZYMaquvpGcDZJeEvtiZUgisvSLHQLX7L9lDsUoX4gZDVebbMnFLcaekj52G+YSj2Z7gbTpxY6K8zIMAJw40na/HLnO3sknMeSvZMiIdx1RpUL7hB2C/Tbvf1W0rD6HJ7WqvRBnKgY+5vBTGeWLGWY8oUlpteORzGIBAIlbumVkxNrUVaHRhRW/KNV9prRwwbET48+xN2E5dcIteNndHJTjzKgKOhtvAxgq+reayMd046YrbGh6103euaq5LQSaa9PxHjgxtP/3sxI7EjLNRcSX23+uQStuWzWVPZK5H9qQ/gLlPNNRM4aej9ES6vH0UBbXJALt/UIg+l9qoG8tftAH52yHXNtb7p4HmQ1Ml2+GRRlsx/DCTuV81yWo+DZknDas+2ag== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: HUbLtWDhK/62UC72yeOCUMOcxTVvssqJHALfhyWho9VVzaNJIJtsnG38tTqZ0lN33urtioUO2+yYI0bs9cVy8lirRofDJ145eahdYkCeDaOsam3fYkgKA7UgW4xO0AZvAbTwSgCQu05qYVUcl892Q/eiSPupMw1KeDkKj5oWwTb1lejNij878SaCF6NFCwmvxdvyTfZpvz7Htxb9plxExTxZERjKXFQ2iYEiUlR3RK5ohKsTDWWC4NF3MNK4WwA0/Vzh4+NT6Q8NXRLkUbqkIpCOb/BIY73c3IPWli1KDSt0z7gWbGHRRavpOPeoroFi3/3OTm7wAur4fVyROEDDWJ6xUiuFGjN5c61N3xPfBYE8NodAyGQ2TrfurS9u5sJ+LRM2WIelueAd3EKQZQZulrvYK0T63BjxSdxIk0tYiUi9mQ/2TRZ5lE8hOQt2l1NIdpgLDQuQEaK5Vko8BVaFAHGuy60kcv44rK+TlChQ4LWXwWmAT0C+iIfS6Xw0QjZlNR+z4vYXzv0C8YACCvt8Pmh2UcQROKaZfuywSwzm685uaMAuf/BGWPjeLaKpc1nrDUINsdc1jnDv2RFJPNOb42Xircb3FdxTKixGW0Db3Ol9onhCMucXWE68MHcmHF4A5YJTjBCjcSuRC/v57B94g57eT7EpQQxTUka1VZscuM5O/q7shlwarXhBNecUnKxl0dsN3/NR8pfILNQ9zbdFv4VJrLa4TKPTs2Me3/5lcguTXeMcAygjy1XLY9oWLnBl2cYy1WmehWHqnnHc9rBDydLLU3tLe9R5tkMAOSma4c139b60GtGMqHRtwfplroYU2mYPT10gDVhJB8LS+yV9DZc8n9gFQZTiRzxOWnSkT5izV40bb/mMR88RYlaGRn5aDmCGHLC+MWQlGkLT2BGYIDY3J1sjZnLlubBXrEoKsIgNcdwcDDLo5kGyK6DtDMtJOiYMQIOSsa89o8earyBo9q3yKyfdKhf/avdONH8ZGa/FDdBZT8D3y0s373nrvyOHfnCpOcYG2MQXrbNnAZGy6zTvOuPm9ALyEssgqtr72B3XRmboyWG/1MNXNlFYiqECsj78r+G+xfkyyYgtrgnZCXaTaNp78TnMW4/YRjLg4MPt30JAUFx58tsf2t/Ht7OYCZYpOgCsH/NTpdTUPlOwixTcc14mqjBFzq+bQg77qN8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4e2b1122-bff9-4520-d771-08dba989faf7 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2023 18:50:29.2182 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: c1jpT9NX6HKUIXnsCA+J4aa6hsaL19SCYYKBxDawvmAJXp3LITvyxtrC1sGQ1e9O92LGGqEm98mhQnhUbqFKRycem7irCnZB1bNUmbV9g2g= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR10MB7897 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-30_15,2023-08-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 suspectscore=0 adultscore=0 phishscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2308300170 X-Proofpoint-GUID: fevrx8sv2rJAwQDtN66tvTm5CqoXCAqk X-Proofpoint-ORIG-GUID: fevrx8sv2rJAwQDtN66tvTm5CqoXCAqk X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 259E5180024 X-Stat-Signature: my4sss5h7mbn6wz6ts6kkgqjm4xm3kwb X-HE-Tag: 1693421443-150544 X-HE-Meta: U2FsdGVkX1+qhgEqudw9POVqJPqWMv31lmywvHZucvCQlmLgjaj/4DrKnoXjcuSiyouAcMPPPGFHsz3AX+ib8E6Mus2LYkhx1yes7FKP/rtnQHrBVdikXW7vJmXU51sUtSRpalBpj8guA8LjPM3wwxSgn2LO+lxpGK0PxwGQklrBgeYiEmhbxVLnMUd1+oGPywyStqzgiBiSmTnRfJe/v+2QjSlTpVKMtOjH7cm8dlxkFduJ19vqOXS+ng/PKfvbdtFjLUlFj3RymcY2Lib1JhE+tqwMp/JL8l1bnRl1ougkVVqLCUD5xOSDQNX5OU9pUjekTIcI4ZNTPt1Y7ZnxCv6gU4G3k2IS1A4mBfOvvjmGUGE1FBghCTwbeR7mPLzn/6EVJdd7O7DnOqxSJAWgCKZuEptJ3bsliSwxNRkYvoedkcpINQyjkRsy2sYkQRFm10cMNt/FO3t/Rptpcobd/oRUxkx1TcqYSm+nFjw+2mOgy6OGWUzgKcG9KgGNCWhVA8NQJ6RK8pq0BfMjz5Ub3588a9cqQOpMx3EnsqTh9R3a15QYXH/A4Gb0S8o7bK5uVIVD8Ag0iRcB2tkn6jl94kC/dHxUHbSFHwqSWAnFe3FV0oZAGH+UuH3pdgDzNGMyp6T1W/GkYtWe44r2FXDo4MUVNA9GfN6h5bwVB8CeBleoQFsc4RxAscio6liCg7Hg2NBEsHwWr5aDCXYg4/6QKSA1+8+JomqfFbyXETbhFkEFOb4NttCUrv6pcJO813+7oFeYbESa2tOhJk1JeU4WvbNOpsSxJBagncEDp2P9Wst20PVrMS3EUc4HDkW6z1mYFaVae2PBQ673nkf3vrV+fohmVsH77E4WWWiXsDcksVFzuP1WXGufIt1mqTgMIZcOztFfCXCQjUZrUG9q51ziTfWzMoqs96zs0V3pRda5VNo/m7eoD5Lex7nU/gp2u5hYrlgPsdq8I/CUF+Sqxvy F+Pw9vuv Efbsoj1uK2Fvn6bS4ov+Py75sSJPYh6RPuJgOvUEQOOL8fFlL6QNi+M7a6a7YJUoWBpIPL54WFSIEOfHqbR47m9Idr0vrDC/4Wghg3cDSizquJpIVX3zvU0Y4FDKHv6XliJ8/CLb14ua98G0a8hqolsA/SimuFPRJTq2GeLTC2QMh/q8KAHhVIwSgAToRBgAmpQpKSY9oCpE0sOKfg08ljTWq34kGGigtsFin5QjSDW1PIQf2MW4z99t9593IjpDQjvdENZFD3PiSHTMFFM2pkFfF4wuoD8Qv3HSKeEWTf0/sBIDxK6ab6k3+GEskqn8BnHXrWzjI15TqjC8hnFtETUEeD9RM/GzyrqsOIf/TZSjvibyxyvpYTkcnIsm1qX2LIX6PZyk9GoXhccRg8MwhS6dvw9I6J57G62k4pr/bNUH/u2rYnAn5NUyDcTpTisSDU3tBcW6RBfFAuLD+quIBltV0MqYv2CQFEwwOVCpRO6s8ws1UFUjAdkQrf4ijY9Uk1LlOzkuO+Yl7BD8w7xAYrjWoK9v6Cbgi8R26dS1JWSyIqwD1FxNdmzHhRifLE//EX9gy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: clear_contig_region() can be used for clearing regions as large as a gigantic page. Allow preemption in irqentry_exit to make sure we don't hold on to the CPU for an arbitrarily long period. Signed-off-by: Ankur Arora --- arch/x86/mm/hugetlbpage.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c index 0b9f7a6dad93..55d1d15ea618 100644 --- a/arch/x86/mm/hugetlbpage.c +++ b/arch/x86/mm/hugetlbpage.c @@ -152,7 +152,12 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr, #ifndef CONFIG_HIGHMEM static void clear_contig_region(struct page *page, unsigned int npages) { + /* + * We might be clearing a large region. Allow rescheduling. + */ + allow_resched(); clear_pages(page_address(page), npages); + disallow_resched(); } /*