From patchwork Thu Dec 14 22:34:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 13493764 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06775C4332F for ; Thu, 14 Dec 2023 22:34:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 796EB6B067D; Thu, 14 Dec 2023 17:34:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 71F736B067E; Thu, 14 Dec 2023 17:34:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E7376B067F; Thu, 14 Dec 2023 17:34:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 48C756B067D for ; Thu, 14 Dec 2023 17:34:55 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 1E262A12CA for ; Thu, 14 Dec 2023 22:34:55 +0000 (UTC) X-FDA: 81566880150.22.C39D2B4 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2126.outbound.protection.outlook.com [40.107.94.126]) by imf28.hostedemail.com (Postfix) with ESMTP id 262BFC0002 for ; Thu, 14 Dec 2023 22:34:51 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=os.amperecomputing.com header.s=selector2 header.b=EGvakLnA; spf=pass (imf28.hostedemail.com: domain of yang@os.amperecomputing.com designates 40.107.94.126 as permitted sender) smtp.mailfrom=yang@os.amperecomputing.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=amperecomputing.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702593292; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=+BbGHNFDqbLs84+grYdyze7BUBnAr0BBq8KJg5XdSIs=; b=gzgX5EON521+ky0fd5884476IQwEcnZyw7WLLbRhKuWn57aSIZoRHSTDvy7Iduw1yQEcUO O1JpYZa5eKXjm7tw1hQJelatAobEXeI3HMJTlZ/GKFCT2qekgvaIeQbrDqTvzUjwUTQmtq ndMkkOT7xd3ANMnVZ144ucW86pWoCTY= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=os.amperecomputing.com header.s=selector2 header.b=EGvakLnA; spf=pass (imf28.hostedemail.com: domain of yang@os.amperecomputing.com designates 40.107.94.126 as permitted sender) smtp.mailfrom=yang@os.amperecomputing.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=amperecomputing.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1702593292; a=rsa-sha256; cv=pass; b=LxnjC2dVMwAKnAUTNbbeQO7iuLKHZx5PDnzz0GUtbQdNb0pztrMOdKd9zFwF4fLNzu2BpQ iHqyIuBTcuDNVneekEx2KzBzOTdzAkRZz3LHO2g6fZReqSMo1HjbReACHyj/IjhG5lm9h2 uJmE98E6tMWEzWGhegoEK13UPqyYhSo= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TdI06iJu09/dCtncdvdurFnNY3xG3XNcTOdwH8cv6O/U9jc5Z6qRiJqJF/eWTQZBFpNV1iiWEY65+SOxlrNxUoeox5k6poP/FG47aE37VYSqPnEOmOsZT5wN31xXigqdvxEqjU0dwe/pXTaVXHTdQDyQnHkShnUyIAOanul1zrpiiRjlbEu6NxsqSViG5C4pOdd0OYeLH7tpe8hG4wXgHn7a0aW4k3P0z+1bs77uJEpMHOM0VqQ5U6ispMuNFrkXSRZOPS/czMTvl4arBWQ4cH+mbTIZxMQxRdVRO0ziNfBozkdRSnNwGm/Keedcp6hQQkri9UhSsZLwNrH1oYsefw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+BbGHNFDqbLs84+grYdyze7BUBnAr0BBq8KJg5XdSIs=; b=ILVQH7gmhA2qV36SJQ1vJRo981bJ8TZAWqtJFE6EXMcvl2ZZSsMssp0x/Ok4rnFUvANDZNiT7F7+wRlxkQkWidXpwhHiL/33vtD8GjjJrbmraaVkZ100pMe1bzxLG4TnU//S/56rGaC/euGCcU9fKnH7qjcjk3CSLDJdO15pnNXddLkJpzrGIxOq57WfVGBS+oMlYEqGTPbeXNyDHbC72CsrCABEgDyHCqM440gL8AYRcHDXTcqkumS7BuKxpxveMoykcB+LFIA8XfNyAe01mM39tAUFujesaLSeYZa+iH80Z2iKc6BEfpmYEqB2jIuc7MoOl4JC/y/yRDFjbcO+tw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+BbGHNFDqbLs84+grYdyze7BUBnAr0BBq8KJg5XdSIs=; b=EGvakLnA/Io13ZX9LdALGtA1xBDVa5yT1HdbU0BeL73z6622Kt6QTyYwG4FtmbEKnErN3pRx3LX6Eh1ZDsimm+Nve2g2aJW7MdckGtwevI13p65wkiW25W40MPNoE4ETotSLaPoLFKrd1aGjfJ2P2XVpZym3x82c0E9Swki86IU= Received: from PH0PR01MB7492.prod.exchangelabs.com (2603:10b6:510:f3::6) by PH0PR01MB8166.prod.exchangelabs.com (2603:10b6:510:293::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.28; Thu, 14 Dec 2023 22:34:44 +0000 Received: from PH0PR01MB7492.prod.exchangelabs.com ([fe80::f6b3:cbb5:65c9:b0ef]) by PH0PR01MB7492.prod.exchangelabs.com ([fe80::f6b3:cbb5:65c9:b0ef%3]) with mapi id 15.20.7091.028; Thu, 14 Dec 2023 22:34:44 +0000 From: Yang Shi To: riel@surriel.com, shy828301@gmail.com, willy@infradead.org, cl@linux.com, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RESEND PATCH] mm: align larger anonymous mappings on THP boundaries Date: Thu, 14 Dec 2023 14:34:23 -0800 Message-ID: <20231214223423.1133074-1-yang@os.amperecomputing.com> X-Mailer: git-send-email 2.41.0 X-ClientProxiedBy: CH2PR20CA0011.namprd20.prod.outlook.com (2603:10b6:610:58::21) To PH0PR01MB7492.prod.exchangelabs.com (2603:10b6:510:f3::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR01MB7492:EE_|PH0PR01MB8166:EE_ X-MS-Office365-Filtering-Correlation-Id: 8604f7ff-84a5-41c3-2a20-08dbfcf4decf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: alfkTjEEtnIj0tgP2MuvCZclvmQRGTnijB88RLQE85GBI/QEv0QQtUR5k6XejXPz/91hsK967dHcsPJi4G3U3vXQjywnujuomyvTMNgje9X9AUW1rQZvGiuCTHEMmwM7qAe5ZUWEyGZ/DDGGjNGxOlwAW5VXck/PxdUnE4M0jyTpzw+or7s3XxsdgPTGp6fVWpsRNtp9I/llsk+Wpo3VXahFuSTGyEXAo1h6IImukDriix053cR4SHwODxG4t67EF5n5aP3EdetZiBThPtvEE65e+CdcSWlpd6Sx9S4iHweEo+uBwxRgn/u8kpwVTbDgi0iwvCFcGmO2cPC/QwFgnS1zHJfmxF531EGNpviAGEcTy4Ln13ICfe/PXigCk1lTIxU0Lq2xxKhHGkOFT2YLad0iZypHGE9eFB1LNfV10Z1VPnrTP/FLTycI3jKJeHXBIcceYQfHpnQEZPtrwAlfDB1lhFBG8XVaJuWs0D4bS6SWSMwvdDJ2Nnqo0xz4Bhkk5YBe+CclmXK1qMowRpCE4+FapLcvifDYEK3tUeApkEhq2rfaJPNsbNxrDrmPWIcRt+KGygKbzjOvwDI0JvvSZXYbVJK/ZRz3c7f4ioc9H2U= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR01MB7492.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230031)(366004)(136003)(39850400004)(396003)(346002)(376002)(230922051799003)(186009)(451199024)(64100799003)(1800799012)(8936002)(8676002)(4326008)(38100700002)(5660300002)(38350700005)(2906002)(966005)(6486002)(478600001)(1076003)(2616005)(26005)(6506007)(52116002)(6512007)(6666004)(86362001)(41300700001)(316002)(66476007)(66946007)(66556008);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Yx9Q+ASuDo4HxeC026AfmbjQPZTTAncbCGBToBQALFf+fMieEacJApSJ34c3Ti8vypclQX6Z66KqQ4vxPgfjHCTcTxLzI/4DcB14DJkMl878bh17bMEbc1xiS5Vi9z9my4xM+yOWElxf8/9Iu08mCjjgAwjq7OvodkgETKNWA4yfxkxOkTvx2wvoK9KhQgSuFKn5CiO2mZ4t4I99jJ6XXkuUG6j61z42xeOH+8KZ+9sey56xzdJ3msIaOmBWLnKckJFvh7JEBABCz14cPYmO8if+J9UEeE+T9JiaxFrp+Nro27jVTGr6zAmuZayFRm87o1XzK3m5UQQEUhfNnE5P8tndWQ05H5MX+URb9j0sTacpkS5uheIAobi/jqILhyyXReGBLKMn4sW+0WrbQQks2ia4DEQskto6NtLsRfOxcDvr5yT77/RLhTgVtg7UaVCn/OAWVIP63usRKY6lxqQFgKSMfJKBEwWToq/NWK0XxOgLAnw4xj2kG6yIG2zSQgmJA/Q1giNqm+8BToyWnNZndgH7v3SbcbSE20xitz1zgtf6fA499poa9OB3kXp1eOGuvUnssahSmvCjWqDgJF11b02mSEMfs70ne3xSJs7K42cd9cqpr+T+LeCpGGhauqBoZB2wvXiOgdZMc2GGUYDBVWmPDJ1J58ytWm78EI7/yOky+sjhRw9n695PSGDMgRr0vkYIxz1wF2BboEGnY2jp9emzA7Bj998wXQdMoFQd4/LoqWR24pzVGZtqm6skhVGsCJ3LGI9G7WsoeRB4sfoQe4e8cHDxvT0e8X4jFeWDr0cM1ec70EKsXq5VfcVXoD7FzOdoAhCz6d0kquA/xUkZ7hRcA+2l2EdDIr1lMuRMJ+/n+xeYtw3Yps16whdpmYrgjm73aeyaKmyYZvSZ8EvlSGRhzjNzesyk0vSsJv+57+4hYN5bbY6Gsr97RC8RABrikuIhkmFQpPsjN81cGQvwg8LAla+NyL8BOJAsy+FkkQc0/1QS0au+S6rA7cKtEg9RnVc6g5WHnoAWzfJjL9l8HCKXi1bUbsMHtOTPhAynkKF876EsJ4NtDtIyYs6sL8bnYnwS8Yw38yRamKkYcgc6aXKlzJfVEux8i112tg2jgovpmWd961pADbi99mM7KT4jIJsqqtvsJ/pucL1y3Kx70t84PLSOm7H6CbbsTZjN0BPoD+WJbxZRHykUF3ijndzrL8M0RvZWKfEcfW/R/oU0abIcwK2XaENSzXdkbcuAVVnD+J9qtTWp+YMHGBjU2Qhrp5eGLnHcvaTr4h/salCOfXuxfBPF5mMIcL5fY+grN9/su8rE8HTSdizCCuBDG4d7owo5AVWHCgVNkEe6O3QlR+zRtOfhtUWpwfkYvNKRPsUcxsRX0S8q6aeITih+HJrXIhDZOgatAYbJFNtEkKc9c/kZHB46ZM6mNoWteuy+R6Z7pClwQU5BnbQ6qkAkdENBiZvL4Y+aAbg6+OpWQBbJ5PwG2dWYKhzWM9hm/FgXEyNY7FRhR5+HkxLWg256ldfw1huGWoXyOazdUjvLxLdNU1SrZK6FGFdJLwKga7AXRdMwsMFVs8yFGYO+rTIQsSTKrGacsVfj7nXsQviTPRDWeGFy5TJq7F0DLhLvFIuGXZM= X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8604f7ff-84a5-41c3-2a20-08dbfcf4decf X-MS-Exchange-CrossTenant-AuthSource: PH0PR01MB7492.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Dec 2023 22:34:44.5659 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nMh+cJuTSCSEaxPuJK5UHrbOqOTVxYik36Yj3sQwUkfRuXorecVFy+GXGgusia3tA4ClrJDFqN3E16YUR0IH7oALZAZUcXC1+wdzPNsD9uE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR01MB8166 X-Rspamd-Queue-Id: 262BFC0002 X-Rspam-User: X-Stat-Signature: 3cwsq6z7ar9wssc3x4rb76j4i48iofzb X-Rspamd-Server: rspam01 X-HE-Tag: 1702593291-913304 X-HE-Meta: U2FsdGVkX19KdilhFEWCW7LFPrub4EIPHJtkj70m1cMcoybSjmQMqSdWytcDS6OddyxVaQ8lD/q+xpdkclrUaIqF2BIaRoY6FEENl4zwGZDL6TNqHKI6mKm4c3g0ycsTNoTquGn08q+zMOlJN0IqD87HVNfnXHtpqJMmjCzRKqMg6yb8kgDt1uwUpsWzuZw5rfFy+iw3Bozs0Q0GwEEPIrfbul3Yj2jhpVb7pWdIHzqUG2cG3ESiKkNa4xYE8dEArvwEPMOr/yFaE0NkErBhRCaOw7gihxEaWUQZ3KB7QD9kFlj3ihwvyb+vQcAjd0tX6w4LTNSvEpSnvdKSzTQ7jdmxviP8B8QoFjY27Ph1YbdUE7DHmup/SVnpYRDnxR1wGenyjv87tat9s84gqD+0cFYGB4LcgtokZKwMHcP+I2ziwOubxqEB05YDqcOPBFOGugo8DWz2y4ryJsbBZJtvwMoI+DkDEpSEWGhHTRH0WOBjUO0WSBQYXQ1mjElWgsvDtKVS3IG4mzxtfSGUUZXiW6t2sIAUR9xZa39B5WpmMADhkmYMbze8ymEfr3khNM8L1ygOnclkK7CjVrJHyJRTua4JMhFesrbj8L0XmG70vjfKzOeDmr/B6gSSS4mC0jiyVcknYqSTcsaBvYcQNNbbFDxtJnOmzHy5uh9AS6TbZFGP5u5pV/P++AAj7+/dJBbo6QEI0cTwd9Kue6ErWX+KOlHbvZGocixuuxH1kpJWjAE4uDwUW1pQhs+mY06EuOH8uHjyrPAnF2vhhpzpZReviSrFd8ev0gVX9appRYIN8UITO7gHOl9DM1bVHAvTatrxtXc+dHUXDh2QywYwoVt/viJ6VS9El9UQW2XQO0/S59vb3fIv0vFv0nvqxPTMuFJJbT/wJxGSlRdDoqpVu2sCzPQTcR4loCbB4pexF46YwUgSyeRzpa93MWXYE1hOZ486yn3Hq8QLTrpWcOX6J3S YR/Xjr+r Wm+6hUtgGMb9qZAWc+UtndQ0erCwK134/dZ3dZCg3m1fg27wqU8K1zBviUCr8PSJ42NLykvGXbsgBU+pzVkf2qgjl+FMrFOc5RhJaKCBb6Jh7uRecpDx1WMJhAhaauYFlkuzlD3beItaHjFROcmLYh1Of6o6llSqNq/grS5q3TFkMT3fEDzR9VSSBr6dyektzXeBMtew2d+wg45bU3qSOsSQ/SXzMtcYueUMBqYrmK665giJ/EJPIW1M3nMNrgHBCZx/IhfW8tMUoJgjqExzzt54sMnKx0m2QWBptZiMFk/YgCJe2tHLFdMoUfAfcRcMaw3WRo/EJoYD6bZoeEkZYCJ5Ho7ln1E7TfphsoSYURHOnZj0TEPmjUPsSC+sUu3U7e3Ix7P4goDeqMdzMsJHH2ngNmdNuflrC7HS6FdP+zuSWK+U0FAbV6PkeYMCW5XcWmJtzogNI+5r8+iM1rEGT4aX1T+qZGjx8/MoOIgwWZwVCNBga7uOeKUAQfB2/9CG1oEJp1rTY61v7zDBU3M4ZYiidn2TbZHDiiRYNd84tFS4iKLV99ag6nXdtz48aaFFMLAJLACbpXftk765l9bjVqww/RA6wI0Ns2f/9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Rik van Riel Align larger anonymous memory mappings on THP boundaries by going through thp_get_unmapped_area if THPs are enabled for the current process. With this patch, larger anonymous mappings are now THP aligned. When a malloc library allocates a 2MB or larger arena, that arena can now be mapped with THPs right from the start, which can result in better TLB hit rates and execution time. Link: https://lkml.kernel.org/r/20220809142457.4751229f@imladris.surriel.com Signed-off-by: Rik van Riel Reviewed-by: Yang Shi Cc: Matthew Wilcox Cc: Christopher Lameter Signed-off-by: Andrew Morton --- This patch was applied to v6.1, but was reverted due to a regression report. However it turned out the regression was not due to this patch. I ping'ed Andrew to reapply this patch, Andrew may forget it. This patch helps promote THP, so I rebased it onto the latest mm-unstable. mm/mmap.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c index 9d780f415be3..dd25a2aa94f7 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2232,6 +2232,9 @@ get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, */ pgoff = 0; get_area = shmem_get_unmapped_area; + } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { + /* Ensures that larger anonymous mappings are THP aligned. */ + get_area = thp_get_unmapped_area; } addr = get_area(file, addr, len, pgoff, flags);