From patchwork Sat Dec 7 17:16:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13898344 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D71BBE7717B for ; Sat, 7 Dec 2024 17:16:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D4D06B0347; Sat, 7 Dec 2024 12:16:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B45A6B0348; Sat, 7 Dec 2024 12:16:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 525756B0349; Sat, 7 Dec 2024 12:16:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 350AF6B0347 for ; Sat, 7 Dec 2024 12:16:44 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D2B06161B87 for ; Sat, 7 Dec 2024 17:16:43 +0000 (UTC) X-FDA: 82868817150.11.8F32996 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2056.outbound.protection.outlook.com [40.107.212.56]) by imf09.hostedemail.com (Postfix) with ESMTP id A9F8A14000C for ; Sat, 7 Dec 2024 17:16:28 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=OdEfi3rh; spf=pass (imf09.hostedemail.com: domain of ziy@nvidia.com designates 40.107.212.56 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733591793; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=LrtZWxRzTQc7SESzKzF5aEtmFkHEZlyLbV8fyHJAHNY=; b=wV7sfWw71ArnlTXuerEBj9vFojB/t5hiID3lwEtyM8g43uYOfj0WVF4+yCSKSxBR+m4BGk zJll7ZCWHaIAvST/eIhs85DxRWeqCL1rb1Ftt2qKAYvlXJsAoOt/81LhjIKmebcjYhp9Mx dZI8QP8Xup6ezqGL82nsimMOBRGrBPU= ARC-Authentication-Results: i=2; imf09.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=OdEfi3rh; spf=pass (imf09.hostedemail.com: domain of ziy@nvidia.com designates 40.107.212.56 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733591793; a=rsa-sha256; cv=pass; b=ghhHaLwcfZEL0bSeUZUoS/W1oFWduqO091OzJOCaqlc/AEkwvXWFRYM+RCoedhXB5BcvmU 782BJqNOpQk62XPFEBSMm8MeNsxyh1GKtmFitcVsNDnmF4ih68BhGh0211peIt8UFHveG2 Os5k681Z61QPf4ZsAqckF87vuKkHY0c= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HNKRGA8XpcaoXgRWP6oHW4OqPpNHx3cmx/GOHcBz4zlz8xNwOuFbf5NSNEP3BcbEJ4RMalhb8CcxEL6f4oQbn6hcm3zeJwimEbsfnXrYpbFoWhPaTQ47KaXreoIrAWzCvblgJjLkAhp8OVxhXzv10J4yl4PZAroK2JykDOdI6grfvn1ix4AbFEIX3cR2uJEvJBTn6ryexHajoYLp7jHJn1JZ3Df4cjwSaJo1ZNWwHexvcVZn2Dj0L1TuQ2W2FYaRawAjVvEbFwDyn7txYznJlLlb/zECjlIQMnIJ6XRtG++B/MEmCHsXMEn9TLZzSKvnJP172q/imO901UJl8Jdh6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LrtZWxRzTQc7SESzKzF5aEtmFkHEZlyLbV8fyHJAHNY=; b=FIN1VuoxdYA6uGp/nOkIPk/oxNAWlOCuzneqgbvOY0RF/DIvAYsA/2C2Q49w6dxCf3MBIuXTi0RHMqGIU5suQ45D8OMUIuLjRBDaLMNSn8grjqkuMQLmq5N10HRtNngFwOSIU5zZdUKivJZq6uf3tj1dSGzsUZzOgTz10WJKwyJSD8fhr4hTi5Zj8E7TpFsr7jXsW2ZG5P4g18082Z2lNVyR+14bSIPSVtz8lWOGlyriK+BIB56z/O7Lo+g4ZjPtJkezkH0iHO+rV51i9V3n1rwVPjpR5shKpnFmlbAXnw00ekBHbdPsO0QgOwbo0vdOHljuw89SS42bbNh0TB1Sfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LrtZWxRzTQc7SESzKzF5aEtmFkHEZlyLbV8fyHJAHNY=; b=OdEfi3rhmtsEP08m+7Ns8je976oP6OctKdt9bhAxpGZDMBpInr1s4yDsu0RbGjsAPswNe+MET5RPfMAZW2Ig9vXqcFfNo/3WYpN923PQin5b02KoWZ8FEh4kQ9ags+jw1wdWV1mcQbSJJ0peN35duQA6I/N+23p030Jj/cgG0f6fKOCPAobtJcG4M00328v0O/sntHEFyhsiQgWHo/sK/SfCvNkT0C4NfYEUbuRUTDrh5Eg3QZHbCblUs8eOHwjtWaUdoGeSFdehStW+ltXiG3JYl7XfPmpY5XtT6UO2pdI7zVwpVGcz/lZQxt5Ql7QPpv6KbJSFZGlmxUyRXB5thA== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by BL3PR12MB6449.namprd12.prod.outlook.com (2603:10b6:208:3b8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.18; Sat, 7 Dec 2024 17:16:36 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%3]) with mapi id 15.20.8230.016; Sat, 7 Dec 2024 17:16:36 +0000 From: Zi Yan To: linux-mm@kvack.org, Andrew Morton , Geert Uytterhoeven , Mathieu Desnoyers Cc: Vlastimil Babka , David Hildenbrand , "Matthew Wilcox (Oracle)" , Miaohe Lin , Kefeng Wang , John Hubbard , "Huang, Ying" , Ryan Roberts , Alexander Potapenko , Kees Cook , Vineet Gupta , linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, Zi Yan Subject: [PATCH v4 1/2] Introduce cpu_icache_is_aliasing() across all architectures Date: Sat, 7 Dec 2024 12:16:30 -0500 Message-ID: <20241207171631.2853336-1-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 X-ClientProxiedBy: BN1PR14CA0004.namprd14.prod.outlook.com (2603:10b6:408:e3::9) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|BL3PR12MB6449:EE_ X-MS-Office365-Filtering-Correlation-Id: ce108cea-58a3-4d01-b940-08dd16e2e7dd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: e4BjkR/sC1yVHcitd95CYFvzqdhIUCPgUftjiyPKICTHalVrrSdl58OyHee5j+9tuxJtEbz3GMXRYQPvDf2hovu3R7W8yEzV/oDietOp3iuVoViUY59QCf4DhdZhlZZeYs0vHrHCPhQvTMHcH5MEPnFqzpaKGK7qWifmmwtuVDApsDsO1fOl9QQxJbaOIFRNRWlTrHofpivJ5EaFt19KR0zySzfJ827GxmBsd6bffopLQVeGPoY7kCDPRbCkgyCkziR6W6qeRKu8gOFUiNpWIvZcxybt60lGbc0MeaKDaQ9d/IL7Kz2icVpHyFO474QFE8eUXTyPeL9aERIzibKVU1CDsRSmw72nLRmD2h+TZF47GkC4nCjESvUgSxlsG+El83qSAAFhpmtMKkZsqXVnY0UcQNuSnyPvunqSx4o3e6vkNIyZ4oigaXYQURyahV/MsLfJRPgMAAUDq9tQVzgoO9fvEqmmc77zN/8KfgwGiJvC31ab3zrZLx+wJ5yOIc6boXst4HYQfmNzDNDCCe/BZzw0pmH7wFQ6Mn8fywnpDA4DjjHTVgVnyARANfIu2Qv3GUfg08CfAPUBqWbbr1Ou4ppdn8y7IohzZArszZJXEXEoFZP6tClCe53d1oO3TBeVId8PkV/8/X84Ddj6ZdAJhf5D4N1ezD72eP9VN3X3ClZa9WZfUOx1lKZfwkeT2WkrklMRzqRT/ApyZ+FYQEiE1DafeANZzlMcQFJEZMgFRn54ZCMY97WGuBq7n91eEaCURG9quwCTtTc2JDXdU1ggjfUxijUx0k6ootkxIjoUkFnd3mIdsYtGCKM+NdtTf78cDf+r2bf20C6/zZqHdB6XDYJWlqe7LEShCrsNlfHX9qFoAQ+5AxtpO8cE8QfmhBYs69l9H1lyFzwhVtbfkmN5AWsmIdjfJR0jX4QPm4q2466SMJQ8w22VnccxbMLy+kozWxE1Ucmdy0vfz1boaUn783dZsNPhzHcI3EaXkbKCDDkK3uSseo7xBliCY7hmxOj3SWeVcPt0303zvBnrAvrl98ZKi9PRpG3gyaIF/beI9WF9807+4fehQZdzWscrmWiBLIm486SaJK+3qx8u1FQQkv/BQO4PI6780tIzaGXlgHdN2ucnDz649aiVC+u6TB3cQqmazbTlftwoFC+s1mbQGbprjGfiNQX8TLdcvRg9QTSoOPiM266dzpS4ji9CPDlFcivWsMD+SsbXmxBqJaSygJd5xrHl8pohWzaEQDVLJUsb42wOpyBnwLewvOSwHIKEN3GMlagRYEnuOBa1D1ermNIVoLw016Ia/Nvaz/gT4jOOLD7clN6BYj7Jw1iZ0+8hwI8sXH11ucNE1S0PQDQopw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 0o9me9pa9+3ZEPt6Z+aJfI/HmepJTephF3Opo0Ue1NW/WwqPx8qDH4WHlIV8tFZ1FFCzJHpeCqJgYgRiZuw9Vm+ZxYX0h1dLnQ6FILRqNykKtPqOnepUMmEF04QppS2gTzgNrd8eGud5SUWu0HLfOOCMimsdv2PKgnBIs32WJCEcqqcudOhjCuofx1WOWNkvqQo+5LjQrWIcassqY1CIP9WLq4meaVxx15FFOw4gtNuzR7iIzoWHeif+JlrTa0Hu6ywJt4NV1T2eLUVewSwbT0OpE8xBVUoLV5ENVUGeKyTgDXB+gLozPQbf3KBGAUZhRcISjRj50ztL6iuPvx6tBdsKNPn4ZlRF4NF1IHsKlfkuOi4Qq3d/YZycpcC1n6zSjiAboSbZo6NS+fTXEmD7s2/ZtmX1jJiz/GYLm8URPfbB7jVG9kaZSSt9kZiWv0+hhdP3za9V3MQe2SwRfEbfhRbEGcaWkcsOaI8hgD5KuSb9iS6JDW/7AtQUCv/JxQQs4iSOzRAQaY8IhIsVMiAIhHvf2YD6bURP6LQhmIjMC/ytamGOR9sUKlxBz3e4QR3frRg//MqxUMq/K9AfY+KEcj9FC61UXdutZLwu+f4aLlZRLM0kihecIP+U4xLkdN4t72BPMZ2RaE93Zspms3cMTtDe46VWtl6PmKFkXhN6g/yF6fJARqbZziwgb9TbjkMkumByhFOvwsC3erkNeAgTP7YA0cu2zi0PV2jN7JYheMw1sdTGdtVOYfWtk42FbOiBkp4JjB4LuBHt14IwzeZB05yjEoAMSplR1XLihfPeJGQN/yAdB74buZnyWcb7Pgxw3IZ/0/NKgOWPMxD9P26BYBOrfhmJzwKlfjJWWhtnbgDCnUJCNSR+Otz4CT55bxBAK7Knl/Bqj8nzqy3H4/1uDho2FG31gNM2UhvMaT8W2Nx3tNkFf8U7YehtpLWLYXLDEtxGHSLtKgUFxqG7rIwllC/p37iUps0XjfUCRID03lhsALBAqE7a6kb2ka9I456aNO0lNOl+iEmHLC6BT+0ajAlUi5J1eSWcJKycRW83HqXM2+hWDi9h1JYEPdeynsx5sSWekkuN3wgh5QbQrM7STs38i3+Od0YeCrAfftYZlU5yZmMgleLqccEhco7/uiC14q4zVQlK4cr+zfprxMOSHCQNJJ+gPo05pZXHYziN8Bt3PeR64dW1M+lc+rlFAFEDjVKs7y49Wi17BZhe/krdJJzMGhwgsc6VbzYbUNq796h/3Wzmwo5H1PQQAEgACUnPVnGhC8JA1eDHWtXivO7H1Lu9DVlxrqrk/do8Ovtd9RX6xzQ39DnW3tnhQfO6+bv+ACmj8uTiYdZm78zKwzQPL5W7rv0mKIKXBQ8I0ZC9eBT7TwBSnJJj0ZvGgsvuHSSdYUOSgHnG+uLzKcA99oRP8RM9lktHVu6lgprBhz5PnPC00CVYaLo38qU+RlG5fMdf+ClDw5CUiFimDw9rdLy9H46dNdfMs45ERKC4aa9tJQprNxdKKpMLLwKyhHE0F9ohBN4qqNf0R/COgqtEpr3xHcCSAlS+QjHBk/yu55P68p0= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: ce108cea-58a3-4d01-b940-08dd16e2e7dd X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Dec 2024 17:16:36.6148 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xQz3EqCFMlESmkdZIqHIbinp4PWkqyQQUZHb6LKP29wK7NgeGcecLEcxfVfU6cX5 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6449 X-Rspamd-Queue-Id: A9F8A14000C X-Stat-Signature: upxugkhhose1mru4zfzrs1ptdkmz9nt6 X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1733591788-709113 X-HE-Meta: U2FsdGVkX1+gw6Q+c1y1NOnVxQRxMRpz17OIJc1HiLIdrnR8Y2W+VuKiFXkgL3rVao5XwVnj4qy78X85krWsTiuFVIaebW8HDGc6UogcaeaQFhWemcdT1CJmU/5nSwQExxum2VMcWHGxT/WO0am/jxKfobA6R0BVCcRAfRaEylz4pTua0M306kXnOzO8VycZO4r005CJQhjeWO1VEg93gL9CTCXqTpaDsy33urq9XylOBpsNSza1KRHFd+D58gaYK1qorqDnebGvZW/PViKfQGIlh11Q1SzNJbgEwsVP3KI+azogcFg2kJbGH+y6UDIsBHS9ZPp0qxyPhI7cu25ExAWKWGeM0PBwue8SaiEdZwRij/p93p1iFR+VrSRt1w7yua+9WpRgy7TcKcH5KpGhUwqm0Q6DzLikO137U0Fa3HkeYAYbZkP68nloFITQ8e8G5p/S1aX4BT7CB/kBji69ZE4YmFoZQd2MGsglgzWJlNvocMMaUghMZWtVD/RRWwtsetQ3EtvvlM6FacPqPweAAjHNcGFWE+sRtcSfqdfbewtn4Jsvnb8kDxCgxbHg4yMIlkjtb35RfHY7uJ5R3edTBUEBcKZdUkRvJjv02rovr2K+57ll+RQ92i6q1rNAYRwsywM8H5MhNF6dkXw8z1szlypJsY1ETKF4RTYEOq0spvDRGsIWxqRO5FYDlVP66bOQJ2cxcHb4/aX8fH3ZNP7Y0KpMJp88eV7r1KmhwlNZOluGECkm5NO0yBwFwjoTlLo8IasgoQsQtn1K5CnzBj7SEnldq//X8yeBBkgpgQcXenN/1StVTdw4jwpuMWfNmP0yyDO7z9H59i9UpYT7AlYU9ADL99y5N1Y2eZwp+jtE+I3o1/wYExIF7an2mMUI6FIz9Y09RvNvB8iSbLvBiUX1ZJBF6I5S2oloaUNf6H8uXCalDlB2eqcaQC9qgh0Cl8tyoAIxZfhh+e5wn1+vyVj y8hRy+qT YInSes+nTpeUmyA7eBfkZPhrLbjwdSQbQpiwnhLp6ITgCtIwANVT3MCifiIfrmeBDUxa6Do7uK/nxY6q4GEdS64m1w1jcY5OOTCdzLzfOrJrZoquykjjb1HcDVmfNJKJ47GFz/56axYCwzO4bCnkI56D6d4pglkHNQe/SeY0ThN0sfUb64T5hdVqc6QmgBpqDS3sk9oiJOvA3uvDU0BpgNcvdyjRl+pktNBQOs/8RSWTKiiO8/2nVbSR7GOsdnIU/3Uqik6Dexxs++Emdcg4SPif7YDSx7PlrAYhxV9tDTH6PW+G5m11heY/LxvFU6m1jbPSQcuG3amvYiVNiroGy4Yl1iWtT/mThh9/YHTcTLrUraXA4/Mp3q2bmIz+oraROjWDSK4qDED91psiYZJi5xPLETdn/qkRixofugLOddwZ3NzA3qwUxoYl8S46bzCxH2EXN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In commit eacd0e950dc2 ("ARC: [mm] Lazy D-cache flush (non aliasing VIPT)"), arc adds the need to flush dcache to make icache see the code page change. This also requires special handling for clear_user_(high)page(). Introduce cpu_icache_is_aliasing() to make MM code query special clear_user_(high)page() easier. This will be used by the following commit. Suggested-by: Mathieu Desnoyers Signed-off-by: Zi Yan Reviewed-by: Mathieu Desnoyers Acked-by: Vlastimil Babka --- arch/arc/Kconfig | 1 + arch/arc/include/asm/cachetype.h | 8 ++++++++ include/linux/cacheinfo.h | 6 ++++++ 3 files changed, 15 insertions(+) create mode 100644 arch/arc/include/asm/cachetype.h diff --git a/arch/arc/Kconfig b/arch/arc/Kconfig index 5b2488142041..e96935373796 100644 --- a/arch/arc/Kconfig +++ b/arch/arc/Kconfig @@ -6,6 +6,7 @@ config ARC def_bool y select ARC_TIMERS + select ARCH_HAS_CPU_CACHE_ALIASING select ARCH_HAS_CACHE_LINE_SIZE select ARCH_HAS_DEBUG_VM_PGTABLE select ARCH_HAS_DMA_PREP_COHERENT diff --git a/arch/arc/include/asm/cachetype.h b/arch/arc/include/asm/cachetype.h new file mode 100644 index 000000000000..acd3b6cb4bf5 --- /dev/null +++ b/arch/arc/include/asm/cachetype.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_ARC_CACHETYPE_H +#define __ASM_ARC_CACHETYPE_H + +#define cpu_dcache_is_aliasing() false +#define cpu_icache_is_aliasing() true + +#endif diff --git a/include/linux/cacheinfo.h b/include/linux/cacheinfo.h index 108060612bb8..7ad736538649 100644 --- a/include/linux/cacheinfo.h +++ b/include/linux/cacheinfo.h @@ -155,8 +155,14 @@ static inline int get_cpu_cacheinfo_id(int cpu, int level) #ifndef CONFIG_ARCH_HAS_CPU_CACHE_ALIASING #define cpu_dcache_is_aliasing() false +#define cpu_icache_is_aliasing() cpu_dcache_is_aliasing() #else #include + +#ifndef cpu_icache_is_aliasing +#define cpu_icache_is_aliasing() cpu_dcache_is_aliasing() +#endif + #endif #endif /* _LINUX_CACHEINFO_H */ From patchwork Sat Dec 7 17:16:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13898345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D506FE7717B for ; Sat, 7 Dec 2024 17:16:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6ACC16B0348; Sat, 7 Dec 2024 12:16:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 65CE66B034A; Sat, 7 Dec 2024 12:16:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D67E6B034B; Sat, 7 Dec 2024 12:16:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2FD9C6B0348 for ; Sat, 7 Dec 2024 12:16:58 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A715241A32 for ; Sat, 7 Dec 2024 17:16:57 +0000 (UTC) X-FDA: 82868817654.15.5F6A45D Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2075.outbound.protection.outlook.com [40.107.96.75]) by imf08.hostedemail.com (Postfix) with ESMTP id 94D1C160004 for ; Sat, 7 Dec 2024 17:16:42 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=jiz1glDA; spf=pass (imf08.hostedemail.com: domain of ziy@nvidia.com designates 40.107.96.75 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733591800; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OE5AjSKRmuU6Rd5zudSUyT6TNtRUji94cy+vQKppN38=; b=vA6/lxBP9Ow9dD9EjmtYMm4BW02Sa3D1hp+hsUOQ47knBCr6hab8blP6cNucg4rco57VW+ +7uwwF3vyyFJTIr2L/YSLjCLwVYpl/DsAmNTwcGfLbwD0z2LwhygvL1AvJXp+Et5niPQq1 dQecjqMRPcItHUqFZjr6ANtQmmCJv9s= ARC-Authentication-Results: i=2; imf08.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=jiz1glDA; spf=pass (imf08.hostedemail.com: domain of ziy@nvidia.com designates 40.107.96.75 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733591800; a=rsa-sha256; cv=pass; b=ZbZS1z0crzcSDEDvstMn33lqcPQjI4pBB5WyNjIBB6gvCaTXiO99VI9ZJqvrzefc76LbgN Y/lML9bVMTfokAAUxyg7/H69LebA+hOhkhCjR/tJzWSWaeoyjqv9mpkDD+2BiX9BTGD1SQ dU0ESCsxtFCbcyL7QN9vCBZiSk7BUBA= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Z4IBbr01Tn2UW6VLPX9QvdB1HUCheC320LdUH5KbdGE1WE/NrDEOLBDDEa6quzbUsqeK5jSmZ99+C7GBXKZvPwnkZpKCWa/m1osBK/k700ydXb/ygtStx7JsvOemMLL7hRRhsW7/6JMksG+3GL/IDex/skspaZoRNp9tROOdttR5cMTw/7t96N4TNvD7zdarXjfd41ralYZuFh7l6J853jwTtgaTCGvxmgNd4eqHcvgNklQNpdUDjD0OYoE7P+6PkgZFEBMnEFs+qiQ5RW4C75QESi69msLReR2mc8hi5amulk4ZaLQ6dPlJPWobhS5/vWmBmtyg8hBEbNRQ9V1aeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OE5AjSKRmuU6Rd5zudSUyT6TNtRUji94cy+vQKppN38=; b=omNqsIqZWRuVZ4QN0EN+3pcklZxWn1oHkmzPYOmlYxdAbJYUMiHI02v0MJ8cAeBarwK2LkC01mAc82Gnfvv8aG6znwakMSWLzOylxT39jgA1dSnaSjG1f4PJd/S82FgBlgLwZuprlGJ7PLKxbLE+R6rI70QlJ69Ng3VHpsG9NtMdo7eFS4BI8dPpiL44UOUpzOxGZslxZgm1EaOgT4Z9KBkxA+cBWtNC47eKtAHmDpWNMBy/PxMgnM64AgSKc/A9R3ISlUihH34cP+XiIbPafz2Eva679kr/LZrPlM/8sAXZkJQtjQdTFzzBkC6Ie6PbP83bhj0A1Ex7YZ4tkJwwpQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OE5AjSKRmuU6Rd5zudSUyT6TNtRUji94cy+vQKppN38=; b=jiz1glDAiJppLPmO19LYEzRDIACo4xrlZ3v2ggEYOxgO18VgYEOATThsk2WLZ1YGj8k/+H00GeChByjXk1j4901G1n8qkRW8ZLyWayqD4vZtAsMZn04ZMeF6exy+wMtJnGYHMW0HgUoH0HyBuygHX6Ibiyc6ZrX3eVKRbHE2DpiOS4vYorJtCM4Dz0nNTvAfTZMAjvbgI7fU8PzWGGNqtbPxoNGRrzN8nP3tvWeBpqRw/yfJmJiE4Lo4SnqdhEvAM0G+UvhbPNbs+MDrL5V6UtREiqRDuGl63MhfymNxprFFlN+TF1+K2I91L1eiWW4BL1EGZ980zChZ7xwLxQuvcg== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by BL3PR12MB6449.namprd12.prod.outlook.com (2603:10b6:208:3b8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.18; Sat, 7 Dec 2024 17:16:38 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%3]) with mapi id 15.20.8230.016; Sat, 7 Dec 2024 17:16:38 +0000 From: Zi Yan To: linux-mm@kvack.org, Andrew Morton , Geert Uytterhoeven , Mathieu Desnoyers Cc: Vlastimil Babka , David Hildenbrand , "Matthew Wilcox (Oracle)" , Miaohe Lin , Kefeng Wang , John Hubbard , "Huang, Ying" , Ryan Roberts , Alexander Potapenko , Kees Cook , Vineet Gupta , linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, Zi Yan , Geert Uytterhoeven Subject: [PATCH v4 2/2] mm: use clear_user_(high)page() for arch with special user folio handling Date: Sat, 7 Dec 2024 12:16:31 -0500 Message-ID: <20241207171631.2853336-2-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241207171631.2853336-1-ziy@nvidia.com> References: <20241207171631.2853336-1-ziy@nvidia.com> X-ClientProxiedBy: BN1PR14CA0001.namprd14.prod.outlook.com (2603:10b6:408:e3::6) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|BL3PR12MB6449:EE_ X-MS-Office365-Filtering-Correlation-Id: a6720ab1-6ec6-4de6-1bbf-08dd16e2e8ba X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: /Y2XFwm26cw2QCdK2B0hx1zRuQ5iStogSx0h+O8oEj0+X/aTB/mmGfscQhquCja7rJvp4CoScdX35deP7ErRnWKcYurx2onJ66gfA+TDyFKyxwDaUxk5QjvSqrJT+zQPDzORlvIjzNk24BpyFcg29r6KBL4NPMg9Z1q9xbDKfvhak12bxee6bRZcLm941iGWUOkM7YWj+RwxXR5KsN2UpU5AysKkD5nkrFYPU6KLPJiAcKGEBvI26bKmnQNJdS9/kq4ZkHcg/Did8BpY6inWINbHWfhx1ICnJwRnt/Opc3ctCaeMAZwU0KWMu4YofYPLTpCaW/TL+UueuTv1ecMsvh/W+1H7kyCPvpWy3KBWX7khJBvpCgRTePFLJZE0I6rOmPE8Pbek4k0+SwHn/4fyPo2dOedkZMB2YTcuHiLf9h3TRRp5uZkqxa/jDbFBP0AxRNtQIQlhymy7WOsGY/TIyxqUxWQ2lo3so3i27IPDPr0Ipqrmx1PtfkGqTnB9czlmfrL3kZhT+IOHYH7YeKTdY2fCP4RCK36+vxPhfzvgU1tj7f+IObnYGaDHuob/5bZFrFG9CO074wYH+yIiLVgGNnu5a+Ug+/MM4tTX4Wb59985mgrZfeqpRFNm7tSV2K0Can1b0mpXfGLoYOeKOS9r6x4MdoUnUBLV4Im2o5FbHb90l/XTOu50haAFG8gGFozoeG4kcnSbNrEJaZFwMgX7FG/1Qg/mqGUUbTWwvIKsGKBgwWWhm9GTiE/320Q/p80caRURFJEXAX7sXeQJk9KYmWN6SOeFJR0PTHuPE65Y3H9FDESL02lMukNYcIVxT3SSlb3c1gV+s0qZlUzO5IGWcM7/BN4oqYejXB+Rk1AuNT/CRkABOA0hfJ3aKRziahu0bR/sQKBoyGvvDVQ4QooCJeFlH5rAIRlWOz/5rWDuPXOAcgvLKcQ+B6pWq9HvIVxeQ8+e6eqpvtHcpoSJWvAXzqMwh/rYKP4oyScXe1hIFGqLnGyVSin/9me6TyQfeyMGdgRvczRLScPB/1TxjiOOqBs7hOvpTpy0VQb3Ius8ymGGhFapS+2Ij367k4l2d8ROh76ACQ/+7ZZ94/7GYJ+aROqYh1Fmll1MPk547n/glbOTUllx5aXCTE91Tpr1Frc/9KamjUiH12eV2CxD8z0Vu02Fn3EA5YJlRQpod815nsjJUSmkcvRIPol1zRQvvw2dPCz7HpaePH0xRXF5fcNsW1EmYAMoabLCiMusLY475Q+/Bsy3hXM36j4JoEO5GebpKFmL70yE9Zzs7dEc728Lu4Du81s4L03yVFJ+jYHgH888pr/mUci09pbCwR+WV6FOgsVItDpvcs40g4PnTQhOv6lnY2fzl0YLXswO30FLTzw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: uLWnkDnmofnhgIeLGaNsC4QTWKQU7U0amFvuILfg1qoGjJq1IW+74rm0HnPkYsyLpjuHQokpMC2E7ekKWiO8Lp5YfjUH/p6fyEE7XtXEb7QzqKvmhcB33jzkxsNQiL28N9hOjEJrRbNh7Kyc+Yb5eV9c4U+DUasK5qZMG+CbgH8+3Nv3GERnVDsZ9LxTbD9mv6dj8cP8yhJ35/tEGVXnXKs8BBSpjVWscNut1T+UKZ7eH+17P+RZCDxaDsxMK9e9XBJ+mm/uJJBmZDRXhJPwgPrCerDkR5stianevzkKTdyGkGqbbTB7CgVf5dbLQOZ3j1nC4RXMjwK59LHyy7w4Nva2TFBFHFVmP2sKn5R5bNg6xTTg0EfaBKkPv0kSj1N1TOVdXR2uLnDnk59M+rA05mqFwU+Zo209gG5ad3blUiWRhm9aFjRUtF4wWJEe7Zcrj+fqxLH+qyVtWK8Jle7s26fLUDJoOq8yIeup0TOqGoEWLZ6wHFakWadAxLI4IlCsSYI/Rt+Ex4XRYpeRWE3PQU85mctAFYP1GHLHXDpy08cI0kUx/wU0dAPSBejEBM3Ef8qZqRMV2E8FMlVgwe8wxMxnvPASKzIicOY3AzLS4T5qj77skfUc+uygmncFnW4EWF+/gQlHEh8KJ6x56xIwmjq0CXgzmN1cGxjI3D2xScoZDelcIQ1elQlYTpVY5OHL6MSokOSn1fHPD0q5qrjCHlY2gMmfzTt3u2IQml799PHG7uEezOPfC8h+Eyhq0lTJ2y1lkE0hl4eWNQgbIjY/bq0cJpK5GiZn00fAgLYnRaOZZ2u4U55Cj7wg0ETbMOvHPoAwrncVbsYkpYAFq+A81ZHb/ssUhm6rSe58WiDhZNGXgRIE/7a6WHGSnprYOKuoEXv5guSF9MpKbG2POoDd7Xr9n9nsgcls5h5vVViJuflc/G7Bs867EL+G0stfakv6z1xGXuEMVPCiUEfjkG8i7o62RSS8YdMsozmu5p3tkaTsUuEnnoB9mxQh+mOuEhcB4CmVRsjuOCGRb+5GdBt9wMVSCp2Nthcr0MIO38PP1+Mj9W/QFLMqJ5yb3Qt1fLCcmuFvm2cK/JNTsqmUtxf5NCPV/iZpRQQKI23wEslf1kFIG1JRkXBXBQ1Uzgwo4KWhly1IECLcfQOC2Y8/j+MfrZL7UA3MS1vVIfwvHnmCZ8jshs+fbdGKL/Y32XHVdXMSdBOYumrjDfWkzQm2rbZfFIyIMAWPBt2p6FtjO8mNKQWRxC0LhEAArPUWbMtTB3WHnu7Dt0GN6IYHkDemQvE13AfPZQsqqOfBE6xIuIGwzbti4JiSh/2TxQfRlRYsgkxx6ckTHeLIVuOc2ksnkLjcp8u9HB1gS1aqyonZnQzuRa0DavbVoVxuNUOKX76xjIcVY996JHHLjhsUbMAHWoQjz0oq0+VE8rswxQwM96KTP2odmFGT+AVQlUEc1MjfeH7QcUcWbxsXzBCO7JbgWL/yM3IHiYbu/WcW5BBiYwSWuijIbxaSjjdhkM1oVvLWCmHlgeKlJCojFDur7IxkWn+LZRfPT66+cHMqAF1SlCOomLw= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a6720ab1-6ec6-4de6-1bbf-08dd16e2e8ba X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Dec 2024 17:16:38.0532 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ox5ibJ/iTdeXkxOjKZ09C0Rfw9WAURpn4E/GHUTKXZq3iCzr1JhH6CB5Ant5tlrc X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6449 X-Rspamd-Queue-Id: 94D1C160004 X-Rspamd-Server: rspam12 X-Stat-Signature: awjzusm861f79nondwhe54zpxzdjdpt6 X-Rspam-User: X-HE-Tag: 1733591802-606493 X-HE-Meta: U2FsdGVkX18S2d1rKfjwecHfRDwrPvM2qGIaAumqq/dan/n0+xKyMICpFUAhvg3d8ndTJLVE8NUac+6d1sCtpdh51kXNPQnAEkU+qhzHbKZfXenwld4keEbhGWVd1pLCEEgS68SdU9qtH29b3CU18XwzQNLBi0UCDv5OHpCMc9y2lFqaZBBIzYbh5HL0cTHHcnWKFrb6OJ0mq2BSXFZL+W/zBED+jLQhedukR8yeZfp/EJmoXaN9aM1zWJYgJtczWhTayJ7uPJvig/A2fIqm96oQqKDopUGOI9yHTk/HiEDU2AZjns+ZxU5aqxkpzdwqPjLsukIi9SK/+Gu6hAs9xG/gWxlzS/nGiQ/EJ3kPCj51cJoctfy42y3Y4/kB2fyKtfRpSs+nK7+TUYfo98xuHlQzjdqaQgEXXH/jmowXHNd+nyWvX/rKsGmIrYFv/ZuvXXNknu+gywFx7ctqx29zm2divkXtmccKoPbXio1ts2uS0U0mPzyXEDS2AfZBohempTx+TQXFMEgt+hPBny402WDg6G5AeZLoxd5WZmKPI0UdiJ7NhVuo/Gu66kqIL7cLi6aeaMxQA3t/qPrxsv2vIZibh+/uYWchdStSei8l1wEfu+AJYcwzWBRNigiJfOBIu0lnRCjkYpZ7TMTbjSwoHSqELIgCHXspkLCgJ4vRSIOXqKbeaFt5zIOzhE7h0AanhgYpILCU2eblzxKb9NTBdKVXtHAup2IpxPsXDg4W98kWpcAHrqQir91RpxwWjwdHJYhROd/dinASuEeIO2yGRzmGsnQjIuGbtazvePSTBEFA8MKPgqfMsleAEd0qxgnGMF5mVS7FLijRNVosxyfsywynbCZV8YnZhRVrtP4H5cHUbt9BiXq/SdounPq68evsriWnagu5ulVIl+a5xE9hSxim4v41IG5e6JzCECsCqLRFFYyh4uBQY0/+eEepb8Cs9axn73W/aWtRs04p+lY 5LwbXyXK qPIvXvg7zQ9ru8J7Szu2a4KL+p/QtNQS/WB9flcoNQD26l9EP69V3OaTaLN24Gqb1MOMbzWbHKdrCtptZ8Y0g2+sDy1x7eW5XyQinuMI2bJVd0otTudAnak6kYXthKrGiYqtOy+tdDbtTTqznyWMqbEAorUuftbPOnctGxEbyJ4tajZyIsuitNkeHEV+HB9B/txjayuYJPBgteHkOd74Vnx806EyWgIbKuSVUoe+7z1Iu04hUevKRm0TJHEDl3nVCbd8p92+KzZOKZDeDbPvEObuuVcCBE0mNjpsg32JgO2VyJ1u5UWeE6kH+PXEmLi+SEqGNqun+led+dY8zca/W8YQP8VYLnRk2E7/3IbxGPE+xS+Tn25/yefNmf0VXamqcSEwrLoo6Nl8uTFULHo9XWlN+IdeKfyszisn8Kk+r1YYLPE/sU8JurigicXU7yx/U2TsBz82YLqQ0oTbp38+jk8L+qI3RyXB6RHWf6hwqDgteTNGdtvok0NVfneBzTiMO8x/vogP42SVcE0U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some architectures have special handling after clearing user folios: architectures, which set cpu_dcache_is_aliasing() to true, require flushing dcache; arc, which sets cpu_icache_is_aliasing() to true, changes folio->flags to make icache coherent to dcache. So __GFP_ZERO using only clear_page() is not enough to zero user folios and clear_user_(high)page() must be used. Otherwise, user data will be corrupted. Fix it by always clearing user folios with clear_user_(high)page() when cpu_dcache_is_aliasing() is true or cpu_icache_is_aliasing() is true. Rename alloc_zeroed() to alloc_need_zeroing() and invert the logic to clarify its intend. Fixes: 5708d96da20b ("mm: avoid zeroing user movable page twice with init_on_alloc=1") Reported-by: Geert Uytterhoeven Closes: https://lore.kernel.org/linux-mm/CAMuHMdV1hRp_NtR5YnJo=HsfgKQeH91J537Gh4gKk3PFZhSkbA@mail.gmail.com/ Tested-by: Geert Uytterhoeven Signed-off-by: Zi Yan Acked-by: Vlastimil Babka --- include/linux/highmem.h | 8 +++++++- include/linux/mm.h | 18 ++++++++++++++++++ mm/huge_memory.c | 9 +++++---- mm/internal.h | 6 ------ mm/memory.c | 10 +++++----- 5 files changed, 35 insertions(+), 16 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index 6e452bd8e7e3..d9beb8371daa 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -224,7 +224,13 @@ static inline struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr) { - return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr); + struct folio *folio; + + folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vaddr); + if (folio && alloc_need_zeroing()) + clear_user_highpage(&folio->page, vaddr); + + return folio; } #endif diff --git a/include/linux/mm.h b/include/linux/mm.h index c39c4945946c..9330b60b926f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -31,6 +31,7 @@ #include #include #include +#include struct mempolicy; struct anon_vma; @@ -4175,6 +4176,23 @@ static inline int do_mseal(unsigned long start, size_t len_in, unsigned long fla } #endif +/* + * alloc_need_zeroing checks if a user folio from page allocator needs to be + * zeroed or not. + */ +static inline bool alloc_need_zeroing(void) +{ + /* + * for user folios, arch with cache aliasing requires cache flush and + * arc changes folio->flags to make icache coherent with dcache, so + * always return false to make caller use + * clear_user_page()/clear_user_highpage(). + */ + return cpu_dcache_is_aliasing() || cpu_icache_is_aliasing() || + !static_branch_maybe(CONFIG_INIT_ON_ALLOC_DEFAULT_ON, + &init_on_alloc); +} + int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *status); int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status); int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index ee335d96fc39..107130a5413a 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1176,11 +1176,12 @@ static struct folio *vma_alloc_anon_folio_pmd(struct vm_area_struct *vma, folio_throttle_swaprate(folio, gfp); /* - * When a folio is not zeroed during allocation (__GFP_ZERO not used), - * folio_zero_user() is used to make sure that the page corresponding - * to the faulting address will be hot in the cache after zeroing. + * When a folio is not zeroed during allocation (__GFP_ZERO not used) + * or user folios require special handling, folio_zero_user() is used to + * make sure that the page corresponding to the faulting address will be + * hot in the cache after zeroing. */ - if (!alloc_zeroed()) + if (alloc_need_zeroing()) folio_zero_user(folio, addr); /* * The memory barrier inside __folio_mark_uptodate makes sure that diff --git a/mm/internal.h b/mm/internal.h index cb8d8e8e3ffa..3bd08bafad04 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1285,12 +1285,6 @@ void touch_pud(struct vm_area_struct *vma, unsigned long addr, void touch_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t *pmd, bool write); -static inline bool alloc_zeroed(void) -{ - return static_branch_maybe(CONFIG_INIT_ON_ALLOC_DEFAULT_ON, - &init_on_alloc); -} - /* * Parses a string with mem suffixes into its order. Useful to parse kernel * parameters. diff --git a/mm/memory.c b/mm/memory.c index 75c2dfd04f72..cf1611791856 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4733,12 +4733,12 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf) folio_throttle_swaprate(folio, gfp); /* * When a folio is not zeroed during allocation - * (__GFP_ZERO not used), folio_zero_user() is used - * to make sure that the page corresponding to the - * faulting address will be hot in the cache after - * zeroing. + * (__GFP_ZERO not used) or user folios require special + * handling, folio_zero_user() is used to make sure + * that the page corresponding to the faulting address + * will be hot in the cache after zeroing. */ - if (!alloc_zeroed()) + if (alloc_need_zeroing()) folio_zero_user(folio, vmf->address); return folio; }