From patchwork Mon Aug 12 17:55:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Lameter via B4 Relay X-Patchwork-Id: 13760900 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FBC4C52D7C for ; Mon, 12 Aug 2024 17:55:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A7B866B008C; Mon, 12 Aug 2024 13:55:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A28866B0098; Mon, 12 Aug 2024 13:55:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 916D36B009A; Mon, 12 Aug 2024 13:55:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 72F196B008C for ; Mon, 12 Aug 2024 13:55:20 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 16F3D1A0316 for ; Mon, 12 Aug 2024 17:55:20 +0000 (UTC) X-FDA: 82444345200.10.B38D6E6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id 13FE9120009 for ; Mon, 12 Aug 2024 17:55:17 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WPUhlG75; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf29.hostedemail.com: domain of devnull+cl.gentwo.org@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=devnull+cl.gentwo.org@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723485240; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=HkxX50kwfImJJEgGsM8nWeJV/AoUeQivDACN/J6zQwI=; b=68eLFb3YpZztXuWhs3uo+hGWjldfKOrvYBzYu9to3QfA9oiwLc0HYPilsiSz6LPibTdxG9 KXxckqvbnAwY9Ihj7oGI7GWtVCCLMaSTI/3BiYMElkmEzEMNUPuiRS4MqOTLB3Q53G7IrO NfQJS5BYkDrtn0dZaiI5kDm8JTg0SZ8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723485240; a=rsa-sha256; cv=none; b=TqeI9bCRb0YRWwxY2LH/q9h0b1Szj/HIkqkQkedt9/ECs/neZUKJ/TwVrE/cPmbI/wC0yS TBzoE69Bi8eM7dsex6+/f+mRS0WeXUfPk+18n2PmrLi4Pbgy/XKD7iBVMoWZuYVLCSGkT5 +mHLInVT+hgKUTTGhduw2TodKDQzTVQ= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WPUhlG75; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf29.hostedemail.com: domain of devnull+cl.gentwo.org@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=devnull+cl.gentwo.org@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 0AC0F61369; Mon, 12 Aug 2024 17:55:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPS id 54836C4AF1B; Mon, 12 Aug 2024 17:55:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723485316; bh=Ay8VaO4Jv/lXv+JmnPgO0eWEOIHNdWaHckuTwRmJZk8=; h=From:Date:Subject:To:Cc:Reply-To:From; b=WPUhlG756UwmloX/ZNx6WVIG3Gz7KgFAkSwPL8PIfLsv/XnJsPd7S8xowSMYh4luW lcc0KfrTiC3kgPzN9OZTvsdapHW2pemGRt/l3F9OEGSqs9JTN9EArcYPgxJ9GSX0iE N74gb4dfbZnK45nPSxPGpri2dG8Vw3FQ12MjL7ZyYCI0X2xtlIs0HY1AAKrzTWgwFU WrSuXSuLig0hU9sSwDZRHf+gZP42P3CJ8QNcSmZHttnv0CYANF0YMeNdhslCzQEfiJ doG3CBqubwgL0dNsX8tnsgfU1qc+BM/mjUzL/s5fXptgpU41qFJcIsa5TuSfxodOXy tA/Mz29d0mPCg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4871AC52D7C; Mon, 12 Aug 2024 17:55:16 +0000 (UTC) From: Christoph Lameter via B4 Relay Date: Mon, 12 Aug 2024 10:55:06 -0700 Subject: [PATCH] Reenable NUMA policy support in the slab allocator MIME-Version: 1.0 Message-Id: <20240812-numa_policy-v1-1-1732602b976c@gentwo.org> X-B4-Tracking: v=1; b=H4sIAHlMumYC/6tWKk4tykwtVrJSqFYqSi3LLM7MzwNyDHUUlJIzE vPSU3UzU4B8JSMDIxMDCwMz3bzS3MT4gvyczORKXVNDC4s0E5OkRIMUCyWgjoKi1LTMCrBp0bG 1tQCMAoOGXQAAAA== To: "Matthew Wilcox (Oracle)" , Johannes Weiner , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Lameter X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=ed25519-sha256; t=1723485315; l=2030; i=cl@gentwo.org; s=20240811; h=from:subject:message-id; bh=oIsOUj+s52JxQUrLe0TMZXCQFfAgTyXqoXTDzKem1MA=; b=Z286VwSdQVPVNRMc582ozWP7NFSxGLmQwBaCtj9kzM80sRwA6XOkZDFrPl4wKma09vPviLxBh p5zaM/WaXQaCAi00LCfffQoEJ69rjA3c7EcHqnOIkayhvpRYMQaRcRQ X-Developer-Key: i=cl@gentwo.org; a=ed25519; pk=I7gqGwDi9drzCReFIuf2k9de1FI1BGibsshXI0DIvq8= X-Endpoint-Received: by B4 Relay for cl@gentwo.org/20240811 with auth_id=194 X-Original-From: Christoph Lameter Reply-To: cl@gentwo.org X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 13FE9120009 X-Stat-Signature: u9wbyq69345qopgirfeq9noee7bexxiw X-Rspam-User: X-HE-Tag: 1723485317-273395 X-HE-Meta: U2FsdGVkX1+8ojo5CwgqM3ZQc36/BQ5i51i0OAONCyPmPbtvbVkV10Azhd9tXHeiGDCa9tCTdzIv44hIntzr4/bqi3HWEwW93rwENB8pbQgDuz408UArQkj9euEQhqvqF/xxC23PvicPo6AlXXEI7UhbRCZoMZ7EjZ78yMaammyFD/M3GBbJvzyo2wZWLUtecJ6rc2M4JLEydDYMXtKJxk+Ct05LqWzvhtOiMhxYsnAryulMEWaUcQhr89zJesrenzjBuLSFb1qdDOTvkqf5R62tfgW5Dk3Ptf0wA/Aykfl06c7mz3a0hwFZirX1cCHHMGI0q+QSuZaTNV4BULnTp4YG2Q3HVZZmEvqJ9haeEKXiK3JU0XUcpQ3FAa+7EyGH1iZUNPf/shVR3e4TMt6trfIYk/o/uie0SXBjCyXkM4/bh6wzlqSTqlu3czdB16KmexBUZLvI/TUTKIKrrnD5ayx/dyLd2JSFzY0Y0FBYu/lO1lfxfCwhZe4wYx+4VkFJtG9h1bNGJlTMdU/fqXt3dRYUtaZMLNQ/LuWJno6GEeJ4VWWHNAnAwHp2+O4HRY5Qdw6fgyEtYAOv8Q/zU3U89fTjZdhHse769v9h0J6UeMVHVJgiYAE3RPjNdZbglOAQtvbgSdrNqXajHsdMZpZ9NChtgauwVa1yXi3Ahh/O+wSUmpYGwtRrdzZTiL6nY9iJF7OdjKTMuJRzQiicLv+VBZ8Sgxrjp4vwy0qRPebM9tYt5b/+oJE+INF9Vo5cjKsDMvQC6ETEUgF2q5M2yJ4OABrE5ALU/Y4Hsg/QkC6zPPZPCwCeuZ5ugSt3Mg95M/2d8I97g4sx5xG+Dfs1oes2sV2WI3VFFQeX3nnc35rYL1viPIxg861jRpJf8dziZQM6tJqteffQqikLCntiC2wvPC03pRnrpDYglJpdq81n/j/XYchGFyhAYpT52Th7TR6sgjeVgjkd5GmpMsyGAxu S7CQBLKy svgoz14hhQuEgQr0ut/SeFKuK453zRNBXrtHUIOrI6RrAzQfSbrLH5MIUHaOdZhJiSPXELN7wQmUuEMhS6nYniArcK+5SEZKeAwwXZg4qCqGt9HiwdH8F8d/xCzM1Z5dhO9RWLapmsoTh3Ijieq/Od0l8cwzXugmVjdNu9Wp0l26KDpeJqkmGdloxQHgy2fyca4cc5YP4t4mXtryq7ItSIm4RZCwjOUWyQvO8yj3LItdRYik046cD4e00VoaHJZHTRG3UuseIEGowgAz8A3Qn4gBAEQTbJZGV2uENMttItqOFRMmqEUwHbp+LNQ2zsLV1eGmIl+ttWqAIiZOeD9J3Iqj1LplUSHIP6gTYB2031goc5zm7Lab4JCFS9inX1No4tMYf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Christoph Lameter Revert commit 8014c46ad991f05b15ffbc0c6ae130bdf911187b ("slub: use alloc_pages_node() in alloc_slab_page()"). The patch disabled the numa policy support in the slab allocator. It did not consider that alloc_pages() uses memory policies but alloc_pages_node() does not. As a result of this patch slab memory allocations are no longer spread via interleave policy across all available NUMA nodes on bootup. Instead all slab memory is allocated close to the boot processor. This leads to an imbalance of memory accesses on NUMA systems. Also applications using MPOL_INTERLEAVE as a memory policy will no longer spread slab allocations over all nodes in the interleave set but allocate memory locally. This may also result in unbalanced allocations on a single node if f.e. a certain process does the memory allocation on behalf of all the other processes. SLUB does not apply memory policies to individual object allocations. However, it relies on the page allocators support of memory policies through alloc_pages() to do the NUMA memory allocations on a per folio or page level. SLUB also applies memory policies when retrieving partial allocated slab pages from the partial list. Signed-off-by: Christoph Lameter --- mm/slub.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) --- base-commit: d74da846046aeec9333e802f5918bd3261fb5509 change-id: 20240806-numa_policy-5188f44ba0d8 Best regards, diff --git a/mm/slub.c b/mm/slub.c index c9d8a2497fd6..4dea3c7df5ad 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2318,7 +2318,11 @@ static inline struct slab *alloc_slab_page(gfp_t flags, int node, struct slab *slab; unsigned int order = oo_order(oo); - folio = (struct folio *)alloc_pages_node(node, flags, order); + if (node == NUMA_NO_NODE) + folio = (struct folio *)alloc_pages(flags, order); + else + folio = (struct folio *)__alloc_pages_node(node, flags, order); + if (!folio) return NULL;