From patchwork Fri Jan 24 12:01:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 13949323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 377CBC02181 for ; Fri, 24 Jan 2025 12:02:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 96CFA280062; Fri, 24 Jan 2025 07:02:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 91C83280059; Fri, 24 Jan 2025 07:02:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E436280062; Fri, 24 Jan 2025 07:02:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 611FA280059 for ; Fri, 24 Jan 2025 07:02:14 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 08B4E120D6D for ; Fri, 24 Jan 2025 12:02:13 +0000 (UTC) X-FDA: 83042207388.04.ABD6DCE Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf03.hostedemail.com (Postfix) with ESMTP id 058BF20012 for ; Fri, 24 Jan 2025 12:02:11 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=zD2fGxD+; spf=pass (imf03.hostedemail.com: domain of 3QoGTZwUKCNU5CM5I7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--elver.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3QoGTZwUKCNU5CM5I7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--elver.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737720132; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=zD6StW0rYzOmQ2GqSmARPsItuuT3+0nfZNTObiJebWo=; b=5gVDlDpVthjE0dzoBV27tpAhNtuV5Zo3ozxv+R39G+3jY1uthsMBZEmKnhiIFQebiXhQ4j hGzd4I8vR8RzE0GlmOm6GzJaj00GwofzY+TZp819URK+m99/XUmomeP/tSu6BZaCiA0yfp 98GnpDyn+CRlohoysOIWFWFmVPZpAOQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=zD2fGxD+; spf=pass (imf03.hostedemail.com: domain of 3QoGTZwUKCNU5CM5I7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--elver.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3QoGTZwUKCNU5CM5I7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--elver.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737720132; a=rsa-sha256; cv=none; b=UeRIMsjUkKtxWkje5NeQoEEw6CtWLWWcb7Qj73MlgWlmIX0trJQojeY1E1avKw19Mo2AdD 2+CpS3L4wA7AebVacnG7kbPBejnWfOeEcUV1Fr+fo2OLbPWxyyeaYh0oqXAmqUbq2b33ky ohP1qQoEclcw9TUm2XhRtpGacaCCyxw= Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-5da0b47115aso2038300a12.1 for ; Fri, 24 Jan 2025 04:02:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1737720130; x=1738324930; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=zD6StW0rYzOmQ2GqSmARPsItuuT3+0nfZNTObiJebWo=; b=zD2fGxD+T13Evfi2eM7y2wGlxhIN1boaO8oOk1y9TzmQQGolnzuAVBiKu8gio2wFpV 1BKaJkdRljcdRxv0OCuKRiUnLp+n62FuHaPgyv45lkT+9O656vB2F17x/qLNUvAXYr0h 40vz5B2rc1jA8CGNE6fhboumJ9UdF1nyM1dlvgH0gXU8GJ7VxdqMSLsYF3ybipDVLpUn lM+oBnJ8Wb+GBfKZwJOrUmhA8jOkK40xAlhkcsWolHSxPVFcshxx2P4HL3+7XHFWaS8Y iNFKYPek/3mYUg9VU9+FXCLc2cAh4T2EhyNSi+KqhxpKAnuT7Eu1kO9FsoBAjwlW/oxd Ph4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737720130; x=1738324930; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=zD6StW0rYzOmQ2GqSmARPsItuuT3+0nfZNTObiJebWo=; b=rVTfOTebe7N2EzEA9cSCNkIsIfxTNkqQtK4X704PyqHVKrg5A6iFtH1GMTOTgtK7W7 CW6p63/BQDJDfVGP7zz6SXvQZUweDtTmX3QYl5YJKNMANPnUxmqd+ZfClao8+2mNuxwM ZmKVub57jhqIce/aK1UmBwC3mQm0fxKCpiY/uslMJakYbppqSfpG5Ub6/bo4AxdLRWYx Hk3/4A7A5rh8pK6yfuEdjPFwlCq11MDSPNNPPFDACCdVyLt0iVvdJSZ4wA0RIy6eXj+P 8tbz+hB2IAsid/ACNShlRuv6qkj7RJVBbfYOiq2GVASu0vJ5VnEfew9ria+pucSjhLRv rYFw== X-Forwarded-Encrypted: i=1; AJvYcCW9pdilVW0/WWzcnHVr2F+exXMv5dakP7AUEizfzP3TGKIDoNKrb0o0v/+PxZ+Wc0XjpCRNBSNWEA==@kvack.org X-Gm-Message-State: AOJu0YxJO9om7P0uqDFaazu4+dUoDua8rogP8HMhjbweRR7mFCqx5zGb mEvI0/r6VFtwDaKWMVlTgjxw/tdxTnk+RZsjOGsr/FOJEV1Pzu2A/8edj9y32KPLp1t/zkRWHw= = X-Google-Smtp-Source: AGHT+IGNFpOULcDrHD56b/yh05c68gjc+fDIs2J39MXi/+4aqEjglZpsvwZBUEMUnzpUaNwRYkyqzdJI9Q== X-Received: from edbel15.prod.google.com ([2002:a05:6402:360f:b0:5db:e8e8:da25]) (user=elver job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:2803:b0:5d2:723c:a568 with SMTP id 4fb4d7f45d1cf-5db7d2f105fmr25991316a12.10.1737720130343; Fri, 24 Jan 2025 04:02:10 -0800 (PST) Date: Fri, 24 Jan 2025 13:01:38 +0100 Mime-Version: 1.0 X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog Message-ID: <20250124120145.410066-1-elver@google.com> Subject: [PATCH] kfence: skip __GFP_THISNODE allocations on NUMA systems From: Marco Elver To: elver@google.com, Andrew Morton Cc: Alexander Potapenko , Dmitry Vyukov , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vlastimil Babka , Christoph Lameter X-Rspamd-Queue-Id: 058BF20012 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: jdk5nbqgteaq5acscgsted3b6zwk49s8 X-HE-Tag: 1737720131-86413 X-HE-Meta: U2FsdGVkX1+zzGPeFPIwRam1iUh30d/WuhEvKNR+YfVhe5StzAKnYzFPEVKCa0jDJeLj0V9wqPCJah/+HStHebSgDimaQrmsbc11tFASZqC8Dkp26UKih+49jMmr5D02SCsphmKvkOClzw2nf7vosk09EbnKgxoRm8Mc9x2E54SbCdD54aFOjw9vOUzwn9LTdCtMz8CpHQQI7KX0F5WW0RRAGFGlPklphSMU8Ti1mQ2j/Ot7++Rx/O5jvvO0qKO294mBZVytufG0mNzc9epH5Xp7n9fzCkV5AuHnt0ZMKJ1SkPjCvpB5t/Llwr9piyncvNh047WIrzVRhQ21klJDd4SfwXQfIx1A5zFvmGVzIvk8BnXqC90dXmZLEoo9Xrf7UzpfHHNBPfmO1mm/sLPGU6GekmGoDTETl5r7UGVCW1c8YKn9FcKG2I83CU61CsKrn7nXz+OPpr0p0zMz5n6TYWYNHgfgSp/fOmvICO+x+tFlDXqO+QdaW85SsYUDzAkIFhHxNdRsJXMQh0U+Sr3tLZt26tWuvyH0y7rm/k8BsK9yXe+DATyhSe0g9STMZFZ75s9xG9EwRNPuIsIasidVV5+GGxFcMs6X9ybkSATlbO934ojRErM4dPVtufgTSKuXBMB/a5Nls8yQH6qA0JGK1LjzEgz9+3jEVTGdnDBilWlTjrm0+q8KcxpUuFSwOisqeJOX3kmiw6N5VmAfoqadscSU75+mBRfPN8VR6VhB2gxujKn0O9cNLQwVGm/j2hjgOYt8upUDUWcqZvlNtdml6AOEhHrkBQsun9IqbC7mfW8oXoootR1tHgNoYBQgFuvEphhkdLh4VaQnGP9N1S7AhdvuBEhsE7YgISBCusFBcYxq/hcFpCaCfYlTDUARQBH7kR+B0A9GsnVgeIbvnQhScP1foa2A/XIGJH7gkDAV3X1hZED2wDaXH/hju8nlVwlmtLVzUfeTkiyTJwKrlY4 ZQELjhso vrSXf4yCjgEtjDZyw0N0nZ/kpN/yLEFcBLsGgJkWLEL2+7uXLtIsYh70DHEtIgIgdPV4Fu3jVsmyt28ANDVrfCR+giJyiw1w2sOsDXY7wi0hvKpfF3q3X3dEcfvT3w8Yay/AcfdnOZfisle2KK2bH9ZxDLdajB+/ixtfMIB3SiwZ6v36HPT/2JRCj1qTb6qkKa2ke1RS/XVL7ztc9iWg1I2hpMO4NyVu2/q2AU2lSX/ZO5M3R+b4urCBvckUHPKYFl/XzaAla6F80Xo58YC05zgf7T9UeNcGn9Sx2bWcQSIeldB6euf584foUXJgh61NJr2kVgtYNTb9HS2szcYD38URJNcmQikM0tulWSlpsQ6ukmSxk4CXhUU44lE8lMpWCqDKJzI2eAvTjnaV+RqcnUpnZeif+G0FggPmEUE3U7FcIJF0HVJQV36gaLE6KB83P2zv2d+Qsqw/kwNVAsPDxpSTNV8yBi0QppBrYhunM5PuKv3Z91lyCXglZiWzEKqk8SsXhrvBeP5w80zFuu4S/zKESWrjHJ8byazZjFWiLu1a/HaMe8nZw2vbyTGLaN0cUCQnsN3kMSl0ej8BxXOiuvTXoxMfTYjyeQkn+22YTE/2I9Xx0qxQwE8qK7YYlr06ONEus X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On NUMA systems, __GFP_THISNODE indicates that an allocation _must_ be on a particular node, and failure to allocate on the desired node will result in a failed allocation. Skip __GFP_THISNODE allocations if we are running on a NUMA system, since KFENCE can't guarantee which node its pool pages are allocated on. Reported-by: Vlastimil Babka Cc: Christoph Lameter Fixes: 236e9f153852 ("kfence: skip all GFP_ZONEMASK allocations") Signed-off-by: Marco Elver Acked-by: Vlastimil Babka --- mm/kfence/core.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 67fc321db79b..102048821c22 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -1084,6 +1085,7 @@ void *__kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags) * properties (e.g. reside in DMAable memory). */ if ((flags & GFP_ZONEMASK) || + ((flags & __GFP_THISNODE) && num_online_nodes() > 1) || (s->flags & (SLAB_CACHE_DMA | SLAB_CACHE_DMA32))) { atomic_long_inc(&counters[KFENCE_COUNTER_SKIP_INCOMPAT]); return NULL;