From patchwork Tue Mar 5 10:10:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 13581908 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A52BCC54E4A for ; Tue, 5 Mar 2024 10:10:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D17CC6B008A; Tue, 5 Mar 2024 05:10:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CD2DC6B008C; Tue, 5 Mar 2024 05:10:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A55396B0092; Tue, 5 Mar 2024 05:10:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 811856B008A for ; Tue, 5 Mar 2024 05:10:32 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 49167C0E05 for ; Tue, 5 Mar 2024 10:10:32 +0000 (UTC) X-FDA: 81862565904.03.4695BFA Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf09.hostedemail.com (Postfix) with ESMTP id 635F514000A for ; Tue, 5 Mar 2024 10:10:30 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=FfYsyFIw; spf=pass (imf09.hostedemail.com: domain of keescook@chromium.org designates 209.85.214.175 as permitted sender) smtp.mailfrom=keescook@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709633430; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=99J+yh1I+zD4qoBsjvOLBvxyDWRgmU0/2arb8UvstmI=; b=CFCwB5IWHQfPeKBejzYN3OEXWUYbUKCe9C6TgTFH2NXaSDtfP4mTpjnQi9TOgDd2fL/ndS p4ycgpi1YfzGh8VCIcZkN402xpc6c1TysBA9cQ/tRKDM9ugHr3wHcvBiDq8QnrdNi+HP6G 9MlPoH5CczzOpaFFdyngxScuG7Nbpqw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709633430; a=rsa-sha256; cv=none; b=kkLk7Z1TTMTYollEar2wo+Ai9Jbn/E+NNA0krr+maZSOhGvPvKcnghAwZ+TmxI+NM/HL+8 4wrJohMYiskrfJUw1u7MWF2TtmXApitJXbG7Bhs6zDjdn7txOfza02FKjrwK+xr8MvO1nN fiiMr1vVK/rczJUZfk8o0WRUSAarUfQ= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=FfYsyFIw; spf=pass (imf09.hostedemail.com: domain of keescook@chromium.org designates 209.85.214.175 as permitted sender) smtp.mailfrom=keescook@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1dd10ae77d8so15610095ad.0 for ; Tue, 05 Mar 2024 02:10:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1709633429; x=1710238229; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=99J+yh1I+zD4qoBsjvOLBvxyDWRgmU0/2arb8UvstmI=; b=FfYsyFIwaGOeQ6KWQkIEuOC6mluRFOfKAcny9Bh87WSNtIHd2P7nvzUy0sly1+JHx8 jJOizy7FD71AgRqJVtg2oPyB53RVLchrzSD5+75DvCXoBsRQ9S5p/TUVhzpiQqafawI6 aMynzY7zE9xVYlqFUeXAOQIPq9mPnJtD8rNho= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709633429; x=1710238229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=99J+yh1I+zD4qoBsjvOLBvxyDWRgmU0/2arb8UvstmI=; b=EOZNjhUI23Xm58pooTwYpOyAN8260kloV3GEufzMaY5nEPRcCASf4RnROC/LKXalri 6tUnpfFn3J3T19Dd1tQXNQKZvz4QVgM3qvk0+k44zzAEB/hbyGN6v7F/+9HAY/lFENPv MmnpEw2I34OUIUWJseGSYNsrtqu41VuDE2+Zj6Zt3nLQi9UtC4jDmp3KL1AoRBcNzlu+ 1cHJREaycofd8/YGF2WzYRg2mcQ8PyHcRlPctmr0cSWAGX0W1BNQAzbuhe9Ez+p5F2Pj SDFC9kwgfxzo0oLT0Rnxa3CfRjdShY1o/xOUPdR2mWNBgVQYf57OG6IMe8+wqMaltp/n YBGQ== X-Forwarded-Encrypted: i=1; AJvYcCVWtlNReJopDJQ5HdUNMYPvLTgL0QVrdetX/gJUfh7gEdWRIZRPSU4i5cFjZ/klQ67ObMXpikpnaIb01IUEz4f+HoQ= X-Gm-Message-State: AOJu0YyVfSg4XYbH7uBgx8Jih9SUcAVeIJIsSlrBEzECxOaXLeC/42Nq jS0nkoZxabzqKBsz6scsfJMkGdw4w6uIMA+8zfM/cpAZ+sythmlmq2Pp2I+N5A== X-Google-Smtp-Source: AGHT+IFqjHRULQpfLuOIfWcrg86BprEdjHxZ7A04s1os533fRHV67h+yavqSGXCERg/CRb9UyIa8KQ== X-Received: by 2002:a17:902:d4c1:b0:1d8:ab27:d76c with SMTP id o1-20020a170902d4c100b001d8ab27d76cmr1643127plg.51.1709633429324; Tue, 05 Mar 2024 02:10:29 -0800 (PST) Received: from www.outflux.net ([198.0.35.241]) by smtp.gmail.com with ESMTPSA id n15-20020a170903110f00b001dcc3a46a6bsm10081558plh.262.2024.03.05.02.10.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Mar 2024 02:10:27 -0800 (PST) From: Kees Cook To: Vlastimil Babka Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, "GONG, Ruiqi" , Xiu Jianfeng , Suren Baghdasaryan , Kent Overstreet , Jann Horn , Matteo Rizzo , linux-kernel@vger.kernel.org, linux-hardening@vger.kernel.org Subject: [PATCH v2 3/9] util: Introduce __kvmalloc_node() that can take kmem_buckets argument Date: Tue, 5 Mar 2024 02:10:19 -0800 Message-Id: <20240305101026.694758-3-keescook@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240305100933.it.923-kees@kernel.org> References: <20240305100933.it.923-kees@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2996; i=keescook@chromium.org; h=from:subject; bh=JqAs2ONNE7kMO16unP+0GI5vQbPsq9Mj6KA7f6Xw/go=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBl5u+QU4fAZ6X0I2VVjtMMEATXAWJJ5dqn0j2T+ P8LqusMSw+JAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCZebvkAAKCRCJcvTf3G3A JmecD/92uSdARNPOF/Ig41R9iblLk9UxCbO1fB2HKnaa/w+yLHjuEWP2fptZr1u4cl7oBi+V+7e XCMsvWK4pkOYAP7NuRX8GU6AxmvUqlZpmQXa283pIL7cWbyoBryIVD0uVp7YjUDWMT3mZYo8kTI WOxJrg2BS22j8FjJ0SmfK28wsBeG/6y4f11iNleBFW5KpXlGaRw2NoFGPv/oYzmVJdpOvno7tv1 SrJhdM++M5F4fg8dy8UZdPFSBIjoz4VC49dlXRzdsM9GoSdV+aakrKnzGaX8WPqr1QTfDcy5qEv RCskJbWRYLonWFp9KcpeVrWeXB43I5oPU6H/DHx0PGqLxhEKdHMseOCtWkAKNPVhtqOufAQLdct 2XPAIyB+CIavIHRn8oZvNB8dhF46U1bKOM9jFISSARfnGEZp8WC4c4OStJpN9nCnwYHPz32sXy5 IFPtLolXYwV9mqfABdpmFQb/QG6+XuvjFWZ+HpxdxuAD/41pG2HrHPS3XlG77pqtqM+/vtWm5LH hf8/tXhzxjo8B8vey28JSnydpkHd63SeElyfCf/JQ/Vbe5wzsRt/oHddPw1OThs95gN/vEjmHXH rP+vV3/i9F7vKu273jzC4MIKRmYWSs3/Lc000Nj3NCnpIp6uaAeGZFpJIzs1h/qLp3rQGBKT8yw TPX8rWd+pV6qhAA== X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 X-Rspamd-Queue-Id: 635F514000A X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: igtkfth6tdj7a5ze46uh4kg4qzjfnkua X-HE-Tag: 1709633430-911072 X-HE-Meta: U2FsdGVkX1/G26VbJ0mEs1o0W7fAfHgHZ0MjNFkm4iJdv1nrsb+M60giQMRisgE8EDnSYFrHsNL6o8kLP7f6BYjV4q2CpGinElklzR2HOmn7M0X8jHNg+jovh2aW4K6HvlRCOEqVqh9RjF0rZD+a3qmcTxqsqgnCYzOEJZEOTd81qT/wHHIfGx26DdwMBAf1Urp1vgtejJGLUvGH+fl96bBUuePOnrkfNjV1oDdX+yX54GF7uBq+WLx3JsAgfAs7Cdd2OJcuhdZIX7Z5qgw2UIDOY31cpnWVM6OZQtzJ+X2nRrHfCI6yEbbgXbJtqdZbFVRKaFfZbqnLj6ZoZFpepLqK6eEcboayacb5cE8F76xW7LKbRd+Xl+ryULx7Ns6iN0vAqvtCrG+DMryRX2wrZqP/iOysLKlV7gdn1F+bmAjbBa5HvOA4/ukVOKXa8Ro/N1t/6+ra7YA/j3/T7R2SIGH2MUBmH1GyQNR5WKhhRiiOb8GL9IKcEAmCNZCk2G76JZkMaRaQbITpUaTVfZ3SuVsoCykuz+4eRAtO2n41cEiBChLQERSXH+dQtiF6dNkYPWcwzR8o0cZtJEqtKgAOubX/K0sbKQKQskEWWr0dFAujyxxVEiRZC+zjsR0XACjsE+BFt0NB6AFza3BJ0GmAz8Q/cRr+pjK6ya9z1A1FhahpW/LW2qGLjgt6vwCCZaC7WXfkzhUNHE8oaZM5J6mpiC7qIC/G6iS/aqkznKcfhAc+kFSUdoHrh8pe4L8AXDO+VfFjdlfqZXQdp3BT2lbu00VTDN/Yz6PNgfM3luq72BhL5cIYLTIQlhioI8D4LFkBGnQBgImt35boHyKg3iOsegklkfUTMAkWNFR6RCJBiV1JDGuQywt0THpRWpaequQxXeQxAkct7IM6hFTSVKNiVbs0tPG1VABQSsfNRl/OK2HVVfFHipGWsqyxn/0K5PFEzjsM7Blz2hP7/fCLNT/ cc+md4J7 6cnMKyDYRUqwp0bIEwuykwODa9b381C3RkOp7RAw5NrzUeJIhK1071jGbpNhpBer7gwa+dTyIeRaHa4uk6CnRJHCD5Qk7e0xTmjz3MIT0/46sHTAgwaNbr0jOrlzIPU4jOJhLF286ec9KONlolY8RQXbx6ji12kKZ8tEMVo1t0+02hAYxejxNEDynUuK9zkjDFfcogwYfV+KGHu5SPaRKsR7/AAM6SOHIDPRxNsKwWkTFURg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Provide an API to perform kvmalloc-style allocations with a particular set of buckets. Signed-off-by: Kees Cook --- Cc: Vlastimil Babka Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim Cc: Andrew Morton Cc: Roman Gushchin Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: linux-mm@kvack.org --- include/linux/slab.h | 9 ++++++++- mm/util.c | 9 +++++---- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 1cc1a7637b56..f26ac9a6ef9f 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -723,7 +723,14 @@ static inline __alloc_size(1) void *kzalloc_node(size_t size, gfp_t flags, int n return kmalloc_node(size, flags | __GFP_ZERO, node); } -extern void *kvmalloc_node(size_t size, gfp_t flags, int node) __alloc_size(1); +void * __alloc_size(2) +__kvmalloc_node(kmem_buckets *b, size_t size, gfp_t flags, int node); + +static inline __alloc_size(1) void *kvmalloc_node(size_t size, gfp_t flags, int node) +{ + return __kvmalloc_node(NULL, size, flags, node); +} + static inline __alloc_size(1) void *kvmalloc(size_t size, gfp_t flags) { return kvmalloc_node(size, flags, NUMA_NO_NODE); diff --git a/mm/util.c b/mm/util.c index 5a6a9802583b..02c895b87a28 100644 --- a/mm/util.c +++ b/mm/util.c @@ -577,8 +577,9 @@ unsigned long vm_mmap(struct file *file, unsigned long addr, EXPORT_SYMBOL(vm_mmap); /** - * kvmalloc_node - attempt to allocate physically contiguous memory, but upon + * __kvmalloc_node - attempt to allocate physically contiguous memory, but upon * failure, fall back to non-contiguous (vmalloc) allocation. + * @b: which set of kmalloc buckets to allocate from. * @size: size of the request. * @flags: gfp mask for the allocation - must be compatible (superset) with GFP_KERNEL. * @node: numa node to allocate from @@ -592,7 +593,7 @@ EXPORT_SYMBOL(vm_mmap); * * Return: pointer to the allocated memory of %NULL in case of failure */ -void *kvmalloc_node(size_t size, gfp_t flags, int node) +void *__kvmalloc_node(kmem_buckets *b, size_t size, gfp_t flags, int node) { gfp_t kmalloc_flags = flags; void *ret; @@ -614,7 +615,7 @@ void *kvmalloc_node(size_t size, gfp_t flags, int node) kmalloc_flags &= ~__GFP_NOFAIL; } - ret = kmalloc_node(size, kmalloc_flags, node); + ret = __kmalloc_node(b, size, kmalloc_flags, node); /* * It doesn't really make sense to fallback to vmalloc for sub page @@ -643,7 +644,7 @@ void *kvmalloc_node(size_t size, gfp_t flags, int node) flags, PAGE_KERNEL, VM_ALLOW_HUGE_VMAP, node, __builtin_return_address(0)); } -EXPORT_SYMBOL(kvmalloc_node); +EXPORT_SYMBOL(__kvmalloc_node); /** * kvfree() - Free memory.