From patchwork Mon Jul 1 19:13:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 13718561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF9FDC2BD09 for ; Mon, 1 Jul 2024 19:13:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3471F6B009E; Mon, 1 Jul 2024 15:13:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 297056B00A3; Mon, 1 Jul 2024 15:13:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D4E116B009A; Mon, 1 Jul 2024 15:13:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 79E266B009E for ; Mon, 1 Jul 2024 15:13:08 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 31EE481AE5 for ; Mon, 1 Jul 2024 19:13:08 +0000 (UTC) X-FDA: 82292131656.09.9C5590D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf26.hostedemail.com (Postfix) with ESMTP id 75297140020 for ; Mon, 1 Jul 2024 19:13:06 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mMao9yoX; spf=pass (imf26.hostedemail.com: domain of kees@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=kees@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719861175; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=siYgYiAhCvUSPA8SJhrIVBJeroAt8eFvCxrlbbpf4yM=; b=oGk+pUwLuuBXWVGk+Fsptthx792ZCL2O9d2ZrrcDDh/AEHG8gLuz7CHqPDXiGdhQVf/Q+T rBtRNV6xowGEA82AuWMB75393Lztk/+1jzKSD1Og5dOB7KFlNT3lIsM4ggb4KBdzmZ6f6K /3SMvLYbhDULXtHpO/RSoBD8PPw6KHc= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mMao9yoX; spf=pass (imf26.hostedemail.com: domain of kees@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=kees@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719861175; a=rsa-sha256; cv=none; b=lW2OJhc971BzX/EKvS0cLKD06qlqScrnkEFjBz4h5Xzyuugjc9/0i4qO370P1isj6wb+h1 qeb8s4ovp5qpprqdt/3DQXPimvjnHCBiQ8vqm7gYgnZ++KQQr717vemBwIvBTLAGAGzRJG W96K7GJUCd6JeKOReVVsNL6JWePOH3E= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6B1DA61789; Mon, 1 Jul 2024 19:13:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B4989C4AF0A; Mon, 1 Jul 2024 19:13:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719861184; bh=xxet30UEfeVq/dWdKRNFzCWZboWOxErsseLhti/TXwg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mMao9yoX18zMVhMQY5kIolTbGVQ1jNtDDjmsTCB1JdDobR3dwbrFWHV4S6VLmWFQ6 8klmPbsi/OtNrbFwfYS/rBPG3KZGTj2fhyS7xNyw5asCH7SzSgCPO5zq4qlbgmJVC4 NlzKgmInTFyAWCzXWeqI/pP1k9mxSB9+5iA/G25w4RRI/8F7Fn4l/TaWWuIZgjbV0O 4PFTz+aPo3ftiGxkUDdGoJVCN+BSH5sVJriXl+rpVGOiq3qanJo2M802ddSfZPGdfg rZo1KXvtSOciCb+zMGA/Wx9Mm8L8/fQQBF2P5oOH8z0fF+CUrt0RzcxI+SL5n/fjyN gvLwRdyObFs6A== From: Kees Cook To: Vlastimil Babka Cc: Kees Cook , "GONG, Ruiqi" , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , jvoisin , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Xiu Jianfeng , Suren Baghdasaryan , Kent Overstreet , Jann Horn , Matteo Rizzo , Thomas Graf , Herbert Xu , "Gustavo A. R. Silva" , Jonathan Corbet , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH v6 3/6] mm/slab: Introduce kvmalloc_buckets_node() that can take kmem_buckets argument Date: Mon, 1 Jul 2024 12:13:00 -0700 Message-Id: <20240701191304.1283894-3-kees@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240701190152.it.631-kees@kernel.org> References: <20240701190152.it.631-kees@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2960; i=kees@kernel.org; h=from:subject; bh=xxet30UEfeVq/dWdKRNFzCWZboWOxErsseLhti/TXwg=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBmgv++rAHw0U+CEnyXA/4XClLK6QAyqHm8wswfo OqdafG1Bj+JAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCZoL/vgAKCRCJcvTf3G3A JnwwEACfJUwABmsGkmhqQnATf7Y0sAvd2R6x4c0rtnoHIuF4bVZHVNfrX+sm0soE98hTF6dJhSb 54g7YCknTpz7GmrfaJ5B+6Nt892AgOuCstHR6q3rMDkpbqRRk7a3VIOEqbNnPUNJpyf4Wj6oTB+ FpKGwepyFVPdvd7Prn6jPRJzZQmnRlJRqG73USLhSw5sSEQjd452tI2JG+gHL/qwx11dQeJHiEr 9+/VZ1zhVvgJ1rrfQ4zOcSEPDuuIKjF75SGADyD6H0y20WrMSuGAHFoJpbJYxDkAJE4Gv1Yq28b NHiH8cJzxNi85ddufgze9mVDh6j/09oxZUB6Q4kZYmqy9Xu+f7qrxqQbLbnKV1XYYP9ZGJN0v8H Ludofn6ojZhgH8KOYlpEMD77aG9nOAYKMFPg5wBqXoXlzfhUVhS1Q3UlodTk7WQR9dhez0JEBpW /jw6xO8QkzUt0yBRT3nry/LJsmZmaeY54ekml6BugsOLniwFjBaq/fs/7a6TIDRO8ESL24TtvrB 1GNHbIRz/GxaxHXegXWceNTTLerX3JSOVEnG3qv1zJsV9XPTMAhmlZAV2DrLasptG6tkwgPC9zo Mf0SCpzv9ElM+HZzsRuDAJL6CkBHzXvbgW0IH4traaTQNdAHVoBKHn0pxZfluvUuUt4eiE9VO3D 14HEVxu1s4wtbJg= = X-Developer-Key: i=kees@kernel.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 75297140020 X-Stat-Signature: 8nohi8s9qbf9icdj6kb7fg1dy1m1sx3s X-Rspam-User: X-HE-Tag: 1719861186-817839 X-HE-Meta: U2FsdGVkX18pLhGOoADupfwIinHwCTJZaAoCXDUJ0RJIJoWjU3DKTmuOXjdZNUo96I8rh0eNmHgezywxpALIgt6rwUbN1VU3rNRS/C2jRpFi9ECdlN2ADM1Xx2EgQg4P2a/4emDjZ9Q+0lZOAm6tqKgAyONyz5eIOR+XpyDhAqLbRKfys2gE45DYMZlI6eAxAlMGueJyeY/MrSc/22ZGKB661dMovJWYERCRUjYbz+QH8Ks1CPyJsWsRTvTjBxwsgJfMgqg8SqVnNslmhEP5Y4nhNkIf3OMV27JwuwAKMUvw3MT6Wwln4N/rXmI6rU9sCGkOtTCieJt6ENzKSOxRFXLkqNUJgcciX+7FC6gkS2IeYz91sTUORUI3C32w75I0cYuUkuAF3Rxyaa/KqaG4IvYXAm5dsOxvPrnSvj1joWkgCFZ15v+8yRPI8m0oYoCtk2EFvIRP5gQ43IZi4RbDFLBtFmyr1phmUeoqYeJX0qSG15NIcKM1eOLdtAvaK1sjVAJgMB/J09BgiEA3Za1QYvdvc5Vq3Vf4bYsG4f2ho1lAayN5JwpZtedjkfMjd2nunYau1HBDKH+wLSZ2tmltvrc4bG6gTr7AvTYjEGcaVXzxquv0+7XcmCbQYAfEIBJNkVDZlTGNUtuja+2hfHZTizoUhITS6672tn5MXw3IWmOYUGsjTlEzHJUczP+orY7VtcegVpUGt838Lqmyj+Ems0soIJykC3bZDEuG8yG04EQ54Zx0lBxf8eHLXUSSijDJYOq+XupmJr/dPjGZOrFf8gUsxL9DfMewlkkCnyrMErrHpxMRRj/s//uXFo5f0Ci1nK/ophISBsZD5wUYbdkCJMDlpEUhFaqEaLhZgFe1fGi8ypd1RmIF+Z7mBc66XKe1zdrnj927izOH9qpm9/JGdxxd+cqhYnow7RBtgu+rHtCtweTYss0Nn43S3mJMHTn4uSGA3AztLgAEBfb9MXu o7vtkX84 RkdnieZhvWQ7JjugJA3gYMZsF2/6WbnsQj1NGIsPyvYIa8NSYtstrHLRvHc9eNLwaft+Od8pvY+pcVj6jQ5nbYvy6C4FP7+h5e+FACyvwtfA6La/GH5LYVUYzSAfY42CxbwAspY4XVcqXnEdQ2HuY9ygF6ZxpFH/I/TXYoLUm4it6u78J/Fblr9iHraf84bGRFAhu7RVZnGHrrPPTEYn9Y/lenDBgDz+R4mbXZnYOT1B8hmO1qY76ymrAlzvHFNmsavqbxhXCpLVISjvm77ell7EsrM7O+U3IY4kt6hdjhJTyqMKXrv3+Ia2zW05U6BhZiCaB3qxAnmYSjVjfMl98oW3K6byM29LYpS6gKmM863WbQYs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Plumb kmem_buckets arguments through kvmalloc_node_noprof() so it is possible to provide an API to perform kvmalloc-style allocations with a particular set of buckets. Introduce kvmalloc_buckets_node() that takes a kmem_buckets argument. Signed-off-by: Kees Cook --- include/linux/slab.h | 4 +++- mm/util.c | 9 +++++---- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 708bde6039f0..8d0800c7579a 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -798,7 +798,9 @@ static inline __alloc_size(1) void *kzalloc_noprof(size_t size, gfp_t flags) #define kzalloc(...) alloc_hooks(kzalloc_noprof(__VA_ARGS__)) #define kzalloc_node(_size, _flags, _node) kmalloc_node(_size, (_flags)|__GFP_ZERO, _node) -extern void *kvmalloc_node_noprof(size_t size, gfp_t flags, int node) __alloc_size(1); +void *__kvmalloc_node_noprof(DECL_BUCKET_PARAMS(size, b), gfp_t flags, int node) __alloc_size(1); +#define kvmalloc_node_noprof(size, flags, node) \ + __kvmalloc_node_noprof(PASS_BUCKET_PARAMS(size, NULL), flags, node) #define kvmalloc_node(...) alloc_hooks(kvmalloc_node_noprof(__VA_ARGS__)) #define kvmalloc(_size, _flags) kvmalloc_node(_size, _flags, NUMA_NO_NODE) diff --git a/mm/util.c b/mm/util.c index c9e519e6811f..28c5356b9f1c 100644 --- a/mm/util.c +++ b/mm/util.c @@ -594,9 +594,10 @@ unsigned long vm_mmap(struct file *file, unsigned long addr, EXPORT_SYMBOL(vm_mmap); /** - * kvmalloc_node - attempt to allocate physically contiguous memory, but upon + * __kvmalloc_node - attempt to allocate physically contiguous memory, but upon * failure, fall back to non-contiguous (vmalloc) allocation. * @size: size of the request. + * @b: which set of kmalloc buckets to allocate from. * @flags: gfp mask for the allocation - must be compatible (superset) with GFP_KERNEL. * @node: numa node to allocate from * @@ -609,7 +610,7 @@ EXPORT_SYMBOL(vm_mmap); * * Return: pointer to the allocated memory of %NULL in case of failure */ -void *kvmalloc_node_noprof(size_t size, gfp_t flags, int node) +void *__kvmalloc_node_noprof(DECL_BUCKET_PARAMS(size, b), gfp_t flags, int node) { gfp_t kmalloc_flags = flags; void *ret; @@ -631,7 +632,7 @@ void *kvmalloc_node_noprof(size_t size, gfp_t flags, int node) kmalloc_flags &= ~__GFP_NOFAIL; } - ret = kmalloc_node_noprof(size, kmalloc_flags, node); + ret = __kmalloc_node_noprof(PASS_BUCKET_PARAMS(size, b), kmalloc_flags, node); /* * It doesn't really make sense to fallback to vmalloc for sub page @@ -660,7 +661,7 @@ void *kvmalloc_node_noprof(size_t size, gfp_t flags, int node) flags, PAGE_KERNEL, VM_ALLOW_HUGE_VMAP, node, __builtin_return_address(0)); } -EXPORT_SYMBOL(kvmalloc_node_noprof); +EXPORT_SYMBOL(__kvmalloc_node_noprof); /** * kvfree() - Free memory.