From patchwork Wed Sep 4 10:21:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790336 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7472198825 for ; Wed, 4 Sep 2024 10:21:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445315; cv=none; b=L5DyyJFHqoOhrJbpDmh31rMxuIIvQ0oOdMGriP0eT9nLEU7kgIQcjm6qnISbI+KoqOd4iFmNpCx6WSF/8hnjDd/+dCcy621+kILhJB1XH8BNSpWE3t6KjHM2hxw7ckTu1WeUIotaUPsAqQIHZ0MVCwWG+Qzxwyzwr4aFQ+ehMPU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445315; c=relaxed/simple; bh=aZEAASsBiFk/RgRLba541LPDxJ17oriKgsD3TIAIOy8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=C/sqVIE+2bjtMF7MlPBiIRly2J/3+3m2/4/TK0j9SFZn3O8JpqVzIWR8+WOWoYuOCM/bIQtiOCTJDCOvU29A/bJ/Y+qKJSUk0XKwqsfrQU+cdzcY8MuSA1moHtLuQ4N5rMX+NBC3IpRLBL7oXBjOok6+TtY/HV4D16D9yLqKxvg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pVRvFeeI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pVRvFeeI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D0CC5C4CEC2; Wed, 4 Sep 2024 10:21:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445315; bh=aZEAASsBiFk/RgRLba541LPDxJ17oriKgsD3TIAIOy8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=pVRvFeeIXH04mtBi0pL304wNLzScNm6KRBcMNa+Kk3rJjFcQUjqct+CJhwG66KIy5 MzkAx0bUoO3qqI85wSK+22x/YC0YPxsIIS4OrO7zFh5MgezfDiawCwkChw8Mn57Orf aOmdR4bdcn2J0YD/lY6MIbAP76giGif1VTmMlO65s77Z25UQ3eMrUb6jE8Z0SL1H1l Oc9LHKBhX5vamFvnc/buanj7jLBG4pMRdENbAkzyWqWkeYy7kLrjmiDvc90WqZpeGB a5XITsPViOqtbzhFKuIOgNpN7kfd1fvQhpIcYhQigq7u81tCFyq281BXUhqi8zMhoU MkMdBVI9gWAQg== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:06 +0200 Subject: [PATCH v3 01/17] slab: s/__kmem_cache_create/do_kmem_cache_create/g Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-1-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=2022; i=brauner@kernel.org; h=from:subject:message-id; bh=aZEAASsBiFk/RgRLba541LPDxJ17oriKgsD3TIAIOy8=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNm5xyytO7rTxql9L3d59dyG0jNp6epyKrNMn79vu jrrKGNWRykLgxgXg6yYIotDu0m43HKeis1GmRowc1iZQIYwcHEKwES6Whj+e89ZwziJweJX1ePQ /xlPfa8++PKG5Sfj1j8JW+PFfyYIrWNkuPyp5eJFc4m2ZcpXXhxkPDL3fYi/R09dzyxzxQZ+7eR gDgA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Free up reusing the double-underscore variant for follow-up patches. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- mm/slab.h | 2 +- mm/slab_common.c | 4 ++-- mm/slub.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/slab.h b/mm/slab.h index a6051385186e..684bb48c4f39 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -424,7 +424,7 @@ kmalloc_slab(size_t size, kmem_buckets *b, gfp_t flags, unsigned long caller) gfp_t kmalloc_fix_flags(gfp_t flags); /* Functions provided by the slab allocators */ -int __kmem_cache_create(struct kmem_cache *, slab_flags_t flags); +int do_kmem_cache_create(struct kmem_cache *, slab_flags_t flags); void __init kmem_cache_init(void); extern void create_boot_cache(struct kmem_cache *, const char *name, diff --git a/mm/slab_common.c b/mm/slab_common.c index 95db3702f8d6..91e0e36e4379 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -234,7 +234,7 @@ static struct kmem_cache *create_cache(const char *name, s->useroffset = useroffset; s->usersize = usersize; #endif - err = __kmem_cache_create(s, flags); + err = do_kmem_cache_create(s, flags); if (err) goto out_free_cache; @@ -778,7 +778,7 @@ void __init create_boot_cache(struct kmem_cache *s, const char *name, s->usersize = usersize; #endif - err = __kmem_cache_create(s, flags); + err = do_kmem_cache_create(s, flags); if (err) panic("Creation of kmalloc slab %s size=%u failed. Reason %d\n", diff --git a/mm/slub.c b/mm/slub.c index 9aa5da1e8e27..23d9d783ff26 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -5902,7 +5902,7 @@ __kmem_cache_alias(const char *name, unsigned int size, unsigned int align, return s; } -int __kmem_cache_create(struct kmem_cache *s, slab_flags_t flags) +int do_kmem_cache_create(struct kmem_cache *s, slab_flags_t flags) { int err; From patchwork Wed Sep 4 10:21:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790337 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9518B1CC89A for ; Wed, 4 Sep 2024 10:21:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445319; cv=none; b=KZwGEW9IYXtFMptTOl8U30V7WhlL7lXLlLubgTbb+XWclYJZqz9MZuqaFBk+r72WvgQ+acwk+UB8F8vRZqJbtViN5ojIQOJgcWfjSxHj3cUfHdeJtk/5jyv4z/E6E45fYcwtZo3nHUS6rhVw/U56WzDn4zXi1tgA3x6bBNjSNWk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445319; c=relaxed/simple; bh=YrF0YLTWmhzO7ohK2/AAEF++N1Zrk2GGMPwzT5f/roQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Y6GY2X9V1HSffBxvSv8R2ULKxKDiU+AzlsS0Xv0TVBp6MVkWMkli+y6A9tIhBBjl/EL79Znua3OfN5R7HxTsb05lGVxV8Tr+MTRie7f1BwaAn/d3+U+GPEavnK1Fz8p8xytfObEOxu8PqDDkUfGpTy/XxFJDmTPYU8ip+REUb4w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=R1y/uMG6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="R1y/uMG6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C0DB0C4CEC6; Wed, 4 Sep 2024 10:21:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445319; bh=YrF0YLTWmhzO7ohK2/AAEF++N1Zrk2GGMPwzT5f/roQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=R1y/uMG6e0aaQJlJr5qnvCpHQ3FGR8xCSzgrJj43XHhClJcTH4aQgaRS0vIgoHBt1 jw1g4OsII1pDgv1IGB/YhIhdBx2fO2Ffq3uhe3aCPNGgyPcF/W3VCB+Q7u79OOzaUD cVhFkmhqSf5uA3i4oRlL38Oe6CMmbIMWymgqw+7gO/ElkY7Mp2Bac/wpTba6o4VAZ/ jEZvnnO9LHyDpghOuFNmjRsPUZl40k3dCPjCFn3294/WQRCsfb7Y6FnjVZQwM8XgOS y4FhlZBWVAFW601oThoHu20D5kW0/nsSXsajbVjzMdaPQqDh3LPyIF+xk17yYUY9Wj WbK8uxxKCt+AA== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:07 +0200 Subject: [PATCH v3 02/17] slab: add struct kmem_cache_args Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-2-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=5895; i=brauner@kernel.org; h=from:subject:message-id; bh=YrF0YLTWmhzO7ohK2/AAEF++N1Zrk2GGMPwzT5f/roQ=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNn5Zy1b43m913E1vBfFXhlN6hf4K6t8o4d1sZH0D NOX0RkaHaUsDGJcDLJiiiwO7Sbhcst5KjYbZWrAzGFlAhnCwMUpABOxncrw32W15y+9mCesh/rF jnjanXMQy1z6QeRw3pwetYV7bfi1LjMyfM0T6tlg35cW4vT058FtGzhnW3B/aNK0zY/a8sgoNP0 6MwA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Currently we have multiple kmem_cache_create*() variants that take up to seven separate parameters with one of the functions having to grow an eigth parameter in the future to handle both usercopy and a custom freelist pointer. Add a struct kmem_cache_args structure and move less common parameters into it. Core parameters such as name, object size, and flags continue to be passed separately. Add a new function __kmem_cache_create_args() that takes a struct kmem_cache_args pointer and port do_kmem_cache_create_usercopy() over to it. In follow-up patches we will port the other kmem_cache_create*() variants over to it as well. Reviewed-by: Kees Cook Reviewed-by: Jens Axboe Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- include/linux/slab.h | 21 ++++++++++++++++ mm/slab_common.c | 67 +++++++++++++++++++++++++++++++++++++++------------- 2 files changed, 72 insertions(+), 16 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 5b2da2cf31a8..79d8c8bca4a4 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -240,6 +240,27 @@ struct mem_cgroup; */ bool slab_is_available(void); +/** + * @align: The required alignment for the objects. + * @useroffset: Usercopy region offset + * @usersize: Usercopy region size + * @freeptr_offset: Custom offset for the free pointer in RCU caches + * @use_freeptr_offset: Whether a @freeptr_offset is used + * @ctor: A constructor for the objects. + */ +struct kmem_cache_args { + unsigned int align; + unsigned int useroffset; + unsigned int usersize; + unsigned int freeptr_offset; + bool use_freeptr_offset; + void (*ctor)(void *); +}; + +struct kmem_cache *__kmem_cache_create_args(const char *name, + unsigned int object_size, + struct kmem_cache_args *args, + slab_flags_t flags); struct kmem_cache *kmem_cache_create(const char *name, unsigned int size, unsigned int align, slab_flags_t flags, void (*ctor)(void *)); diff --git a/mm/slab_common.c b/mm/slab_common.c index 91e0e36e4379..0f13c045b8d1 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -248,14 +248,24 @@ static struct kmem_cache *create_cache(const char *name, return ERR_PTR(err); } -static struct kmem_cache * -do_kmem_cache_create_usercopy(const char *name, - unsigned int size, unsigned int freeptr_offset, - unsigned int align, slab_flags_t flags, - unsigned int useroffset, unsigned int usersize, - void (*ctor)(void *)) +/** + * __kmem_cache_create_args - Create a kmem cache + * @name: A string which is used in /proc/slabinfo to identify this cache. + * @object_size: The size of objects to be created in this cache. + * @args: Arguments for the cache creation (see struct kmem_cache_args). + * @flags: See %SLAB_* flags for an explanation of individual @flags. + * + * Cannot be called within a interrupt, but can be interrupted. + * + * Return: a pointer to the cache on success, NULL on failure. + */ +struct kmem_cache *__kmem_cache_create_args(const char *name, + unsigned int object_size, + struct kmem_cache_args *args, + slab_flags_t flags) { struct kmem_cache *s = NULL; + unsigned int freeptr_offset = UINT_MAX; const char *cache_name; int err; @@ -275,7 +285,7 @@ do_kmem_cache_create_usercopy(const char *name, mutex_lock(&slab_mutex); - err = kmem_cache_sanity_check(name, size); + err = kmem_cache_sanity_check(name, object_size); if (err) { goto out_unlock; } @@ -296,12 +306,14 @@ do_kmem_cache_create_usercopy(const char *name, /* Fail closed on bad usersize of useroffset values. */ if (!IS_ENABLED(CONFIG_HARDENED_USERCOPY) || - WARN_ON(!usersize && useroffset) || - WARN_ON(size < usersize || size - usersize < useroffset)) - usersize = useroffset = 0; - - if (!usersize) - s = __kmem_cache_alias(name, size, align, flags, ctor); + WARN_ON(!args->usersize && args->useroffset) || + WARN_ON(object_size < args->usersize || + object_size - args->usersize < args->useroffset)) + args->usersize = args->useroffset = 0; + + if (!args->usersize) + s = __kmem_cache_alias(name, object_size, args->align, flags, + args->ctor); if (s) goto out_unlock; @@ -311,9 +323,11 @@ do_kmem_cache_create_usercopy(const char *name, goto out_unlock; } - s = create_cache(cache_name, size, freeptr_offset, - calculate_alignment(flags, align, size), - flags, useroffset, usersize, ctor); + if (args->use_freeptr_offset) + freeptr_offset = args->freeptr_offset; + s = create_cache(cache_name, object_size, freeptr_offset, + calculate_alignment(flags, args->align, object_size), + flags, args->useroffset, args->usersize, args->ctor); if (IS_ERR(s)) { err = PTR_ERR(s); kfree_const(cache_name); @@ -335,6 +349,27 @@ do_kmem_cache_create_usercopy(const char *name, } return s; } +EXPORT_SYMBOL(__kmem_cache_create_args); + +static struct kmem_cache * +do_kmem_cache_create_usercopy(const char *name, + unsigned int size, unsigned int freeptr_offset, + unsigned int align, slab_flags_t flags, + unsigned int useroffset, unsigned int usersize, + void (*ctor)(void *)) +{ + struct kmem_cache_args kmem_args = { + .align = align, + .use_freeptr_offset = freeptr_offset != UINT_MAX, + .freeptr_offset = freeptr_offset, + .useroffset = useroffset, + .usersize = usersize, + .ctor = ctor, + }; + + return __kmem_cache_create_args(name, size, &kmem_args, flags); +} + /** * kmem_cache_create_usercopy - Create a cache with a region suitable From patchwork Wed Sep 4 10:21:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790338 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96CE614658C for ; Wed, 4 Sep 2024 10:22:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445323; cv=none; b=mC1NsCZIyvdhO3G6qvu+n+XFJ1yGjyRvganju8kxxEtwTe2U/qZUTwM79b/q/49SnJp03dnUfxnd2035sLUJ2zfKrcS35gLsA3LQMn1PCgXolPPtop5UTaA7up5KC6dWY1PIpGUgfgMlPI/F8/xqffc8EX0YBazEUAV4sAhy2NU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445323; c=relaxed/simple; bh=0yvBsmIzvOyKMPTmQD/+Lg+Nh4nfzS75mqd2tBAiPek=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ilJM/764SObyL8kqUzlLDSMsmVgG3x+dv9QscZF0oMU/DUwsfMaeL3LoOMpm0xpKgL4ytDpC0bQTIP9Z63FFKqSPErV8hi0vlPYHmJGMPosCUivifgxJQuLXVaHE56UyTD9ZSTc08+pOKoRk5oKqZ8NAWzc1Kt7JGVXNgAcYhC0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ATXYzaED; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ATXYzaED" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BBE8DC4CEC6; Wed, 4 Sep 2024 10:21:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445323; bh=0yvBsmIzvOyKMPTmQD/+Lg+Nh4nfzS75mqd2tBAiPek=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ATXYzaEDvpyqO1cHVGL9BkKUYRZSfC3s2XK116B4f+XLDx3OR0OqXp7vJQk46lpOB o0+rzieBKLa9Ck3jreS4t1TXQ2/X2f2ZJiWgyRJbWBepXzXY3p+72PP1P4A3enMKTp YJRpuc78eW/3QcKhW5wPqjj4YnoFgpSAk2Io02F/kROX5bxT+VVUVojSV6CVAKX4Pu vytuhD1n/pj06Tld94xW8mBjHjSWEJuQeg790cjshKRbKdFqcY2fp1zrjgPLDERZ+a JuJN7gsBnG4CbYClaIaCTeKJjJthJTBWZcboXJvlqzjvLiKUz4QcaE3SIbQ5x/xzlp WU6mG6l3Yp7Pg== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:08 +0200 Subject: [PATCH v3 03/17] slab: port kmem_cache_create() to struct kmem_cache_args Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-3-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=925; i=brauner@kernel.org; h=from:subject:message-id; bh=0yvBsmIzvOyKMPTmQD/+Lg+Nh4nfzS75mqd2tBAiPek=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNnJbu88Z6+bms7x+X9qq2pO9d8V69rFpSOosqRs4 dV4zTlNHaUsDGJcDLJiiiwO7Sbhcst5KjYbZWrAzGFlAhnCwMUpABN5HMfw38X95L1Xu9QYhQ+t eJ13zN/OOL72oUYV06oQzmyNjE57dkaGuc/0N6ZdNHR3KmN4suviqSP+rQ0ru98LPu7ZYKId783 KAwA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Port kmem_cache_create() to struct kmem_cache_args. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- mm/slab_common.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 0f13c045b8d1..ac0832dac01e 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -439,8 +439,12 @@ struct kmem_cache * kmem_cache_create(const char *name, unsigned int size, unsigned int align, slab_flags_t flags, void (*ctor)(void *)) { - return do_kmem_cache_create_usercopy(name, size, UINT_MAX, align, flags, - 0, 0, ctor); + struct kmem_cache_args kmem_args = { + .align = align, + .ctor = ctor, + }; + + return __kmem_cache_create_args(name, size, &kmem_args, flags); } EXPORT_SYMBOL(kmem_cache_create); From patchwork Wed Sep 4 10:21:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790339 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99A6A1CCB24 for ; Wed, 4 Sep 2024 10:22:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445327; cv=none; b=VgMOrcQiLcPKd+4Qnuxnh2u+BAFSjemnkA9+TKhkDE/QTgE+D5+kBGm2NvJmndGknEb70u9MdtnWoNN5rRiRC3+FNUn1XQBonMheFj5ofG9lzzZN0zVodkLBtjh23p7Jqo6VQMvbbpkUgtXzpeUEqAzv8J7yAGeyDOqJetsfpyM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445327; c=relaxed/simple; bh=PzF+2+t3y9QvUaK7XGtoJxZCJ/C5QzrTNKeCBjRAdK8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JAKyo5D/WUDxSHgheW+y6oNVfPgHsA6259u+Ec8Z57Zjmv0QFlBob6xgCcTvaRmv7sKzjP/nmBGKjvW0O6o/jQwbAbrDcHxzkXCCAt/C/df2UpoBlf83yoy2IP0NCP2+ZkQm34WsunT37ls3mFbG7tKkZu0KJuCiZRF8KhUqi5E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Fr54x1ND; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Fr54x1ND" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3B2FC4CECA; Wed, 4 Sep 2024 10:22:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445327; bh=PzF+2+t3y9QvUaK7XGtoJxZCJ/C5QzrTNKeCBjRAdK8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Fr54x1ND6hGLkeb2A+9fO433Az2ZzEZSs5uo+FsIlP5Rw2c5k2FFT8CkidVw9waS0 w7VCYJqFVwFep3ul/BzzO5m1crxq1AiiijoEUK9ir5la6ndOW8jydEEDMNalFx4fbJ 5tm+i+5GCM2/3bMDIFNMRNQSQdRbw8OL5axjA3I2mXDhjNrNVQdUkCrOboSZx0Mg5I FokxkNFZmpuxIUo5vu27G5LISDln8afLXkDB809xn+aw5LfGyV6VVIRiml77B9FxWq T0gOE7uAzLjDGMpykeHfbEWYNRdoq7cO+fPIT4gfPh5JBnTMu6uNXWErgihqRQcbVE bxzsSnzCnTYfA== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:09 +0200 Subject: [PATCH v3 04/17] slab: port kmem_cache_create_rcu() to struct kmem_cache_args Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-4-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=1036; i=brauner@kernel.org; h=from:subject:message-id; bh=PzF+2+t3y9QvUaK7XGtoJxZCJ/C5QzrTNKeCBjRAdK8=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNm1Se/tBhcrk60f62veG7IUPfrovW+R6MMvMkZr7 xkn+f3+2lHKwiDGxSArpsji0G4SLrecp2KzUaYGzBxWJpAhDFycAjCRresZGVZfuV3EN+lIY7Zv nejLMMEIbQedvUoXnpmtd1omvz0y/B0jw6e5HyJSY5waTD/eWWn7ce3MCg/f1Q6nuDpd735I06x +zQ0A X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Port kmem_cache_create_rcu() to struct kmem_cache_args. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- mm/slab_common.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index ac0832dac01e..da62ed30f95d 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -481,9 +481,13 @@ struct kmem_cache *kmem_cache_create_rcu(const char *name, unsigned int size, unsigned int freeptr_offset, slab_flags_t flags) { - return do_kmem_cache_create_usercopy(name, size, freeptr_offset, 0, - flags | SLAB_TYPESAFE_BY_RCU, 0, 0, - NULL); + struct kmem_cache_args kmem_args = { + .freeptr_offset = freeptr_offset, + .use_freeptr_offset = true, + }; + + return __kmem_cache_create_args(name, size, &kmem_args, + flags | SLAB_TYPESAFE_BY_RCU); } EXPORT_SYMBOL(kmem_cache_create_rcu); From patchwork Wed Sep 4 10:21:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790340 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 160DD1CC89A for ; Wed, 4 Sep 2024 10:22:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445331; cv=none; b=FusCQaA5rdM3qcW3myLPMiiEdjGGn4z1Fccv5gkP0VgNTxRs7mRxe+Xppmie8q7DeFk4KDwa2HRQGhT3p0X1eM9slYyVhZaPRWdPQcII6ohAgO8AyRtLuel8RiKK01AGDWr9hW8VoGnfoG6zPbhZZ2U7JPzXmBOytrhzGW+T+Do= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445331; c=relaxed/simple; bh=t6CjazxfkyG3rY4OORK6/WvzeiMUzf2dJjeC2mkHY5Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LHE+SpEbyFxKrlhekgV1UNWM0D3yh65mp8uzOae++4Xc5uz/NGwSFtXse3x8C3CQx8qBWTabXGuveqOtX6u0ioyl/c9AtPAu0YFPAboR6OeWEeEkbO7zmSVwZn1yjLM3ZvzXP2RtLaJI/Wn0O27/7EuzxyzB/rLhoInFEvsdeL0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=B/f4yXXq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="B/f4yXXq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A5044C4CEC2; Wed, 4 Sep 2024 10:22:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445331; bh=t6CjazxfkyG3rY4OORK6/WvzeiMUzf2dJjeC2mkHY5Q=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=B/f4yXXqFQ5vSEkfdHwtQuqANWB/P0R8vg2RYv1QfjxFiwI6B4jTz9W2XYox8RnVm Z6ewd99jp6fgqA4fe65Io6FD7qgiSiDkoiTXmBf6KQAdODA+gNEda0V4kozGjVNG7j XrcNweG+qb00+grRhqri5Ak6yVTy0XlpAlwXoabLBVfBSMQHAppX4uZyDY5mTfdNFn aOQmTTDICjs94eG6xNLw+AOQMGtqXvxNW+oE2qg9v1HRbwngViSuLWVKB+Zr0I6gR4 nO0V1WPSENxkaRMm12hnLq0+uJCw9w2ian4gZdTotnfz2H2FKSVqYzFbGgp8mo2iAf 0mxWu5/Dzs2eg== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:10 +0200 Subject: [PATCH v3 05/17] slab: port kmem_cache_create_usercopy() to struct kmem_cache_args Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-5-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=1992; i=brauner@kernel.org; h=from:subject:message-id; bh=t6CjazxfkyG3rY4OORK6/WvzeiMUzf2dJjeC2mkHY5Q=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNmV7rWmntNfIcDgadi0xuMKx5YKclqElxXs2/fBb sYCi6UHOkpZGMS4GGTFFFkc2k3C5ZbzVGw2ytSAmcPKBDKEgYtTACbC8oOR4a3j0sVZ1x/0LDOz P6LpWNzw3OpOs+thuVsvmm91hH3ffZGRYYHB01rVNsnGvdNNOYtfLrpoyTK/SMLTavONcpd8T/8 X7AA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Port kmem_cache_create_usercopy() to struct kmem_cache_args and remove the now unused do_kmem_cache_create_usercopy() helper. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- mm/slab_common.c | 30 ++++++++---------------------- 1 file changed, 8 insertions(+), 22 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index da62ed30f95d..16c36a946135 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -351,26 +351,6 @@ struct kmem_cache *__kmem_cache_create_args(const char *name, } EXPORT_SYMBOL(__kmem_cache_create_args); -static struct kmem_cache * -do_kmem_cache_create_usercopy(const char *name, - unsigned int size, unsigned int freeptr_offset, - unsigned int align, slab_flags_t flags, - unsigned int useroffset, unsigned int usersize, - void (*ctor)(void *)) -{ - struct kmem_cache_args kmem_args = { - .align = align, - .use_freeptr_offset = freeptr_offset != UINT_MAX, - .freeptr_offset = freeptr_offset, - .useroffset = useroffset, - .usersize = usersize, - .ctor = ctor, - }; - - return __kmem_cache_create_args(name, size, &kmem_args, flags); -} - - /** * kmem_cache_create_usercopy - Create a cache with a region suitable * for copying to userspace @@ -405,8 +385,14 @@ kmem_cache_create_usercopy(const char *name, unsigned int size, unsigned int useroffset, unsigned int usersize, void (*ctor)(void *)) { - return do_kmem_cache_create_usercopy(name, size, UINT_MAX, align, flags, - useroffset, usersize, ctor); + struct kmem_cache_args kmem_args = { + .align = align, + .ctor = ctor, + .useroffset = useroffset, + .usersize = usersize, + }; + + return __kmem_cache_create_args(name, size, &kmem_args, flags); } EXPORT_SYMBOL(kmem_cache_create_usercopy); From patchwork Wed Sep 4 10:21:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790341 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23D57198825 for ; Wed, 4 Sep 2024 10:22:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445335; cv=none; b=d0qKT59TfNIuXL3bIHTOqffqhbUejGhNWcOv1S+phtaRHNXHlK0FgCb9uihvgsM0gqSzDqNtdjw6BaQIWmzxqVHQadLhFmqdNvNfuyBtum5IgYHWe7p5Sz4QFBL/hW+8/qa77jpkCNOx1JGaqo7/AyMdZnITzxBojVjOIH3md2I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445335; c=relaxed/simple; bh=wlErePvTE8gEuAQI7b5rPaz+icXnXkiXiNHwXkcPVSE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dPX2B4NUL8ULBrk9FjXupLVwkAJfJak/y3WJVRpNzy/ERduph3EO7gtZcv7XWB0douk78dXV7r5MPyqb1/RBbNDY+ovQIn6iYG0GQ4CCCeFxwBDSPAB5RIvw84IrU4mlOBqmaDbicmQxuEznZDsKRJR4m2dWxqjvfyLBsf1UvMk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RcQXs353; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RcQXs353" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 84756C4CEC6; Wed, 4 Sep 2024 10:22:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445335; bh=wlErePvTE8gEuAQI7b5rPaz+icXnXkiXiNHwXkcPVSE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=RcQXs353asnqMtizb8ucN9fB/kmrdEonJJa4WhFzECgl1zSEYCask+0iHaYbU7A0z +WT8J2sxrWx2DYfPN/Wp0oJO3cpPXzoGsp6g5FhdnDbxO7BgD15iCDZh8iC6F55AX6 tfdEmhho3yPQgkUQ8YDVdjVp/rtS5mItx6Tzp19daAiBkCSHovIv67mP8rvt+HVn6u iRosz7EPbnsH157G/WRr7qwowiavGfbdahCmEcloUYfOv6UIpynTERb7ORi/fbyqXc U9GjQr7pb/4Z0r01sVOv/ezRE6PGYyMPqa3bv3wOEPhXC91/lNC6Z+GVtwHG51Fwnc jofw4P+pfNMZw== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:11 +0200 Subject: [PATCH v3 06/17] slab: pass struct kmem_cache_args to create_cache() Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-6-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=3153; i=brauner@kernel.org; h=from:subject:message-id; bh=wlErePvTE8gEuAQI7b5rPaz+icXnXkiXiNHwXkcPVSE=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNl1R3mW7FIn46d5Uw7FCL8OX7j68p7IPTOUeZ6nW fQ4evzz6yhlYRDjYpAVU2RxaDcJl1vOU7HZKFMDZg4rE8gQBi5OAZhIkQrDPwMljWLOnBfnlr85 ev34+sX3utvinm91l5RrsfgnVFdw8S3DX5meJ8+41y4wiH14V6ymv6fdjn3j+rfRbbdULWL17pj WsgAA X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Pass struct kmem_cache_args to create_cache() so that we can later simplify further helpers. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- mm/slab_common.c | 39 +++++++++++++++++++-------------------- 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 16c36a946135..9baa61c9c670 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -202,22 +202,22 @@ struct kmem_cache *find_mergeable(unsigned int size, unsigned int align, } static struct kmem_cache *create_cache(const char *name, - unsigned int object_size, unsigned int freeptr_offset, - unsigned int align, slab_flags_t flags, - unsigned int useroffset, unsigned int usersize, - void (*ctor)(void *)) + unsigned int object_size, + struct kmem_cache_args *args, + slab_flags_t flags) { struct kmem_cache *s; int err; - if (WARN_ON(useroffset + usersize > object_size)) - useroffset = usersize = 0; + if (WARN_ON(args->useroffset + args->usersize > object_size)) + args->useroffset = args->usersize = 0; /* If a custom freelist pointer is requested make sure it's sane. */ err = -EINVAL; - if (freeptr_offset != UINT_MAX && - (freeptr_offset >= object_size || !(flags & SLAB_TYPESAFE_BY_RCU) || - !IS_ALIGNED(freeptr_offset, sizeof(freeptr_t)))) + if (args->use_freeptr_offset && + (args->freeptr_offset >= object_size || + !(flags & SLAB_TYPESAFE_BY_RCU) || + !IS_ALIGNED(args->freeptr_offset, sizeof(freeptr_t)))) goto out; err = -ENOMEM; @@ -227,12 +227,15 @@ static struct kmem_cache *create_cache(const char *name, s->name = name; s->size = s->object_size = object_size; - s->rcu_freeptr_offset = freeptr_offset; - s->align = align; - s->ctor = ctor; + if (args->use_freeptr_offset) + s->rcu_freeptr_offset = args->freeptr_offset; + else + s->rcu_freeptr_offset = UINT_MAX; + s->align = args->align; + s->ctor = args->ctor; #ifdef CONFIG_HARDENED_USERCOPY - s->useroffset = useroffset; - s->usersize = usersize; + s->useroffset = args->useroffset; + s->usersize = args->usersize; #endif err = do_kmem_cache_create(s, flags); if (err) @@ -265,7 +268,6 @@ struct kmem_cache *__kmem_cache_create_args(const char *name, slab_flags_t flags) { struct kmem_cache *s = NULL; - unsigned int freeptr_offset = UINT_MAX; const char *cache_name; int err; @@ -323,11 +325,8 @@ struct kmem_cache *__kmem_cache_create_args(const char *name, goto out_unlock; } - if (args->use_freeptr_offset) - freeptr_offset = args->freeptr_offset; - s = create_cache(cache_name, object_size, freeptr_offset, - calculate_alignment(flags, args->align, object_size), - flags, args->useroffset, args->usersize, args->ctor); + args->align = calculate_alignment(flags, args->align, object_size); + s = create_cache(cache_name, object_size, args, flags); if (IS_ERR(s)) { err = PTR_ERR(s); kfree_const(cache_name); From patchwork Wed Sep 4 10:21:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790342 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58187198825 for ; Wed, 4 Sep 2024 10:22:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445339; cv=none; b=Pbg92H0jGZe8/MzDKNUcMN5jWaXGpt/AEy5dspCg6YIGrMW68gz4wWhNJIM4cezb1lw2vFNYh4A2xf8BOHcofPLBVvi9ItO9YdtyFvY3tSKLzEJpp0HiCt5PW/vPV82hh75XDTno8fgSYAqhtaJEpCHFDPmViI2QeiRh8rCuIfY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445339; c=relaxed/simple; bh=XvA3OjNvZTXuO+1s1Yt+F4vUj5bmDXzlHdYPfx3s9dQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LDafSapCqxqmbITlI6NuyKWu6Ah8hWsdsPSeXPttRLMzjTQbQByoI/YWvhUkFb8ECBnc554gyZxSBfUBhfksW9IJzCtZpiQOTQRCV8dHktPiKEz0JmlIwmEdD/zHLCgtbsDhj8gwhx8Den5g+c0m/32C0T1cJQ5ZL7UOwtk14lA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dHNDkNyo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dHNDkNyo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6FCF6C4CEC8; Wed, 4 Sep 2024 10:22:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445338; bh=XvA3OjNvZTXuO+1s1Yt+F4vUj5bmDXzlHdYPfx3s9dQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=dHNDkNyof6PXzmIXAY/8Nxv2qxyJEp6i9DMdtkFlhAvTa/5gvBP4elVrdQWCQbs/g UqHo6xYX/3ja7Mla50VPP7g0eWGgcGcFZPdp6Xe/K9ksOQzVGHjDcZ/x/lg9hZrpI4 w7jrv19fpk/YrutcUmR+3sVhnY1AW/KTGxJ3a8PzpMu42uUBzCanWtarU3eHK/6i3w GB39Wfo27w8EWKr07F9s5ue6ob2fFV97Z/9RKVmKTXiCez04wM5+4dsdQkfyKvkXPn mGrUwTUx6TgmWhjgdb/JXbR9H/VGapduDWqswy7FQXdvKM3csBKfacz50Nm1GSP3J2 tLntU20O+Ds8A== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:12 +0200 Subject: [PATCH v3 07/17] slab: pull kmem_cache_open() into do_kmem_cache_create() Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-7-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=4287; i=brauner@kernel.org; h=from:subject:message-id; bh=XvA3OjNvZTXuO+1s1Yt+F4vUj5bmDXzlHdYPfx3s9dQ=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNm19VTQ/A31/Pcu2kyd5/7N9tnrK+eEj5TmndY0t DG4qP9lXkcpC4MYF4OsmCKLQ7tJuNxynorNRpkaMHNYmUCGMHBxCsBEvK4wMrzY0Sn98yJDqfmu HNOPRaseRR7wXDt5jfvSf15amn62AQ8Z/oocyBH68vthrP6iL/4TN0+xFO8VmnIh98LPPQ5Lb9u 1HeYHAA== X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 do_kmem_cache_create() is the only caller and we're going to pass down struct kmem_cache_args in a follow-up patch. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- mm/slub.c | 132 +++++++++++++++++++++++++++++--------------------------------- 1 file changed, 62 insertions(+), 70 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 23d9d783ff26..30f4ca6335c7 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -5290,65 +5290,6 @@ static int calculate_sizes(struct kmem_cache *s) return !!oo_objects(s->oo); } -static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags) -{ - s->flags = kmem_cache_flags(flags, s->name); -#ifdef CONFIG_SLAB_FREELIST_HARDENED - s->random = get_random_long(); -#endif - - if (!calculate_sizes(s)) - goto error; - if (disable_higher_order_debug) { - /* - * Disable debugging flags that store metadata if the min slab - * order increased. - */ - if (get_order(s->size) > get_order(s->object_size)) { - s->flags &= ~DEBUG_METADATA_FLAGS; - s->offset = 0; - if (!calculate_sizes(s)) - goto error; - } - } - -#ifdef system_has_freelist_aba - if (system_has_freelist_aba() && !(s->flags & SLAB_NO_CMPXCHG)) { - /* Enable fast mode */ - s->flags |= __CMPXCHG_DOUBLE; - } -#endif - - /* - * The larger the object size is, the more slabs we want on the partial - * list to avoid pounding the page allocator excessively. - */ - s->min_partial = min_t(unsigned long, MAX_PARTIAL, ilog2(s->size) / 2); - s->min_partial = max_t(unsigned long, MIN_PARTIAL, s->min_partial); - - set_cpu_partial(s); - -#ifdef CONFIG_NUMA - s->remote_node_defrag_ratio = 1000; -#endif - - /* Initialize the pre-computed randomized freelist if slab is up */ - if (slab_state >= UP) { - if (init_cache_random_seq(s)) - goto error; - } - - if (!init_kmem_cache_nodes(s)) - goto error; - - if (alloc_kmem_cache_cpus(s)) - return 0; - -error: - __kmem_cache_release(s); - return -EINVAL; -} - static void list_slab_objects(struct kmem_cache *s, struct slab *slab, const char *text) { @@ -5904,26 +5845,77 @@ __kmem_cache_alias(const char *name, unsigned int size, unsigned int align, int do_kmem_cache_create(struct kmem_cache *s, slab_flags_t flags) { - int err; + int err = -EINVAL; - err = kmem_cache_open(s, flags); - if (err) - return err; + s->flags = kmem_cache_flags(flags, s->name); +#ifdef CONFIG_SLAB_FREELIST_HARDENED + s->random = get_random_long(); +#endif + + if (!calculate_sizes(s)) + goto out; + if (disable_higher_order_debug) { + /* + * Disable debugging flags that store metadata if the min slab + * order increased. + */ + if (get_order(s->size) > get_order(s->object_size)) { + s->flags &= ~DEBUG_METADATA_FLAGS; + s->offset = 0; + if (!calculate_sizes(s)) + goto out; + } + } + +#ifdef system_has_freelist_aba + if (system_has_freelist_aba() && !(s->flags & SLAB_NO_CMPXCHG)) { + /* Enable fast mode */ + s->flags |= __CMPXCHG_DOUBLE; + } +#endif + + /* + * The larger the object size is, the more slabs we want on the partial + * list to avoid pounding the page allocator excessively. + */ + s->min_partial = min_t(unsigned long, MAX_PARTIAL, ilog2(s->size) / 2); + s->min_partial = max_t(unsigned long, MIN_PARTIAL, s->min_partial); + + set_cpu_partial(s); + +#ifdef CONFIG_NUMA + s->remote_node_defrag_ratio = 1000; +#endif + + /* Initialize the pre-computed randomized freelist if slab is up */ + if (slab_state >= UP) { + if (init_cache_random_seq(s)) + goto out; + } + + if (!init_kmem_cache_nodes(s)) + goto out; + + if (!alloc_kmem_cache_cpus(s)) + goto out; /* Mutex is not taken during early boot */ - if (slab_state <= UP) - return 0; + if (slab_state <= UP) { + err = 0; + goto out; + } err = sysfs_slab_add(s); - if (err) { - __kmem_cache_release(s); - return err; - } + if (err) + goto out; if (s->flags & SLAB_STORE_USER) debugfs_slab_add(s); - return 0; +out: + if (err) + __kmem_cache_release(s); + return err; } #ifdef SLAB_SUPPORTS_SYSFS From patchwork Wed Sep 4 10:21:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790343 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22E2F433C8 for ; Wed, 4 Sep 2024 10:22:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445343; cv=none; b=dOWUb3yW+x5Hi3XbPLRnlMd5XzA8TqqYMorFVQoHKaBDCqs+UMzC1dKCt00I8O9kzEYwkiWA5r91qZmsljaFgCdijRklJuanHkySSW5TlVNZwQJZBzat8dXRVO/geAvj90rCXN1rCCvQTaIo5IflCilz7jmYpN72Vkoh4DE3YlM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445343; c=relaxed/simple; bh=Lf0x5OF2wU4gj/MS9gGKSdAWPtAXYg+UCtXI3xTY+oM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=fEmy0ZzvhbTqse38zTGZqpnSoEvnwllAiI3KeFsAdjoMqYRgSkccpKCpVDbFO3iQfevUlfp04w1DSlQedV0XpKfRzfHihqoN2avG/JEhqDKeBJt8Z8Ijh+F80YJtYDhDyx9brLv3UX0C5cWtXtCem/W2so3TxwM33i+cUzaeI4Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=k2CX6esi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="k2CX6esi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5504FC4CEC2; Wed, 4 Sep 2024 10:22:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445342; bh=Lf0x5OF2wU4gj/MS9gGKSdAWPtAXYg+UCtXI3xTY+oM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=k2CX6esieR2DSrAm/zlh8hGQRtYTUzTrBeRwPO4dtam+p9PJiYfG34rAxs0jMItFa cOl2HPnbM7Ctg0Hs636nvcXa0OH8jdpw76Pu6lDoc32JoJjX7kyHUu4wHEzLpsfyyW lYLPJETA7CzrJI3yMFTjxAGcUudA9zwJavBSnj39Nu7hjByuCpXZGz0fzWsyFJEgI0 pCXrMQuJqu2DWDPxk2rTXSaC2wvegM1eoS4ZR+atlIyIqKiEJtfQSOm0YkyEWBnmlq VX8QOQWnaX6kz6xvhOCUP5lXZP0Ol7wkQWdBTPg+Z51NwMZ6G5hf1J0Fck4Wg3SmoY Cau2hk1OwJSBg== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:13 +0200 Subject: [PATCH v3 08/17] slab: pass struct kmem_cache_args to do_kmem_cache_create() Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-8-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=3919; i=brauner@kernel.org; h=from:subject:message-id; bh=Lf0x5OF2wU4gj/MS9gGKSdAWPtAXYg+UCtXI3xTY+oM=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNl1+96yN7XuIidXBs55eepY8tVfXOd+KTMHvNO3z D1WGHAut6OUhUGMi0FWTJHFod0kXG45T8Vmo0wNmDmsTCBDGLg4BWAisf4M/zSZEzhbP0dUxJov OM0mnnX6RPbX2bdmNPX+Dz/1tjzplirD/5QuHrUktw1HP5dd3Mr0Xr8gq9tjo7WD85GFlvdMNRl 4GQE= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 and initialize most things in do_kmem_cache_create(). In a follow-up patch we'll remove rcu_freeptr_offset from struct kmem_cache. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- mm/slab.h | 4 +++- mm/slab_common.c | 27 ++++++--------------------- mm/slub.c | 17 ++++++++++++++++- 3 files changed, 25 insertions(+), 23 deletions(-) diff --git a/mm/slab.h b/mm/slab.h index 684bb48c4f39..c7a4e0fc3cf1 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -424,7 +424,9 @@ kmalloc_slab(size_t size, kmem_buckets *b, gfp_t flags, unsigned long caller) gfp_t kmalloc_fix_flags(gfp_t flags); /* Functions provided by the slab allocators */ -int do_kmem_cache_create(struct kmem_cache *, slab_flags_t flags); +int do_kmem_cache_create(struct kmem_cache *s, const char *name, + unsigned int size, struct kmem_cache_args *args, + slab_flags_t flags); void __init kmem_cache_init(void); extern void create_boot_cache(struct kmem_cache *, const char *name, diff --git a/mm/slab_common.c b/mm/slab_common.c index 9baa61c9c670..19ae3dd6e36f 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -224,20 +224,7 @@ static struct kmem_cache *create_cache(const char *name, s = kmem_cache_zalloc(kmem_cache, GFP_KERNEL); if (!s) goto out; - - s->name = name; - s->size = s->object_size = object_size; - if (args->use_freeptr_offset) - s->rcu_freeptr_offset = args->freeptr_offset; - else - s->rcu_freeptr_offset = UINT_MAX; - s->align = args->align; - s->ctor = args->ctor; -#ifdef CONFIG_HARDENED_USERCOPY - s->useroffset = args->useroffset; - s->usersize = args->usersize; -#endif - err = do_kmem_cache_create(s, flags); + err = do_kmem_cache_create(s, name, object_size, args, flags); if (err) goto out_free_cache; @@ -788,9 +775,7 @@ void __init create_boot_cache(struct kmem_cache *s, const char *name, { int err; unsigned int align = ARCH_KMALLOC_MINALIGN; - - s->name = name; - s->size = s->object_size = size; + struct kmem_cache_args kmem_args = {}; /* * kmalloc caches guarantee alignment of at least the largest @@ -799,14 +784,14 @@ void __init create_boot_cache(struct kmem_cache *s, const char *name, */ if (flags & SLAB_KMALLOC) align = max(align, 1U << (ffs(size) - 1)); - s->align = calculate_alignment(flags, align, size); + kmem_args.align = calculate_alignment(flags, align, size); #ifdef CONFIG_HARDENED_USERCOPY - s->useroffset = useroffset; - s->usersize = usersize; + kmem_args.useroffset = useroffset; + kmem_args.usersize = usersize; #endif - err = do_kmem_cache_create(s, flags); + err = do_kmem_cache_create(s, name, size, &kmem_args, flags); if (err) panic("Creation of kmalloc slab %s size=%u failed. Reason %d\n", diff --git a/mm/slub.c b/mm/slub.c index 30f4ca6335c7..4719b60215b8 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -5843,14 +5843,29 @@ __kmem_cache_alias(const char *name, unsigned int size, unsigned int align, return s; } -int do_kmem_cache_create(struct kmem_cache *s, slab_flags_t flags) +int do_kmem_cache_create(struct kmem_cache *s, const char *name, + unsigned int size, struct kmem_cache_args *args, + slab_flags_t flags) { int err = -EINVAL; + s->name = name; + s->size = s->object_size = size; + s->flags = kmem_cache_flags(flags, s->name); #ifdef CONFIG_SLAB_FREELIST_HARDENED s->random = get_random_long(); #endif + if (args->use_freeptr_offset) + s->rcu_freeptr_offset = args->freeptr_offset; + else + s->rcu_freeptr_offset = UINT_MAX; + s->align = args->align; + s->ctor = args->ctor; +#ifdef CONFIG_HARDENED_USERCOPY + s->useroffset = args->useroffset; + s->usersize = args->usersize; +#endif if (!calculate_sizes(s)) goto out; From patchwork Wed Sep 4 10:21:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790344 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5A3E433C8 for ; Wed, 4 Sep 2024 10:22:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445346; cv=none; b=S+XadaKZkxGqluXPJXw4Cs8eLA31NVPASw3mCWkFWLP1uFigT5OssSpnOOhbURc+jAYePWSj8GArmWY7/kfjkRRiWcR61EchAATSr3xeadyCsW5vzmFUtvSmQkxsyGT2F2b5nR/SByaGKWdso30lWcpERABGeG8KI+zRGxvC7tY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445346; c=relaxed/simple; bh=8hORSdr9O/EqElT2ajhJQ8gEdIqY6bU8s9Nue9pitwk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Vu1CXY1mqusXhyLq1a7/q/hRcjerI3jOOKnhHZ/3BVJZFE8OwYr37IaRH+qDc/NbzRw40uOt2n1SDXATZbMea/zz+KyUeXzPUg9RbALcKDQKiscXB7Xqb5ZnxRaHQEYj4muU5jjolVix2GY379foIjKA/VvDk9fs97q4M+DsaPs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=us9HEmIz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="us9HEmIz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 35D50C4CEC6; Wed, 4 Sep 2024 10:22:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445346; bh=8hORSdr9O/EqElT2ajhJQ8gEdIqY6bU8s9Nue9pitwk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=us9HEmIzCgJ1DP9lrROrQhAb2zX+L5BEnnhdKkWq6pq0V3MhZk2lZ4bevheiquKzS 60XaBJaYfYppDbcqI/WDixM0L7NIaJA4M8LKXgTWMK0fphczOVIiFOsf9Atupsb1bj UFfnioh3A2OD/lsX+87sFGf2cMJ/Xtal5GknHUeK0aBeZV2Hc3G2OFN09CIPtZSCNH Bf9Xglj3Y62QKfyzei6gxD5Ilr4phgEjtoQX5dDDD2zqvwnJ9Fxs8c2MqhIwTYArAf Q8EafrAHwYLvHu3mDnBU/gu2RKAk59ffY6BBz7AxxxC9klbM/s1gJ+x4RApnjLJkVA iLHmH+bMcC5Gw== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:14 +0200 Subject: [PATCH v3 09/17] slab: remove rcu_freeptr_offset from struct kmem_cache Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-9-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=3911; i=brauner@kernel.org; h=from:subject:message-id; bh=8hORSdr9O/EqElT2ajhJQ8gEdIqY6bU8s9Nue9pitwk=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNk1Y2pnvP31qKVpOcJ5do+eigrNe3hBOWNSeo3uI obMvRKBHaUsDGJcDLJiiiwO7Sbhcst5KjYbZWrAzGFlAhnCwMUpABNRMWFkeMCSwrpzXcROLyPf JzwXoxOe3hR/ovf8iI7sgtCN83NWVjMytJ4XPf57hZ7huRP6s2Z5+YTtMrRZIt6vpv7B7bDZisW +fAA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Pass down struct kmem_cache_args to calculate_sizes() so we can use args->{use}_freeptr_offset directly. This allows us to remove ->rcu_freeptr_offset from struct kmem_cache. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- mm/slab.h | 2 -- mm/slub.c | 25 +++++++------------------ 2 files changed, 7 insertions(+), 20 deletions(-) diff --git a/mm/slab.h b/mm/slab.h index c7a4e0fc3cf1..36ac38e21fcb 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -261,8 +261,6 @@ struct kmem_cache { unsigned int object_size; /* Object size without metadata */ struct reciprocal_value reciprocal_size; unsigned int offset; /* Free pointer offset */ - /* Specific free pointer requested (if not UINT_MAX) */ - unsigned int rcu_freeptr_offset; #ifdef CONFIG_SLUB_CPU_PARTIAL /* Number of per cpu partial objects to keep around */ unsigned int cpu_partial; diff --git a/mm/slub.c b/mm/slub.c index 4719b60215b8..a23c7036cd61 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3916,8 +3916,7 @@ static void *__slab_alloc_node(struct kmem_cache *s, * If the object has been wiped upon free, make sure it's fully initialized by * zeroing out freelist pointer. * - * Note that we also wipe custom freelist pointers specified via - * s->rcu_freeptr_offset. + * Note that we also wipe custom freelist pointers. */ static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s, void *obj) @@ -5141,17 +5140,11 @@ static void set_cpu_partial(struct kmem_cache *s) #endif } -/* Was a valid freeptr offset requested? */ -static inline bool has_freeptr_offset(const struct kmem_cache *s) -{ - return s->rcu_freeptr_offset != UINT_MAX; -} - /* * calculate_sizes() determines the order and the distribution of data within * a slab object. */ -static int calculate_sizes(struct kmem_cache *s) +static int calculate_sizes(struct kmem_cache_args *args, struct kmem_cache *s) { slab_flags_t flags = s->flags; unsigned int size = s->object_size; @@ -5192,7 +5185,7 @@ static int calculate_sizes(struct kmem_cache *s) */ s->inuse = size; - if (((flags & SLAB_TYPESAFE_BY_RCU) && !has_freeptr_offset(s)) || + if (((flags & SLAB_TYPESAFE_BY_RCU) && !args->use_freeptr_offset) || (flags & SLAB_POISON) || s->ctor || ((flags & SLAB_RED_ZONE) && (s->object_size < sizeof(void *) || slub_debug_orig_size(s)))) { @@ -5214,8 +5207,8 @@ static int calculate_sizes(struct kmem_cache *s) */ s->offset = size; size += sizeof(void *); - } else if ((flags & SLAB_TYPESAFE_BY_RCU) && has_freeptr_offset(s)) { - s->offset = s->rcu_freeptr_offset; + } else if ((flags & SLAB_TYPESAFE_BY_RCU) && args->use_freeptr_offset) { + s->offset = args->freeptr_offset; } else { /* * Store freelist pointer near middle of object to keep @@ -5856,10 +5849,6 @@ int do_kmem_cache_create(struct kmem_cache *s, const char *name, #ifdef CONFIG_SLAB_FREELIST_HARDENED s->random = get_random_long(); #endif - if (args->use_freeptr_offset) - s->rcu_freeptr_offset = args->freeptr_offset; - else - s->rcu_freeptr_offset = UINT_MAX; s->align = args->align; s->ctor = args->ctor; #ifdef CONFIG_HARDENED_USERCOPY @@ -5867,7 +5856,7 @@ int do_kmem_cache_create(struct kmem_cache *s, const char *name, s->usersize = args->usersize; #endif - if (!calculate_sizes(s)) + if (!calculate_sizes(args, s)) goto out; if (disable_higher_order_debug) { /* @@ -5877,7 +5866,7 @@ int do_kmem_cache_create(struct kmem_cache *s, const char *name, if (get_order(s->size) > get_order(s->object_size)) { s->flags &= ~DEBUG_METADATA_FLAGS; s->offset = 0; - if (!calculate_sizes(s)) + if (!calculate_sizes(args, s)) goto out; } } From patchwork Wed Sep 4 10:21:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790345 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9181C14658C for ; Wed, 4 Sep 2024 10:22:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445350; cv=none; b=BWwrcFSrJfRzV+dzpSNdNIIkuuO05MumdI193I80ViSnioJWC2Uf32B9RzCLPibm4S3PqNsmBzIjgkC59/9EQDm/c4ch3lP6cDH5BlBIrl0SPYOrMDCATmRB4SpmQ0VyT26xPpB79LCVUST5nYWEorC4IXjt7GZEZk2X/rUg1kQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445350; c=relaxed/simple; bh=/U/EdH2xjh8wlswrDAib0TyvkUltBmXomIATwtZQzXM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=j/cNEvYReKjKkThVUNrSa8gneAowhYXWRHe/6RcfhMePWimyMMJF3S5pX3I15Z3IF+vabRmUbGGhqu1TQAp+q1aeMuD2m+LUs9Oon72d9AdQFMfErME+QwVfmV9j6fzIpxR0KBgfZ4Q6cHTX3BRSnsG8W/gDQkRMqkhS1w1IiDQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ng4+fX0s; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ng4+fX0s" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1F8C5C4CECA; Wed, 4 Sep 2024 10:22:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445350; bh=/U/EdH2xjh8wlswrDAib0TyvkUltBmXomIATwtZQzXM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Ng4+fX0s+8DqSHCR5z9UFG82jzXO8WDWHFkEdHUi9jQ+e6ydvcsUhDcnPzOss7yio dLO4oWouJTmhgVH/3C4C6a9ju/RIGsmQiuDTcIqUwxTrzQqI/+RQ/777jJQpjGiLSB A+/xoiff9qLDxM/465PUiXMat7Lr3YyFKz8b0fmKKk4fs39EMdvIj7ewjAbOl3w/x5 0Z0kfj9Ohvs2pLp/lty6ljCM24y3jD1VnWrq7IjWzVavn53N15ekpNeJIst9gJ62Jo xR03wiqIsnNZsFzHH10rsWoMeZs7UJCsvQJ6xyckYgxCOKnTlfYjYFUa0f3lOK+gnM 17/RgHPMb0kEQ== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:15 +0200 Subject: [PATCH v3 10/17] slab: port KMEM_CACHE() to struct kmem_cache_args Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-10-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=1217; i=brauner@kernel.org; h=from:subject:message-id; bh=/U/EdH2xjh8wlswrDAib0TyvkUltBmXomIATwtZQzXM=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNl1oG5JPOf83ttis7ZNrFp7sPrsoz65lH2ntZ427 Zh+zOjevI5SFgYxLgZZMUUWh3aTcLnlPBWbjTI1YOawMoEMYeDiFICJeBUxMvx76K26/FDZJaco 2x96nI93rMk6O7OU9/2r7Sv46r59qw9nZNi5vaQsb84tv0tfMpQaD03zbGLs1VmtW9a8+9ut8H1 MUxgB X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Make KMEM_CACHE() use struct kmem_cache_args. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- include/linux/slab.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 79d8c8bca4a4..d9c2ed5bc02f 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -283,9 +283,12 @@ int kmem_cache_shrink(struct kmem_cache *s); * f.e. add ____cacheline_aligned_in_smp to the struct declaration * then the objects will be properly aligned in SMP configurations. */ -#define KMEM_CACHE(__struct, __flags) \ - kmem_cache_create(#__struct, sizeof(struct __struct), \ - __alignof__(struct __struct), (__flags), NULL) +#define KMEM_CACHE(__struct, __flags) \ + __kmem_cache_create_args(#__struct, sizeof(struct __struct), \ + &(struct kmem_cache_args) { \ + .align = __alignof__(struct __struct), \ + .ctor = NULL, \ + }, (__flags)) /* * To whitelist a single field for copying to/from usercopy, use this From patchwork Wed Sep 4 10:21:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790346 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3B4B14658C for ; Wed, 4 Sep 2024 10:22:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445354; cv=none; b=mR5N9UyoLMnoO/nhKbUiLgRBdJen+I94tz/aRxdA2G0x1SZSqmZkWY5PuccIKJIzo5BR2hIAgT0Ra2dJ2IOkhEU0QlF1U1zbTlYJO+n8LCugFqQj6/YHf8RGeqrFczfKF7rDNdIOpdMb5DBmqhOG95SReXe3Kksi4yhXmmVqBMs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445354; c=relaxed/simple; bh=82J8Bf3LCC365/T9k9A+CuN1jBmnIwSLQl/UyUW6uSk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=lyGgR3PRXPPclvfvh0ieiLqPaYe3A5Sug3SRHdl8xAZzRO3j8PE+e7rac0f7RV1l7WEAqsLu5y5seu08JgAXznhl7mEq3sgy1BvnS1wmy9BfPlN8GDiSi2JE99DhiKSTF9jcqDHdTl+q6pECh//eLjaSawv7CbM9GAScPks6VcM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VftW6h/4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VftW6h/4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E6A98C4CEC8; Wed, 4 Sep 2024 10:22:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445354; bh=82J8Bf3LCC365/T9k9A+CuN1jBmnIwSLQl/UyUW6uSk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=VftW6h/4nP22SErP4ZUBode4BsdoX/rn/Oe2nrMJfHyJqoUNq9k9mqtieylVbFrTu /Rb5LHSBVEgxWAM9gIifQg79JG+qOOzP/g8cwhWLXPwoc2WzvHbr5OZHPRpVixzW2l AKWOunLfQ9TRuoKUIG3mnHOxjlmDxnng0QCl05BKY5qCy8L+XRktHVZKA3DXb3867Y HA8zx3rGwx39UgKzj3y6Fg+yIsXcKDa9dbmI2AxaEY1xfFE/EvGXLNN+qYq/I6Hf11 PRy9tXEt6NHf3W0hoe9oDsg4S12TYYAPavNlvJK0s6nenxgW6T+/4NtAYHxdE4LVYl 9ZURfLw0qrAuQ== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:16 +0200 Subject: [PATCH v3 11/17] slab: port KMEM_CACHE_USERCOPY() to struct kmem_cache_args Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-11-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=1417; i=brauner@kernel.org; h=from:subject:message-id; bh=82J8Bf3LCC365/T9k9A+CuN1jBmnIwSLQl/UyUW6uSk=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNllvcm9/S2D0dtHnVb2ptY3cvLiTpxk4XvUvaP8Z 0LVsbwTHaUsDGJcDLJiiiwO7Sbhcst5KjYbZWrAzGFlAhnCwMUpABM5e5uRYUHpdQ8euaUhvDaS vbvYfR1nHDyetjnWZVWPwKNH2bXGwgz/w2R25VxKdOL6q/okJ5z7zd4zAb46QseLvVh5ah5X/Jn JCAA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Make KMEM_CACHE_USERCOPY() use struct kmem_cache_args. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- include/linux/slab.h | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index d9c2ed5bc02f..aced16a08700 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -294,12 +294,14 @@ int kmem_cache_shrink(struct kmem_cache *s); * To whitelist a single field for copying to/from usercopy, use this * macro instead for KMEM_CACHE() above. */ -#define KMEM_CACHE_USERCOPY(__struct, __flags, __field) \ - kmem_cache_create_usercopy(#__struct, \ - sizeof(struct __struct), \ - __alignof__(struct __struct), (__flags), \ - offsetof(struct __struct, __field), \ - sizeof_field(struct __struct, __field), NULL) +#define KMEM_CACHE_USERCOPY(__struct, __flags, __field) \ + __kmem_cache_create_args(#__struct, sizeof(struct __struct), \ + &(struct kmem_cache_args) { \ + .align = __alignof__(struct __struct), \ + .useroffset = offsetof(struct __struct, __field), \ + .usersize = sizeof_field(struct __struct, __field), \ + .ctor = NULL, \ + }, (__flags)) /* * Common kmalloc functions provided by all allocators From patchwork Wed Sep 4 10:21:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790347 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3A1514658C for ; Wed, 4 Sep 2024 10:22:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445359; cv=none; b=kwq34Fl+VHggO0u0HZh1LQlBxwfD2WASYMhS8dR0BylqvOXeLDu7CNQmzteoYEpIhEzW2drwJioBXoSU8xSWifEJAoLJOzIJ5DWXR5MR4y1rx3psxPBMOLFnGrd7dv09qiTq0VQ1MVljnMh2fE25azRXr1Jbr9n5JpWLlU94+/0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445359; c=relaxed/simple; bh=dnwP/c5HlqpuuYs6TocRpCymaBfpftG+kdwK6XKc1E0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=DcQ1ML2zOFl/HO7j86JudRwFGHyodyFkxJyW5MmPt33lzqHBVMRc3HVzaqvKY4X1ehMEQyXQH4hFcGtJt7A5HbPugQF6dv4u52vh0/0pgIvrrTy36Vkexlb/03Oj5nBRH8V0N20xeVisTZVSjIIxo8c/gj+aj5oh+baPphoHCI8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HhZh9DXp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HhZh9DXp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F26DAC4CEC6; Wed, 4 Sep 2024 10:22:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445358; bh=dnwP/c5HlqpuuYs6TocRpCymaBfpftG+kdwK6XKc1E0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=HhZh9DXpXCPba9MpQZVKOy0Ru27OS0HEskSXwAoDug2DUBzvgpz64S98j7Hw1t/bC BVj+HYC6e8qqlAqINxPAkWw3OhJM3Ou2+TOjNYfe+EPeKyLXxkaN31I3Uo6sUMGr+9 MmCk+iSDrssxm7ruDUw4wx6fAOPkz/++IWZjV1gD66ymm4gV9WFq+aOJGQH309H/7M EFpGvoa7aVx0dDTnBfaX01Bd5+U3LjFbuYSjbLn/nqf9tHWtEepUycPqo+VXfDFM8L 9BehLMX+Y0F3ogbV8kHXUl0vnhsfc0ExXBSSmNz5GI5GDFX0yUHmZeScAnze6TYIEU NN53xl5rTni6w== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:17 +0200 Subject: [PATCH v3 12/17] slab: create kmem_cache_create() compatibility layer Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-12-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=3373; i=brauner@kernel.org; h=from:subject:message-id; bh=dnwP/c5HlqpuuYs6TocRpCymaBfpftG+kdwK6XKc1E0=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNl15rbBxssX1ASll7Ntr2iuelPPt6tHsviFY87f+ 97vVoi4dZSyMIhxMciKKbI4tJuEyy3nqdhslKkBM4eVCWQIAxenAEzk7g2Gv/IhxWev+F9Q+PAl 1O+De+AROQ+p2wpTl6p7Js2erjNNvoLhf7Z27Q3zJWyBfJVrtRhcdZ/UPjV03BB5ksl+6sHqBXl 5bAA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Use _Generic() to create a compatibility layer that type switches on the third argument to either call __kmem_cache_create() or __kmem_cache_create_args(). This can be kept in place until all callers have been ported to struct kmem_cache_args. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- include/linux/slab.h | 13 ++++++++++--- mm/slab_common.c | 10 +++++----- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index aced16a08700..4292d67094c3 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -261,9 +261,10 @@ struct kmem_cache *__kmem_cache_create_args(const char *name, unsigned int object_size, struct kmem_cache_args *args, slab_flags_t flags); -struct kmem_cache *kmem_cache_create(const char *name, unsigned int size, - unsigned int align, slab_flags_t flags, - void (*ctor)(void *)); + +struct kmem_cache *__kmem_cache_create(const char *name, unsigned int size, + unsigned int align, slab_flags_t flags, + void (*ctor)(void *)); struct kmem_cache *kmem_cache_create_usercopy(const char *name, unsigned int size, unsigned int align, slab_flags_t flags, @@ -272,6 +273,12 @@ struct kmem_cache *kmem_cache_create_usercopy(const char *name, struct kmem_cache *kmem_cache_create_rcu(const char *name, unsigned int size, unsigned int freeptr_offset, slab_flags_t flags); + +#define kmem_cache_create(__name, __object_size, __args, ...) \ + _Generic((__args), \ + struct kmem_cache_args *: __kmem_cache_create_args, \ + default: __kmem_cache_create)(__name, __object_size, __args, __VA_ARGS__) + void kmem_cache_destroy(struct kmem_cache *s); int kmem_cache_shrink(struct kmem_cache *s); diff --git a/mm/slab_common.c b/mm/slab_common.c index 19ae3dd6e36f..418459927670 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -383,7 +383,7 @@ kmem_cache_create_usercopy(const char *name, unsigned int size, EXPORT_SYMBOL(kmem_cache_create_usercopy); /** - * kmem_cache_create - Create a cache. + * __kmem_cache_create - Create a cache. * @name: A string which is used in /proc/slabinfo to identify this cache. * @size: The size of objects to be created in this cache. * @align: The required alignment for the objects. @@ -407,9 +407,9 @@ EXPORT_SYMBOL(kmem_cache_create_usercopy); * * Return: a pointer to the cache on success, NULL on failure. */ -struct kmem_cache * -kmem_cache_create(const char *name, unsigned int size, unsigned int align, - slab_flags_t flags, void (*ctor)(void *)) +struct kmem_cache *__kmem_cache_create(const char *name, unsigned int size, + unsigned int align, slab_flags_t flags, + void (*ctor)(void *)) { struct kmem_cache_args kmem_args = { .align = align, @@ -418,7 +418,7 @@ kmem_cache_create(const char *name, unsigned int size, unsigned int align, return __kmem_cache_create_args(name, size, &kmem_args, flags); } -EXPORT_SYMBOL(kmem_cache_create); +EXPORT_SYMBOL(__kmem_cache_create); /** * kmem_cache_create_rcu - Create a SLAB_TYPESAFE_BY_RCU cache. From patchwork Wed Sep 4 10:21:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790348 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86E47433C8 for ; Wed, 4 Sep 2024 10:22:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445362; cv=none; b=eZx23esnmb18uzcUbsSZ3666s4A92Ok232x3zTcFuHmAXhSwOlmpRB8+4qjwJTbm1IqHRZwjOUvblMlH3XHd3q8D83LidGd9EfEWsSMRe2DcjnE7ObupUgP+T1vbYeX8/VNi2uyTAa132dBUTM63lLNYTgTrOHy9Fc0muxNn+TA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445362; c=relaxed/simple; bh=MhdNhrflcCHIlYw4WJW1aVuEAIulx0BazYlkdUtItAE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PNeDBEDKCWsDq93eGrsTeUIxmgmFZalUngW62hvmZfVc6JLvcv9X1J1PP+ZHtayrK07dcEyFYtnMb/0Oh04ym1ywiLOUpcsGRa2jtdIRSWNf4/ljS8WnNkdyFc3wSUeksVRUdoQB4aRsyLaYQLHlOmhchFFAtU32H0R/SZROFoo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Y5mD2qJI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y5mD2qJI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 15AE3C4CEC6; Wed, 4 Sep 2024 10:22:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445362; bh=MhdNhrflcCHIlYw4WJW1aVuEAIulx0BazYlkdUtItAE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Y5mD2qJIqza8xUSaKCniqGT6GyvL5XOqqUqSen1zwDmgse53XxgDkB/IaZ6OsdYYY HYib9vb7fhSa0D2q4dqzyj/8xC1BxXzCl4IA1dLaiKDhsnZ+025pBjcaYYIt56ewzj ImyELYiUYXyDW0VEvr/BVWQpJGlLykJqI7wS3YOuAMzGJDRvoCGiHtUw2UzduwguzR 940m3KaGmUe9hQRpBz8qObWAKghEKOZLvzpTjFxNZAyCcxk3DlWHiHrlgM1BkhzusM Ae1/8K1zvEazD1LkNYQfJJ85PlwjmAy3UoiuSb8kDBKXmGvelED+4HYUZVLErZA8O1 nniIQcrF4IcHQ== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:18 +0200 Subject: [PATCH v3 13/17] file: port to struct kmem_cache_args Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-13-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=1043; i=brauner@kernel.org; h=from:subject:message-id; bh=MhdNhrflcCHIlYw4WJW1aVuEAIulx0BazYlkdUtItAE=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNn1OddWmEttJccRu4daT9PNdz/3OvbG1UZGZaKCS 9Ckp24WHaUsDGJcDLJiiiwO7Sbhcst5KjYbZWrAzGFlAhnCwMUpABMpcWD4n7e4eGv9V5XWu/md cffOJzq8mfAveRnXW84XuW2nErR+iDD8FV3y7dXBLL7PUvqJBmJsp52Yuva55RXdt1xyuPSwDXM hNwA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Port filp_cache to struct kmem_cache_args. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- fs/file_table.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/fs/file_table.c b/fs/file_table.c index 3ef558f27a1c..861c03608e83 100644 --- a/fs/file_table.c +++ b/fs/file_table.c @@ -511,9 +511,14 @@ EXPORT_SYMBOL(__fput_sync); void __init files_init(void) { - filp_cachep = kmem_cache_create_rcu("filp", sizeof(struct file), - offsetof(struct file, f_freeptr), - SLAB_HWCACHE_ALIGN | SLAB_PANIC | SLAB_ACCOUNT); + struct kmem_cache_args args = { + .use_freeptr_offset = true, + .freeptr_offset = offsetof(struct file, f_freeptr), + }; + + filp_cachep = kmem_cache_create("filp", sizeof(struct file), &args, + SLAB_HWCACHE_ALIGN | SLAB_PANIC | + SLAB_ACCOUNT | SLAB_TYPESAFE_BY_RCU); percpu_counter_init(&nr_files, 0, GFP_KERNEL); } From patchwork Wed Sep 4 10:21:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790349 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA26114658C for ; Wed, 4 Sep 2024 10:22:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445366; cv=none; b=rqrhqVj+bUJIXOXIvNCQwhUxE2vzlvl0bBxReigYyXvuvG0OPUxGUdJrH9lQCp5jruzvq2tPommAFHyGeLFPxefZSjG/veAocRdno5TIVSK7/ju4ty65FjjPx0IhFdkb41lfXNFST4jIeAbxIzTetd+OttKcmVksUtwG6a+XvZI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445366; c=relaxed/simple; bh=ob7tfSkKKaOFfsUNw54A8NnR55QWcUPRYRE0hGJ3b7c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=l9al6XwOUE2EwV/gvxm0k5X50lNCBsv9HyMr5juld+b+UjniUmUwNco73vRTcNo/yVmrUHtYYXpRUngy7QDjyW7+RNe8ERj8rEeaUriclfzGOJTybaxJiqiUIYTRWQYlhL0PCutSevj6P8TOB2Fn4+THyngdXanaKtF9MS/QDPU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JRb8YchA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JRb8YchA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DD028C4CEC2; Wed, 4 Sep 2024 10:22:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445366; bh=ob7tfSkKKaOFfsUNw54A8NnR55QWcUPRYRE0hGJ3b7c=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=JRb8YchA1ASqQ3gjaPNCb2ql7VsuECo83v0Lq+OcjYRob92h21b4kbDsuHmjJqRZW qoSlLkVDQOxp2eAaBmMBDJ19GHaa/AAaD1r4SxRhIz4RQm1a1ONGWK/lFYPQDNafQ3 j7LA0AAPa4Zan7gvOn0W+nVU8RSDISc4ZRmuAKoXHPNA2jDH+On/sNA6mDW1anw+KL 4W54tzz2lLjzH42yg2+1h2f2SD6WBOQRajfBNgC5Mb/51bSWfOYXipxW/tHfzOaDjG E3iiNtpx601BMCVUfuwfr5VoFgpooZwUU1wPcE8rttxqQ/My7vpCohzgD/t9OKIBvt SL8JA2SyIL6Wg== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:19 +0200 Subject: [PATCH v3 14/17] slab: remove kmem_cache_create_rcu() Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-14-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=3146; i=brauner@kernel.org; h=from:subject:message-id; bh=ob7tfSkKKaOFfsUNw54A8NnR55QWcUPRYRE0hGJ3b7c=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNl9h8VC/ZrFR6Mzc6ZER+rs5dr4x2dKw/Pwh95OW tcSTRtkOkpZGMS4GGTFFFkc2k3C5ZbzVGw2ytSAmcPKBDKEgYtTACYSf57hn45wDQOP2darxaJt NXwPNXi71KIeTUt7Nksl5J+fyc9l5xj+u3W69D4PXbsz+/GX+XE+Zgv5VPV9b/zcezK0m+nk2g/ 8TAA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Now that we have ported all users of kmem_cache_create_rcu() to struct kmem_cache_args the function is unused and can be removed. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- include/linux/slab.h | 3 --- mm/slab_common.c | 43 ------------------------------------------- 2 files changed, 46 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 4292d67094c3..1176b30cd4b2 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -270,9 +270,6 @@ struct kmem_cache *kmem_cache_create_usercopy(const char *name, slab_flags_t flags, unsigned int useroffset, unsigned int usersize, void (*ctor)(void *)); -struct kmem_cache *kmem_cache_create_rcu(const char *name, unsigned int size, - unsigned int freeptr_offset, - slab_flags_t flags); #define kmem_cache_create(__name, __object_size, __args, ...) \ _Generic((__args), \ diff --git a/mm/slab_common.c b/mm/slab_common.c index 418459927670..9133b9fafcb1 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -420,49 +420,6 @@ struct kmem_cache *__kmem_cache_create(const char *name, unsigned int size, } EXPORT_SYMBOL(__kmem_cache_create); -/** - * kmem_cache_create_rcu - Create a SLAB_TYPESAFE_BY_RCU cache. - * @name: A string which is used in /proc/slabinfo to identify this cache. - * @size: The size of objects to be created in this cache. - * @freeptr_offset: The offset into the memory to the free pointer - * @flags: SLAB flags - * - * Cannot be called within an interrupt, but can be interrupted. - * - * See kmem_cache_create() for an explanation of possible @flags. - * - * By default SLAB_TYPESAFE_BY_RCU caches place the free pointer outside - * of the object. This might cause the object to grow in size. Callers - * that have a reason to avoid this can specify a custom free pointer - * offset in their struct where the free pointer will be placed. - * - * Note that placing the free pointer inside the object requires the - * caller to ensure that no fields are invalidated that are required to - * guard against object recycling (See SLAB_TYPESAFE_BY_RCU for - * details.). - * - * Using zero as a value for @freeptr_offset is valid. To request no - * offset UINT_MAX must be specified. - * - * Note that @ctor isn't supported with custom free pointers as a @ctor - * requires an external free pointer. - * - * Return: a pointer to the cache on success, NULL on failure. - */ -struct kmem_cache *kmem_cache_create_rcu(const char *name, unsigned int size, - unsigned int freeptr_offset, - slab_flags_t flags) -{ - struct kmem_cache_args kmem_args = { - .freeptr_offset = freeptr_offset, - .use_freeptr_offset = true, - }; - - return __kmem_cache_create_args(name, size, &kmem_args, - flags | SLAB_TYPESAFE_BY_RCU); -} -EXPORT_SYMBOL(kmem_cache_create_rcu); - static struct kmem_cache *kmem_buckets_cache __ro_after_init; /** From patchwork Wed Sep 4 10:21:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790350 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 816F11CF5D7 for ; Wed, 4 Sep 2024 10:22:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445370; cv=none; b=d0krS8K11Qs9ensFiDIdiWtiVtroqZm0mRYVT2zA7E2i62tuVO9WUc5ewB5ifg7E0VEp+YEEiEIDDi6pmJiS2Hv73aTxAZBjvTB2grrzjPmeroIiJwXjqjMS4sWUicCH01lhsbCKscv1wYTkG1vaNHV1rIJt+bSfw8Tsw4s559Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445370; c=relaxed/simple; bh=otBPVDb5wSFqFX+emtlwUyqyvPQGIcX7yx9PMQsOqjw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=i1yU1f6E7UbhUmxxm9i1QcYtA76zyFey1Ex+Qp32dcSr5fkuNLAQRNlEloOixPH1lYLvBcZ++JoBLvHd5agTAW9GBW9TcsgAfl3au9J2Af54lBvop9IROJsag5q1x6k5hcVDR92IatQYG7jRTy+6opCYZoVnHr0VSppOF4AbhK8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bDGa/Ivc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bDGa/Ivc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D4F27C4CEC6; Wed, 4 Sep 2024 10:22:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445370; bh=otBPVDb5wSFqFX+emtlwUyqyvPQGIcX7yx9PMQsOqjw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=bDGa/IvcMSu4GQ4Irqb6p7xSlqntR0jiy/9X9+lLUcAnsYW4ttuniVZERzqaXh1Ud L5gf8yBN5UrenlR2eXCqo6BbRFAIaCB3HS3bV6FfMmD5e7Hqls1vAKAXTJzFrET2yd FmjH6Vci+98N//pLFtJiUJbDjmPzo5QXksmnLpMHV54wVjSgAK49ZZ81Te553Me8IL 62zOQ5vI8sERsbwrxVrAcjY02sCf/wPLkcUo0KNao3wJxWXMxsgvvXxx4uSN+6XxeB f+bf1rb1OTj7j3A3Yy4zzHM7G2px1cBaa4TlMgvyO5h4+KNnEJ34t/hrIibFJufYrC GFx5Z2ilhYzFA== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:20 +0200 Subject: [PATCH v3 15/17] slab: make kmem_cache_create_usercopy() static inline Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-15-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=4638; i=brauner@kernel.org; h=from:subject:message-id; bh=otBPVDb5wSFqFX+emtlwUyqyvPQGIcX7yx9PMQsOqjw=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNm97Phk78obioayNwPatj1TMDp+ZF9VYkVXH/O+B /Jt7LurOkpZGMS4GGTFFFkc2k3C5ZbzVGw2ytSAmcPKBDKEgYtTACby05ORYcKJiwsts2xFFzUX ugcwrBC8HHNuub3Sn9Q1jwL0kzvnP2X4n9DQlrdkZt8qM0XHb44egXPmcayWS+TjN9y5yis1Zgc zFwA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Make kmem_cache_create_usercopy() a static inline function. Signed-off-by: Christian Brauner --- include/linux/slab.h | 49 ++++++++++++++++++++++++++++++++++++++++++++----- mm/slab_common.c | 45 --------------------------------------------- 2 files changed, 44 insertions(+), 50 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 1176b30cd4b2..e224a1a9bcbc 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -265,11 +265,50 @@ struct kmem_cache *__kmem_cache_create_args(const char *name, struct kmem_cache *__kmem_cache_create(const char *name, unsigned int size, unsigned int align, slab_flags_t flags, void (*ctor)(void *)); -struct kmem_cache *kmem_cache_create_usercopy(const char *name, - unsigned int size, unsigned int align, - slab_flags_t flags, - unsigned int useroffset, unsigned int usersize, - void (*ctor)(void *)); + +/** + * kmem_cache_create_usercopy - Create a cache with a region suitable + * for copying to userspace + * @name: A string which is used in /proc/slabinfo to identify this cache. + * @size: The size of objects to be created in this cache. + * @align: The required alignment for the objects. + * @flags: SLAB flags + * @useroffset: Usercopy region offset + * @usersize: Usercopy region size + * @ctor: A constructor for the objects. + * + * Cannot be called within a interrupt, but can be interrupted. + * The @ctor is run when new pages are allocated by the cache. + * + * The flags are + * + * %SLAB_POISON - Poison the slab with a known test pattern (a5a5a5a5) + * to catch references to uninitialised memory. + * + * %SLAB_RED_ZONE - Insert `Red` zones around the allocated memory to check + * for buffer overruns. + * + * %SLAB_HWCACHE_ALIGN - Align the objects in this cache to a hardware + * cacheline. This can be beneficial if you're counting cycles as closely + * as davem. + * + * Return: a pointer to the cache on success, NULL on failure. + */ +static inline struct kmem_cache * +kmem_cache_create_usercopy(const char *name, unsigned int size, + unsigned int align, slab_flags_t flags, + unsigned int useroffset, unsigned int usersize, + void (*ctor)(void *)) +{ + struct kmem_cache_args kmem_args = { + .align = align, + .ctor = ctor, + .useroffset = useroffset, + .usersize = usersize, + }; + + return __kmem_cache_create_args(name, size, &kmem_args, flags); +} #define kmem_cache_create(__name, __object_size, __args, ...) \ _Generic((__args), \ diff --git a/mm/slab_common.c b/mm/slab_common.c index 9133b9fafcb1..3477a3918afd 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -337,51 +337,6 @@ struct kmem_cache *__kmem_cache_create_args(const char *name, } EXPORT_SYMBOL(__kmem_cache_create_args); -/** - * kmem_cache_create_usercopy - Create a cache with a region suitable - * for copying to userspace - * @name: A string which is used in /proc/slabinfo to identify this cache. - * @size: The size of objects to be created in this cache. - * @align: The required alignment for the objects. - * @flags: SLAB flags - * @useroffset: Usercopy region offset - * @usersize: Usercopy region size - * @ctor: A constructor for the objects. - * - * Cannot be called within a interrupt, but can be interrupted. - * The @ctor is run when new pages are allocated by the cache. - * - * The flags are - * - * %SLAB_POISON - Poison the slab with a known test pattern (a5a5a5a5) - * to catch references to uninitialised memory. - * - * %SLAB_RED_ZONE - Insert `Red` zones around the allocated memory to check - * for buffer overruns. - * - * %SLAB_HWCACHE_ALIGN - Align the objects in this cache to a hardware - * cacheline. This can be beneficial if you're counting cycles as closely - * as davem. - * - * Return: a pointer to the cache on success, NULL on failure. - */ -struct kmem_cache * -kmem_cache_create_usercopy(const char *name, unsigned int size, - unsigned int align, slab_flags_t flags, - unsigned int useroffset, unsigned int usersize, - void (*ctor)(void *)) -{ - struct kmem_cache_args kmem_args = { - .align = align, - .ctor = ctor, - .useroffset = useroffset, - .usersize = usersize, - }; - - return __kmem_cache_create_args(name, size, &kmem_args, flags); -} -EXPORT_SYMBOL(kmem_cache_create_usercopy); - /** * __kmem_cache_create - Create a cache. * @name: A string which is used in /proc/slabinfo to identify this cache. From patchwork Wed Sep 4 10:21:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790351 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82AF2126C08 for ; Wed, 4 Sep 2024 10:22:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445374; cv=none; b=hRs9qmgOV0AqPp9KH7o8wDP0Z6D1JG24o0lRl9rAWi3GNeYZVqXD9MZJQMlWcv5F8TIXL+qsV5BRaKBdY8kwo82V+4f8wDIrjtchpjPBg+WGvpyVRhjh85SXTUP/XzDTeO2VKd1347kXZ6CZsouolCT8T0x53i9ERBDbSvJ5Yrk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445374; c=relaxed/simple; bh=YjxO66hJ+BVi0bwZydpu1jfLI/x58A7MkM0YNuLTP98=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QVZYav90qWyqiY9kiu9X+6i5+aNkzCJDHrtUgX1o8nA7z8EMsIPRhXigzSjW8IIcRKj5n4x4JQt2WiUv43qyYW2hA9FPzWJ/L4OTxw+2HMxoJ9MZn7jfJqExGXo13PmVznJ/sK9CgO7YRY9nCb8LiCyo7PV4QhhrZ1QbruVhf2s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AEfBh4WZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AEfBh4WZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC9B5C4CECC; Wed, 4 Sep 2024 10:22:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445374; bh=YjxO66hJ+BVi0bwZydpu1jfLI/x58A7MkM0YNuLTP98=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=AEfBh4WZJ7pNUvjtrYEMp5wXlLriDBDcZYoIuOd4xpupauIsqPzuDzphXZBV4kLV6 TcOZFri2K0HWuchx6mo47Vr+pxy16ovN7pPwNNUVXmGx0I8umtB4a6zQLkYasZBer6 EgvxZrHDnb2A9oa6qB+INgGpn+OS9/RinxZzTzVs8cq0WTzCD8+08jglfGHct+ZnCJ 4iQw5og6J5qluAYydkJv1Us4UU8C7+2Z6LC7mmnDe55ZZgS5PESq77XyMSI0d0jhf4 QX4v/aadFfTR4hnlXSjUP/2yB1sLSbB2OOapeSjVCTVtJ/5ispRbDWwt1+Kuy0mOVV XzTmmcjshI5EQ== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:21 +0200 Subject: [PATCH v3 16/17] slab: make __kmem_cache_create() static inline Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-16-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=2885; i=brauner@kernel.org; h=from:subject:message-id; bh=YjxO66hJ+BVi0bwZydpu1jfLI/x58A7MkM0YNuLTP98=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNm9XjXnZfdPg79/rR6baEe3TpARutzz7eEu2WcW0 a/LpL4zdJSyMIhxMciKKbI4tJuEyy3nqdhslKkBM4eVCWQIAxenAExkHw/D/xKxXKeZEyNXL72z d+5/TQFl7seBn70y714QVhcpi3M61s3w38H/W79TX2P+vgf8U5M2bJ+yfGGtosqbiz4uVt1Cu89 m8AMA X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Make __kmem_cache_create() a static inline function. Signed-off-by: Christian Brauner --- include/linux/slab.h | 13 ++++++++++--- mm/slab_common.c | 38 -------------------------------------- 2 files changed, 10 insertions(+), 41 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index e224a1a9bcbc..70a0f530b89f 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -261,10 +261,17 @@ struct kmem_cache *__kmem_cache_create_args(const char *name, unsigned int object_size, struct kmem_cache_args *args, slab_flags_t flags); +static inline struct kmem_cache * +__kmem_cache_create(const char *name, unsigned int size, unsigned int align, + slab_flags_t flags, void (*ctor)(void *)) +{ + struct kmem_cache_args kmem_args = { + .align = align, + .ctor = ctor, + }; -struct kmem_cache *__kmem_cache_create(const char *name, unsigned int size, - unsigned int align, slab_flags_t flags, - void (*ctor)(void *)); + return __kmem_cache_create_args(name, size, &kmem_args, flags); +} /** * kmem_cache_create_usercopy - Create a cache with a region suitable diff --git a/mm/slab_common.c b/mm/slab_common.c index 3477a3918afd..30000dcf0736 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -337,44 +337,6 @@ struct kmem_cache *__kmem_cache_create_args(const char *name, } EXPORT_SYMBOL(__kmem_cache_create_args); -/** - * __kmem_cache_create - Create a cache. - * @name: A string which is used in /proc/slabinfo to identify this cache. - * @size: The size of objects to be created in this cache. - * @align: The required alignment for the objects. - * @flags: SLAB flags - * @ctor: A constructor for the objects. - * - * Cannot be called within a interrupt, but can be interrupted. - * The @ctor is run when new pages are allocated by the cache. - * - * The flags are - * - * %SLAB_POISON - Poison the slab with a known test pattern (a5a5a5a5) - * to catch references to uninitialised memory. - * - * %SLAB_RED_ZONE - Insert `Red` zones around the allocated memory to check - * for buffer overruns. - * - * %SLAB_HWCACHE_ALIGN - Align the objects in this cache to a hardware - * cacheline. This can be beneficial if you're counting cycles as closely - * as davem. - * - * Return: a pointer to the cache on success, NULL on failure. - */ -struct kmem_cache *__kmem_cache_create(const char *name, unsigned int size, - unsigned int align, slab_flags_t flags, - void (*ctor)(void *)) -{ - struct kmem_cache_args kmem_args = { - .align = align, - .ctor = ctor, - }; - - return __kmem_cache_create_args(name, size, &kmem_args, flags); -} -EXPORT_SYMBOL(__kmem_cache_create); - static struct kmem_cache *kmem_buckets_cache __ro_after_init; /** From patchwork Wed Sep 4 10:21:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Brauner X-Patchwork-Id: 13790352 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B58611CF7B5 for ; Wed, 4 Sep 2024 10:22:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445378; cv=none; b=mnOC3e892PKO8hRBJeokNZpq/Jyi43w3oWJ75KxWY5tFRMyJQV5WugNXcg3IXQgYjEoxV6PEMlF55yG4e7JP4cYiBN9ToO5oJ7rwWCrGr/NOMGthir+oDITW1IJ2ir/HjM8ucGXfuaFcORafyzxRCa4rpZHYPLKJgZsW+QByelw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725445378; c=relaxed/simple; bh=fyePhUeJgoPiuTJClIk+6OpwiFIuMNlpyvRSeOBVO/I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FpHQHLs0A6D1Ls5FF5NbSBQGysK3Uk51uG/1cUq28T5KjEHClYOn1BRWaZSrwx+8f9Bmy4MxbYQtNO/N4NdI4BxE4g4EVDdkfBWfi1/gUQnP594H1o+E7IXmiZDkuvR0kUnB1lF1S1bURqBgetHkj3YAjDAxpaCLeArUvrezSd8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lrPSuG+i; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lrPSuG+i" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EA1E4C4CEC2; Wed, 4 Sep 2024 10:22:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725445378; bh=fyePhUeJgoPiuTJClIk+6OpwiFIuMNlpyvRSeOBVO/I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=lrPSuG+iQHm9RvIy0WSzIC2PaCeqRuJifT/frNLorTp/Y+si8I22TATwm54EoN0D/ wF2P4/AMPbEdKK8eAUsdGJnFov14GBDKvKX233goB9TE5E+/nTOFTrtxZydchwpjK1 +0ROLJQWn2g/nElnAFEGmPaXKWER5LFg02TTLWE8FQYKvkqBiIj0JdJHaHPRCsfNeW z/GvHBdEK/9S6wPDwW2H85vXa1ChSZCNWtDyoMoRQhfHefYlRAxgclb4rvVPxQlxEL HaG1ACOdWYQDoDWn9c4EU632lFH62HODflmben8lJ2I1zQb7trHaHaGJlfVK93r/hW 5MIvW4MbSGP9w== From: Christian Brauner Date: Wed, 04 Sep 2024 12:21:22 +0200 Subject: [PATCH v3 17/17] io_uring: port to struct kmem_cache_args Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240904-work-kmem_cache_args-v3-17-05db2179a8c2@kernel.org> References: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> In-Reply-To: <20240904-work-kmem_cache_args-v3-0-05db2179a8c2@kernel.org> To: Vlastimil Babka , Jens Axboe , Jann Horn , Linus Torvalds , Mike Rapoport Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Christian Brauner X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=1645; i=brauner@kernel.org; h=from:subject:message-id; bh=fyePhUeJgoPiuTJClIk+6OpwiFIuMNlpyvRSeOBVO/I=; b=owGbwMvMwCU28Zj0gdSKO4sYT6slMaTdMNm954/H99mFK48VyBu05f33uZhWXFSRZ9Oa3Nj54 xK3WOHZjlIWBjEuBlkxRRaHdpNwueU8FZuNMjVg5rAygQxh4OIUgInU8jH8jxfh7RF5IrdmtcJi O4uEiafyNqqu2X9kyg62j7sr95lqdjH8D1pQYvVGdIZVkvf/uVXinWfkBDd91tHiPr1kRtkmwQZ rLgA= X-Developer-Key: i=brauner@kernel.org; a=openpgp; fpr=4880B8C9BD0E5106FC070F4F7B3C391EFEA93624 Port req_cachep to struct kmem_cache_args. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Vlastimil Babka Signed-off-by: Christian Brauner --- io_uring/io_uring.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 3942db160f18..d9d721d1424e 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -3638,6 +3638,11 @@ SYSCALL_DEFINE2(io_uring_setup, u32, entries, static int __init io_uring_init(void) { + struct kmem_cache_args kmem_args = { + .useroffset = offsetof(struct io_kiocb, cmd.data), + .usersize = sizeof_field(struct io_kiocb, cmd.data), + }; + #define __BUILD_BUG_VERIFY_OFFSET_SIZE(stype, eoffset, esize, ename) do { \ BUILD_BUG_ON(offsetof(stype, ename) != eoffset); \ BUILD_BUG_ON(sizeof_field(stype, ename) != esize); \ @@ -3722,12 +3727,9 @@ static int __init io_uring_init(void) * range, and HARDENED_USERCOPY will complain if we haven't * correctly annotated this range. */ - req_cachep = kmem_cache_create_usercopy("io_kiocb", - sizeof(struct io_kiocb), 0, - SLAB_HWCACHE_ALIGN | SLAB_PANIC | - SLAB_ACCOUNT | SLAB_TYPESAFE_BY_RCU, - offsetof(struct io_kiocb, cmd.data), - sizeof_field(struct io_kiocb, cmd.data), NULL); + req_cachep = kmem_cache_create("io_kiocb", sizeof(struct io_kiocb), &kmem_args, + SLAB_HWCACHE_ALIGN | SLAB_PANIC | SLAB_ACCOUNT | + SLAB_TYPESAFE_BY_RCU); io_buf_cachep = KMEM_CACHE(io_buffer, SLAB_HWCACHE_ALIGN | SLAB_PANIC | SLAB_ACCOUNT);