From patchwork Fri Feb 14 16:27:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13975229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15152C021A4 for ; Fri, 14 Feb 2025 16:28:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A9B556B0095; Fri, 14 Feb 2025 11:28:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9F72D280005; Fri, 14 Feb 2025 11:28:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84BA9280004; Fri, 14 Feb 2025 11:28:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 613226B0095 for ; Fri, 14 Feb 2025 11:28:11 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6081580628 for ; Fri, 14 Feb 2025 16:28:10 +0000 (UTC) X-FDA: 83119082340.25.EF6B518 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf02.hostedemail.com (Postfix) with ESMTP id 2E74280006 for ; Fri, 14 Feb 2025 16:28:07 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=u8jqOZ8I; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=0qa1KWSV; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=u8jqOZ8I; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=0qa1KWSV; spf=pass (imf02.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739550488; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NLtEQ/q9lO1Z0OFCXgBkkhN+q6XZ6NGPEru6gyzhnws=; b=DFkYbx5UNdWWRgm1lLsLe0hf2lkbXaX4IfqPACmBUIcHDE3jR6NR90Vvn0eYUxHzE8Y+dZ 2SVro9oO+JqkSMvU1gtHnO2xUdNGKR2CIirB2aSUlInxhwRH8YfaIcoEcsxaPJUPEkbReM JSY4WCyOpEMAXVimXj2/nDBIHA/H/7A= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=u8jqOZ8I; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=0qa1KWSV; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=u8jqOZ8I; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=0qa1KWSV; spf=pass (imf02.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739550488; a=rsa-sha256; cv=none; b=2q4VB8owux6f717PYzpkchqkF3iiqhKERIcq9C9cL2ykBjbCTb3oOZIFebuuLq1cWs5f99 xGL0q3WFD/p8Kq/iwZxsu3FKVqBErEU2NrGJa1UPvrOMVyDRlY4C+mUvBNQOX7naRnd4Pi 9RS3gxWnK3V3/fRFgXORyQBOgyJXMA0= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 1060721181; Fri, 14 Feb 2025 16:27:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1739550463; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NLtEQ/q9lO1Z0OFCXgBkkhN+q6XZ6NGPEru6gyzhnws=; b=u8jqOZ8I83tYUP2QKPEWC8KiQEnlRgtDT8KaFa8nU1mFZO2ZOQNPO5cz1KrcEsvZ9IoVdd 6BIKrHuXaBDHLGMBM3NzrdF1jtgvHr6Mg7ave+rF2evhrGtVtuB3gh8ZG0W3yvc3nH3dG1 fw1yxlK9Crhx07OsvaS3eUMyX/j9iA0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1739550463; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NLtEQ/q9lO1Z0OFCXgBkkhN+q6XZ6NGPEru6gyzhnws=; b=0qa1KWSVi4cqBGYpYfsLyobPouXCrxLgFIk0jemoeuQ9nH/qplkae8T9y1ScxKDhKkCX/V 9nZh3P+ap91ZDgAA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1739550463; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NLtEQ/q9lO1Z0OFCXgBkkhN+q6XZ6NGPEru6gyzhnws=; b=u8jqOZ8I83tYUP2QKPEWC8KiQEnlRgtDT8KaFa8nU1mFZO2ZOQNPO5cz1KrcEsvZ9IoVdd 6BIKrHuXaBDHLGMBM3NzrdF1jtgvHr6Mg7ave+rF2evhrGtVtuB3gh8ZG0W3yvc3nH3dG1 fw1yxlK9Crhx07OsvaS3eUMyX/j9iA0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1739550463; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NLtEQ/q9lO1Z0OFCXgBkkhN+q6XZ6NGPEru6gyzhnws=; b=0qa1KWSVi4cqBGYpYfsLyobPouXCrxLgFIk0jemoeuQ9nH/qplkae8T9y1ScxKDhKkCX/V 9nZh3P+ap91ZDgAA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id EFC7B13AF0; Fri, 14 Feb 2025 16:27:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kM5IOv5ur2eHSAAAD6G6ig (envelope-from ); Fri, 14 Feb 2025 16:27:42 +0000 From: Vlastimil Babka Date: Fri, 14 Feb 2025 17:27:44 +0100 Subject: [PATCH RFC v2 08/10] tools: Add testing support for changes to rcu and slab for sheaves MIME-Version: 1.0 Message-Id: <20250214-slub-percpu-caches-v2-8-88592ee0966a@suse.cz> References: <20250214-slub-percpu-caches-v2-0-88592ee0966a@suse.cz> In-Reply-To: <20250214-slub-percpu-caches-v2-0-88592ee0966a@suse.cz> To: Suren Baghdasaryan , "Liam R. Howlett" , Christoph Lameter , David Rientjes Cc: Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Uladzislau Rezki , linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, maple-tree@lists.infradead.org, "Liam R. Howlett" X-Mailer: b4 0.14.2 X-Rspam-User: X-Rspamd-Queue-Id: 2E74280006 X-Stat-Signature: t35p9gf9kwrhhepgffsu5fxdsr8ru68i X-Rspamd-Server: rspam03 X-HE-Tag: 1739550487-982633 X-HE-Meta: U2FsdGVkX19Fedgd22mJcdqJNlifLok0xL/UQowIV74PHxz2wXOuorxvYnk0G3bo20uqlDnNhNUn3sOiQBpuIDQvMb11Vsg0mYim6FMPEKVnIM7YTaNudT8gatbazaYqAiPJ+tyDCignT87Cs59Fg7b45KHYmJiVLZGdu6NNeftcObqZfe0VG03P2/ez/SF0eAE1qbbQBcHG3oOK2Fph7jPhpalEXxbIEtix6i+UTobJVWM9bNSI/V+HVpDCuqUEB4WV8aeDCtwmNF7hzM54S73Y7I/wd/kkNKlCU3Fq2Z/X3NDdlgyMB2j/1KJ4IwpTNoWx+9CFF4r3W/VBysGMNGUoeQrY8n5NQNwxmqY8ZDCvh25G8yIjvZIwNBZGBFF8dBK+4lk22HT/vjmHh6PQN6O0/NwCwWlOuIsdzT3sHqZJt2uPSmDPj1iyX1cu2sDCvAE+bmNE0+YnT1zrV2Gpc7yRvHU+r9AsVA/5kDLvT4i4Y+65Oq1EpQ8GLKWVSLL7sZj+RJo5l8CCw3WI9QGMmKqi765Td8ha+o/As9UxxihxaHCT/XI+KEDNDkn9JrKNWTzt/vx7i5RNMRJUWLcBnmerxxcRJRgsfdkZIYUvi2afA+de0CHBXruGBI5zuuftv9r/WKp4hEspjglhAxsyDbhPNfGH5NdMFcixNAracvwd5YjioKQvt71LHi8+qHX99R5x1lee39/IJw4EyHqp8L/2jOMX49iDFRKsWKLdExNOiRdxO0R9HocigKN3cSx9++gp6+OmiikeukkF5ZHrUYifChoMPcgQbyeYuodCfS//t09UhmfaYgxfr46HvvvpByZ4N6bVud/QhEU8K1o/iGWTsFJrlcdbK6nPQ7I6c0rMj3N/ympmcgNdN51FEAPqmT7arxuYKsYwss5EpXtxFABROsp0un0saTd6n7VVJ383EHWPkPe7VFZLDAkVee8GyJa3ScKa1cs6nK8Wv++ kFUrT/dV C35XBMywqqCHFDs+Fs1yVgr6Nryc17ce5vRdVOfpHDkQrkwJeMe/ElqAI6ZsN5lgMOO/2FMQmws4ZV7zT5nBUIyJXJ8q5S8S7yTChOcSJuzt2q4hMD+AcMgnzr0fTrTLqCIKEYYGZoL+3RMLZMiwco6DUENiN/F3q68zrytso7NSjC2qKoMCfxLAY6Sea+OXFKVRl2uZhf/ligKxD5WdcwZry8/vkpj6J+aL1iC+2sOTNR31irJyMfJ2Dbe/uKAYoSmWDyY6VXYZzFTXVozXlolDh0gbXHoVACHiLn+HJki3VyfLIEn52xkLItB5CY7Pfp2jZPfATvnRFnkjMv1QH69Rtb4BZCK2BMsWYLAgkhdGC3Q3uvPo9TjMQBw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Liam R. Howlett" Make testing work for the slab and rcu changes that have come in with the sheaves work. This only works with one kmem_cache, and only the first one used. Subsequent setting of keme_cache will not update the active kmem_cache and will be silently dropped because there are other tests which happen after the kmem_cache of interest is set. The saved active kmem_cache is used in the rcu callback, which passes the object to be freed. The rcu call takes the rcu_head, which is passed in as the field in the struct (in this case rcu in the maple tree node), which is calculated by pointer math. The offset of which is saved (in a global variable) for restoring the node pointer on the callback after the rcu grace period expires. Don't use any of this outside of testing, please. Signed-off-by: Liam R. Howlett --- tools/include/linux/slab.h | 41 ++++++++++++++++++++++++++++++++--- tools/testing/shared/linux.c | 24 ++++++++++++++++---- tools/testing/shared/linux/rcupdate.h | 22 +++++++++++++++++++ 3 files changed, 80 insertions(+), 7 deletions(-) diff --git a/tools/include/linux/slab.h b/tools/include/linux/slab.h index 51b25e9c4ec7b66bdf4c68cc1353c6faf1ca7bb8..a475364cfd9fcdb10db252aab18ea3a620326b6b 100644 --- a/tools/include/linux/slab.h +++ b/tools/include/linux/slab.h @@ -22,6 +22,12 @@ enum slab_state { FULL }; +struct kmem_cache_args { + unsigned int align; + unsigned int sheaf_capacity; + void (*ctor)(void *); +}; + static inline void *kzalloc(size_t size, gfp_t gfp) { return kmalloc(size, gfp | __GFP_ZERO); @@ -36,9 +42,38 @@ static inline void *kmem_cache_alloc(struct kmem_cache *cachep, int flags) } void kmem_cache_free(struct kmem_cache *cachep, void *objp); -struct kmem_cache *kmem_cache_create(const char *name, unsigned int size, - unsigned int align, unsigned int flags, - void (*ctor)(void *)); + +struct kmem_cache * +__kmem_cache_create_args(const char *name, unsigned int size, + struct kmem_cache_args *args, unsigned int flags); + +/* If NULL is passed for @args, use this variant with default arguments. */ +static inline struct kmem_cache * +__kmem_cache_default_args(const char *name, unsigned int size, + struct kmem_cache_args *args, unsigned int flags) +{ + struct kmem_cache_args kmem_default_args = {}; + + return __kmem_cache_create_args(name, size, &kmem_default_args, flags); +} + +static inline struct kmem_cache * +__kmem_cache_create(const char *name, unsigned int size, unsigned int align, + unsigned int flags, void (*ctor)(void *)) +{ + struct kmem_cache_args kmem_args = { + .align = align, + .ctor = ctor, + }; + + return __kmem_cache_create_args(name, size, &kmem_args, flags); +} + +#define kmem_cache_create(__name, __object_size, __args, ...) \ + _Generic((__args), \ + struct kmem_cache_args *: __kmem_cache_create_args, \ + void *: __kmem_cache_default_args, \ + default: __kmem_cache_create)(__name, __object_size, __args, __VA_ARGS__) void kmem_cache_free_bulk(struct kmem_cache *cachep, size_t size, void **list); int kmem_cache_alloc_bulk(struct kmem_cache *cachep, gfp_t gfp, size_t size, diff --git a/tools/testing/shared/linux.c b/tools/testing/shared/linux.c index 66dbb362385f3c3d923233448cc591adfe6dc9e7..9f5fd722f27f1d3877be8927be30409cd74ab3c3 100644 --- a/tools/testing/shared/linux.c +++ b/tools/testing/shared/linux.c @@ -20,6 +20,7 @@ struct kmem_cache { pthread_mutex_t lock; unsigned int size; unsigned int align; + unsigned int sheaf_capacity; int nr_objs; void *objs; void (*ctor)(void *); @@ -31,6 +32,8 @@ struct kmem_cache { void *private; }; +static struct kmem_cache *kmem_active = NULL; + void kmem_cache_set_callback(struct kmem_cache *cachep, void (*callback)(void *)) { cachep->callback = callback; @@ -147,6 +150,14 @@ void kmem_cache_free(struct kmem_cache *cachep, void *objp) pthread_mutex_unlock(&cachep->lock); } +void kmem_cache_free_active(void *objp) +{ + if (!kmem_active) + printf("WARNING: No active kmem_cache\n"); + + kmem_cache_free(kmem_active, objp); +} + void kmem_cache_free_bulk(struct kmem_cache *cachep, size_t size, void **list) { if (kmalloc_verbose) @@ -234,23 +245,28 @@ int kmem_cache_alloc_bulk(struct kmem_cache *cachep, gfp_t gfp, size_t size, } struct kmem_cache * -kmem_cache_create(const char *name, unsigned int size, unsigned int align, - unsigned int flags, void (*ctor)(void *)) +__kmem_cache_create_args(const char *name, unsigned int size, + struct kmem_cache_args *args, + unsigned int flags) { struct kmem_cache *ret = malloc(sizeof(*ret)); pthread_mutex_init(&ret->lock, NULL); ret->size = size; - ret->align = align; + ret->align = args->align; + ret->sheaf_capacity = args->sheaf_capacity; ret->nr_objs = 0; ret->nr_allocated = 0; ret->nr_tallocated = 0; ret->objs = NULL; - ret->ctor = ctor; + ret->ctor = args->ctor; ret->non_kernel = 0; ret->exec_callback = false; ret->callback = NULL; ret->private = NULL; + if (!kmem_active) + kmem_active = ret; + return ret; } diff --git a/tools/testing/shared/linux/rcupdate.h b/tools/testing/shared/linux/rcupdate.h index fed468fb0c78db6f33fb1900c7110ab5f3c19c65..c95e2f0bbd93798e544d7d34e0823ed68414f924 100644 --- a/tools/testing/shared/linux/rcupdate.h +++ b/tools/testing/shared/linux/rcupdate.h @@ -9,4 +9,26 @@ #define rcu_dereference_check(p, cond) rcu_dereference(p) #define RCU_INIT_POINTER(p, v) do { (p) = (v); } while (0) +void kmem_cache_free_active(void *objp); +static unsigned long kfree_cb_offset = 0; + +static inline void kfree_rcu_cb(struct rcu_head *head) +{ + void *objp = (void *) ((unsigned long)head - kfree_cb_offset); + + kmem_cache_free_active(objp); +} + +#ifndef offsetof +#define offsetof(TYPE, MEMBER) __builtin_offsetof(TYPE, MEMBER) +#endif + +#define kfree_rcu(ptr, rhv) \ +do { \ + if (!kfree_cb_offset) \ + kfree_cb_offset = offsetof(typeof(*(ptr)), rhv); \ + \ + call_rcu(&ptr->rhv, kfree_rcu_cb); \ +} while (0) + #endif