From patchwork Mon Nov 20 18:34:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13461880 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DE58C2BB3F for ; Mon, 20 Nov 2023 18:35:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E8976B039D; Mon, 20 Nov 2023 13:34:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0493C6B03AA; Mon, 20 Nov 2023 13:34:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD9086B039C; Mon, 20 Nov 2023 13:34:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5939D6B03AA for ; Mon, 20 Nov 2023 13:34:45 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3434480146 for ; Mon, 20 Nov 2023 18:34:45 +0000 (UTC) X-FDA: 81479183730.29.3E48D1E Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf20.hostedemail.com (Postfix) with ESMTP id 027AD1C0017 for ; Mon, 20 Nov 2023 18:34:42 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=UYk+nbOd; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=q32ZDjRb; dmarc=none; spf=pass (imf20.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700505283; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=f1ToxzDptD57b6Vsib7GAjPh04DR/y9DB96sNqNT2KE=; b=tRswNoHy+ZpEjqDTt+fuFtssWpfJHRNuwc21ZrlSJldvvRiYbFni5NXuI1A3WJrscuH5p0 BkJAFgXigG5qsGmij6qnkRmHzPJIQhQeOCEcdmJlhuIaoYwzi7RyRp5Turw6PYWznqZJ27 U5NTTE6smDyZitxL7ZZhs9f32yrxvuo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=UYk+nbOd; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=q32ZDjRb; dmarc=none; spf=pass (imf20.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700505283; a=rsa-sha256; cv=none; b=JWIbcv1AlL7dJeDhXmQT/6KMG8Q69aVAgTvHN785E2Ya2Ggl4zrgFSg5QncIOg0C/9fhYf BNXuO+y7pCSGtUwkQThgLZr35B6k7yg9iGJwb5QDILpIGswSLRgIkBptqeMhv66xbRbUcw yuH7aUCj791FIXEX+eZNvdaGK7JGRZA= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 2B5711F8A8; Mon, 20 Nov 2023 18:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1700505281; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f1ToxzDptD57b6Vsib7GAjPh04DR/y9DB96sNqNT2KE=; b=UYk+nbOdpwt7lTEROM4R9R/VybWxUsZNmkJ6xbBvRHIlibYVyjKf0xMYUUD5mRAxzZ1nYm 8YuyVrnDdN/4r8f2BpnRbBT9UOibhIk2oeEW/xMg4iUETJa6r240JQQlVhh/+wrO5nMNaz GGaxETZpNTnVjpcdQee4/bZSKagO6iw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1700505281; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f1ToxzDptD57b6Vsib7GAjPh04DR/y9DB96sNqNT2KE=; b=q32ZDjRbFefawcKstIaTcDIBwhSahPd7d0kPJGhClVzYjFjtGumHVTj08Yz7Fh3lMJpylO PItDrslB5HufLZCw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id ED71413499; Mon, 20 Nov 2023 18:34:40 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id mOKCOcCmW2UUMgAAMHmgww (envelope-from ); Mon, 20 Nov 2023 18:34:40 +0000 From: Vlastimil Babka Date: Mon, 20 Nov 2023 19:34:18 +0100 Subject: [PATCH v2 07/21] mm/slab: remove CONFIG_SLAB code from slab common code MIME-Version: 1.0 Message-Id: <20231120-slab-remove-slab-v2-7-9c9c70177183@suse.cz> References: <20231120-slab-remove-slab-v2-0-9c9c70177183@suse.cz> In-Reply-To: <20231120-slab-remove-slab-v2-0-9c9c70177183@suse.cz> To: David Rientjes , Christoph Lameter , Pekka Enberg , Joonsoo Kim Cc: Andrew Morton , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Roman Gushchin , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Marco Elver , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Kees Cook , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org, linux-hardening@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.12.4 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 027AD1C0017 X-Stat-Signature: kwmx3f97fkatetb4d5n1insm5cedggdj X-Rspam-User: X-HE-Tag: 1700505282-227034 X-HE-Meta: U2FsdGVkX1/nIq/c1yphL76rnpW9qRzVGt0xUbmmei0H17Mmms3yO9QdfUiZQB9h6FdjjPhQ/uUMOdrpqsBG3pI85D13sxcxc3KBmGmNQZGaTU+yI0kBcr3HN4/VY1iG/ezx+2AmmxxwO1afq2PsWtN2cLNCFrJqfW7NHHwcrdLhr2lRCClmTKZ7dSNT60xUsY1mFjVhrtqTOL0TbWPjGvs22CR+WruGn0eehaOAHl88TUtv7DJfHqGWdFq0LXoHcxmZuw1uL8MdS/5oTa5rATMMSfxVmpUODhr23BYEJRe5JCqNnEyIgYnXZrxnS5Vhsh93uI2v3uQtCqA9a0DFiDFRzfa+1eGZ2NaHA3X19yceB4ECEcpnwdEkD3n/7PAPCE+45fgQJsqFNcnzG4raEEiRNZkXNsdCATfIWx+aBhIcHIv48fB46grJKwBrIkfNBNOATaIeWOdKObGXvo6eNcAT2ZQbjNZaufTmhWJgE6zoRoW1NByWKWc/Fpda9ZmmrGkf04NNM35yTLjpZyH3UucsaNUmuWqmd83Qy/w6G0znd3/TRdMSmiccyp/42tMaa/BsN61hn0Y66qU7b8+ZRMdzPT1hnGOzudPehjNNpuX2vvLrNTiQHc4M+xbqb722kvutaVJIXbwPL1op/EMjAJW6/iw4IECRtF4yGdYSR6LpxZ2VzyaoplRT1gnWnvjmdBqrh8XhcGmuIEHMPfXIgasTPKj9PqrBMvmpHVpkeOehuOoi5jyAU6k/GScBsO1ATTAo9BhxBpzgPeSp9cYU8tvYRDyjXgRYyV8kLHKxlpz+d/C3Pb87CgRj7LuJX99zuJGNAMv0HOMzX+gz8wl+0FdeBR1HTnQSZNdZOilv1goVXoeFQGYpkrLOSPU/1RtRboXixsyZIRD2uLngFUvzrI7Yt3UjdySgwqrDtJxoRXNgVD3J0/E3906od+2ak9IwORGVFJHzKycdQZSK+6q hwTPaGBo x8dOXhbSKF4j0F3TVI85UoH3ldhWshsXqGh9614H6TBhSf/unmqESUJ6eYlXz8tOuNsE+mbwo345IOimaU6zcPquGMvT24X6w7mjFV+2a9BhTVNBJ7u0upPJnk+yMor+JVkBrDeTSMvPdqzm8iFLaWlxNuv8yP1sOtrPuZYF7huNqTa3cDGowG3D0fvm1ILi/XzBnLv+pMblS8Jf55JJ9X9Kmj/R2/+sEMD7A84hKB+UK3/QmBvu+JC64KJCMVBnJWIvANnXeD3N7XiqdFLxq1jVaBE2BA7ihN5dGBsftKYG9fNGcmumzxdPNp1KjAvj7NWTXrR/Mn8aDkI2BnYTJ+N/CRYG4xfm44HOfFM/KDnR5HxK7EY3OR3ymnQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In slab_common.c and slab.h headers, we can now remove all code behind CONFIG_SLAB and CONFIG_DEBUG_SLAB ifdefs, and remove all CONFIG_SLUB ifdefs. Reviewed-by: Kees Cook Signed-off-by: Vlastimil Babka Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- include/linux/slab.h | 14 ++--------- mm/slab.h | 69 ++++------------------------------------------------ mm/slab_common.c | 22 ++--------------- 3 files changed, 9 insertions(+), 96 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 34e43cddc520..b2015d0e01ad 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -24,7 +24,7 @@ /* * Flags to pass to kmem_cache_create(). - * The ones marked DEBUG are only valid if CONFIG_DEBUG_SLAB is set. + * The ones marked DEBUG need CONFIG_SLUB_DEBUG enabled, otherwise are no-op */ /* DEBUG: Perform (expensive) checks on alloc/free */ #define SLAB_CONSISTENCY_CHECKS ((slab_flags_t __force)0x00000100U) @@ -302,25 +302,15 @@ static inline unsigned int arch_slab_minalign(void) * Kmalloc array related definitions */ -#ifdef CONFIG_SLAB /* - * SLAB and SLUB directly allocates requests fitting in to an order-1 page + * SLUB directly allocates requests fitting in to an order-1 page * (PAGE_SIZE*2). Larger requests are passed to the page allocator. */ #define KMALLOC_SHIFT_HIGH (PAGE_SHIFT + 1) #define KMALLOC_SHIFT_MAX (MAX_ORDER + PAGE_SHIFT) #ifndef KMALLOC_SHIFT_LOW -#define KMALLOC_SHIFT_LOW 5 -#endif -#endif - -#ifdef CONFIG_SLUB -#define KMALLOC_SHIFT_HIGH (PAGE_SHIFT + 1) -#define KMALLOC_SHIFT_MAX (MAX_ORDER + PAGE_SHIFT) -#ifndef KMALLOC_SHIFT_LOW #define KMALLOC_SHIFT_LOW 3 #endif -#endif /* Maximum allocatable size */ #define KMALLOC_MAX_SIZE (1UL << KMALLOC_SHIFT_MAX) diff --git a/mm/slab.h b/mm/slab.h index 3d07fb428393..014c36ea51fa 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -42,21 +42,6 @@ typedef union { struct slab { unsigned long __page_flags; -#if defined(CONFIG_SLAB) - - struct kmem_cache *slab_cache; - union { - struct { - struct list_head slab_list; - void *freelist; /* array of free object indexes */ - void *s_mem; /* first object */ - }; - struct rcu_head rcu_head; - }; - unsigned int active; - -#elif defined(CONFIG_SLUB) - struct kmem_cache *slab_cache; union { struct { @@ -91,10 +76,6 @@ struct slab { }; unsigned int __unused; -#else -#error "Unexpected slab allocator configured" -#endif - atomic_t __page_refcount; #ifdef CONFIG_MEMCG unsigned long memcg_data; @@ -111,7 +92,7 @@ SLAB_MATCH(memcg_data, memcg_data); #endif #undef SLAB_MATCH static_assert(sizeof(struct slab) <= sizeof(struct page)); -#if defined(system_has_freelist_aba) && defined(CONFIG_SLUB) +#if defined(system_has_freelist_aba) static_assert(IS_ALIGNED(offsetof(struct slab, freelist), sizeof(freelist_aba_t))); #endif @@ -228,13 +209,7 @@ static inline size_t slab_size(const struct slab *slab) return PAGE_SIZE << slab_order(slab); } -#ifdef CONFIG_SLAB -#include -#endif - -#ifdef CONFIG_SLUB #include -#endif #include #include @@ -320,26 +295,16 @@ static inline bool is_kmalloc_cache(struct kmem_cache *s) SLAB_CACHE_DMA32 | SLAB_PANIC | \ SLAB_TYPESAFE_BY_RCU | SLAB_DEBUG_OBJECTS ) -#if defined(CONFIG_DEBUG_SLAB) -#define SLAB_DEBUG_FLAGS (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER) -#elif defined(CONFIG_SLUB_DEBUG) +#ifdef CONFIG_SLUB_DEBUG #define SLAB_DEBUG_FLAGS (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_CONSISTENCY_CHECKS) #else #define SLAB_DEBUG_FLAGS (0) #endif -#if defined(CONFIG_SLAB) -#define SLAB_CACHE_FLAGS (SLAB_MEM_SPREAD | SLAB_NOLEAKTRACE | \ - SLAB_RECLAIM_ACCOUNT | SLAB_TEMPORARY | \ - SLAB_ACCOUNT | SLAB_NO_MERGE) -#elif defined(CONFIG_SLUB) #define SLAB_CACHE_FLAGS (SLAB_NOLEAKTRACE | SLAB_RECLAIM_ACCOUNT | \ SLAB_TEMPORARY | SLAB_ACCOUNT | \ SLAB_NO_USER_FLAGS | SLAB_KMALLOC | SLAB_NO_MERGE) -#else -#define SLAB_CACHE_FLAGS (SLAB_NOLEAKTRACE) -#endif /* Common flags available with current configuration */ #define CACHE_CREATE_MASK (SLAB_CORE_FLAGS | SLAB_DEBUG_FLAGS | SLAB_CACHE_FLAGS) @@ -672,18 +637,14 @@ size_t __ksize(const void *objp); static inline size_t slab_ksize(const struct kmem_cache *s) { -#ifndef CONFIG_SLUB - return s->object_size; - -#else /* CONFIG_SLUB */ -# ifdef CONFIG_SLUB_DEBUG +#ifdef CONFIG_SLUB_DEBUG /* * Debugging requires use of the padding between object * and whatever may come after it. */ if (s->flags & (SLAB_RED_ZONE | SLAB_POISON)) return s->object_size; -# endif +#endif if (s->flags & SLAB_KASAN) return s->object_size; /* @@ -697,7 +658,6 @@ static inline size_t slab_ksize(const struct kmem_cache *s) * Else we can use all the padding etc for the allocation */ return s->size; -#endif } static inline struct kmem_cache *slab_pre_alloc_hook(struct kmem_cache *s, @@ -775,23 +735,6 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, * The slab lists for all objects. */ struct kmem_cache_node { -#ifdef CONFIG_SLAB - raw_spinlock_t list_lock; - struct list_head slabs_partial; /* partial list first, better asm code */ - struct list_head slabs_full; - struct list_head slabs_free; - unsigned long total_slabs; /* length of all slab lists */ - unsigned long free_slabs; /* length of free slab list only */ - unsigned long free_objects; - unsigned int free_limit; - unsigned int colour_next; /* Per-node cache coloring */ - struct array_cache *shared; /* shared per node */ - struct alien_cache **alien; /* on other nodes */ - unsigned long next_reap; /* updated without locking */ - int free_touched; /* updated without locking */ -#endif - -#ifdef CONFIG_SLUB spinlock_t list_lock; unsigned long nr_partial; struct list_head partial; @@ -800,8 +743,6 @@ struct kmem_cache_node { atomic_long_t total_objects; struct list_head full; #endif -#endif - }; static inline struct kmem_cache_node *get_node(struct kmem_cache *s, int node) @@ -818,7 +759,7 @@ static inline struct kmem_cache_node *get_node(struct kmem_cache *s, int node) if ((__n = get_node(__s, __node))) -#if defined(CONFIG_SLAB) || defined(CONFIG_SLUB_DEBUG) +#ifdef CONFIG_SLUB_DEBUG void dump_unreclaimable_slab(void); #else static inline void dump_unreclaimable_slab(void) diff --git a/mm/slab_common.c b/mm/slab_common.c index 8d431193c273..63b8411db7ce 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -71,10 +71,8 @@ static int __init setup_slab_merge(char *str) return 1; } -#ifdef CONFIG_SLUB __setup_param("slub_nomerge", slub_nomerge, setup_slab_nomerge, 0); __setup_param("slub_merge", slub_merge, setup_slab_merge, 0); -#endif __setup("slab_nomerge", setup_slab_nomerge); __setup("slab_merge", setup_slab_merge); @@ -197,10 +195,6 @@ struct kmem_cache *find_mergeable(unsigned int size, unsigned int align, if (s->size - size >= sizeof(void *)) continue; - if (IS_ENABLED(CONFIG_SLAB) && align && - (align > s->align || s->align % align)) - continue; - return s; } return NULL; @@ -1222,12 +1216,8 @@ void cache_random_seq_destroy(struct kmem_cache *cachep) } #endif /* CONFIG_SLAB_FREELIST_RANDOM */ -#if defined(CONFIG_SLAB) || defined(CONFIG_SLUB_DEBUG) -#ifdef CONFIG_SLAB -#define SLABINFO_RIGHTS (0600) -#else +#ifdef CONFIG_SLUB_DEBUG #define SLABINFO_RIGHTS (0400) -#endif static void print_slabinfo_header(struct seq_file *m) { @@ -1235,18 +1225,10 @@ static void print_slabinfo_header(struct seq_file *m) * Output format version, so at least we can change it * without _too_ many complaints. */ -#ifdef CONFIG_DEBUG_SLAB - seq_puts(m, "slabinfo - version: 2.1 (statistics)\n"); -#else seq_puts(m, "slabinfo - version: 2.1\n"); -#endif seq_puts(m, "# name "); seq_puts(m, " : tunables "); seq_puts(m, " : slabdata "); -#ifdef CONFIG_DEBUG_SLAB - seq_puts(m, " : globalstat "); - seq_puts(m, " : cpustat "); -#endif seq_putc(m, '\n'); } @@ -1370,7 +1352,7 @@ static int __init slab_proc_init(void) } module_init(slab_proc_init); -#endif /* CONFIG_SLAB || CONFIG_SLUB_DEBUG */ +#endif /* CONFIG_SLUB_DEBUG */ static __always_inline __realloc_size(2) void * __do_krealloc(const void *p, size_t new_size, gfp_t flags)