[56/62] mm: Convert slub to use struct slab

Message ID	20211004134650.4031813-57-willy@infradead.org (mailing list archive)
State	New
Headers	show Return-Path: <SRS0=Fowb=OY=kvack.org=owner-linux-mm@kernel.org> DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1304D61186 From: "Matthew Wilcox (Oracle)" <willy@infradead.org> To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org> Subject: [PATCH 56/62] mm: Convert slub to use struct slab Date: Mon, 4 Oct 2021 14:46:44 +0100 Message-Id: <20211004134650.4031813-57-willy@infradead.org> In-Reply-To: <20211004134650.4031813-1-willy@infradead.org> References: <20211004134650.4031813-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org Precedence: bulk
Series	Separate struct slab from struct page \| expand [00/62] Separate struct slab from struct page [01/62] mm: Convert page_to_section() to pgflags_section() [02/62] mm: Add pgflags_nid() [03/62] mm: Split slab into its own type [04/62] mm: Add account_slab() and unaccount_slab() [05/62] mm: Convert virt_to_cache() to use struct slab [06/62] mm: Convert __ksize() to struct slab [07/62] mm: Use struct slab in kmem_obj_info() [08/62] mm: Convert check_heap_object() to use struct slab [09/62] mm/slub: Convert process_slab() to take a struct slab [10/62] mm/slub: Convert detached_freelist to use a struct slab [11/62] mm/slub: Convert kfree() to use a struct slab [12/62] mm/slub: Convert __slab_free() to take a struct slab [13/62] mm/slub: Convert new_slab() to return a struct slab [14/62] mm/slub: Convert early_kmem_cache_node_alloc() to use struct slab [15/62] mm/slub: Convert kmem_cache_cpu to struct slab [16/62] mm/slub: Convert show_slab_objects() to struct slab [17/62] mm/slub: Convert validate_slab() to take a struct slab [18/62] mm/slub: Convert count_partial() to struct slab [19/62] mm/slub: Convert bootstrap() to struct slab [20/62] mm/slub: Convert __kmem_cache_do_shrink() to struct slab [21/62] mm/slub: Convert free_partial() to use struct slab [22/62] mm/slub: Convert list_slab_objects() to take a struct slab [23/62] mm/slub: Convert slab_alloc_node() to use a struct slab [24/62] mm/slub: Convert get_freelist() to take a struct slab [25/62] mm/slub: Convert node_match() to take a struct slab [26/62] mm/slub: Convert slab flushing to struct slab [27/62] mm/slub: Convert __unfreeze_partials to take a struct slab [28/62] mm/slub: Convert deactivate_slab() to take a struct slab [29/62] mm/slub: Convert acquire_slab() to take a struct page [30/62] mm/slub: Convert partial slab management to struct slab [31/62] mm/slub: Convert slab freeing to struct slab [32/62] mm/slub: Convert shuffle_freelist to struct slab [33/62] mm/slub: Remove struct page argument to next_freelist_entry() [34/62] mm/slub: Remove struct page argument from setup_object() [35/62] mm/slub: Convert freelist_corrupted() to struct slab [36/62] mm/slub: Convert full slab management to struct slab [37/62] mm/slub: Convert free_consistency_checks() to take a struct slab [38/62] mm/slub: Convert alloc_debug_processing() to struct slab [39/62] mm/slub: Convert check_object() to struct slab [40/62] mm/slub: Convert on_freelist() to struct slab [41/62] mm/slub: Convert check_slab() to struct slab [42/62] mm/slub: Convert check_valid_pointer() to struct slab [43/62] mm/slub: Convert object_err() to take a struct slab [44/62] mm/slub: Convert print_trailer() to struct slab [45/62] mm/slub: Convert slab_err() to take a struct slab [46/62] mm/slub: Convert print_page_info() to print_slab_info() [47/62] mm/slub: Convert trace() to take a struct slab [48/62] mm/slub: Convert cmpxchg_double_slab to struct slab [49/62] mm/slub: Convert get_map() and __fill_map() to struct slab [50/62] mm/slub: Convert slab_lock() and slab_unlock() to struct slab [51/62] mm/slub: Convert setup_page_debug() to setup_slab_debug() [52/62] mm/slub: Convert pfmemalloc_match() to take a struct slab [53/62] mm/slub: Remove pfmemalloc_match_unsafe() [54/62] mm: Convert slab to use struct slab [55/62] mm: Convert slob to use struct slab [56/62] mm: Convert slub to use struct slab [57/62] memcg: Convert object cgroups from struct page to struct slab [58/62] mm/kasan: Convert to struct slab [59/62] zsmalloc: Stop using slab fields in struct page [60/62] bootmem: Use page->index instead of page->freelist [61/62] iommu: Use put_pages_list [62/62] mm: Remove slab from struct page

Message ID

20211004134650.4031813-57-willy@infradead.org (mailing list archive)

State

New

Headers

DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1304D61186
From: "Matthew Wilcox (Oracle)" <willy@infradead.org>
To: linux-mm@kvack.org
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Subject: [PATCH 56/62] mm: Convert slub to use struct slab
Date: Mon,  4 Oct 2021 14:46:44 +0100
Message-Id: <20211004134650.4031813-57-willy@infradead.org>
In-Reply-To: <20211004134650.4031813-1-willy@infradead.org>
References: <20211004134650.4031813-1-willy@infradead.org>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Sender: owner-linux-mm@kvack.org
Precedence: bulk

Series

Separate struct slab from struct page | expand

Commit Message

Matthew Wilcox Oct. 4, 2021, 1:46 p.m. UTC

Remaining bits & pieces.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 mm/slub.c | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 229fc56809c2..51ead3838fc1 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -64,19 +64,19 @@ 
  *
  *   The slab_lock is only used for debugging and on arches that do not
  *   have the ability to do a cmpxchg_double. It only protects:
- *	A. slab->freelist	-> List of object free in a page
+ *	A. slab->freelist	-> List of object free in a slab
  *	B. slab->inuse		-> Number of objects in use
- *	C. slab->objects	-> Number of objects in page
+ *	C. slab->objects	-> Number of objects in slab
  *	D. slab->frozen		-> frozen state
  *
  *   Frozen slabs
  *
  *   If a slab is frozen then it is exempt from list management. It is not
  *   on any list except per cpu partial list. The processor that froze the
- *   slab is the one who can perform list operations on the page. Other
+ *   slab is the one who can perform list operations on the slab. Other
  *   processors may put objects onto the freelist but the processor that
  *   froze the slab is the only one that can retrieve the objects from the
- *   page's freelist.
+ *   slab's freelist.
  *
  *   list_lock
  *
@@ -135,7 +135,7 @@ 
  * minimal so we rely on the page allocators per cpu caches for
  * fast frees and allocs.
  *
- * page->frozen		The slab is frozen and exempt from list processing.
+ * slab->frozen		The slab is frozen and exempt from list processing.
  * 			This means that the slab is dedicated to a purpose
  * 			such as satisfying allocations for a specific
  * 			processor. Objects may be freed in the slab while
@@ -250,7 +250,7 @@  static inline bool kmem_cache_has_cpu_partial(struct kmem_cache *s)
 
 #define OO_SHIFT	16
 #define OO_MASK		((1 << OO_SHIFT) - 1)
-#define MAX_OBJS_PER_PAGE	32767 /* since page.objects is u15 */
+#define MAX_OBJS_PER_PAGE	32767 /* since slab.objects is u15 */
 
 /* Internal SLUB flags */
 /* Poison object */
@@ -1753,14 +1753,21 @@  static inline struct slab *alloc_slab(struct kmem_cache *s, gfp_t flags,
 				int node, struct kmem_cache_order_objects oo)
 {
 	struct page *page;
+	struct slab *slab;
 	unsigned int order = oo_order(oo);
 
 	if (node == NUMA_NO_NODE)
 		page = alloc_pages(flags, order);
 	else
 		page = __alloc_pages_node(node, flags, order);
+	if (!page)
+		return NULL;
 
-	return (struct slab *)page;
+	__SetPageSlab(page);
+	slab = (struct slab *)page;
+	if (page_is_pfmemalloc(page))
+		slab_set_pfmemalloc(slab);
+	return slab;
 }
 
 #ifdef CONFIG_SLAB_FREELIST_RANDOM
@@ -1781,7 +1788,7 @@  static int init_cache_random_seq(struct kmem_cache *s)
 		return err;
 	}
 
-	/* Transform to an offset on the set of pages */
+	/* Transform to an offset on the set of slabs */
 	if (s->random_seq) {
 		unsigned int i;
 
@@ -1911,10 +1918,6 @@  static struct slab *allocate_slab(struct kmem_cache *s, gfp_t flags, int node)
 	account_slab(slab, oo_order(oo), s, flags);
 
 	slab->slab_cache = s;
-	__SetPageSlab(slab_page(slab));
-	if (page_is_pfmemalloc(slab_page(slab)))
-		slab_set_pfmemalloc(slab);
-
 	kasan_poison_slab(slab_page(slab));
 
 	start = slab_address(slab);
@@ -3494,7 +3497,7 @@  static inline void free_nonslab_page(struct page *page, void *object)
 {
 	unsigned int order = compound_order(page);
 
-	VM_BUG_ON_PAGE(!PageCompound(page), page);
+	VM_BUG_ON_PAGE(!PageHead(page), page);
 	kfree_hook(object);
 	mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B, -(PAGE_SIZE << order));
 	__free_pages(page, order);

[56/62] mm: Convert slub to use struct slab

Commit Message

Patch