[v2,11/33] mm/slub: Convert kfree() to use a struct slab

Message ID	20211201181510.18784-12-vbabka@suse.cz (mailing list archive)
State	New
Headers	show Return-Path: <owner-linux-mm@kvack.org> From: Vlastimil Babka <vbabka@suse.cz> To: Matthew Wilcox <willy@infradead.org>, Christoph Lameter <cl@linux.com>, David Rientjes <rientjes@google.com>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, Pekka Enberg <penberg@kernel.org> Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, patches@lists.linux.dev, Vlastimil Babka <vbabka@suse.cz> Subject: [PATCH v2 11/33] mm/slub: Convert kfree() to use a struct slab Date: Wed, 1 Dec 2021 19:14:48 +0100 Message-Id: <20211201181510.18784-12-vbabka@suse.cz> In-Reply-To: <20211201181510.18784-1-vbabka@suse.cz> References: <20211201181510.18784-1-vbabka@suse.cz> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org Precedence: bulk
Series	Separate struct slab from struct page \| expand [v2,00/33] Separate struct slab from struct page [v2,01/33] mm: add virt_to_folio() and folio_address() [v2,02/33] mm/slab: Dissolve slab_map_pages() in its caller [v2,03/33] mm/slub: Make object_err() static [v2,04/33] mm: Split slab into its own type [v2,05/33] mm: Add account_slab() and unaccount_slab() [v2,06/33] mm: Convert virt_to_cache() to use struct slab [v2,07/33] mm: Convert __ksize() to struct slab [v2,08/33] mm: Use struct slab in kmem_obj_info() [v2,09/33] mm: Convert check_heap_object() to use struct slab [v2,10/33] mm/slub: Convert detached_freelist to use a struct slab [v2,11/33] mm/slub: Convert kfree() to use a struct slab [v2,12/33] mm/slub: Convert __slab_lock() and __slab_unlock() to struct slab [v2,13/33] mm/slub: Convert print_page_info() to print_slab_info() [v2,14/33] mm/slub: Convert alloc_slab_page() to return a struct slab [v2,15/33] mm/slub: Convert __free_slab() to use struct slab [v2,16/33] mm/slub: Convert pfmemalloc_match() to take a struct slab [v2,17/33] mm/slub: Convert most struct page to struct slab by spatch [v2,18/33] mm/slub: Finish struct page to struct slab conversion [v2,19/33] mm/slab: Convert kmem_getpages() and kmem_freepages() to struct slab [v2,20/33] mm/slab: Convert most struct page to struct slab by spatch [v2,21/33] mm/slab: Finish struct page to struct slab conversion [v2,22/33] mm: Convert struct page to struct slab in functions used by other subsystems [v2,23/33] mm/memcg: Convert slab objcgs from struct page to struct slab [v2,24/33] mm/slob: Convert SLOB to use struct slab [v2,25/33] mm/kasan: Convert to struct folio and struct slab [v2,26/33] mm/kfence: Convert kfence_guarded_alloc() to struct slab [v2,27/33] zsmalloc: Stop using slab fields in struct page [v2,28/33] bootmem: Use page->index instead of page->freelist [v2,29/33] iommu: Use put_pages_list [v2,30/33] mm: Remove slab from struct page [v2,31/33] mm/slb: Differentiate struct slab fields by slb implementations [v2,32/33] mm/slub: Simplify struct slab slabs field definition [v2,33/33] mm/slub: Define struct slab fields for CONFIG_SLUB_CPU_PARTIAL only when enabled

Message ID

20211201181510.18784-12-vbabka@suse.cz (mailing list archive)

State

New

Headers

From: Vlastimil Babka <vbabka@suse.cz>
To: Matthew Wilcox <willy@infradead.org>,
	Christoph Lameter <cl@linux.com>,
	David Rientjes <rientjes@google.com>,
	Joonsoo Kim <iamjoonsoo.kim@lge.com>,
	Pekka Enberg <penberg@kernel.org>
Cc: linux-mm@kvack.org,
	Andrew Morton <akpm@linux-foundation.org>,
	patches@lists.linux.dev,
	Vlastimil Babka <vbabka@suse.cz>
Subject: [PATCH v2 11/33] mm/slub: Convert kfree() to use a struct slab
Date: Wed,  1 Dec 2021 19:14:48 +0100
Message-Id: <20211201181510.18784-12-vbabka@suse.cz>
In-Reply-To: <20211201181510.18784-1-vbabka@suse.cz>
References: <20211201181510.18784-1-vbabka@suse.cz>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Sender: owner-linux-mm@kvack.org
Precedence: bulk

Series

Separate struct slab from struct page | expand

Commit Message

Vlastimil Babka Dec. 1, 2021, 6:14 p.m. UTC

From: "Matthew Wilcox (Oracle)" <willy@infradead.org>

Convert kfree(), kmem_cache_free() and ___cache_free() to resolve object
addresses to struct slab, using folio as intermediate step where needed. Keep
passing the result as struct page for now in preparation for mass conversion of
internal functions.

[ vbabka@suse.cz: Use folio as intermediate step when checking for
  large kmalloc pages ]

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
 mm/slub.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 9bdb0a5d6ab2..2d9c1b3c9f37 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -3517,7 +3517,7 @@  static __always_inline void slab_free(struct kmem_cache *s, struct page *page,
 #ifdef CONFIG_KASAN_GENERIC
 void ___cache_free(struct kmem_cache *cache, void *x, unsigned long addr)
 {
-	do_slab_free(cache, virt_to_head_page(x), x, NULL, 1, addr);
+	do_slab_free(cache, slab_page(virt_to_slab(x)), x, NULL, 1, addr);
 }
 #endif
 
@@ -3527,7 +3527,7 @@  void kmem_cache_free(struct kmem_cache *s, void *x)
 	if (!s)
 		return;
 	trace_kmem_cache_free(_RET_IP_, x, s->name);
-	slab_free(s, virt_to_head_page(x), x, NULL, 1, _RET_IP_);
+	slab_free(s, slab_page(virt_to_slab(x)), x, NULL, 1, _RET_IP_);
 }
 EXPORT_SYMBOL(kmem_cache_free);
 
@@ -4546,7 +4546,8 @@  EXPORT_SYMBOL(__ksize);
 
 void kfree(const void *x)
 {
-	struct page *page;
+	struct folio *folio;
+	struct slab *slab;
 	void *object = (void *)x;
 
 	trace_kfree(_RET_IP_, x);
@@ -4554,12 +4555,13 @@  void kfree(const void *x)
 	if (unlikely(ZERO_OR_NULL_PTR(x)))
 		return;
 
-	page = virt_to_head_page(x);
-	if (unlikely(!PageSlab(page))) {
-		free_nonslab_page(page, object);
+	folio = virt_to_folio(x);
+	if (unlikely(!folio_test_slab(folio))) {
+		free_nonslab_page(folio_page(folio, 0), object);
 		return;
 	}
-	slab_free(page->slab_cache, page, object, NULL, 1, _RET_IP_);
+	slab = folio_slab(folio);
+	slab_free(slab->slab_cache, slab_page(slab), object, NULL, 1, _RET_IP_);
 }
 EXPORT_SYMBOL(kfree);

[v2,11/33] mm/slub: Convert kfree() to use a struct slab

Commit Message

Patch