From patchwork Mon Oct 4 13:46:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12534129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F492C433F5 for ; Mon, 4 Oct 2021 14:28:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3537961251 for ; Mon, 4 Oct 2021 14:28:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3537961251 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id BD28B940036; Mon, 4 Oct 2021 10:28:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B82A594000B; Mon, 4 Oct 2021 10:28:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A49D4940036; Mon, 4 Oct 2021 10:28:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0253.hostedemail.com [216.40.44.253]) by kanga.kvack.org (Postfix) with ESMTP id 90D1994000B for ; Mon, 4 Oct 2021 10:28:02 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 453811828EE05 for ; Mon, 4 Oct 2021 14:28:02 +0000 (UTC) X-FDA: 78658984404.31.B5D5494 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf01.hostedemail.com (Postfix) with ESMTP id E75F250714DC for ; Mon, 4 Oct 2021 14:28:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=psKfLaxwvqSXN5SoFl0tcEZxdCcqcbuJyQ75i+0JEok=; b=es7X7I4rmYrS8GvWwJmk5uKaWi WKqKegcc7uaJk2g13vGhs2iRV7priobht/rctFrwOaOTl5D1oMjTL7hmHsstNQuDQoqvnjH8Lu/Bn dwi5ej62JwCZNm5X2IKtovXiOEj59l0CZ5M2KXq64dgKd02Y0Gm9H79rKa8ZjZkaDhxzYEcyNXq4b cR+wT2f/Pv1B3j9SxaSQuiyXPJukU/2qAQGqWpA09ZrkFQ6WRpLa/v7Rqn5u4COnHC8SF63YHuT2b I3aFndtWCPsguOvLbbc8AFghUTKnV5E9H/vtGO6tBeVoC0P2xzTDMhs9g4+/MDfUp6/SlPUu0CCUb tMm5ioRg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1mXOtx-00GzFE-3S; Mon, 04 Oct 2021 14:25:33 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 30/62] mm/slub: Convert partial slab management to struct slab Date: Mon, 4 Oct 2021 14:46:18 +0100 Message-Id: <20211004134650.4031813-31-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211004134650.4031813-1-willy@infradead.org> References: <20211004134650.4031813-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E75F250714DC X-Stat-Signature: km6ymca9dtee4zanra8mrefwcudrmdd4 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=es7X7I4r; dmarc=none; spf=none (imf01.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-HE-Tag: 1633357681-748904 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert __add_partial(), add_partial() and remove_partial(). Improves type safety and removes calls to slab_page(). Signed-off-by: Matthew Wilcox (Oracle) --- mm/slub.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 3468f2b2fe3a..e3c8893f9bd5 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2013,27 +2013,27 @@ static void discard_slab(struct kmem_cache *s, struct page *page) * Management of partially allocated slabs. */ static inline void -__add_partial(struct kmem_cache_node *n, struct page *page, int tail) +__add_partial(struct kmem_cache_node *n, struct slab *slab, int tail) { n->nr_partial++; if (tail == DEACTIVATE_TO_TAIL) - list_add_tail(&page->slab_list, &n->partial); + list_add_tail(&slab->slab_list, &n->partial); else - list_add(&page->slab_list, &n->partial); + list_add(&slab->slab_list, &n->partial); } static inline void add_partial(struct kmem_cache_node *n, - struct page *page, int tail) + struct slab *slab, int tail) { lockdep_assert_held(&n->list_lock); - __add_partial(n, page, tail); + __add_partial(n, slab, tail); } static inline void remove_partial(struct kmem_cache_node *n, - struct page *page) + struct slab *slab) { lockdep_assert_held(&n->list_lock); - list_del(&page->slab_list); + list_del(&slab->slab_list); n->nr_partial--; } @@ -2078,7 +2078,7 @@ static inline void *acquire_slab(struct kmem_cache *s, "acquire_slab")) return NULL; - remove_partial(n, slab_page(slab)); + remove_partial(n, slab); WARN_ON(!freelist); return freelist; } @@ -2405,12 +2405,12 @@ static void deactivate_slab(struct kmem_cache *s, struct slab *slab, if (l != m) { if (l == M_PARTIAL) - remove_partial(n, slab_page(slab)); + remove_partial(n, slab); else if (l == M_FULL) remove_full(s, n, slab_page(slab)); if (m == M_PARTIAL) - add_partial(n, slab_page(slab), tail); + add_partial(n, slab, tail); else if (m == M_FULL) add_full(s, n, slab_page(slab)); } @@ -2479,7 +2479,7 @@ static void __unfreeze_partials(struct kmem_cache *s, struct slab *partial_slab) slab->next = unusable; unusable = slab; } else { - add_partial(n, slab_page(slab), DEACTIVATE_TO_TAIL); + add_partial(n, slab, DEACTIVATE_TO_TAIL); stat(s, FREE_ADD_PARTIAL); } } @@ -3367,7 +3367,7 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, */ if (!kmem_cache_has_cpu_partial(s) && unlikely(!prior)) { remove_full(s, n, slab_page(slab)); - add_partial(n, slab_page(slab), DEACTIVATE_TO_TAIL); + add_partial(n, slab, DEACTIVATE_TO_TAIL); stat(s, FREE_ADD_PARTIAL); } spin_unlock_irqrestore(&n->list_lock, flags); @@ -3378,7 +3378,7 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, /* * Slab on the partial list. */ - remove_partial(n, slab_page(slab)); + remove_partial(n, slab); stat(s, FREE_REMOVE_PARTIAL); } else { /* Slab must be on the full list */ @@ -3922,7 +3922,7 @@ static void early_kmem_cache_node_alloc(int node) * No locks need to be taken here as it has just been * initialized and there is no concurrent access. */ - __add_partial(n, slab_page(slab), DEACTIVATE_TO_HEAD); + __add_partial(n, slab, DEACTIVATE_TO_HEAD); } static void free_kmem_cache_nodes(struct kmem_cache *s) @@ -4180,7 +4180,7 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags) #endif /* - * The larger the object size is, the more pages we want on the partial + * The larger the object size is, the more slabs we want on the partial * list to avoid pounding the page allocator excessively. */ set_min_partial(s, ilog2(s->size) / 2); @@ -4247,7 +4247,7 @@ static void free_partial(struct kmem_cache *s, struct kmem_cache_node *n) spin_lock_irq(&n->list_lock); list_for_each_entry_safe(slab, h, &n->partial, slab_list) { if (!slab->inuse) { - remove_partial(n, slab_page(slab)); + remove_partial(n, slab); list_add(&slab->slab_list, &discard); } else { list_slab_objects(s, slab,