From patchwork Mon May 24 23:39:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 12277237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBDEAC47086 for ; Mon, 24 May 2021 23:40:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 75DD16141F for ; Mon, 24 May 2021 23:40:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 75DD16141F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 239946B0078; Mon, 24 May 2021 19:40:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D4086B0080; Mon, 24 May 2021 19:40:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3E8E6B0078; Mon, 24 May 2021 19:40:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0081.hostedemail.com [216.40.44.81]) by kanga.kvack.org (Postfix) with ESMTP id ABF436B0072 for ; Mon, 24 May 2021 19:40:51 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4E15F181AEF2A for ; Mon, 24 May 2021 23:40:51 +0000 (UTC) X-FDA: 78177747102.01.9F3479D Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf16.hostedemail.com (Postfix) with ESMTP id CC06A80192D8 for ; Mon, 24 May 2021 23:40:45 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1621899648; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1xmWeJ1b7AZlmflgkuweNV6NpHrLePBdboBsTBPbV/k=; b=JoaTF4rqBbwWtdHjrYUyCM/QqY0hcx8rqKdEwXsbGgayd9D7nrxrKjRRG96f1rr0pxkojB Q4Wq26PiCK1wcYmstYjpoDfT5IXgmDdqeCpHih3/V1+qCbGY6PockH5rxi7413yGns9vVO ZiBbgyoQxkBY5FQs/AGCcri2u4csPV4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1621899648; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1xmWeJ1b7AZlmflgkuweNV6NpHrLePBdboBsTBPbV/k=; b=E7m4pF7yX975qQIzoPzuGk7jsqfXsunkwKDsF+v818VG1L4hZR5aSd1yRJVN9IMmKuc/cL KfzO/8ZKmHl/pJDQ== Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 1BAFFAF21; Mon, 24 May 2021 23:40:48 +0000 (UTC) From: Vlastimil Babka To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Lameter , David Rientjes , Pekka Enberg , Joonsoo Kim Cc: Sebastian Andrzej Siewior , Thomas Gleixner , Mel Gorman , Jesper Dangaard Brouer , Peter Zijlstra , Jann Horn , Vlastimil Babka Subject: [RFC 08/26] mm, slub: restructure new page checks in ___slab_alloc() Date: Tue, 25 May 2021 01:39:28 +0200 Message-Id: <20210524233946.20352-9-vbabka@suse.cz> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210524233946.20352-1-vbabka@suse.cz> References: <20210524233946.20352-1-vbabka@suse.cz> MIME-Version: 1.0 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=JoaTF4rq; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=E7m4pF7y; dmarc=none; spf=pass (imf16.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.15 as permitted sender) smtp.mailfrom=vbabka@suse.cz X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CC06A80192D8 X-Stat-Signature: ocjued889czzy5c9yc1teo9ip8a11mr1 X-HE-Tag: 1621899645-733752 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When we allocate slab object from a newly acquired page (from node's partial list or page allocator), we usually also retain the page as a new percpu slab. There are two exceptions - when pfmemalloc status of the page doesn't match our gfp flags, or when the cache has debugging enabled. The current code for these decisions is not easy to follow, so restructure it and add comments. No functional change. Signed-off-by: Vlastimil Babka Acked-by: Mel Gorman --- mm/slub.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index f240e424c861..06f30c9ad361 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2743,13 +2743,29 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, c->page = page; check_new_page: - if (likely(!kmem_cache_debug(s) && pfmemalloc_match(page, gfpflags))) - goto load_freelist; - /* Only entered in the debug case */ - if (kmem_cache_debug(s) && - !alloc_debug_processing(s, page, freelist, addr)) - goto new_slab; /* Slab failed checks. Next slab needed */ + if (kmem_cache_debug(s)) { + if (!alloc_debug_processing(s, page, freelist, addr)) + /* Slab failed checks. Next slab needed */ + goto new_slab; + else + /* + * For debug case, we don't load freelist so that all + * allocations go through alloc_debug_processing() + */ + goto return_single; + } + + if (unlikely(!pfmemalloc_match(page, gfpflags))) + /* + * For !pfmemalloc_match() case we don't load freelist so that + * we don't make further mismatched allocations easier. + */ + goto return_single; + + goto load_freelist; + +return_single: deactivate_slab(s, page, get_freepointer(s, freelist), c); return freelist;