From patchwork Sun Sep 15 17:08:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pengfei Li X-Patchwork-Id: 11146045 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A3C61747 for ; Sun, 15 Sep 2019 17:09:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 49AC121479 for ; Sun, 15 Sep 2019 17:09:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ANNQcHBe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 49AC121479 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5B66A6B026B; Sun, 15 Sep 2019 13:09:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5406D6B026D; Sun, 15 Sep 2019 13:09:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 407416B026E; Sun, 15 Sep 2019 13:09:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0084.hostedemail.com [216.40.44.84]) by kanga.kvack.org (Postfix) with ESMTP id 1C3A06B026D for ; Sun, 15 Sep 2019 13:09:13 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 95921824CA36 for ; Sun, 15 Sep 2019 17:09:12 +0000 (UTC) X-FDA: 75937790544.21.metal04_333e344c76726 X-Spam-Summary: 50,3,0,b130e0f7db268a18,d41d8cd98f00b204,lpf.vector@gmail.com,:akpm@linux-foundation.org:vbabka@suse.cz:cl@linux.com:penberg@kernel.org:rientjes@google.com:iamjoonsoo.kim@lge.com::linux-kernel@vger.kernel.org:guro@fb.com:lpf.vector@gmail.com,RULES_HIT:41:69:355:379:541:800:960:968:973:982:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1543:1711:1730:1747:1777:1792:1801:2198:2199:2393:2504:2559:2562:2693:2731:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3870:3871:3872:3874:4117:4321:4605:5007:6261:6653:7514:7903:8603:9121:9413:9592:10004:11026:11232:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12683:12895:13156:13228:13894:14040:14093:14181:14394:14687:14721:21080:21444:21450:21451:21627:21666:30029:30054,0,RBL:209.85.215.196:@gmail.com:.lbl8.mailshell.net-62.50.175.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: metal04_333e344c76726 X-Filterd-Recvd-Size: 6321 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Sun, 15 Sep 2019 17:09:11 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id 4so18023036pgm.12 for ; Sun, 15 Sep 2019 10:09:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3ghBrDy6wMKk43GDL/rFQB3qO4WlHynWL3G2LPomcOY=; b=ANNQcHBepw6eqK2TUsOonsPcZHpQ31HyjAqQ5GpjC1GXjc3vkSr7FVAF2g9z64OmzM tiaOv4HLHT9eMX+otd9hySdwzsxt240NNfBB270xdh909hG73HSDUXCCeGjTlqc7+zgM tCUb7tdc9gFK45+zYxujcpquqA3QxzQ9S4EKUaMRgF4sW8TCEO94WmhLzSGclFJBNBfI ODnsAc2L2tEcBUfEOpB62gH6lGfg3MgEy9057ZKX312vC21jG3llCUE87k2hbinoMEB8 jF5U3kFsZobrVujxFynsCiHBvRN38bJF/HKTCmjWrJdYpZSDMW8gmMl6McQrVxK0DttK 7Bnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3ghBrDy6wMKk43GDL/rFQB3qO4WlHynWL3G2LPomcOY=; b=LJ0/jrmTEkqI5qgBh66m079plcwS8yY80KZvUVL8ncBIfEe16aeCMHQT6yA6apGjPI mndqpws2SUM7I4I026FCGdxNBh74/fh+t+UqYNakGWqER9g9bUufTsP5XUZmtdutOY+C G00ayvfV8bOJzeYbPReeT0/RjB1HJNGzfqz9FYJ9MwjNHjJ616nnk5DD38ohhIn+5Tjs FBZAhLbC04MHiiv56OfglAso89j7Jijyw285L1nGV50KA+sid0YV+4/e7500ISJ5HNEA +q4Zg+1e19t2MnRkbjv0Z8fnUyTei/TZXPBymfmm1f62aJUUUR28grYTu7YLq5Z9ImzA +o7w== X-Gm-Message-State: APjAAAVsbWkXRoFAIaXchbLuINL8siXdrw+SBVFfIvwMZmO3hr8VZFFR nGgBc/KxdBxJdkwxnTm8L18= X-Google-Smtp-Source: APXvYqwd1Gu7Oj+hAUr55jham6Nk02gWInO/JZ+/9i9r2KKbdnRq111MTKjlBkJUa2qkqz6p+hpyRA== X-Received: by 2002:a65:5082:: with SMTP id r2mr51718854pgp.170.1568567351236; Sun, 15 Sep 2019 10:09:11 -0700 (PDT) Received: from localhost.localdomain.localdomain ([2408:823c:c11:160:b8c3:8577:bf2f:3]) by smtp.gmail.com with ESMTPSA id r28sm62279134pfg.62.2019.09.15.10.09.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Sep 2019 10:09:10 -0700 (PDT) From: Pengfei Li To: akpm@linux-foundation.org Cc: vbabka@suse.cz, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, guro@fb.com, Pengfei Li Subject: [RESEND v4 6/7] mm, slab_common: Initialize the same size of kmalloc_caches[] Date: Mon, 16 Sep 2019 01:08:08 +0800 Message-Id: <20190915170809.10702-7-lpf.vector@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190915170809.10702-1-lpf.vector@gmail.com> References: <20190915170809.10702-1-lpf.vector@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In the current implementation, KMALLOC_RECLAIM is not initialized until all the KMALLOC_NORMAL sizes have been initialized. But for a particular size, create_kmalloc_caches() can be executed faster by initializing different types of kmalloc in order. $ ./scripts/bloat-o-meter vmlinux.old vmlinux.patch_1-5 add/remove: 1/2 grow/shrink: 6/64 up/down: 872/-1113 (-241) Function old new delta create_kmalloc_caches 270 214 -56 $ ./scripts/bloat-o-meter vmlinux.old vmlinux.patch_1-6 add/remove: 1/2 grow/shrink: 6/64 up/down: 872/-1172 (-300) Function old new delta create_kmalloc_caches 270 155 -115 We can see that it really gets the benefits. Besides, KMALLOC_DMA will be initialized after "slab_state = UP", this does not seem to be necessary. Commit f97d5f634d3b ("slab: Common function to create the kmalloc array") introduces create_kmalloc_caches(). And I found that for SLAB, KMALLOC_DMA is initialized before "slab_state = UP". But for SLUB, KMALLOC_DMA is initialized after "slab_state = UP". Based on this fact, I think it is okay to initialize KMALLOC_DMA before "slab_state = UP". Signed-off-by: Pengfei Li --- mm/slab_common.c | 35 ++++++++++++----------------------- 1 file changed, 12 insertions(+), 23 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 2aed30deb071..e7903bd28b1f 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1165,12 +1165,9 @@ void __init setup_kmalloc_cache_index_table(void) size_index[size_index_elem(i)] = 0; } -static void __init +static __always_inline void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { - if (type == KMALLOC_RECLAIM) - flags |= SLAB_RECLAIM_ACCOUNT; - kmalloc_caches[type][idx] = create_kmalloc_cache( kmalloc_info[idx].name[type], kmalloc_info[idx].size, flags, 0, @@ -1185,30 +1182,22 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) void __init create_kmalloc_caches(slab_flags_t flags) { int i; - enum kmalloc_cache_type type; - for (type = KMALLOC_NORMAL; type <= KMALLOC_RECLAIM; type++) { - for (i = 0; i < KMALLOC_CACHE_NUM; i++) { - if (!kmalloc_caches[type][i]) - new_kmalloc_cache(i, type, flags); - } - } + for (i = 0; i < KMALLOC_CACHE_NUM; i++) { + if (!kmalloc_caches[KMALLOC_NORMAL][i]) + new_kmalloc_cache(i, KMALLOC_NORMAL, flags); - /* Kmalloc array is now usable */ - slab_state = UP; + new_kmalloc_cache(i, KMALLOC_RECLAIM, + flags | SLAB_RECLAIM_ACCOUNT); #ifdef CONFIG_ZONE_DMA - for (i = 0; i < KMALLOC_CACHE_NUM; i++) { - struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL][i]; - - if (s) { - kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache( - kmalloc_info[i].name[KMALLOC_DMA], - kmalloc_info[i].size, - SLAB_CACHE_DMA | flags, 0, 0); - } - } + new_kmalloc_cache(i, KMALLOC_DMA, + flags | SLAB_CACHE_DMA); #endif + } + + /* Kmalloc array is now usable */ + slab_state = UP; } #endif /* !CONFIG_SLOB */