From patchwork Wed Apr 28 09:49:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12228437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3384C43460 for ; Wed, 28 Apr 2021 09:54:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7890361426 for ; Wed, 28 Apr 2021 09:54:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7890361426 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 00FC76B006E; Wed, 28 Apr 2021 05:54:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F01316B0070; Wed, 28 Apr 2021 05:54:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D79FB6B0071; Wed, 28 Apr 2021 05:54:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0169.hostedemail.com [216.40.44.169]) by kanga.kvack.org (Postfix) with ESMTP id BB5156B006E for ; Wed, 28 Apr 2021 05:54:06 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 66F9F180AD817 for ; Wed, 28 Apr 2021 09:54:06 +0000 (UTC) X-FDA: 78081314892.36.DE25A94 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf16.hostedemail.com (Postfix) with ESMTP id B635280192F8 for ; Wed, 28 Apr 2021 09:54:01 +0000 (UTC) Received: by mail-pl1-f174.google.com with SMTP id h20so32520922plr.4 for ; Wed, 28 Apr 2021 02:54:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9oUrOqpAChDXaXDoRREdcI5L9MHP6oIlBRQpnJzUeII=; b=KjmVrYt4/mY4WAr2kGNuBttDXe558+QJuBQ0FarvEbVpYaJKm1byHI0esEdym/Sbms ro0NbPCCKfv5HOUetWMK0UzUV/wBvWj1pdq6vDvbMZJSDoWUsAEaM5qnn0Zn94mGyu76 CIdM0kJHKdvY2YznYTJNRCQjsDJ89lLa/6V9en+m4TrMBSXszIAsfpheCjK7mXscPLii f4WMz1stInaW9MZwHqi5sRS8/FdZ3f75T59LuLxAzP9za4ip+85ii2835JA9zLFnKkwg /7LBEqHUDEYbsKjDOXcN00GLzP6ZQcqmZwgmfvE5wno6F5pVKaC2ocCOaWtiQmLDvMik WWRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9oUrOqpAChDXaXDoRREdcI5L9MHP6oIlBRQpnJzUeII=; b=ByQYj+l/4y9GoBd5bZICDEmagTIUZdHmWDNQPDClZHq7Z4w3z3e+Uidbnq7mzVT5+h j/hXpKUSTlY9IjzJCw7VvMWSUYwLjFAiNyRoS4faCT14N8p5wx7/JT8M7SBWxE88rpot FI7cZHDTnAqioaW8wZEcy0dY7EnVVKKJzIMZ2Ul8RvGxSYbLqGyzMbHiE49m82ai3uPs 95rAvI/GNoHYwmm35GaMjYDV+epbMAd3lbXfj0mc9Z7S5kPY998US7JHXu8U/VCuVGCi 9hvDIhaJEyPRdIGg+M9roEr3mgwwase7cXJkoil137rx2WxUn7zt7h+sR0EDYH+rnvkm JTCg== X-Gm-Message-State: AOAM531kni/plYuMAnunYoSiX48Nf3fadRXHFUqttDtwMg8uZOedItRh 5JRstM1xyYy9uDNIKPDsZ4doQg== X-Google-Smtp-Source: ABdhPJxBl/GM8wieywRB0WXvNHzOvK1MwlC3y0XOZRlQNnVIeOPVORj003QMYJqVZivfXIWTUHMcqg== X-Received: by 2002:a17:902:7d89:b029:ec:c084:d4bc with SMTP id a9-20020a1709027d89b02900ecc084d4bcmr29571045plm.18.1619603645050; Wed, 28 Apr 2021 02:54:05 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.233]) by smtp.gmail.com with ESMTPSA id x77sm4902365pfc.19.2021.04.28.02.54.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Apr 2021 02:54:04 -0700 (PDT) From: Muchun Song To: willy@infradead.org, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, shakeelb@google.com, guro@fb.com, shy828301@gmail.com, alexs@kernel.org, alexander.h.duyck@linux.intel.com, richard.weiyang@gmail.com Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 1/9] mm: list_lru: fix list_lru_count_one() return value Date: Wed, 28 Apr 2021 17:49:41 +0800 Message-Id: <20210428094949.43579-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210428094949.43579-1-songmuchun@bytedance.com> References: <20210428094949.43579-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: h9hoi5yw38h4bcj4noont99budhqkb9u X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B635280192F8 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail-pl1-f174.google.com; client-ip=209.85.214.174 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619603641-797789 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since commit 2788cf0c401c ("memcg: reparent list_lrus and free kmemcg_id on css offline"), the ->nr_items can be negative during memory cgroup reparenting. In this case, list_lru_count_one() can returns an unusually large value. In order to not surprise the user. So return zero when ->nr_items is negative. Signed-off-by: Muchun Song --- mm/list_lru.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/mm/list_lru.c b/mm/list_lru.c index cd58790d0fb3..4962d48d4410 100644 --- a/mm/list_lru.c +++ b/mm/list_lru.c @@ -176,13 +176,16 @@ unsigned long list_lru_count_one(struct list_lru *lru, { struct list_lru_node *nlru = &lru->node[nid]; struct list_lru_one *l; - unsigned long count; + long count; rcu_read_lock(); l = list_lru_from_memcg_idx(nlru, memcg_cache_id(memcg)); count = READ_ONCE(l->nr_items); rcu_read_unlock(); + if (unlikely(count < 0)) + count = 0; + return count; } EXPORT_SYMBOL_GPL(list_lru_count_one); From patchwork Wed Apr 28 09:49:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12228439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 396B8C433B4 for ; Wed, 28 Apr 2021 09:54:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A0F1B613E7 for ; Wed, 28 Apr 2021 09:54:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A0F1B613E7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 27D676B0070; Wed, 28 Apr 2021 05:54:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 22E1D6B0071; Wed, 28 Apr 2021 05:54:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A7736B0072; Wed, 28 Apr 2021 05:54:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0212.hostedemail.com [216.40.44.212]) by kanga.kvack.org (Postfix) with ESMTP id E2E026B0070 for ; Wed, 28 Apr 2021 05:54:11 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9CD2252BF for ; Wed, 28 Apr 2021 09:54:11 +0000 (UTC) X-FDA: 78081315102.09.F81DEF3 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf11.hostedemail.com (Postfix) with ESMTP id 12F632000241 for ; Wed, 28 Apr 2021 09:53:55 +0000 (UTC) Received: by mail-pg1-f182.google.com with SMTP id j189so613751pgd.13 for ; Wed, 28 Apr 2021 02:54:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m1CFbU+mCktpHQ1dKqvusnGfjIT06imnu2MiKnHY6nk=; b=v4VjCZlNqoTtno1oSp5L516Q2Oq7puICK/nvOGS2G21b+Zf8dDbqpbPxbXBFaxZyPX +9XX3GF59psC7dYErKQ1Wd1WNsQ+vI7dYeuNft1Ozh+O1wGMo22QQASc86eHBg+p/taX 0nt80fb8lIzpAY70hZqGDF1imMiyDa8M/NsPH3xct7MNOLYD0cR3yy9FOLpKSE51hOFh hoyOIPv/848DsYPW8PmuRPui51Ovt41Nq0xw0czJphN9QGOR5l8mxqA0xVizRBGv80Dm V7n+ezDBiebnuPwdwgVwZO118XmqHJVIwU+D7QP07FQozcA5XNWoYM4DmScmZFDqI3Fp cdGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m1CFbU+mCktpHQ1dKqvusnGfjIT06imnu2MiKnHY6nk=; b=hL1WghzfMHcIeWZbaelg5Zc80dSdQ1B4+7ZFVRFeNZ1m1q2837bc22qKkfNkOrE7en DtMRTlufa2YqNr2xx4pJ4muOVbfc9X/cCDQ82hZZ8gyVwI/tN0CMPC+4pIR8GbTA8g4J 5EhmYy+qZms/SFe62kaRLFcQbb0hF6DKbE1/Vl5FhI1dK+TxwA3ah0SI38dbtW4T+dbn 535NyOh9nqV9kj7HvxyzzkGIw4G+PLkgB6XE8uAVktgVGwsdYgY68AUgFfpDwqTCLyq+ TXHjDmNz7ZWrNGG19SpfbqYbX65rFbrrqxJtIrcS/gWQsi1t1qkjkBKZsZes+zNQYGec WElg== X-Gm-Message-State: AOAM530CypgGeJCddMK/wJ8fsDq6/KxsHRyHBqE0q6JDg6Na3gftnrCZ Dvq2wMRBcu0iO8bsi1jazbWAFA== X-Google-Smtp-Source: ABdhPJw6KaQEv1gmE2Ru2d5aoT7DK6/dT4q9Q9aoINopiR0PMuD4/fmPAfJYaLngGk2J5Pm10PwxBA== X-Received: by 2002:a62:7646:0:b029:27a:29b6:e10b with SMTP id r67-20020a6276460000b029027a29b6e10bmr8361616pfc.14.1619603650322; Wed, 28 Apr 2021 02:54:10 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.233]) by smtp.gmail.com with ESMTPSA id x77sm4902365pfc.19.2021.04.28.02.54.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Apr 2021 02:54:10 -0700 (PDT) From: Muchun Song To: willy@infradead.org, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, shakeelb@google.com, guro@fb.com, shy828301@gmail.com, alexs@kernel.org, alexander.h.duyck@linux.intel.com, richard.weiyang@gmail.com Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 2/9] mm: memcontrol: remove kmemcg_id reparenting Date: Wed, 28 Apr 2021 17:49:42 +0800 Message-Id: <20210428094949.43579-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210428094949.43579-1-songmuchun@bytedance.com> References: <20210428094949.43579-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 12F632000241 X-Stat-Signature: sm9n3omr7ffqpz36fqfamc4gdymotdp5 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail-pg1-f182.google.com; client-ip=209.85.215.182 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619603635-251028 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since slab objects and kmem pages are charged to object cgroup instead of memory cgroup, memcg_reparent_objcgs() will reparent this cgroup and all its descendants to the parent cgroup. It means that the new parent memory cgroup can be returned by mem_cgroup_from_obj() which is called from list_lru_from_kmem(). This can make further list_lru_add()'s add elements to the parent's list. So we do not need to change kmemcg_id of an offline cgroup to its parent's id. This is just waste CPU cycles. Just remove those redundant code. Signed-off-by: Muchun Song --- mm/memcontrol.c | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 64ada9e650a5..21e12312509c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3485,8 +3485,7 @@ static int memcg_online_kmem(struct mem_cgroup *memcg) static void memcg_offline_kmem(struct mem_cgroup *memcg) { - struct cgroup_subsys_state *css; - struct mem_cgroup *parent, *child; + struct mem_cgroup *parent; int kmemcg_id; if (memcg->kmem_state != KMEM_ONLINE) @@ -3503,22 +3502,7 @@ static void memcg_offline_kmem(struct mem_cgroup *memcg) kmemcg_id = memcg->kmemcg_id; BUG_ON(kmemcg_id < 0); - /* - * Change kmemcg_id of this cgroup and all its descendants to the - * parent's id, and then move all entries from this cgroup's list_lrus - * to ones of the parent. After we have finished, all list_lrus - * corresponding to this cgroup are guaranteed to remain empty. The - * ordering is imposed by list_lru_node->lock taken by - * memcg_drain_all_list_lrus(). - */ - rcu_read_lock(); /* can be called from css_free w/o cgroup_mutex */ - css_for_each_descendant_pre(css, &memcg->css) { - child = mem_cgroup_from_css(css); - BUG_ON(child->kmemcg_id != kmemcg_id); - child->kmemcg_id = parent->kmemcg_id; - } - rcu_read_unlock(); - + /* memcg_reparent_objcgs() must be called before this. */ memcg_drain_all_list_lrus(kmemcg_id, parent); memcg_free_cache_id(kmemcg_id); From patchwork Wed Apr 28 09:49:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12228441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 573FCC433ED for ; Wed, 28 Apr 2021 09:54:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CA67061419 for ; Wed, 28 Apr 2021 09:54:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA67061419 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 563846B0071; Wed, 28 Apr 2021 05:54:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4EDE56B0072; Wed, 28 Apr 2021 05:54:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 318126B0073; Wed, 28 Apr 2021 05:54:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0061.hostedemail.com [216.40.44.61]) by kanga.kvack.org (Postfix) with ESMTP id 119C76B0071 for ; Wed, 28 Apr 2021 05:54:17 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C36AC181AF5CA for ; Wed, 28 Apr 2021 09:54:16 +0000 (UTC) X-FDA: 78081315312.27.301690E Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf17.hostedemail.com (Postfix) with ESMTP id CD38040002DD for ; Wed, 28 Apr 2021 09:54:12 +0000 (UTC) Received: by mail-pf1-f171.google.com with SMTP id v191so1594246pfc.8 for ; Wed, 28 Apr 2021 02:54:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DbfZmKXxQSOn7wHkCejhev2qWZcfPUS5M9gRiT8vHvs=; b=BqOFNyjA/Es8BddpzZ+9QxaIwEeZvSIuQO419b/fBSreqOqcUtciS6XnVplZDrZlG/ Imy6gNobcHN9VQvDMV/23CyMkdByJAqNWYXOjnXrqLQnlBCL3Q5MGNlt5tUXlbGe2fsh EzWGLAY4HuEZi8OeUdvhfwXr+AJyKi8UUff6lbxmCS5S1oHkh6xPZtngzOHzBOlXvwa7 mXj6jgUf+OY2OCpdvZPqdFr/f0e9CYhODs0mfdwqr9OPaDCGojvLQSHtoTaEfwj0nl7d rN6sbJqX27CKHTBysTUwLvfX/Ky7+llOZ3baxVl62+P2oov3Ev4KJJ6+DO2fgHGZetZS JHzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DbfZmKXxQSOn7wHkCejhev2qWZcfPUS5M9gRiT8vHvs=; b=JOKN/lrdmW0IXr7Be5+hYeITLQnebdv6nTr4aX1xckife48OZR0InK2DI3Cm+7Z8Q6 9OJtpMDHe6yFEL4CnMdgrFb9mrNaIlQ171ucCSjlnMn3slYTM/K5uk1QtVSDwW6iWtYK gQvM7zkApeduE/rZLU6YXQb1gVkug9+RHmDpM0OuuOc+dCR75j+iqq4LBDhApi244g1k ZuRyL6FsBwQzfATCrr/WhDtBt+Z8aAWZ2fiRnWtfDhIGPhSHELY3whcYpY5deMo5BM6s 1Dkjnkb1Xee+DlN/QRrj9joVEt3yScycArY+ZNHuwmuU3jJBC0QRChnnH7MJ5tqxPayG NAqQ== X-Gm-Message-State: AOAM5336R7DVCkM7yq8ti5REBMX/NxF0QtFmTIP/CpRKxgikrDhi22lx Wa8uT28vF/5jd1BGKK6BaZ7ATw== X-Google-Smtp-Source: ABdhPJw1XENVld3cLK+I71antXFfgkPHZ+eowp5+rG7qg4jm1nTaFVssJ6CW0IB96l43Y9shWLLlEQ== X-Received: by 2002:aa7:9806:0:b029:253:e613:4ada with SMTP id e6-20020aa798060000b0290253e6134adamr27192427pfl.65.1619603655719; Wed, 28 Apr 2021 02:54:15 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.233]) by smtp.gmail.com with ESMTPSA id x77sm4902365pfc.19.2021.04.28.02.54.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Apr 2021 02:54:15 -0700 (PDT) From: Muchun Song To: willy@infradead.org, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, shakeelb@google.com, guro@fb.com, shy828301@gmail.com, alexs@kernel.org, alexander.h.duyck@linux.intel.com, richard.weiyang@gmail.com Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 3/9] mm: list_lru: rename memcg_drain_all_list_lrus to memcg_reparent_list_lrus Date: Wed, 28 Apr 2021 17:49:43 +0800 Message-Id: <20210428094949.43579-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210428094949.43579-1-songmuchun@bytedance.com> References: <20210428094949.43579-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: CD38040002DD X-Stat-Signature: fe5j1gfteddnck8ynck9ck5ug8d484kd Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf17; identity=mailfrom; envelope-from=""; helo=mail-pf1-f171.google.com; client-ip=209.85.210.171 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619603652-687083 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since we do not change memcg->kmemcg_id before calling memcg_drain_all_list_lrus(), so we do not need to take kmemcg_id as parameter. The two parameters of memcg_drain_all_list_lrus() seems odd, one is kmemcg_id, another is memcg. Now we can change the kmemcg_id to the memcg. It is more consistent. Since the purpose of the memcg_drain_all_list_lrus() is list_lru reparenting. So also rename it to memcg_reparent_list_lrus(). The name is also consistent with memcg_reparent_objcgs(). Signed-off-by: Muchun Song --- include/linux/list_lru.h | 2 +- mm/list_lru.c | 23 ++++++++++++----------- mm/memcontrol.c | 2 +- 3 files changed, 14 insertions(+), 13 deletions(-) diff --git a/include/linux/list_lru.h b/include/linux/list_lru.h index 9dcaa3e582c9..e8a5e3a2c0dd 100644 --- a/include/linux/list_lru.h +++ b/include/linux/list_lru.h @@ -70,7 +70,7 @@ int __list_lru_init(struct list_lru *lru, bool memcg_aware, __list_lru_init((lru), true, NULL, shrinker) int memcg_update_all_list_lrus(int num_memcgs); -void memcg_drain_all_list_lrus(int src_idx, struct mem_cgroup *dst_memcg); +void memcg_reparent_list_lrus(struct mem_cgroup *memcg, struct mem_cgroup *parent); /** * list_lru_add: add an element to the lru list's tail diff --git a/mm/list_lru.c b/mm/list_lru.c index 4962d48d4410..d78dba5a6dab 100644 --- a/mm/list_lru.c +++ b/mm/list_lru.c @@ -523,11 +523,11 @@ int memcg_update_all_list_lrus(int new_size) goto out; } -static void memcg_drain_list_lru_node(struct list_lru *lru, int nid, - int src_idx, struct mem_cgroup *dst_memcg) +static void memcg_reparent_list_lru_node(struct list_lru *lru, int nid, + struct mem_cgroup *memcg, + struct mem_cgroup *parent) { struct list_lru_node *nlru = &lru->node[nid]; - int dst_idx = dst_memcg->kmemcg_id; struct list_lru_one *src, *dst; /* @@ -536,22 +536,23 @@ static void memcg_drain_list_lru_node(struct list_lru *lru, int nid, */ spin_lock_irq(&nlru->lock); - src = list_lru_from_memcg_idx(nlru, src_idx); - dst = list_lru_from_memcg_idx(nlru, dst_idx); + src = list_lru_from_memcg_idx(nlru, memcg->kmemcg_id); + dst = list_lru_from_memcg_idx(nlru, parent->kmemcg_id); list_splice_init(&src->list, &dst->list); if (src->nr_items) { dst->nr_items += src->nr_items; - set_shrinker_bit(dst_memcg, nid, lru_shrinker_id(lru)); + set_shrinker_bit(parent, nid, lru_shrinker_id(lru)); src->nr_items = 0; } spin_unlock_irq(&nlru->lock); } -static void memcg_drain_list_lru(struct list_lru *lru, - int src_idx, struct mem_cgroup *dst_memcg) +static void memcg_reparent_list_lru(struct list_lru *lru, + struct mem_cgroup *memcg, + struct mem_cgroup *parent) { int i; @@ -559,16 +560,16 @@ static void memcg_drain_list_lru(struct list_lru *lru, return; for_each_node(i) - memcg_drain_list_lru_node(lru, i, src_idx, dst_memcg); + memcg_reparent_list_lru_node(lru, i, memcg, parent); } -void memcg_drain_all_list_lrus(int src_idx, struct mem_cgroup *dst_memcg) +void memcg_reparent_list_lrus(struct mem_cgroup *memcg, struct mem_cgroup *parent) { struct list_lru *lru; mutex_lock(&list_lrus_mutex); list_for_each_entry(lru, &list_lrus, list) - memcg_drain_list_lru(lru, src_idx, dst_memcg); + memcg_reparent_list_lru(lru, memcg, parent); mutex_unlock(&list_lrus_mutex); } #else diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 21e12312509c..c1ce4fdba028 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3503,7 +3503,7 @@ static void memcg_offline_kmem(struct mem_cgroup *memcg) BUG_ON(kmemcg_id < 0); /* memcg_reparent_objcgs() must be called before this. */ - memcg_drain_all_list_lrus(kmemcg_id, parent); + memcg_reparent_list_lrus(memcg, parent); memcg_free_cache_id(kmemcg_id); } From patchwork Wed Apr 28 09:49:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12228443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDFECC43460 for ; Wed, 28 Apr 2021 09:54:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 446F46113D for ; Wed, 28 Apr 2021 09:54:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 446F46113D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BA6B56B0072; Wed, 28 Apr 2021 05:54:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B60E46B0073; Wed, 28 Apr 2021 05:54:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D05D6B0074; Wed, 28 Apr 2021 05:54:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0220.hostedemail.com [216.40.44.220]) by kanga.kvack.org (Postfix) with ESMTP id 8341D6B0072 for ; Wed, 28 Apr 2021 05:54:22 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3A4AB52CE for ; Wed, 28 Apr 2021 09:54:22 +0000 (UTC) X-FDA: 78081315564.11.FED7E2E Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf28.hostedemail.com (Postfix) with ESMTP id 533712000242 for ; Wed, 28 Apr 2021 09:54:23 +0000 (UTC) Received: by mail-pj1-f50.google.com with SMTP id l10-20020a17090a850ab0290155b06f6267so2630926pjn.5 for ; Wed, 28 Apr 2021 02:54:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NP/rxZesEiIIrfKiG+UYq1kkvOJzyQmEnIKOjy9l8kg=; b=Svyqhok6H6Syu6MAMe9UCLt3sv86mUklBNHn2V++/4jxLrT2f5AFZv9tKBSqsHVgiP ZoCf3jsDk5APQTo72uUAk2ZNYE4xWEseg/ya9aj/kyQzqUV0P6jWd7Mxu/vdl2t1ZEy5 Qo1yA2mysgdJliLeZxMv2OtZsL83h8tp0G5w837IQwOgUKEbPn/1f/zs2TfaS2/YR0s8 AKF0JFo43aaX3cyIP+5m3KmlrByf3LlDwqDNQM/sjmmhbHjgaVHa+UAb6KnrpRUuyc4X lGNSqcaj1YgmDEAaerlHT9cm+BQvw4HtgjshaqMQQV/vAZFH8Vz2aw87Eo8Gcx4MW+pC 0avg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NP/rxZesEiIIrfKiG+UYq1kkvOJzyQmEnIKOjy9l8kg=; b=Iw6YFfIKxSCd/kN1fhwG7SlVZsbDeE2oPItMoYRJWSVQEMXZnj2Q0hQ5ch2xdd9oUC ZyYo12w2sgz84mAZee9VbABL2g+U1Qrs75qo007M8BpUNRDFTPaNgUjGAo0unBBN6Vze DUr5QPMRgO8CNI/GXRGx9KhOoZG9KgT+mKyDr3lP57b7i+PwclAX0Cw0d5tffnBCNvXG VQdBDxLPTNhAe0EQWmDjPG9ZM+/ExuqDMKtbSzm4/hXum3GUO1WdLZmxVolIt+Ib/wkJ 20dofafITLnMQQHrllh/oXd8eTHgbKvv5nAriE74jj0oKJBw7KPS4LSpF398QuSIznsa myAg== X-Gm-Message-State: AOAM532DDIMb1J+KODfhy3EeCdS2wjZ9PRdaDAvy5IrKd88gTi0yjT4t OZSCo3SkHSvDFiutsr55o17JHg== X-Google-Smtp-Source: ABdhPJy2UAfP3e/NNwuaLzNawPPbdd+obpHPXRqeQPBax1hGR6109fKOvajSwFFeB/oc84VrtkReyg== X-Received: by 2002:a17:90a:c717:: with SMTP id o23mr25749439pjt.214.1619603661139; Wed, 28 Apr 2021 02:54:21 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.233]) by smtp.gmail.com with ESMTPSA id x77sm4902365pfc.19.2021.04.28.02.54.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Apr 2021 02:54:20 -0700 (PDT) From: Muchun Song To: willy@infradead.org, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, shakeelb@google.com, guro@fb.com, shy828301@gmail.com, alexs@kernel.org, alexander.h.duyck@linux.intel.com, richard.weiyang@gmail.com Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 4/9] mm: memcontrol: remove the kmem states Date: Wed, 28 Apr 2021 17:49:44 +0800 Message-Id: <20210428094949.43579-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210428094949.43579-1-songmuchun@bytedance.com> References: <20210428094949.43579-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 533712000242 X-Stat-Signature: 7qqroodkrsrh8gta1noydnorw3dqaft3 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf28; identity=mailfrom; envelope-from=""; helo=mail-pj1-f50.google.com; client-ip=209.85.216.50 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619603663-972759 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now the kmem states is only used to indicate whether the kmem is offlined. But we can use ->kmemcg_id to do the same things. So remove the kmem states to simplify the code. Signed-off-by: Muchun Song --- include/linux/memcontrol.h | 7 ------- mm/memcontrol.c | 10 ++-------- 2 files changed, 2 insertions(+), 15 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index c193be760709..6350c563c7b8 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -182,12 +182,6 @@ struct mem_cgroup_thresholds { struct mem_cgroup_threshold_ary *spare; }; -enum memcg_kmem_state { - KMEM_NONE, - KMEM_ALLOCATED, - KMEM_ONLINE, -}; - #if defined(CONFIG_SMP) struct memcg_padding { char x[0]; @@ -320,7 +314,6 @@ struct mem_cgroup { #ifdef CONFIG_MEMCG_KMEM int kmemcg_id; - enum memcg_kmem_state kmem_state; struct obj_cgroup __rcu *objcg; struct list_head objcg_list; /* list of inherited objcgs */ #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c1ce4fdba028..9b9a5368a3e9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3461,7 +3461,6 @@ static int memcg_online_kmem(struct mem_cgroup *memcg) return 0; BUG_ON(memcg->kmemcg_id >= 0); - BUG_ON(memcg->kmem_state); memcg_id = memcg_alloc_cache_id(); if (memcg_id < 0) @@ -3478,7 +3477,6 @@ static int memcg_online_kmem(struct mem_cgroup *memcg) static_branch_enable(&memcg_kmem_enabled_key); memcg->kmemcg_id = memcg_id; - memcg->kmem_state = KMEM_ONLINE; return 0; } @@ -3488,11 +3486,6 @@ static void memcg_offline_kmem(struct mem_cgroup *memcg) struct mem_cgroup *parent; int kmemcg_id; - if (memcg->kmem_state != KMEM_ONLINE) - return; - - memcg->kmem_state = KMEM_ALLOCATED; - parent = parent_mem_cgroup(memcg); if (!parent) parent = root_mem_cgroup; @@ -3506,12 +3499,13 @@ static void memcg_offline_kmem(struct mem_cgroup *memcg) memcg_reparent_list_lrus(memcg, parent); memcg_free_cache_id(kmemcg_id); + memcg->kmemcg_id = -1; } static void memcg_free_kmem(struct mem_cgroup *memcg) { /* css_alloc() failed, offlining didn't happen */ - if (unlikely(memcg->kmem_state == KMEM_ONLINE)) + if (unlikely(memcg->kmemcg_id != -1)) memcg_offline_kmem(memcg); } #else From patchwork Wed Apr 28 09:49:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12228445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9919BC433B4 for ; Wed, 28 Apr 2021 09:54:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 262C361419 for ; Wed, 28 Apr 2021 09:54:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 262C361419 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A1E126B0073; Wed, 28 Apr 2021 05:54:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A6E16B0074; Wed, 28 Apr 2021 05:54:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7FA346B0075; Wed, 28 Apr 2021 05:54:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0175.hostedemail.com [216.40.44.175]) by kanga.kvack.org (Postfix) with ESMTP id 5DD136B0073 for ; Wed, 28 Apr 2021 05:54:28 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1DC15180AD802 for ; Wed, 28 Apr 2021 09:54:28 +0000 (UTC) X-FDA: 78081315816.29.D0231D5 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf04.hostedemail.com (Postfix) with ESMTP id 137C43C4 for ; Wed, 28 Apr 2021 09:54:23 +0000 (UTC) Received: by mail-pl1-f175.google.com with SMTP id a11so1626668plh.3 for ; Wed, 28 Apr 2021 02:54:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rR+OvhVrb+QAHQ+jiVrkYgB4Dzl1YoXeDesY75cVJEI=; b=QvZJqE3ta2E/c3ZVQ+FxrxKL1VJJR+NIP4aPVZ3vheUWyux1jDovf//k/iN0z68sQ3 ui2Zg4Owg7kHHJQnT+BuuFMMfwjIVtkab2zyIXz4WwGRigQA57PzG9HvF5d3kGm6kgPL ikcC3jjEBpbjZH8wnBtMWy158RctA+ANe6wb0kGi0s/bPMGhnu/yJ3nnbsP0Wzvcr3Ma yDMDnyyAPXKwlJJqCqCBanhk1okLxlS5l5OYz+QbIWLLn+h/6o+RIdFj9jQHbpxGZltj 8hXbvNQpnhO8PG8FRAUqnqC3y15yMwipWjvJYNQQygKtYiUsSS6InGx4uvkhdT/6CAus eyVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rR+OvhVrb+QAHQ+jiVrkYgB4Dzl1YoXeDesY75cVJEI=; b=XKfwW4b1G/8C9UuI4Thx/rPhO0YSItToAGbWb50a7S1X2NDSCNh69+Dfam2mrAgQF3 DjEUTR+2nhHVIYimZWjKsBuLeSJ+SZlBQuqUuQ+jlmI7ECf0CEz3rXnBqrVu2FlD4xQ1 hIj0HWTaN1Kr39DhKi3bTvrHdzlpAZpzUhUKIdzjfq3X6zyLBCNgmygMj/u9GAu1VJd3 iZKOWtePex+ygDBIG/76yoSldXBW2X5itbIx9L7HZhIa+l56XsAqlUYPmmgSBF+TaWrh ZNHKRWGQhINQNblYk6ah5iBZPz2ba2O3gvrFi3nxmVFfCeD7GnNLveyqLegAq3eSHXjS kI0A== X-Gm-Message-State: AOAM532KKfcaG4bCIYnw47kP+euZ0OPttqI91Qydus2lAvxx00uM7Pq5 bbvA86qkR8gd4vo5FQYOFqzwkQ== X-Google-Smtp-Source: ABdhPJx+BQwvCa+tWyaLztEOm3PIxKcql1j60oXvl+7x6kH9BoqBdjRVuQn6aVNZA3U5QJvl60B37w== X-Received: by 2002:a17:90a:6c23:: with SMTP id x32mr18667949pjj.62.1619603666977; Wed, 28 Apr 2021 02:54:26 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.233]) by smtp.gmail.com with ESMTPSA id x77sm4902365pfc.19.2021.04.28.02.54.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Apr 2021 02:54:26 -0700 (PDT) From: Muchun Song To: willy@infradead.org, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, shakeelb@google.com, guro@fb.com, shy828301@gmail.com, alexs@kernel.org, alexander.h.duyck@linux.intel.com, richard.weiyang@gmail.com Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 5/9] mm: memcontrol: move memcg_online_kmem() to mem_cgroup_css_online() Date: Wed, 28 Apr 2021 17:49:45 +0800 Message-Id: <20210428094949.43579-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210428094949.43579-1-songmuchun@bytedance.com> References: <20210428094949.43579-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 137C43C4 X-Stat-Signature: hm3kuumh351zqq3dghin73hbou1hsbof Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail-pl1-f175.google.com; client-ip=209.85.214.175 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619603663-558687 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move memcg_online_kmem() to mem_cgroup_css_online() to simplify the code. In this case, we can remove memcg_free_kmem(). Signed-off-by: Muchun Song --- mm/memcontrol.c | 41 ++++++++++++++++------------------------- 1 file changed, 16 insertions(+), 25 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 9b9a5368a3e9..1610d501e7b5 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3460,6 +3460,9 @@ static int memcg_online_kmem(struct mem_cgroup *memcg) if (cgroup_memory_nokmem) return 0; + if (mem_cgroup_is_root(memcg)) + return 0; + BUG_ON(memcg->kmemcg_id >= 0); memcg_id = memcg_alloc_cache_id(); @@ -3486,6 +3489,9 @@ static void memcg_offline_kmem(struct mem_cgroup *memcg) struct mem_cgroup *parent; int kmemcg_id; + if (mem_cgroup_is_root(memcg)) + return; + parent = parent_mem_cgroup(memcg); if (!parent) parent = root_mem_cgroup; @@ -3499,14 +3505,6 @@ static void memcg_offline_kmem(struct mem_cgroup *memcg) memcg_reparent_list_lrus(memcg, parent); memcg_free_cache_id(kmemcg_id); - memcg->kmemcg_id = -1; -} - -static void memcg_free_kmem(struct mem_cgroup *memcg) -{ - /* css_alloc() failed, offlining didn't happen */ - if (unlikely(memcg->kmemcg_id != -1)) - memcg_offline_kmem(memcg); } #else static int memcg_online_kmem(struct mem_cgroup *memcg) @@ -3516,9 +3514,6 @@ static int memcg_online_kmem(struct mem_cgroup *memcg) static void memcg_offline_kmem(struct mem_cgroup *memcg) { } -static void memcg_free_kmem(struct mem_cgroup *memcg) -{ -} #endif /* CONFIG_MEMCG_KMEM */ static int memcg_update_kmem_max(struct mem_cgroup *memcg, @@ -5047,7 +5042,6 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css) { struct mem_cgroup *parent = mem_cgroup_from_css(parent_css); struct mem_cgroup *memcg, *old_memcg; - long error = -ENOMEM; old_memcg = set_active_memcg(parent); memcg = mem_cgroup_alloc(); @@ -5077,38 +5071,36 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css) } /* The following stuff does not apply to the root */ - error = memcg_online_kmem(memcg); - if (error) - goto fail; - if (cgroup_subsys_on_dfl(memory_cgrp_subsys) && !cgroup_memory_nosocket) static_branch_inc(&memcg_sockets_enabled_key); return &memcg->css; -fail: - mem_cgroup_id_remove(memcg); - mem_cgroup_free(memcg); - return ERR_PTR(error); } static int mem_cgroup_css_online(struct cgroup_subsys_state *css) { struct mem_cgroup *memcg = mem_cgroup_from_css(css); + if (memcg_online_kmem(memcg)) + goto remove_id; + /* * A memcg must be visible for expand_shrinker_info() * by the time the maps are allocated. So, we allocate maps * here, when for_each_mem_cgroup() can't skip it. */ - if (alloc_shrinker_info(memcg)) { - mem_cgroup_id_remove(memcg); - return -ENOMEM; - } + if (alloc_shrinker_info(memcg)) + goto offline_kmem; /* Online state pins memcg ID, memcg ID pins CSS */ refcount_set(&memcg->id.ref, 1); css_get(css); return 0; +offline_kmem: + memcg_offline_kmem(memcg); +remove_id: + mem_cgroup_id_remove(memcg); + return -ENOMEM; } static void mem_cgroup_css_offline(struct cgroup_subsys_state *css) @@ -5166,7 +5158,6 @@ static void mem_cgroup_css_free(struct cgroup_subsys_state *css) cancel_work_sync(&memcg->high_work); mem_cgroup_remove_from_trees(memcg); free_shrinker_info(memcg); - memcg_free_kmem(memcg); mem_cgroup_free(memcg); } From patchwork Wed Apr 28 09:49:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12228447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6AA4C433B4 for ; Wed, 28 Apr 2021 09:54:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 689F261429 for ; Wed, 28 Apr 2021 09:54:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 689F261429 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EB2C36B0074; Wed, 28 Apr 2021 05:54:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E61AB6B0075; Wed, 28 Apr 2021 05:54:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CDC9F6B0078; Wed, 28 Apr 2021 05:54:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0030.hostedemail.com [216.40.44.30]) by kanga.kvack.org (Postfix) with ESMTP id B20AB6B0074 for ; Wed, 28 Apr 2021 05:54:33 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6DA9B181AF5CA for ; Wed, 28 Apr 2021 09:54:33 +0000 (UTC) X-FDA: 78081316026.08.54FFE34 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf13.hostedemail.com (Postfix) with ESMTP id E2A05E000128 for ; Wed, 28 Apr 2021 09:54:24 +0000 (UTC) Received: by mail-pf1-f181.google.com with SMTP id j6so3983640pfh.5 for ; Wed, 28 Apr 2021 02:54:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tsZpxnSCiHU3yOSRKuQ5Nn0zsBnruhO0+WHjObzR7x8=; b=CVN6j6GWu+63QHDHliyf2AZ1WZ20KwHrSvk5XqYiXCO7XP6C3pKHdUDBT4SxVeMhrP L7tMKIH47RxVvddI2lTDpeJ8zoSoyEYMuQ45LVFeb5DXE+Fh6X6ChHpirl3PFzwWAMLR Mq16ofgtQ7FlNKOSAT4pY+afeqGjvzzZCTuj0MZdV7TOq3YnjHw7tiV/vRs6qP2M2Xgm th2Wnk7lULXJi9N8kST37AJ0xLNcyrFfwLaX9ZH7VPC/rOXuYy0wadwRq00TeBP8sxq3 UWfh5WsU99QG0r6ObL1e/F7tLvVLs7RbR4LYSVxhVou+9gePD8fjRXBe3VlMtaRlw2Xe LMlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tsZpxnSCiHU3yOSRKuQ5Nn0zsBnruhO0+WHjObzR7x8=; b=RQGV9Z+JW3OUQOQWkBq18gTv6i+8hr7gBg2zqFtSvHs6rLVCr1MpYm+5Q2l5gHQ9ga bIcqFwN2YtDmj6HZTSJ8TNbjPG2S0p1H15NR3rbGMPHg332FQJhmJpjsvu9wu+WedjQF pjrGNCz3jG28IECUmqLToi8lkjcgK8A++tiZW8NqrIrdsq7UFrnFL9gDfQUaNDv8ywTM fdIszceifjDoLfOGxidp6lflVqFKTcjpXrM6nPKM7fs+ebTfzjNB68PKjt0u5o4/U49x T0Wigu/hNSEu18yGEmaBBFVGi+Fu4m7A0fvEOLcANAjRuX0cr05hkYDGmEkIAzhPLz27 xVRg== X-Gm-Message-State: AOAM532QZOrWsqEEmncjcqvsetY4TcFhZ5daT2C4x7xGQn09K+vyPZ6Z jzJqp2W8pOszpWAb0xYQ8wejfw== X-Google-Smtp-Source: ABdhPJxeJwwMM87CerPEYoMJbuNWzwWZUAqJ1LCPLGY2k3lesIokTfkNVrUPpW+Jem/IzuHxzll8pA== X-Received: by 2002:a63:570b:: with SMTP id l11mr26184726pgb.193.1619603672214; Wed, 28 Apr 2021 02:54:32 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.233]) by smtp.gmail.com with ESMTPSA id x77sm4902365pfc.19.2021.04.28.02.54.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Apr 2021 02:54:31 -0700 (PDT) From: Muchun Song To: willy@infradead.org, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, shakeelb@google.com, guro@fb.com, shy828301@gmail.com, alexs@kernel.org, alexander.h.duyck@linux.intel.com, richard.weiyang@gmail.com Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 6/9] mm: list_lru: support for shrinking list lru Date: Wed, 28 Apr 2021 17:49:46 +0800 Message-Id: <20210428094949.43579-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210428094949.43579-1-songmuchun@bytedance.com> References: <20210428094949.43579-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: E2A05E000128 X-Stat-Signature: ojonka53ctjfh897z5u439h4whtun8x5 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail-pf1-f181.google.com; client-ip=209.85.210.181 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619603664-837353 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now memcg_update_all_list_lrus() only can increase the size of all the list lrus. This patch adds an ability to make it can shrink the size of all the list lrus. This can help us save memory when the user want to shrink the size. Signed-off-by: Muchun Song --- mm/list_lru.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 49 insertions(+), 4 deletions(-) diff --git a/mm/list_lru.c b/mm/list_lru.c index d78dba5a6dab..3ee5239922c9 100644 --- a/mm/list_lru.c +++ b/mm/list_lru.c @@ -383,13 +383,11 @@ static void memcg_destroy_list_lru_node(struct list_lru_node *nlru) kvfree(memcg_lrus); } -static int memcg_update_list_lru_node(struct list_lru_node *nlru, - int old_size, int new_size) +static int memcg_list_lru_node_inc(struct list_lru_node *nlru, + int old_size, int new_size) { struct list_lru_memcg *old, *new; - BUG_ON(old_size > new_size); - old = rcu_dereference_protected(nlru->memcg_lrus, lockdep_is_held(&list_lrus_mutex)); new = kvmalloc(sizeof(*new) + new_size * sizeof(void *), GFP_KERNEL); @@ -418,11 +416,58 @@ static int memcg_update_list_lru_node(struct list_lru_node *nlru, return 0; } +/* This function always returns 0. */ +static int memcg_list_lru_node_dec(struct list_lru_node *nlru, + int old_size, int new_size) +{ + struct list_lru_memcg *old, *new; + + old = rcu_dereference_protected(nlru->memcg_lrus, + lockdep_is_held(&list_lrus_mutex)); + __memcg_destroy_list_lru_node(old, new_size, old_size); + + /* Reuse the old array if the allocation failures here. */ + new = kvmalloc(sizeof(*new) + new_size * sizeof(void *), GFP_KERNEL); + if (!new) + return 0; + + memcpy(&new->lru, &old->lru, new_size * sizeof(void *)); + + /* + * The locking below allows readers that hold nlru->lock avoid taking + * rcu_read_lock (see list_lru_from_memcg_idx). + * + * Since list_lru_{add,del} may be called under an IRQ-safe lock, + * we have to use IRQ-safe primitives here to avoid deadlock. + */ + spin_lock_irq(&nlru->lock); + rcu_assign_pointer(nlru->memcg_lrus, new); + spin_unlock_irq(&nlru->lock); + + kvfree_rcu(old, rcu); + return 0; +} + +static int memcg_update_list_lru_node(struct list_lru_node *nlru, + int old_size, int new_size) +{ + if (new_size > old_size) + return memcg_list_lru_node_inc(nlru, old_size, new_size); + else if (new_size < old_size) + return memcg_list_lru_node_dec(nlru, old_size, new_size); + + return 0; +} + static void memcg_cancel_update_list_lru_node(struct list_lru_node *nlru, int old_size, int new_size) { struct list_lru_memcg *memcg_lrus; + /* Nothing to do for the shrinking case. */ + if (old_size >= new_size) + return; + memcg_lrus = rcu_dereference_protected(nlru->memcg_lrus, lockdep_is_held(&list_lrus_mutex)); /* do not bother shrinking the array back to the old size, because we From patchwork Wed Apr 28 09:49:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12228449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6DCAC433ED for ; Wed, 28 Apr 2021 09:54:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5C23D613E7 for ; Wed, 28 Apr 2021 09:54:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5C23D613E7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E8AAD6B0075; Wed, 28 Apr 2021 05:54:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E3A336B0078; Wed, 28 Apr 2021 05:54:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB59C6B007B; Wed, 28 Apr 2021 05:54:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0104.hostedemail.com [216.40.44.104]) by kanga.kvack.org (Postfix) with ESMTP id AB1D86B0075 for ; Wed, 28 Apr 2021 05:54:38 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 69A4452AA for ; Wed, 28 Apr 2021 09:54:38 +0000 (UTC) X-FDA: 78081316236.30.9D10F28 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf13.hostedemail.com (Postfix) with ESMTP id EFEA2E000111 for ; Wed, 28 Apr 2021 09:54:29 +0000 (UTC) Received: by mail-pl1-f179.google.com with SMTP id a11so1626920plh.3 for ; Wed, 28 Apr 2021 02:54:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tOlUteNHINhRpZtpztBeELGSpydzeET01dFGAMvcB9w=; b=LEEZH5u64AUh+ynJYV212VHcTTq3jo7SOak1Z80frSXjEr35j/gMZeBApZgIac5EtK oyQLEU220WgDEPzQQt937Zemxc7veTiPvTy2Rem1vxKR5EMp3d0T0RkE6upT1Nx13pd3 ct69tBbCAl5PkeCcTgc9IYO1AqQ4C5KPm82xwGXsp+4PGuex1H2KlazxCnxlGyVZwc5j Fs3/VxauZ9qh06JmV+1Po+DoXkgn8GDqHvtx/snioyeBACqoFzuMhU4SBVX7lh2/yQ8s hU0c7U9jmWxlmiUBhSuIaccnhBlEYqFEurNWm6cRlhEbmb8D8OXWCCXdL2sH6LGIxV++ yNOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tOlUteNHINhRpZtpztBeELGSpydzeET01dFGAMvcB9w=; b=Xx+20nZ8I5zAwCg6TbwVcxb5lYjFJ1chcUSda1qTm4NpSDplr6QMlpBhpVG0wkgvVC hSG8tADrEBP52aKYQ/EyNLxwbUxHllpR8cOKW3bZRKCgmlkeDp5knE4QxcZHYhooXCaT 5wzLffaOYtCTijOQ61URVsGN6i/lTx/U3MhCusEJ9sxftMcKWGxDIIgIWkUC1e+s9DYQ oeKT9/drVJaB4q7BvtfwbFCh2u2irjNGO09Xj5IUcqnHsxKP+KLztfbu/hjWzvy4DXE4 P4rDQG6wrbte172DH3jODlJqBW/Zn/Na82ed6dK2Q3uzM0jkkoMtfflT0QhIPoBV1RBH 7TLA== X-Gm-Message-State: AOAM532UaEeFu7qxvDBV9paIqq5XUwxrz6AKjRTDUr8rnZz+QTMH9wDe Pz54fiEMMJmDfJWxa0eb/iNYHQ== X-Google-Smtp-Source: ABdhPJyrBDmZk/mG3JBVhVPM7uIF8K2OuTdn8hm9YJpozguzJZ16PR0+D5tUGPQdGb8zskw4RSE7hg== X-Received: by 2002:a17:90b:1bc1:: with SMTP id oa1mr30557077pjb.46.1619603677248; Wed, 28 Apr 2021 02:54:37 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.233]) by smtp.gmail.com with ESMTPSA id x77sm4902365pfc.19.2021.04.28.02.54.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Apr 2021 02:54:36 -0700 (PDT) From: Muchun Song To: willy@infradead.org, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, shakeelb@google.com, guro@fb.com, shy828301@gmail.com, alexs@kernel.org, alexander.h.duyck@linux.intel.com, richard.weiyang@gmail.com Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 7/9] ida: introduce ida_max() to return the maximum allocated ID Date: Wed, 28 Apr 2021 17:49:47 +0800 Message-Id: <20210428094949.43579-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210428094949.43579-1-songmuchun@bytedance.com> References: <20210428094949.43579-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: EFEA2E000111 X-Stat-Signature: 8kgcsnbxym4qecznmab77zpkp43udid3 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail-pl1-f179.google.com; client-ip=209.85.214.179 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619603669-790969 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce ida_max() to return the maximum allocated ID. This will be used by memory cgroup in the later patch. Signed-off-by: Muchun Song --- include/linux/idr.h | 1 + lib/idr.c | 40 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/include/linux/idr.h b/include/linux/idr.h index a0dce14090a9..c3968a6348d1 100644 --- a/include/linux/idr.h +++ b/include/linux/idr.h @@ -255,6 +255,7 @@ struct ida { int ida_alloc_range(struct ida *, unsigned int min, unsigned int max, gfp_t); void ida_free(struct ida *, unsigned int id); void ida_destroy(struct ida *ida); +int ida_max(struct ida *ida); /** * ida_alloc() - Allocate an unused ID. diff --git a/lib/idr.c b/lib/idr.c index f4ab4f4aa3c7..bcfcaae89aa7 100644 --- a/lib/idr.c +++ b/lib/idr.c @@ -553,6 +553,46 @@ void ida_destroy(struct ida *ida) } EXPORT_SYMBOL(ida_destroy); +/** + * ida_max() - Return the maximum allocated ID. + * @ida: IDA handle. + * + * Context: Any context. It is safe to call this function without + * locking in your code. + * + * Return: The maximum allocated ID, or %-ENOSPC if the @ida is empty + */ +int ida_max(struct ida *ida) +{ + XA_STATE(xas, &ida->xa, 0); + struct ida_bitmap *curr, *prev; + unsigned long flags; + unsigned int bit, index; + + xas_lock_irqsave(&xas, flags); + if (ida_is_empty(ida)) { + xas_unlock_irqrestore(&xas, flags); + return -ENOSPC; + } + + xas_for_each(&xas, curr, ULONG_MAX) { + prev = curr; + index = xas.xa_index; + } + + if (xa_is_value(prev)) { + unsigned long val = xa_to_value(prev); + + bit = find_last_bit(&val, BITS_PER_XA_VALUE); + } else { + bit = find_last_bit(prev->bitmap, IDA_BITMAP_BITS); + } + + xas_unlock_irqrestore(&xas, flags); + + return index * IDA_BITMAP_BITS + bit; +} + #ifndef __KERNEL__ extern void xa_dump_index(unsigned long index, unsigned int shift); #define IDA_CHUNK_SHIFT ilog2(IDA_BITMAP_BITS) From patchwork Wed Apr 28 09:49:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12228451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00AAFC433B4 for ; Wed, 28 Apr 2021 09:54:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8A25661424 for ; Wed, 28 Apr 2021 09:54:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8A25661424 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 059F66B0078; Wed, 28 Apr 2021 05:54:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 030946B007B; Wed, 28 Apr 2021 05:54:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DED066B007D; Wed, 28 Apr 2021 05:54:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0204.hostedemail.com [216.40.44.204]) by kanga.kvack.org (Postfix) with ESMTP id C2B6E6B0078 for ; Wed, 28 Apr 2021 05:54:49 -0400 (EDT) Received: from smtpin39.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 85DE852AA for ; Wed, 28 Apr 2021 09:54:49 +0000 (UTC) X-FDA: 78081316698.39.8BBB4AE Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf27.hostedemail.com (Postfix) with ESMTP id CD9A280192F6 for ; Wed, 28 Apr 2021 09:54:26 +0000 (UTC) Received: by mail-pf1-f169.google.com with SMTP id 10so831633pfl.1 for ; Wed, 28 Apr 2021 02:54:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hY8MMvczpGg13vjFG0pVN31ztr+LQshtCwbuU4Brm+8=; b=OVbU3euh7GNOL90p5IiOxYxd4FcV1VfX0Y9vH/DEq9PQ+YiGaCtAbF9kfcosIOpdNt +/+lxJEW370B9C2wMjLbygjflseJ/Bjqq3IiZmMZrz+aDvE97hM9gBX/TnJna/XR4twH ghtJswmWGKVLMbNUuVlgg+yv1Ui/e8jn7fEiLnj0my55Xpciv3n8/sSpBrnNSV17kx6g 6WqjfZh1HFaIBmkPWTn8wwv6NFUYza8HXULyd3PUKJpw32v5videUL6humajUOB0SVuH W95VSs9KRquN+EZQVRhc35Ml9pUhxcqB966YC+EuQPDvO8LdqrpLONVSsK59dHAyLB4I NXAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hY8MMvczpGg13vjFG0pVN31ztr+LQshtCwbuU4Brm+8=; b=qSL7xgUnehLrw+P+52oVmNdC6VJKhBEuw1G1Ew38MhoNm7XEV+7p2JPMo1N6bogQYS eIA0ymx6jYtcc6Kg4tJ69tHR6tTms1lRfz0DYssu4vdTWagBkEoP6BHhs2lpk+jycpay yvAflebjyS9gF02mmjEvO1DocHJQTHlZlpfFeL6xC2XhCiECRkQRQGAWqLcomiWd6EsJ /mQDSQs1Bd77U3yu//NLcDNW+2RTOY670mu3LFRgVCMj04jdx/1In/wPOIb0WZHnIy3W EhVY71JS9vPlmtEz1SAcI9/g28RK2QO3C0s8TnnUWRruaMQ6uwbEdMY5HtpV1kuR4SDZ 2NvA== X-Gm-Message-State: AOAM530Z0XorTXgYJ9nNsnpTeMGaxfkHbhoR40lGE3cy+htNex6zVlOi wUhYFGdmBkrQua+TV9W5RL6s0g== X-Google-Smtp-Source: ABdhPJyzJQIe5mcRTW0GlTyxzjQmNhmM1GNGv6hhvKBTTupwNZQlaXeqkzuhovHXs5jNeEzNcBllrw== X-Received: by 2002:a63:b52:: with SMTP id a18mr26475766pgl.276.1619603688418; Wed, 28 Apr 2021 02:54:48 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.233]) by smtp.gmail.com with ESMTPSA id x77sm4902365pfc.19.2021.04.28.02.54.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Apr 2021 02:54:48 -0700 (PDT) From: Muchun Song To: willy@infradead.org, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, shakeelb@google.com, guro@fb.com, shy828301@gmail.com, alexs@kernel.org, alexander.h.duyck@linux.intel.com, richard.weiyang@gmail.com Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 9/9] mm: memcontrol: rename memcg_{get,put}_cache_ids to memcg_list_lru_resize_{lock,unlock} Date: Wed, 28 Apr 2021 17:49:49 +0800 Message-Id: <20210428094949.43579-10-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210428094949.43579-1-songmuchun@bytedance.com> References: <20210428094949.43579-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CD9A280192F6 X-Stat-Signature: x79687u6frybrnxsb3i56w8ehmjgi8z1 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=mail-pf1-f169.google.com; client-ip=209.85.210.169 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619603666-509944 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The rwsem is held for writing during list lru arrays relocation and memcg_nr_cache_ids updates. Therefore memcg_get_cache_ids implies memcg_nr_cache_ids cannot be updated. It acts as a lock primitive. So rename it to a more suitable name. Signed-off-by: Muchun Song --- include/linux/memcontrol.h | 8 ++++---- mm/list_lru.c | 8 ++++---- mm/memcontrol.c | 4 ++-- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6350c563c7b8..e8ba6ee1b369 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1635,8 +1635,8 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size); extern struct static_key_false memcg_kmem_enabled_key; extern int memcg_nr_cache_ids; -void memcg_get_cache_ids(void); -void memcg_put_cache_ids(void); +void memcg_list_lru_resize_lock(void); +void memcg_list_lru_resize_unlock(void); /* * Helper macro to loop through all memcg-specific caches. Callers must still @@ -1711,11 +1711,11 @@ static inline int memcg_cache_id(struct mem_cgroup *memcg) return -1; } -static inline void memcg_get_cache_ids(void) +static inline void memcg_list_lru_resize_lock(void) { } -static inline void memcg_put_cache_ids(void) +static inline void memcg_list_lru_resize_unlock(void) { } diff --git a/mm/list_lru.c b/mm/list_lru.c index 3ee5239922c9..e0ba0641b4e1 100644 --- a/mm/list_lru.c +++ b/mm/list_lru.c @@ -640,7 +640,7 @@ int __list_lru_init(struct list_lru *lru, bool memcg_aware, else lru->shrinker_id = -1; #endif - memcg_get_cache_ids(); + memcg_list_lru_resize_lock(); lru->node = kcalloc(nr_node_ids, sizeof(*lru->node), GFP_KERNEL); if (!lru->node) @@ -663,7 +663,7 @@ int __list_lru_init(struct list_lru *lru, bool memcg_aware, list_lru_register(lru); out: - memcg_put_cache_ids(); + memcg_list_lru_resize_unlock(); return err; } EXPORT_SYMBOL_GPL(__list_lru_init); @@ -674,7 +674,7 @@ void list_lru_destroy(struct list_lru *lru) if (!lru->node) return; - memcg_get_cache_ids(); + memcg_list_lru_resize_lock(); list_lru_unregister(lru); @@ -685,6 +685,6 @@ void list_lru_destroy(struct list_lru *lru) #ifdef CONFIG_MEMCG_KMEM lru->shrinker_id = -1; #endif - memcg_put_cache_ids(); + memcg_list_lru_resize_unlock(); } EXPORT_SYMBOL_GPL(list_lru_destroy); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f8cdd87cf693..437465611845 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -367,12 +367,12 @@ static int kmemcg_max_id; /* Protects memcg_nr_cache_ids */ static DECLARE_RWSEM(memcg_cache_ids_sem); -void memcg_get_cache_ids(void) +void memcg_list_lru_resize_lock(void) { down_read(&memcg_cache_ids_sem); } -void memcg_put_cache_ids(void) +void memcg_list_lru_resize_unlock(void) { up_read(&memcg_cache_ids_sem); }