From patchwork Tue Apr 13 06:51:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12199453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0471AC433B4 for ; Tue, 13 Apr 2021 06:57:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8193A60FDB for ; Tue, 13 Apr 2021 06:57:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8193A60FDB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 19FA46B0096; Tue, 13 Apr 2021 02:57:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17BDA6B0099; Tue, 13 Apr 2021 02:57:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 016B86B009A; Tue, 13 Apr 2021 02:57:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0096.hostedemail.com [216.40.44.96]) by kanga.kvack.org (Postfix) with ESMTP id DAAFD6B0096 for ; Tue, 13 Apr 2021 02:57:40 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 90AA28249980 for ; Tue, 13 Apr 2021 06:57:40 +0000 (UTC) X-FDA: 78026438280.05.9E2F46B Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf18.hostedemail.com (Postfix) with ESMTP id D3B972000250 for ; Tue, 13 Apr 2021 06:57:40 +0000 (UTC) Received: by mail-pj1-f50.google.com with SMTP id s14so3125151pjl.5 for ; Mon, 12 Apr 2021 23:57:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Nm2qLQR55IXL2VTvCCbc055js3zSget/PmvTRGcIhzI=; b=SYDc5NQqBBJ4brJ6Ss8Q9ZO0pgg43LtamF9vVa3EBuNPT/+gjwy/G/AyELJxGFwXAZ P8ZoSUANNh/NKhe0oVZet49FfHqNcp3lSalETQA3VvfLMOShFa/dNhJizgaodB3smwlT Yp0bD/F0tizefNn6OwzwrwfvOKsY545GxAw+69nhxUzKyXCjumko+Dr+0tfMGL/AHFm+ eeezTgw0hdZDv3842+nD+mZSyZJGiEWp3N3/1EWr2KpISZnXDOXScBZPXtLo0YuMX3SE 1ZIytNcS8hN5jQODYw5YG0fSX/sSRYtMYIAWvqZZcRzY6M5WBlNHjibnjzEatgSzUUus Wqbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Nm2qLQR55IXL2VTvCCbc055js3zSget/PmvTRGcIhzI=; b=QDmxhH9/JdT3H4K7JWxgJ2gGdEM4URXFH6A/w9foFR6sNCSKabYz9Nuwet2720D/Cc S374kQ/wkpYx1Kwwya05Wt2uNImci1dgcmQkFdxV9J6V0pKnWO1r/HnPcn5+4PU29Ach 7JOV4Ld4e5bAEVZBHbVhDWFIN3x+IGdMECxFxcIZ64U+upgRKxo+uvTIUcM6YeLzndx8 ZN1KqbGTEN4e3bZ5JK/6m2No2veDJjjBMq8NEXYFh6Gu+ReVY2xXaPX5HzSe/ZJwgQcn gdq1JD9FQce3Hak9nPdckGrlh9R5+gVzd34bp4pRJwpOm73r0XXr3uOgddwzLdM+K1Ll bmnw== X-Gm-Message-State: AOAM533EokBtIeHJXtkkVvZ15cHDa5VDiEyU73qpEv4kJDHq7161p3Rh 7MebtN1aWLU0bCwvW2eIlWA+2h8YccdLUCUj X-Google-Smtp-Source: ABdhPJzLAqcpFJPdGm9ND1hLM45nc0Xm5wg7mCzjWEcmGLjSKdPWnXxuyar0Nueum0yIyWvj2cuUbg== X-Received: by 2002:a17:902:8e89:b029:e9:a576:886e with SMTP id bg9-20020a1709028e89b02900e9a576886emr22954128plb.65.1618297059217; Mon, 12 Apr 2021 23:57:39 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.241]) by smtp.gmail.com with ESMTPSA id 78sm11508562pfu.40.2021.04.12.23.57.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Apr 2021 23:57:38 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH 1/7] mm: memcontrol: fix page charging in page replacement Date: Tue, 13 Apr 2021 14:51:47 +0800 Message-Id: <20210413065153.63431-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210413065153.63431-1-songmuchun@bytedance.com> References: <20210413065153.63431-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: D3B972000250 X-Stat-Signature: hfap5ny463txtswb1rtjhkjcwrugwi8r Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail-pj1-f50.google.com; client-ip=209.85.216.50 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618297060-681632 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The pages aren't accounted at the root level, so do not charge the page to the root memcg in page replacement. Although we do not display the value (mem_cgroup_usage) so there shouldn't be any actual problem, but there is a WARN_ON_ONCE in the page_counter_cancel(). Who knows if it will trigger? So it is better to fix it. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- mm/memcontrol.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 64ada9e650a5..f229de925aa5 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6806,9 +6806,11 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage) /* Force-charge the new page. The old one will be freed soon */ nr_pages = thp_nr_pages(newpage); - page_counter_charge(&memcg->memory, nr_pages); - if (do_memsw_account()) - page_counter_charge(&memcg->memsw, nr_pages); + if (!mem_cgroup_is_root(memcg)) { + page_counter_charge(&memcg->memory, nr_pages); + if (do_memsw_account()) + page_counter_charge(&memcg->memsw, nr_pages); + } css_get(&memcg->css); commit_charge(newpage, memcg); From patchwork Tue Apr 13 06:51:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12199455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33D46C433ED for ; Tue, 13 Apr 2021 06:57:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CA9A360FDB for ; Tue, 13 Apr 2021 06:57:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA9A360FDB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6D09D6B0036; Tue, 13 Apr 2021 02:57:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A5336B0099; Tue, 13 Apr 2021 02:57:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 546C86B009B; Tue, 13 Apr 2021 02:57:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0225.hostedemail.com [216.40.44.225]) by kanga.kvack.org (Postfix) with ESMTP id 384F66B0036 for ; Tue, 13 Apr 2021 02:57:46 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id F3E0D8249980 for ; Tue, 13 Apr 2021 06:57:45 +0000 (UTC) X-FDA: 78026438490.22.8A8D4CD Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf04.hostedemail.com (Postfix) with ESMTP id 03F4D13A for ; Tue, 13 Apr 2021 06:57:43 +0000 (UTC) Received: by mail-pf1-f181.google.com with SMTP id d124so10788212pfa.13 for ; Mon, 12 Apr 2021 23:57:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OtjjYxy6fbQMjrzo3+4P30oz5j5AuPoBCZ8FaKjHWIE=; b=y1rkSqiViCIp3wTOZ5QlJPLXK+Id87VZYWjp1ZyjjVXGVoEX1B6l280oeHYcRpB1KY 9ESkPxUcqthFMWeaWmpwBsPAs3AcKDJrYuUE9hbn1eTCht6iIiiMtfmCcApQBtZCaikL 5dqWFDHSIebonXesBLGxJVCC/6inWWBnQ4RI/iagNc9h3ODaXBWPgo9+IvWilMCTYrpe iQNvJUteDMc1T/AO22JtRuTo36GFapFcy5LB2fzqZTqtpysOxnQsFFqjKSt5VJlECY+I 1pI6Cizi6IduIfyUxUUxlQRg9UqWAuAupLaQsD8V9H8d8Ra3B1qX4jDQpnVQsFnvvMnT yZqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OtjjYxy6fbQMjrzo3+4P30oz5j5AuPoBCZ8FaKjHWIE=; b=Lc+AcNCG8Ijhc0VxTTCI8A6cuuzxuzf5hJYrgzeTeMFUb9JvopO0pBlKaqf0Utgcnt PgTRyHKMTKzf7lEyIhbR8fYkyati7ay8pKjGWJC5UVgVg3IuX4Dy5Gx876JJ5gS7E0J7 x2usEB7zwmMAC1eRXFRknj8V8VoY5cAeHydpuCVSTewq/x9bmeoFMvbt7WcnkDe7q/BT v7QrgA9IYHwsqPTNuWDHFEcCNhPuAnmTejGWU9lP7rrPCHAbjCunZVaUGN3Vfl/tzZcJ Huu8aYWVTVYQw7uqv9jwzjm4K9w3Woa1YjXwrLc77frbRTU6dLhybJE7a/4YP4Ya/qHd zRTw== X-Gm-Message-State: AOAM5330wnJNj6/ntzfe9G2K824s5E7rGXWmcYD3QwwhwTvkwbZilMs9 Zi3PX+8uvYPGsQ1ULR5VlocF+g== X-Google-Smtp-Source: ABdhPJy7qTBayD0MjKH9mpcq9ysXSWq3mVkOTjl13qDTm/tP1B8q6Px9uYLgupMSHVay7ks/UL7ooQ== X-Received: by 2002:a62:7b0e:0:b029:24c:3d81:8028 with SMTP id w14-20020a627b0e0000b029024c3d818028mr9200627pfc.58.1618297064697; Mon, 12 Apr 2021 23:57:44 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.241]) by smtp.gmail.com with ESMTPSA id 78sm11508562pfu.40.2021.04.12.23.57.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Apr 2021 23:57:44 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH 2/7] mm: memcontrol: bail out early when !mm in get_mem_cgroup_from_mm Date: Tue, 13 Apr 2021 14:51:48 +0800 Message-Id: <20210413065153.63431-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210413065153.63431-1-songmuchun@bytedance.com> References: <20210413065153.63431-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 03F4D13A X-Stat-Signature: 5ftojkz5a9brexo1foubh8o9iggd1y93 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail-pf1-f181.google.com; client-ip=209.85.210.181 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618297063-794004 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When mm is NULL, we do not need to hold rcu lock and call css_tryget for the root memcg. And we also do not need to check !mm in every loop of while. So bail out early when !mm. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- mm/memcontrol.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f229de925aa5..9cbfff59b171 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -901,20 +901,19 @@ struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm) if (mem_cgroup_disabled()) return NULL; + /* + * Page cache insertions can happen without an + * actual mm context, e.g. during disk probing + * on boot, loopback IO, acct() writes etc. + */ + if (unlikely(!mm)) + return root_mem_cgroup; + rcu_read_lock(); do { - /* - * Page cache insertions can happen without an - * actual mm context, e.g. during disk probing - * on boot, loopback IO, acct() writes etc. - */ - if (unlikely(!mm)) + memcg = mem_cgroup_from_task(rcu_dereference(mm->owner)); + if (unlikely(!memcg)) memcg = root_mem_cgroup; - else { - memcg = mem_cgroup_from_task(rcu_dereference(mm->owner)); - if (unlikely(!memcg)) - memcg = root_mem_cgroup; - } } while (!css_tryget(&memcg->css)); rcu_read_unlock(); return memcg; From patchwork Tue Apr 13 06:51:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12199457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0ED8C43460 for ; Tue, 13 Apr 2021 06:57:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 43287613AE for ; Tue, 13 Apr 2021 06:57:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 43287613AE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B889B6B006E; Tue, 13 Apr 2021 02:57:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B6C546B0070; Tue, 13 Apr 2021 02:57:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A02276B0071; Tue, 13 Apr 2021 02:57:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0225.hostedemail.com [216.40.44.225]) by kanga.kvack.org (Postfix) with ESMTP id 83EB86B006E for ; Tue, 13 Apr 2021 02:57:54 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 49795688B for ; Tue, 13 Apr 2021 06:57:54 +0000 (UTC) X-FDA: 78026438868.26.353C87C Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by imf25.hostedemail.com (Postfix) with ESMTP id C39F06000117 for ; Tue, 13 Apr 2021 06:57:51 +0000 (UTC) Received: by mail-pg1-f173.google.com with SMTP id t140so11236306pgb.13 for ; Mon, 12 Apr 2021 23:57:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=71NH+uMXEa3vImQxjJijYTe54is/RT9JbLc/98rym48=; b=z6HE6C/LCwCAC+kFg3CfimNwgpDd8pULqr7veW101Uhlcp9bcab1cyTfzU/AoiTpk0 J5OVBs/JiRbYsbs1868vdPTkjsVEttS0o2fO8GejsOngfQSqngsxc7rZ78m1cqFO/S3c yz78kiY6ITDzK0ujBc3k6UdwRZTBzqTrW04yJNEv/07vbdwIpXCruLI2mvCU5jydD8nz GoVxaDFHvHrgPM/DLJFd+9iq2ersHPph0Y9rE9MBuaC4/Iu4MrA2rXMKU1wRE31xIS9w HazXZrwHBpBfbMXWf/kp8Yqs9WLTG7KpaYClxNGrtSu+2VjdT1LfnT3A11Hj0/yN3lDr j7mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=71NH+uMXEa3vImQxjJijYTe54is/RT9JbLc/98rym48=; b=NM+F8aj2pzGZ4NoxdcaOgfSYT2tY0xHGcT1Kz53A7vXQkRHMtQx9yO5FAhwv7yUg/J 7tJGKA74zCrWHL+h1rJZ1Qlnrew4KxO4Z3u2WkhF90GJOTz6NmcoRzxgIY7EayaXQd6s aEvBNrSvbJKgLYuA/n90xNrUFlTHgn8p26ykohCA8H7rEVrlco2/gFFCWblGPK/rsHSd G5c54g8LZvqs2pe2Gw2gJ+iq9FsDcpglej2NTHdba2JI74r2Y2gZtjGpG7dwW/oK/FCi eFhNBs3DY4jB/0ndEpIHNYAVa1rd4rCpXaK6f7SvZqhtKtY0prwPDFHqe+3ZUjWmgQM1 Bmvg== X-Gm-Message-State: AOAM530PeMGzzqUqoemAZMzjitFcMbRo0Pec5c1BQzI3bJvJTKtVb2O4 1Fa2whPYe3BX8obnMPSPLBc2GQ== X-Google-Smtp-Source: ABdhPJy9ZkUOPYnaAFXFhbgIJ0DFD88zmBjK/GeAzc9o1siOzWhvf0y7oYEyUj4rdqxVJtsrdnW1fQ== X-Received: by 2002:a62:ab11:0:b029:242:4c58:d46b with SMTP id p17-20020a62ab110000b02902424c58d46bmr3940418pff.15.1618297073178; Mon, 12 Apr 2021 23:57:53 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.241]) by smtp.gmail.com with ESMTPSA id 78sm11508562pfu.40.2021.04.12.23.57.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Apr 2021 23:57:52 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH 4/7] mm: memcontrol: simplify lruvec_holds_page_lru_lock Date: Tue, 13 Apr 2021 14:51:50 +0800 Message-Id: <20210413065153.63431-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210413065153.63431-1-songmuchun@bytedance.com> References: <20210413065153.63431-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C39F06000117 X-Stat-Signature: t4kup5xrbfzd5iu6k6qx95kwax9m6yuh Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf25; identity=mailfrom; envelope-from=""; helo=mail-pg1-f173.google.com; client-ip=209.85.215.173 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618297071-724283 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We already have a helper lruvec_memcg() to get the memcg from lruvec, we do not need to do it ourselves in the lruvec_holds_page_lru_lock(). So use lruvec_memcg() instead. And if mem_cgroup_disabled() returns false, the page_memcg(page) (the LRU pages) cannot be NULL. So remove the odd logic of "memcg = page_memcg(page) ? : root_mem_cgroup". And use lruvec_pgdat to simplify the code. We can have a single definition for this function that works for !CONFIG_MEMCG, CONFIG_MEMCG + mem_cgroup_disabled() and CONFIG_MEMCG. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- include/linux/memcontrol.h | 31 +++++++------------------------ 1 file changed, 7 insertions(+), 24 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 4f49865c9958..38b8d3fb24ff 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -755,22 +755,6 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page) return mem_cgroup_lruvec(memcg, pgdat); } -static inline bool lruvec_holds_page_lru_lock(struct page *page, - struct lruvec *lruvec) -{ - pg_data_t *pgdat = page_pgdat(page); - const struct mem_cgroup *memcg; - struct mem_cgroup_per_node *mz; - - if (mem_cgroup_disabled()) - return lruvec == &pgdat->__lruvec; - - mz = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - memcg = page_memcg(page) ? : root_mem_cgroup; - - return lruvec->pgdat == pgdat && mz->memcg == memcg; -} - struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p); struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm); @@ -1229,14 +1213,6 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page) return &pgdat->__lruvec; } -static inline bool lruvec_holds_page_lru_lock(struct page *page, - struct lruvec *lruvec) -{ - pg_data_t *pgdat = page_pgdat(page); - - return lruvec == &pgdat->__lruvec; -} - static inline void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page) { } @@ -1518,6 +1494,13 @@ static inline void unlock_page_lruvec_irqrestore(struct lruvec *lruvec, spin_unlock_irqrestore(&lruvec->lru_lock, flags); } +static inline bool lruvec_holds_page_lru_lock(struct page *page, + struct lruvec *lruvec) +{ + return lruvec_pgdat(lruvec) == page_pgdat(page) && + lruvec_memcg(lruvec) == page_memcg(page); +} + /* Don't lock again iff page's lruvec locked */ static inline struct lruvec *relock_page_lruvec_irq(struct page *page, struct lruvec *locked_lruvec) From patchwork Tue Apr 13 06:51:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12199459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11FC7C433ED for ; Tue, 13 Apr 2021 06:58:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9EAE460FDB for ; Tue, 13 Apr 2021 06:57:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9EAE460FDB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3E8BF6B0070; Tue, 13 Apr 2021 02:57:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3BF666B0072; Tue, 13 Apr 2021 02:57:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 260436B0099; Tue, 13 Apr 2021 02:57:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id 0B70B6B0070 for ; Tue, 13 Apr 2021 02:57:59 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id C475F81D1 for ; Tue, 13 Apr 2021 06:57:58 +0000 (UTC) X-FDA: 78026439036.32.7D571D3 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf18.hostedemail.com (Postfix) with ESMTP id 18470200025D for ; Tue, 13 Apr 2021 06:57:58 +0000 (UTC) Received: by mail-pj1-f54.google.com with SMTP id i4so7969796pjk.1 for ; Mon, 12 Apr 2021 23:57:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2FD/YxJiXhb6Cqn7dGyIHqv5BROgaaIUxtbd/p90WjE=; b=ON6KiOCCTjcGcoOkHzXj1pJvR4mRoIT7hOPDgNKxVoRqOyrfkENxRRwe4JEVHfsquX C2ptIpImrLFi01ukcvaePpwmBK17BAVtOXtrB6YFcnAPhRqTXiV1P8CSMiD4XceSPca+ s1fdLLTorjlvdVo/4+UXWv9+E7DTlYF5dioGf4ptggNEwkXVFIuoC+DjpwU8R6am+CrW rFzwj4Z4H0oCd2o8/B05axkCMQokpYzyK7V0ZAMvP2N6+d1KDgtv0czSEw8qxd0L7nTK x2ezcfsydFc9sP/ADeSMpEWLLAvJaYth5Efr7oay0cNJm2TjjGROJqtkmIRQf4vXZq+r hliA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2FD/YxJiXhb6Cqn7dGyIHqv5BROgaaIUxtbd/p90WjE=; b=dcSzZ6b5OTR4CZDUsThQ+hFE0AK+D3JPdIvjmUOr1jHHCB4GNEFnTgjkbEeXu0mI6U wO2Hh7bsoAiuYSD06jWSnbwuy60rxIHwViANfEutW2cYHI8Brrdxq81Q7n69XfrwKnyl nc64mytC1UCEwIonbvkAEdit4oEOPhlZn3CrisV2Xr1zAkYIpfAN2FmoZFlXnoK2mGqY fTsQnbJdz3e7ukSfgvLfZfrf8+BdtVUovFNMngQY8DIKuLm0cUNiXX/vf4prdG6NtCiJ 1FTR8ZlVq4ABaQdxPqSNb6khWnr+5PitawT7lh8J7zvBznP7Cx4+stWMiGqCyl8ESXuG 1KbA== X-Gm-Message-State: AOAM5315cjKJ/OuE+Nh2F6PX2Ryg8XNkv7P8o6bPSv0q9rgNt+rKffVR TBU/QGqTE94Uz1i4kI+Auk35EA== X-Google-Smtp-Source: ABdhPJwzVPCZNKsdmtjXdTO2whihSoz7AxS92CX6eF1MOYBAjKNAw0sm3rds4M/24NuYVkeznKTcXg== X-Received: by 2002:a17:90a:528b:: with SMTP id w11mr3371687pjh.162.1618297077558; Mon, 12 Apr 2021 23:57:57 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.241]) by smtp.gmail.com with ESMTPSA id 78sm11508562pfu.40.2021.04.12.23.57.53 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Apr 2021 23:57:57 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH 5/7] mm: memcontrol: simplify the logic of objcg pinning memcg Date: Tue, 13 Apr 2021 14:51:51 +0800 Message-Id: <20210413065153.63431-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210413065153.63431-1-songmuchun@bytedance.com> References: <20210413065153.63431-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 18470200025D X-Stat-Signature: zz9fbjcjks8grx4rge39ncjpc31t1noh Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail-pj1-f54.google.com; client-ip=209.85.216.54 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618297078-259698 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The obj_cgroup_release() and memcg_reparent_objcgs() are serialized by the css_set_lock. We do not need to care about objcg->memcg being released in the process of obj_cgroup_release(). So there is no need to pin memcg before releasing objcg. Remove those pinning logic to simplfy the code. There are only two places that modifies the objcg->memcg. One is the initialization to objcg->memcg in the memcg_online_kmem(), another is objcgs reparenting in the memcg_reparent_objcgs(). It is also impossible for the two to run in parallel. So xchg() is unnecessary and it is enough to use WRITE_ONCE(). Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin --- mm/memcontrol.c | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 1f807448233e..42d8c0f4ab1d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -261,7 +261,6 @@ static void obj_cgroup_uncharge_pages(struct obj_cgroup *objcg, static void obj_cgroup_release(struct percpu_ref *ref) { struct obj_cgroup *objcg = container_of(ref, struct obj_cgroup, refcnt); - struct mem_cgroup *memcg; unsigned int nr_bytes; unsigned int nr_pages; unsigned long flags; @@ -291,11 +290,9 @@ static void obj_cgroup_release(struct percpu_ref *ref) nr_pages = nr_bytes >> PAGE_SHIFT; spin_lock_irqsave(&css_set_lock, flags); - memcg = obj_cgroup_memcg(objcg); if (nr_pages) obj_cgroup_uncharge_pages(objcg, nr_pages); list_del(&objcg->list); - mem_cgroup_put(memcg); spin_unlock_irqrestore(&css_set_lock, flags); percpu_ref_exit(ref); @@ -330,17 +327,12 @@ static void memcg_reparent_objcgs(struct mem_cgroup *memcg, spin_lock_irq(&css_set_lock); - /* Move active objcg to the parent's list */ - xchg(&objcg->memcg, parent); - css_get(&parent->css); - list_add(&objcg->list, &parent->objcg_list); - - /* Move already reparented objcgs to the parent's list */ - list_for_each_entry(iter, &memcg->objcg_list, list) { - css_get(&parent->css); - xchg(&iter->memcg, parent); - css_put(&memcg->css); - } + /* 1) Ready to reparent active objcg. */ + list_add(&objcg->list, &memcg->objcg_list); + /* 2) Reparent active objcg and already reparented objcgs to parent. */ + list_for_each_entry(iter, &memcg->objcg_list, list) + WRITE_ONCE(iter->memcg, parent); + /* 3) Move already reparented objcgs to the parent's list */ list_splice(&memcg->objcg_list, &parent->objcg_list); spin_unlock_irq(&css_set_lock); From patchwork Tue Apr 13 06:51:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12199461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61236C433B4 for ; Tue, 13 Apr 2021 06:58:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 00622613AE for ; Tue, 13 Apr 2021 06:58:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 00622613AE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9681A6B0072; Tue, 13 Apr 2021 02:58:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 93DDB6B009C; Tue, 13 Apr 2021 02:58:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DE6E6B009D; Tue, 13 Apr 2021 02:58:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0036.hostedemail.com [216.40.44.36]) by kanga.kvack.org (Postfix) with ESMTP id 615106B0072 for ; Tue, 13 Apr 2021 02:58:03 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1CC37180ACF75 for ; Tue, 13 Apr 2021 06:58:03 +0000 (UTC) X-FDA: 78026439246.24.894E9BE Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf20.hostedemail.com (Postfix) with ESMTP id 404B7DE for ; Tue, 13 Apr 2021 06:57:58 +0000 (UTC) Received: by mail-pl1-f172.google.com with SMTP id m18so5479020plc.13 for ; Mon, 12 Apr 2021 23:58:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rlxS2NP/wFG07icClsRwQltSV/pe4IUbel0wnvFN2yM=; b=kKWMZutIU29NqKffRf7Rn3aW5tj6h3ArneM4CjccRqhy83CknU6Gjh1hdO/OGbN9XD /d1X8d4GxEXJfe28RTBxeBtSIC/pRUZCLQ96RzX42IWktE6+V4s0YNzR5GCwuPrtv+ub uA3slpqDt404SoidWY6YzIg5SBoKC+FvmC8b5ADeGs1v85JFo+AoOULYxXA+ndidbZfm 6AzqCNtTMtve4/uLh/ID2es0t1iHBlF2/ac2pnRjtwTpQpO8ZQAq45dVJ5jO52h8Y57A chftzkHYqBMkhc3ji88AaC8sNoUayvBuxLOp6AyqFKxjEglR2CqYIuE5NFz8JkaFA00k PTQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rlxS2NP/wFG07icClsRwQltSV/pe4IUbel0wnvFN2yM=; b=i4ke688UvLFNFvj6RlF88cjCvcfrUkLWKhF5itkNtfuikLLbeEbTAmvKLifbKAClw9 eN4NMIm23X4C84IAvvGd4nWjoJ8zrUXE1dg2IydXCDpex0SoK2r3a8NaAUf8Y0M8zdnB GRtlpAx0z1OfBbTNmepdtqs7YKCFbsmMpnqWNX0cux8DJufCi08k24a9/dYg4D6kGor1 mWDfcipZ5EiDmdPR0jtVxZxBulkj2XIkxwbI/Fz7IdWJ4UWqSfgNtON5Q5PqCDY4NxQ1 dYc29Rcl5qKBqRhCXQTMVwR/LN1JoGciFrm69q8SNS3kTO1GfgPc75Ub/Oc6c13QFwJK r1Xg== X-Gm-Message-State: AOAM532LzLjV+t1c98lHJTbfWX+IJ1NYfZW7lcjigrLsQG8xvEFzbef5 l6pXKUl+zmk07R9h7J6L8nO3bw== X-Google-Smtp-Source: ABdhPJxxd8+LsLVbpUFdRYV2g1kphK8cC2VJ47h5BOjiz+ON6vX30VKce7mGgwpcf0LEwmJYYA4EKw== X-Received: by 2002:a17:90a:e50a:: with SMTP id t10mr3417568pjy.32.1618297082039; Mon, 12 Apr 2021 23:58:02 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.241]) by smtp.gmail.com with ESMTPSA id 78sm11508562pfu.40.2021.04.12.23.57.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Apr 2021 23:58:01 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH 6/7] mm: memcontrol: move obj_cgroup_uncharge_pages() out of css_set_lock Date: Tue, 13 Apr 2021 14:51:52 +0800 Message-Id: <20210413065153.63431-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210413065153.63431-1-songmuchun@bytedance.com> References: <20210413065153.63431-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 404B7DE X-Stat-Signature: r5d87q64fzxdhqr6dtxfs3r444uopcqx X-Rspamd-Server: rspam02 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail-pl1-f172.google.com; client-ip=209.85.214.172 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618297078-125583 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The css_set_lock is used to guard the list of inherited objcgs. So there is no need to uncharge kernel memory under css_set_lock. Just move it out of the lock. Signed-off-by: Muchun Song Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Johannes Weiner --- mm/memcontrol.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 42d8c0f4ab1d..d9c7e44abcd0 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -289,9 +289,10 @@ static void obj_cgroup_release(struct percpu_ref *ref) WARN_ON_ONCE(nr_bytes & (PAGE_SIZE - 1)); nr_pages = nr_bytes >> PAGE_SHIFT; - spin_lock_irqsave(&css_set_lock, flags); if (nr_pages) obj_cgroup_uncharge_pages(objcg, nr_pages); + + spin_lock_irqsave(&css_set_lock, flags); list_del(&objcg->list); spin_unlock_irqrestore(&css_set_lock, flags); From patchwork Tue Apr 13 06:51:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12199463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80A37C433B4 for ; Tue, 13 Apr 2021 06:58:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 19792613AE for ; Tue, 13 Apr 2021 06:58:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 19792613AE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A9F366B009C; Tue, 13 Apr 2021 02:58:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A75656B009E; Tue, 13 Apr 2021 02:58:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93F296B009F; Tue, 13 Apr 2021 02:58:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0062.hostedemail.com [216.40.44.62]) by kanga.kvack.org (Postfix) with ESMTP id 78C936B009C for ; Tue, 13 Apr 2021 02:58:07 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 3AFF08792 for ; Tue, 13 Apr 2021 06:58:07 +0000 (UTC) X-FDA: 78026439414.02.B007AA6 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf20.hostedemail.com (Postfix) with ESMTP id 6E36513A for ; Tue, 13 Apr 2021 06:58:02 +0000 (UTC) Received: by mail-pg1-f182.google.com with SMTP id q10so11247935pgj.2 for ; Mon, 12 Apr 2021 23:58:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bdhaQ5mP8Ps2L1OS7f25L6onCgsPCveY4DfI6OcUTcA=; b=zGHAEn5+HYX5vVZwfFXCcJNdGf6EJdpTl1UmeK5gSzvLt16cZ4/g1Qs53J0H0mtrdt HumqY6jWtfForTqeYwAAPCzyywtO+c+A0kEohE6BZCcpzejZt36TkbLlfj4xitbnfGRR zv7DhDVV53ex62qdLzmAjz7sb41qMJ+awZfmCAK636xBgZYJF3Hr7ykdOMkgmyGJRKiu 3Uj7ti3Sc9tyU5RYMveeV0ABz+EJwCc73UZZdLVXdjAUiLI70A37vdWl7UIY5HITqh+7 4cpZ1lJmVsms91ZQAn0kiELwhnBQuSH3ZEl2c3RibhgZsjAsWIAk2ObGXXCT2x19KRZU MOOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bdhaQ5mP8Ps2L1OS7f25L6onCgsPCveY4DfI6OcUTcA=; b=etMCv3K7HfbH8cl0O2GfYZB1zwAmlk6Uji10QUjxlJgG3rmHuRRRGKQX9UfeQ4ab6f efvj2A5d0s+iDc1p3SPsaVCJYkmHDpQ3AMbDZ9VHf+K84ABtWGiM/5jhYvaod4oT8TSr BzWW15+scVyjFII/ac+VtdWOd27arNQ4KJLOjU6RPmgwdnj6WED06fzm2NA5w0frR7GQ XXHUoqDg599vG1UIcxxh0jRPRH/hyPaq5waHq/JcAnOSscIeIET8noEdHiss4/3ZeQHM /dTXt1KHN4egWTwD3mH8PMYTkyEC0Jc6qrlrZqcEi1DsIeYMmL+T7VAwqZpEMunyHHeL meww== X-Gm-Message-State: AOAM531us96yMyo/LfyemUGfec5X87M55FRS++sYNEHBsCHl9VIEum5o FCMdkUug/pIunHiJ1Lghm/6ajA== X-Google-Smtp-Source: ABdhPJxkqc0xpu3YaF1nM5me0VVuSkfci2Ug8vJGDZJJiMNgpqzmmeNsvLCbaDDCl7NfZxWPPWIj8w== X-Received: by 2002:a63:e707:: with SMTP id b7mr30425925pgi.389.1618297086205; Mon, 12 Apr 2021 23:58:06 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.241]) by smtp.gmail.com with ESMTPSA id 78sm11508562pfu.40.2021.04.12.23.58.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Apr 2021 23:58:05 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH 7/7] mm: vmscan: remove noinline_for_stack Date: Tue, 13 Apr 2021 14:51:53 +0800 Message-Id: <20210413065153.63431-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210413065153.63431-1-songmuchun@bytedance.com> References: <20210413065153.63431-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6E36513A X-Stat-Signature: 3nf14hg7udq8yunh89g1jnodxjg4yp8w X-Rspamd-Server: rspam02 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail-pg1-f182.google.com; client-ip=209.85.215.182 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618297082-739763 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The noinline_for_stack is introduced by commit 666356297ec4 ("vmscan: set up pagevec as late as possible in shrink_inactive_list()"), its purpose is to delay the allocation of pagevec as late as possible to save stack memory. But the commit 2bcf88796381 ("mm: take pagevecs off reclaim stack") replace pagevecs by lists of pages_to_free. So we do not need noinline_for_stack, just remove it (let the compiler decide whether to inline). Signed-off-by: Muchun Song Acked-by: Johannes Weiner Acked-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko --- mm/vmscan.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 64bf07cc20f2..e40b21298d77 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2015,8 +2015,8 @@ static int too_many_isolated(struct pglist_data *pgdat, int file, * * Returns the number of pages moved to the given lruvec. */ -static unsigned noinline_for_stack move_pages_to_lru(struct lruvec *lruvec, - struct list_head *list) +static unsigned int move_pages_to_lru(struct lruvec *lruvec, + struct list_head *list) { int nr_pages, nr_moved = 0; LIST_HEAD(pages_to_free); @@ -2096,7 +2096,7 @@ static int current_may_throttle(void) * shrink_inactive_list() is a helper for shrink_node(). It returns the number * of reclaimed pages */ -static noinline_for_stack unsigned long +static unsigned long shrink_inactive_list(unsigned long nr_to_scan, struct lruvec *lruvec, struct scan_control *sc, enum lru_list lru) {