From patchwork Fri Apr 16 05:14:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12206695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FDB8C433ED for ; Fri, 16 Apr 2021 05:15:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CB7DD610FB for ; Fri, 16 Apr 2021 05:15:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CB7DD610FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1B1056B0070; Fri, 16 Apr 2021 01:15:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 15E2B6B0072; Fri, 16 Apr 2021 01:15:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EFE5D6B0071; Fri, 16 Apr 2021 01:15:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0065.hostedemail.com [216.40.44.65]) by kanga.kvack.org (Postfix) with ESMTP id D1B806B0036 for ; Fri, 16 Apr 2021 01:15:45 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 969C61F1B for ; Fri, 16 Apr 2021 05:15:45 +0000 (UTC) X-FDA: 78037067850.16.CE92A06 Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) by imf18.hostedemail.com (Postfix) with ESMTP id 4023F2000254 for ; Fri, 16 Apr 2021 05:15:46 +0000 (UTC) Received: by mail-pg1-f171.google.com with SMTP id t22so18479786pgu.0 for ; Thu, 15 Apr 2021 22:15:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8k6YXYK85mvc31ajI7k2coqgwaOfr7Ej3V6vhYCGVRI=; b=efWsErrRy6Y+q7AbkIPIOy5Upya1t/n2SZE33fPb3OfiCt8/AoLJQ8+y3xKWrG3giU /ufhSnUuwyEPh9AXrW1YrV1mDst07Js6C7JCixrgEMisbo+cOfQEAS3csT9k4iKIxQxB MlQ92g9HdhU7uf7ItfpepWKimxH+L7RVtrvC4bbC9xr+PAgkSgdVvMna7FXQ2CtBuOar hRpDe7r7VPNd/d4DlGDeGMwhHK+jpovUERNHZtSxPOu4z1WWMgp2J5DDsMQZ2XuL+PLs r45bBJPkpvHKzN+U+a6SsByLwC/ijypTht26AWuoHyInoGpG/6fG3hlXiq77OGnPtmUt raNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8k6YXYK85mvc31ajI7k2coqgwaOfr7Ej3V6vhYCGVRI=; b=X2y+lsbp34vRUIJmQ6w88ynQ/NqCdOlHxJSXrpfd6jXGFr2XjrOJlzvGbV2Z6equ5g 7J7fvwmqrqtwZmDw4n5Mq3aXrGJAKKvbdthYtHXouaSuka3EALV/jf7o5EBD5w0w2UIn +AKE6Lo9KmmnG3omnZOadGc2Cj8/yfFptwogSaWFBUEERLbilZ5ho2CujA10125rZk47 pn06z0I6J4ffxZ74NrCeCbcyA4B3x0bDOw700+qz7rXNJJ/c7OXgdCctcCzyWfBdiNm4 cUx+Z6E/NOMovcLrcOnSb2aDhgHOVihICDNC8ZqQkZ/ZNKVW/EurQ8p1f0H7Xg6x/jhc KQAA== X-Gm-Message-State: AOAM530TmXFgaMVnXBdugEVCNQ+hnIYvb/1RqRQwmBkGAJlPVpRNEZCB uROBOwTmtVQC8geGnT0xdyfd5el5WtXC0Pid X-Google-Smtp-Source: ABdhPJypRzf2gPZt+cnsgCnfpajGcfQTSzsxaz5LpZts4H35QQ/kLfNwffTKJXeXBHRxNEagMZVCfQ== X-Received: by 2002:a63:5857:: with SMTP id i23mr6629207pgm.152.1618550144354; Thu, 15 Apr 2021 22:15:44 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.235]) by smtp.gmail.com with ESMTPSA id j7sm3518540pfd.129.2021.04.15.22.15.40 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Apr 2021 22:15:44 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v2 1/8] mm: memcontrol: fix page charging in page replacement Date: Fri, 16 Apr 2021 13:14:00 +0800 Message-Id: <20210416051407.54878-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210416051407.54878-1-songmuchun@bytedance.com> References: <20210416051407.54878-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 4023F2000254 X-Stat-Signature: q6p6hc1qad3z1yqf45ybx3tq7rsrrd54 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail-pg1-f171.google.com; client-ip=209.85.215.171 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618550146-41510 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The pages aren't accounted at the root level, so do not charge the page to the root memcg in page replacement. Although we do not display the value (mem_cgroup_usage) so there shouldn't be any actual problem, but there is a WARN_ON_ONCE in the page_counter_cancel(). Who knows if it will trigger? So it is better to fix it. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- mm/memcontrol.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 64ada9e650a5..f229de925aa5 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6806,9 +6806,11 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage) /* Force-charge the new page. The old one will be freed soon */ nr_pages = thp_nr_pages(newpage); - page_counter_charge(&memcg->memory, nr_pages); - if (do_memsw_account()) - page_counter_charge(&memcg->memsw, nr_pages); + if (!mem_cgroup_is_root(memcg)) { + page_counter_charge(&memcg->memory, nr_pages); + if (do_memsw_account()) + page_counter_charge(&memcg->memsw, nr_pages); + } css_get(&memcg->css); commit_charge(newpage, memcg); From patchwork Fri Apr 16 05:14:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12206699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1F09C433B4 for ; Fri, 16 Apr 2021 05:15:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2E7336113B for ; Fri, 16 Apr 2021 05:15:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E7336113B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A87556B006C; Fri, 16 Apr 2021 01:15:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A358D6B0071; Fri, 16 Apr 2021 01:15:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FFD26B0072; Fri, 16 Apr 2021 01:15:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0112.hostedemail.com [216.40.44.112]) by kanga.kvack.org (Postfix) with ESMTP id 696316B006C for ; Fri, 16 Apr 2021 01:15:50 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 31DFE5848 for ; Fri, 16 Apr 2021 05:15:50 +0000 (UTC) X-FDA: 78037068060.17.B2EEEE6 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by imf24.hostedemail.com (Postfix) with ESMTP id 3D7A5A00038C for ; Fri, 16 Apr 2021 05:15:43 +0000 (UTC) Received: by mail-pj1-f41.google.com with SMTP id r13so9526273pjf.2 for ; Thu, 15 Apr 2021 22:15:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NE/BcoGMuqPyqQGyjWQGMTxC1WpQbz3TqD3QbwUvSZY=; b=x3mRR8adpG+SIPQvbajyUHRAzVYGQ9kXgMEs7JE0sm5iBBYsM7za2rneJ2x+zx7Alm nvjA0ySFpVE0MWfbuJUl+trTsqrxpnkGXWgmQ1htfWCvNWccRgTDGYGaEs7Feyw4sZ41 Nyo0Z2ol/a0h+wdqzCXC58YJtzlmXejXT0oumYu+JvdMKtxzxN6FdzbxwmbUg8/S9Eap H4ZZve05NA7mvYp2zP0e740L9pte8vGxHJ68iJKbrfp6Xk5BJVu0G0bgdZWFZhcNG0MR WAdfx26I8k/nlMtz6X2Uhe00xyCK4COeU4jU9z0DBaM4bXqoCGlrnT/S13+W2n51u6bb wnPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NE/BcoGMuqPyqQGyjWQGMTxC1WpQbz3TqD3QbwUvSZY=; b=BLqsrgwzGPbOy3Dq67PZMNVyil+8j1ufweIJ5e4yHXnrrjz+d7tzMASURTbkgM9Flh GcWdoSJ+jKETpiBIMFWWctp9RWX+jfuokmYvbIL+7IkvlqUp6JMiPxpwb6XqtjVH7osc QqMLP2otnU/XluG9exjGVpOHAi/nS6t4nK5VJz83jDIjvh0SrlMFQBr6tUfk2c5M6/Ru tIL2kp2DoC4rPp0niRIdMlKrdKQBMB810Jhtm3nyDQwaRGaTEYDmWORxw3Ydk+c41dUv /b9BkVudC8W5mDUUnPTlcoMAGTiNyfRrsxQ30kPbnHG77c/b+RJrVK1lRcl3ouf3urP5 B6Hg== X-Gm-Message-State: AOAM533c4UyXMOKKkveY2ZCD0veUWd5L7WkfcOSSDZ4NggNpqGm2nmK2 8G4Ezct+e+vnQmY2TObaffJeuA== X-Google-Smtp-Source: ABdhPJyRUkfUK1Gz4fhB+MEe5DNop4sWVG+HsPBKJEMPEQLpoPtCsMG3b1+SKM+CihVIFoRzjH8IfQ== X-Received: by 2002:a17:90a:9405:: with SMTP id r5mr8098835pjo.139.1618550148926; Thu, 15 Apr 2021 22:15:48 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.235]) by smtp.gmail.com with ESMTPSA id j7sm3518540pfd.129.2021.04.15.22.15.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Apr 2021 22:15:48 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v2 2/8] mm: memcontrol: bail out early when !mm in get_mem_cgroup_from_mm Date: Fri, 16 Apr 2021 13:14:01 +0800 Message-Id: <20210416051407.54878-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210416051407.54878-1-songmuchun@bytedance.com> References: <20210416051407.54878-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: sq7axp9xi768sxkqy8ugsqrtw9d84xnw X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3D7A5A00038C Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail-pj1-f41.google.com; client-ip=209.85.216.41 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618550143-413218 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When mm is NULL, we do not need to hold rcu lock and call css_tryget for the root memcg. And we also do not need to check !mm in every loop of while. So bail out early when !mm. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- mm/memcontrol.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f229de925aa5..50e3cf1e263e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -901,20 +901,23 @@ struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm) if (mem_cgroup_disabled()) return NULL; + /* + * Page cache insertions can happen without an + * actual mm context, e.g. during disk probing + * on boot, loopback IO, acct() writes etc. + * + * No need to css_get on root memcg as the reference + * counting is disabled on the root level in the + * cgroup core. See CSS_NO_REF. + */ + if (unlikely(!mm)) + return root_mem_cgroup; + rcu_read_lock(); do { - /* - * Page cache insertions can happen without an - * actual mm context, e.g. during disk probing - * on boot, loopback IO, acct() writes etc. - */ - if (unlikely(!mm)) + memcg = mem_cgroup_from_task(rcu_dereference(mm->owner)); + if (unlikely(!memcg)) memcg = root_mem_cgroup; - else { - memcg = mem_cgroup_from_task(rcu_dereference(mm->owner)); - if (unlikely(!memcg)) - memcg = root_mem_cgroup; - } } while (!css_tryget(&memcg->css)); rcu_read_unlock(); return memcg; From patchwork Fri Apr 16 05:14:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12206701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AE10C433B4 for ; Fri, 16 Apr 2021 05:15:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AC6666113D for ; Fri, 16 Apr 2021 05:15:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC6666113D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 296FB6B0073; Fri, 16 Apr 2021 01:15:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 26FCC6B0074; Fri, 16 Apr 2021 01:15:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 15E0C6B0075; Fri, 16 Apr 2021 01:15:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0027.hostedemail.com [216.40.44.27]) by kanga.kvack.org (Postfix) with ESMTP id E61986B0073 for ; Fri, 16 Apr 2021 01:15:54 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B27453489 for ; Fri, 16 Apr 2021 05:15:54 +0000 (UTC) X-FDA: 78037068228.33.5A90B19 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf02.hostedemail.com (Postfix) with ESMTP id A3FBE40002C0 for ; Fri, 16 Apr 2021 05:15:36 +0000 (UTC) Received: by mail-pf1-f170.google.com with SMTP id a12so17574933pfc.7 for ; Thu, 15 Apr 2021 22:15:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CUAxTMYGPKcXUKklEvyCvysOG+ZnpjCi0FGjCzotqSU=; b=sKTfHmqSsWDRGf9SVi9pCO6Hmse+SZHM+tv3/GJl2rgPhf5LhsNUzYPF8GIbtfPjmi RLQSfJGWAZtFFG5WKBMO/UEP4u+7lJ86fcGy3qMcMBQRs16QYPNN4osVE2B7q3wPm3IM 6k/5JyFHsfTjzDmOAZ2WZ21L69Y6as0JcteWUeVjQxAJsR7i3UC4Bj6clihPikjYzYdB MkpMLzKj7ZirphY+yFO6Wt5BcH6ICOyYy2mJsVMItKl0chzciFMZA+B16wt9PEMImgIe 0mu9QbgV/a2EWzTQvbMRBfNglm3uT3KAt50tf3sstmbeqzHvrmzQeC+1dNyS4ySuO74L kJLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CUAxTMYGPKcXUKklEvyCvysOG+ZnpjCi0FGjCzotqSU=; b=ZOGP8j0TZUvzUX2NabvKxmvLSt8ZCLGDFf7f0nOnfZKGBhrd4E3fBN5ATvrxxhspD3 bPN5cKnR/CHO5bVm4unpVPoKl+QuIQ4Dd7J9MyzuVNnjhT6V+1YaeTykBQd1diYXabBW xC8A6c+VWcV08fHwRjXnnCGCwNTgEQrjHFb+pA5/CXO/jSkBFD6M28nqKXQTM/zglO7C 267zI1420aL/wWAxbxaOIW5PB2BM3u1sUqWJHoyl+wx/4mt/bv4n0MIjtZMknxH+uWVD 0YZ3hhjogoVPnSWK3v7UNOVVkVPA4Ybwd0fOuaIU2q9ZScWgvn2ptL8PKSJV7oN1wI6T Ip4Q== X-Gm-Message-State: AOAM531Bf2uQQur8CP562SdUFHb0QO2Y5PKE87kM7jPdqp5uyl/dVroH tXVAN0U4TE140mESH+m/Mmf3lA== X-Google-Smtp-Source: ABdhPJylZLX97P+G2iOwpp72wGJA5EJe8joOYASW+03d0LyiRtMtN0GtaCcX9czglyk4+IJR8Rxong== X-Received: by 2002:a62:6883:0:b029:220:4426:449c with SMTP id d125-20020a6268830000b02902204426449cmr6160037pfc.14.1618550153443; Thu, 15 Apr 2021 22:15:53 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.235]) by smtp.gmail.com with ESMTPSA id j7sm3518540pfd.129.2021.04.15.22.15.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Apr 2021 22:15:53 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v2 3/8] mm: memcontrol: remove the pgdata parameter of mem_cgroup_page_lruvec Date: Fri, 16 Apr 2021 13:14:02 +0800 Message-Id: <20210416051407.54878-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210416051407.54878-1-songmuchun@bytedance.com> References: <20210416051407.54878-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A3FBE40002C0 X-Stat-Signature: hocd95xr8stankyzb15beap6ww7f8ptk Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from=""; helo=mail-pf1-f170.google.com; client-ip=209.85.210.170 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618550136-669906 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All the callers of mem_cgroup_page_lruvec() just pass page_pgdat(page) as the 2nd parameter to it (except isolate_migratepages_block()). But for isolate_migratepages_block(), the page_pgdat(page) is also equal to the local variable of @pgdat. So mem_cgroup_page_lruvec() do not need the pgdat parameter. Just remove it to simplify the code. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- include/linux/memcontrol.h | 10 +++++----- mm/compaction.c | 2 +- mm/memcontrol.c | 9 +++------ mm/swap.c | 2 +- mm/workingset.c | 2 +- 5 files changed, 11 insertions(+), 14 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index c193be760709..f2a5aaba3577 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -743,13 +743,12 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg, /** * mem_cgroup_page_lruvec - return lruvec for isolating/putting an LRU page * @page: the page - * @pgdat: pgdat of the page * * This function relies on page->mem_cgroup being stable. */ -static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page, - struct pglist_data *pgdat) +static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page) { + pg_data_t *pgdat = page_pgdat(page); struct mem_cgroup *memcg = page_memcg(page); VM_WARN_ON_ONCE_PAGE(!memcg && !mem_cgroup_disabled(), page); @@ -1221,9 +1220,10 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg, return &pgdat->__lruvec; } -static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page, - struct pglist_data *pgdat) +static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page) { + pg_data_t *pgdat = page_pgdat(page); + return &pgdat->__lruvec; } diff --git a/mm/compaction.c b/mm/compaction.c index 8c5028bfbd56..1c500e697c88 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -994,7 +994,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, if (!TestClearPageLRU(page)) goto isolate_fail_put; - lruvec = mem_cgroup_page_lruvec(page, pgdat); + lruvec = mem_cgroup_page_lruvec(page); /* If we already hold the lock, we can skip some rechecking */ if (lruvec != locked) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 50e3cf1e263e..caf193088beb 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1181,9 +1181,8 @@ void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page) struct lruvec *lock_page_lruvec(struct page *page) { struct lruvec *lruvec; - struct pglist_data *pgdat = page_pgdat(page); - lruvec = mem_cgroup_page_lruvec(page, pgdat); + lruvec = mem_cgroup_page_lruvec(page); spin_lock(&lruvec->lru_lock); lruvec_memcg_debug(lruvec, page); @@ -1194,9 +1193,8 @@ struct lruvec *lock_page_lruvec(struct page *page) struct lruvec *lock_page_lruvec_irq(struct page *page) { struct lruvec *lruvec; - struct pglist_data *pgdat = page_pgdat(page); - lruvec = mem_cgroup_page_lruvec(page, pgdat); + lruvec = mem_cgroup_page_lruvec(page); spin_lock_irq(&lruvec->lru_lock); lruvec_memcg_debug(lruvec, page); @@ -1207,9 +1205,8 @@ struct lruvec *lock_page_lruvec_irq(struct page *page) struct lruvec *lock_page_lruvec_irqsave(struct page *page, unsigned long *flags) { struct lruvec *lruvec; - struct pglist_data *pgdat = page_pgdat(page); - lruvec = mem_cgroup_page_lruvec(page, pgdat); + lruvec = mem_cgroup_page_lruvec(page); spin_lock_irqsave(&lruvec->lru_lock, *flags); lruvec_memcg_debug(lruvec, page); diff --git a/mm/swap.c b/mm/swap.c index a75a8265302b..e0d5699213cc 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -313,7 +313,7 @@ void lru_note_cost(struct lruvec *lruvec, bool file, unsigned int nr_pages) void lru_note_cost_page(struct page *page) { - lru_note_cost(mem_cgroup_page_lruvec(page, page_pgdat(page)), + lru_note_cost(mem_cgroup_page_lruvec(page), page_is_file_lru(page), thp_nr_pages(page)); } diff --git a/mm/workingset.c b/mm/workingset.c index b7cdeca5a76d..4f7a306ce75a 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -408,7 +408,7 @@ void workingset_activation(struct page *page) memcg = page_memcg_rcu(page); if (!mem_cgroup_disabled() && !memcg) goto out; - lruvec = mem_cgroup_page_lruvec(page, page_pgdat(page)); + lruvec = mem_cgroup_page_lruvec(page); workingset_age_nonresident(lruvec, thp_nr_pages(page)); out: rcu_read_unlock(); From patchwork Fri Apr 16 05:14:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12206703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7F7DC433B4 for ; Fri, 16 Apr 2021 05:16:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 76F0D6113D for ; Fri, 16 Apr 2021 05:16:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76F0D6113D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 15E546B0075; Fri, 16 Apr 2021 01:16:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 135666B0078; Fri, 16 Apr 2021 01:16:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F18316B007B; Fri, 16 Apr 2021 01:16:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0240.hostedemail.com [216.40.44.240]) by kanga.kvack.org (Postfix) with ESMTP id C304B6B0075 for ; Fri, 16 Apr 2021 01:16:00 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 7E71D180995C2 for ; Fri, 16 Apr 2021 05:16:00 +0000 (UTC) X-FDA: 78037068480.34.5A37086 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf24.hostedemail.com (Postfix) with ESMTP id 7AFDEA0003A2 for ; Fri, 16 Apr 2021 05:15:53 +0000 (UTC) Received: by mail-pf1-f181.google.com with SMTP id i190so17574938pfc.12 for ; Thu, 15 Apr 2021 22:16:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EITONQXFUiMqxrQ5NUsYlzJaHXQ6TTbcz+uz38aCyj8=; b=SuUeEWi3hkyym9C+nZPi7l4X3qRxD6TQytpUw/1EHGepojFemgYBIwmEu4XWEYjzRU an93ortrd6A7N9bn9ygLjYAeG6tEhXijd+Gp2dFmewjtt8Y1BXdbX0UYctCOSDbYaGJe b9hQRoOKYWATBes4K+Q4crKShSdfA1/k3ry8JPuwHhRcJ2oFjLBYQDL3wGIYmxrBe8Jg V9+K60ehr1o+PctGj1dJdVcEYAsvz0WkuOBMUBhiriTc1dXJcLyKJdTLeV1uBVAUYxh+ J2vGVMqehdk5QvqMcgGo39GgUTujIeA5A48sPaQtouMdkT2OTMa0pXP76k7LVxBXs+23 +VHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EITONQXFUiMqxrQ5NUsYlzJaHXQ6TTbcz+uz38aCyj8=; b=DRl/ffGFfSZAwnE/2u6KRqTfm60XTVReXuhpsN6o2Y5TT1zvyKke6XKWvLoYq4IgcA 6HwgOd5kgjZ2O3PSUgNJ5dQ922IEXH1YERiwLAA4Tf/8ryZUG2OhCCTGAHzrwU28MHvr k7TGRZdOeiOau4XJQbXBmadzE+azt23SpsWZoQsTBuhmQ8ZNCCRSQcLjf1Ilb4EMW8Tw k3UVvIoZPRlle85OHr6Ba75PLcS+4OSOG/BagSGU+vN+XgZafZn62A8qUHS5fCoT0hvE 5hqjpij4C0nioJlJdoGMqpvEe2tglepLfMJum0pb4KUj9sXJTSlb/E3OFjsUhrYqZ+QK 2gAw== X-Gm-Message-State: AOAM533YFTZkp1VVZrRQwgmPGkcqHPak3FUE2fykCZcQnO9yLCES65oi j0qWkB5b92F8e14fG3wCT7/Nqg== X-Google-Smtp-Source: ABdhPJxQ15461bx5MeGSdaLrFIJ1l7r1oqawxEatcnKC8zUEEH8blyrZiTvRaboYm6nB5iYTJ3lp/g== X-Received: by 2002:aa7:8593:0:b029:259:c31b:945b with SMTP id w19-20020aa785930000b0290259c31b945bmr2025912pfn.27.1618550159354; Thu, 15 Apr 2021 22:15:59 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.235]) by smtp.gmail.com with ESMTPSA id j7sm3518540pfd.129.2021.04.15.22.15.53 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Apr 2021 22:15:59 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v2 4/8] mm: memcontrol: simplify lruvec_holds_page_lru_lock Date: Fri, 16 Apr 2021 13:14:03 +0800 Message-Id: <20210416051407.54878-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210416051407.54878-1-songmuchun@bytedance.com> References: <20210416051407.54878-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: o5xwjfnzq1owu9tyg7u4oa4kza93ghuy X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7AFDEA0003A2 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail-pf1-f181.google.com; client-ip=209.85.210.181 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618550153-192501 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We already have a helper lruvec_memcg() to get the memcg from lruvec, we do not need to do it ourselves in the lruvec_holds_page_lru_lock(). So use lruvec_memcg() instead. And if mem_cgroup_disabled() returns false, the page_memcg(page) (the LRU pages) cannot be NULL. So remove the odd logic of "memcg = page_memcg(page) ? : root_mem_cgroup". And use lruvec_pgdat to simplify the code. We can have a single definition for this function that works for !CONFIG_MEMCG, CONFIG_MEMCG + mem_cgroup_disabled() and CONFIG_MEMCG. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- include/linux/memcontrol.h | 31 +++++++------------------------ 1 file changed, 7 insertions(+), 24 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index f2a5aaba3577..2fc728492c9b 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -755,22 +755,6 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page) return mem_cgroup_lruvec(memcg, pgdat); } -static inline bool lruvec_holds_page_lru_lock(struct page *page, - struct lruvec *lruvec) -{ - pg_data_t *pgdat = page_pgdat(page); - const struct mem_cgroup *memcg; - struct mem_cgroup_per_node *mz; - - if (mem_cgroup_disabled()) - return lruvec == &pgdat->__lruvec; - - mz = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - memcg = page_memcg(page) ? : root_mem_cgroup; - - return lruvec->pgdat == pgdat && mz->memcg == memcg; -} - struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p); struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm); @@ -1227,14 +1211,6 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page) return &pgdat->__lruvec; } -static inline bool lruvec_holds_page_lru_lock(struct page *page, - struct lruvec *lruvec) -{ - pg_data_t *pgdat = page_pgdat(page); - - return lruvec == &pgdat->__lruvec; -} - static inline void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page) { } @@ -1516,6 +1492,13 @@ static inline void unlock_page_lruvec_irqrestore(struct lruvec *lruvec, spin_unlock_irqrestore(&lruvec->lru_lock, flags); } +static inline bool lruvec_holds_page_lru_lock(struct page *page, + struct lruvec *lruvec) +{ + return lruvec_pgdat(lruvec) == page_pgdat(page) && + lruvec_memcg(lruvec) == page_memcg(page); +} + /* Don't lock again iff page's lruvec locked */ static inline struct lruvec *relock_page_lruvec_irq(struct page *page, struct lruvec *locked_lruvec) From patchwork Fri Apr 16 05:14:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12206705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70D33C433B4 for ; Fri, 16 Apr 2021 05:16:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 06468610FB for ; Fri, 16 Apr 2021 05:16:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 06468610FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 974DC6B007B; Fri, 16 Apr 2021 01:16:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 94AEA6B007D; Fri, 16 Apr 2021 01:16:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7EC646B007E; Fri, 16 Apr 2021 01:16:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0114.hostedemail.com [216.40.44.114]) by kanga.kvack.org (Postfix) with ESMTP id 60A2F6B007B for ; Fri, 16 Apr 2021 01:16:05 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 234681822B821 for ; Fri, 16 Apr 2021 05:16:05 +0000 (UTC) X-FDA: 78037068690.27.D53BAF8 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf16.hostedemail.com (Postfix) with ESMTP id 27BF980192DA for ; Fri, 16 Apr 2021 05:16:04 +0000 (UTC) Received: by mail-pf1-f176.google.com with SMTP id 10so8699791pfl.1 for ; Thu, 15 Apr 2021 22:16:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=quvF2EjF/xPK4PyQGSt8Z2m2h60Cl4Xz0YmhFi/qiI8=; b=wT8QOhtF0Xk3DzJURDe0I2FPnhhBjN6WF3ukTAggd/ZvLUS8hIyGkQMjuXxsfhlJNK MA7nkojnWv4ZUHPHV0/jKpXU9mf79GZsZcfx5Qgo+YDZiNXpmMp80mPzbja4fl+Z0nkr 7ny44DwYeAIT8iTY+qHqEulVRhBNOdmT7Z2pM82H9op1rRDKA5x8zQeQY+WP0mjjZYj4 xBDpQc4C7hAbt7Q3AtV6SlDYT2kaS6AK+XuBEeVzg48e+Wk+cyxRS95bJvwJZLx9T2bx PfzfJkPH/kiRnFiQqdhVAZYclcXAQl8mz/mlu8d45AoSRk+Z6AE2rKpAPNBNGgtk8lor Wtzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=quvF2EjF/xPK4PyQGSt8Z2m2h60Cl4Xz0YmhFi/qiI8=; b=I6eJtrMdjTpai4dC+auvYiYzUiwi60Uzsd9ew8YaAt1gmjN262QLQvr0I8XqbzV7V9 5ToxLn3txuzl99wQhnfGOt52CZ6+eQNG7YVOJXBm3fYUkVjpyiHZ0YtKdyt6x5b1wTqg URC0eM7h28dB/a6vMV1z8VVwQ+2nrPID+osa5BxjgKZvwCfAqTu4TOIiNHUxiECM+LH1 xp0bbaOnpd8472ev47TAdgiq/ZvgC7fWBx378OjY78xMevGmv4ffGSNVtOGvfAh0dIUB /ke1LjC82TsaILK6msqwNa3rjCdbQb9MUu9iqVovsefklzTniKQw8LQ5fjQkLdoaS8Q+ VUbg== X-Gm-Message-State: AOAM530WE6VKi8Cb4t5iS4eeRe1+U3SYji0O2iZWp6bCe/ujaEDSz4l4 cNM9QrR0vn+6Koed2Ix9pSPy/A== X-Google-Smtp-Source: ABdhPJwLW+IwtkWVKRKXZQMrb5e0hyLANL5+//jsyPZ/VYECJqDhd2JzUG8ROI5TDfmpH9bmgwID1A== X-Received: by 2002:a62:1ec1:0:b029:24d:b3de:25be with SMTP id e184-20020a621ec10000b029024db3de25bemr6421893pfe.17.1618550163986; Thu, 15 Apr 2021 22:16:03 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.235]) by smtp.gmail.com with ESMTPSA id j7sm3518540pfd.129.2021.04.15.22.15.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Apr 2021 22:16:03 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song Subject: [PATCH v2 5/8] mm: memcontrol: rename lruvec_holds_page_lru_lock to page_matches_lruvec Date: Fri, 16 Apr 2021 13:14:04 +0800 Message-Id: <20210416051407.54878-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210416051407.54878-1-songmuchun@bytedance.com> References: <20210416051407.54878-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 27BF980192DA X-Stat-Signature: 8t59qq6kysa5bt4jpsxfb177ocdfab7n Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail-pf1-f176.google.com; client-ip=209.85.210.176 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618550164-335934 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: lruvec_holds_page_lru_lock() doesn't check anything about locking and is used to check whether the page belongs to the lruvec. So rename it to page_matches_lruvec(). Signed-off-by: Muchun Song Acked-by: Michal Hocko Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt --- include/linux/memcontrol.h | 7 +++---- mm/vmscan.c | 2 +- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 2fc728492c9b..40b0c31ea4ba 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1492,8 +1492,7 @@ static inline void unlock_page_lruvec_irqrestore(struct lruvec *lruvec, spin_unlock_irqrestore(&lruvec->lru_lock, flags); } -static inline bool lruvec_holds_page_lru_lock(struct page *page, - struct lruvec *lruvec) +static inline bool page_matches_lruvec(struct page *page, struct lruvec *lruvec) { return lruvec_pgdat(lruvec) == page_pgdat(page) && lruvec_memcg(lruvec) == page_memcg(page); @@ -1504,7 +1503,7 @@ static inline struct lruvec *relock_page_lruvec_irq(struct page *page, struct lruvec *locked_lruvec) { if (locked_lruvec) { - if (lruvec_holds_page_lru_lock(page, locked_lruvec)) + if (page_matches_lruvec(page, locked_lruvec)) return locked_lruvec; unlock_page_lruvec_irq(locked_lruvec); @@ -1518,7 +1517,7 @@ static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page, struct lruvec *locked_lruvec, unsigned long *flags) { if (locked_lruvec) { - if (lruvec_holds_page_lru_lock(page, locked_lruvec)) + if (page_matches_lruvec(page, locked_lruvec)) return locked_lruvec; unlock_page_lruvec_irqrestore(locked_lruvec, *flags); diff --git a/mm/vmscan.c b/mm/vmscan.c index bb8321026c0c..2bc5cf409958 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2062,7 +2062,7 @@ static unsigned noinline_for_stack move_pages_to_lru(struct lruvec *lruvec, * All pages were isolated from the same lruvec (and isolation * inhibits memcg migration). */ - VM_BUG_ON_PAGE(!lruvec_holds_page_lru_lock(page, lruvec), page); + VM_BUG_ON_PAGE(!page_matches_lruvec(page, lruvec), page); add_page_to_lru_list(page, lruvec); nr_pages = thp_nr_pages(page); nr_moved += nr_pages; From patchwork Fri Apr 16 05:14:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12206707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A932EC433ED for ; Fri, 16 Apr 2021 05:16:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4B99C610FC for ; Fri, 16 Apr 2021 05:16:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4B99C610FC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DE0F56B007E; Fri, 16 Apr 2021 01:16:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DB8176B0080; Fri, 16 Apr 2021 01:16:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C593C6B0081; Fri, 16 Apr 2021 01:16:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0231.hostedemail.com [216.40.44.231]) by kanga.kvack.org (Postfix) with ESMTP id A1F906B007E for ; Fri, 16 Apr 2021 01:16:10 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 650E08248D52 for ; Fri, 16 Apr 2021 05:16:10 +0000 (UTC) X-FDA: 78037068900.17.036E5D9 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf17.hostedemail.com (Postfix) with ESMTP id AB1FD40002CF for ; Fri, 16 Apr 2021 05:16:07 +0000 (UTC) Received: by mail-pl1-f182.google.com with SMTP id p16so9387498plf.12 for ; Thu, 15 Apr 2021 22:16:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=j8wWJIpwyMTu3hxIlWXzmIud8VzdA+zaDtz12NM5n5U=; b=iHBp4oj9JUNupFLCeYguBqBHnt/v6XaFLarmLsBE+1tkt3jxMhzFrhBvE5l8AN997Q yoR3q08gqyt+Sx6AKbEcIzEhpyu0SaMeCtIyPXxDWEBmmc8d+jxn8sPUbnCHWffKbanT /c72tbHOahxBwRDDNhxZTRoZr/HSGYnPyiKa2252M9kQLGncdK4AWFI9Nb42osoA7MsV oawzC8OlTQdgpjx5CaRugEGbD6uaUGlx8fjNkGmJ3d4OSCeLlIlDlN5cLhA0Qh/tb5Ok AfnV+x3DYN9fEgAcJbV0sXzSIl8BcX4pCPs/6UlWSYZkFna0+lK1qCsK8xEfiDTxpAG3 TjLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=j8wWJIpwyMTu3hxIlWXzmIud8VzdA+zaDtz12NM5n5U=; b=jbiGjsJfswr3E74btBwyLx7miH1rc0nZf9YMuQZw4XtsWVNWJxScZzOSmfPI6c1lNM WBPqJM2cEntxFwTfagGgwknRHwNcB8TnRV5hjMq7N4NzcyDdECNwUL91wDOb1uAr6U1O AXZzzEXh9FnGYVp/WsnKDxGmoqapPyqDa/fMUSigOuHm9EkdUY3YUyxgw02A42eZHy0+ 0Y/qE+gSreqICoQJLlBHMEAB4G1Vu1TfUdreoV2KzkfvWv1Do5o/sRjiBsHFL+J9PdJq knDMwONxEVsG5+/fs3b6rrkcrc5QKhttQuIZJxXIeLSlj1E7bR0o5TXeSVN9DIebGDHB DTlA== X-Gm-Message-State: AOAM533wvxhs/6xHrlPpmQFs/oTWbRONlIwyZPOQ6ntBgT8Nk4MFp/Xo Np4TXplgCj6i00MS4Gh6PPh3tQ== X-Google-Smtp-Source: ABdhPJxiIy16z6+pQ2PwCIV5UB57x16+veoagt6ZKlXWDE19ptYCd7nfBf8ivXxvb72aa+VHvmL9bw== X-Received: by 2002:a17:90a:7893:: with SMTP id x19mr7908730pjk.3.1618550169181; Thu, 15 Apr 2021 22:16:09 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.235]) by smtp.gmail.com with ESMTPSA id j7sm3518540pfd.129.2021.04.15.22.16.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Apr 2021 22:16:08 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song Subject: [PATCH v2 6/8] mm: memcontrol: simplify the logic of objcg pinning memcg Date: Fri, 16 Apr 2021 13:14:05 +0800 Message-Id: <20210416051407.54878-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210416051407.54878-1-songmuchun@bytedance.com> References: <20210416051407.54878-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: AB1FD40002CF X-Stat-Signature: 118t4gu3kpr31t1zbtnzg6ecyfor8c9n Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf17; identity=mailfrom; envelope-from=""; helo=mail-pl1-f182.google.com; client-ip=209.85.214.182 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618550167-368575 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The obj_cgroup_release() and memcg_reparent_objcgs() are serialized by the css_set_lock. We do not need to care about objcg->memcg being released in the process of obj_cgroup_release(). So there is no need to pin memcg before releasing objcg. Remove those pinning logic to simplfy the code. There are only two places that modifies the objcg->memcg. One is the initialization to objcg->memcg in the memcg_online_kmem(), another is objcgs reparenting in the memcg_reparent_objcgs(). It is also impossible for the two to run in parallel. So xchg() is unnecessary and it is enough to use WRITE_ONCE(). Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin --- mm/memcontrol.c | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index caf193088beb..c4eebe2a2914 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -261,7 +261,6 @@ static void obj_cgroup_uncharge_pages(struct obj_cgroup *objcg, static void obj_cgroup_release(struct percpu_ref *ref) { struct obj_cgroup *objcg = container_of(ref, struct obj_cgroup, refcnt); - struct mem_cgroup *memcg; unsigned int nr_bytes; unsigned int nr_pages; unsigned long flags; @@ -291,11 +290,9 @@ static void obj_cgroup_release(struct percpu_ref *ref) nr_pages = nr_bytes >> PAGE_SHIFT; spin_lock_irqsave(&css_set_lock, flags); - memcg = obj_cgroup_memcg(objcg); if (nr_pages) obj_cgroup_uncharge_pages(objcg, nr_pages); list_del(&objcg->list); - mem_cgroup_put(memcg); spin_unlock_irqrestore(&css_set_lock, flags); percpu_ref_exit(ref); @@ -330,17 +327,12 @@ static void memcg_reparent_objcgs(struct mem_cgroup *memcg, spin_lock_irq(&css_set_lock); - /* Move active objcg to the parent's list */ - xchg(&objcg->memcg, parent); - css_get(&parent->css); - list_add(&objcg->list, &parent->objcg_list); - - /* Move already reparented objcgs to the parent's list */ - list_for_each_entry(iter, &memcg->objcg_list, list) { - css_get(&parent->css); - xchg(&iter->memcg, parent); - css_put(&memcg->css); - } + /* 1) Ready to reparent active objcg. */ + list_add(&objcg->list, &memcg->objcg_list); + /* 2) Reparent active objcg and already reparented objcgs to parent. */ + list_for_each_entry(iter, &memcg->objcg_list, list) + WRITE_ONCE(iter->memcg, parent); + /* 3) Move already reparented objcgs to the parent's list */ list_splice(&memcg->objcg_list, &parent->objcg_list); spin_unlock_irq(&css_set_lock); From patchwork Fri Apr 16 05:14:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12206709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E002BC43460 for ; Fri, 16 Apr 2021 05:16:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8AB776113D for ; Fri, 16 Apr 2021 05:16:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8AB776113D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2964B6B0081; Fri, 16 Apr 2021 01:16:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 26F7C6B0082; Fri, 16 Apr 2021 01:16:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E85C6B0083; Fri, 16 Apr 2021 01:16:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0225.hostedemail.com [216.40.44.225]) by kanga.kvack.org (Postfix) with ESMTP id DF1CC6B0081 for ; Fri, 16 Apr 2021 01:16:14 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id AA9AE182371AB for ; Fri, 16 Apr 2021 05:16:14 +0000 (UTC) X-FDA: 78037069068.25.EC90D7C Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf28.hostedemail.com (Postfix) with ESMTP id D6D952000261 for ; Fri, 16 Apr 2021 05:16:15 +0000 (UTC) Received: by mail-pl1-f169.google.com with SMTP id e2so9038576plh.8 for ; Thu, 15 Apr 2021 22:16:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y8TwextheIbDJq4JEPMsgtPW4imrEmOBZladzG6/EIY=; b=HoswKfBgJEUAeaMh1rPnmApf19VxOGH4LxCzAAWsLcCDsEJ8GosQcQPJblt+T7JZQN r71ipPvNggvX+8yVTjlCBkzMqiV9mO1+R0ztqecdTYYhVEYAdxJCRsulXBT29kTm5GMF yIIw/0cWyOm7IL4zWeV8RMSiZx4ntXvj6H4+sCRnDq7GheeYJUNEq2Fo+0HV63X4l9mk 9Bd+o5MXhoiGnagKy7YFgSOjH4P5wsMdUUQiXFiVo997n7F1nZvBadfLSDtkCr+1xJgs aQJ4kVAQAaUaXgzq/bo9yvy+PMbauZJycOroQXYvrMj8bT8zseAK4eESsLH38n9wC1o/ BXCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y8TwextheIbDJq4JEPMsgtPW4imrEmOBZladzG6/EIY=; b=EPHLkPbyNMnn8kYNEAJscbeO+VNsIX8NLskosMCLa3OGAwyrllSNTgiYi0AKcBHpOZ /hnHwrLkmuAMA0PC+Tktyuvp2Cc0E1wlDqcj7MnBJ7axIknrFcgrpweqEACqc2aFF7Lw VmixphdPw1BQl9znL/vbcTicaTU1mb5S6H082nuS7+kirRPmPNV6x8TZ2YO1bXjnyZet Di/MKuNO+9ZMSYKkHdRBDxNlde4rE2w7EqE1lrA6WCgCvAaWq2kstB/EGYbb84DKVbRQ tefmKlanzZTV5TQbUXuLA4Bfxy53XenV2p2JdsGkNGtcFhAjyJ5lQpW/tE5vyIAFNRxj /3GQ== X-Gm-Message-State: AOAM532fH6Mzc0F/5/HKgQ0Z6mUQcR8Yf6bpVtB8+GbSxHWTu7Wzrh/L Dr0FT1MAlEf0gHAoV4Mui/Afmg== X-Google-Smtp-Source: ABdhPJw9GKAMCuwJ22HjA/xotV/FWoBOFr8YI7Kfw+R1/pWOK3g+g9MUgjT0ClJCL20xK6itrBQIPA== X-Received: by 2002:a17:902:b602:b029:e6:cabb:10b9 with SMTP id b2-20020a170902b602b02900e6cabb10b9mr7627812pls.47.1618550173628; Thu, 15 Apr 2021 22:16:13 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.235]) by smtp.gmail.com with ESMTPSA id j7sm3518540pfd.129.2021.04.15.22.16.09 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Apr 2021 22:16:13 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song Subject: [PATCH v2 7/8] mm: memcontrol: move obj_cgroup_uncharge_pages() out of css_set_lock Date: Fri, 16 Apr 2021 13:14:06 +0800 Message-Id: <20210416051407.54878-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210416051407.54878-1-songmuchun@bytedance.com> References: <20210416051407.54878-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: D6D952000261 X-Stat-Signature: xa7kat7q841naoohdey97gn1t7asbaps X-Rspamd-Server: rspam02 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf28; identity=mailfrom; envelope-from=""; helo=mail-pl1-f169.google.com; client-ip=209.85.214.169 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618550175-238066 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The css_set_lock is used to guard the list of inherited objcgs. So there is no need to uncharge kernel memory under css_set_lock. Just move it out of the lock. Signed-off-by: Muchun Song Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Johannes Weiner --- mm/memcontrol.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c4eebe2a2914..e0c398fe7443 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -289,9 +289,10 @@ static void obj_cgroup_release(struct percpu_ref *ref) WARN_ON_ONCE(nr_bytes & (PAGE_SIZE - 1)); nr_pages = nr_bytes >> PAGE_SHIFT; - spin_lock_irqsave(&css_set_lock, flags); if (nr_pages) obj_cgroup_uncharge_pages(objcg, nr_pages); + + spin_lock_irqsave(&css_set_lock, flags); list_del(&objcg->list); spin_unlock_irqrestore(&css_set_lock, flags); From patchwork Fri Apr 16 05:14:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12206711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C099C433B4 for ; Fri, 16 Apr 2021 05:16:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EE0AE610FB for ; Fri, 16 Apr 2021 05:16:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE0AE610FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8DBF86B0083; Fri, 16 Apr 2021 01:16:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 887316B0085; Fri, 16 Apr 2021 01:16:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 729D56B0087; Fri, 16 Apr 2021 01:16:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id 557426B0083 for ; Fri, 16 Apr 2021 01:16:20 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1CE58182371CB for ; Fri, 16 Apr 2021 05:16:20 +0000 (UTC) X-FDA: 78037069320.28.67EC0AD Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf20.hostedemail.com (Postfix) with ESMTP id 4B05212E for ; Fri, 16 Apr 2021 05:16:14 +0000 (UTC) Received: by mail-pj1-f46.google.com with SMTP id q14-20020a17090a430eb02901503aaee02bso2898255pjg.3 for ; Thu, 15 Apr 2021 22:16:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uQtbD7P4vc1vn6sj+rqk5iopinYS+ejCLWKURjwvog0=; b=mRtbHxsAw7+QrX8ggNXOLMpdmdVUi7Qf1Lpa81SHEUBjmwEClOHj/sH4gvAYCq8zPZ xWibDLFCrAzBmZDv70d88yqFQse8c9LDa2GH++SJosfSXH710S1MN91bD9nh5dZhoJ+Z k0lLVEqXe3+hW7vEtcqBkSlhNOw9VTcVBej93UknAw8Nnf+qblXwim/FTZBj8ayS/sT2 9NOkyYaLvrpYcu5oKbIJ7zcZJgeAmkEZyf5w8lja3um9r8ULyUkEgRgQ0hnfg7qKs0Eq F5EShwqc6drtDgM19zTdg9A10CWl1boL9f0t+t3Bg3N9uDCy5PRtVitu16kREDAQyi/E XbHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uQtbD7P4vc1vn6sj+rqk5iopinYS+ejCLWKURjwvog0=; b=g8JestJM82slUovQOSURexwQiB3Gvd83s9Z7mV8vNaP4SS5F5E9Pws09ORGVzBpxyZ T7yPL7XxwIaU0wBPRSNG6qjuXeB+ugH4kxFq0TCdWGjnaFYZuaGOLaalICP3dLfhYT06 drXDrvomzUEDx4hZHp7d+AthBnpamYFjWf9YT6YhCLXJJTL2aqw2xbHvMyO0cOPkspiC g7qNCv6hZHmxgEgmQC03ofiVDUS4mJwISu9llKVp6Za3qpCm9zOg0iF9FwzCtPO4SfQS 31aUaZIJkfiZKuBFRaUawO+OGZr4KvLD4H2N507NIDd8sgWh+590Mase9AuJyFHe4Dpg +O2w== X-Gm-Message-State: AOAM532MMnpjcjoajk7AH9qLFbxooj2tWRrwF1fpCHdUqMktMMnH9sEa saOXaznI2PyCK9ig1jAa66nzLQ== X-Google-Smtp-Source: ABdhPJzsgKapGnuNFn0l7TP15/XU9x0NbvQ/D9jy8GoSu4oybHVT5scnv3+3n7GIFQav84toXIR3Nw== X-Received: by 2002:a17:902:c948:b029:e9:8f01:fa8e with SMTP id i8-20020a170902c948b02900e98f01fa8emr7765675pla.37.1618550179022; Thu, 15 Apr 2021 22:16:19 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.235]) by smtp.gmail.com with ESMTPSA id j7sm3518540pfd.129.2021.04.15.22.16.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Apr 2021 22:16:18 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v2 8/8] mm: vmscan: remove noinline_for_stack Date: Fri, 16 Apr 2021 13:14:07 +0800 Message-Id: <20210416051407.54878-9-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210416051407.54878-1-songmuchun@bytedance.com> References: <20210416051407.54878-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 4B05212E X-Stat-Signature: 7ggsj9b3kbxhczmohqk4ahy8i3jss49s Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail-pj1-f46.google.com; client-ip=209.85.216.46 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618550174-536778 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The noinline_for_stack is introduced by commit 666356297ec4 ("vmscan: set up pagevec as late as possible in shrink_inactive_list()"), its purpose is to delay the allocation of pagevec as late as possible to save stack memory. But the commit 2bcf88796381 ("mm: take pagevecs off reclaim stack") replace pagevecs by lists of pages_to_free. So we do not need noinline_for_stack, just remove it (let the compiler decide whether to inline). Signed-off-by: Muchun Song Acked-by: Johannes Weiner Acked-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko --- mm/vmscan.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 2bc5cf409958..2d2727b78df9 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2014,8 +2014,8 @@ static int too_many_isolated(struct pglist_data *pgdat, int file, * * Returns the number of pages moved to the given lruvec. */ -static unsigned noinline_for_stack move_pages_to_lru(struct lruvec *lruvec, - struct list_head *list) +static unsigned int move_pages_to_lru(struct lruvec *lruvec, + struct list_head *list) { int nr_pages, nr_moved = 0; LIST_HEAD(pages_to_free); @@ -2095,7 +2095,7 @@ static int current_may_throttle(void) * shrink_inactive_list() is a helper for shrink_node(). It returns the number * of reclaimed pages */ -static noinline_for_stack unsigned long +static unsigned long shrink_inactive_list(unsigned long nr_to_scan, struct lruvec *lruvec, struct scan_control *sc, enum lru_list lru) {