From patchwork Sat Apr 17 04:35:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12209523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9D37C433B4 for ; Sat, 17 Apr 2021 04:38:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6BCC0611AC for ; Sat, 17 Apr 2021 04:38:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6BCC0611AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0B9B96B006E; Sat, 17 Apr 2021 00:38:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0918E6B0070; Sat, 17 Apr 2021 00:38:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9A786B0071; Sat, 17 Apr 2021 00:38:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0072.hostedemail.com [216.40.44.72]) by kanga.kvack.org (Postfix) with ESMTP id D27E76B006E for ; Sat, 17 Apr 2021 00:38:03 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 8E24C8248047 for ; Sat, 17 Apr 2021 04:38:03 +0000 (UTC) X-FDA: 78040601646.27.510DD7F Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf07.hostedemail.com (Postfix) with ESMTP id 04345A0000FF for ; Sat, 17 Apr 2021 04:38:02 +0000 (UTC) Received: by mail-pl1-f176.google.com with SMTP id o16so1395167plg.5 for ; Fri, 16 Apr 2021 21:38:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8k6YXYK85mvc31ajI7k2coqgwaOfr7Ej3V6vhYCGVRI=; b=ps1+a0+z21qwgsRdYo2hIYK2avotOHkYx3BP4oOagRnFhl/blcLB1v1fiGfsYeWwjB r+HCFHxNBDPj/n5UF3SbNMRjYnh5N1VDtGH+FAZcThLFwwbBbCyEj9jUE0E1TwBndNWY iIdiA/YDNmAIyMyySRsHmmSWSAMHEIk5RmAZPMs1NC0Zd/UkbQAFbHUFs1bqFBmx3c4Z AnjWuaxinOTVY2me31a3RJ2F8XmoQh1VHiiXbLZoNBdUm39+DG/dTRxIKzhC3PqabIpV SvcijOy+eHbybeP/SLVydbxRl2iZhP2NuurrjBs9JenKG1CRMKQgqP4z+s89SCieg+DB zHSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8k6YXYK85mvc31ajI7k2coqgwaOfr7Ej3V6vhYCGVRI=; b=EPs3zUtIxd5y6dE9bZwUX77nvxIElkd5LFN3+tb56PjjRl4cbiDBkLfUeuu9QdJXn+ QoP21090B2VPVcLCzpkbm4+po4xlMKsnllKifrEYADWuHAABvKkFCEAgoJd0Rcw0mxjS gGKpedz46Oi729BY0bxoHOTKIhhv9QJxoKKtVt6noFCbaVEuuDnydAQ/d9thCvX+BAiT ZZ1klhVTG5SHwjc9QC9qDZVRFWKzWYhUqIXWXpPgyWlf8b4OGVCDsJ1W+/NVPA8yWXG+ WtrSw9TDEagSryNSy5WgEDF1JjUV07wMDydg7O4rPuZGjTyFWt98cAIq10aYoLYHqlS5 kC8Q== X-Gm-Message-State: AOAM530PN57lgnIMt/E8JvE5u2OsIdmxBc+tHZh4iwp1RhKVboYRRQDy s5WChUcI5toSmE6NDTXk2SJOeg== X-Google-Smtp-Source: ABdhPJxf0T3t8Wb69/tbGsZ2lpLCfYMfOgx16Q4hD5OSt32utfReq+PsYCafvJR3TPaUnVvUlqb2zw== X-Received: by 2002:a17:902:264:b029:eb:3d3a:a09c with SMTP id 91-20020a1709020264b02900eb3d3aa09cmr12795807plc.0.1618634282457; Fri, 16 Apr 2021 21:38:02 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id w21sm7064775pjy.21.2021.04.16.21.37.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Apr 2021 21:38:02 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v3 1/8] mm: memcontrol: fix page charging in page replacement Date: Sat, 17 Apr 2021 12:35:31 +0800 Message-Id: <20210417043538.9793-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210417043538.9793-1-songmuchun@bytedance.com> References: <20210417043538.9793-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 04345A0000FF X-Stat-Signature: krr88be7m6iakdszq7cqopae6s1wdxyt Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail-pl1-f176.google.com; client-ip=209.85.214.176 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618634282-446791 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The pages aren't accounted at the root level, so do not charge the page to the root memcg in page replacement. Although we do not display the value (mem_cgroup_usage) so there shouldn't be any actual problem, but there is a WARN_ON_ONCE in the page_counter_cancel(). Who knows if it will trigger? So it is better to fix it. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- mm/memcontrol.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 64ada9e650a5..f229de925aa5 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6806,9 +6806,11 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage) /* Force-charge the new page. The old one will be freed soon */ nr_pages = thp_nr_pages(newpage); - page_counter_charge(&memcg->memory, nr_pages); - if (do_memsw_account()) - page_counter_charge(&memcg->memsw, nr_pages); + if (!mem_cgroup_is_root(memcg)) { + page_counter_charge(&memcg->memory, nr_pages); + if (do_memsw_account()) + page_counter_charge(&memcg->memsw, nr_pages); + } css_get(&memcg->css); commit_charge(newpage, memcg); From patchwork Sat Apr 17 04:35:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12209525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71872C433B4 for ; Sat, 17 Apr 2021 04:38:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1B3E5611AB for ; Sat, 17 Apr 2021 04:38:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1B3E5611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B413D6B0070; Sat, 17 Apr 2021 00:38:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B17C56B0071; Sat, 17 Apr 2021 00:38:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B9B56B0072; Sat, 17 Apr 2021 00:38:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0246.hostedemail.com [216.40.44.246]) by kanga.kvack.org (Postfix) with ESMTP id 80BB76B0070 for ; Sat, 17 Apr 2021 00:38:09 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 38C5A45BB for ; Sat, 17 Apr 2021 04:38:09 +0000 (UTC) X-FDA: 78040601898.06.6F4C570 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf24.hostedemail.com (Postfix) with ESMTP id CE76AA0000FD for ; Sat, 17 Apr 2021 04:38:01 +0000 (UTC) Received: by mail-pl1-f169.google.com with SMTP id u7so13190786plr.6 for ; Fri, 16 Apr 2021 21:38:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NE/BcoGMuqPyqQGyjWQGMTxC1WpQbz3TqD3QbwUvSZY=; b=HCYBSuWL5lNSfyR1c5D+bsBKj9KEfSpvl6qC9ksYnjL148APwpW4TAUtqKdnJ7pC7A TcpivOKDUXNMLR3P+FWv0R6KBx0FXOHEtaFTOTJjSlGGMEKxleb0kqY6b+3DAzr/+qMO yUn6B0VTyigbyVwQw5NZ3llxK3JzpPsR4mNmCT8d1ttTFw24RIUEdIoPxa7Hr+KPWBxb P8YHj4MA8iPuorRIuQPnQgP4eCAw+CDbYwkVLDQxwWWIhoiTTe8bIpNqwVa1ED87nl6A fW/wJXy4jF0w1hZj+aZIv7N5SUCY68+4Hz3w1OYGvkhz307Khf9vrRxWNo24RfqfKG8A gaRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NE/BcoGMuqPyqQGyjWQGMTxC1WpQbz3TqD3QbwUvSZY=; b=XgzIKNnECTV5mTx25HuQwpf7VV894kCV5aMKdeeBjfsFkVrjOpeFO5HR/8jVxxe9Te 7WejnqAaxuf0R53aApMMCVMxvZEkBBSVWu9MK63z+Ond0fV0+eHIe7AGL8JvArCSpXbK 7DCYjKYGALUlhYu9w28WkPghk+RF4a7Gg7hV70/skICsvJ1ti6dSK8unaO6r+rdLNZ3o ewAUnHKkVqCyjFR9/wpNUKAqx6EOq9QrsuVp5qadNYbFf3X6kcAnv7MCeW8ZouJth80t LyQhwzVv0WLBrfclqDfbctWMOa8Kxlj3wzBHXjkGUe9mR4NN3/fiaCJes6wK9ln6GcR+ 4ddQ== X-Gm-Message-State: AOAM531GSudNkzl4arGGI/rq6YabRFoepfJyUP/5lE4Qm/Cpt0asDJAP m3elThh0YM6HY2oldoC3ksl70Q== X-Google-Smtp-Source: ABdhPJzWa9DWpbE9nY7cRes3aa7lKCkPT7KXRZi0kgxvIDJ4A4qJBPj3gK6804vLQ/irrw+KbLogpQ== X-Received: by 2002:a17:902:cec4:b029:e9:58eb:562b with SMTP id d4-20020a170902cec4b02900e958eb562bmr12659796plg.63.1618634288058; Fri, 16 Apr 2021 21:38:08 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id w21sm7064775pjy.21.2021.04.16.21.38.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Apr 2021 21:38:07 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v3 2/8] mm: memcontrol: bail out early when !mm in get_mem_cgroup_from_mm Date: Sat, 17 Apr 2021 12:35:32 +0800 Message-Id: <20210417043538.9793-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210417043538.9793-1-songmuchun@bytedance.com> References: <20210417043538.9793-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CE76AA0000FD X-Stat-Signature: jpesk6jcj8r8tsm6q781k91m3wig6x4p Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail-pl1-f169.google.com; client-ip=209.85.214.169 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618634281-103563 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When mm is NULL, we do not need to hold rcu lock and call css_tryget for the root memcg. And we also do not need to check !mm in every loop of while. So bail out early when !mm. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- mm/memcontrol.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f229de925aa5..50e3cf1e263e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -901,20 +901,23 @@ struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm) if (mem_cgroup_disabled()) return NULL; + /* + * Page cache insertions can happen without an + * actual mm context, e.g. during disk probing + * on boot, loopback IO, acct() writes etc. + * + * No need to css_get on root memcg as the reference + * counting is disabled on the root level in the + * cgroup core. See CSS_NO_REF. + */ + if (unlikely(!mm)) + return root_mem_cgroup; + rcu_read_lock(); do { - /* - * Page cache insertions can happen without an - * actual mm context, e.g. during disk probing - * on boot, loopback IO, acct() writes etc. - */ - if (unlikely(!mm)) + memcg = mem_cgroup_from_task(rcu_dereference(mm->owner)); + if (unlikely(!memcg)) memcg = root_mem_cgroup; - else { - memcg = mem_cgroup_from_task(rcu_dereference(mm->owner)); - if (unlikely(!memcg)) - memcg = root_mem_cgroup; - } } while (!css_tryget(&memcg->css)); rcu_read_unlock(); return memcg; From patchwork Sat Apr 17 04:35:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12209527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDC7AC433B4 for ; Sat, 17 Apr 2021 04:38:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 76822611AB for ; Sat, 17 Apr 2021 04:38:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76822611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 155FF6B0071; Sat, 17 Apr 2021 00:38:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 12D486B0072; Sat, 17 Apr 2021 00:38:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F10AB8D0002; Sat, 17 Apr 2021 00:38:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id D6D926B0071 for ; Sat, 17 Apr 2021 00:38:15 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 9485F18015CAC for ; Sat, 17 Apr 2021 04:38:15 +0000 (UTC) X-FDA: 78040602150.26.F217B0F Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf08.hostedemail.com (Postfix) with ESMTP id 7FC9880192C0 for ; Sat, 17 Apr 2021 04:37:59 +0000 (UTC) Received: by mail-pj1-f43.google.com with SMTP id e8-20020a17090a7288b029014e51f5a6baso10460264pjg.2 for ; Fri, 16 Apr 2021 21:38:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CUAxTMYGPKcXUKklEvyCvysOG+ZnpjCi0FGjCzotqSU=; b=rZV3Lx/DG+nk2yOJopVmLtWk63dUiCzC/kXZfxaop1AJ0FAZfmyf2w4SM/saLYzAK8 4Rj59/gtH6GGeuQl+ONJaMCilputqDh0R8QOjVdpKnHnxcsOJ7jS5D8uVqHnJfT2XbxK JqzKeKWfqOSMFt6ZQPnPw3RD+SQ4cYiF7thutI51Ck8Vso9AaDvxszCXA8GROWHHn92j IagTTI3d6dZsxbDqusXTDbBIOS5eCGGudc+aw1mS2SYKwLc/T3wp8Ntto3X/StjH6ZE6 JGw1vidb5iDt484PLTbmBeout5mXkw9GnEITsLtKRkltPYpq0MxEAK9U88jF/pwN7L2U a18w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CUAxTMYGPKcXUKklEvyCvysOG+ZnpjCi0FGjCzotqSU=; b=kG6M7evy77aBxYfEyGF3XlRJ+M23RSCQBuxNCW1ANaEUvThEmtkQDC1l2JGD6nHEb/ 43fYOVkCmaN6dOr23TzO4pqdmbwvH/UU6KxLOj0RXIeaFhuomrDBBFeSbDON/NPyb8yM 82iRrN/rDLK0Y6wLx+/J/k3XD6yK2Epxn+Uk4x8v1n8UV8WX3wKQ+0ZS6HgBk2KtB0yn oiFCBrGCMD0ASg1XxIicz0HLPiH2DGSsHOCN1BGoVRLEZiHEu46GGVzHMxFw7bcGbVFv vDlLv/Vk1Fpsv//koYyS03JPR0Li8QXQK2ykg9PAGG/hpKbkgcL0A/swaLD49CHiVHa6 iTKA== X-Gm-Message-State: AOAM530ucu0el+6J6HfiAfyP9zHPk4l3zYeiaA7TXUS9cojbBkk9RQZb G5ZGjxBEwyIzk6RnA+eRq5TceQ== X-Google-Smtp-Source: ABdhPJx1urRpgkyMn0GN9kMp05vLR8ipUAEHnpuOQwNghNNxGjpdwtQ1A8eZT+trOPcx+kGUbKD7Zg== X-Received: by 2002:a17:90a:6396:: with SMTP id f22mr13217394pjj.91.1618634294427; Fri, 16 Apr 2021 21:38:14 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id w21sm7064775pjy.21.2021.04.16.21.38.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Apr 2021 21:38:14 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v3 3/8] mm: memcontrol: remove the pgdata parameter of mem_cgroup_page_lruvec Date: Sat, 17 Apr 2021 12:35:33 +0800 Message-Id: <20210417043538.9793-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210417043538.9793-1-songmuchun@bytedance.com> References: <20210417043538.9793-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 7FC9880192C0 X-Stat-Signature: 8hxy8upyqm4cknopesb17wkjm43j3dc3 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf08; identity=mailfrom; envelope-from=""; helo=mail-pj1-f43.google.com; client-ip=209.85.216.43 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618634279-119373 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All the callers of mem_cgroup_page_lruvec() just pass page_pgdat(page) as the 2nd parameter to it (except isolate_migratepages_block()). But for isolate_migratepages_block(), the page_pgdat(page) is also equal to the local variable of @pgdat. So mem_cgroup_page_lruvec() do not need the pgdat parameter. Just remove it to simplify the code. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- include/linux/memcontrol.h | 10 +++++----- mm/compaction.c | 2 +- mm/memcontrol.c | 9 +++------ mm/swap.c | 2 +- mm/workingset.c | 2 +- 5 files changed, 11 insertions(+), 14 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index c193be760709..f2a5aaba3577 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -743,13 +743,12 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg, /** * mem_cgroup_page_lruvec - return lruvec for isolating/putting an LRU page * @page: the page - * @pgdat: pgdat of the page * * This function relies on page->mem_cgroup being stable. */ -static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page, - struct pglist_data *pgdat) +static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page) { + pg_data_t *pgdat = page_pgdat(page); struct mem_cgroup *memcg = page_memcg(page); VM_WARN_ON_ONCE_PAGE(!memcg && !mem_cgroup_disabled(), page); @@ -1221,9 +1220,10 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg, return &pgdat->__lruvec; } -static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page, - struct pglist_data *pgdat) +static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page) { + pg_data_t *pgdat = page_pgdat(page); + return &pgdat->__lruvec; } diff --git a/mm/compaction.c b/mm/compaction.c index 8c5028bfbd56..1c500e697c88 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -994,7 +994,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, if (!TestClearPageLRU(page)) goto isolate_fail_put; - lruvec = mem_cgroup_page_lruvec(page, pgdat); + lruvec = mem_cgroup_page_lruvec(page); /* If we already hold the lock, we can skip some rechecking */ if (lruvec != locked) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 50e3cf1e263e..caf193088beb 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1181,9 +1181,8 @@ void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page) struct lruvec *lock_page_lruvec(struct page *page) { struct lruvec *lruvec; - struct pglist_data *pgdat = page_pgdat(page); - lruvec = mem_cgroup_page_lruvec(page, pgdat); + lruvec = mem_cgroup_page_lruvec(page); spin_lock(&lruvec->lru_lock); lruvec_memcg_debug(lruvec, page); @@ -1194,9 +1193,8 @@ struct lruvec *lock_page_lruvec(struct page *page) struct lruvec *lock_page_lruvec_irq(struct page *page) { struct lruvec *lruvec; - struct pglist_data *pgdat = page_pgdat(page); - lruvec = mem_cgroup_page_lruvec(page, pgdat); + lruvec = mem_cgroup_page_lruvec(page); spin_lock_irq(&lruvec->lru_lock); lruvec_memcg_debug(lruvec, page); @@ -1207,9 +1205,8 @@ struct lruvec *lock_page_lruvec_irq(struct page *page) struct lruvec *lock_page_lruvec_irqsave(struct page *page, unsigned long *flags) { struct lruvec *lruvec; - struct pglist_data *pgdat = page_pgdat(page); - lruvec = mem_cgroup_page_lruvec(page, pgdat); + lruvec = mem_cgroup_page_lruvec(page); spin_lock_irqsave(&lruvec->lru_lock, *flags); lruvec_memcg_debug(lruvec, page); diff --git a/mm/swap.c b/mm/swap.c index a75a8265302b..e0d5699213cc 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -313,7 +313,7 @@ void lru_note_cost(struct lruvec *lruvec, bool file, unsigned int nr_pages) void lru_note_cost_page(struct page *page) { - lru_note_cost(mem_cgroup_page_lruvec(page, page_pgdat(page)), + lru_note_cost(mem_cgroup_page_lruvec(page), page_is_file_lru(page), thp_nr_pages(page)); } diff --git a/mm/workingset.c b/mm/workingset.c index b7cdeca5a76d..4f7a306ce75a 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -408,7 +408,7 @@ void workingset_activation(struct page *page) memcg = page_memcg_rcu(page); if (!mem_cgroup_disabled() && !memcg) goto out; - lruvec = mem_cgroup_page_lruvec(page, page_pgdat(page)); + lruvec = mem_cgroup_page_lruvec(page); workingset_age_nonresident(lruvec, thp_nr_pages(page)); out: rcu_read_unlock(); From patchwork Sat Apr 17 04:35:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12209529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 992BCC433ED for ; Sat, 17 Apr 2021 04:38:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3C170611CD for ; Sat, 17 Apr 2021 04:38:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3C170611CD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BB6626B0072; Sat, 17 Apr 2021 00:38:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B8D766B0073; Sat, 17 Apr 2021 00:38:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A07396B0074; Sat, 17 Apr 2021 00:38:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0226.hostedemail.com [216.40.44.226]) by kanga.kvack.org (Postfix) with ESMTP id 87B666B0072 for ; Sat, 17 Apr 2021 00:38:21 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4102C180AD838 for ; Sat, 17 Apr 2021 04:38:21 +0000 (UTC) X-FDA: 78040602402.33.6EF8482 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf27.hostedemail.com (Postfix) with ESMTP id B1E0E80192C0 for ; Sat, 17 Apr 2021 04:38:07 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id x21-20020a17090a5315b029012c4a622e4aso15654761pjh.2 for ; Fri, 16 Apr 2021 21:38:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EITONQXFUiMqxrQ5NUsYlzJaHXQ6TTbcz+uz38aCyj8=; b=whLRw2HCvXl3pHg6mNAV15FULhOHrpn62y8Z8WKJKFY20H6AULTb5aEwzcp8eDCYtw ov1aiOKypF9MkXKicm+fYwhabKgE5rafxXQlyV9ykdSYtSO9QncIl8RWuZvTwd+VEo5P 1e6TQQiIb5rY1dJ1PYZO+hG8aEMtvl/AB2yOQk1Cx6qUDh6IlxGWze4ZnTI8QgNhj4GR ciF8lmElZj/GpCJl/KHuLRQl0Q6MaQTcUU8CEq9pFgu95wxqjljkBG+PSSPR3L4nBW7R foBQUryfXNvNEwh5/yhgtbz2Vl+2kmuLu/9YjSzZ/YNRfoVDpTbWib7DnSFO5t/suN/m 65xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EITONQXFUiMqxrQ5NUsYlzJaHXQ6TTbcz+uz38aCyj8=; b=Y71S9wNGUJ5/ymzTysvSZL02gsyYywVwRov0bfjR9jLKiHd9zfhQS0CjiJajDXd2c5 OU1xFbUXFOOhwASsN6emsnLp+RjTDCUSRIg5YHJTCPCa+9kMpycbnoxF0v6zdh3NwOvy 6t/Thnlp7gTpItbmPu7CnjJlqMgZoe495uukPg5QnWzuWE74qQqLPQhptkPdr+wdmADu j19aFLAf8yNCBQ7uzzgvKGwL37bpP55fK9hyBGfko33WBh2m2UN8Ekz7sfmlW/F/sjki IzM6tcxN2Wzly+GhDxfMWWJDhuiRmpJKExJBiSxWiVGfU5QyTIx1HRfCUfXFC046XXe6 rTKA== X-Gm-Message-State: AOAM530gGecOdWnFz8Mkwq2zk14pMVGyki/c3b+wgGXdLjCsqw/1Qsoz jkMhl7rQV/utPctSzSQI//RdRQ== X-Google-Smtp-Source: ABdhPJx3/kW197VAi+ZCYN+jYpLWGU93A/ZF7SmxrsBMkISTuQDrOIqPceWxOybJqEEuIfDvhrq82A== X-Received: by 2002:a17:90a:854b:: with SMTP id a11mr13378213pjw.17.1618634300159; Fri, 16 Apr 2021 21:38:20 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id w21sm7064775pjy.21.2021.04.16.21.38.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Apr 2021 21:38:19 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v3 4/8] mm: memcontrol: simplify lruvec_holds_page_lru_lock Date: Sat, 17 Apr 2021 12:35:34 +0800 Message-Id: <20210417043538.9793-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210417043538.9793-1-songmuchun@bytedance.com> References: <20210417043538.9793-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: jjomhjj9t4bjjnazf8dtufasagcnb8fe X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B1E0E80192C0 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=mail-pj1-f53.google.com; client-ip=209.85.216.53 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618634287-96940 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We already have a helper lruvec_memcg() to get the memcg from lruvec, we do not need to do it ourselves in the lruvec_holds_page_lru_lock(). So use lruvec_memcg() instead. And if mem_cgroup_disabled() returns false, the page_memcg(page) (the LRU pages) cannot be NULL. So remove the odd logic of "memcg = page_memcg(page) ? : root_mem_cgroup". And use lruvec_pgdat to simplify the code. We can have a single definition for this function that works for !CONFIG_MEMCG, CONFIG_MEMCG + mem_cgroup_disabled() and CONFIG_MEMCG. Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko --- include/linux/memcontrol.h | 31 +++++++------------------------ 1 file changed, 7 insertions(+), 24 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index f2a5aaba3577..2fc728492c9b 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -755,22 +755,6 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page) return mem_cgroup_lruvec(memcg, pgdat); } -static inline bool lruvec_holds_page_lru_lock(struct page *page, - struct lruvec *lruvec) -{ - pg_data_t *pgdat = page_pgdat(page); - const struct mem_cgroup *memcg; - struct mem_cgroup_per_node *mz; - - if (mem_cgroup_disabled()) - return lruvec == &pgdat->__lruvec; - - mz = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - memcg = page_memcg(page) ? : root_mem_cgroup; - - return lruvec->pgdat == pgdat && mz->memcg == memcg; -} - struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p); struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm); @@ -1227,14 +1211,6 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page) return &pgdat->__lruvec; } -static inline bool lruvec_holds_page_lru_lock(struct page *page, - struct lruvec *lruvec) -{ - pg_data_t *pgdat = page_pgdat(page); - - return lruvec == &pgdat->__lruvec; -} - static inline void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page) { } @@ -1516,6 +1492,13 @@ static inline void unlock_page_lruvec_irqrestore(struct lruvec *lruvec, spin_unlock_irqrestore(&lruvec->lru_lock, flags); } +static inline bool lruvec_holds_page_lru_lock(struct page *page, + struct lruvec *lruvec) +{ + return lruvec_pgdat(lruvec) == page_pgdat(page) && + lruvec_memcg(lruvec) == page_memcg(page); +} + /* Don't lock again iff page's lruvec locked */ static inline struct lruvec *relock_page_lruvec_irq(struct page *page, struct lruvec *locked_lruvec) From patchwork Sat Apr 17 04:35:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12209531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49C21C433ED for ; Sat, 17 Apr 2021 04:38:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C6148611AB for ; Sat, 17 Apr 2021 04:38:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C6148611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 65F136B0073; Sat, 17 Apr 2021 00:38:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 634736B0074; Sat, 17 Apr 2021 00:38:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D6666B0075; Sat, 17 Apr 2021 00:38:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0055.hostedemail.com [216.40.44.55]) by kanga.kvack.org (Postfix) with ESMTP id 340176B0073 for ; Sat, 17 Apr 2021 00:38:27 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id EE5791801587F for ; Sat, 17 Apr 2021 04:38:26 +0000 (UTC) X-FDA: 78040602612.03.1CD83EB Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf07.hostedemail.com (Postfix) with ESMTP id 498BCA0000FD for ; Sat, 17 Apr 2021 04:38:26 +0000 (UTC) Received: by mail-pl1-f172.google.com with SMTP id t22so14678396ply.1 for ; Fri, 16 Apr 2021 21:38:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CzztEYThC68iA+/H3mXFzISa6K0UiM/T7EUy8N400kM=; b=ZwQpZdb4P0z3JJJBaiDRn34BsDhgV7N2sSEU+RfSVAHDy77bCYUa42yOlGOKs0A7TU Kj/xw6XNgYv+qS/dwBKrd0p60qYp8yqtTQL3ftFDEKcKFFzEeWz0rK/vStgWPGu4vVXi Ekr8fx3oRHT4pJI4I3clORCRCha9uLyLk1MfV1EJQuayfM3Yzqo3rDKwziEv+xt7lfBX 8/9AC9jvuQdU5UU3o0mHBbePSwRw8Gw3WvWgATwgkfdqt5vxl4KGi2Va3k23f+nUNrt3 7wbApWm3nKoM1btFRS9BmewuR1HjYvLBDNKauYYE1i9M8bWV4DB+JqUdbbxCFb6xTO0n OCgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CzztEYThC68iA+/H3mXFzISa6K0UiM/T7EUy8N400kM=; b=Hakm8N+eYVOJ4G7402ya6fRSaqLpLyl5wPR0Rv8HujbN4tXzLD+4p2JHIp+PQ8VOW0 zPxqfX5Je5HA6ITR8Wx9njvzSmSinatB9jl5dutHgSPwU6EoqPqU/aZxWAxtnRSFUKBK noaZZ+y0GCrf8/IQUWDJHF/PHhEbz2N5K8rEL2icev2Zc1+0WWMEaESMvi5Hw4zW7/El VDPWk7xeOY8skQ29lHYzI20gNVvUB/EPxI1DgU7ZeEE8z8j4ERdSCMuw4M4jV4AcQuRE mEDB+KkcBKjmHtdrySyiFF/PeDXnBFVXtA+Js81hzHzVFutYn2LO89SoFDheJlbl0vXI A7gA== X-Gm-Message-State: AOAM532XRHDgP6Y/URU4+CWlB+xTY0NbGp2szMpVgUfImwZaWX2u74hc tlSvTVYJyQp9lK1IXm0W+iAPIQ== X-Google-Smtp-Source: ABdhPJwHJOXNxCRxaiJNn5wSMEycirv4bY2+aWCTu05ryyJoj5mcg7Rh8aN85bWBwHXTFU/RExwEkQ== X-Received: by 2002:a17:90a:9312:: with SMTP id p18mr12986286pjo.171.1618634305772; Fri, 16 Apr 2021 21:38:25 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id w21sm7064775pjy.21.2021.04.16.21.38.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Apr 2021 21:38:25 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v3 5/8] mm: memcontrol: rename lruvec_holds_page_lru_lock to page_matches_lruvec Date: Sat, 17 Apr 2021 12:35:35 +0800 Message-Id: <20210417043538.9793-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210417043538.9793-1-songmuchun@bytedance.com> References: <20210417043538.9793-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 498BCA0000FD X-Stat-Signature: 9frg9ze5d4tnihpxbcqns86nmnejts58 X-Rspamd-Server: rspam02 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail-pl1-f172.google.com; client-ip=209.85.214.172 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618634306-732759 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: lruvec_holds_page_lru_lock() doesn't check anything about locking and is used to check whether the page belongs to the lruvec. So rename it to page_matches_lruvec(). Signed-off-by: Muchun Song Acked-by: Michal Hocko Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt --- include/linux/memcontrol.h | 8 ++++---- mm/vmscan.c | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 2fc728492c9b..0ce97eff79e2 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1492,8 +1492,8 @@ static inline void unlock_page_lruvec_irqrestore(struct lruvec *lruvec, spin_unlock_irqrestore(&lruvec->lru_lock, flags); } -static inline bool lruvec_holds_page_lru_lock(struct page *page, - struct lruvec *lruvec) +/* Test requires a stable page->memcg binding, see page_memcg() */ +static inline bool page_matches_lruvec(struct page *page, struct lruvec *lruvec) { return lruvec_pgdat(lruvec) == page_pgdat(page) && lruvec_memcg(lruvec) == page_memcg(page); @@ -1504,7 +1504,7 @@ static inline struct lruvec *relock_page_lruvec_irq(struct page *page, struct lruvec *locked_lruvec) { if (locked_lruvec) { - if (lruvec_holds_page_lru_lock(page, locked_lruvec)) + if (page_matches_lruvec(page, locked_lruvec)) return locked_lruvec; unlock_page_lruvec_irq(locked_lruvec); @@ -1518,7 +1518,7 @@ static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page, struct lruvec *locked_lruvec, unsigned long *flags) { if (locked_lruvec) { - if (lruvec_holds_page_lru_lock(page, locked_lruvec)) + if (page_matches_lruvec(page, locked_lruvec)) return locked_lruvec; unlock_page_lruvec_irqrestore(locked_lruvec, *flags); diff --git a/mm/vmscan.c b/mm/vmscan.c index bb8321026c0c..2bc5cf409958 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2062,7 +2062,7 @@ static unsigned noinline_for_stack move_pages_to_lru(struct lruvec *lruvec, * All pages were isolated from the same lruvec (and isolation * inhibits memcg migration). */ - VM_BUG_ON_PAGE(!lruvec_holds_page_lru_lock(page, lruvec), page); + VM_BUG_ON_PAGE(!page_matches_lruvec(page, lruvec), page); add_page_to_lru_list(page, lruvec); nr_pages = thp_nr_pages(page); nr_moved += nr_pages; From patchwork Sat Apr 17 04:35:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12209533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A62B1C433B4 for ; Sat, 17 Apr 2021 04:38:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 53F0B611AB for ; Sat, 17 Apr 2021 04:38:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 53F0B611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E755F6B0074; Sat, 17 Apr 2021 00:38:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E4BAC6B0075; Sat, 17 Apr 2021 00:38:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CECDD6B0078; Sat, 17 Apr 2021 00:38:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0228.hostedemail.com [216.40.44.228]) by kanga.kvack.org (Postfix) with ESMTP id B37B96B0074 for ; Sat, 17 Apr 2021 00:38:32 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6B50B180177F5 for ; Sat, 17 Apr 2021 04:38:32 +0000 (UTC) X-FDA: 78040602864.36.31FEF76 Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by imf26.hostedemail.com (Postfix) with ESMTP id 109D640002C2 for ; Sat, 17 Apr 2021 04:38:26 +0000 (UTC) Received: by mail-pj1-f48.google.com with SMTP id j21-20020a17090ae615b02901505b998b45so2157953pjy.0 for ; Fri, 16 Apr 2021 21:38:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=j8wWJIpwyMTu3hxIlWXzmIud8VzdA+zaDtz12NM5n5U=; b=gOlDpI9W4hNk8OJOLpqJsXe7e3d8XW3uIRecNncUmDast0FIsNwEoukqsd92tIuXYS 4S9+Hg1qJokG2rU5GBwTlujrn6cb7eUE/O5soU6MDUsB6ZnMlldD6aeX97Vf0sjZUH6J rOOyszSYismDKyNKucJHCd9d7lB3hROGCDbpF57945bR5Lh/+0wSFO9LnmJaZJKqIX4N oL/XuAorQSZlwsgaDUvjvU5w6A/dZh3lZuzTzXKZyR0Fj/+TGjI4tbqSUScR4IvQig6O Mfwui3WpKH1vmNkVObxYjB9OtobnsE1qegYgcKfLd2VCr0pxUCUoGNJCfvboZYFIVTQH V7tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=j8wWJIpwyMTu3hxIlWXzmIud8VzdA+zaDtz12NM5n5U=; b=AMRcA1zhNVZmyrgCRQJEa0b2OkjhiPi4uqeTeNSbzQ1vmEKegbOcuxLDTPZnj0DzIY aSzGeMmRnEdW+9MrYeQpXPUOZ+2aPo/2Qpc+q2u8+1i6/VSt9my/bCqQNkwk3mrrLi8p WZu1pp9GRcEAocSXJTtWUlm8Ljg7p92X8KDuG9JhpNaeMP0hbZOVxd2A1PA9D9tJ9rBo PaIV1EOZPkcqYNts6OJ/1Hj75BpjhoAGUB5a7mMIJDAI/Vbys98DI1N/en1aVCoKn4cK 42xqeqnB3D9uFvoyKYyaDFGnN34sb3nkwQ0PhaCVKkAKFmcrvHcDxuOl1a8oxMYDQ98b WLag== X-Gm-Message-State: AOAM531fdemUZeEzU68vM/3IaEfyJj2vMzulqHTqbfeWi+tYNgoXmC8/ GGuWBk3th5JfctHWFoEV/98QTQ== X-Google-Smtp-Source: ABdhPJyjWUSSWRy9zIYjqf2tg/9AOXpV4Sq7NMnXrLiBovzHs2ttyWQ1tvrW4y8CSIsagtmxFg5dTQ== X-Received: by 2002:a17:902:501:b029:eb:7a1b:c96c with SMTP id 1-20020a1709020501b02900eb7a1bc96cmr11483702plf.76.1618634311277; Fri, 16 Apr 2021 21:38:31 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id w21sm7064775pjy.21.2021.04.16.21.38.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Apr 2021 21:38:31 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song Subject: [PATCH v3 6/8] mm: memcontrol: simplify the logic of objcg pinning memcg Date: Sat, 17 Apr 2021 12:35:36 +0800 Message-Id: <20210417043538.9793-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210417043538.9793-1-songmuchun@bytedance.com> References: <20210417043538.9793-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 109D640002C2 X-Stat-Signature: yz4q56frcat4gq9or4dub5zs315bkit7 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf26; identity=mailfrom; envelope-from=""; helo=mail-pj1-f48.google.com; client-ip=209.85.216.48 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618634306-930859 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The obj_cgroup_release() and memcg_reparent_objcgs() are serialized by the css_set_lock. We do not need to care about objcg->memcg being released in the process of obj_cgroup_release(). So there is no need to pin memcg before releasing objcg. Remove those pinning logic to simplfy the code. There are only two places that modifies the objcg->memcg. One is the initialization to objcg->memcg in the memcg_online_kmem(), another is objcgs reparenting in the memcg_reparent_objcgs(). It is also impossible for the two to run in parallel. So xchg() is unnecessary and it is enough to use WRITE_ONCE(). Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin --- mm/memcontrol.c | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index caf193088beb..c4eebe2a2914 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -261,7 +261,6 @@ static void obj_cgroup_uncharge_pages(struct obj_cgroup *objcg, static void obj_cgroup_release(struct percpu_ref *ref) { struct obj_cgroup *objcg = container_of(ref, struct obj_cgroup, refcnt); - struct mem_cgroup *memcg; unsigned int nr_bytes; unsigned int nr_pages; unsigned long flags; @@ -291,11 +290,9 @@ static void obj_cgroup_release(struct percpu_ref *ref) nr_pages = nr_bytes >> PAGE_SHIFT; spin_lock_irqsave(&css_set_lock, flags); - memcg = obj_cgroup_memcg(objcg); if (nr_pages) obj_cgroup_uncharge_pages(objcg, nr_pages); list_del(&objcg->list); - mem_cgroup_put(memcg); spin_unlock_irqrestore(&css_set_lock, flags); percpu_ref_exit(ref); @@ -330,17 +327,12 @@ static void memcg_reparent_objcgs(struct mem_cgroup *memcg, spin_lock_irq(&css_set_lock); - /* Move active objcg to the parent's list */ - xchg(&objcg->memcg, parent); - css_get(&parent->css); - list_add(&objcg->list, &parent->objcg_list); - - /* Move already reparented objcgs to the parent's list */ - list_for_each_entry(iter, &memcg->objcg_list, list) { - css_get(&parent->css); - xchg(&iter->memcg, parent); - css_put(&memcg->css); - } + /* 1) Ready to reparent active objcg. */ + list_add(&objcg->list, &memcg->objcg_list); + /* 2) Reparent active objcg and already reparented objcgs to parent. */ + list_for_each_entry(iter, &memcg->objcg_list, list) + WRITE_ONCE(iter->memcg, parent); + /* 3) Move already reparented objcgs to the parent's list */ list_splice(&memcg->objcg_list, &parent->objcg_list); spin_unlock_irq(&css_set_lock); From patchwork Sat Apr 17 04:35:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12209535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1357C43460 for ; Sat, 17 Apr 2021 04:38:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 79EF4611AB for ; Sat, 17 Apr 2021 04:38:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 79EF4611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1A41D6B0075; Sat, 17 Apr 2021 00:38:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17AC36B0078; Sat, 17 Apr 2021 00:38:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 01B876B007B; Sat, 17 Apr 2021 00:38:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0172.hostedemail.com [216.40.44.172]) by kanga.kvack.org (Postfix) with ESMTP id DDE246B0075 for ; Sat, 17 Apr 2021 00:38:37 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A40E94428 for ; Sat, 17 Apr 2021 04:38:37 +0000 (UTC) X-FDA: 78040603074.26.0591140 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf19.hostedemail.com (Postfix) with ESMTP id E227490009EF for ; Sat, 17 Apr 2021 04:38:18 +0000 (UTC) Received: by mail-pj1-f52.google.com with SMTP id b8-20020a17090a5508b029014d0fbe9b64so17461939pji.5 for ; Fri, 16 Apr 2021 21:38:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y8TwextheIbDJq4JEPMsgtPW4imrEmOBZladzG6/EIY=; b=USYqdzUiBYMhAwyN2kR/fOuJc5bdKJjvIFBfGlRsQbiFuKbvl018KHNrEDB6ZR5hD3 g7Eb7MjfHwqdbzwjZzHqSgKn3iN0np5fP1fvqLDkt7kuvnVfjS6qEcxVT/HC1z5wCMcq UJSsW2z0vk7Tym2fshCANCoQ+ILRU0a3KCB6lPp0zjw7kGEhJYSd+EmJli0rrX4PU+fa ey3C9Zhh6qFULYaL8zYEVqNSOXyONHWxPDFrKjweK2ag8PUoyXsGL6thcX6S15QUgS7a EMoKzQxZakqREJcohPUtHCRxupw0a/ijaFh7rGwamu9yLBVnXWqZQRBwZQz+mH0vms9N pnmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y8TwextheIbDJq4JEPMsgtPW4imrEmOBZladzG6/EIY=; b=opryrOObmTZI2Aczqdl7ZcU7NUSEYQ9WmKOZYW2OQH6iVpnu2iMZ/7rpCgsGJcArTM aG2DTlO2eMjuBfHr2pbQ1n4USmnzeR8JBXBUpxOU54EjwP4y2kfmF8NnULOSuWWFbIHb tErDpIaQyyViCegjQaTfLvW9LHKh9+hJdqQQ0gB9IcQhrE9Iw9DcfDtWm6PF1B+T8SeO OiQUHLv83gTzbkucPr/gpXZPOhOQHpl6E1/6ZEVUI4+ippG8E9Qr12koryqySVRJ/F+y CtWOfodHXslm3RhmzDCMPq4u3ETLsMVOu53OBFcN6rlwmILpMIccO2Ixzb9pPDBpOm1h xXsw== X-Gm-Message-State: AOAM531ZA8RE1YGBAr00wrv/o3aBxb4FD0yg3DEMfrWA86WHtsdRsQgC 1GZyUD0zK4pa4vxWNxYDbguHWw== X-Google-Smtp-Source: ABdhPJzQts5UlUhjYfu2k84c2/GRxhOv1r45Guo8b4WWs2RaMDXIQiw0p76Vd5VjErfo2giWbBJ1YQ== X-Received: by 2002:a17:902:b601:b029:eb:1207:86ec with SMTP id b1-20020a170902b601b02900eb120786ecmr12472948pls.3.1618634316645; Fri, 16 Apr 2021 21:38:36 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id w21sm7064775pjy.21.2021.04.16.21.38.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Apr 2021 21:38:36 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song Subject: [PATCH v3 7/8] mm: memcontrol: move obj_cgroup_uncharge_pages() out of css_set_lock Date: Sat, 17 Apr 2021 12:35:37 +0800 Message-Id: <20210417043538.9793-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210417043538.9793-1-songmuchun@bytedance.com> References: <20210417043538.9793-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: E227490009EF X-Stat-Signature: gew8km7pd5jh3cqapweiw8hti67ezfqb Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf19; identity=mailfrom; envelope-from=""; helo=mail-pj1-f52.google.com; client-ip=209.85.216.52 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618634298-67381 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The css_set_lock is used to guard the list of inherited objcgs. So there is no need to uncharge kernel memory under css_set_lock. Just move it out of the lock. Signed-off-by: Muchun Song Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Johannes Weiner --- mm/memcontrol.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c4eebe2a2914..e0c398fe7443 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -289,9 +289,10 @@ static void obj_cgroup_release(struct percpu_ref *ref) WARN_ON_ONCE(nr_bytes & (PAGE_SIZE - 1)); nr_pages = nr_bytes >> PAGE_SHIFT; - spin_lock_irqsave(&css_set_lock, flags); if (nr_pages) obj_cgroup_uncharge_pages(objcg, nr_pages); + + spin_lock_irqsave(&css_set_lock, flags); list_del(&objcg->list); spin_unlock_irqrestore(&css_set_lock, flags); From patchwork Sat Apr 17 04:35:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12209537 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA3D4C433B4 for ; Sat, 17 Apr 2021 04:38:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 76CFC611AC for ; Sat, 17 Apr 2021 04:38:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76CFC611AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1A33F6B0078; Sat, 17 Apr 2021 00:38:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17A726B007B; Sat, 17 Apr 2021 00:38:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0423D6B007D; Sat, 17 Apr 2021 00:38:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0178.hostedemail.com [216.40.44.178]) by kanga.kvack.org (Postfix) with ESMTP id DF5236B0078 for ; Sat, 17 Apr 2021 00:38:43 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A19878248047 for ; Sat, 17 Apr 2021 04:38:43 +0000 (UTC) X-FDA: 78040603326.01.1C6A49E Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by imf26.hostedemail.com (Postfix) with ESMTP id 3E77040002C6 for ; Sat, 17 Apr 2021 04:38:38 +0000 (UTC) Received: by mail-pj1-f41.google.com with SMTP id u14-20020a17090a1f0eb029014e38011b09so10826403pja.5 for ; Fri, 16 Apr 2021 21:38:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uQtbD7P4vc1vn6sj+rqk5iopinYS+ejCLWKURjwvog0=; b=b5xz3eTkB17OddKFPj9hsXB5jtbQcup9TatoeWEFMiLXgVOzdU/wufc+cbr67DMZbj pcL65OQ+PfpKe9nyEAdQ7pJwBauH6prg0T52NdN7LIVPLXlgnhRwD9J7C64wFliCY02C vOpWyJq/xqMrXg+v7VrKmwAysRwv1yXube8PuOhJKdy3v6ZRt1VK9Q6oNo5WjwGkznCw 1MCvHnf18yaeW9ZHhowTtnu/aT0mYYIWc1UpD54stMe+sYVN+mkwS+Zb81hLvNz+cXD9 6iImyuGI6UO+plKqcv/GGIed7d4nrLA/Z5ytpzOJZPncUXhLFopRtKdXO15m6t28SelW u3qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uQtbD7P4vc1vn6sj+rqk5iopinYS+ejCLWKURjwvog0=; b=KfrxNev+T0B9nqnR5prii/YXcKMk6plW0GUveJmsXdsaqIcMs6+h84t23x7ryh+cpt /KfFYkZRpgB9UQvkfz+Akgjanb7I/ey+KYeawkPL3U1shSfail4nqt/mh56ABaATxGUi mI4FuqxCmG3FuBwamUjPWPFI0EqF3uRqL++NzxPRRCgHxiczGkP3VOGstArs+Ky/WwVd YEBcs83jO/krhng+dQ23qaxtxlSyd79MTTQK68qvkJlVgswKfi+tj2t4g4v5AExGHBUF zQONCdxuimC9Y18bJis0bZ4VNhprxX5jIb6jPKu6mpSCf9R1ki9zUVAbo15wtC9Uon26 Ollg== X-Gm-Message-State: AOAM531+Xp4pj5GfL1TizuLUV4lrMTXFP341sc443aHYxVIYHiKKLUxJ biuMFSnlSDW0EQk8dvC/4gC+Iw== X-Google-Smtp-Source: ABdhPJy8VZmbHWqkDJIrzx05dg1KyMDeoaA6atPoRgpSAyr6k02STX0M9UPOZ/Gg1bLONaZEENhGhg== X-Received: by 2002:a17:90a:c81:: with SMTP id v1mr12749439pja.23.1618634322480; Fri, 16 Apr 2021 21:38:42 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id w21sm7064775pjy.21.2021.04.16.21.38.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Apr 2021 21:38:42 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song , Michal Hocko Subject: [PATCH v3 8/8] mm: vmscan: remove noinline_for_stack Date: Sat, 17 Apr 2021 12:35:38 +0800 Message-Id: <20210417043538.9793-9-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210417043538.9793-1-songmuchun@bytedance.com> References: <20210417043538.9793-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3E77040002C6 X-Stat-Signature: g4174d94ifattx8npk5rqgxyuubpihk9 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf26; identity=mailfrom; envelope-from=""; helo=mail-pj1-f41.google.com; client-ip=209.85.216.41 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618634318-344412 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The noinline_for_stack is introduced by commit 666356297ec4 ("vmscan: set up pagevec as late as possible in shrink_inactive_list()"), its purpose is to delay the allocation of pagevec as late as possible to save stack memory. But the commit 2bcf88796381 ("mm: take pagevecs off reclaim stack") replace pagevecs by lists of pages_to_free. So we do not need noinline_for_stack, just remove it (let the compiler decide whether to inline). Signed-off-by: Muchun Song Acked-by: Johannes Weiner Acked-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko --- mm/vmscan.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 2bc5cf409958..2d2727b78df9 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2014,8 +2014,8 @@ static int too_many_isolated(struct pglist_data *pgdat, int file, * * Returns the number of pages moved to the given lruvec. */ -static unsigned noinline_for_stack move_pages_to_lru(struct lruvec *lruvec, - struct list_head *list) +static unsigned int move_pages_to_lru(struct lruvec *lruvec, + struct list_head *list) { int nr_pages, nr_moved = 0; LIST_HEAD(pages_to_free); @@ -2095,7 +2095,7 @@ static int current_may_throttle(void) * shrink_inactive_list() is a helper for shrink_node(). It returns the number * of reclaimed pages */ -static noinline_for_stack unsigned long +static unsigned long shrink_inactive_list(unsigned long nr_to_scan, struct lruvec *lruvec, struct scan_control *sc, enum lru_list lru) {