From patchwork Sun Apr 25 07:54:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12223051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DFA6C433ED for ; Sun, 25 Apr 2021 07:57:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9B1E061354 for ; Sun, 25 Apr 2021 07:57:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9B1E061354 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C85566B0036; Sun, 25 Apr 2021 03:57:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C0E836B006C; Sun, 25 Apr 2021 03:57:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A60C66B006E; Sun, 25 Apr 2021 03:57:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0119.hostedemail.com [216.40.44.119]) by kanga.kvack.org (Postfix) with ESMTP id 8740C6B0036 for ; Sun, 25 Apr 2021 03:57:02 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 380358248047 for ; Sun, 25 Apr 2021 07:57:02 +0000 (UTC) X-FDA: 78070133484.10.51FAB04 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf15.hostedemail.com (Postfix) with ESMTP id D3FC0A00038D for ; Sun, 25 Apr 2021 07:56:57 +0000 (UTC) Received: by mail-pj1-f52.google.com with SMTP id j6-20020a17090adc86b02900cbfe6f2c96so3538625pjv.1 for ; Sun, 25 Apr 2021 00:57:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=DTSkEph7qL8ctEwVYodw+espySzWTEr2PJqwjnU2IwA=; b=YArwpms4v/ebs6QBCf1CGl2uuaUyu+UF1v0h7p108viBd2U9qf+F2CEuWeL1v1SQYA hkn/IuRVgXsDg2MsWYBr6dQJIYosab3LRXZODBHGsGEgSnMM1qO0sM5T/O1BKhmU0xgd 2JqiNp7PSBPQQIVFxkmXIsqbmCaVOkGaQe59xXb+pRACMxHXSFq5aD/jE+hsrd0vIW2d Ji0ga2wU2ahPoF7clxKb5BmD/Bx5jJ5XM76+S46Q+AL4UhDbi14ihQXoL5H7XFVC8Bxn BvSmdixlYxfyPMH5aZL0I+Jq742Z0nZcAj9LA4z7WSFGNgzmrdEFi+RQmAlB4oksXgF3 GOZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=DTSkEph7qL8ctEwVYodw+espySzWTEr2PJqwjnU2IwA=; b=HoDVcojo6pccqOsLfCfYIoWUGbI2HWFNJ5AX/VduKIkmpl6qIoBh4KKYJ0+HaKcERg gnSku84eVP2wpOe4a12ELMcKif06KAgfFLA16tfmPHM6MQNDfdnzbQVB4ZlYLnGlWZSA RvG2ANph+MdyNvTrbUPlJVB8GsCt7OYd5VPappWrZey/FKR8ByJsGK8Jg3ZIvZrvy/pZ pORO7jh0CUTueGGKzmznFg4Tri5ZwPJvSFb2UbBzcHuoPv9P4uYbQkqpT2zGIDgFddH7 aLIOTkHKhTE/Vyn6+P2WTTpcueX9RgvZXp+DC/SnIMAx9akfLfR/CCaTjzEKRnFwwi0m //iQ== X-Gm-Message-State: AOAM531CL/iky2tQgkPCoEISulmqfucu4ZLiBJGEs/xz9UeN3eYdAqa6 b/2g5W4yvDJpvPgxvfayMfPixQ== X-Google-Smtp-Source: ABdhPJxt94EaODQ6xUr6u0EpOWiZ6SVb3NyMWhIkriwxWC5qFE6d12Ze1J4FmoZNRU9GljOR+A9WCA== X-Received: by 2002:a17:902:6ac3:b029:e6:c6a3:a697 with SMTP id i3-20020a1709026ac3b02900e6c6a3a697mr12560454plt.2.1619337419848; Sun, 25 Apr 2021 00:56:59 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id c193sm8689444pfc.11.2021.04.25.00.56.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 25 Apr 2021 00:56:59 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, zhengqi.arch@bytedance.com, Muchun Song Subject: [PATCH v2] mm: memcontrol: fix root_mem_cgroup charging Date: Sun, 25 Apr 2021 15:54:10 +0800 Message-Id: <20210425075410.19255-1-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) MIME-Version: 1.0 X-Rspamd-Queue-Id: D3FC0A00038D X-Stat-Signature: d3ewx8mmdtjtjbm6ukm4b87p1mw5xecn X-Rspamd-Server: rspam02 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from=""; helo=mail-pj1-f52.google.com; client-ip=209.85.216.52 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619337417-407694 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The below scenario can cause the page counters of the root_mem_cgroup to be out of balance. CPU0: CPU1: objcg = get_obj_cgroup_from_current() obj_cgroup_charge_pages(objcg) memcg_reparent_objcgs() // reparent to root_mem_cgroup WRITE_ONCE(iter->memcg, parent) // memcg == root_mem_cgroup memcg = get_mem_cgroup_from_objcg(objcg) // do not charge to the root_mem_cgroup try_charge(memcg) obj_cgroup_uncharge_pages(objcg) memcg = get_mem_cgroup_from_objcg(objcg) // uncharge from the root_mem_cgroup refill_stock(memcg) drain_stock(memcg) page_counter_uncharge(&memcg->memory) get_obj_cgroup_from_current() never returns a root_mem_cgroup's objcg, so we never explicitly charge the root_mem_cgroup. And it's not going to change. It's all about a race when we got an obj_cgroup pointing at some non-root memcg, but before we were able to charge it, the cgroup was gone, objcg was reparented to the root and so we're skipping the charging. Then we store the objcg pointer and later use to uncharge the root_mem_cgroup. This can cause the page counter to be less than the actual value. Although we do not display the value (mem_cgroup_usage) so there shouldn't be any actual problem, but there is a WARN_ON_ONCE in the page_counter_cancel(). Who knows if it will trigger? So it is better to fix it. Signed-off-by: Muchun Song Acked-by: Roman Gushchin Reviewed-by: Shakeel Butt --- Changeslog in v2: - Update commit log. - Rename __try_charge to try_charge_memcg. mm/memcontrol.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 64ada9e650a5..42dee9798ab8 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2504,8 +2504,8 @@ void mem_cgroup_handle_over_high(void) css_put(&memcg->css); } -static int try_charge(struct mem_cgroup *memcg, gfp_t gfp_mask, - unsigned int nr_pages) +static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, + unsigned int nr_pages) { unsigned int batch = max(MEMCG_CHARGE_BATCH, nr_pages); int nr_retries = MAX_RECLAIM_RETRIES; @@ -2517,8 +2517,6 @@ static int try_charge(struct mem_cgroup *memcg, gfp_t gfp_mask, bool drained = false; unsigned long pflags; - if (mem_cgroup_is_root(memcg)) - return 0; retry: if (consume_stock(memcg, nr_pages)) return 0; @@ -2698,6 +2696,15 @@ static int try_charge(struct mem_cgroup *memcg, gfp_t gfp_mask, return 0; } +static inline int try_charge(struct mem_cgroup *memcg, gfp_t gfp_mask, + unsigned int nr_pages) +{ + if (mem_cgroup_is_root(memcg)) + return 0; + + return try_charge_memcg(memcg, gfp_mask, nr_pages); +} + #if defined(CONFIG_MEMCG_KMEM) || defined(CONFIG_MMU) static void cancel_charge(struct mem_cgroup *memcg, unsigned int nr_pages) { @@ -2925,7 +2932,7 @@ static int obj_cgroup_charge_pages(struct obj_cgroup *objcg, gfp_t gfp, memcg = get_mem_cgroup_from_objcg(objcg); - ret = try_charge(memcg, gfp, nr_pages); + ret = try_charge_memcg(memcg, gfp, nr_pages); if (ret) goto out;