From patchwork Fri Feb 12 17:01:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12085689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54C20C433E0 for ; Fri, 12 Feb 2021 17:02:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C13DE64E05 for ; Fri, 12 Feb 2021 17:02:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C13DE64E05 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4CBFA8D006E; Fri, 12 Feb 2021 12:02:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 47D6C8D0060; Fri, 12 Feb 2021 12:02:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36CC28D006E; Fri, 12 Feb 2021 12:02:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0090.hostedemail.com [216.40.44.90]) by kanga.kvack.org (Postfix) with ESMTP id 20B518D0060 for ; Fri, 12 Feb 2021 12:02:18 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D8F008248047 for ; Fri, 12 Feb 2021 17:02:17 +0000 (UTC) X-FDA: 77810233914.01.drop55_140c02a27622 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id B9D4310057A32 for ; Fri, 12 Feb 2021 17:02:17 +0000 (UTC) X-HE-Tag: drop55_140c02a27622 X-Filterd-Recvd-Size: 4173 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Fri, 12 Feb 2021 17:02:17 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id z9so758596pjl.5 for ; Fri, 12 Feb 2021 09:02:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=hdVPqrFgHTgC5mTllAq9fBNSua/ngygoblWbV1oigUI=; b=Wqq++mXdhn6HE2fF23Wh8h9AFqcynW3GPMQs21nMApsmcn9L4h6O8KtYPRjAUeNCDn iNMu2YDjYyY8JDBGuTJAEVoQcXdkJcwdAMiDMk/43qpb0yi2y/gjcH86jMmW4oRF16x3 L8UNDDz8q5XJPbV/D8jN4450QYrAUnCaJ5cUuSHsVoO/e/3r98A0TR3WAK0qaFp0ufcz 5FcU7+Yz6okzH9k+R28b7nmm6Z9f63wUb2h6VTNDmLX3s/VAfDqzX8JRwrHt08eJ5okY cIbjgV0hM5kJGL5B1Ttc4NuCgqtZIs5c37uIrW6rDWoTarmMsvNlMxDDAHHWfGQAiIkl 0GaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=hdVPqrFgHTgC5mTllAq9fBNSua/ngygoblWbV1oigUI=; b=eLhCGFCOa3yZWuKQ5DMKqmmIekYNy5FvVrLUroCD8f6HBbpayk67dTmcWsWDTdz491 DLd7HbwORMD62x41qw+gLOd9KYLF92eA3psoMR3FFdemz+UIJzd1fbcyqiTp2VmTOKvu aELt9j0nBIUZnEML4z99omsISgqahBt/WfUPjXU1OUOSO5qq96q5BFHPHtoHCC5L84ht Eip2kMa/7AVPw8jS1Krv9qp16rDyzPPH1v2GKQTTloeWI8PNWBRzSTS1ZFcCYVmFD/mx TtcFCnba95mapE89SsLjNEYaomYAts6VKgvhAtFhMLUgvkehvwfuhZYSCiOv6cRSP5Ul duGw== X-Gm-Message-State: AOAM530qk3d9M1z5YWVJErXe18qaVdqlRk/51p+edkBcx4GYit7HlREG C7tzMQb7kf3zDwAQ/ClqNN+2jg== X-Google-Smtp-Source: ABdhPJybZwdjimV2P5ym8dkQiRVRyC5DByFMgygAkBaUP5rkFW8O6XL/Sn6SVVsiqOEugp4e7rkVow== X-Received: by 2002:a17:90b:2312:: with SMTP id mt18mr3636553pjb.81.1613149336083; Fri, 12 Feb 2021 09:02:16 -0800 (PST) Received: from localhost.bytedance.net ([139.177.225.254]) by smtp.gmail.com with ESMTPSA id e21sm9317815pgv.74.2021.02.12.09.02.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Feb 2021 09:02:15 -0800 (PST) From: Muchun Song To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org Cc: cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song Subject: [PATCH 1/4] mm: memcontrol: remove memcg check from memcg_oom_recover Date: Sat, 13 Feb 2021 01:01:56 +0800 Message-Id: <20210212170159.32153-1-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The memcg_oom_recover() almost never do anything but the test (because oom_disabled is a rarely used) is just waste of cycles in some hot paths (e.g. kmem uncharge). And it is very small, so it is better to make it inline. Also, the parameter of memcg cannot be NULL, so removing the check can reduce useless check. Signed-off-by: Muchun Song --- mm/memcontrol.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 8c035846c7a4..7afca9677693 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1925,7 +1925,7 @@ static int memcg_oom_wake_function(wait_queue_entry_t *wait, return autoremove_wake_function(wait, mode, sync, arg); } -static void memcg_oom_recover(struct mem_cgroup *memcg) +static inline void memcg_oom_recover(struct mem_cgroup *memcg) { /* * For the following lockless ->under_oom test, the only required @@ -1935,7 +1935,7 @@ static void memcg_oom_recover(struct mem_cgroup *memcg) * achieved by invoking mem_cgroup_mark_under_oom() before * triggering notification. */ - if (memcg && memcg->under_oom) + if (memcg->under_oom) __wake_up(&memcg_oom_waitq, TASK_NORMAL, 0, memcg); } From patchwork Fri Feb 12 17:01:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12085691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AD9AC433DB for ; Fri, 12 Feb 2021 17:02:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 375D764E42 for ; Fri, 12 Feb 2021 17:02:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 375D764E42 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D57298D006F; Fri, 12 Feb 2021 12:02:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D08148D0060; Fri, 12 Feb 2021 12:02:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA8CC8D006F; Fri, 12 Feb 2021 12:02:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id A39408D0060 for ; Fri, 12 Feb 2021 12:02:25 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 6183C8248047 for ; Fri, 12 Feb 2021 17:02:25 +0000 (UTC) X-FDA: 77810234250.17.EF41B5A Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf11.hostedemail.com (Postfix) with ESMTP id 6B68320053DA for ; Fri, 12 Feb 2021 17:02:19 +0000 (UTC) Received: by mail-pl1-f176.google.com with SMTP id a16so163873plh.8 for ; Fri, 12 Feb 2021 09:02:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bcYt0NSihifXZf17H55Cj5Hl2GUjQyX5Z/whSKEEhxI=; b=QXdiNtz4icRE9wYOkzwNj/MEYjknOPfl+3qraA/tVpTL1yVUQ1uW/Egvq3zevZw11m 20DnFlwvQipf18yyHzSbhvGpWQ214vwvIYstobFglcVWyBKXAYFVQKqdebGoExkRRSK5 eiQft2Jyww09TKCwpm9fRe6tkSks1kczSXEdcv/laOSq5Cy74ohKIYxLVjPANARtpaLa liZ+acD0FILIz+lrXrA0/L9tZrDVhVAsNAGcQONgLpjgPGdqpzEP9NdCgVeVK6fjDxQo CXm7xNS6alE/sydQVkPcpDYcXQZrJZE64mOIICrRUaLwpDpPS5NX11mhQxdktolhNWPw vO4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bcYt0NSihifXZf17H55Cj5Hl2GUjQyX5Z/whSKEEhxI=; b=Cw4pF5XsWLGRf0axhiztjImZx6wDE7fKinu9Db/DLpdK0y+JvT+/pp5BtDcIw8blsJ bx9gMAhPJGdEPjZrDPkiiFwIPWLn4EFvete2KNch/43Uyvq0w9kWna9tvZjlTkJBH95L +mDDXKP8Y6CGm9uIGygpZCuoqgR9MS9DDQ0qbdZOYxS/kBV4Ha/yM64B2jCKCD2436Ar wtSXnhwvhRqVhG1aGrzXwKCsYoSSuugzXc70gK4k+WO7xwrJ/yyNfcKh4UXzILeW3isK IJwuyadELH79Zn3TWTtzwckDCm59LdxZRawuwl6MJwuEGxDk9tqFXKbDDVSc/eWxl2V6 DYPA== X-Gm-Message-State: AOAM5304zOcVDSrqRk2yOPjF4mRiD7LuBMjslPrN9fnHrmlC0bNl9w9w 9gJfeFYDijSdsk1/PIEyXQn9iQ== X-Google-Smtp-Source: ABdhPJy3fzcN97TILH5Pd2p9CjYCM+6s/EotNHOwcpsaLTt2cNdlvuafFJYbAs//bwycN8+3HlIfWA== X-Received: by 2002:a17:903:31c4:b029:e1:8840:8ab9 with SMTP id v4-20020a17090331c4b02900e188408ab9mr3441164ple.70.1613149340708; Fri, 12 Feb 2021 09:02:20 -0800 (PST) Received: from localhost.bytedance.net ([139.177.225.254]) by smtp.gmail.com with ESMTPSA id e21sm9317815pgv.74.2021.02.12.09.02.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Feb 2021 09:02:19 -0800 (PST) From: Muchun Song To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org Cc: cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song Subject: [PATCH 2/4] mm: memcontrol: add missing memcg_oom_recover() when uncharge slab page Date: Sat, 13 Feb 2021 01:01:57 +0800 Message-Id: <20210212170159.32153-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210212170159.32153-1-songmuchun@bytedance.com> References: <20210212170159.32153-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: d4mtcuz5o1f77pr7azbgytt5rjii49m5 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 6B68320053DA Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail-pl1-f176.google.com; client-ip=209.85.214.176 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1613149339-98059 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When we uncharge a page, we wake up oom victims when the memcg oom handling is outsourced to the userspace. The uncharge_batch do that for normal and kmem pages but not slab pages. It is likely an omission. So add the missing memcg_oom_recover() to __memcg_kmem_uncharge(). And the function of memory.oom_control is only suitable for cgroup v1. So guard this test (memcg->under_oom) by the cgroup_subsys_on_dfl(memory_cgrp_subsys). Signed-off-by: Muchun Song --- mm/memcontrol.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7afca9677693..a3f26522765a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3126,8 +3126,10 @@ static int __memcg_kmem_charge(struct mem_cgroup *memcg, gfp_t gfp, */ static void __memcg_kmem_uncharge(struct mem_cgroup *memcg, unsigned int nr_pages) { - if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) { page_counter_uncharge(&memcg->kmem, nr_pages); + memcg_oom_recover(memcg); + } refill_stock(memcg, nr_pages); } @@ -6806,11 +6808,15 @@ static void uncharge_batch(const struct uncharge_gather *ug) if (!mem_cgroup_is_root(ug->memcg)) { page_counter_uncharge(&ug->memcg->memory, ug->nr_pages); - if (do_memsw_account()) - page_counter_uncharge(&ug->memcg->memsw, ug->nr_pages); - if (!cgroup_subsys_on_dfl(memory_cgrp_subsys) && ug->nr_kmem) - page_counter_uncharge(&ug->memcg->kmem, ug->nr_kmem); - memcg_oom_recover(ug->memcg); + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) { + if (!cgroup_memory_noswap) + page_counter_uncharge(&ug->memcg->memsw, + ug->nr_pages); + if (ug->nr_kmem) + page_counter_uncharge(&ug->memcg->kmem, + ug->nr_kmem); + memcg_oom_recover(ug->memcg); + } } local_irq_save(flags); From patchwork Fri Feb 12 17:01:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12085693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16AC4C433E0 for ; Fri, 12 Feb 2021 17:02:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A110764E39 for ; Fri, 12 Feb 2021 17:02:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A110764E39 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DCBFD8D0070; Fri, 12 Feb 2021 12:02:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D31F18D0060; Fri, 12 Feb 2021 12:02:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B5F7F8D0070; Fri, 12 Feb 2021 12:02:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0250.hostedemail.com [216.40.44.250]) by kanga.kvack.org (Postfix) with ESMTP id 909B78D0060 for ; Fri, 12 Feb 2021 12:02:26 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4135818039797 for ; Fri, 12 Feb 2021 17:02:26 +0000 (UTC) X-FDA: 77810234292.19.vase58_0416a7d27622 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id E67AA1ACC35 for ; Fri, 12 Feb 2021 17:02:25 +0000 (UTC) X-HE-Tag: vase58_0416a7d27622 X-Filterd-Recvd-Size: 3826 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Fri, 12 Feb 2021 17:02:25 +0000 (UTC) Received: by mail-pj1-f43.google.com with SMTP id gx20so765845pjb.1 for ; Fri, 12 Feb 2021 09:02:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uMiCMlqqsGt4v3DIDBbc5Tzpa+oMrTR3N8I/PD9p2nk=; b=NmStyGwgQEnVNrqN5Ubtgd25sYfpL8IXtOClOXbtc4s/sjMIQFqvb31t0DYjJZidM1 gCU3y8v0DWb3bByea+46x2tO/zxaZI0DLjF8Y1CVsBZu6wD68sRHZXdAlzW5+0pqOwTT TM3Jb8qnl9xOL4mWSIZj8Q1QL2ERlMxEY4F9uzUa1wjlotEQZKxZVjviz3OXd8WsF/Rg jFVabi1XtYlsOwbqnSNgovnWpCfmsOpZKBCmrxZU1f8vkqYrTp2sVvrX738HsEIw8MFn blBu6Yfthn9vxzvgemOvsKgO0ZWZHZw99ZIC5oXvfM3NPLKZaY4jXeVvJfxkeLuu24Wy 6/NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uMiCMlqqsGt4v3DIDBbc5Tzpa+oMrTR3N8I/PD9p2nk=; b=oIfIxWwNyVtuqc8JTqEVyq+ei+0911WTHeA8oU7zQidLpQ9SyW0bSf7s1MX+R7nWCq VQirhyPfITa3Pn6i/CjtGkak4GDm/jaXoHr7l1z3bIplnhiQptRP3omr/P0VVv55wBvB H66IheBoi273mQd0hXMyVMwZxXVnMkviI+XET9yHYbXva/4w6sgxcqP8Nh+/wWU1r356 JOpNOLqxIZkqK57azWBC5fAnkPsO+2NKClO/jiFKXWFcGkTQpy6tYgOCcB3m5/KsyM58 aSuVEJmb+QWySPW2j4rgqxsaGjbKH0bZb48w2WqVB3CrwbyDgMgVyUo1HjjqXcsdEhI4 9bGQ== X-Gm-Message-State: AOAM5321gOSyy7xdCxZo0sFPokuoXegI/T/kzxb1wyUfvXWLvghyGpDB 2mvQoAD82waGsXP1JlnLFQIB7w== X-Google-Smtp-Source: ABdhPJzNZ8r1f8u/Wzepj9mLIFlxDAW2JOdq6rVUNEpN5TjGRjw0rdpCvkeGjJveQHXo8f+etJVqEw== X-Received: by 2002:a17:902:d886:b029:e1:7784:4db5 with SMTP id b6-20020a170902d886b02900e177844db5mr3677168plz.72.1613149344235; Fri, 12 Feb 2021 09:02:24 -0800 (PST) Received: from localhost.bytedance.net ([139.177.225.254]) by smtp.gmail.com with ESMTPSA id e21sm9317815pgv.74.2021.02.12.09.02.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Feb 2021 09:02:23 -0800 (PST) From: Muchun Song To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org Cc: cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song Subject: [PATCH 3/4] mm: memcontrol: bail out early when id is zero Date: Sat, 13 Feb 2021 01:01:58 +0800 Message-Id: <20210212170159.32153-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210212170159.32153-1-songmuchun@bytedance.com> References: <20210212170159.32153-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The memcg ID cannot be zero, but we can pass zero to mem_cgroup_from_id, so idr_find() is pointless and wastes CPU cycles. Signed-off-by: Muchun Song --- mm/memcontrol.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index a3f26522765a..68ed4b297c13 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5173,6 +5173,9 @@ static inline void mem_cgroup_id_put(struct mem_cgroup *memcg) struct mem_cgroup *mem_cgroup_from_id(unsigned short id) { WARN_ON_ONCE(!rcu_read_lock_held()); + /* The memcg ID cannot be zero. */ + if (id == 0) + return NULL; return idr_find(&mem_cgroup_idr, id); } From patchwork Fri Feb 12 17:01:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12085695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 984DDC433E0 for ; Fri, 12 Feb 2021 17:02:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4F50564E39 for ; Fri, 12 Feb 2021 17:02:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4F50564E39 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DCCDE8D0071; Fri, 12 Feb 2021 12:02:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D82778D0060; Fri, 12 Feb 2021 12:02:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C6BFA8D0071; Fri, 12 Feb 2021 12:02:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0052.hostedemail.com [216.40.44.52]) by kanga.kvack.org (Postfix) with ESMTP id AF0C48D0060 for ; Fri, 12 Feb 2021 12:02:29 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 62E3918209BD2 for ; Fri, 12 Feb 2021 17:02:29 +0000 (UTC) X-FDA: 77810234418.23.cry53_3609c3c27622 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 4124D37608 for ; Fri, 12 Feb 2021 17:02:29 +0000 (UTC) X-HE-Tag: cry53_3609c3c27622 X-Filterd-Recvd-Size: 4112 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Fri, 12 Feb 2021 17:02:28 +0000 (UTC) Received: by mail-pf1-f177.google.com with SMTP id b145so6180655pfb.4 for ; Fri, 12 Feb 2021 09:02:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7mHQ3glb31sFuwvoGcKO+DumCO9HSr0Bb7Nri9jWV50=; b=mAnF3VGeuW6pSRvu2LN2xqj+dzYyLwh/OPEDqMeHluwaz2poT5if4WhFnoBubDskRD WhemKL3Sc0cI5yOpXFf9KOI9IxspEKlb0Yx00kM0w2z3lZWpHDh9nEqUqtIJIKQPqLA5 xcpTmN54VOVH83PLYEfe7rFxnQwCu3TC8LcM+lp8sY5f2JdsTwXNHNUz/KY0RPxxBpqK ZJB6hm8JCvRjPWPpLFw2sWnkKZDw26g3HarWaEIjaNl5953EBj7UaCDr50tnWTqaScTC rvhmQZW1Vgq9yaElWbZpHKpFG09EA5M7Ga5tjZxGq+ijpVfQQ8fzo1Axg+qd6FtjiW1J 52Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7mHQ3glb31sFuwvoGcKO+DumCO9HSr0Bb7Nri9jWV50=; b=OuSCPPv78SZfC3gJ5kgYxqv1nat8rRyIcJWPOGdWxKYYrX3YUih6UWGbBjw07uTdBt XcJihwwyvW+fLtZD4rFKpaJit7lDLPKOXGyvoCK/Gepj0qhbKKFnNoGwP5NUSk4CgNsM MPvunAjI/NORB4wzwuSczMSG56qz4BvKOM4/y3N0KsnY7d66bSSJVZ4wj0Gib29Lp+z0 rrVv3Cvp6GqW8Yg5jvXtKRciYzNR2jdQ/ann0JZjITNiPdYw0tB2y1EXa+iyaRooP3Cr mWm8jmyYYCVZJx0eWqTeYdNZvsuD5sd5gZZ2UOQIF75PbxqqiLrxdbA4AS/SAAaj6ZnZ sUeQ== X-Gm-Message-State: AOAM5325h/L3UQLkLWJAPUDpWN789ayk0mJ2L+rmwzWek6hhqYS8vvO3 BGu9GnDUezt4VE4c2wrIs/+RuQ== X-Google-Smtp-Source: ABdhPJyYIF4+lkvlNFXhM8SAayi2rNZb5rZhRBWKr/m9i+0EfQyNSsW8uvtRc94AUdYeVdY9ep214Q== X-Received: by 2002:a62:7fcb:0:b029:1da:36b1:8ac7 with SMTP id a194-20020a627fcb0000b02901da36b18ac7mr3749647pfd.13.1613149347766; Fri, 12 Feb 2021 09:02:27 -0800 (PST) Received: from localhost.bytedance.net ([139.177.225.254]) by smtp.gmail.com with ESMTPSA id e21sm9317815pgv.74.2021.02.12.09.02.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Feb 2021 09:02:27 -0800 (PST) From: Muchun Song To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org Cc: cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song Subject: [PATCH 4/4] mm: memcontrol: fix swap uncharge on cgroup v2 Date: Sat, 13 Feb 2021 01:01:59 +0800 Message-Id: <20210212170159.32153-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210212170159.32153-1-songmuchun@bytedance.com> References: <20210212170159.32153-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The swap charges the actual number of swap entries on cgroup v2. If a swap cache page is charged successful, and then we uncharge the swap counter. It is wrong on cgroup v2. Because the swap entry is not freed. Fixes: 2d1c498072de ("mm: memcontrol: make swap tracking an integral part of memory control") Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Michal Hocko --- mm/memcontrol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c737c8f05992..be6bc5044150 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6753,7 +6753,7 @@ int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask) memcg_check_events(memcg, page); local_irq_enable(); - if (PageSwapCache(page)) { + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys) && PageSwapCache(page)) { swp_entry_t entry = { .val = page_private(page) }; /* * The swap entry might not get freed for a long time,