From patchwork Mon Jul 13 11:05:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Down X-Patchwork-Id: 11659481 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 576B814DD for ; Mon, 13 Jul 2020 11:05:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1654D20773 for ; Mon, 13 Jul 2020 11:05:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="AT52M10G" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1654D20773 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4DFF98D0005; Mon, 13 Jul 2020 07:05:58 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4C0AD8D0001; Mon, 13 Jul 2020 07:05:58 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A65C8D0005; Mon, 13 Jul 2020 07:05:58 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0037.hostedemail.com [216.40.44.37]) by kanga.kvack.org (Postfix) with ESMTP id 232C98D0001 for ; Mon, 13 Jul 2020 07:05:58 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id DD5DA180AD81F for ; Mon, 13 Jul 2020 11:05:57 +0000 (UTC) X-FDA: 77032772754.28.hook41_221366226ee7 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id AA5196C11 for ; Mon, 13 Jul 2020 11:05:57 +0000 (UTC) X-Spam-Summary: 1,0,0,d16e648653a2a708,d41d8cd98f00b204,chris@chrisdown.name,,RULES_HIT:2:41:355:379:800:960:973:988:989:1260:1277:1312:1313:1314:1345:1359:1437:1516:1518:1519:1535:1593:1594:1595:1596:1605:1730:1747:1777:1792:2393:2553:2559:2562:2897:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4049:4120:4250:4321:4470:4605:5007:6261:6653:7514:7576:7875:7903:7904:10004:11026:11473:11658:11914:12043:12219:12291:12295:12296:12297:12438:12517:12519:12555:12664:12679:12683:12895:12986:13149:13161:13229:13230:13255:13439:13895:14096:14097:14394:21080:21433:21444:21451:21627:21740:21939:21966:21990:30054:30056:30064:30074:30090,0,RBL:209.85.208.68:@chrisdown.name:.lbl8.mailshell.net-62.2.0.100 66.100.201.201;04yg6osm5ae1fatiyjf8becbk9mzoycduwi3gd9mjjfx46ca48qauuchk4scfth.qxo3sgfpnjwgndknkmxje8qzi9qaezfq3uhhzjhubmqjj5tdd4ncafcxnu1y47n.w-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSB L:none,C X-HE-Tag: hook41_221366226ee7 X-Filterd-Recvd-Size: 9853 Received: from mail-ed1-f68.google.com (mail-ed1-f68.google.com [209.85.208.68]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Mon, 13 Jul 2020 11:05:57 +0000 (UTC) Received: by mail-ed1-f68.google.com with SMTP id bm28so11154635edb.2 for ; Mon, 13 Jul 2020 04:05:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=from:date:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=aFXMEHhkBKemwA/QoFtxiRqyIHqOIfwBr0LYDpNQG/w=; b=AT52M10Glz6TiDoZ4uGS3qPTmg2vqCLijtoFBQ8ribBod0g/b47E9eyS4hn+Po7CIZ TfzqlI8FPBvDocG12P2yQK9i58fLZPBMa/ODoiYM4PgyrcNGC48huN8RO/sJncf/9QA/ Hz1D17cq1FNCR9cfEpnzBpuSAXsA5oUre28c8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=aFXMEHhkBKemwA/QoFtxiRqyIHqOIfwBr0LYDpNQG/w=; b=CKJyzHQAoqBjYelaOlLKBk7s/TZ2gvkz+W3jVpYbyf7afthv2Cxezjsv31J6bSQT9Q fvWM92IRL72S35U5rTBi700pJEIMhhfQb9vCtKqZK4mCXeof6grEyNllBRHIHwNIANd2 Zhw4SUls/Xi70UgGt5o/5wXLT6PLdHoEC0znGbQQRddOhYXm1vdqh69VFGsQupdiOQvt oHDXCRXqfk2TO2uGUvH2b9ENBjIDmXMRjt3/jrEuriceXH37+zJoakbI4L1vhL60eX9U GnZZO+obzSBWiiuno9DqO73Ouj1eaHAaiMpQWc7P+SvDbTjJJ8bZKvPWUETyIVQcnXJm uO0A== X-Gm-Message-State: AOAM530nH84ruI9ztSTKZMgleUAaHnTVE3RtdYwb+FBaGN7waHNsVYCC RDGouj2k7XSI/TyPhVHoJMqd4A== X-Google-Smtp-Source: ABdhPJwB4lLUUI0SASDFJIPJn2ZlyEEUDi9MRdcYhRBhVhW9BwKCmdfSLCoSFZ4vG91w9G9uO+W6JA== X-Received: by 2002:a50:ab52:: with SMTP id t18mr92428592edc.195.1594638356061; Mon, 13 Jul 2020 04:05:56 -0700 (PDT) Received: from localhost ([2620:10d:c093:400::5:ef88]) by smtp.gmail.com with ESMTPSA id bq8sm9165988ejb.103.2020.07.13.04.05.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 04:05:55 -0700 (PDT) From: Yafang Shao X-Google-Original-From: Yafang Shao Date: Mon, 13 Jul 2020 12:05:54 +0100 To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Yafang Shao , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 1/2] mm, memcg: Avoid stale protection values when cgroup is above protection Message-ID: <044fb8ecffd001c7905d27c0c2ad998069fdc396.1594638158.git.chris@chrisdown.name> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.14.5 (2020-06-23) X-Rspamd-Queue-Id: AA5196C11 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yafang Shao A cgroup can have both memory protection and a memory limit to isolate it from its siblings in both directions - for example, to prevent it from being shrunk below 2G under high pressure from outside, but also from growing beyond 4G under low pressure. Commit 9783aa9917f8 ("mm, memcg: proportional memory.{low,min} reclaim") implemented proportional scan pressure so that multiple siblings in excess of their protection settings don't get reclaimed equally but instead in accordance to their unprotected portion. During limit reclaim, this proportionality shouldn't apply of course: there is no competition, all pressure is from within the cgroup and should be applied as such. Reclaim should operate at full efficiency. However, mem_cgroup_protected() never expected anybody to look at the effective protection values when it indicated that the cgroup is above its protection. As a result, a query during limit reclaim may return stale protection values that were calculated by a previous reclaim cycle in which the cgroup did have siblings. When this happens, reclaim is unnecessarily hesitant and potentially slow to meet the desired limit. In theory this could lead to premature OOM kills, although it's not obvious this has occurred in practice. Workaround the problem by special casing reclaim roots in mem_cgroup_protection. These memcgs are never participating in the reclaim protection because the reclaim is internal. We have to ignore effective protection values for reclaim roots because mem_cgroup_protected might be called from racing reclaim contexts with different roots. Calculation is relying on root -> leaf tree traversal therefore top-down reclaim protection invariants should hold. The only exception is the reclaim root which should have effective protection set to 0 but that would be problematic for the following setup: Let's have global and A's reclaim in parallel: | A (low=2G, usage = 3G, max = 3G, children_low_usage = 1.5G) |\ | C (low = 1G, usage = 2.5G) B (low = 1G, usage = 0.5G) for A reclaim we have B.elow = B.low C.elow = C.low For the global reclaim A.elow = A.low B.elow = min(B.usage, B.low) because children_low_usage <= A.elow C.elow = min(C.usage, C.low) With the effective values resetting we have A reclaim A.elow = 0 B.elow = B.low C.elow = C.low and global reclaim could see the above and then B.elow = C.elow = 0 because children_low_usage > A.elow Which means that protected memcgs would get reclaimed. In future we would like to make mem_cgroup_protected more robust against racing reclaim contexts but that is likely more complex solution than this simple workaround. [hannes@cmpxchg.org - large part of the changelog] [mhocko@suse.com - workaround explanation] [chris@chrisdown.name - retitle] Fixes: 9783aa9917f8 ("mm, memcg: proportional memory.{low,min} reclaim") Signed-off-by: Yafang Shao Acked-by: Michal Hocko Acked-by: Johannes Weiner Acked-by: Chris Down Acked-by: Roman Gushchin Signed-off-by: Chris Down --- include/linux/memcontrol.h | 42 ++++++++++++++++++++++++++++++++++++-- mm/memcontrol.c | 8 ++++++++ mm/vmscan.c | 3 ++- 3 files changed, 50 insertions(+), 3 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index b8f52a3fed90..33d834a187e5 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -363,12 +363,49 @@ static inline bool mem_cgroup_disabled(void) return !cgroup_subsys_enabled(memory_cgrp_subsys); } -static inline unsigned long mem_cgroup_protection(struct mem_cgroup *memcg, +static inline unsigned long mem_cgroup_protection(struct mem_cgroup *root, + struct mem_cgroup *memcg, bool in_low_reclaim) { if (mem_cgroup_disabled()) return 0; + /* + * There is no reclaim protection applied to a targeted reclaim. + * We are special casing this specific case here because + * mem_cgroup_protected calculation is not robust enough to keep + * the protection invariant for calculated effective values for + * parallel reclaimers with different reclaim target. This is + * especially a problem for tail memcgs (as they have pages on LRU) + * which would want to have effective values 0 for targeted reclaim + * but a different value for external reclaim. + * + * Example + * Let's have global and A's reclaim in parallel: + * | + * A (low=2G, usage = 3G, max = 3G, children_low_usage = 1.5G) + * |\ + * | C (low = 1G, usage = 2.5G) + * B (low = 1G, usage = 0.5G) + * + * For the global reclaim + * A.elow = A.low + * B.elow = min(B.usage, B.low) because children_low_usage <= A.elow + * C.elow = min(C.usage, C.low) + * + * With the effective values resetting we have A reclaim + * A.elow = 0 + * B.elow = B.low + * C.elow = C.low + * + * If the global reclaim races with A's reclaim then + * B.elow = C.elow = 0 because children_low_usage > A.elow) + * is possible and reclaiming B would be violating the protection. + * + */ + if (root == memcg) + return 0; + if (in_low_reclaim) return READ_ONCE(memcg->memory.emin); @@ -899,7 +936,8 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm, { } -static inline unsigned long mem_cgroup_protection(struct mem_cgroup *memcg, +static inline unsigned long mem_cgroup_protection(struct mem_cgroup *root, + struct mem_cgroup *memcg, bool in_low_reclaim) { return 0; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 0145a77aa074..21b620e36aa0 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6566,6 +6566,14 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root, if (!root) root = root_mem_cgroup; + + /* + * Effective values of the reclaim targets are ignored so they + * can be stale. Have a look at mem_cgroup_protection for more + * details. + * TODO: calculation should be more robust so that we do not need + * that special casing. + */ if (memcg == root) return MEMCG_PROT_NONE; diff --git a/mm/vmscan.c b/mm/vmscan.c index 5215840ee217..89921a12acae 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2326,7 +2326,8 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc, unsigned long protection; lruvec_size = lruvec_lru_size(lruvec, lru, sc->reclaim_idx); - protection = mem_cgroup_protection(memcg, + protection = mem_cgroup_protection(sc->target_mem_cgroup, + memcg, sc->memcg_low_reclaim); if (protection) {