From patchwork Thu Nov 28 04:17:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 11265373 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B523115AB for ; Thu, 28 Nov 2019 04:17:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5DD9A215F2 for ; Thu, 28 Nov 2019 04:17:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dQ+jP802" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5DD9A215F2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E3BE66B04F9; Wed, 27 Nov 2019 23:17:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DEC3D6B04FA; Wed, 27 Nov 2019 23:17:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CDB076B04FB; Wed, 27 Nov 2019 23:17:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0153.hostedemail.com [216.40.44.153]) by kanga.kvack.org (Postfix) with ESMTP id B31A66B04F9 for ; Wed, 27 Nov 2019 23:17:38 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 609FB180AD807 for ; Thu, 28 Nov 2019 04:17:38 +0000 (UTC) X-FDA: 76204377396.21.body84_5fc08a7780e25 X-Spam-Summary: 2,0,0,2d2d6ec7bf36faf4,d41d8cd98f00b204,laoar.shao@gmail.com,:mhocko@kernel.org:hannes@cmpxchg.org:vdavydov.dev@gmail.com:akpm@linux-foundation.org::laoar.shao@gmail.com:david@redhat.com,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1345:1437:1535:1543:1605:1711:1730:1747:1777:1792:2194:2198:2199:2200:2393:2553:2559:2562:2693:2741:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3873:3874:4250:4423:5007:6119:6261:6653:7514:7875:7903:9413:10004:11026:11232:11658:11914:12043:12048:12295:12296:12297:12438:12517:12519:12555:12895:13161:13229:14093:14096:14181:14394:14687:14721:21080:21444:21451:21627:21666:30054:30070:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:26,LUA_SUMMARY:none X-HE-Tag: body84_5fc08a7780e25 X-Filterd-Recvd-Size: 5856 Received: from mail-pj1-f68.google.com (mail-pj1-f68.google.com [209.85.216.68]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Thu, 28 Nov 2019 04:17:37 +0000 (UTC) Received: by mail-pj1-f68.google.com with SMTP id f7so11260119pjw.5 for ; Wed, 27 Nov 2019 20:17:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=X3+V+tzDN5FFywLOA66m82+E3YB5eFZUMuHbocI2TJs=; b=dQ+jP802j8NEOT2b7knXd0wJAJFNxZMj3SjtMkRPb6c+7GM46SB2d+AT1+G1/oLKsL Xs0oJPgq6/NaepyRHsspRPyBL/ZPguhiSolPAsqnKRDTUlCWOj/e0HWeUBvx+D4+rAwD AHwiFIO1o1vYhjCnN7STnI8zCnAxso5mzNjjKCgoCBCW98Lm7jcPN+YYuBPyrnka8+HB XStzd1sltLlJg/qAzPn4PNsd+qfV3mYgEx44yF08O/772EyYWWy6FzIxbxFNGTBZeULO vxNwu1TlPk/305D4Pvtq+48T7gzLegGVVsTTkM9RbgBu0Z1ZKxxM5FKBnFvLyJrHzAK2 fozw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=X3+V+tzDN5FFywLOA66m82+E3YB5eFZUMuHbocI2TJs=; b=ol14Ehqc4XDIDi8aWyIjBn0VMSwEL/Pja7TXfjVxtk3jUKNNg/ypFgvnWYMNTUjp6I jJcutv0df9c6KmP2CmcXqGPw1cIbWTKZP+4MiL15V3H/ibKa5Xy2QvY8iTPJoFjviP/Z ry45Mjn1KBU+yv8ULa/12rjX1tZJeOZZnZn2Gln9ZhJ+QZA69x67naA3viBRmLqd9KLk CjAGPbdTBqHY8/nKmrk3bzkPvu23FvU8O5X98v53p7NB5q/QOY/Pzjrh6HXBdTUX6ATz 6TyiIsJqATScw0vVd97hCxj12ivCoMn5jJviUKtR7Q8wrQOhesRMK7snakG/jc+3Zy3D fYVw== X-Gm-Message-State: APjAAAXI7OlndIJPCfb4qswYl29pz+vw5pp8oTHj5onEMuCZvfpMygmz QdLGQh7sAOQt+gvzlW6oDCY= X-Google-Smtp-Source: APXvYqySj52xOX2XYx57Cxq7dfL2oHBgvRttHvWtBw0AoE+Ahk+vXz2MDrZNHrSdegsiJmbrkkAAvQ== X-Received: by 2002:a17:902:854c:: with SMTP id d12mr7743751plo.264.1574914656622; Wed, 27 Nov 2019 20:17:36 -0800 (PST) Received: from dev.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id x21sm17763451pfi.122.2019.11.27.20.17.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Nov 2019 20:17:35 -0800 (PST) From: Yafang Shao To: mhocko@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, Yafang Shao , David Hildenbrand Subject: [PATCH v3] mm, memcg: fix the stupid OOM killer when shrinking memcg hard limit Date: Wed, 27 Nov 2019 23:17:13 -0500 Message-Id: <1574914633-2020-1-git-send-email-laoar.shao@gmail.com> X-Mailer: git-send-email 1.8.3.1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When there are no more processes in a memcg (e.g., due to OOM group), we can still have file pages in the page cache. If these pages are protected by memory.min, they can't be reclaimed. Especially if there won't be another process in this memcg and the memcg is kept online, we do want to drop these pages from the page cache. By dropping these page caches we can avoid reclaimers (e.g., kswapd or direct) to scan and reclaim pages from all memcgs in the system - because the reclaimers will try to fairly reclaim pages from all memcgs in the system when under memory pressure. By setting the hard limit of such a memcg to 0, we allow to drop the page cache of such memcgs. Unfortunately, this may invoke the OOM killer and generate a lot of output. The OOM output is not expected by an admin who wants to drop these caches and knows that there are no processes in this memcg anymore. Therefore, if a memcg is not populated, we should not invoke the OOM killer - there is nothing to kill. The next time a new process is started in the memcg and the "max" is still below usage, the OOM killer will be invoked and the new process will be killed. [ Above commit log is contributed by David ] What's worse about this issue is that when there're killable tasks and the OOM killer killed the last task, and what will happen then ? As nr_reclaims is already 0 and drained is alreay true, the OOM killer will try to kill nothing (because he knows he has killed the last task), what's a stupid behavior. Someone may worry that the admins may not going to see that the memcg was OOM due to the limit change. But this is not a issue, because the admins changes the limit and then the admins must check the result of his change - by checking memory.{max, current, stat} he can get all he wants. Cc: David Hildenbrand Nacked-by: Michal Hocko Signed-off-by: Yafang Shao --- Changes since v2: Refresh the subject and commit log. The original subject is "mm, memcg: avoid oom if cgroup is not populated" --- mm/memcontrol.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 1c4c08b..e936f1b 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6139,9 +6139,20 @@ static ssize_t memory_max_write(struct kernfs_open_file *of, continue; } - memcg_memory_event(memcg, MEMCG_OOM); - if (!mem_cgroup_out_of_memory(memcg, GFP_KERNEL, 0)) + /* If there's no procesess, we don't need to invoke the OOM + * killer. Then next time when you try to start a process + * in this memcg, the max may still bellow usage, and then + * this OOM killer will be invoked. This can be considered + * as lazy OOM, that is we have been always doing in the + * kernel. Pls. Michal, that is really consistency. + */ + if (cgroup_is_populated(memcg->css.cgroup)) { + memcg_memory_event(memcg, MEMCG_OOM); + if (!mem_cgroup_out_of_memory(memcg, GFP_KERNEL, 0)) + break; + } else { break; + } } memcg_wb_domain_size_changed(memcg);