From patchwork Mon Mar 16 10:39:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 11440127 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 012D51667 for ; Mon, 16 Mar 2020 10:40:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B75C220663 for ; Mon, 16 Mar 2020 10:40:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="U7jolHM3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B75C220663 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 00C316B0008; Mon, 16 Mar 2020 06:40:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EFEC16B000A; Mon, 16 Mar 2020 06:40:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF0FC6B000C; Mon, 16 Mar 2020 06:40:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0222.hostedemail.com [216.40.44.222]) by kanga.kvack.org (Postfix) with ESMTP id C3FF76B0008 for ; Mon, 16 Mar 2020 06:40:45 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A25FB8777 for ; Mon, 16 Mar 2020 10:40:45 +0000 (UTC) X-FDA: 76600882050.09.cent51_23adbcec35d55 X-Spam-Summary: 2,0,0,6eac881b45d6b440,d41d8cd98f00b204,laoar.shao@gmail.com,,RULES_HIT:2:41:355:379:541:800:960:965:966:973:988:989:1260:1345:1359:1437:1535:1605:1730:1747:1777:1792:1801:2196:2198:2199:2200:2393:2553:2559:2562:2693:2897:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4049:4120:4250:4321:4385:4390:4395:4470:4605:5007:6261:6653:7514:7903:9010:9121:9413:10004:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12683:12895:12986:13161:13180:13229:14096:14394:14687:21080:21444:21450:21451:21611:21627:21666:21796:21990:30036:30054:30090,0,RBL:209.85.216.66:@gmail.com:.lbl8.mailshell.net-66.100.201.100 62.50.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: cent51_23adbcec35d55 X-Filterd-Recvd-Size: 9063 Received: from mail-pj1-f66.google.com (mail-pj1-f66.google.com [209.85.216.66]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Mon, 16 Mar 2020 10:40:45 +0000 (UTC) Received: by mail-pj1-f66.google.com with SMTP id hg10so4156203pjb.1 for ; Mon, 16 Mar 2020 03:40:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=L3eSlYEGbu4PoC6YG4Y6sJy5VuyO654A9z9jaMwfoII=; b=U7jolHM3bRjcBPeF4kiQ8Qv4bCZPDeiyy2doENtGgy5ldWAt2Sh715JT8EcVxaAbHk et0mFXOJ2tTKBs9DA6+eX0rnjUvNcytq5XOJxDzfpDs1TGdXTAwT87twSqg0ZptuTxcY 2cpCxGFrCjs+LNtDQbR/hNxqUHd+65sSNGd5lxS5BKsyoph7cuPrMAKjah2SF9XHT88l qBUmIW/YwEathMLVvk6vSbBBx9EQQNz+a96dZ+uHp4x0amLEe17H3KyBvgtg/ul28uIz Pu7RdNIGr3G8rdCBi870VZCeY+EShDhB8PvNOhbnXEmJeX5m1D9rXkDoBr+ygl0DPCiy LzGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=L3eSlYEGbu4PoC6YG4Y6sJy5VuyO654A9z9jaMwfoII=; b=ius7evysBM8/Ny/K3N6gj9o9mDcdpj9X0tZlcSM3R+7gH55JAqZRwHpmFEF7/AhXZl sPW7IvgCNK6puKyknFxkXjXEugf7zV5b8GVKYx/o7jen5gIXeKX1/JnhvNnsz0iHsqaw uRwDHdhuhr/P1mryrCtqbu3X7x7h1/Nk0eXXLiPNDfLD1sTrn9yy08XsGO6uxe+uutRo 2fZJ2nrcIZyunWlcvgZ8BLFIxfQijg2Fqsk/nleySkG9xbfaMDYiOw6mtXIvkP+2Ibot QQctSMsOkmBYL2YOu1b1eTxvixCWA33AiZyR9fST2AlM0QIguHwVE5VMzqehkCrJNbfx B+Yg== X-Gm-Message-State: ANhLgQ0VIeN+cdeBrJ6+pV8KzSZVH5yk5RyrB1lVOfKE77zh2ctBrS0K 4ZX21wCCydWuZyLx49K+8iaULh6FJ++83g== X-Google-Smtp-Source: ADFU+vsFPfI9KgXUp4a42DVqAeDWUuRxQKl66czJZMnK/JrjUBixHW1pReSK4/A+y/ljHFP6Sj9lnQ== X-Received: by 2002:a17:902:8498:: with SMTP id c24mr25960172plo.233.1584355244072; Mon, 16 Mar 2020 03:40:44 -0700 (PDT) Received: from dev.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id h2sm19834276pjc.7.2020.03.16.03.40.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Mar 2020 03:40:43 -0700 (PDT) From: Yafang Shao To: dchinner@redhat.com, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com, akpm@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Cc: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Yafang Shao Subject: [PATCH v6 3/3] inode: protect page cache from freeing inode Date: Mon, 16 Mar 2020 06:39:58 -0400 Message-Id: <1584355198-10137-4-git-send-email-laoar.shao@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1584355198-10137-1-git-send-email-laoar.shao@gmail.com> References: <1584355198-10137-1-git-send-email-laoar.shao@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On my server there're some running MEMCGs protected by memory.{min, low}, but I found the usage of these MEMCGs abruptly became very small, which were far less than the protect limit. It confused me and finally I found that was because of inode stealing. Once an inode is freed, all its belonging page caches will be dropped as well, no matter how may page caches it has. So if we intend to protect the page caches in a memcg, we must protect their host (the inode) first. Otherwise the memcg protection can be easily bypassed with freeing inode, especially if there're big files in this memcg. Supposes we have a memcg, and the stat of this memcg is, memory.current = 1024M memory.min = 512M And in this memcg there's a inode with 800M page caches. Once this memcg is scanned by kswapd or other regular reclaimers, kswapd <<<< It can be either of the regular reclaimers. shrink_node_memcgs switch (mem_cgroup_protected()) <<<< Not protected case MEMCG_PROT_NONE: <<<< Will scan this memcg beak; shrink_lruvec() <<<< Reclaim the page caches shrink_slab() <<<< It may free this inode and drop all its page caches(800M). So we must protect the inode first if we want to protect page caches. Note that this inode may be a cold inode (in the tail of list lru), because memcg protection protects all slabs and page cache pages whatever they are cold or hot. IOW, this is a memcg-protection-specific issue. The inherent mismatch between memcg and inode is a trouble. One inode can be shared by different MEMCGs, but it is a very rare case. If an inode is shared, its belonging page caches may be charged to different MEMCGs. Currently there's no perfect solution to fix this kind of issue, but the inode majority-writer ownership switching can help it more or less. Cc: Dave Chinner Cc: Johannes Weiner Signed-off-by: Yafang Shao --- fs/inode.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 72 insertions(+), 3 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 93d9252..f5a9537 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -55,6 +55,12 @@ * inode_hash_lock */ +struct inode_isolate_control { + struct list_head *freeable; + struct mem_cgroup *memcg; /* derived from shrink_control */ + bool memcg_low_reclaim; /* derived from scan_control */ +}; + static unsigned int i_hash_mask __read_mostly; static unsigned int i_hash_shift __read_mostly; static struct hlist_head *inode_hashtable __read_mostly; @@ -715,6 +721,58 @@ int invalidate_inodes(struct super_block *sb, bool kill_dirty) return busy; } +#ifdef CONFIG_MEMCG_KMEM +/* + * Once an inode is freed, all its belonging page caches will be dropped as + * well, even if there're lots of page caches. So if we intend to protect + * page caches in a memcg, we must protect their host(the inode) first. + * Otherwise the memcg protection can be easily bypassed with freeing inode, + * especially if there're big files in this memcg. + * Note that it may happen that the page caches are already charged to the + * memcg, but the inode hasn't been added to this memcg yet. In this case, + * this inode is not protected. + * The inherent mismatch between memcg and inode is a trouble. One inode + * can be shared by different MEMCGs, but it is a very rare case. If + * an inode is shared, its belonging page caches may be charged to + * different MEMCGs. Currently there's no perfect solution to fix this + * kind of issue, but the inode majority-writer ownership switching can + * help it more or less. + */ +static bool memcg_can_reclaim_inode(struct inode *inode, + struct inode_isolate_control *iic) +{ + unsigned long protection; + struct mem_cgroup *memcg; + bool reclaimable = true; + + if (!inode->i_data.nrpages) + goto out; + + /* Excludes freeing inode via drop_caches */ + if (!current->reclaim_state) + goto out; + + memcg = iic->memcg; + if (!memcg || memcg == root_mem_cgroup) + goto out; + + protection = mem_cgroup_protection(memcg, iic->memcg_low_reclaim); + if (!protection) + goto out; + + reclaimable = false; + +out: + return reclaimable; +} +#else /* CONFIG_MEMCG_KMEM */ +static bool memcg_can_reclaim_inode(struct inode *inode, + struct inode_isolate_control *iic) +{ + return true; +} +#endif /* CONFIG_MEMCG_KMEM */ + /* * Isolate the inode from the LRU in preparation for freeing it. * @@ -733,8 +791,9 @@ int invalidate_inodes(struct super_block *sb, bool kill_dirty) static enum lru_status inode_lru_isolate(struct list_head *item, struct list_lru_one *lru, spinlock_t *lru_lock, void *arg) { - struct list_head *freeable = arg; - struct inode *inode = container_of(item, struct inode, i_lru); + struct inode_isolate_control *iic = arg; + struct list_head *freeable = iic->freeable; + struct inode *inode = container_of(item, struct inode, i_lru); /* * we are inverting the lru lock/inode->i_lock here, so use a trylock. @@ -743,6 +802,11 @@ static enum lru_status inode_lru_isolate(struct list_head *item, if (!spin_trylock(&inode->i_lock)) return LRU_SKIP; + if (!memcg_can_reclaim_inode(inode, iic)) { + spin_unlock(&inode->i_lock); + return LRU_ROTATE; + } + /* * Referenced or dirty inodes are still in use. Give them another pass * through the LRU as we canot reclaim them now. @@ -800,9 +864,14 @@ long prune_icache_sb(struct super_block *sb, struct shrink_control *sc) { LIST_HEAD(freeable); long freed; + struct inode_isolate_control iic = { + .freeable = &freeable, + .memcg = sc->memcg, + .memcg_low_reclaim = sc->memcg_low_reclaim, + }; freed = list_lru_shrink_walk(&sb->s_inode_lru, sc, - inode_lru_isolate, &freeable); + inode_lru_isolate, &iic); dispose_list(&freeable); return freed; }