From patchwork Wed Jan 9 12:20:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10753981 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A2C8746 for ; Wed, 9 Jan 2019 12:20:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 892CB28E8E for ; Wed, 9 Jan 2019 12:20:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 878CE28EBB; Wed, 9 Jan 2019 12:20:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D950E28EBA for ; Wed, 9 Jan 2019 12:20:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DBDB8E009F; Wed, 9 Jan 2019 07:20:39 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6CB108E0038; Wed, 9 Jan 2019 07:20:39 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 542F08E009F; Wed, 9 Jan 2019 07:20:39 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lj1-f197.google.com (mail-lj1-f197.google.com [209.85.208.197]) by kanga.kvack.org (Postfix) with ESMTP id D9A838E0038 for ; Wed, 9 Jan 2019 07:20:38 -0500 (EST) Received: by mail-lj1-f197.google.com with SMTP id v24-v6so1779971ljj.10 for ; Wed, 09 Jan 2019 04:20:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=5tY899Xd2X+9Z3CRn+G6UgzyTSWMoq5hPLIK/1l/VBc=; b=mcEfSBwvNxvWgKvH0jBgezn/t0bE4bDdvPbQPA2F2EGC0zd8iBEnKKEfWd7Y9udpKe dMsGKbgcGglFuX8782V3EaJh2gpdsIf9Sh56dsps7Ipzseq9fr/OvLuueqa8H/wXQKgc YLdq5fhKvHQOaC6osELaJApN4dDExHz8HvN7ntAA+GgpZX6Jy5WFYxOgpzyQXpvO3ZLc Lb+m7t3oW6EzwZq1Atv85aC3YY1XnSnf8xR61Ua4IgT0j5Tvqbb6TNmfXwFTRvtcu8LG 89IJN8BxOZnyOk6skf9XpQI+7FAC2V9zonHXoFLk2gA8YQZ6pi10L4CCwfoE6lFnRm2A 9EVQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: AJcUukfgBmRdXTd+739CFY95SrTeUA+9BBRlAxsCjaSM4b/+BhKtE01A FrF8fHFm0ZzzD60gscWe27MKd0tZxZ0pxC6Xi3yq7MW+g2g4wKFy4FVnCrKZLuJUeyEj8qrRaG7 4w+EuDZxznAPLUsHXchRiQGtzk5Zft/xFU1CdqY2qKM0dAd0eA1lgUoGqaTWXlGv0xg== X-Received: by 2002:a2e:197:: with SMTP id f23-v6mr3309296lji.144.1547036438162; Wed, 09 Jan 2019 04:20:38 -0800 (PST) X-Google-Smtp-Source: ALg8bN4EKDkZuRzn+8Iz/majJKX712vWZq4aPRPBKasfRKTUusr8eQI3nrvMX6LRzVzNXH6GYQAJ X-Received: by 2002:a2e:197:: with SMTP id f23-v6mr3309247lji.144.1547036436943; Wed, 09 Jan 2019 04:20:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547036436; cv=none; d=google.com; s=arc-20160816; b=mr9thyfy2QNfe3CbiJtl7Q9pJtF9Kx7WUhgCxesxSunxXT/pyfIb7H1Rdoysq7qujt Bmh1zzOjyYFl5ibL2cgPwoM0R/r1xtqFO1m7oWbN3LgPeKT7/jT7GTSZGYGsAxwdTMUU ITj0+wr5eY262rBl/Tm09r1mDumY9tDvBjSGT9kWvQdHA8zsB5pUtO2LU55rsEJpE4gw 7HYRt5T73kTMO5qk1z3aeK4eKwRBalecFb9wiu8mgZgYu2W/aIYNp7rbrYNWGMVExjBD ZEwUc5YdL8zI1SjFF+eGrQzWZP6hiUqEePXNhGHyqMbuWnfpjWKgHRZT/TRBXzCL80BJ +b/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=5tY899Xd2X+9Z3CRn+G6UgzyTSWMoq5hPLIK/1l/VBc=; b=S+gUYJbyVZg4JzTkBRYYXGGu4E9L0Kj16kGPzPAgwBvGQW9mxnGN95AaoO51w52PrM D5zWiJWTng0WROmGYCRwTAB5Iv3FJe/8YXJayy8GzwCDFDdG853VR6e1Fc/NU3nB7W17 2JkoksrBuAPgiflk9yXSY+tGM4iQl2ECPBrw+xRHXyeCvKPNt4i7LtGKpXctXBY6cpJe d6KNh5LCtfqDsxcD6hirK9JSj/3QOtt93Di8+Vwf17997+B+WX8Pl86qjfdf+nillvhS iaWEjVMvs7l0XtRFO+pMYOpSNqNvMlRzzKnMcwV0WMqe5VSP1XLNqn1Id/RpxVWpPh7v IzFg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id v23-v6si63981269ljh.63.2019.01.09.04.20.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 09 Jan 2019 04:20:36 -0800 (PST) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1ghCqO-0004gt-O2; Wed, 09 Jan 2019 15:20:24 +0300 Subject: [PATCH 1/3] mm: Uncharge and keep page in pagecache on memcg reclaim From: Kirill Tkhai To: akpm@linux-foundation.org, hannes@cmpxchg.org, josef@toxicpanda.com, jack@suse.cz, hughd@google.com, ktkhai@virtuozzo.com, darrick.wong@oracle.com, mhocko@suse.com, aryabinin@virtuozzo.com, guro@fb.com, mgorman@techsingularity.net, shakeelb@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Wed, 09 Jan 2019 15:20:24 +0300 Message-ID: <154703642447.32690.5604527676583713589.stgit@localhost.localdomain> In-Reply-To: <154703479840.32690.6504699919905946726.stgit@localhost.localdomain> References: <154703479840.32690.6504699919905946726.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch makes __remove_mapping() not remove a page from pagecache on memcg reclaim. After all mappings are removed and refcounter is freezed, we uncharge page memcg. Further putback_lru_page() places page into root_mem_cgroup, so it remains in pagecache till global reclaim. This gives memcg tasks extra possibility to obtain page from pagecache instead of launching IO. Next patch makes pagecache_get_page() to recharge a page in case of its memcg is NULL (i.e., on first access after uncharging). It looks to be the only function, which is used by filesystems to obtain a pagecache page. Here we introduce AS_KEEP_MEMCG_RECLAIM flag to mark the filesystems, which are reviewed, that they really follow this way. It has a sense to keep pages in __remove_mapping() only for them. Later, we remove this flags after all filesystems are reviewed. Signed-off-by: Kirill Tkhai --- include/linux/pagemap.h | 1 + mm/vmscan.c | 22 ++++++++++++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 1020e6f40880..1b880da85868 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -29,6 +29,7 @@ enum mapping_flags { AS_EXITING = 4, /* final truncate in progress */ /* writeback related tags are not used */ AS_NO_WRITEBACK_TAGS = 5, + AS_KEEP_MEMCG_RECLAIM = 6, }; /** diff --git a/mm/vmscan.c b/mm/vmscan.c index a714c4f800e9..7237603c8973 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -887,7 +887,7 @@ static pageout_t pageout(struct page *page, struct address_space *mapping, * gets returned with a refcount of 0. */ static int __remove_mapping(struct address_space *mapping, struct page *page, - bool reclaimed) + bool reclaimed, bool memcg_reclaim) { unsigned long flags; int refcount; @@ -963,7 +963,20 @@ static int __remove_mapping(struct address_space *mapping, struct page *page, if (reclaimed && page_is_file_cache(page) && !mapping_exiting(mapping) && !dax_mapping(mapping)) shadow = workingset_eviction(mapping, page); - __delete_from_page_cache(page, shadow); +#ifdef CONFIG_MEMCG + if (memcg_reclaim && + test_bit(AS_KEEP_MEMCG_RECLAIM, &mapping->flags)) { + /* + * Page is not dirty/writeback/mapped, so we may avoid + * taking mem_cgroup::move_lock for changing its memcg. + * See mem_cgroup_move_account() for details. + */ + mem_cgroup_uncharge(page); + page_ref_unfreeze(page, refcount); + goto cannot_free; + } else +#endif + __delete_from_page_cache(page, shadow); xa_unlock_irqrestore(&mapping->i_pages, flags); if (freepage != NULL) @@ -985,7 +998,7 @@ static int __remove_mapping(struct address_space *mapping, struct page *page, */ int remove_mapping(struct address_space *mapping, struct page *page) { - if (__remove_mapping(mapping, page, false)) { + if (__remove_mapping(mapping, page, false, false)) { /* * Unfreezing the refcount with 1 rather than 2 effectively * drops the pagecache ref for us without requiring another @@ -1458,7 +1471,8 @@ static unsigned long shrink_page_list(struct list_head *page_list, count_vm_event(PGLAZYFREED); count_memcg_page_event(page, PGLAZYFREED); - } else if (!mapping || !__remove_mapping(mapping, page, true)) + } else if (!mapping || !__remove_mapping(mapping, page, true, + !global_reclaim(sc))) goto keep_locked; unlock_page(page);