From patchwork Tue Jun 19 05:13:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 10473181 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0325360383 for ; Tue, 19 Jun 2018 05:13:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E757228179 for ; Tue, 19 Jun 2018 05:13:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DBC5628C04; Tue, 19 Jun 2018 05:13:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49CAC28179 for ; Tue, 19 Jun 2018 05:13:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C1316B0266; Tue, 19 Jun 2018 01:13:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F12086B0269; Tue, 19 Jun 2018 01:13:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D65936B026A; Tue, 19 Jun 2018 01:13:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg0-f71.google.com (mail-pg0-f71.google.com [74.125.83.71]) by kanga.kvack.org (Postfix) with ESMTP id 94AD26B0266 for ; Tue, 19 Jun 2018 01:13:50 -0400 (EDT) Received: by mail-pg0-f71.google.com with SMTP id o19-v6so6040659pgn.14 for ; Mon, 18 Jun 2018 22:13:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=Xw+SjgYvIwITvP/432MUmWBrfLlGMOh9oKDuJ2FIurQ=; b=uO57uxZt2HfN/oLxeua48ZbYqfHjdrjlKW7yiAkjKdHMaTDtC7soHNZL6NdT37JyZ5 D0icqCs0vmNTpXSWiFU9TaASkG0DN1DQBJh3YXPAvr6nVQrYJfg7hTiriYJ/xoBAhNty MivryUqoPL5+YkDB9jjZbpBljcRxjqGsePMHnqK8ouGFCIXh1UjRrxMCMudyO5al1rPe XdBACgbgqBpvYx7gBogfRD1VSlAv7LT+bS5V8YBY0cGHY75gjgES7ColQnxKegyBl3YW aD9f4mnQ8UfCf1tFwX5s99uiHyvmpMYN7OoNU+IcObbl1PlFKk6lXKzaySgJAFqr+MLg vNAg== X-Gm-Message-State: APt69E1Ft6sSAq96u6RGr5CJn6BW1BdBKBMuPC/qvQ58CIjYuJZM2h4V 3vxbbEbHdo4cVRq5ZEDM3L5nFm/a+hEBZw4dY4qqubTwqLLefVJEjmXoul+hB5XKerGjOueNhWG rZPH4iPrPpVs8OJH1TzHZ5GmUCD3encSGmh2PoYRYETZyWnyMfWL9lDUmOspb6TSTqCCHBsmbcY dJoztx/1hx/wj8RWM7NIKyS6adVGIMDKuOWpj7N6xow5kgbe2iHrcwxKFwq6ec9AxoxjRGN66Zw bx+fLZF7FBjbCqdaX/7Vsfk6XCUnmnZi5xHHXyYLZMneA+xTtLYKCE6vh2NlrGuuw9bVqgP8zfe 1t0a4YxNqIrkQ0zgIa9hbfkuzU2fGFzT6xixnWeTJl/1/7ClCxNo1kfCxBuayP4NHJb/hHuPzrM k X-Received: by 2002:a62:8a0a:: with SMTP id y10-v6mr16541590pfd.237.1529385230280; Mon, 18 Jun 2018 22:13:50 -0700 (PDT) X-Received: by 2002:a62:8a0a:: with SMTP id y10-v6mr16541556pfd.237.1529385229539; Mon, 18 Jun 2018 22:13:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529385229; cv=none; d=google.com; s=arc-20160816; b=quBXq24eD014uMOAanG4XCAxz9xL1zIhe5HZOJz2/kFAWITGTeXgQz1YpoC5u+5GLw xzMKVDGVVMbk2VA3aPaInU6alKARLiZd6SswPbb0eDPDM9wrlTF8qSy5Buy+Y7tIycM4 9jA08sTz8rqA/6Zyl5iJJO3Ja3SiMsvGSLnvg3+JsACD3FrI4+vDWG0qtkEBtxghjN4v V0eqpRNZ25142Atu1Aae8zuehQs7Y3jnFd3+Fh9aj7kQd3bAEspdQmKvXitZgx3WhrT9 alYc7I31EFn+NnhD+nKx15SGB8kV8PG6sUU2UecwXdLrMbv3AXDtyjywlXcAt51S0gpn rD5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=Xw+SjgYvIwITvP/432MUmWBrfLlGMOh9oKDuJ2FIurQ=; b=FNLdV9MSTgqpRwkU+ZLMMjOUeG525cmu5QzJ5f+46//5oUTr4aiztACPFS6wVVpzy4 tU948A4BTR96RK5VXKyShj/yOq8KG+S8/XUgwiMmOvLLhH0g8vDKCG8mf/heS0VYzRID n/jbpcT1vV+bK3UCzSJBsheDuaeFqErvEuHjKvUl0k9CVaNEhTpvublc746dnIXt3y8D ayOqlGBwaQsd+cgf5E6OqDAxgGxOJL/FjbtIQpoULn10IEMp31Zm/hIZuo1Oa8OBmcSl MoA4Xo1eR73v+w7hQ1V4MUozY+kx1WwckkgYpBYyCPCgdWlvoUzOH1cc/KEoT6Xoejz+ xSKA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=RO8enGtI; spf=pass (google.com: domain of shakeelb@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id l66-v6sor4850184pfi.22.2018.06.18.22.13.49 for (Google Transport Security); Mon, 18 Jun 2018 22:13:49 -0700 (PDT) Received-SPF: pass (google.com: domain of shakeelb@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=RO8enGtI; spf=pass (google.com: domain of shakeelb@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Xw+SjgYvIwITvP/432MUmWBrfLlGMOh9oKDuJ2FIurQ=; b=RO8enGtIamWi5l9P1KAPe90ParEH71qWpJjU/zOTnjbiEbEozJ56Lv0W3qFAPyK2uN M6UGpxDr+JrPp2uVjkoSlcmRfLBzozs9na2pP//S4dzNRDn2EGW3bLWM6r+7IrWjJDEr QXgevl/IbhFJBMiBL8v7A2mUCSDKd1Q8HxzZmQfenC9ZKOzw10OGdFCo7Sz1fdM3ZcKo TQoXea0zr6GxhquC9QSBPRodB+g/dPtLojCrW9sDJPKWQEwIXMGvoq5hMAIV6c4PUywe 4MwGMyt4Yu12SZtJ+J73TommRFBMN01PvlfiK8ZC3uZQWmFXYd2IVbBfU5DRBsOxtxLN tEuQ== X-Google-Smtp-Source: ADUXVKLngP++HS5H86cz4AtIu7lSldmnPBWkdUz5xyPrlsUJgAcLTgiBftq+QBz/HaMn95+kFGOcfg== X-Received: by 2002:a62:701:: with SMTP id b1-v6mr16574381pfd.252.1529385228852; Mon, 18 Jun 2018 22:13:48 -0700 (PDT) Received: from shakeelb.mtv.corp.google.com ([2620:15c:2cb:201:3a5f:3a4f:fa44:6b63]) by smtp.gmail.com with ESMTPSA id a23-v6sm28390504pfj.117.2018.06.18.22.13.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 18 Jun 2018 22:13:46 -0700 (PDT) From: Shakeel Butt To: Andrew Morton Cc: Michal Hocko , Johannes Weiner , Vladimir Davydov , Jan Kara , Greg Thelen , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Shakeel Butt , Jan Kara , Alexander Viro Subject: [PATCH 3/3] fs, mm: account buffer_head to kmemcg Date: Mon, 18 Jun 2018 22:13:27 -0700 Message-Id: <20180619051327.149716-4-shakeelb@google.com> X-Mailer: git-send-email 2.18.0.rc1.244.gcf134e6275-goog In-Reply-To: <20180619051327.149716-1-shakeelb@google.com> References: <20180619051327.149716-1-shakeelb@google.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The buffer_head can consume a significant amount of system memory and is directly related to the amount of page cache. In our production environment we have observed that a lot of machines are spending a significant amount of memory as buffer_head and can not be left as system memory overhead. Charging buffer_head is not as simple as adding __GFP_ACCOUNT to the allocation. The buffer_heads can be allocated in a memcg different from the memcg of the page for which buffer_heads are being allocated. One concrete example is memory reclaim. The reclaim can trigger I/O of pages of any memcg on the system. So, the right way to charge buffer_head is to extract the memcg from the page for which buffer_heads are being allocated and then use targeted memcg charging API. Signed-off-by: Shakeel Butt Cc: Jan Kara Cc: Greg Thelen Cc: Michal Hocko Cc: Johannes Weiner Cc: Vladimir Davydov Cc: Alexander Viro Cc: Andrew Morton --- fs/buffer.c | 14 +++++++++++++- include/linux/memcontrol.h | 7 +++++++ mm/memcontrol.c | 21 +++++++++++++++++++++ 3 files changed, 41 insertions(+), 1 deletion(-) diff --git a/fs/buffer.c b/fs/buffer.c index 8194e3049fc5..26389b7a3cab 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -815,10 +815,17 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size, struct buffer_head *bh, *head; gfp_t gfp = GFP_NOFS; long offset; + struct mem_cgroup *old_memcg; + struct mem_cgroup *memcg = get_mem_cgroup_from_page(page); if (retry) gfp |= __GFP_NOFAIL; + if (memcg) { + gfp |= __GFP_ACCOUNT; + old_memcg = memalloc_memcg_save(memcg); + } + head = NULL; offset = PAGE_SIZE; while ((offset -= size) >= 0) { @@ -835,6 +842,11 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size, /* Link the buffer to its page */ set_bh_page(bh, page, offset); } +out: + if (memcg) { + memalloc_memcg_restore(old_memcg); +#ifdef CONFIG_MEMCG + css_put(&memcg->css); +#endif + } return head; /* * In case anything failed, we just free everything we got. @@ -848,7 +860,7 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size, } while (head); } - return NULL; + goto out; } EXPORT_SYMBOL_GPL(alloc_page_buffers); diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6c857be8a9b7..d53609978eb7 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -380,6 +380,8 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p); struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm); +struct mem_cgroup *get_mem_cgroup_from_page(struct page *page); + static inline struct mem_cgroup *mem_cgroup_from_css(struct cgroup_subsys_state *css){ return css ? container_of(css, struct mem_cgroup, css) : NULL; @@ -864,6 +866,11 @@ static inline struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm) return NULL; } +static inline struct mem_cgroup *get_mem_cgroup_from_page(struct page *page) +{ + return NULL; +} + static inline void mem_cgroup_put(struct mem_cgroup *memcg) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c481e661e051..f9a9a79117b9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -712,6 +712,27 @@ struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm) return memcg; } +/** + * get_mem_cgroup_from_page: Obtain a reference on given page's memcg. + * @page: page from which memcg should be extracted. + * + * Obtain a reference on page->memcg and returns it if successful. Otherwise + * NULL is returned. + */ +struct mem_cgroup *get_mem_cgroup_from_page(struct page *page) +{ + struct mem_cgroup *memcg = page->mem_cgroup; + + if (mem_cgroup_disabled() || !memcg) + return NULL; + + rcu_read_lock(); + if (!css_tryget_online(&memcg->css)) + memcg = NULL; + rcu_read_unlock(); + return memcg; +} + static __always_inline struct mem_cgroup *get_mem_cgroup( struct mem_cgroup *memcg, struct mm_struct *mm) {