From patchwork Mon Jun 25 23:06:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 10487531 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 15484601D5 for ; Mon, 25 Jun 2018 23:07:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E24BE286A8 for ; Mon, 25 Jun 2018 23:07:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D68D1286BD; Mon, 25 Jun 2018 23:07:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 58C27286A8 for ; Mon, 25 Jun 2018 23:07:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 69B6B6B000A; Mon, 25 Jun 2018 19:07:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6497D6B000C; Mon, 25 Jun 2018 19:07:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C34A6B000D; Mon, 25 Jun 2018 19:07:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf0-f197.google.com (mail-pf0-f197.google.com [209.85.192.197]) by kanga.kvack.org (Postfix) with ESMTP id 095AE6B000C for ; Mon, 25 Jun 2018 19:07:16 -0400 (EDT) Received: by mail-pf0-f197.google.com with SMTP id n19-v6so7696794pff.8 for ; Mon, 25 Jun 2018 16:07:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=5Y314oyiiNmyPZcaXu7Y9D2B9E/QFhwvNSMq6xgiZiY=; b=kCDrjwLdH8eywjzBZciuAxKMR3qKJpwp6VUIr0ki4ivPuHWO20rqX/jv/exKcV1AwM Dx7WsZvLBDY0ATJTNVKHppj6XrjFbRMk03XOIEitH7Rut6bzm7ze4It4QhkrvVB1wGiv Q8h4hWDn+MluDJm6rktIhj4tW9lQac3ylFqwgA0z1xPZnTVi/3FlKgWr0nmCHsSfIXf6 IsHa/3A5aBtOYOr7iZ5DNszYbOdZzAaYJASc1IFKIzEFFeEM5SiQJ2Ma58Lmz/N5PLf8 gO0v8G5WZ9KnBH9thXkwM1LSQa8fcHeFgYS++N3OzMefHqCGPg0SvXouBE+yoWKf6Ult vXEA== X-Gm-Message-State: APt69E1onKndoQ2kjpGq2DU6A6KTSwW9asNWpPBIANekZitc3s1185mI b4jaLacILh3V1rATrzCeJCIW+LF38pDjBvg0tY68bgKTnE+GgGg2abRdDeWjhqCrshXH4dOdMvX QRJHF32UrBfrMUEVBGNV0GcATgDyTOfs6SQv8aSYWMCEuGGA3Cio0cR9wogSleIQeUlepmkEW/Q 3r3YQLLLKyAoCd823ExAwC8fEWehVp3JSyaLw36H1p58nlVki8VdfalTQR+kt4boHDD6XSS31xl D6h27llHl58NTZsXKOAu4aptA6py5oOxxU+xkp7iUuObGghDPsapQp2Yux5B6geWiQRGToUQGZ1 lllHkTv+WzSLHwfrQ4+hj9BbyaKwFmfLttQeJ4nh7fOv6pq0djfYed9Jj8uwnfbQ+853zoxgcdx H X-Received: by 2002:a63:2b88:: with SMTP id r130-v6mr12000295pgr.170.1529968035683; Mon, 25 Jun 2018 16:07:15 -0700 (PDT) X-Received: by 2002:a63:2b88:: with SMTP id r130-v6mr12000265pgr.170.1529968034797; Mon, 25 Jun 2018 16:07:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529968034; cv=none; d=google.com; s=arc-20160816; b=bpGkxOteqggZpxhfzK/yoOnZy0okw7UuVAd9HewpQ0qTHApp2z/9xTAWZMPlsBV2av n/9xxPg/LuMz4Pwol4+2JE0U/o426Dq2RpgM3tdZ6a5n1oACOoMOMasx6Vcq8B1O+pHV fz6FWCz7FOujbOwIb51dg3EzQAVEPPyka2b6V2g27H1kyCjEk0L6+jB9YUA3+1qHkdC4 7mrB3OWpMFFMRCZhdZwxkYNfgGagHJ0kQ6s9tFNL/OVKonTu7opmSHHPhoYcPw6uZh5L j2L8AAc6Z4T2BTBOzxrYqZ7ft2IwPGVVa/Yw58F6dTGWKlpzHuNClL2C0ZB3X2w+wvZm h30g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=5Y314oyiiNmyPZcaXu7Y9D2B9E/QFhwvNSMq6xgiZiY=; b=H+F00y82+7VgsJOwRPvwEUAVDsPAw09xZFVc4GdI3tZD9QsPZJTgZ0wE1psxNMvBBP djznW5TZXKqCHuofkrBQi5G0/lwQBuq8Nl9uWc7PjAsIDnpC/Xs3uzxQG1bhFT+Ib6Zm dAuqzezOPc1gbdEKjLPxWCiR/ZW0FxmRJrp5+UoICHvi3QFpiSTvZd8uS7jh+B+krXYp RFDRrZTuzAhUZFxl3/V2z2Q2H8cLY/iZffVxabtFkw8wu6o4qE6kIuon0K8klsgKrIys RCeciTY4HSaWPKLHMqdVO3AYn6g7Ed7Wx500U8mDleflNdW1B1F6HNitzq3/FxRyu1AW wKUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=K60YuCwU; spf=pass (google.com: domain of shakeelb@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id p1-v6sor35664plq.135.2018.06.25.16.07.14 for (Google Transport Security); Mon, 25 Jun 2018 16:07:14 -0700 (PDT) Received-SPF: pass (google.com: domain of shakeelb@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=K60YuCwU; spf=pass (google.com: domain of shakeelb@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5Y314oyiiNmyPZcaXu7Y9D2B9E/QFhwvNSMq6xgiZiY=; b=K60YuCwU/i11Ov3zZBeP8KbVIGoBLTcB5GVbzuN0OFQFme3CvW46fFShogp9nxybid 6Y6F99FOv5zGYeOn17jA/iVSRLfzE8742Qe5OmnncR0l7vnalXoKyZJ6j1SiIW6yklHi G/j2lh5T//8kqzYANwaiCmCNEBp05RnQZa+gO139YBHMUwvJ4mEn57w/kpEw29jRxnE8 tKjE5Rx+kG8oam8ygnEk3fR4p1Kt173YVxPzxCJOOzR4Bh62vBta70YX6nPuHtgKq6bS asaQYoGDK3l8xr1Zcefq5/F7Jer+rFeNE+yB53W8s+bjXisY646Wl1G95/lh7buNDWut h8eg== X-Google-Smtp-Source: ADUXVKJ6x5RBWdPrEHu3E3oG025Jr4l9kM9NYOnGR92zF+w+CImnxWs2ng3qXMm3b+bhRWznvJ2phA== X-Received: by 2002:a17:902:6e09:: with SMTP id u9-v6mr472346plk.13.1529968034074; Mon, 25 Jun 2018 16:07:14 -0700 (PDT) Received: from shakeelb.mtv.corp.google.com ([2620:15c:2cb:201:3a5f:3a4f:fa44:6b63]) by smtp.gmail.com with ESMTPSA id p73-v6sm116657pfa.142.2018.06.25.16.07.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 25 Jun 2018 16:07:11 -0700 (PDT) From: Shakeel Butt To: Andrew Morton Cc: Michal Hocko , Johannes Weiner , Vladimir Davydov , Jan Kara , Greg Thelen , Amir Goldstein , Roman Gushchin , Alexander Viro , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Shakeel Butt Subject: [PATCH 2/2] fs, mm: account buffer_head to kmemcg Date: Mon, 25 Jun 2018 16:06:59 -0700 Message-Id: <20180625230659.139822-3-shakeelb@google.com> X-Mailer: git-send-email 2.18.0.rc2.346.g013aa6912e-goog In-Reply-To: <20180625230659.139822-1-shakeelb@google.com> References: <20180625230659.139822-1-shakeelb@google.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The buffer_head can consume a significant amount of system memory and is directly related to the amount of page cache. In our production environment we have observed that a lot of machines are spending a significant amount of memory as buffer_head and can not be left as system memory overhead. Charging buffer_head is not as simple as adding __GFP_ACCOUNT to the allocation. The buffer_heads can be allocated in a memcg different from the memcg of the page for which buffer_heads are being allocated. One concrete example is memory reclaim. The reclaim can trigger I/O of pages of any memcg on the system. So, the right way to charge buffer_head is to extract the memcg from the page for which buffer_heads are being allocated and then use targeted memcg charging API. Signed-off-by: Shakeel Butt --- Changelog since v1: - simple code cleanups fs/buffer.c | 15 ++++++++++++++- include/linux/memcontrol.h | 7 +++++++ mm/memcontrol.c | 22 ++++++++++++++++++++++ 3 files changed, 43 insertions(+), 1 deletion(-) diff --git a/fs/buffer.c b/fs/buffer.c index 8194e3049fc5..47682cc935fb 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -45,6 +45,7 @@ #include #include #include +#include #include static int fsync_buffers_list(spinlock_t *lock, struct list_head *list); @@ -815,10 +816,17 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size, struct buffer_head *bh, *head; gfp_t gfp = GFP_NOFS; long offset; + struct mem_cgroup *old_memcg = NULL, *memcg; if (retry) gfp |= __GFP_NOFAIL; + memcg = get_mem_cgroup_from_page(page); + if (memcg) { + gfp |= __GFP_ACCOUNT; + old_memcg = memalloc_use_memcg(memcg); + } + head = NULL; offset = PAGE_SIZE; while ((offset -= size) >= 0) { @@ -835,6 +843,11 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size, /* Link the buffer to its page */ set_bh_page(bh, page, offset); } +out: + if (memcg) { + memalloc_unuse_memcg(old_memcg); + mem_cgroup_put(memcg); + } return head; /* * In case anything failed, we just free everything we got. @@ -848,7 +861,7 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size, } while (head); } - return NULL; + goto out; } EXPORT_SYMBOL_GPL(alloc_page_buffers); diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6c857be8a9b7..d53609978eb7 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -380,6 +380,8 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p); struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm); +struct mem_cgroup *get_mem_cgroup_from_page(struct page *page); + static inline struct mem_cgroup *mem_cgroup_from_css(struct cgroup_subsys_state *css){ return css ? container_of(css, struct mem_cgroup, css) : NULL; @@ -864,6 +866,11 @@ static inline struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm) return NULL; } +static inline struct mem_cgroup *get_mem_cgroup_from_page(struct page *page) +{ + return NULL; +} + static inline void mem_cgroup_put(struct mem_cgroup *memcg) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 6b1a8f8e0a82..13d30b37935f 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -713,6 +713,28 @@ struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm) } EXPORT_SYMBOL(get_mem_cgroup_from_mm); +/** + * get_mem_cgroup_from_page: Obtain a reference on given page's memcg. + * @page: page from which memcg should be extracted. + * + * Obtain a reference on page->memcg and returns it if successful. Otherwise + * NULL is returned. + */ +struct mem_cgroup *get_mem_cgroup_from_page(struct page *page) +{ + struct mem_cgroup *memcg = page->mem_cgroup; + + if (mem_cgroup_disabled() || !memcg) + return NULL; + + rcu_read_lock(); + if (!css_tryget_online(&memcg->css)) + memcg = NULL; + rcu_read_unlock(); + return memcg; +} +EXPORT_SYMBOL(get_mem_cgroup_from_page); + /** * First try to obtain reference on current->active_memcg. On failure, try to * obtain reference on current->mm->memcg. On further failure root_mem_cgroup