From patchwork Mon Jan 23 19:17:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "T.J. Mercier" X-Patchwork-Id: 13112831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4667CC54EAA for ; Mon, 23 Jan 2023 19:18:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC9476B0078; Mon, 23 Jan 2023 14:18:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A78C06B007B; Mon, 23 Jan 2023 14:18:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 942F16B007D; Mon, 23 Jan 2023 14:18:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 83EDA6B0078 for ; Mon, 23 Jan 2023 14:18:00 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4B56783523 for ; Mon, 23 Jan 2023 19:18:00 +0000 (UTC) X-FDA: 80387023920.18.5BDE591 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf15.hostedemail.com (Postfix) with ESMTP id 7A473A0028 for ; Mon, 23 Jan 2023 19:17:57 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Subdt7mb; spf=pass (imf15.hostedemail.com: domain of 3ZN3OYwkKCH8wmphuflhujrrjoh.frpolqx0-ppnydfn.ruj@flex--tjmercier.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3ZN3OYwkKCH8wmphuflhujrrjoh.frpolqx0-ppnydfn.ruj@flex--tjmercier.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674501477; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=29eNHuXkN4Biw7YjDro7DKxD9MLgh2JPpCUKKyMlqpA=; b=reOHKxCywZSGDlzNnDsILdU/5WVslpq8+OxoH82v/A9dAnwRh/TBem+wmzqGUvrFJdcd52 XcqZT/zBRNwM5a88VYUi2CKmffUpq/VFEsxaI/+Zz5VcxjmNOsrtBGoE/RtleDujZeSih/ /NjklPLM9006EJWbxuYzcsfePtkAcHo= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Subdt7mb; spf=pass (imf15.hostedemail.com: domain of 3ZN3OYwkKCH8wmphuflhujrrjoh.frpolqx0-ppnydfn.ruj@flex--tjmercier.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3ZN3OYwkKCH8wmphuflhujrrjoh.frpolqx0-ppnydfn.ruj@flex--tjmercier.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674501477; a=rsa-sha256; cv=none; b=xoGQ7AWkUOIkWWMIQly3LtP72h2KAHAtj4qlrGVSZCFbZZSB39v6SEMegnCiHX/+Bg0rPS mPFEZWFsgIFMLSka50gEAw5EUdiyb+/EwOboQKBK3jB9dq3lOP3+XAQEAMQ2wTIJxLpJdR HWn4eWGkxkn17wZH1jJYwawv3YJw/PQ= Received: by mail-yb1-f201.google.com with SMTP id k15-20020a5b0a0f000000b007eba3f8e3baso13943900ybq.4 for ; Mon, 23 Jan 2023 11:17:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=29eNHuXkN4Biw7YjDro7DKxD9MLgh2JPpCUKKyMlqpA=; b=Subdt7mbpSJ3q++6bJrXZjJZZw2B5aXVhcYIOjpWp1e7w65W6r5oAUZpnmuhuuxtGC m6yHwVIfAM74kNHnxjivy2kMGAg6vCfv+3hhWBYw2BJN0Dja1Y7NQKOgWr6COa+8oUkk rUnUidJgv1sRL2Bt6YdHCPqVzpfizdrAy9YnM1stYfRMcfzP4DXyiUe0ObHRm1rO0ZdX Z1ZA/fI74arWwHwvDOYrrcmDZGCsMH3Nq+OQolvzt4CNc9tJFsaB0vn3Yw+H2aqetp8n bmppFH7vBRP4aW++HEHSq4uD8ir3GpzrY/G/loULfk/4Fe0e7jWIDY4IGgw0L9zKW6Fz 9YCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=29eNHuXkN4Biw7YjDro7DKxD9MLgh2JPpCUKKyMlqpA=; b=pZ5F9mT+dEWS2LPQ5uGPLynFst6VIFIyhyYRcb8jMIB+SnOHFYP3oPvgzlX0FhG6Az VUIaO82XmoGTk0f052vD6HjLmtXE7b06pcX3pPOsxbb1KLJOnYNDSWRuWK4tEkwx63UV 3ef7drNdhY0D3wHgrArOGYBbSZwhj4r1ZPKM+F+kfkfI+apMhFTLNVoo3SHhgQUi3JhN swOx5FHjNPq47sIXeGZY3N2PhzsrBN/nJEMOjr5V9nFFmwyYGe16KBkuiKM0uETvTJQX 7XgI98eAmhNTAJUJ31llri1bO3D8qopaGdaYWpIYcXu5hGo17CiWy0K0zWJ9OPPzKxd0 XXUQ== X-Gm-Message-State: AFqh2kp/ROxycHzeU7m5kY3w8P662Tprap4NtxxnBHtZKnGQ5ZCBDdUy rqVwiLsCy3KTHD9Cw52ax+VkJjQ2pJDekEc= X-Google-Smtp-Source: AMrXdXuIEmBCRRQ29mCX2Lz8F60RAV61DpR49m4PqdkPLl3Fou0sh8htd5KZF+eNK2HXmLDFr4FD/qKGK/2gVtE= X-Received: from tj.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:53a]) (user=tjmercier job=sendgmr) by 2002:a81:5553:0:b0:4d1:959d:fa49 with SMTP id j80-20020a815553000000b004d1959dfa49mr3602848ywb.33.1674501476646; Mon, 23 Jan 2023 11:17:56 -0800 (PST) Date: Mon, 23 Jan 2023 19:17:23 +0000 In-Reply-To: <20230123191728.2928839-1-tjmercier@google.com> Mime-Version: 1.0 References: <20230123191728.2928839-1-tjmercier@google.com> X-Mailer: git-send-email 2.39.0.246.g2a6d74b583-goog Message-ID: <20230123191728.2928839-2-tjmercier@google.com> Subject: [PATCH v2 1/4] memcg: Track exported dma-buffers From: "T.J. Mercier" To: tjmercier@google.com, Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Sumit Semwal , " =?utf-8?q?Christian_K=C3=B6nig?= " , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: daniel.vetter@ffwll.ch, android-mm@google.com, jstultz@google.com, jeffv@google.com, cmllamas@google.com, linux-security-module@vger.kernel.org, selinux@vger.kernel.org, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-mm@kvack.org X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 7A473A0028 X-Rspam-User: X-Stat-Signature: wggta1aoxizm9tx1yhsgmbq4hf3oxa6f X-HE-Tag: 1674501477-617579 X-HE-Meta: U2FsdGVkX1/twUPZ4sL0dakdkRDK/gvJcsS1ZMZERYmR7fJxlLuH7+JdH9WAfX3RuZ9VrRWSoReHNfIJIwH4nlQHk5H84VPei7WqCVBdrL5O8Jj8zoywUzdEUUL7NHja0FB3WekeHQxpOVZLFj0v+xDrqXuSAxDVfOvI32h4Rtk845AAJx42ZE3LunHg2LwHwd8JPJl5X1i5Sl+KPxP9M4u4ocZlHHt635OI/TiniLT5w56ONoXbtJvenb7J76hV5oRKB8WUDnOljz7sVf7gBRqmF74y8D1AtCA7HSFKBxD8T5HxuL6scdu4Shi6x84lx78F1ZflewuGhnHQRGhOhfs0AtySlHpqA4g0iH5C10pBi5XPaJHFJ1PVPMnLFpEq0WvVs3oThgccnuIeSmR6wWWjmRekpf7JqKpFKevtgsyMZRBY1wl7Yrd7OuGldC4iQO+1nEmlUQIrphNyqhDs5idMlqRTEewyEJJghqb/LR2sGgHQv75j9bcGfFwPYWlLF7m9/zR3WAMp+ZAbahC4Qqkadj2Cs/+jHcn29KmRl2V87eKQzjQd24VgXly5P86lgawq3i3Famg1SapMBamW5aeDS1E28IjfDNlS3rqvJyLdPTHl9hiS1eXQihS8xS752gbPUWInhJGkkFDc1akId5LhJNIjonllrDsOEd6EVrZtaheUJx5yltvYaEB5ZvZgwvRS+vKy8ONSFSq1i2k9OoP5sOl3TFMaAZP/KQw5wa0c1XKfbqMZ8SkcmFpssXXT8QOoT9bDjnPr9NMYmf1XYh47KSZwGZ8SoFFZrMH6c5/5gJfIiXO5Z24GWWuMlBCddMLvKsc6xHu4YgZLgymecY/vFl0Mu4g1OmUrdQ6Gj7Afvqxu86yamNECLocVu4s9vCUyey7OEfaA7PaOQplo6ZVL8toksQIfQ6aiPjb8LHLk6LMNJgxXgqh/1Yd+wuxOOFgYaBOOlcspNRguQRC H/fynY1t pF7AkClrFVvGwva6OD1wUixnQTNb8+k5GZOdg9PtdxIcE397Hcyt+bjHsVOVGjMAqMXHFfRztXqZCdznace4OGlZZzAW7ZX0V92DqTXvau0tynaxh3dNnER1rzpV5vpgmD8jucAgyNuGMzps/Y3UwOvWr+KU4CdqQpCyXwOpcR5UE4WZ1Uyhu/eZC1wfVrzjNpGuW78FsPBVCl8qUlXkPaxufYp13aQc91Xxaooc3Rrsi782E5TCQzuBHV2bQgJdTKuzGzpT4IZkKrmSFJtSzR7hnDCnzSVptfXlecsJv0uAjcHWOTeewjdrGIAzekcUZyDQnyZ2ythp6ZStmkbTen+UAgPJjWeJwfTEXTeRjD0AB12pShXeu6SPEReSWKB6XFaZL1537LAPvS0X44FtU7Pabi8ztGRPcOjn8gH8i5xiPgHn2cu5jF/x4oJ+8PwSLVbFcFETOGNVyV4eSSUrsDJS75jDGI71EKMnUyXGrVG2sUlanfU6sLC6AdWiycy3VrP2IvPbBgNct3vdZat6mZKBz/R74NqbwfxTvd0Ij4fxur58Spdv+1dWaUCs+9TRU+GPbJeUAJep+XpINW8Qi8xU9SL1kqUJ7yM8vmn272vBj6a5iRkd2OqCmVVYb/giOITnT/H/j0SRD0IsgH0bjfnpywA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When a buffer is exported to userspace, use memcg to attribute the buffer to the allocating cgroup until all buffer references are released. Unlike the dmabuf sysfs stats implementation, this memcg accounting avoids contention over the kernfs_rwsem incurred when creating or removing nodes. Signed-off-by: T.J. Mercier --- Documentation/admin-guide/cgroup-v2.rst | 4 +++ drivers/dma-buf/dma-buf.c | 13 +++++++++ include/linux/dma-buf.h | 3 ++ include/linux/memcontrol.h | 38 +++++++++++++++++++++++++ mm/memcontrol.c | 19 +++++++++++++ 5 files changed, 77 insertions(+) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index c8ae7c897f14..538ae22bc514 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1455,6 +1455,10 @@ PAGE_SIZE multiple when read back. Amount of memory used for storing in-kernel data structures. + dmabuf (npn) + Amount of memory used for exported DMA buffers allocated by the cgroup. + Stays with the allocating cgroup regardless of how the buffer is shared. + workingset_refault_anon Number of refaults of previously evicted anonymous pages. diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index e6528767efc7..a6a8cb5cb32d 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -75,6 +75,9 @@ static void dma_buf_release(struct dentry *dentry) */ BUG_ON(dmabuf->cb_in.active || dmabuf->cb_out.active); + mem_cgroup_uncharge_dmabuf(dmabuf->memcg, PAGE_ALIGN(dmabuf->size) / PAGE_SIZE); + mem_cgroup_put(dmabuf->memcg); + dma_buf_stats_teardown(dmabuf); dmabuf->ops->release(dmabuf); @@ -673,6 +676,13 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) if (ret) goto err_dmabuf; + dmabuf->memcg = get_mem_cgroup_from_mm(current->mm); + if (!mem_cgroup_charge_dmabuf(dmabuf->memcg, PAGE_ALIGN(dmabuf->size) / PAGE_SIZE, + GFP_KERNEL)) { + ret = -ENOMEM; + goto err_memcg; + } + file->private_data = dmabuf; file->f_path.dentry->d_fsdata = dmabuf; dmabuf->file = file; @@ -683,6 +693,9 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) return dmabuf; +err_memcg: + mem_cgroup_put(dmabuf->memcg); + dma_buf_stats_teardown(dmabuf); err_dmabuf: if (!resv) dma_resv_fini(dmabuf->resv); diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 6fa8d4e29719..1f0ffb8e4bf5 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -22,6 +22,7 @@ #include #include #include +#include struct device; struct dma_buf; @@ -446,6 +447,8 @@ struct dma_buf { struct dma_buf *dmabuf; } *sysfs_entry; #endif + /* The cgroup to which this buffer is currently attributed */ + struct mem_cgroup *memcg; }; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index d3c8203cab6c..c10b8565fdbf 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -37,6 +37,7 @@ enum memcg_stat_item { MEMCG_KMEM, MEMCG_ZSWAP_B, MEMCG_ZSWAPPED, + MEMCG_DMABUF, MEMCG_NR_STAT, }; @@ -673,6 +674,25 @@ static inline int mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, int mem_cgroup_swapin_charge_folio(struct folio *folio, struct mm_struct *mm, gfp_t gfp, swp_entry_t entry); + +/** + * mem_cgroup_charge_dmabuf - Charge dma-buf memory to a cgroup and update stat counter + * @memcg: memcg to charge + * @nr_pages: number of pages to charge + * @gfp_mask: reclaim mode + * + * Charges @nr_pages to @memcg. Returns %true if the charge fit within + * @memcg's configured limit, %false if it doesn't. + */ +bool __mem_cgroup_charge_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages, gfp_t gfp_mask); +static inline bool mem_cgroup_charge_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages, + gfp_t gfp_mask) +{ + if (mem_cgroup_disabled()) + return 0; + return __mem_cgroup_charge_dmabuf(memcg, nr_pages, gfp_mask); +} + void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry); void __mem_cgroup_uncharge(struct folio *folio); @@ -690,6 +710,14 @@ static inline void mem_cgroup_uncharge(struct folio *folio) __mem_cgroup_uncharge(folio); } +void __mem_cgroup_uncharge_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages); +static inline void mem_cgroup_uncharge_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) +{ + if (mem_cgroup_disabled()) + return; + __mem_cgroup_uncharge_dmabuf(memcg, nr_pages); +} + void __mem_cgroup_uncharge_list(struct list_head *page_list); static inline void mem_cgroup_uncharge_list(struct list_head *page_list) { @@ -1242,6 +1270,12 @@ static inline int mem_cgroup_swapin_charge_folio(struct folio *folio, return 0; } +static inline bool mem_cgroup_charge_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages, + gfp_t gfp_mask) +{ + return true; +} + static inline void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry) { } @@ -1250,6 +1284,10 @@ static inline void mem_cgroup_uncharge(struct folio *folio) { } +static inline void mem_cgroup_uncharge_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) +{ +} + static inline void mem_cgroup_uncharge_list(struct list_head *page_list) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ab457f0394ab..375d18370f4b 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1502,6 +1502,7 @@ static const struct memory_stat memory_stats[] = { { "unevictable", NR_UNEVICTABLE }, { "slab_reclaimable", NR_SLAB_RECLAIMABLE_B }, { "slab_unreclaimable", NR_SLAB_UNRECLAIMABLE_B }, + { "dmabuf", MEMCG_DMABUF }, /* The memory events */ { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, @@ -4042,6 +4043,7 @@ static const unsigned int memcg1_stats[] = { WORKINGSET_REFAULT_ANON, WORKINGSET_REFAULT_FILE, MEMCG_SWAP, + MEMCG_DMABUF, }; static const char *const memcg1_stat_names[] = { @@ -4057,6 +4059,7 @@ static const char *const memcg1_stat_names[] = { "workingset_refault_anon", "workingset_refault_file", "swap", + "dmabuf", }; /* Universal VM events cgroup1 shows, original sort order */ @@ -7299,6 +7302,22 @@ void mem_cgroup_uncharge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +bool __mem_cgroup_charge_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages, gfp_t gfp_mask) +{ + if (try_charge(memcg, gfp_mask, nr_pages) == 0) { + mod_memcg_state(memcg, MEMCG_DMABUF, nr_pages); + return true; + } + + return false; +} + +void __mem_cgroup_uncharge_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) +{ + mod_memcg_state(memcg, MEMCG_DMABUF, -nr_pages); + refill_stock(memcg, nr_pages); +} + static int __init cgroup_memory(char *s) { char *token;