From patchwork Fri Sep 6 00:10:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13793111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96052CE7AA5 for ; Fri, 6 Sep 2024 00:11:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 198086B008C; Thu, 5 Sep 2024 20:11:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 149126B0092; Thu, 5 Sep 2024 20:11:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F2BAD6B0093; Thu, 5 Sep 2024 20:11:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id D67136B008C for ; Thu, 5 Sep 2024 20:11:28 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5EC92A9F5D for ; Fri, 6 Sep 2024 00:11:28 +0000 (UTC) X-FDA: 82532384256.30.0925156 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf22.hostedemail.com (Postfix) with ESMTP id 8F1A7C0003 for ; Fri, 6 Sep 2024 00:11:25 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BBw2uoCs; spf=pass (imf22.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725581460; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ywsxeCwLk01Zemkk/XL229z2/fbHB1/7YpgelTeC42M=; b=zQs5jpR0dUXiuRuS0rV090xfUrDGqXbGJLN58wgL9lEpP4Ct1js0wF1QeYtx2WXFFWVJis QKJ5+wfMs9Xta+QsMfp70KrUojoCnvN5tV8Eqy2bpyB0tmifr1t6Qy/skzzsSAq1mlYCks w3I/aoxxXfV1bRinzPBC9+am415ZtVk= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BBw2uoCs; spf=pass (imf22.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725581460; a=rsa-sha256; cv=none; b=vrrBpq857TDSWUgvFApQmp+Moc9BARcXFC/Xb9jPA81TDsTDCON1Qm5H8X4x4VzOCnEtXH tkKWv6Zj/nQPdEAYRAL7mnb2Fa8kkPZPL+E7xdGaRTxxmw7UlhNeGenbMpnAPlXVaTMqqw RRmosd/C6Acr5e8kOV7C24oyAgcasUM= Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-718d6ad6050so162622b3a.0 for ; Thu, 05 Sep 2024 17:11:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725581484; x=1726186284; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ywsxeCwLk01Zemkk/XL229z2/fbHB1/7YpgelTeC42M=; b=BBw2uoCsIU6c6zGxExPqh69Dcco1r+VfFbLaeJdyLyFgCVeGBd1HOxLgcSaIleXzEl 8hNcHOP9/xfsJaDuWDgeU9rL5N4BOq4GNGm4cX5Y/yCw8AitQMYvM89hE4dj40t30z3g 4qxkLYsyeGp2mr3teO0670n7vF6yw7LUImhTYJWtgHdYl39BR8l1I6nnb3M6d3PEnTlu XxpXQf6aftNfcqtJZt1HmTJEaij0vnevD6oy47jvPWwlfwoABuT9xnpZcxGVwKvwj9wm rTCxVbyUQjtAyGYxFtAIVdppN6Lqm96zd8mWLifvr0Qr2qGTxxH9Qnto2mE9blw3KGL/ toQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725581484; x=1726186284; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ywsxeCwLk01Zemkk/XL229z2/fbHB1/7YpgelTeC42M=; b=gRPc+6lV5U+4dNWRJKxr9poSsw/jlGzOR/SycbaOe5jElV07ceosjG5idB1FxMrDbM 3ooW4pna2p93yGzv+wsuPqKNjTR1KBk+JIx1ksvOGvqVoTX918ssD2jxl+0HJjgsbrAo Xsz5RvdTQyBVHH//+vjhHNLzLm+bioGxXK93DRwIrIatiJT3YvM88z8gy54LX8JNZZ9x I/KG8WQ8dmb2H/lcNeN3U6mvQD7j5UtwhyoDjSDv2HoEmSmGBEXtY3dcvQFrkoOpY4+X O9qfemMMValoL9zPM06Uh/mLAcbHVU9qS2B9Sa38xST8IaWV6Sqk1Vu+hPU2dn+Lf6R8 6vkg== X-Forwarded-Encrypted: i=1; AJvYcCVZgkf8YEStpbnz0TcgT0bJPYQIwcjYZksAPgyBYn2puuTI8OlEloNsL+OKlrt0NTCy0eax2m2bSA==@kvack.org X-Gm-Message-State: AOJu0Yytmjw5jc2xksOgpBZ3MA0UI+6BBde4mjlMSDavqD255AcwJX79 Icuk3RPmre46Z6HrR2nu+VZI/Ezlychot4AcytZPJzz8BpIWepdAFRTaYQzl X-Google-Smtp-Source: AGHT+IFsT4gnjeGJ6BsOu0cQW9p32ygkcd0pzMLcI1W+NEiU9e52v6MkwZke3r/as0onruSbW4ToHQ== X-Received: by 2002:a05:6a20:cf83:b0:1ce:d418:a42c with SMTP id adf61e73a8af0-1cf1d0eff88mr747550637.19.1725581484279; Thu, 05 Sep 2024 17:11:24 -0700 (PDT) Received: from Barrys-MBP.hub ([2407:7000:8942:5500:cd42:f8ae:5222:5ab7]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206aea3548dsm33447445ad.140.2024.09.05.17.11.15 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 05 Sep 2024 17:11:23 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: hanchuanhua@oppo.com, baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hannes@cmpxchg.org, hch@infradead.org, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, mhocko@suse.com, minchan@kernel.org, nphamcs@gmail.com, ryan.roberts@arm.com, ryncsn@gmail.com, senozhatsky@chromium.org, shakeel.butt@linux.dev, shy828301@gmail.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yosryahmed@google.com Subject: [PATCH v8 2/3] mm: add nr argument in mem_cgroup_swapin_uncharge_swap() helper to support large folios Date: Fri, 6 Sep 2024 12:10:46 +1200 Message-Id: <20240906001047.1245-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240906001047.1245-1-21cnbao@gmail.com> References: <20240906001047.1245-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 8p5417u4kxa1n73oedncjxmr4x6yyu5x X-Rspamd-Queue-Id: 8F1A7C0003 X-Rspamd-Server: rspam11 X-HE-Tag: 1725581485-361244 X-HE-Meta: U2FsdGVkX19CuahZ+sreadBuk/ydCRnWX9Q8jpQW9M5w2Lhb5bsrg0Xm2dHVh847lMvRHQZE9WIMQXOMN7+tPEgJf6hENrJfaHv6nlQEcbrLb5lpo85VURZ2x5KycZWhcAYjWpRUs+6dU/P5/Fi+KBxmW5PPVMZdnRb9L1uBBCMUMZ8iQjLpAioN0ylZulO+AbWUEKluuFFnbsvaG4wbamyaJ7un3MpwYvEt/ZVaZYiw51AQh87FRTzTQq3hlTestKFzjANW7Oey9g32q+2ItdgtLWFGcuT1VmzrU/k/5dNEH5vxl4PbuOfmTSli7fVX0X6Ey/M+78myPzUaj8vrTGZAbUDZ2NcZxo+ReR6sidyiESaETn4hFfhTMEpEIzZkybUof4gwDo25zXB3fjhGlwW0KnWxKsfdwlEcVGJOS9l4DHY7rS52FAdZob6XnAli2SZ2WvFeSMO3kKCfeZ1YctWbuSoWg+yBvAER1gWGJbR0RLvEwPE3Yg1CdbxMMhfZAzz+KXIeUg6CwsCOvDIJ3LaAWGwMcY/iqC/taxdU1JeivLuzgyE19HzvQo0Am89EIWfJcpABLLqzf0HMSJrS51nExxJt1vbJBbqKMS5JtQ5kojQp/Ih9Ly6kSRfQNiIXIRnIS2RniyZVSdYjnMBSeLZLazQS/aEBD2+Gu0y1nIo3ckOlfPIyPT78VaUUDofff50v4wN8b7hB1IY3Q0Ujm/j0vv11+O3AqlVaKgR7E50xQze1Nt4Rd28/vDb0FrmHaUVJt+FRqe5EHo9lkNfTGMYOv9HkuL4OvVqmd30DwkSxSMzW9P/5++HwtHH4yiWnnNMKzD4IWsulcl9EeeiYha0w5foXGRmpGlPkeIPc19Q88virAgYBC6y0gg9FS+C0V4Rz/A0/rnYtmOR+4F3rUoIygdv0UNgqfRnRIfSpaL+trFGjzaC5v1gK7MyZJxwidDRYI+7lJDv9MVhBF9Y x3S2dABs FcjD/Dt2ysJuRCL578/TK6PdjTv3rnk2v1GVZ0vIy8awidleWaY9Y4lWGcmX8i5Gc+eEuNWWv49gU4J3z2wyaCPHv4F4hB/c1K724Lrh+fP8lVJNs0/fDC+SC8hQFejhH4qaezAAi835jMAcIhdND4GVSn/hP5SkD2QrxG/ScwNoS9kNeOKIDnQLqzZFz7FN0ZWJDrSFklJ4HSNG6iiQ9N7fNxgvzrHUU5QR/2ejZXF7Um2fo24+fhLSG47QBK2JR2Avay15AMrnFTlLP1E8KV2WLMg/z530nRYSydgcZGOzccTlOaavF1cEyWrLYB1RjJ5QsVITV5YBy+wxNXF3em33Oc/nG5f81vpqk3vO2n1J7TpXMR3t8f/I4uivU2d3H1PHkimRyoUzaqIjdNvgNAj5xsd1b7fDOzcLkSr1VraRiSLb7aK6pe4Jj8e0XneVv1diQTxq8Ymx4XixXueg7J99qLQvWVZ8x+ccj2fICYDnrRXjxK6QhgWMwWpA0TesPkjABSX15MS+wxH+XOaoPplRuKatwlvVtwVhirrEc5iRzrGggEH0kRNjYfR9eu2LRPMNJQp0IEKHGq5dozQNu9dY1Eh13iuEnRVUD7xGXlvb5rW0DtTMP8dS4+6eHPjxHrVxDxVGAjpyi6VAmPxoZ3gUrhXp8jRheL0Q+2KB28VA/WQCQquwwAe9bHKyfy735vfU5S5AfcaacB3GkE5hlucCl4d2fcv8vHt42XeQhArwCdTezgjx3CKjCS9E71LxSBJCalKfOkPy1bdYmSn6VSN7AVuXYRn66fe+omoEhPzZE8uVsflVgIU6hAz/x5YdtzfGdYk9wPeAB7fiYptU1ERUSI4BTUQP2vRD5PV1XxXO2qQyk+fFF4ZOoszsF5EDvQqg4//PjNiZlGXptIe+xKtxoJgWbhmr7KO9xMex1H+sQ2E95mpDInfpVkIySXIiRXlbstdc5QCv9n1HJeLlntJGn9S0w 4Po+oqQj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song With large folios swap-in, we might need to uncharge multiple entries all together, add nr argument in mem_cgroup_swapin_uncharge_swap(). For the existing two users, just pass nr=1. Signed-off-by: Barry Song Acked-by: Chris Li Cc: Shakeel Butt Cc: Baolin Wang Cc: Christoph Hellwig Cc: David Hildenbrand Cc: Gao Xiang Cc: "Huang, Ying" Cc: Hugh Dickins Cc: Johannes Weiner Cc: Kairui Song Cc: Kairui Song Cc: Kalesh Singh Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Minchan Kim Cc: Nhat Pham Cc: Ryan Roberts Cc: Sergey Senozhatsky Cc: Suren Baghdasaryan Cc: Yang Shi Cc: Yosry Ahmed Reviewed-by: Yosry Ahmed --- include/linux/memcontrol.h | 5 +++-- mm/memcontrol.c | 7 ++++--- mm/memory.c | 2 +- mm/swap_state.c | 2 +- 4 files changed, 9 insertions(+), 7 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 2ef94c74847d..34d2da05f2f1 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -699,7 +699,8 @@ int mem_cgroup_hugetlb_try_charge(struct mem_cgroup *memcg, gfp_t gfp, int mem_cgroup_swapin_charge_folio(struct folio *folio, struct mm_struct *mm, gfp_t gfp, swp_entry_t entry); -void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry); + +void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry, unsigned int nr_pages); void __mem_cgroup_uncharge(struct folio *folio); @@ -1206,7 +1207,7 @@ static inline int mem_cgroup_swapin_charge_folio(struct folio *folio, return 0; } -static inline void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry) +static inline void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry, unsigned int nr) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index bda6f75d22ff..c0d36ca20332 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4559,14 +4559,15 @@ int mem_cgroup_swapin_charge_folio(struct folio *folio, struct mm_struct *mm, /* * mem_cgroup_swapin_uncharge_swap - uncharge swap slot - * @entry: swap entry for which the page is charged + * @entry: the first swap entry for which the pages are charged + * @nr_pages: number of pages which will be uncharged * * Call this function after successfully adding the charged page to swapcache. * * Note: This function assumes the page for which swap slot is being uncharged * is order 0 page. */ -void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry) +void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry, unsigned int nr_pages) { /* * Cgroup1's unified memory+swap counter has been charged with the @@ -4586,7 +4587,7 @@ void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry) * let's not wait for it. The page already received a * memory+swap charge, drop the swap entry duplicate. */ - mem_cgroup_uncharge_swap(entry, 1); + mem_cgroup_uncharge_swap(entry, nr_pages); } } diff --git a/mm/memory.c b/mm/memory.c index 42674c0748cb..cdf03b39a92c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4100,7 +4100,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) ret = VM_FAULT_OOM; goto out_page; } - mem_cgroup_swapin_uncharge_swap(entry); + mem_cgroup_swapin_uncharge_swap(entry, 1); shadow = get_shadow_from_swap_cache(entry); if (shadow) diff --git a/mm/swap_state.c b/mm/swap_state.c index a042720554a7..4669f29cf555 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -522,7 +522,7 @@ struct folio *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, if (add_to_swap_cache(new_folio, entry, gfp_mask & GFP_RECLAIM_MASK, &shadow)) goto fail_unlock; - mem_cgroup_swapin_uncharge_swap(entry); + mem_cgroup_swapin_uncharge_swap(entry, 1); if (shadow) workingset_refault(new_folio, shadow);