From patchwork Sun Sep 8 23:21:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13795690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9C6FCD4F4C for ; Sun, 8 Sep 2024 23:21:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6B7B66B00EC; Sun, 8 Sep 2024 19:21:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 641536B00EE; Sun, 8 Sep 2024 19:21:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BA8C6B00EF; Sun, 8 Sep 2024 19:21:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2A3696B00EC for ; Sun, 8 Sep 2024 19:21:56 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E3AE940A4E for ; Sun, 8 Sep 2024 23:21:55 +0000 (UTC) X-FDA: 82543145790.02.81CE1DE Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf10.hostedemail.com (Postfix) with ESMTP id 08E15C0002 for ; Sun, 8 Sep 2024 23:21:52 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bMwt8RG7; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725837686; a=rsa-sha256; cv=none; b=RIIAQ1gvtHCYNhuhrWawa1BVZhAZH9WNCLUNmnlYIDtQ2mgTiuLaSCTAWWS/0ZdYuPxscm HxjmklFzCO+OFgU50kx9Wnor1v/KMeARR22aNen769ffqRJnzGWWxDNr2h9Jn7F9ySd/jT Up/TZEV5EMS1Q++ZkmvqS/2ZB3KiIxU= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bMwt8RG7; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725837686; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OlYXV3Fv3CqOaRAKUuC/QzTZ9pL0veLk2T3qfKeSJ+c=; b=L8DB35w1waBsvSoTWVmUBk92v7oI0+beaUJssW8w8q9wOI5Ok+ryJtUt67/mXXuvCMDc3h bLJZ6ZUlGe1UJFyHxYmrMkssjAVD9yTmhh4HpqBp/BVvDzPy4Qv8AqiMY+SkCxd/zEBNIJ sZtc6ndMs/TAAbkyAM1SauCZDyF6Z3o= Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-71798661a52so2109676b3a.0 for ; Sun, 08 Sep 2024 16:21:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725837712; x=1726442512; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OlYXV3Fv3CqOaRAKUuC/QzTZ9pL0veLk2T3qfKeSJ+c=; b=bMwt8RG7TODIQDo3z7YI8fiCkGLEb1zXKpiZiZRyRCJmYEdIOGhF92SvkRvF3hstnD G28NCYYjgnTV2P/t40d0GJPrrwoHLUO6R5gIpJeJPH5GkSEL75rdBOzvxEE+OTO7HQT8 /AWb9j9XHJxJqiXUi7WvboxQToJKiNO6I+oZxpj0oipeMSeeWMDOw+iH8gN11TVz/vKs K32fQh8JYvrikMbQjS64eaz/YwvaNxZVQw/JvqifzPvd+Finhsnx2xov/mzGQF/d1dSF 0D41covrFo1arU9ejYueO8eCdBmkgjIm58hJ5SBrspFSZekHKs5Htu/R3t5wA+NVMeac 2WfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725837712; x=1726442512; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OlYXV3Fv3CqOaRAKUuC/QzTZ9pL0veLk2T3qfKeSJ+c=; b=emRVdoV1nXlyELdvgC7vNq5Jq1Sao56DprL3doNoxzek3ehTXSICxqe+sfg1z+6Olu +GVx+97hoNkI7LBW48i5ZJV2aDQabizeq9qmXvVOMsUrV8M+73tv7Sy/wViCmCpvgD+C 61Ft3aEwmcg4t1rFTINEZ5GYEUSWWZBUGNk6dOc/GsASAlMnDOjj4MLcYoZ2jGOm765x O/lXioPrXtoB3SWHirdnf/AxUPcC7NXHnJLpTcfwfvVQURes+jliDERX2UROcRs/hdw+ skOvfY/YwGsznqgPVddX6/P4S00zOitxeYQiE/pbB7ENK/IDjIlHlcjNYl7dk112M9/O z5tQ== X-Forwarded-Encrypted: i=1; AJvYcCW1M1FgYaFFa5RFRqbIUheudw9bvRdO3ZWJZiJCc8vZa6Xi/BLJ0sE2FA2mwQhPGqlW4TsgoXmeKg==@kvack.org X-Gm-Message-State: AOJu0YyXJxIBVhCFTbRSI6xa7z63zLke59rIw2c5jAT+UjADMzURzO+z DLqpXq5KcCf8/wsrF2AxVQyqCBrMKCqsWMcoxBLGJgU3Dtb1ocZX X-Google-Smtp-Source: AGHT+IGL5o84rj87mdfLAxsPpLUvBk6PXR77RV9OsfYtgvZ14Unfm2nHOrZAzsQDfp012KISjhaCqw== X-Received: by 2002:a62:a503:0:b0:717:92d8:ca5c with SMTP id d2e1a72fcca58-71792d8cadamr17178636b3a.3.1725837711653; Sun, 08 Sep 2024 16:21:51 -0700 (PDT) Received: from Barrys-MBP.hub ([2407:7000:8942:5500:ecbd:b95f:f7b2:8158]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-718e58c703asm2447643b3a.82.2024.09.08.16.21.43 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 08 Sep 2024 16:21:51 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, hannes@cmpxchg.org, hch@infradead.org, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, mhocko@suse.com, minchan@kernel.org, nphamcs@gmail.com, ryan.roberts@arm.com, ryncsn@gmail.com, senozhatsky@chromium.org, shakeel.butt@linux.dev, shy828301@gmail.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yosryahmed@google.com Subject: [PATCH v9 2/3] mm: add nr argument in mem_cgroup_swapin_uncharge_swap() helper to support large folios Date: Mon, 9 Sep 2024 11:21:18 +1200 Message-Id: <20240908232119.2157-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240908232119.2157-1-21cnbao@gmail.com> References: <20240908232119.2157-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 08E15C0002 X-Rspamd-Server: rspam01 X-Stat-Signature: uxqo8sw9hpgeuy5xw7wuottzmopd5i1o X-HE-Tag: 1725837712-756069 X-HE-Meta: U2FsdGVkX1/wy2IyeAOlE7LENN8k/WKjDtIEaHuiDi8CBo1xSi+RCJVpIKTYrqpceDT1lnqLDyR4mzDtP9MGnrYcEaHJ3ixsQTL+Z9qFsXoHWmh1bdkXcKlYDDoNIf3WCTaV96BlCeL1zY1qx3KAsNqCSooZhAzhE0ZrD9QeeuNQBEL/6XMW7v+t/7tAoLo5SEYY9E5v7e4cdXWKpJEWUOTek+9MuCxOWAPX8IUArMbMPBSHkDLl5PYdXqRDNmrev/8y1CP77beha4flXHrIwN+uULdkyz4rIKTP081D6YW2HJxdTq8qamqKQzyUaqIK66cBgdBMyE8WuvHbxS2NTPimjrkEP3FXp9F4o82pZHa/g9OgOnnL2TDmeLdy5s6dvZ6num8pJoiKWg4KeG4p7zSjWhWtR/BJSkROkt4EVkn0XN4/Jk+S7WJ0rj2sAFDd4kmjLRM3g9Za3FJVzzRURj9pyQTYBCMKBFKe3cBH0SzIzsFUDKHSlBnx1pTbnduc1kwdOeH2MY5iTDPdwTZ5jXPAAs77vZDvFB/wdWUqYNSkZ+b4McBgcgaD3lvVIvxONxSkQFajzB7Es8Ypr9eayfRTf4nyDk6RVgEx/DMwdAJ2CZv/9NYzuO/Fbo7Vp/gGLuBQ4NkzL/iBiFG5vLTns+16RoQxxU+uyHrbInpR6noYupXEsbb010CNW0BcPvBHqWAMergh0BvkhpFDIBpmdIH/5/iowilKMI2qcK5rVDAEr9q6F9Iylm9lWzD0VWJ5huH9uryBIGn7R1OUAy6XZIOIgyQe1IeHxaYpf5z8G2a+XZVmR5ECY23OaOiI6Iq8p44vjZhBJd/hpatxix3/SSVeyU7fzwn+heCZ8UoMTM6feg9Mb233AyvoKuxxu8ECnIKx2baM6SBasaR+tWRhpSvrOSpbcG0hO/qH38vs2OzUXNl4+zKxYBi1uOz7jx6r15Ze9Sc4Y5+HSuSN0ag 8+fmGGAF n66t/ElhAp0QKp9W+UZwBORk3YFOrFgzPE5OFtfEOOgwQseETBj0H4vEXLXI8JiQ6FQS3pQdMiaMEfFqveZfzA0VLP3Dfrb69qGtB0T2qNXz8IgftQK2hOHB16ASFcK4aiHHsA3kbmwOMXwleWdUxqmqWFr4BXx5w6tVHSn4Y214gXKn8Ycvq3x4ZjCei017igpxurHBpf62AHRX05pNRoLNmEVSRzzb7vrrPD2+tPOwEylw+y/zyGwe1AAsiX0JqoX9gwh1rPxgkZafmDMEO8O/DlJSSv6+lYjSjN21I96zcYDeLF14XWekS6CLNZJl5N1vMQBD0wVT4aPGEs+YdhGAVFmDmFwrKRHMvHXe3zSQaRyVV12hAL0wYQtxI6182DvHlMGRJQaO3dLXwJ+/xxJnefxm0BaPQcqBuGCfIpauVdXJ2jxREqOVyfbT2GrtOb0FLVHIuUINShFczYsSbOFHYyFN8onZE4lmH6WY+wUVfvKb/4yGc1HJcB6uFZVMKcgTK8KEkDkeklJ8W40AjBjOeIIDhJMwNinoGWZAN8ljIlgMzkJ3Z2ipitvXJRwE4Jx+pe+3iR/q3d23w+OQGBHti+oXWuZceBEOC28yAjtupR6tLvXLTJZ7DYSp8epCaMGC5w6BoeD5q2a+qR4TOQAur6PHR9yV80yUcWLuF9crA6qo2s8++Q9D/N8OfwEaMCFxGI9bRwhIXunyqH/HwAePMyiqzqtxI8bqGPnRNfrtihMEpfxYehE8YSBbGNdBCO//P9CH+nLA4CT9kiVayEtBKd/CQ7bcwPc2kIn1rLhMxh33w4OYELV/WHZP/R2OenJXt+JEl9nPnlWuVToC1tb4KQHyWJRlS+gio X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song With large folios swap-in, we might need to uncharge multiple entries all together, add nr argument in mem_cgroup_swapin_uncharge_swap(). For the existing two users, just pass nr=1. Signed-off-by: Barry Song Acked-by: Chris Li Reviewed-by: Yosry Ahmed Cc: Shakeel Butt Cc: Baolin Wang Cc: Christoph Hellwig Cc: David Hildenbrand Cc: Gao Xiang Cc: "Huang, Ying" Cc: Hugh Dickins Cc: Johannes Weiner Cc: Kairui Song Cc: Kairui Song Cc: Kalesh Singh Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Minchan Kim Cc: Nhat Pham Cc: Ryan Roberts Cc: Sergey Senozhatsky Cc: Suren Baghdasaryan Cc: Yang Shi --- include/linux/memcontrol.h | 5 +++-- mm/memcontrol.c | 7 ++++--- mm/memory.c | 2 +- mm/swap_state.c | 2 +- 4 files changed, 9 insertions(+), 7 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 2ef94c74847d..34d2da05f2f1 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -699,7 +699,8 @@ int mem_cgroup_hugetlb_try_charge(struct mem_cgroup *memcg, gfp_t gfp, int mem_cgroup_swapin_charge_folio(struct folio *folio, struct mm_struct *mm, gfp_t gfp, swp_entry_t entry); -void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry); + +void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry, unsigned int nr_pages); void __mem_cgroup_uncharge(struct folio *folio); @@ -1206,7 +1207,7 @@ static inline int mem_cgroup_swapin_charge_folio(struct folio *folio, return 0; } -static inline void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry) +static inline void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry, unsigned int nr) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index bda6f75d22ff..c0d36ca20332 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4559,14 +4559,15 @@ int mem_cgroup_swapin_charge_folio(struct folio *folio, struct mm_struct *mm, /* * mem_cgroup_swapin_uncharge_swap - uncharge swap slot - * @entry: swap entry for which the page is charged + * @entry: the first swap entry for which the pages are charged + * @nr_pages: number of pages which will be uncharged * * Call this function after successfully adding the charged page to swapcache. * * Note: This function assumes the page for which swap slot is being uncharged * is order 0 page. */ -void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry) +void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry, unsigned int nr_pages) { /* * Cgroup1's unified memory+swap counter has been charged with the @@ -4586,7 +4587,7 @@ void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry) * let's not wait for it. The page already received a * memory+swap charge, drop the swap entry duplicate. */ - mem_cgroup_uncharge_swap(entry, 1); + mem_cgroup_uncharge_swap(entry, nr_pages); } } diff --git a/mm/memory.c b/mm/memory.c index 42674c0748cb..cdf03b39a92c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4100,7 +4100,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) ret = VM_FAULT_OOM; goto out_page; } - mem_cgroup_swapin_uncharge_swap(entry); + mem_cgroup_swapin_uncharge_swap(entry, 1); shadow = get_shadow_from_swap_cache(entry); if (shadow) diff --git a/mm/swap_state.c b/mm/swap_state.c index a042720554a7..4669f29cf555 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -522,7 +522,7 @@ struct folio *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, if (add_to_swap_cache(new_folio, entry, gfp_mask & GFP_RECLAIM_MASK, &shadow)) goto fail_unlock; - mem_cgroup_swapin_uncharge_swap(entry); + mem_cgroup_swapin_uncharge_swap(entry, 1); if (shadow) workingset_refault(new_folio, shadow);