From patchwork Mon Aug 12 07:42:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13760267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A5F6C3DA7F for ; Mon, 12 Aug 2024 07:42:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C4C5D6B009E; Mon, 12 Aug 2024 03:42:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BFAFE6B00A0; Mon, 12 Aug 2024 03:42:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE91E6B00A1; Mon, 12 Aug 2024 03:42:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 90F376B009E for ; Mon, 12 Aug 2024 03:42:27 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 348C9A0940 for ; Mon, 12 Aug 2024 07:42:27 +0000 (UTC) X-FDA: 82442800734.27.7947E06 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by imf14.hostedemail.com (Postfix) with ESMTP id F1734100013 for ; Mon, 12 Aug 2024 07:42:22 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Oq4bQ0S4; spf=pass (imf14.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723448534; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o6+ZLJUP4gIVFn0Lkl5fLomnFxVaP80TbLuhIXh/Zp0=; b=UPdLVn8Gacef360n5lHA+4BkaxRW2WfPl4KtvgpBjmSrGNGDx4eRtb/dMxi8r2S5cC5lWM ojLzTqKXBge7FRkow9YWjKcDMSGH8R+8qdwUQwfvPeHd+HJmxxtfsDOUWDhifcn16Cf3Vs g00t0nzxUJkSY8lMPUBnJGQ0K1n3bdo= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Oq4bQ0S4; spf=pass (imf14.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723448534; a=rsa-sha256; cv=none; b=O6k4fWTqcTZIqJutAhHqXb5yogg3o2LMIt7yztnQOpOT+28zJaRBtv5VsGcBI97Nbs/mGS dy/sG++ZVD6WtP19UXa7xpT5mlfRVuDmMxOAskOO1Q3pMsjV+rElf3wfO7RAK450V1DdZs s5tVLvx4hnCIAQrVAMZ+8itu0P//GUw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1723448539; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=o6+ZLJUP4gIVFn0Lkl5fLomnFxVaP80TbLuhIXh/Zp0=; b=Oq4bQ0S4sRYKmxobP1G6SuQvWw8Y7VgYAIgZ2GPgBnC1zii6a+gZLYlgssvj0VvpLBUHjC/0uvOMIRjFLt5GMDuMzqgEPf5/F5gxjJmL3gGXGx2oj8MC3DHPvNwQCLS1CVq7D5CGuHtaXZsBjuE+8qwdEcMi3f842OriSxb3lG0= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WCbAKO1_1723448538) by smtp.aliyun-inc.com; Mon, 12 Aug 2024 15:42:18 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 1/9] mm: swap: extend swap_shmem_alloc() to support batch SWAP_MAP_SHMEM flag setting Date: Mon, 12 Aug 2024 15:42:02 +0800 Message-Id: <99f64115d04b285e009580eb177352c57119ffd0.1723434324.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: yfphbwgg8jta6psay85fazx5booba7in X-Rspamd-Queue-Id: F1734100013 X-Rspamd-Server: rspam11 X-HE-Tag: 1723448542-582450 X-HE-Meta: U2FsdGVkX18PoreRoolrJ3oXlKl/IBk1Kz5z3Vth0wWpDCtt/3lTLOtrenpZ56DAXOdCZXs0/cvTUCx63rj2/ZojIeO0U/MWacQaj7K9sYXcFssmhE8yUFa119gIo0PIZYzGegqzlEEMH3BsLWvrChwJ87il40tx+iHL/9vceq8OoTFxe+YF668UGvLhPLowMnv1ohp1WkJrdPlNCeNhL74RAthxuD3dSnA2sW7HkaJgByZDsgcpRm/Vyy2kpn87NddzqpHrDxhIyR+KERK8xsOdlBiHNtCL84Rvb3H9tvDG00TaI4oD0r/vxMH9juTOtKJj2pQgP/dhKTetmDYCml8Ozkxd2neDjPqFiyMo6/QKGdwNvyEdOEL9hituGfp9wzNoBuOc/DyJWmEyZG28GIUyYqKcl3K3gVQ05OciKoJs8d+mtCii95v2kuy0GG9hQhE6uRWXR3tQeLfCJuPLFwt9/6EZ6/fSLdBoAflewcpL47bFMv4VtjGx2INSKmi9uA+JBv4TLxBz+Q5yBCqdtiLFnZuDJ1EhTZwcvE9XMOyBvzUbadCC5B0PhqK7OBH84c6QHvzeiFPiw/r1ZMUMYeOLDUq4t03gtOOxoU4efftarpEFhCSC6LUyXIr1l3anzuhOgajpQx0ZSbaEeF9idD3akwsufhEk39abzJ7VLvklOi5fc571NmhamU99iE/8+GJ5Jg1Z4Dxad7dqwvzH9DgMmGCcBpJMr+JpohBEOyQQIlvRDThb1o/pdYYpULq7p87SdZZByJF59RoxnJFQsw/sihB6wBoBlkoc503OwH8zJTTmizDMpo20Gw4S/vwPP8a9Z82Ue1RoBcBMbZWZKz6hrxKLs+ir4G8jZgufc+35vsJPQ0VjsyfCH+OF5pkrMNt5NAKyv2OAJlAfqJ+A2sWC0SeRlTLbiJ50zOhV0aW8x9hYO1TWVIxWzIUrfMWwEzsORZBZ/S2TyI1qWex V6iWSNvm YE8GOUV/zuUqbnL3TMQbrHwLgtuqzTyxtpuIg+oDQz03l/Z7c6lzBn4U7tERPJ9H2/t09uYR7KQTl+QOVxzw/kL5Yx9WOxaIj1360/5SI47JjxZ9nYc9yvvH8h2+/QwMVe4ShpdZYHQPZk5wF4AjZzZvV2Oi3222DXOEO5RM+wI4bBlkk5nb4lDX2mLhxBRshDlpJp7bs7z9np8OVaKBOT8MHx7perumhITezvzASmKhkXt70d/tDSp7LfgDdkuLHjGGfHZb1bWcShHZ88jDNnHNNtCVKkTeldTCwUU3/Q/A77m+/Px7gCH38An6hfwjLZiCx1L0HyFTKP/jpKv3hDjsSQg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support shmem large folio swap operations, add a new parameter to swap_shmem_alloc() that allows batch SWAP_MAP_SHMEM flag setting for shmem swap entries. While we are at it, using folio_nr_pages() to get the number of pages of the folio as a preparation. Signed-off-by: Baolin Wang Reviewed-by: Barry Song --- include/linux/swap.h | 4 ++-- mm/shmem.c | 6 ++++-- mm/swapfile.c | 4 ++-- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 1c8f844a9f0f..248db1dd7812 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -481,7 +481,7 @@ void put_swap_folio(struct folio *folio, swp_entry_t entry); extern swp_entry_t get_swap_page_of_type(int); extern int get_swap_pages(int n, swp_entry_t swp_entries[], int order); extern int add_swap_count_continuation(swp_entry_t, gfp_t); -extern void swap_shmem_alloc(swp_entry_t); +extern void swap_shmem_alloc(swp_entry_t, int); extern int swap_duplicate(swp_entry_t); extern int swapcache_prepare(swp_entry_t entry, int nr); extern void swap_free_nr(swp_entry_t entry, int nr_pages); @@ -548,7 +548,7 @@ static inline int add_swap_count_continuation(swp_entry_t swp, gfp_t gfp_mask) return 0; } -static inline void swap_shmem_alloc(swp_entry_t swp) +static inline void swap_shmem_alloc(swp_entry_t swp, int nr) { } diff --git a/mm/shmem.c b/mm/shmem.c index 4a5254bfd610..22cdc10f27ea 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1452,6 +1452,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb); swp_entry_t swap; pgoff_t index; + int nr_pages; /* * Our capabilities prevent regular writeback or sync from ever calling @@ -1484,6 +1485,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) } index = folio->index; + nr_pages = folio_nr_pages(folio); /* * This is somewhat ridiculous, but without plumbing a SWAP_MAP_FALLOC @@ -1536,8 +1538,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) if (add_to_swap_cache(folio, swap, __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN, NULL) == 0) { - shmem_recalc_inode(inode, 0, 1); - swap_shmem_alloc(swap); + shmem_recalc_inode(inode, 0, nr_pages); + swap_shmem_alloc(swap, nr_pages); shmem_delete_from_page_cache(folio, swp_to_radix_entry(swap)); mutex_unlock(&shmem_swaplist_mutex); diff --git a/mm/swapfile.c b/mm/swapfile.c index 6de12d712c7e..1caeee676696 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -3604,9 +3604,9 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage, int nr) * Help swapoff by noting that swap entry belongs to shmem/tmpfs * (in which case its reference count is never incremented). */ -void swap_shmem_alloc(swp_entry_t entry) +void swap_shmem_alloc(swp_entry_t entry, int nr) { - __swap_duplicate(entry, SWAP_MAP_SHMEM, 1); + __swap_duplicate(entry, SWAP_MAP_SHMEM, nr); } /* From patchwork Mon Aug 12 07:42:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13760270 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E655FC3DA7F for ; Mon, 12 Aug 2024 07:42:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E3566B00A4; Mon, 12 Aug 2024 03:42:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 56BB66B00A2; Mon, 12 Aug 2024 03:42:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3948E6B00A5; Mon, 12 Aug 2024 03:42:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 040726B00A2 for ; Mon, 12 Aug 2024 03:42:28 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AFA96A1BD6 for ; Mon, 12 Aug 2024 07:42:28 +0000 (UTC) X-FDA: 82442800776.12.D489D98 Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) by imf29.hostedemail.com (Postfix) with ESMTP id 95162120025 for ; Mon, 12 Aug 2024 07:42:26 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=uJE1VEiW; spf=pass (imf29.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.124 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723448477; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2cIcVzSoLVgPRYlybE3t8RKibMfrcxZj8JgqApi1OSo=; b=jbhBSlCPwYJhnTO2fJs7zm7+XCeMMGFffSDxoO3/x8TTIY2sHataZVxFjm4MesdQz1wNHK jKT0H45RYm4VUPwCSiLxsroYQujaMlXMU9TJqdIAr7FXkwa7HSntmWgOrKtcfME/Dstlin cQeLp3WxSElYl0yo2+9zbQzkBFI3rIM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723448477; a=rsa-sha256; cv=none; b=5BJ3hnOsgGatN/HDo78XNWpb1d4o7R6hHI8JRHaImUZJH4QeLWoqAzYh0ovbC0EX7gGeJn nXDJDe/cb+upjSDC250tyZs2M1TC78uYVKgsp1gW9xmHjowL+EReitH0ghvRwzQ5D0Jtkt V+mnE/pphGJ5JdUvEBUcVQjtwXfrvw8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=uJE1VEiW; spf=pass (imf29.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.124 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1723448542; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=2cIcVzSoLVgPRYlybE3t8RKibMfrcxZj8JgqApi1OSo=; b=uJE1VEiWOPfMo9lqgitzw/Rv0cBMkRqrhdPbbiH+ks7VCoMITHmKKgGzpUK0PndssJsJbGpQNVWalpYTu3GN52Qnlr3XpGUhtbP1gHhMdwlkpFfe328GrLMmXsMrmQKbMgKIbVAbyTxpV4VfX9QlaOsTyeRuVYkmegBi6Isudw0= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WCazOhr_1723448539) by smtp.aliyun-inc.com; Mon, 12 Aug 2024 15:42:19 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 2/9] mm: shmem: extend shmem_partial_swap_usage() to support large folio swap Date: Mon, 12 Aug 2024 15:42:03 +0800 Message-Id: <60b130b9fc3e422bb91293a172c2113c85e9233a.1723434324.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: cai7h7z4o376ke3g77k4wi39adigkyjt X-Rspamd-Queue-Id: 95162120025 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1723448546-402343 X-HE-Meta: U2FsdGVkX19EZI8Arb/ZN8zJlMx0Iauj1THlJwcAIGm4a9hUnV8BAsu2OWPp4YaSde0l4Ijmm4wuW5rlQ730FoCmk4fr0JhZ8/RjTjTGWntIrqXmhaZgwg/37VRIVF402g0zJCjyo4BVsFKu59mj4DKGO9doCeR/ahOqnVyXcXX30urmMZKRIwwyk2j9srjc5j+NwHZWS1y9HkNaX0Xca54uuYJY7PlYsGKR6GMczUe2KFUtcbdxbt4+6itDSICkLU3q9GtXJM914eO+/VnbgiKyUopLlmfPvIVvklzlhtVS+sUAfhSIlM/1SNodiK6f6+ahBJQO5y8zFIDByUK8aOpzqZSox+LeNVduZGkkyz0liUAsBCyME0qM0cAXxR0fz10tvhy6Yj6d2WKu2s7zWCrg3IsoMoxjNKRKVUsi60V2UzC+SvIkmcO4p7rHYF+CutV4LXJLzcM+pdSY6b18IVi620Uk6T+8+mdiQ0aQ+i3HF6FXQ7JhoGI1iZO/GwQqM0H2zXRDMnjZOl41Mgb6oeDLgqFXWiY1NkzHbunOs0ZYjfG8OnpH9cXyUFYnhL/RTxvkJlvcztC00owGnACw88WjM6VZWlx51BTUv/tITdu01fkEF0P0nhDbV7PdzqHCVQ65G+uvmQXWCThcnLRU1UhffoiAxfNuBrYyLsfggf5Yg/asaqHXnl/o2fuHPXssjvjaVhEc7B8IvTMB3OGca8wXf65volR29QpNiKDu1GL0fEl/RqKSSrwHjpj0AYaTW19aAC9crnqa4xXMMJCmaikgVHmId0mu6MMNi9BPVmzxKW1aewo81O8VaXg7zUYiDnfS5NsGa4OV3QpDJEnDeBUvrMcHvMINMu9zk1WMir3fw/L4OZQRqw6KJ98e3256O8PQFC1nQjmDU5mkFpa/5Zwho4OGXtaDPK/LSQufx+7Nsq8f6a2dBmb9dAq9JGsPI1oOrf3CSuy3MJegV4U 9qbfdH/8 Z6nSEmnM+KFubGEpCcm956jp/YnIVyaSAiOsXCv4Bjf98BnPUXCXYY/lwsL2mJgzoWgz2zBj1wwWmTV161cXgG0vF4oFtCrSOMuPun+23IzIteFDvdkmMnZ50d4vSKzOsxOc+mI7/l2LBKDa6Qpa+b+HPEBm2MblDnQfGmswsVkAuSdQONyreJC+ZGl9MroLPftswH222W1S+DlNLa7y09hFoEgiLBkgQBTxx1QpoJMQAgVCZcazfJ7d1gg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support shmem large folio swapout in the following patches, using xa_get_order() to get the order of the swap entry to calculate the swap usage of shmem. Signed-off-by: Baolin Wang --- mm/shmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index 22cdc10f27ea..02fb188d627f 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -890,7 +890,7 @@ unsigned long shmem_partial_swap_usage(struct address_space *mapping, if (xas_retry(&xas, page)) continue; if (xa_is_value(page)) - swapped++; + swapped += 1 << xa_get_order(xas.xa, xas.xa_index); if (xas.xa_index == max) break; if (need_resched()) { From patchwork Mon Aug 12 07:42:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13760268 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B056AC531DA for ; Mon, 12 Aug 2024 07:42:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 71C6A6B00A0; Mon, 12 Aug 2024 03:42:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6CB786B00A2; Mon, 12 Aug 2024 03:42:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 592806B00A3; Mon, 12 Aug 2024 03:42:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 390B36B00A0 for ; Mon, 12 Aug 2024 03:42:28 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 87F2116127D for ; Mon, 12 Aug 2024 07:42:27 +0000 (UTC) X-FDA: 82442800734.04.A831DDD Received: from out30-111.freemail.mail.aliyun.com (out30-111.freemail.mail.aliyun.com [115.124.30.111]) by imf06.hostedemail.com (Postfix) with ESMTP id 6B1D918000F for ; Mon, 12 Aug 2024 07:42:25 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=tU8ArFSW; spf=pass (imf06.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.111 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723448511; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=71u6PF0T766CyIr04AjwEpJvSI8v+1Ntw3LK3owxVuQ=; b=QETa1ru/JqroSTflm5FS7CDTDaRMqw0VvSiHYy5jbRsi9TnrrPrzY58kIFF2cVPxdA6zZ1 AxZEgOQ8no3JHLvW60avZdHXk3ANnBr3QATXjHTrdEIEZnVmJJm5Evm+m1tzPojifwP5fS s5vcwP3M/QOjUqfzdJSlV1JLksH+aew= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=tU8ArFSW; spf=pass (imf06.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.111 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723448511; a=rsa-sha256; cv=none; b=VaNINW6sYye4DPnSkdDFGNrJwSBjyVeQkkQTCBkph0+xs4tgNLjw/rThgwLvwbsdS+y648 hrxrI+8AilsuZcB7Ol8PkSaYvLhCoyXvjGlVf/eiFJNBFh4mA7oYP7VMoT4pnQ7StgzF8p 5GDQRYEWT/PO/29jheN+Hk9Mq6VtrAA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1723448542; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=71u6PF0T766CyIr04AjwEpJvSI8v+1Ntw3LK3owxVuQ=; b=tU8ArFSWaw0R/n1rHBU30YyW2KKq4GhtVbV10IOgZ6nenZ0Q4CYRtGJpd/VYSErwu6tVLLA2Vp7fuCq0zFgwAWp5iOE0xYl/fkB2WB92SYf9Mg4VzpRd41rmQU8+6PyZMzm2dk+jllh5GViOiIWE2FlT1+m4yrJx6qTF/bsq8e8= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WCbAKOu_1723448540) by smtp.aliyun-inc.com; Mon, 12 Aug 2024 15:42:21 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 3/9] mm: shmem: return number of pages beeing freed in shmem_free_swap Date: Mon, 12 Aug 2024 15:42:04 +0800 Message-Id: <9623e863c83d749d5ab407f6fdf0a8e5a3bdf052.1723434324.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 6B1D918000F X-Stat-Signature: 8onjpfuntbmudrnutuk69r76i3g4eftg X-HE-Tag: 1723448545-936321 X-HE-Meta: U2FsdGVkX1/mqd41QKfcfh60XgPhbUw4YHkDTuwrTsCSl02dNOL9iko5m02BxTw0TDiwrBmoJ6kjQ4Z8QswoSQvAd+iXYA6sy6nvgo2X5/67xMDYTQAPWntNSQcIiNRSqmO69OsJrkR2DczoLG7kVIAWk9sNI1SXdTsuwJrUIP0nANO3Hqa1Yy/bowaMjpRb5I0RiX3aKo2E5+dODn3UDme5bKFJRNApENLweSw2xZX32bdrfS2Cr5fJppTt7uag3QZDjk9VflfNHqPQeiYUJLM8n8YiivJ5HlLARFnl6QlUHchbqMi+IFPZ59GoLYo9GPt01AJTB7/pZNGuV2Iw67fcBvVkth/IfK4uRlJOlISJnh2bl5DPY5HKBx1yectUA01f8MZ7VLKbZHymw8rBTEUcAxOun3cwLuXsew+CI1q9IPgzh4Zy4lSA8grqEX57G1zzwvlwoilU4uhDKtRV71Eh1W7YKZ2YwFwJPYBDpzigFdXA276NeKjChj1wJ8sJqkePQGdOII2jlQJUIXzF6Xm0QXv7XR50TBAoj1a1dqHiw70akdjNpYvWsElkaMJlkFi1Y9yxTYXJ29Ibuz+xTtV1Dk6lLwmuIXFG9UYlI9gN4jyuFMuBskHOM7tZzdrGW+vGlEsa48BloOgE3flmEqLe+XUpsrYmGrwWAwntEiAi7gaiCppLbFEp7etpWrj1gRhexiA7pPKFnUV2Xk/xwJ4MUOyDjyMje4scGvYtzk0LwP6uEzt+GAJOoZ1lT/UrthwDb9F6fQLEg+mQ436hjXWlALxlc9iEGBOCp3lfGdm2iPM3pJuSfgFtFk56PYk95vqndMrshTW9sYqUaFP2sDAW3PLY6XvDDq0z7PcB9eLg/WeTGb+VO20ZFfyTE2XulMqvTR+/xkGLTuysKga5eaA8XcxrunSz4sIfXN/oh9zs0LbRWEcBs7a6/JI1YTxBC7fgD7KqlscylkQVVY7 dUkVmdIV wNFUebPNpP7i8OiG6YPB2ChLZKOoJMXumKq5ZqQ6tIi2q0IehrKhMXMw5xNEiEKI7eylUBo/MIMmObAYnZbJCd1k2TX16Fajg36o0+pFSVvVtQ9AF2ERc4VdoJ73qrbOJyGmrz2aNNSQAmknf0oHbGqhexFegc60DRBDe70YaK9I8MZDdejeoFOMVZfAY3/yuig/Y06XXrdrQrPOQGYMdCIPTs2PQS/iRbRyOX1CJFcY48m3IGRLaYaDcFxEjy7RjMMjUQUSbBMbwsIyuKePw4C0TUpf8mIddhZo1aGDkxvVKOhU5F6JkhBI8l+1GkMrh430GkHbCFIPF2jeehdDXu/JuKnI4c7GP6MALJ48VC8Ts/mg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Daniel Gomez Both shmem_free_swap callers expect the number of pages being freed. In the large folios context, this needs to support larger values other than 0 (used as 1 page being freed) and -ENOENT (used as 0 pages being freed). In preparation for large folios adoption, make shmem_free_swap routine return the number of pages being freed. So, returning 0 in this context, means 0 pages being freed. While we are at it, changing to use free_swap_and_cache_nr() to free large order swap entry by Baolin Wang. Suggested-by: Matthew Wilcox Signed-off-by: Daniel Gomez Signed-off-by: Baolin Wang --- mm/shmem.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 02fb188d627f..d0d54939da48 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -856,18 +856,22 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap) } /* - * Remove swap entry from page cache, free the swap and its page cache. + * Remove swap entry from page cache, free the swap and its page cache. Returns + * the number of pages being freed. 0 means entry not found in XArray (0 pages + * being freed). */ -static int shmem_free_swap(struct address_space *mapping, - pgoff_t index, void *radswap) +static long shmem_free_swap(struct address_space *mapping, + pgoff_t index, void *radswap) { + int order = xa_get_order(&mapping->i_pages, index); void *old; old = xa_cmpxchg_irq(&mapping->i_pages, index, radswap, NULL, 0); if (old != radswap) - return -ENOENT; - free_swap_and_cache(radix_to_swp_entry(radswap)); - return 0; + return 0; + free_swap_and_cache_nr(radix_to_swp_entry(radswap), 1 << order); + + return 1 << order; } /* @@ -1019,7 +1023,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, if (xa_is_value(folio)) { if (unfalloc) continue; - nr_swaps_freed += !shmem_free_swap(mapping, + nr_swaps_freed += shmem_free_swap(mapping, indices[i], folio); continue; } @@ -1086,14 +1090,17 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, folio = fbatch.folios[i]; if (xa_is_value(folio)) { + long swaps_freed; + if (unfalloc) continue; - if (shmem_free_swap(mapping, indices[i], folio)) { + swaps_freed = shmem_free_swap(mapping, indices[i], folio); + if (!swaps_freed) { /* Swap was replaced by page: retry */ index = indices[i]; break; } - nr_swaps_freed++; + nr_swaps_freed += swaps_freed; continue; } From patchwork Mon Aug 12 07:42:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13760269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78417C52D7C for ; Mon, 12 Aug 2024 07:42:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F6CE6B00A6; Mon, 12 Aug 2024 03:42:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 133D46B00A5; Mon, 12 Aug 2024 03:42:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 014F66B00A4; Mon, 12 Aug 2024 03:42:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D75746B00A2 for ; Mon, 12 Aug 2024 03:42:28 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 83CDB1608FE for ; Mon, 12 Aug 2024 07:42:28 +0000 (UTC) X-FDA: 82442800776.30.FF2F928 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by imf20.hostedemail.com (Postfix) with ESMTP id 88AC51C0020 for ; Mon, 12 Aug 2024 07:42:26 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="TN9Bti/P"; spf=pass (imf20.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723448492; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kqzOiSUcg37jmhpXPpVX6LPzNgjUKcTxKBX3p6fMXZU=; b=Rp2vWs79hI8kRsGQDoqdqlQ1QMm/4ObiPcA+gYu7CEXL40QSnK93oiMGTvqPeg9G42lfzh GOy2OAU9dzGvW60Cn+iQGPpAVxbuljJQKQ6Qn1BJR2LlsZYfhLwcpgX9W2bfcmHSsVxaIl t9K6UerxYy4EWBKsB4CM7ubxVG9Hbvc= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="TN9Bti/P"; spf=pass (imf20.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723448492; a=rsa-sha256; cv=none; b=uo/GnyG4QMnIKCDihIx8nnp6WvktAX/6Eb3RBdCwt/jyksoTH2YqMn3cQpYuO/SvU6t4sz rfvMdD/gBbMlBm8LBjXQnIOpBu1okJJtbt2FFlBvbQLZ3k6Lj5pMFgW0bRcwejCLzHHPGq BxOtzL1lbHdZGXGLE5PbEtvph1JrLis= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1723448543; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=kqzOiSUcg37jmhpXPpVX6LPzNgjUKcTxKBX3p6fMXZU=; b=TN9Bti/PbtYAdna4Ta+AHAUoPUMngHXCbi9/ycTQjr9DtoiV3uBdqqaaEASO4xr+sYFPHWX6YAOxcbEuJhJYFZf04I29SI2qDouWWRLEeOqfKO9Ftfk0ucjp6UED4UyzRDY9ziwRlKJQhRYNwfUYnz8UEj8E1FYRegX7Kc1drjA= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WCbU5uG_1723448541) by smtp.aliyun-inc.com; Mon, 12 Aug 2024 15:42:22 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 4/9] mm: filemap: use xa_get_order() to get the swap entry order Date: Mon, 12 Aug 2024 15:42:05 +0800 Message-Id: <6876d55145c1cc80e79df7884aa3a62e397b101d.1723434324.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: 8fjqgge9y7hmmc34akdzi17uub9b9ui9 X-Rspam-User: X-Rspamd-Queue-Id: 88AC51C0020 X-Rspamd-Server: rspam02 X-HE-Tag: 1723448546-310816 X-HE-Meta: U2FsdGVkX1+vl2+YQPZiIUeeJ/3zhiya6dqd8ZqV3nN6TN+DaaLhn2BmXxniH8aB8mYukRGfRpu/aT9rKVSyk3MJWPXof/7oavrKyFbzTS0o7/+JZc5Gns2Trcie50kLqMj4GATRNb7ghC1qaFEO6jmYgONEUdZi88SWJxwbPYHKSaxnB9MgWreZnpj+9T60xmuzg/MsrJJ3n+R7TKFbShLU6euCIXl2hctCc9PC+ujJyF630ecsKASxslceEIA0S/0WcAnyNZsEyhgreC9OTjD8pXLhOo/UJm8QKvIRkcUYRKUd0JZqM5BCfVrcpo7E2JFwtRCbO8I4T+TILxKhR7dtiIqc/2R0O/xLNZ75FBFVHkfsqrkawpTt3xeuJZQpLgk9aOWuIAVmQ51IkXB5U+Xnmt8kVAgWVEpjglVZjlHGktsUKtweNOCdxlKFpqARBGQ0DZrcZQ6rKJN8SKkYTA8NiFkLeTxfU2dziFoxkw+Y19QAc7mQ4htGTp8/eVBfnjgc0kmRmz5rkDNj67fYAKLRZsdaocvCFCvXAsPsFEZ5cSjp1JGP32Zj6qlMeTPVQyOIMpHm+17nkESwHbfN7uM+Pb1ieCG7KyG0BEPDFqWnw+gWFlK3GdcS1szZCrdUSWJNKZfAQCHJJUEi+9c10lQSpig3T1sP15I67awhQpqDbvtJeHd9NxJlEyYDJkbTrn0CKgobyWxYcCdA9bAPcmd8FOBf1xNFdJ1TOX0FOd7VC/yKIhUUOKDORPI1IXOOIC5jaJADcCCGSgdKBfQnR8eTQdKIGTkd8yF3XVdY1j4wStv9bpxR3QLbxl6Jn6wZ4/8hGx/5jX3KlHucM1kUUCAZ0esQWfENgyrgKHVA1ouWkI86+p6FMCX8zBDPoWObAr2KwQvkunfxhB+52ut3SqiodAIBsIS0G4KHridv5vI5vK1+/Mz9+X0q9Jgarz6hUyuHRQ+Yfd9H+JIZbBp B+TS7AKw w5T+HE0g+vvm5Z9kvxoZz5NJCbG7BilYNhaTDuTWSYMcff30coKFnvsgznJABvHdr09BZehM3CQED2hGitzNrTiHopt3W58eYwXHDQfjQeO/vxLXlBDGvzwB3QG7Zw4HHQIL2tAlJ1SCzNdAjsGUYIZh8A0x8/0syjkodo9cjl2tgOdZR26UxRMKsKUESjCcL2j+hDuLv9ZAugEHt+voNbHNT4tKFw5uKBqbnXIkNh/LxSstJ/ptrL6lJ/w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the following patches, shmem will support the swap out of large folios, which means the shmem mappings may contain large order swap entries, so using xa_get_order() to get the folio order of the shmem swap entry to update the '*start' correctly. Signed-off-by: Baolin Wang Signed-off-by: Hugh Dickins --- mm/filemap.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 4130be74f6fd..4c312aab8b1f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2056,6 +2056,8 @@ unsigned find_get_entries(struct address_space *mapping, pgoff_t *start, folio = fbatch->folios[idx]; if (!xa_is_value(folio)) nr = folio_nr_pages(folio); + else + nr = 1 << xa_get_order(&mapping->i_pages, indices[idx]); *start = indices[idx] + nr; } return folio_batch_count(fbatch); @@ -2120,6 +2122,8 @@ unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start, folio = fbatch->folios[idx]; if (!xa_is_value(folio)) nr = folio_nr_pages(folio); + else + nr = 1 << xa_get_order(&mapping->i_pages, indices[idx]); *start = indices[idx] + nr; } return folio_batch_count(fbatch); From patchwork Mon Aug 12 07:42:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13760271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83D41C52D7C for ; Mon, 12 Aug 2024 07:42:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8A0CD6B00A2; Mon, 12 Aug 2024 03:42:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7683F6B00A5; Mon, 12 Aug 2024 03:42:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51DB06B00A7; Mon, 12 Aug 2024 03:42:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 189666B00A4 for ; Mon, 12 Aug 2024 03:42:29 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BE6AA80872 for ; Mon, 12 Aug 2024 07:42:28 +0000 (UTC) X-FDA: 82442800776.09.9240A89 Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) by imf13.hostedemail.com (Postfix) with ESMTP id C229B20010 for ; Mon, 12 Aug 2024 07:42:26 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=YqK8RcEK; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf13.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.124 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723448536; a=rsa-sha256; cv=none; b=02BTXJ1T3vZ4BddFMD+urubJ60N2aeEzKreHEJFGvm/TtQhYIo/KoMjcN83CsflFCVIDxN NymkEpEM8lK9GBDYEb71KHwV51TDR+sRfg8VLI5Wm1alQAGgIKcE55BTiTgY1StYVZviYx NRF4A8DT0Ehhc46akcWP2oyp/NykzXc= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=YqK8RcEK; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf13.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.124 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723448536; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CJ17mJWuqe6d52h6IBrZyk6CffbeYGvkfB1+lwHpEl4=; b=Cpy4ljjDBM+hfz4lTXTDhc3wFUdwn5BLuDOGBnsfiv7bAvsAbWXzbz3pTk3qQKadY4A2rj w73ZUd99XjnP4zOuYkg9gQlZt6Mgb6P+rd3z2cHwUOt/xcqSrhgXCHzVuXn5BztIZ4QlM5 4Zy2BjeFZf2hHsvE4AV95dlVh3D4wBI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1723448544; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=CJ17mJWuqe6d52h6IBrZyk6CffbeYGvkfB1+lwHpEl4=; b=YqK8RcEK97o5sDSbuo9hOYNwKT1ZNBSWmLzGGWQ/3gW5BMP0VeNaVm+JTvPKTW2kvy8bEd3E4BTfs20QEyMLlPNq0pHOa2uMbyMn27noS/F1426h8BidOfpHz4RfF3Uj60a0twx+9TGfLLtL+jPcU9cL/+2j+5kZaGdywgwvDNI= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WCazOiu_1723448542) by smtp.aliyun-inc.com; Mon, 12 Aug 2024 15:42:23 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 5/9] mm: shmem: use swap_free_nr() to free shmem swap entries Date: Mon, 12 Aug 2024 15:42:06 +0800 Message-Id: <342207fa679fc88a447dac2e101ad79e6050fe79.1723434324.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: C229B20010 X-Rspamd-Server: rspam01 X-Stat-Signature: q6aem8ihz9hkkic156f3mk1fkdoqwc5a X-HE-Tag: 1723448546-509845 X-HE-Meta: U2FsdGVkX19AOX+tPGSf/Ad8RMbG/hRE7NoPOJoF/mpoDUMpowVAXKAPmozCwgsZypTsZaUBg2E+F4byWqMhb3V7vBJFiDOT7HNYZd159HgxLTeaOOhLKB+VWrKmikMpSM9gZNS8/dZX63zgErrB49mEqIBDTgjqUy7jg1VqP+TaAclCwZ2aVvkL7cPj6ZwrpCHl9k2SgtSjq33+cPgj0El7qV6P4tTBe5FYLO96gFogyZ+paC11mqeBbCt3zbUjFTEIA9c54olEhdkVx9HNJYA7xg46lUfdvcahhGbBOvyTqWRbi/+TE31whwghkYEmwYvoJesGRRbGPL64AvV8U4uTmrJGT+K7HyRBqTI0MUjKJE0YJPAqapPe6GF4NFE/OWI7sHruDwsuTQuS/vjdK36non9jt3Put3lT9s+7eZEH0D9uKSo+mH/t1xK2Hc7IPh/73V6rrKiAztAlToxr2Sto63mnZvLU1SC3Y6yDHWPFVXAsuahW7CRtreWinUtpjndAryR/25/+3qVsf6aAvsuO5obVIft1xqG3tCjYItoR7PfH2mDh4yRwooFDfYsEx0wYDILkDkVorS2XL3Rii3agvcFz/JRJU0XzafzQMQjO1RT6K55/9P4JRHtEJogm7O8qVJ8UEd7b7tJSK5aY7O1uB/q3wYiTf3uYUHJ+BJeNdW5jCOdxWkFbbxfJXi/9psnTLcdACcBDrEZsEY4AYrGcxaG+HrxJKJ+0wIqumvV+PlIa0Rs1Wacewb8n7+5A1Kwfh3mLyQiSgMgDO2uJB6YGHiCmxNbdeFIwkmK80aD/wINUL3CvjTiK9smw5/iv/gtanxQzj6wxlVfRU9SF1V+OnS1M0aY+f0gOsiY2UwcOYgffdjV48xVH13AXAD5QvNAB6iuWeUjgUWaujlfsn1tFoO/W2Rw+gA8QsMuoxfJRaOII4EVaoyg6zcsRdnhrRsXMQDZY8Ivm4xa+jGn WA4XXD1o uGyddXcaKazE+HZMSEs4uveY0bGU617Ieg23czMtnu117xpuMUOJs0HcMFkkNcalxNIKEfrXV1emuwBZYPouBCnpjGkBN753lIueet/mIHcVKxNbe/yPCK7eEzvr2ZM7dOMzVKWy5bvi47rDx3im/2Fp32kpleF6jJUFaIQEC/irD3/PA6UARubhZVjJwRT58ljBNUqoRnsfc9rdTWoLPOQai7pCikeMaaj1c8O6QvD8dbWA4t/rq4Wogtw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As a preparation for supporting shmem large folio swapout, use swap_free_nr() to free some continuous swap entries of the shmem large folio when the large folio was swapped in from the swap cache. In addition, the index should also be round down to the number of pages when adding the swapin folio into the pagecache. Signed-off-by: Baolin Wang --- mm/shmem.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index d0d54939da48..f6bab42180ea 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1961,6 +1961,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, struct address_space *mapping = inode->i_mapping; swp_entry_t swapin_error; void *old; + int nr_pages; swapin_error = make_poisoned_swp_entry(); old = xa_cmpxchg_irq(&mapping->i_pages, index, @@ -1969,6 +1970,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, if (old != swp_to_radix_entry(swap)) return; + nr_pages = folio_nr_pages(folio); folio_wait_writeback(folio); delete_from_swap_cache(folio); /* @@ -1976,8 +1978,8 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, * won't be 0 when inode is released and thus trigger WARN_ON(i_blocks) * in shmem_evict_inode(). */ - shmem_recalc_inode(inode, -1, -1); - swap_free(swap); + shmem_recalc_inode(inode, -nr_pages, -nr_pages); + swap_free_nr(swap, nr_pages); } /* @@ -1996,7 +1998,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, struct swap_info_struct *si; struct folio *folio = NULL; swp_entry_t swap; - int error; + int error, nr_pages; VM_BUG_ON(!*foliop || !xa_is_value(*foliop)); swap = radix_to_swp_entry(*foliop); @@ -2043,6 +2045,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, goto failed; } folio_wait_writeback(folio); + nr_pages = folio_nr_pages(folio); /* * Some architectures may have to restore extra metadata to the @@ -2056,19 +2059,20 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, goto failed; } - error = shmem_add_to_page_cache(folio, mapping, index, + error = shmem_add_to_page_cache(folio, mapping, + round_down(index, nr_pages), swp_to_radix_entry(swap), gfp); if (error) goto failed; - shmem_recalc_inode(inode, 0, -1); + shmem_recalc_inode(inode, 0, -nr_pages); if (sgp == SGP_WRITE) folio_mark_accessed(folio); delete_from_swap_cache(folio); folio_mark_dirty(folio); - swap_free(swap); + swap_free_nr(swap, nr_pages); put_swap_device(si); *foliop = folio; From patchwork Mon Aug 12 07:42:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13760274 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93352C52D7C for ; Mon, 12 Aug 2024 07:42:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 859C96B00AA; Mon, 12 Aug 2024 03:42:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7DE8C6B00AC; Mon, 12 Aug 2024 03:42:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 631B56B00AB; Mon, 12 Aug 2024 03:42:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3C7D76B00A8 for ; Mon, 12 Aug 2024 03:42:32 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B82CDA0907 for ; Mon, 12 Aug 2024 07:42:31 +0000 (UTC) X-FDA: 82442800902.19.4553020 Received: from out199-9.us.a.mail.aliyun.com (out199-9.us.a.mail.aliyun.com [47.90.199.9]) by imf15.hostedemail.com (Postfix) with ESMTP id A5B48A000B for ; Mon, 12 Aug 2024 07:42:29 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=ip3Oj6W2; spf=pass (imf15.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 47.90.199.9 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723448481; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+R9mz0zB2CI8dFveIvPvJwpF+8EOZp0Xn71CWrw5Br0=; b=v9IgzN3K3hRZODXcKWZMO9iPtbIwgwrkDwr7IkK+bjqo3Kv2qfD2GetyoJWUFZRlsKk2oH 4nCTYpYdMufXoPBsIERORCK5ZKW3WWWlNkHHLbEL/yvIIjT6ivfjo83vJC0jNXDrxuqzzW +WfwXsSSf7HG1CXqWTJ/RCQl7pUeYGY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723448481; a=rsa-sha256; cv=none; b=6bIH8DfZACmw4vB2U/ehyO1c0b14aXPgV+kT8ynoL9OKAF77VRCByas4K950OHevuPqV2O TrCSy/iewbi4738CkoMwFRfNCbTSUVIf33xWm+GOMDCL0wxuLIv2p1oU3/GmkLyxzDgGPW LZSXr8UwY71HDKRXaW+W78g74aYxJpQ= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=ip3Oj6W2; spf=pass (imf15.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 47.90.199.9 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1723448544; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=+R9mz0zB2CI8dFveIvPvJwpF+8EOZp0Xn71CWrw5Br0=; b=ip3Oj6W2XCZ74vYfMfX1iusCZMqiYuSf+pMV1iqjGgrE9iMQwZSLhUujtkP1dzt+8R7JpQda9AlMr1mJqQjnI91tHWcqgdYOoUWPRHOeeIDqBsYbLEFld4BRkt4cS7c5XKS4ScjoqzftI7inxemqIJeP5Xo1FvaaNavRVGBwoPk= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WCbAKPi_1723448543) by smtp.aliyun-inc.com; Mon, 12 Aug 2024 15:42:24 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 6/9] mm: shmem: support large folio allocation for shmem_replace_folio() Date: Mon, 12 Aug 2024 15:42:07 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: A5B48A000B X-Stat-Signature: 3eyg7m4mbqprdi3oimxkamcsghyc7hhy X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1723448549-336719 X-HE-Meta: U2FsdGVkX1/2a9dQDPS8oWuBxOyyneWsAMEdSKfByUI2qObYJrj4qT7xqsiejjkN+HNlI0a8BU+kMa2AEzdb97KfRr/gNpkdxsaiKQDyetINNwzmdQMfMrZLR0gOsdbyifHUJl8/G5w41Qb6vVNJrXUw7v9KR18ESns7j6vymuF9vKoJdELdqhSCH3ll5qmSK1YU9v1o7xQrjRxa8FTBV2DcDmbYjHwLDd3uueH9SCs9tLJ4rtnrdsJ7koWnbhc/8qW6l46WrUKk5vUTOopIfg6YW/197VgjiK5FlrJLYjuNJWMT3OqyhLAZv8r7mYN9Jhmr5gdeUhDnSuXxD7tnoVXyRaR9aqdLSeeVj4WlgxQLI6wAfgCHPRfHS6iB8DblHilQKt2jUJNkTNm1AfyHgw49X89RCEALUTTj3d3kqD6kfrJ4cQCEehGxwI4VzwNCLkxNk86xENtYFefqfPRbMLHCikF2/baLa0nr6P3DzzN7oZ6NOH5oB1voi54caMpyM1WGep8tqL5GZFBYt+hNu9jGUC72O+t7ZMH6q5oLAUmdTzWNoOsKoQ/tvyREiMIXUTLR0xU/HBud3MIYsD35YvbmHMcJ9DH9UfjLk15xs+hRsxkRSs2NyDdFKbucsZmSAC3no4YNYUGBW1oaI4VLTMBHF5LQ/KLhMkjI31Qj7ujAwvA/h+315M0WCSWMuYNqvkjwax4bSzOPfOFJATWccbSqUtmvpnZHpedacEcROt057eYS6++eZ4kKqwUZwX764RD3O6wGHPRmryI/DeHBAQlYYP/VX925fW2uysWyDUOUYhh7oeo/Dlh1LMHRhOnjXoHb4QKi4MqkdC9cUF4hriRTfk/BFNoj+uVLE1ACCyIMBWrouD0pBetIJ7f/LI5tlHVtpYrJ/YyeoeFLmfT59Pd9T+oPb9iKniOj/2sWzXwSMvB4B4ENxcsdcYPnQDv17MoBfxq44Ct2GGYqLqe RNtgrlP3 APXUDu5Ahjq6aLB0WEunbmhn/qNcBv/Yx6+cAa9xg9xaUJ0YYjqgcUx8wclSunc5M80Lg9tyDA9JA38X+fo/qRWUYqP9MhYx+JprEaas+g0Admwy/3YuX54Fi1K+ia4Ktz6wACB4rZsUcbqwA1ZVGi7at2ZoEsQ1m5bd523bHc6Qtld8u2ahS8QxSU7N9txmYjRjVfCm4T6x6U9VtwbG9z6sW0PX7BJJ+98hSI41SA8PbV5Ohnf0xLa9mqQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support large folio swapin for shmem in the following patches, add large folio allocation for the new replacement folio in shmem_replace_folio(). Moreover large folios occupy N consecutive entries in the swap cache instead of using multi-index entries like the page cache, therefore we should replace each consecutive entries in the swap cache instead of using the shmem_replace_entry(). As well as updating statistics and folio reference count using the number of pages in the folio. Signed-off-by: Baolin Wang Signed-off-by: Baolin Wang --- mm/shmem.c | 54 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 31 insertions(+), 23 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index f6bab42180ea..d94f02ad7bd1 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1889,28 +1889,24 @@ static bool shmem_should_replace_folio(struct folio *folio, gfp_t gfp) static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, struct shmem_inode_info *info, pgoff_t index) { - struct folio *old, *new; - struct address_space *swap_mapping; - swp_entry_t entry; - pgoff_t swap_index; - int error; - - old = *foliop; - entry = old->swap; - swap_index = swap_cache_index(entry); - swap_mapping = swap_address_space(entry); + struct folio *new, *old = *foliop; + swp_entry_t entry = old->swap; + struct address_space *swap_mapping = swap_address_space(entry); + pgoff_t swap_index = swap_cache_index(entry); + XA_STATE(xas, &swap_mapping->i_pages, swap_index); + int nr_pages = folio_nr_pages(old); + int error = 0, i; /* * We have arrived here because our zones are constrained, so don't * limit chance of success by further cpuset and node constraints. */ gfp &= ~GFP_CONSTRAINT_MASK; - VM_BUG_ON_FOLIO(folio_test_large(old), old); - new = shmem_alloc_folio(gfp, 0, info, index); + new = shmem_alloc_folio(gfp, folio_order(old), info, index); if (!new) return -ENOMEM; - folio_get(new); + folio_ref_add(new, nr_pages); folio_copy(new, old); flush_dcache_folio(new); @@ -1920,18 +1916,25 @@ static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, new->swap = entry; folio_set_swapcache(new); - /* - * Our caller will very soon move newpage out of swapcache, but it's - * a nice clean interface for us to replace oldpage by newpage there. - */ + /* Swap cache still stores N entries instead of a high-order entry */ xa_lock_irq(&swap_mapping->i_pages); - error = shmem_replace_entry(swap_mapping, swap_index, old, new); + for (i = 0; i < nr_pages; i++) { + void *item = xas_load(&xas); + + if (item != old) { + error = -ENOENT; + break; + } + + xas_store(&xas, new); + xas_next(&xas); + } if (!error) { mem_cgroup_replace_folio(old, new); - __lruvec_stat_mod_folio(new, NR_FILE_PAGES, 1); - __lruvec_stat_mod_folio(new, NR_SHMEM, 1); - __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -1); - __lruvec_stat_mod_folio(old, NR_SHMEM, -1); + __lruvec_stat_mod_folio(new, NR_FILE_PAGES, nr_pages); + __lruvec_stat_mod_folio(new, NR_SHMEM, nr_pages); + __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -nr_pages); + __lruvec_stat_mod_folio(old, NR_SHMEM, -nr_pages); } xa_unlock_irq(&swap_mapping->i_pages); @@ -1951,7 +1954,12 @@ static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, old->private = NULL; folio_unlock(old); - folio_put_refs(old, 2); + /* + * The old folio are removed from swap cache, drop the 'nr_pages' + * reference, as well as one temporary reference getting from swap + * cache. + */ + folio_put_refs(old, nr_pages + 1); return error; } From patchwork Mon Aug 12 07:42:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13760273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C43DC52D7C for ; Mon, 12 Aug 2024 07:42:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 326796B00A7; Mon, 12 Aug 2024 03:42:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2D7076B00A9; Mon, 12 Aug 2024 03:42:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 176576B00A8; Mon, 12 Aug 2024 03:42:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E715F6B00A7 for ; Mon, 12 Aug 2024 03:42:30 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A148D1C510A for ; Mon, 12 Aug 2024 07:42:30 +0000 (UTC) X-FDA: 82442800860.26.678497E Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) by imf01.hostedemail.com (Postfix) with ESMTP id 9DF8440016 for ; Mon, 12 Aug 2024 07:42:28 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=bAE8veZu; spf=pass (imf01.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.124 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723448537; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HTq6ByUKYopHbBOucaa7TnUS458Ur1YounHQqWHqN9Y=; b=NExq6XJxl21EhZS3P3cadcrjT1rYNhW9p+v6bIwMNKFkHb9Ru4l0jBbTl2mo40+1856R2q e4zBJxPIhu8Zg56aX7bHBeV+mq5ENhshxnEKrV6FmGUkrHRPH1qNZrmsfhOQDCkHNnrdm6 CkxCqogt08T/NtZYj1JrPTEJpZLRzGs= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=bAE8veZu; spf=pass (imf01.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.124 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723448537; a=rsa-sha256; cv=none; b=5JqOG5pW9t8VxmgmS1Msc85Z+T6Gi8ieXetzBsLa38u1cfBwyhOnERGP7HKWUUeZ0bFLLR BDNVFXy7vM4P7ATYePBxN+GS4QwDssv2jFocupJsxbLAMCCx5/xPw/og2hxrZjk9FeXZ+E Yjts5sM4koSvPstVLxYHPtB0kYQEUTs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1723448546; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=HTq6ByUKYopHbBOucaa7TnUS458Ur1YounHQqWHqN9Y=; b=bAE8veZuDTdgOTjU5mbOztE6V5fXtk1bkAFgwjJkMZyXhIZGNKNUWxil4iLfRAt5R/7y73IW1f5Cq5DzOQi8IV+ZYuVOwFUEwc/aixtu/OtZ1YhUoTf57zEwLAM50teME5MV3Qf8L4zEDgQN9ahwWeobJcUI1wFVTp6zS67utg4= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WCazOjj_1723448544) by smtp.aliyun-inc.com; Mon, 12 Aug 2024 15:42:25 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 7/9] mm: shmem: drop folio reference count using 'nr_pages' in shmem_delete_from_page_cache() Date: Mon, 12 Aug 2024 15:42:08 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: s5ucpypb6tn3jpn96kzridfs5aqfreyx X-Rspamd-Queue-Id: 9DF8440016 X-Rspamd-Server: rspam11 X-HE-Tag: 1723448548-854248 X-HE-Meta: U2FsdGVkX18ZBAwZFXnYvd9pZM0Zt9BRdCwXcdC1JBP723RMrdilqB5lJ8iQmQf7Y3dOb+dnjPrKyzUGGQ5l6YifacR/tny9bjJdAzaS7cy/Ut2XI8UhmA0boOxaQF+Zji1uaS2giW61lZ2NWP9s+B0Ud7tqSWt/gg8uD7IAWeQEcXCFAipOaOh6/VyRTkF8nl9bdb/lXrcFeKSNw/YfGXLqeFZ1Vo1hDJBCCr8WRF3Qg7q3W3sx8bAnoZI4f8TmM1kStMGDoAV3PKcS4kb/qjDVmTiqds031mXMcP5XJMH9ZDHRp42HnYJtGLOpMMPRUFNvC23ehI061HjK2lgUSV8UE3A5FiBhAoPThWnmri30aIpJdsuPLXhCBRRkI4OeyeQJSPB+AUF/SFKMk1rNTrGNJW9IxH6gpDdgKvjZQvqDCHLZIfio7ys374vifrJvQlQJokV3RnK4DYdKMTNLvVOgjaqNHmrQZRswOtClMx75F3/HmExYKbKLtDi6uZbYZQaZ/gvnt2S8TFGIO+cV4apqq5Ho+HF6N8YS2Uze7bPB2Xkok0WJjwI/3JaoXPIQLSYBbP2BZwK0MThfiWUXQhETb+jxfgBMDOeJEnVN+tJzM+2N3tx5hbscbKqIizCfup2S0Usf1b5J7l/Vmm703bn7UrVO6nj3+uR277cS9ODPBsDvKZL4uHCrJArTeQH9CJrFW/NK/rAjM6KHrVD7MzxLfiw0+nJQP+kJvLxcYVUVgIDaoGU8XqrYPPdPGgLQEDa4FTv07SKbqGQSLq1/57U4kphtkaUMQpevFZh5nhIBkHMotNHrfS3bBYZuuWzJtJfXuQZ5MkXdJEU9z5HG61zRLJ5N9mT3ByY7OUBphYpKqszT7HxJjvJWbIJez4mro9QAil5YX3yb7JxLkJz5z+gglNhrPRi8O+EON6yMjMhW/FeCzN1N8S3WplOmidzIT6yHyQndgxcrmReuEkp Oui1EjQ7 jfrY+TqvZ6cjCysWNx9CcXQWYS3KRMrsYo0tl9z9copeqzOTjn4Q3IwNgiBP/wrJKHkS07USiMssWW8Mwg+hNvO0Ah8Uv8bL8T1rXMKcvVkGU2g8lJvXyEymk6st0olGALwHgcDE+VFDJQa4ISwDlDpGVwBh/iIKSvObybn9e6fwy7vTB3PT+GriRxYXopg2EXXYTfhS4BT1qb+vnEjJ76gujYWsrJLaQHEDSWYkmo8dyB8IL8fj6PMm+Tg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support large folio swapin/swapout for shmem in the following patches, drop the folio's reference count by the number of pages contained in the folio when a shmem folio is deleted from shmem pagecache after adding into swap cache. Signed-off-by: Baolin Wang --- mm/shmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index d94f02ad7bd1..345e25425e37 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -851,7 +851,7 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap) __lruvec_stat_mod_folio(folio, NR_FILE_PAGES, -nr); __lruvec_stat_mod_folio(folio, NR_SHMEM, -nr); xa_unlock_irq(&mapping->i_pages); - folio_put(folio); + folio_put_refs(folio, nr); BUG_ON(error); } From patchwork Mon Aug 12 07:42:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13760275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 052EDC52D7C for ; Mon, 12 Aug 2024 07:42:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFE4B6B00A8; Mon, 12 Aug 2024 03:42:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A7FB06B00AB; Mon, 12 Aug 2024 03:42:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 855646B00A8; Mon, 12 Aug 2024 03:42:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 580076B00AA for ; Mon, 12 Aug 2024 03:42:32 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 0D1991A08AD for ; Mon, 12 Aug 2024 07:42:32 +0000 (UTC) X-FDA: 82442800944.24.2BACE08 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by imf02.hostedemail.com (Postfix) with ESMTP id CE2718001C for ; Mon, 12 Aug 2024 07:42:29 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=sDCzgibt; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf02.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723448539; a=rsa-sha256; cv=none; b=USySwocakmTOCN13HiHALj5bcdSselJxZ+HfeLeXSgz1uIr+oRO/E7ywB8lNNwJc6b2Kra kzm8wPCLCV6r4cGNad5eXl4pjg+gEq+kS6knjPWyLjDpDV0hN057Oa9v8k4KrSGPgvNxXO ejDGFhyHGWoS/rWl5lhAHb6MKQ3fVxw= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=sDCzgibt; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf02.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723448539; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PZslh3xDjv8KO1NsIQv9gTqVJj0244TUIWHfHI4JVZY=; b=VkYU142QSrzBVMVUZsBq7lDAlrO5S1x4CayJB1Jc2JFMjQKXja/gWqJT/BYSVgM9KRPgkI fOEW1br57SYGjn/eEwTGepZPOHYivTdqNp4wrcpKhCWOAW/j5oRm3bCo2K+tOblrNQLIO4 srnj96lNsMewPFTEh8ZnY9vTcj3VmVk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1723448547; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=PZslh3xDjv8KO1NsIQv9gTqVJj0244TUIWHfHI4JVZY=; b=sDCzgibtglDth7yEb06ZF+JPV3a7P8kXlyjen9puTazzWqLXGurUo8JQLXMwluUBT2ZI2zlqp6a2wd2202r6sJhrj95PZg2es7ua5U+8+ZzeZALsl7bVluIDLiYwHcl9Z5vC4mBgPexIiw4XxvUHBXd3x3pUf1SgzJ2VMIFzXZs= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WCbU5vC_1723448545) by smtp.aliyun-inc.com; Mon, 12 Aug 2024 15:42:26 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 8/9] mm: shmem: split large entry if the swapin folio is not large Date: Mon, 12 Aug 2024 15:42:09 +0800 Message-Id: <4a0f12f27c54a62eb4d9ca1265fed3a62531a63e.1723434324.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: CE2718001C X-Rspamd-Server: rspam01 X-Stat-Signature: jgoa4jzg5gz4z7kcf4pqnfwwqy937poe X-HE-Tag: 1723448549-348797 X-HE-Meta: U2FsdGVkX1/mpEDtY23zsYDp6yo5C8zeNMXSzSJpOlT2hDR5sOr1mEjiTbd/8CyBxrl2ajUKsXi/+HE7H92n5Is+iCF7bIzRFJzBZ3dbC0UJrGDsufmzyN+2OgFTvdxxViONwQWp3pmUdR0JYu5X2fGPiQXyL19hCP5ML4UKQbsV87xKKkpDDRLzAa62W3Kx4GJszpTHkpk5BW16upOK1/H/eEeky017OI5wBNBM26wktCQh/8GxFylY6BMkDZ0dwKH/rrNW1xWqVic1pvRLNMJRxUIwZlK7wxUf3g8EzU8FIBxsx0iGpbIXzoyiBb3HOZ0t/BzCBfJaz2GEo7roB0WzD6ycKJft1LCH0tan+0G5rG4B8059EH3W1QDAaxa7eViwWwquRXgBGSN1szO+VtiLmMhoARJgL5amc3x8EnrAKi1iAAqFWf70bWmwHLQMpX/KSAOCXvxSArE8gFvrIPbUlyEwGQrDuaN1S1Dp01Sg19mOOycfQaMHJNOojV5BXbwXdu6bTr5bsCmBYgpg6PCL4IwlmSNW2sKmf7iz5PDYBXdFJm7O7VhANG1IykJZtGhKBG9op6+bZj9OCSpD4KnYiwOQps4JGSjJO3VzrJshQ/6EZQ8fR/ALaKggLm26zY02RBxuoPBC2NL5Io8NC4JWQGczm02AxRaCWvkhiiE05LZKGnUId6REqQGs+XgDnVZdslSUJBHo+TfSY0mjohGDejBJQF858C7bKPGNvc4EQWZ2+JyLVNhm8WA9GCewxE6QSDssMEM6ubFcpRMVZTrzN5SgVLEhR+KZhaBXXAYZbKoKpSXGTQ/pEd8OUHmRQl3wwKC/EpqtXn//5EyqwfIbNImal7H51RVVqCWj7biWim/W/VCXWkLjM8IGOhvU2neqgZspF5k4BpwwsRke3SSepzjrl1u/srdywsEaUZ3D3J/o2oGnAn/2GpZ/FpkWK1C3dY9UyzVy9qiw9ug A89UBa0o pMtHyUB53CIITdukXwzRPDXEMnmYPhXi4g6CEo1uq9YBxp6VdJhRQxg4BgbTuX0EDm/mFppM3Pk+ixFg/30HhBmX/N7BqB1xkSOiJxyzgwdRbpIylavUS7WWvGygQv+tVM77fk6gSq1FUqigBL3526ZVSt0RCDX5zYP7+0SXJ6yPTzKEVqnuSIZd1tmeQsAbNRGTRBsz0cd1Ae7Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now the swap device can only swap-in order 0 folio, even though a large folio is swapped out. This requires us to split the large entry previously saved in the shmem pagecache to support the swap in of small folios. Signed-off-by: Baolin Wang Signed-off-by: Hugh Dickins Signed-off-by: Baolin Wang --- mm/shmem.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/mm/shmem.c b/mm/shmem.c index 345e25425e37..996062dc196b 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1990,6 +1990,81 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, swap_free_nr(swap, nr_pages); } +static int shmem_split_large_entry(struct inode *inode, pgoff_t index, + swp_entry_t swap, int new_order, gfp_t gfp) +{ + struct address_space *mapping = inode->i_mapping; + XA_STATE_ORDER(xas, &mapping->i_pages, index, new_order); + void *alloced_shadow = NULL; + int alloced_order = 0, i; + + for (;;) { + int order = -1, split_order = 0; + void *old = NULL; + + xas_lock_irq(&xas); + old = xas_load(&xas); + if (!xa_is_value(old) || swp_to_radix_entry(swap) != old) { + xas_set_err(&xas, -EEXIST); + goto unlock; + } + + order = xas_get_order(&xas); + + /* Swap entry may have changed before we re-acquire the lock */ + if (alloced_order && + (old != alloced_shadow || order != alloced_order)) { + xas_destroy(&xas); + alloced_order = 0; + } + + /* Try to split large swap entry in pagecache */ + if (order > 0 && order > new_order) { + if (!alloced_order) { + split_order = order; + goto unlock; + } + xas_split(&xas, old, order); + + /* + * Re-set the swap entry after splitting, and the swap + * offset of the original large entry must be continuous. + */ + for (i = 0; i < 1 << order; i += (1 << new_order)) { + pgoff_t aligned_index = round_down(index, 1 << order); + swp_entry_t tmp; + + tmp = swp_entry(swp_type(swap), swp_offset(swap) + i); + __xa_store(&mapping->i_pages, aligned_index + i, + swp_to_radix_entry(tmp), 0); + } + } + +unlock: + xas_unlock_irq(&xas); + + /* split needed, alloc here and retry. */ + if (split_order) { + xas_split_alloc(&xas, old, split_order, gfp); + if (xas_error(&xas)) + goto error; + alloced_shadow = old; + alloced_order = split_order; + xas_reset(&xas); + continue; + } + + if (!xas_nomem(&xas, gfp)) + break; + } + +error: + if (xas_error(&xas)) + return xas_error(&xas); + + return alloced_order; +} + /* * Swap in the folio pointed to by *foliop. * Caller has to make sure that *foliop contains a valid swapped folio. @@ -2026,12 +2101,37 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, /* Look it up and read it in.. */ folio = swap_cache_get_folio(swap, NULL, 0); if (!folio) { + int split_order; + /* Or update major stats only when swapin succeeds?? */ if (fault_type) { *fault_type |= VM_FAULT_MAJOR; count_vm_event(PGMAJFAULT); count_memcg_event_mm(fault_mm, PGMAJFAULT); } + + /* + * Now swap device can only swap in order 0 folio, then we + * should split the large swap entry stored in the pagecache + * if necessary. + */ + split_order = shmem_split_large_entry(inode, index, swap, 0, gfp); + if (split_order < 0) { + error = split_order; + goto failed; + } + + /* + * If the large swap entry has already been split, it is + * necessary to recalculate the new swap entry based on + * the old order alignment. + */ + if (split_order > 0) { + pgoff_t offset = index - round_down(index, 1 << split_order); + + swap = swp_entry(swp_type(swap), swp_offset(swap) + offset); + } + /* Here we actually start the io */ folio = shmem_swapin_cluster(swap, gfp, info, index); if (!folio) { From patchwork Mon Aug 12 07:42:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13760276 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E45FC52D7C for ; Mon, 12 Aug 2024 07:42:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7BA616B00A9; Mon, 12 Aug 2024 03:42:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7434A6B00AB; Mon, 12 Aug 2024 03:42:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 593CC6B00AC; Mon, 12 Aug 2024 03:42:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 32C756B00A9 for ; Mon, 12 Aug 2024 03:42:33 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id EDDC81A0871 for ; Mon, 12 Aug 2024 07:42:32 +0000 (UTC) X-FDA: 82442800944.04.031DCFE Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) by imf18.hostedemail.com (Postfix) with ESMTP id F2A161C000F for ; Mon, 12 Aug 2024 07:42:30 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=JdMEKtyN; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf18.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.110 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723448495; a=rsa-sha256; cv=none; b=KQQyYdMbwNTkqQD7cofcRn9ZlCtFwnJ1nIQ46BE0k5XLKesSirpbU75btTY3F1MxqAOAHF NmXp0SOxaLeQo3WpDidiqhBaOscpRBpqR3Ae4X5QKGXxwxxEnby+ypGKyK6/leMTUcMNhM H4+icjd8c5EmCaTdVDj0KjHmt5UiGQ0= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=JdMEKtyN; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf18.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.110 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723448495; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=d32a7jBZCt/LbWrkrcBvwBUp97vywsx+8D5iSkO5umo=; b=o39M3uJ/kj/9Snv4P4tFaDEaiOl9wt9wjFShh0L/PZiJNdwEe/Z3dqALl6IAOfK6wSJ3cg RGxH1QQncjyYD7iMpOiAvTmFopgMkpFH9U4Zxd2AyGLBEomBZ4aAq/wzbDwW21J6WMOksM sZFKUn8YXfy4VlYLefCLFWfpCSXRH9M= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1723448548; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=d32a7jBZCt/LbWrkrcBvwBUp97vywsx+8D5iSkO5umo=; b=JdMEKtyNcKmbXkvsgBxmOUadPjq6PRiOn6b0y0PmmUsaAhgLGuSxwVKT+TfFBSewz6N/+OVc5NWcIb5KHoOb2ylmTrJrz/6KJvos5uahGJMIyednlKEU6Jn8JdHmwtak+FzjaTyZIB2/AB6A39hq0hH2Oggy2zPzf+BcHdmE/8A= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WCbAKQr_1723448546) by smtp.aliyun-inc.com; Mon, 12 Aug 2024 15:42:27 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 9/9] mm: shmem: support large folio swap out Date: Mon, 12 Aug 2024 15:42:10 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: F2A161C000F X-Stat-Signature: c1wpbbcahcdx5twspstbmcmdk5bt6588 X-Rspam-User: X-HE-Tag: 1723448550-13536 X-HE-Meta: U2FsdGVkX1/oVhsvECGlp6kMujs9Ra1Lr4oAXoDGIICjCowmJNVjS+Q+q0m2A0rQ3y6WOHoHfjEq6Trq+/yw/O+eHagZ2o07B7Bv1aQl1CypTAZXgblPc3nRVSLkCmY+AV0nQ42gQyZOXlH7YZc/UOgveR45n7cCd4Qa5qPs2iFRlFyVzMffL5REP8CBty/m3R+x8odbc/urRUYIqybJ+KyAKPcC2CcXk20aa4YbAWpnONqnby5Gw2WC2AuFImw3UhIS9WGSwM1HYvDcC13ovgJDs0gG13mGLxes7iP5m+LhE2RxvYJRd5wuC0LYh+nc2/4kpyQs5kxxYu3YDTNdW0ax2GpP2cSnUOXqL590tLOAcgTlbSbXOGNPRqpOqf7LuwNroLfsY5xMdy0kVZHdtnigSwLhsAYfLkU7IZAx22zAM8JtediSFZScxPR+7xumGqMn9RvhV1d4suknduE05dbewxxF1L8Fw5j4LprVc3jDDRgU324vzrW2jr5+V+CO/FRXG2IKJ/wt7hEeHPeYr1iRoHEsJPxqG3wTeYnMbMkM4pYR/jyaYm9YugKn7hNeqe10QBSTA8nyaLCNo0vvZPPt+4Cyzg6yBlYwi7Az5LJZGEazIvuEyC0Y9GkEo8q9Cs1qAYtCnzsKpbhgJ2kpXGQHPv4fg+ID9Lx1W9VFcyny/XLPQnUXGl1pewEYoaX44krShJkuUeKrTjcM7ZgVyvXpniLRKWHHn0FeCktsoOUS1jZR6//3PdGywBOQLl0npBKqpM4ABeWBIBFEMoHwWtiWOFbgHhFCBPPy+uLrUPrqqGnfdK11Jpwf4QfDtDGIV+FmKpPzUkFDhT000Z+qmk6p7AFhWmCX/DkbniyBH8hwmEmdBucz5H0RwDvDJrAo8kSw6dL6KXE2GCu4m/eY/8DMwmYZMK2aUsWShLovg9TjBANo3muB0Rtlr6E/LINJJoP63Jo0YK9XPC+wpmT 1O4sNBH0 7hhCMIcGVwWj7QbX35CvRPfcuYXDqsdONE3GHTnohcgscc5j1O9QGuRwJXMJWE3CeQHUR/K7jSDiEuab1yrIOURPsm9WUxUZu4/wC7FYpyttYEf3ONcghJCKBvz9hb1gZYSYZ+73SJBlljMqhWFidXGHoUktWHsZ93Fz4GGK1XRFiMjhq0XvEjPvuRLufIT2AcC8q2ii1T7FPmV+4jnxWRhP80atMiS//SXbgJz94Vd9+F/U4/L+PCWnoj6utPXPRhCoQ8LUXeF5NTB3czPweYNSzWW3xKpvLlQAIp7DBjSgpJj7yfgrdy4lnIg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Shmem will support large folio allocation [1] [2] to get a better performance, however, the memory reclaim still splits the precious large folios when trying to swap out shmem, which may lead to the memory fragmentation issue and can not take advantage of the large folio for shmeme. Moreover, the swap code already supports for swapping out large folio without split, hence this patch set supports the large folio swap out for shmem. Note the i915_gem_shmem driver still need to be split when swapping, thus add a new flag 'split_large_folio' for writeback_control to indicate spliting the large folio. [1] https://lore.kernel.org/all/cover.1717495894.git.baolin.wang@linux.alibaba.com/ [2] https://lore.kernel.org/all/20240515055719.32577-1-da.gomez@samsung.com/ Signed-off-by: Baolin Wang Signed-off-by: Hugh Dickins --- drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 1 + include/linux/writeback.h | 4 +++ mm/shmem.c | 12 ++++++--- mm/vmscan.c | 32 ++++++++++++++++++----- 4 files changed, 38 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c index c5e1c718a6d2..c66cb9c585e1 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c @@ -308,6 +308,7 @@ void __shmem_writeback(size_t size, struct address_space *mapping) .range_start = 0, .range_end = LLONG_MAX, .for_reclaim = 1, + .split_large_folio = 1, }; unsigned long i; diff --git a/include/linux/writeback.h b/include/linux/writeback.h index 1a54676d843a..10100e22d5c6 100644 --- a/include/linux/writeback.h +++ b/include/linux/writeback.h @@ -63,6 +63,7 @@ struct writeback_control { unsigned range_cyclic:1; /* range_start is cyclic */ unsigned for_sync:1; /* sync(2) WB_SYNC_ALL writeback */ unsigned unpinned_netfs_wb:1; /* Cleared I_PINNING_NETFS_WB */ + unsigned split_large_folio:1; /* Split large folio for shmem writeback */ /* * When writeback IOs are bounced through async layers, only the @@ -79,6 +80,9 @@ struct writeback_control { */ struct swap_iocb **swap_plug; + /* Target list for splitting a large folio */ + struct list_head *list; + /* internal fields used by the ->writepages implementation: */ struct folio_batch fbatch; pgoff_t index; diff --git a/mm/shmem.c b/mm/shmem.c index 996062dc196b..50aeb03c4d34 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -795,7 +795,6 @@ static int shmem_add_to_page_cache(struct folio *folio, VM_BUG_ON_FOLIO(index != round_down(index, nr), folio); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(!folio_test_swapbacked(folio), folio); - VM_BUG_ON(expected && folio_test_large(folio)); folio_ref_add(folio, nr); folio->mapping = mapping; @@ -1482,10 +1481,11 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) * "force", drivers/gpu/drm/i915/gem/i915_gem_shmem.c gets huge pages, * and its shmem_writeback() needs them to be split when swapping. */ - if (folio_test_large(folio)) { + if (wbc->split_large_folio && folio_test_large(folio)) { +try_split: /* Ensure the subpages are still dirty */ folio_test_set_dirty(folio); - if (split_huge_page(page) < 0) + if (split_huge_page_to_list_to_order(page, wbc->list, 0)) goto redirty; folio = page_folio(page); folio_clear_dirty(folio); @@ -1527,8 +1527,12 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) } swap = folio_alloc_swap(folio); - if (!swap.val) + if (!swap.val) { + if (nr_pages > 1) + goto try_split; + goto redirty; + } /* * Add inode to shmem_unuse()'s list of swapped-out inodes, diff --git a/mm/vmscan.c b/mm/vmscan.c index 96ce889ea3d0..ba7b67218caf 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -628,7 +628,7 @@ typedef enum { * Calls ->writepage(). */ static pageout_t pageout(struct folio *folio, struct address_space *mapping, - struct swap_iocb **plug) + struct swap_iocb **plug, struct list_head *folio_list) { /* * If the folio is dirty, only perform writeback if that write @@ -676,6 +676,16 @@ static pageout_t pageout(struct folio *folio, struct address_space *mapping, .swap_plug = plug, }; + /* + * The large shmem folio can be split if CONFIG_THP_SWAP is + * not enabled or contiguous swap entries are failed to + * allocate. + */ + if (shmem_mapping(mapping) && folio_test_large(folio)) { + wbc.list = folio_list; + wbc.split_large_folio = !IS_ENABLED(CONFIG_THP_SWAP); + } + folio_set_reclaim(folio); res = mapping->a_ops->writepage(&folio->page, &wbc); if (res < 0) @@ -1257,11 +1267,6 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, goto activate_locked_split; } } - } else if (folio_test_swapbacked(folio) && - folio_test_large(folio)) { - /* Split shmem folio */ - if (split_folio_to_list(folio, folio_list)) - goto keep_locked; } /* @@ -1362,12 +1367,25 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, * starts and then write it out here. */ try_to_unmap_flush_dirty(); - switch (pageout(folio, mapping, &plug)) { + switch (pageout(folio, mapping, &plug, folio_list)) { case PAGE_KEEP: goto keep_locked; case PAGE_ACTIVATE: + /* + * If shmem folio is split when writeback to swap, + * the tail pages will make their own pass through + * this function and be accounted then. + */ + if (nr_pages > 1 && !folio_test_large(folio)) { + sc->nr_scanned -= (nr_pages - 1); + nr_pages = 1; + } goto activate_locked; case PAGE_SUCCESS: + if (nr_pages > 1 && !folio_test_large(folio)) { + sc->nr_scanned -= (nr_pages - 1); + nr_pages = 1; + } stat->nr_pageout += nr_pages; if (folio_test_writeback(folio))