From patchwork Tue Jun 18 06:54:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13701836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F9EDC2BA1A for ; Tue, 18 Jun 2024 06:54:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA5186B013A; Tue, 18 Jun 2024 02:54:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B180B6B0144; Tue, 18 Jun 2024 02:54:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8EF5A6B013A; Tue, 18 Jun 2024 02:54:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 65C416B013F for ; Tue, 18 Jun 2024 02:54:35 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 08E79140319 for ; Tue, 18 Jun 2024 06:54:35 +0000 (UTC) X-FDA: 82243096110.11.6F7B8E4 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by imf09.hostedemail.com (Postfix) with ESMTP id 03BA7140002 for ; Tue, 18 Jun 2024 06:54:32 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=jPV10q7o; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf09.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718693666; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yoIYIBJL5vLTtJOJekDK/vHyVKGOuJ66AVuG1N/Iti4=; b=a8SwqYHJJM8ROGNkStKI2rJdMLSFbEo3g8PJpAR+f3e6xHmet7SJzxXgJ1RWm/DzSPLgdg dN2+JkGFG0iygshG3dbvgmRqUyC7e36Vpe4cvlUNjjqe+L1DHAe0CyhsoKeYdqPMntN9bq zj99kBx2Va4s9mvkP7fk7DGWo8u8htc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718693666; a=rsa-sha256; cv=none; b=oU0YEWAYuot/usb+9vvzu0tEy3rEaMhI+7Gym46slj4vMkeyjwn1fUUwFqCfT3XRod8tJ8 eZrgVrwpjUefKHQGF0p/iIjdqBsDAPaVH4nRWq4K6qG/VIM5pJFO7W4DMytCzkLbiaisGK qXpsNGXJ35mTg96lcJb0Z7f0oTMl5us= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=jPV10q7o; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf09.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693670; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=yoIYIBJL5vLTtJOJekDK/vHyVKGOuJ66AVuG1N/Iti4=; b=jPV10q7olMSR8lQVPiAVl3vfo2TAPTkGrKgZXo3xtI56kZoEUXzLPXwZzQvh6nki6PLOEXw94WwZmqvUs/4USnES2yUHRFaJE4DD7xLqPz+v6F1kg7IKhG8NAfmRLKE1qRS4XxzEby7hfRHnSm2LBa+0ZDgXtbcn7yD9ShaN4+A= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R191e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067112;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jKUXF_1718693668; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jKUXF_1718693668) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:28 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/9] mm: vmscan: add validation before spliting shmem large folio Date: Tue, 18 Jun 2024 14:54:13 +0800 Message-Id: <413866f2f2d996cfbb5c906467b36c96b1a82e2d.1718690645.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 03BA7140002 X-Stat-Signature: ribn7eacw7j971u866o41k6a3dnqgtnu X-Rspam-User: X-HE-Tag: 1718693672-96741 X-HE-Meta: U2FsdGVkX18jQsbQykhXZqtHV6U+PVS3+h8ZNxIiqpMvHjlWzklSuhkCFSilfD0pQfKglTf0UgPMElqNK6eJ34QQoluQJR0AeR+2WEU3Dfz1PIGElvGBi/S/i//18/WF2KXshSihG9cyXlW+MnrHp/2Wzksxgyh8dsPzYRSEVuskR174cD6m0ny/DnA2xUkhmBOqoUX/ZZCNwm1ZXXJbt6rOTqyJEw73jpA1n/RVWT3fLxasTJJyAzwbw8DBCLrVjQLFsql6JszRUMI5KArJZq/PzXYPI788tJKwRgZjQKNDaks7esp6TAdqczTq9h7c9QGZDhLjT3BvaZn5tpvJt69vwpQHuVNF0MulsgA2QtOkiTcMWErV+VIgw+mfMtaReFfQJnbZ4bO0l0w1axAlgoqfuru4pGGYmwyxGb0LsetppTlYjQYJ+2U0Q+lHf2NY0i+JiTYlTTNbN9ZrPjOhIUEDWIqKZ0qSTPAjVASg5XWkWKtT5ldf74EzNuRhRvD1GtxVqtUZJG5C2xBN8uwDB8uP4KyljwXSE0HNF2w8/wLaYqZFwtg8Mr1mhhANQAwyuYwlcR87XBfrL2QZI/GXGmXfHgb47LdyxlfCrxecyQIw0FIYq4ANZvQeRMxkwV82PU1nnUk1LcRwzIH6c9iUV3kITmFTygvtqIdXX/xNVGaycW33E+fY47UC1M8BOqL6SCO4lfPQyioWTze/90XXxR7wE6pJgjrjKqjolkxqcqzVtmqxr34nuQczu0iogw2P0jPRm/wKHp54okJA9Wn3C9zo0u8ry6QKP7L3Lb78aNaUlMhYfxKnA+acMAage+kMLXF9Sh8Wv+0w9cmOAFWN/v23+T45p9GZewFvQ2xlOPnDTEjt/7lYXQuOPSkBKJaWPBtXmpw7Ya4WNAQZ5K2m3TgP51wqskeUc0CK0ENHxfnN/lNQTQ6u9ng+Fzgw6QfYPTaS177G2NmZ124D1gI 0RDUa0zi 3go+1JUt+xROKUpeWyxrNAi4RD4c5L4Qi2jN1jwCn7+5FOsNWP0D1SCBxhPrN2wX0eWr1r0ns72YVTXq2wRTVZYVX5txC22V+Lt+nQCW7w9L35H0/4k9Uk8f8+i5wy8g+LMf91z/tghkdDKkcALSQUKubuGXCh9eqfEN0ohaNjZLq1rWsztx0pCmVNNLrCYiw7LE10n2ptyDGk1WKsN551hmbPvc5PiqUa/19ShMkvdCxaIJivscY+UeTIFN+WXaj5sIDhej+QsVmBCw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add swap available space validation before spliting shmem large folio to avoid redundant split, since we can not write shmem folio to the swap device in this case. Signed-off-by: Baolin Wang --- mm/vmscan.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/vmscan.c b/mm/vmscan.c index 1807e5d95dda..61465f92283f 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1237,6 +1237,14 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, } } else if (folio_test_swapbacked(folio) && folio_test_large(folio)) { + + /* + * Do not split shmem folio if no swap memory + * available. + */ + if (!total_swap_pages) + goto activate_locked; + /* Split shmem folio */ if (split_folio_to_list(folio, folio_list)) goto keep_locked; From patchwork Tue Jun 18 06:54:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13701837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FDD6C27C4F for ; Tue, 18 Jun 2024 06:54:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 573E36B0142; Tue, 18 Jun 2024 02:54:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4AA426B0144; Tue, 18 Jun 2024 02:54:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 349D26B0148; Tue, 18 Jun 2024 02:54:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 16E546B0142 for ; Tue, 18 Jun 2024 02:54:36 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C7EE11A03A7 for ; Tue, 18 Jun 2024 06:54:35 +0000 (UTC) X-FDA: 82243096110.28.DABEB84 Received: from out30-101.freemail.mail.aliyun.com (out30-101.freemail.mail.aliyun.com [115.124.30.101]) by imf18.hostedemail.com (Postfix) with ESMTP id 8E89B1C0003 for ; Tue, 18 Jun 2024 06:54:33 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="SdmM4p/h"; spf=pass (imf18.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.101 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718693667; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=A1nuL6+6NmFkbTac9imOaGsyA3d5PGdnrfoW9BMoYYs=; b=WEk1gwZ2y5qjHhun7ZD4ghRnFKCTr4wEhxr1+AAcSmpyKIBB2bpbzvy9xEOSD8dWjElUE+ PY2tVuhtLBRwWKXZoatKM5cVy5bJ/3RgjBS5LVOfZUZ3JW3ObC0yyB0mQeGOwMCjZLFJT6 OI+uE/fmPLRBBjxN3Tosq2hJf6eemPQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718693667; a=rsa-sha256; cv=none; b=Z9+DA9cVt0gB0t5xZmvRSFf5OxOJxolmksCyLwu/4u5CBppFiWeRFS/SSOinKz7aUrWJqm LcQHQGDYJh31O9yIAWnjzcCX1/HcZmN7KJzHRjGaGOd2mcKdZVWBnpFlLfANxnuehxynZL cO0pDM32GHFK1J6vX42TC56pvpCgkdI= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="SdmM4p/h"; spf=pass (imf18.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.101 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693671; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=A1nuL6+6NmFkbTac9imOaGsyA3d5PGdnrfoW9BMoYYs=; b=SdmM4p/hL0pBaDI+D+wT/aVXDvnuD6x5K84xMmQFz4Uf9psxbNi3uFDaAVevcy0lwdNiRqmJ9Dx4cIDYkzCGog8zVFzD+UnRRgDBX00WtMKfZ0ouoZqDNpJWLS9QxPxHZ3nnUVEWp0Ft4QVLsSltfear6YK1JDHRaDF0W0PsVW0= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033032014031;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jH7lG_1718693669; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jH7lG_1718693669) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:29 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/9] mm: swap: extend swap_shmem_alloc() to support batch SWAP_MAP_SHMEM flag setting Date: Tue, 18 Jun 2024 14:54:14 +0800 Message-Id: <4909c4bf2f28665c6a35cd7e77990c5defc3d7fb.1718690645.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: zpjsutp4ye7f9n5snumcpc5dhmtq9p8e X-Rspamd-Queue-Id: 8E89B1C0003 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1718693673-193418 X-HE-Meta: U2FsdGVkX19d/6TIH6CNjVBWHc7I6WvIDVwWDT/Fmamqc4fih0UtL3C+JTUWfy2SBm52U+JmKJheNQiYKlasqJXqcjdTwWo98+O8it+aaPHuUThYIkOVEi1IyWkSlZ/jO/v9GAU6oRe2fEo0YA42mKP6LYyu212cIWTuhQyBdKz8x8idSi11F2OI8VVZlAkTegfQPD833i1z8eN00h/526rV92G/hQ+Wcigt6EM85I4u9vc0S3Vu3qwXBJOZsq08vkv2a7hEjfaA9Gy3ntUsqEV+daTMXYpS80SEI44Pp4y70aA7zCBXRCq8yVzmj+BRhd/HewJAs2Fvg+cF0UsCPpYQcdotlHZMxqIBf8GgKIyDufaaTxwbpWlPm4Up+NztHMoVS1iPKfoFxQvM7VWF+Ie3CWLckbN18IwDvTkvTOo4mxld7gcG7re2B19xNxhx5g+7282vDEkmlePgRjP1cMI4EgijbDONHQQSnxpmXZf2qRGOd40cQ9hioqQYOJjFJC/xjUoZOIcpjfNbsQHGYNXyEPSz2pybCM7B1K+uZFlSd3HkKlMXWNxWS0nWDSI2yCXhhUx1f94XSUS7+gaMXZV9qhmcaAxMnvLaA6xNYWv9MrmyyGBeElrAznV3yIFNKGhIee0IwBpBe5V+1NPl0rkL6JmwKmx8c9q4Td+ZzKRvWgZQ6pJc0nFLPB9HQS9BBTupCmWlB6UjEAdf+WmEA+dVb8FTKTuoJEIyKC8KvCkaC1am5JhxDHOzyi+A/tm2Udl/fno3FdteWOFfIhUPb6BEd6YGb8i4tn8kOo54GuanHmCrlZw9MyWEeYHAbBYgap+HIXTLrQG3rFyZr6QDtTHgNMFu5mZXD0hjKgw1bZePWUBErX+pja7uWjJihbbwiKnLGTIum9xkPogl9gZ7Lh0aglgL/WtXOI0IwC0W+G2Hcu37inuNYEZ6g+/moIDdAFzbEA8v1O2dCyi34Dq rO/bE9Hk NZm8c619ZNitZepVFj8vLi7ttj+CgmbpcNagB+pGcyg+negv9igh7eMzhh/mMwo0BaXUAGqYlKkgG4oxllp5XramR3Tq8XhIet4hyQ+VMREeuiAc8dSZ8dhyb4/KjxkI/nE7l9gPVodAmEj6Da1p+Su8+nFczcqj78ALW3pHFbqBJjHFtbUKMn6URPrOjSTU1s947zxUkpHdrLcZLuOsziH5nffElcFRuCKUwC8iIpl1bLujB8y1P9FRnpwXIeFtRl6klCiNWsTXPRfHDkUYcMemng/gceQqLQx1p X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support shmem large folio swap operations, add a new parameter to swap_shmem_alloc() that allows batch SWAP_MAP_SHMEM flag setting for shmem swap entries. While we are at it, using folio_nr_pages() to get the number of pages of the folio as a preparation. Signed-off-by: Baolin Wang --- include/linux/swap.h | 4 +- mm/shmem.c | 6 ++- mm/swapfile.c | 98 +++++++++++++++++++++++--------------------- 3 files changed, 57 insertions(+), 51 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index d33ce740b695..bffb2281840d 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -474,7 +474,7 @@ void put_swap_folio(struct folio *folio, swp_entry_t entry); extern swp_entry_t get_swap_page_of_type(int); extern int get_swap_pages(int n, swp_entry_t swp_entries[], int order); extern int add_swap_count_continuation(swp_entry_t, gfp_t); -extern void swap_shmem_alloc(swp_entry_t); +extern void swap_shmem_alloc(swp_entry_t, int); extern int swap_duplicate(swp_entry_t); extern int swapcache_prepare(swp_entry_t); extern void swap_free_nr(swp_entry_t entry, int nr_pages); @@ -541,7 +541,7 @@ static inline int add_swap_count_continuation(swp_entry_t swp, gfp_t gfp_mask) return 0; } -static inline void swap_shmem_alloc(swp_entry_t swp) +static inline void swap_shmem_alloc(swp_entry_t swp, int nr) { } diff --git a/mm/shmem.c b/mm/shmem.c index d09c6bf1f28a..b90965486631 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1432,6 +1432,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb); swp_entry_t swap; pgoff_t index; + int nr_pages; /* * Our capabilities prevent regular writeback or sync from ever calling @@ -1464,6 +1465,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) } index = folio->index; + nr_pages = folio_nr_pages(folio); /* * This is somewhat ridiculous, but without plumbing a SWAP_MAP_FALLOC @@ -1516,8 +1518,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) if (add_to_swap_cache(folio, swap, __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN, NULL) == 0) { - shmem_recalc_inode(inode, 0, 1); - swap_shmem_alloc(swap); + shmem_recalc_inode(inode, 0, nr_pages); + swap_shmem_alloc(swap, nr_pages); shmem_delete_from_page_cache(folio, swp_to_radix_entry(swap)); mutex_unlock(&shmem_swaplist_mutex); diff --git a/mm/swapfile.c b/mm/swapfile.c index 9c6d8e557c0f..1dde413264e2 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -3362,62 +3362,58 @@ void si_swapinfo(struct sysinfo *val) * - swap-cache reference is requested but the entry is not used. -> ENOENT * - swap-mapped reference requested but needs continued swap count. -> ENOMEM */ -static int __swap_duplicate(swp_entry_t entry, unsigned char usage) +static int __swap_duplicate(struct swap_info_struct *p, unsigned long offset, + int nr, unsigned char usage) { - struct swap_info_struct *p; struct swap_cluster_info *ci; - unsigned long offset; unsigned char count; unsigned char has_cache; - int err; + int err, i; - p = swp_swap_info(entry); - - offset = swp_offset(entry); ci = lock_cluster_or_swap_info(p, offset); - count = p->swap_map[offset]; - - /* - * swapin_readahead() doesn't check if a swap entry is valid, so the - * swap entry could be SWAP_MAP_BAD. Check here with lock held. - */ - if (unlikely(swap_count(count) == SWAP_MAP_BAD)) { - err = -ENOENT; - goto unlock_out; - } - - has_cache = count & SWAP_HAS_CACHE; - count &= ~SWAP_HAS_CACHE; - err = 0; - - if (usage == SWAP_HAS_CACHE) { + for (i = 0; i < nr; i++) { + count = p->swap_map[offset + i]; - /* set SWAP_HAS_CACHE if there is no cache and entry is used */ - if (!has_cache && count) - has_cache = SWAP_HAS_CACHE; - else if (has_cache) /* someone else added cache */ - err = -EEXIST; - else /* no users remaining */ + /* + * swapin_readahead() doesn't check if a swap entry is valid, so the + * swap entry could be SWAP_MAP_BAD. Check here with lock held. + */ + if (unlikely(swap_count(count) == SWAP_MAP_BAD)) { err = -ENOENT; + break; + } - } else if (count || has_cache) { + has_cache = count & SWAP_HAS_CACHE; + count &= ~SWAP_HAS_CACHE; + err = 0; + + if (usage == SWAP_HAS_CACHE) { + /* set SWAP_HAS_CACHE if there is no cache and entry is used */ + if (!has_cache && count) + has_cache = SWAP_HAS_CACHE; + else if (has_cache) /* someone else added cache */ + err = -EEXIST; + else /* no users remaining */ + err = -ENOENT; + } else if (count || has_cache) { + if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) + count += usage; + else if ((count & ~COUNT_CONTINUED) > SWAP_MAP_MAX) + err = -EINVAL; + else if (swap_count_continued(p, offset + i, count)) + count = COUNT_CONTINUED; + else + err = -ENOMEM; + } else + err = -ENOENT; /* unused swap entry */ - if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) - count += usage; - else if ((count & ~COUNT_CONTINUED) > SWAP_MAP_MAX) - err = -EINVAL; - else if (swap_count_continued(p, offset, count)) - count = COUNT_CONTINUED; - else - err = -ENOMEM; - } else - err = -ENOENT; /* unused swap entry */ + if (err) + break; - if (!err) - WRITE_ONCE(p->swap_map[offset], count | has_cache); + WRITE_ONCE(p->swap_map[offset + i], count | has_cache); + } -unlock_out: unlock_cluster_or_swap_info(p, ci); return err; } @@ -3426,9 +3422,12 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage) * Help swapoff by noting that swap entry belongs to shmem/tmpfs * (in which case its reference count is never incremented). */ -void swap_shmem_alloc(swp_entry_t entry) +void swap_shmem_alloc(swp_entry_t entry, int nr) { - __swap_duplicate(entry, SWAP_MAP_SHMEM); + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); + + __swap_duplicate(p, offset, nr, SWAP_MAP_SHMEM); } /* @@ -3440,9 +3439,11 @@ void swap_shmem_alloc(swp_entry_t entry) */ int swap_duplicate(swp_entry_t entry) { + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); int err = 0; - while (!err && __swap_duplicate(entry, 1) == -ENOMEM) + while (!err && __swap_duplicate(p, offset, 1, 1) == -ENOMEM) err = add_swap_count_continuation(entry, GFP_ATOMIC); return err; } @@ -3457,7 +3458,10 @@ int swap_duplicate(swp_entry_t entry) */ int swapcache_prepare(swp_entry_t entry) { - return __swap_duplicate(entry, SWAP_HAS_CACHE); + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); + + return __swap_duplicate(p, offset, 1, SWAP_HAS_CACHE); } void swapcache_clear(struct swap_info_struct *si, swp_entry_t entry) From patchwork Tue Jun 18 06:54:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13701840 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BAD0C27C4F for ; Tue, 18 Jun 2024 06:54:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2155A6B014C; Tue, 18 Jun 2024 02:54:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 176146B0154; Tue, 18 Jun 2024 02:54:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 03C8C6B0155; Tue, 18 Jun 2024 02:54:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D361E6B014C for ; Tue, 18 Jun 2024 02:54:38 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9C42CA1F76 for ; Tue, 18 Jun 2024 06:54:38 +0000 (UTC) X-FDA: 82243096236.01.B18A079 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by imf18.hostedemail.com (Postfix) with ESMTP id 959CA1C000E for ; Tue, 18 Jun 2024 06:54:36 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="yRYct/UF"; spf=pass (imf18.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718693673; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eA6kyRQ9XEpy05GdpwvXfCDvN4imeajWIAzdeARsgKo=; b=HPm/6CjfI5qOc/H5vhm5Iw8OXfRAA8UAyyZfAdeKjaP8PTi1/1O1DEonyzMNUieNsonHV5 6aKTEwTwqf+XlXP2C90Y4HtxaqVum2QXdV3C0zzqpgYEZmMUfGmfGeuTdZlqSy+gZbkYGV G1iMAx9tquY2W9UOQ7V6a+aGFohaeUM= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="yRYct/UF"; spf=pass (imf18.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718693673; a=rsa-sha256; cv=none; b=Q0DwaAb5T7YTgzH5tFLp10hEdO2rKs6R8UDcqkiFK+0ZfuVh7vBdgaxSziDGdlbywbj12O 5kn8W3jnY/wAeXSyrXZ3/H7RnwV8qLq9xUWF9kpf4/irPi2ialwgRn0y1t94h13y2b3qlB gw9UaKOrYfuMKxSvTqTcp8RxOy/x4uk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693673; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=eA6kyRQ9XEpy05GdpwvXfCDvN4imeajWIAzdeARsgKo=; b=yRYct/UFaYNmmeZK5qcyA8QOZ3CAsn+smy8uqjBqePugsHlZ44p8OjReZj4Xu5pUhK+XvsQ9P7e4q6fPIbCx5rSgGPfU09vJUsnvUnFKZLhPFiI6aTOcuv7WAFYnecuylPRwEOqP8w9uuiH0IHfX+QYLZI9M0w3GKBDi/+BSEMg= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R531e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045046011;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jH7ld_1718693670; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jH7ld_1718693670) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:30 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/9] mm: shmem: extend shmem_partial_swap_usage() to support large folio swap Date: Tue, 18 Jun 2024 14:54:15 +0800 Message-Id: <61c49050a8aeffecade8c2c407280bf6b92f359e.1718690645.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 959CA1C000E X-Stat-Signature: cn8yscf3zerkdfd9m55d55ndcdyosgpo X-Rspam-User: X-HE-Tag: 1718693676-353167 X-HE-Meta: U2FsdGVkX1+F6ZZgJVnmnVoLWDEuZckssA1NrcB0oP9PHAbZEHu/G5gciC/VpON1V/z+vQn3bINc7nYvsWJg10phsnFUiYIOo8p2RY5rQFShoWaCadsa1UoQDfB6MxUrbFdedFqfcv11ZPrapz94pBxDSh+H/fpnpWPFJyL3B75HkpAARP7+SPwrNMHwFJ4NGiZrGhf3aupQ6OW1X7Yqf4TPama4T0kPtmrFRa7e20wgctpTLZfCAZbgifyJGEfkWeVRb2S99SGPf8DftjQLeSOw5yQbRC5QWvfcIoApN/rhO5oQhsBAk9aDBqNkEdGi30Ayli7Yg9uyxDHmXVc8QmpzDLQtGdMXc22+Hb2TtJ83r/FGr8uupQt8LkWvrbRc21VZH3/v97/mfb1QuCRUjOPOAnDiH6TNsGbiHShyTiu/uBYNwZcp31eSwzwtkKOFRlTX1nr101Una9v3b4qJEZJ63AG+5veKyq3mUta2xKVCaesVzwFjROrtTvgrRWSt+rPLAS9OT6oIhkGeditGqIybDYgFPnNRgc86/nqakcQEejVlPzWHuDn2xJmHeLEiKJu3xHVBMaF9KwLkgZz1Da6+r6I1aHxaLWPKmY5QLaTPiNxdAwnAsf0OcClQOMGBDOQxEblr/2snciegOrL1XZqaH+EbjlCbWwkNUCH0HbyVIepxNS0+3SQDXlmq8oyxTQr8koLfnu8mphPEF/WtA5IAygiMyOPB8YJttQVZcpP8F1v9b/svRvLL0tCgHysI3ujL+tYV5gQw1r18wueAxY/IXivmqib827r7/WohvY9c7I7aXEbb0DmGYj4YDPbTB6RN1uH4+W26pQMmztoF+bDzdPNnLWFvlpvLC+T9SdhfM+n1TtjeatiXP3J6FsZh8gJe8dbw+3sYTQViof+zKkYSVwyWSlHoVK3SU/OMXP6QRMovFKbn4dbyCshlYP7J9FVjUF+2rPAv7xshgsz 1cGuJfO/ f+rG55lQW9fwyb9hQc/3r8soGgoSPciw8q9NiyLP1FyXQSQVSXUCyHSnHOwExTkuh9kNUbVOIGQVLyPYL9B39t7G3i4Aodbo8qNwVvhegXGyBUka/KMql8bII/ZRDfsBBy9Ab3zXm8vyLfC7fr2/twocaMP2TUh3djKnfCW7eXSlPNdZjQQ7qcB2lmkYjSKDq1297xzwXMy6iWbpIoNjGlsfgEgrhwMCH6IG0NSjp8x1La5lYi7XYIUt/bJ0Zlgz54lY0pQEXr8FJDKjVR4SGj3SSEQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support shmem large folio swapout in the following patches, using xa_get_order() to get the order of the swap entry to calculate the swap usage of shmem. Signed-off-by: Baolin Wang --- mm/shmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index b90965486631..012a06ef39aa 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -870,7 +870,7 @@ unsigned long shmem_partial_swap_usage(struct address_space *mapping, if (xas_retry(&xas, page)) continue; if (xa_is_value(page)) - swapped++; + swapped += 1 << xa_get_order(xas.xa, xas.xa_index); if (xas.xa_index == max) break; if (need_resched()) { From patchwork Tue Jun 18 06:54:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13701838 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 217C9C27C4F for ; Tue, 18 Jun 2024 06:54:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 266D86B0148; Tue, 18 Jun 2024 02:54:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C4E96B014A; Tue, 18 Jun 2024 02:54:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 015F08D0001; Tue, 18 Jun 2024 02:54:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D09166B0148 for ; Tue, 18 Jun 2024 02:54:37 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7F240804BA for ; Tue, 18 Jun 2024 06:54:37 +0000 (UTC) X-FDA: 82243096194.27.D249057 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by imf27.hostedemail.com (Postfix) with ESMTP id 702DE40018 for ; Tue, 18 Jun 2024 06:54:35 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=kzN8QKL4; spf=pass (imf27.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718693669; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oU/PcFIl06vSDqEtjZujhmAzGqRrrYre+wfF0/f2UwU=; b=MFTkQN6RQf9dZQDcvlMQFSHeJ2tbptirGhfdhe5jAg4MhyJCpjXjk8WMiR0bJMa3eRi5i3 mDqfUejIJhhDFswRGQOUaJvKL3Qf2gKlpbITZN7mIDTHxy3bKhl595fu9UlMAUWevVdch4 NQj11SYAtdtZcnplHH47WG6hgRt7uxg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718693669; a=rsa-sha256; cv=none; b=yy7RD1w7LGwNoqQvXrFmqIo/BYwBKjQL3p2yFjw7rYe/7FzmmnI5V0zVzVLslhL5ssxYDv u4DZXyzLN2dwk/lzErEMbgXpDrfVaHtzPOuuTiuZr6lMc86BIhL9WZMdnGnriHxFmXkFJm EZ7ctNStBnlgEV1itDmpyIvJJarvEC4= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=kzN8QKL4; spf=pass (imf27.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693672; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=oU/PcFIl06vSDqEtjZujhmAzGqRrrYre+wfF0/f2UwU=; b=kzN8QKL4HqHEkDyChBPWZYhwgAemYiLzM/NES7nFLQsMialpNZY3Mvpl1CdyuBfpsc1u8u6gPdIFi0suB/cMYPAv2fvn/xLy0rxZM7yqwFBe136kCncXEVnohlr2shwnHj4528Z2EgYf1KismQSB5b5/idVp0u83tzJcz6tnBaU= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R861e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045075189;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jFFL._1718693671; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jFFL._1718693671) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:31 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 4/9] mm: shmem: return number of pages beeing freed in shmem_free_swap Date: Tue, 18 Jun 2024 14:54:16 +0800 Message-Id: <6e567bfbb6bac0bfbdd63e60718bd1aaf8eade6c.1718690645.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 702DE40018 X-Stat-Signature: r3znm9y1z7x9o7suptzkncqcojyhymgp X-HE-Tag: 1718693675-957502 X-HE-Meta: U2FsdGVkX19glpTEPLxhF+5R8jS1OlhDADcI+IiwIJ5WIfNfgUhCfJmlWFxuGxAvmQhoI3CTZ6rMX73VdQPFLhtIuf8ZmaLis2FPlTXtigtqbzOD4g0Da7a0yOaMwrSfUApN3JqYWzHmV6r85V3I5eNwiqb3zqrVd+6ySWrKKvTbYBe/II/bw1P3OSIZK6R7EYiLpgz7m9L/8m7lYikvOWFt3ZbH+UfaQKCobLz8JKyX0WiHj0nLoDwdapkIRJBgtDLrvIlvXujo0HlivuauXmt40t+HiT0wPpKhyBaL/V+XD2/L7+YTAc1uJn2KpJv7fg+gwGk5Z0lF6TadFnJ4jmyQAz4rFmJGrjmpfe4LEKZSVyssrHyawCyQFr5vKa4080luCafJ+rucmBYvFgsjWRQM5f4HglzqBKT17ZN2HkzfmkbeHfvDzPBjXwpxZQyTv2R7ju6aV2IbcB9dbkdCwnEzpK6mPioY5zlAPbyRGi2vt9+KImy5KQ/2TxWFBMic9YAaHjQDMfmDIj+QdjCsptg9ocQ/xAMMZtIlSLtQJnXsEGP90B9x2/lFeDQ8WTM+mxYn/Tu91w2TETcAdiIG4ckGU4bEDMqxCUWGsayCDH7RJKyTYJB682vmhcKIsfoZ/9gmJgSRK/QxCrpGHnU9t0+bupGOroCjjwGDg8cL1x+sYEccBEWUI0IS6WxHbG5WizJjEo2vmJUYm08G3USxhPWoZZyfYUVD8hILvISWotmNm7l8CNXea8j9vbjptXOf5LZDFMvB2hhTtxdtfgImo3YLK3d3DL5p2Q04YtuVzPVe5TFcRJLXmWaEyeexFfudW0Ka6Hx3tEmD4gdGjnCdHOgUYwJJ4PaO9YteToc39EKanJgRLJ/jpItBp5+8kJLHvzAQCe9PpnDZWix4aQIuPRPhE0TUViymO1pfuhVFWjjkpYTZq+7JBTP3jlvytAxjQE7OIK9ZNHSX694CGSw A9a+QSJ9 53QoNDdnMOIS5V7jhTWvo7eSpDCzdtaZOGu6C4q4cbB9XYINHChlH/Se+BoIw7ZBvHCAXvZ1Y6qvIe4NFIQkTdc2PE744CXUJFKYPFpLnf+l1RGQ+Cl6s1t1yfyecK9q6SJI90UbeOIZ6oqTS3frs6GhpSbEeHCQGBDa9hvTBPzBVXkFaSDPnlvFqD2vkMaMjiWEVaGFFjsL3DDvd82I+baxtzNX+xmVwLpo0EUPBtoBxBJewM3/nRmzFDsZ21MUDWg7UmLztr/tfU27K+gQ705LwWJeNV1ORLx0XwdXvTyC3toZyZJAKW1xvLl8vX8Ox2cs2b4f6zKW0AwgV6Jk3WTSHROXYK3r4OwfCybEjNIIkHbd6DQtorqjABA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Daniel Gomez Both shmem_free_swap callers expect the number of pages being freed. In the large folios context, this needs to support larger values other than 0 (used as 1 page being freed) and -ENOENT (used as 0 pages being freed). In preparation for large folios adoption, make shmem_free_swap routine return the number of pages being freed. So, returning 0 in this context, means 0 pages being freed. While we are at it, changing to use free_swap_and_cache_nr() to free large order swap entry by Baolin Wang. Suggested-by: Matthew Wilcox Signed-off-by: Daniel Gomez Signed-off-by: Baolin Wang --- mm/shmem.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 012a06ef39aa..a73d2da54897 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -836,18 +836,22 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap) } /* - * Remove swap entry from page cache, free the swap and its page cache. + * Remove swap entry from page cache, free the swap and its page cache. Returns + * the number of pages being freed. 0 means entry not found in XArray (0 pages + * being freed). */ -static int shmem_free_swap(struct address_space *mapping, - pgoff_t index, void *radswap) +static long shmem_free_swap(struct address_space *mapping, + pgoff_t index, void *radswap) { + int order = xa_get_order(&mapping->i_pages, index); void *old; old = xa_cmpxchg_irq(&mapping->i_pages, index, radswap, NULL, 0); if (old != radswap) - return -ENOENT; - free_swap_and_cache(radix_to_swp_entry(radswap)); - return 0; + return 0; + free_swap_and_cache_nr(radix_to_swp_entry(radswap), 1 << order); + + return 1 << order; } /* @@ -999,7 +1003,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, if (xa_is_value(folio)) { if (unfalloc) continue; - nr_swaps_freed += !shmem_free_swap(mapping, + nr_swaps_freed += shmem_free_swap(mapping, indices[i], folio); continue; } @@ -1066,14 +1070,17 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, folio = fbatch.folios[i]; if (xa_is_value(folio)) { + long swaps_freed; + if (unfalloc) continue; - if (shmem_free_swap(mapping, indices[i], folio)) { + swaps_freed = shmem_free_swap(mapping, indices[i], folio); + if (!swaps_freed) { /* Swap was replaced by page: retry */ index = indices[i]; break; } - nr_swaps_freed++; + nr_swaps_freed += swaps_freed; continue; } From patchwork Tue Jun 18 06:54:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13701839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDB93C27C4F for ; Tue, 18 Jun 2024 06:54:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 71EFE6B014A; Tue, 18 Jun 2024 02:54:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 630EE6B014C; Tue, 18 Jun 2024 02:54:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 460886B0154; Tue, 18 Jun 2024 02:54:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 29A9F6B014C for ; Tue, 18 Jun 2024 02:54:38 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E3772140338 for ; Tue, 18 Jun 2024 06:54:37 +0000 (UTC) X-FDA: 82243096194.14.ABF9911 Received: from out30-98.freemail.mail.aliyun.com (out30-98.freemail.mail.aliyun.com [115.124.30.98]) by imf24.hostedemail.com (Postfix) with ESMTP id E3B2B18000C for ; Tue, 18 Jun 2024 06:54:35 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=QA0sfXlY; spf=pass (imf24.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.98 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718693673; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OhkcTOgQo6MdSZjxEddX9KsK/pU+Rag0OqwoL0KeI6s=; b=OH1ZijgMRSCgzudF9DKZfif6SRJAL+PVG1yqWdC2/KJakU1XvmrHNybJKHBHhYArhXbfXX zbgSj/eHJvbktdWRjl852V0VXRdQX/Iuq43x2MaUaZAIsbD9jO5KtLQ94tYGldYfFxHiYl Ykf3VmyxtOMh3clTVnxGezchMAorq4w= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=QA0sfXlY; spf=pass (imf24.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.98 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718693673; a=rsa-sha256; cv=none; b=qrem/AKuVimlvSVlllouZuOhAHfS2ekunVOqrlHGxCmmSNpktv0mMzkIIPzhfGUao36ewc sYPifnQ7MVOO0Ul/nnNJBpuqZLeIu/x+aEHydvMyjepaSl8gwUqWzdF3Ry9VxIp1cjxvmi AVzJo0YRoEFHHsRNXTyUn85jShpm5tM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693673; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=OhkcTOgQo6MdSZjxEddX9KsK/pU+Rag0OqwoL0KeI6s=; b=QA0sfXlYV15E15qdzkw0ZovnsnlHgBubvGcUr0c/kim4zdkpUJg2wT3f/uPl/wRVZoXggpCIYnu/cmnl+lXnygF0BSyrAOK/EfSXNSQB9eQY/tPCGA3e7+tQvRiae1GdK06v093ovwVxq4Od6ZDOtIpYTO7XHoaxhNTY1IwpkHQ= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R881e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045046011;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jFFLh_1718693672; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jFFLh_1718693672) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:32 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 5/9] mm: filemap: use xa_get_order() to get the swap entry order Date: Tue, 18 Jun 2024 14:54:17 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: E3B2B18000C X-Stat-Signature: j68hknggw5a86wsrz7xaejidy5zqspn1 X-HE-Tag: 1718693675-966117 X-HE-Meta: U2FsdGVkX1/MrMWD7LBux8s6zzBpVrnckAaNGpWn+o3FPgFwXhOPwVJbNoj9hzxURtv9vwLQQTRxVUqUPYzdhR30vUXOPUhHkMlo5bSMdvHj8Q/js9yWyj3Jnsn3A8OwaemUUD8iSPjwtFFtL3ZvneA/FC3v1zlaJDhuEx4F5n6BBjSHd1cwgUrFnE3p4pZa6TErQSHCPSLq3TCXhbD6zxQq8++HDP2KVl1e6FARd6Xb+goJ7iKbzdvbnSG9nWOp1eeFdPmQUJiswmH9W69mvy68R+8036kVNqO04UYMP7ToP8qwOm1IyaNiy1pPqXwCmTmvB3LtRG5WnJvXAEiwc/XdYWUmqHv/1PK0uy4WzRyH9HeZtYWw7aHj9xRdfX3KFSiFcDzwtn34goCm0ZK05Z+tIjP3DDb1nQh7mblL75SwX7PAHlesyEuqki5xhmh1Z4KBCYdLByelcdpj4x0K3Lf8FLefJNsjOBkgSIkpeKt8dp99TAbZJX3574ywMLEhUApSg2D4ATYWMIm4yfWafuMeNiZAJV8a0n4vdl5F7QhRPWV3FbjZCle1fAWCb/uZ4L/aL4hUxh3gjiHwHf4mAka5R6lsHmIYQ3N9LRT96+skSd8jdJopDGrYKOAfHmmE3OmTqiIWhvQXmkzSQhXKj11oGlhnxqPwzxXLV9w68WNR33jOIr2Mefq0rkwYxXx12kon8l+SQDvTB7pzM7wa1I/DOGMuK5HcrjHIXd9bUtXx2NLQ7QYBHEo0oe76Umx5ayyxTAtfw4O7tXmP62bxYqFFYb/JxZDbG/PGd/2jHjnJYgu9a1hPv9kb34kzaPZPNkC3ZRW9ZKSyNMp9C25bDHs9vLIfXK8Ok/7pIu0bn5smSRgaILW14JrEYG8P8k2sJ8VJhbiSFDZO3FA9o1tm/vnIt8URQUxyNrl7H/08Zd2P11isxCMv28Vg1+ZpRqcNPr8WHmlVy9aElQ8Lo8D YV1YXBOM UV/TMkTg321piM/GAw1kmzW3Tj/TEAJZc6ILiPg4eM/jloLl66DUTMGpqGdHNEqrSxBGUaiDWWnSe7dpmAb2IBpH0NRepACtn9qIl2sLNmktpT1bcULXb3nfGFaLb6YYziLynWIqRpTVxJOKP7e3c2FugobU2JHox0UQtgu4sZZbeX8KOyWjjMBgyy1jmG4D9xvZjdde2VmNv6yDCRkz6FUPfKqRSBxPk45zGXcVZ2OeGWHRMdmuoPDMk1KuvFnoFY+xlk62l2Oavp6tr+flGYuwWIA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the following patches, shmem will support the swap out of large folios, which means the shmem mappings may contain large order swap entries, so using xa_get_order() to get the folio order of the shmem swap entry to update the '*start' correctly. Signed-off-by: Baolin Wang --- mm/filemap.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 876cc64aadd7..ecc785758aaf 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2056,6 +2056,8 @@ unsigned find_get_entries(struct address_space *mapping, pgoff_t *start, folio = fbatch->folios[idx]; if (!xa_is_value(folio)) nr = folio_nr_pages(folio); + else + nr = 1 << xa_get_order(&mapping->i_pages, indices[idx]); *start = indices[idx] + nr; } return folio_batch_count(fbatch); @@ -2120,6 +2122,8 @@ unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start, folio = fbatch->folios[idx]; if (!xa_is_value(folio)) nr = folio_nr_pages(folio); + else + nr = 1 << xa_get_order(&mapping->i_pages, indices[idx]); *start = indices[idx] + nr; } return folio_batch_count(fbatch); From patchwork Tue Jun 18 06:54:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13701842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20E54C27C4F for ; Tue, 18 Jun 2024 06:54:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 246776B0155; Tue, 18 Jun 2024 02:54:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C6956B0157; Tue, 18 Jun 2024 02:54:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6F2E6B0159; Tue, 18 Jun 2024 02:54:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BD09F6B0155 for ; Tue, 18 Jun 2024 02:54:40 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7B17D8047A for ; Tue, 18 Jun 2024 06:54:40 +0000 (UTC) X-FDA: 82243096320.22.4E627C5 Received: from out30-111.freemail.mail.aliyun.com (out30-111.freemail.mail.aliyun.com [115.124.30.111]) by imf08.hostedemail.com (Postfix) with ESMTP id 62461160008 for ; Tue, 18 Jun 2024 06:54:37 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Ez6ad5pk; spf=pass (imf08.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.111 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718693672; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Y0p1KmYvp9sydPHpYYHL11MSmx/npKoL7TRhS1PPQNA=; b=eXovOPiGkewuCKsNS8qDjD7xJrIwNbhnuhz5WkG+z6q2ywT2ECFqlK7bjGmRU/nSWxp/g7 RDMo2241ENQqRRsNU2slZWexsiYbYRuqnv4XvDxVaEZGkkeXOFEvMR0Q9huDRN2ijFvwKD enF/LxF6Tl5hCcHjCUikh+sTncVmX8w= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718693672; a=rsa-sha256; cv=none; b=daGp2H7kqcWUP3j1XfrAnO0ixx3DnV/ysFhMXks+E7xrwfYY9ZXjJikUhvkJBhO2cDy5Ke KApI4Vj/FrRJ/spY894Qu2SCRB985fpNg2Og2ROGcrcUlsBwenqBRfACjUZX7SUnrSJhfm 9UwrXMCuRTBXYQOZSRTQWGg5zocbyU0= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Ez6ad5pk; spf=pass (imf08.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.111 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693674; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=Y0p1KmYvp9sydPHpYYHL11MSmx/npKoL7TRhS1PPQNA=; b=Ez6ad5pkvaQ5CO4OnICZhrFr2C+EuX8PzqZkFFpv4PN26dqCCRE20+Lh91F11AA3ePxYKxyeg1640JhkZZ38fXsnCrcW4eFvA9OUfY9me2UfPvOvsDf8j/l7CkHo0vj6I+NiVjolzeJ9v5Swc2gxzYiFB+/yRzgt1PAesqvlbYA= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R981e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067111;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jKUYu_1718693673; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jKUYu_1718693673) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:33 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 6/9] mm: shmem: use swap_free_nr() to free shmem swap entries Date: Tue, 18 Jun 2024 14:54:18 +0800 Message-Id: <373a0d959930494b83f68169eddb62d0d49a29cf.1718690645.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 62461160008 X-Stat-Signature: abn39d9eai8t496mwsz3s3kbzp8ho8tm X-HE-Tag: 1718693677-131805 X-HE-Meta: U2FsdGVkX1+Pj0haVDKIXHyAlgRRhHqMHigmCTn3z35sK8BBqvbplj1u2u1KAHRJiGPMq/OmJdPNyuQvCnDoubZjksOptPiva76oa5+xfEKHTkU6J/tkge9UfZs7rpUkv5K2fBp0OUr7Kar3cBh2zIX+mtC7m0xyTibaVqvuxOAEfLXlbqVBrZPMOAiOLFSIZuI1CPVl3LV3VldmZb8aeDhj6j1243WQiGtCtwSC0+7CdgG5rnFpShFr3obD3LJXc73tg8NKjPizh6mbP0OE7J4pyfgUD5CrRgN4QmVw3T8ubeA6d9QBQXREX56eE1vTlmpKmzcbQ5VDTO9lFrhVndv7DRcCFdxH0vqeSzgAZfW/r+144n3zzcoVJyz+ZpTZekr8/eh/xs/l4Y6qIxA994mHOjE8MaUsnqzFCqpRgjo6N34F4VeOEl7vw7SpcBNfVI8ThRlfHXESiUYo630khGAYJiaAmIK5ygGtRmVywQx4GXoJaO+3X5mF+ntwPkHqRSVhj1VORZGr0RaacCe3zbj7Zxw+TmSPSGZ1viMfuzgohrfEecu2Tb+26X8NcLtgpWkQA5PvnTmAei/ZTxqBNVlxNa+DHb8CspZAjuPaHrG5aUqRVAdyjKieia+G4Bgwa7pum1rOwWGNUYjmw1p+8QoCnei3GjFJit9CJbmJbDPE5QZZgD9XBwSmXtL/pQZZ45S3+LsdXPeefPCJbZMP/98+feqhXnMejUppeRGjMRBsxmRWah+GNo6Vj9EhARlifIduPwYee37LlIusm3ufCfxZYGocA7m37O6bHehdDIzk+5Bg39AWsqu/r8DqGHAg60LNVIdawWeQ9ikCpCviykPISorU954ZGFgazT+stst6KkFLNBv05YzvQlZURJt8McOnbdt2cyC8XmTZqbTGyXO/3dAHcTH4KTT9afvwMG19p/mmGLzqNnq3aNL18otrsDgcuxP0HUf7OuMG4Sg YIPMcNUI JSAJNQ7JwarFxy4TWe0raLmQGWMzsG7U0le8wgDwssbeaNfrDs3RfpjFQYIieJJhZwiIOiyRQ7c4NoH4ZaEWSFiFjss2qt4OV8h02xYj9BkwcVg0H+sQUbgpm03ofNZv/S0BD+56mO9UseQY87IOf51RKsERe3YTORmhPOuKHIt+EAeFqhWwpERfvZzzHDeXI2wxxc49mrmCUbEt19j7Kwgu9+uBhOrGNS4NxzibkP/IaVO/D53y60ftMCn44ymV4zn7tI+mjbUuNpICx8UdYqAliJV4ylxpufmt6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As a preparation for supporting shmem large folio swapout, use swap_free_nr() to free some continuous swap entries of the shmem large folio when the large folio was swapped in from the swap cache. In addition, the index should also be round down to the number of pages when adding the swapin folio into the pagecache. Signed-off-by: Baolin Wang --- mm/shmem.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index a73d2da54897..4d7996962388 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1950,6 +1950,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, struct address_space *mapping = inode->i_mapping; swp_entry_t swapin_error; void *old; + int nr_pages; swapin_error = make_poisoned_swp_entry(); old = xa_cmpxchg_irq(&mapping->i_pages, index, @@ -1958,6 +1959,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, if (old != swp_to_radix_entry(swap)) return; + nr_pages = folio_nr_pages(folio); folio_wait_writeback(folio); delete_from_swap_cache(folio); /* @@ -1965,8 +1967,8 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, * won't be 0 when inode is released and thus trigger WARN_ON(i_blocks) * in shmem_evict_inode(). */ - shmem_recalc_inode(inode, -1, -1); - swap_free(swap); + shmem_recalc_inode(inode, -nr_pages, -nr_pages); + swap_free_nr(swap, nr_pages); } /* @@ -1985,7 +1987,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, struct swap_info_struct *si; struct folio *folio = NULL; swp_entry_t swap; - int error; + int error, nr_pages; VM_BUG_ON(!*foliop || !xa_is_value(*foliop)); swap = radix_to_swp_entry(*foliop); @@ -2032,6 +2034,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, goto failed; } folio_wait_writeback(folio); + nr_pages = folio_nr_pages(folio); /* * Some architectures may have to restore extra metadata to the @@ -2045,19 +2048,20 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, goto failed; } - error = shmem_add_to_page_cache(folio, mapping, index, + error = shmem_add_to_page_cache(folio, mapping, + round_down(index, nr_pages), swp_to_radix_entry(swap), gfp); if (error) goto failed; - shmem_recalc_inode(inode, 0, -1); + shmem_recalc_inode(inode, 0, -nr_pages); if (sgp == SGP_WRITE) folio_mark_accessed(folio); delete_from_swap_cache(folio); folio_mark_dirty(folio); - swap_free(swap); + swap_free_nr(swap, nr_pages); put_swap_device(si); *foliop = folio; From patchwork Tue Jun 18 06:54:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13701841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEA00C2BA15 for ; Tue, 18 Jun 2024 06:54:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 80A806B0154; Tue, 18 Jun 2024 02:54:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7B78B6B0155; Tue, 18 Jun 2024 02:54:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 631D76B0157; Tue, 18 Jun 2024 02:54:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 413E76B0154 for ; Tue, 18 Jun 2024 02:54:40 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BFFADA0496 for ; Tue, 18 Jun 2024 06:54:39 +0000 (UTC) X-FDA: 82243096278.05.638C693 Received: from out30-100.freemail.mail.aliyun.com (out30-100.freemail.mail.aliyun.com [115.124.30.100]) by imf09.hostedemail.com (Postfix) with ESMTP id AF7A914000D for ; Tue, 18 Jun 2024 06:54:37 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Udpbvqa8; spf=pass (imf09.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.100 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718693673; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=koLTHCsWXBAa3N27f7lrGDpaxVfvYWdW4ehMIBy6BWA=; b=dxGfe6wl+cEMKGgnYRnVxTYr+UJAYS+rRxx1HHeqdAtI2YM81pbQCoxX0btvn1JpakCoST cMXhEE0Bem9NLA+vMBobW56/BeMVgPdw8NPe1rOE4mfx6/+bvHEw0O3ntXn6l4vftDTpx7 F14m6zNqRDyJxxznxek/A6eRFpv3vE0= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Udpbvqa8; spf=pass (imf09.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.100 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718693673; a=rsa-sha256; cv=none; b=MXUoyOyctbJUGRqtnKEEKbmNXrQ0JRKmB5JBpQdFcbrLtDjRcFtqcaDXFHN5IRVv4qzOen UFgviahliJF0eHwFn0ypezQn+TO38DYxx/uoC/5eOEMhMPHQONYIIdM1y5g+wfdxyZ6bx5 dJE7uZGPWIIiR7FgKBweSudPVodbxRA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693675; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=koLTHCsWXBAa3N27f7lrGDpaxVfvYWdW4ehMIBy6BWA=; b=Udpbvqa8C9ywNR4Yd0WabC/RIfeEcOx2FL0N5jolBWFkzUk+OmGThUyCtZS05ueDOQBB6eFmtN8m82W9tO8H3jmWA3WXyrYhWB6n3J6QokY6xreIxVq1Xf8BdWjVkEYE5bm9Nn4RLcKV8SlML61FYBl2Xg7RJgUtryQdtEqqqfM= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045046011;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jKUZG_1718693674; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jKUZG_1718693674) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:34 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 7/9] mm: shmem: support large folio allocation for shmem_replace_folio() Date: Tue, 18 Jun 2024 14:54:19 +0800 Message-Id: <75d1c1f2fbc88dce476a53fa90f085b3bebb83c5.1718690645.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: jsukdhgiwx56pnu5t36emzeku9yy8oyi X-Rspam-User: X-Rspamd-Queue-Id: AF7A914000D X-Rspamd-Server: rspam02 X-HE-Tag: 1718693677-315579 X-HE-Meta: U2FsdGVkX1+Monm45quVjmmIXiwxYuL/XDjtI/P8UKwoQ998hWUDHMya65zuFbkuzVYv5avyzccWMwPecdv9bOY1pyz6BuDlXQT/pp6FnBquRFgUlPUv6MBfU+Z6BzJczTPe8Rzem/9fN5IkOOPrjXCEgqqMCaDzIMYtorDuTPq+ykRskGYb1rsWu5E3uFz+lnTmnqoTlXPLKSuSnW0f70AnsMLzZ/HK15X/OzMg3HYuxeYG23IWY31PSgcJRCkjqn3E+yyD68zFn+UA2gZUQuOD8CRzwB7zxz1ilx5+UKbUZOSNx+PGiUM6+2/YHTYaQ2SGQQ0x+IbyTe9Ruu/wL/XFrJ5Bs++kBr+nTGL19cHSoYezzDnR0yELyuS1aRAfIFcuhYpRf9qcsw5BqMFSKO6Z+wUnQqLqBQJ6MNDBSk6cXQjT4Guuq0vQjhGoJvaPC9YtRUxSZoPVoBs0zjG/28dxc6vKIMvv1C62dOBBWwZsGECPoaXTyaePoqbGRxaTQDwqVCzm0S6v5/4tWd20zcIrNRiMXj8aJKeYIMzGQ8IuHs9VkM5W2q58+U2Efm32tIZo0rIJi/pKIC5RwkTfkbpaAPthgARZxruG96frpJR0SbVGpHidKkDduUSgOoF69mIBNXv+LdPLG814aK3dLwiwHSWAEi2svU99ysm2x86ODw6p3CJIf8WPPmLVra4uL3wBFpjWHOIQNcOZHGY89fTnpVBBlzKHPXAsNMpBAJ1mB12Rrj73NAtu3485gDGbR/Qm432/evOYKWus8Db/sDmwJHEMYrjfQqJsJx8C0igrArSvGjOkuSJMGNZlgl/KCE00H5VP5G8+L7H22ubVOMYyBygYY9+g3ZSZws/2S07uprNQj9aUcoVkzjuXzFI9NoGZ8wvQ3E7aRsZKINooSEeYaizDW1FMuEh0D6gU+qWi7ZqFEcDAuitqQWoboKHqbLEgFHfhWgkezazAFld NXFcsn4t 1jcHEsAYdJ7GX2eqp9pn60gZ7xmAOXvGLp+B84x25RK4XxMKrxk2Hd4bEzXU5zF/nSzeF7L/uBuw66o1rnMmrTmXLdHQHc+nWC1b0x0A51mJ1tkxCQGvtbW9XMvtpZQIFku/QdWsM66FLjysxYwYCtozz6BNg2pq08zU49ASeaE8rQ/WhTw7IQ/2Trqg27XSFhABtC1ulpM6bE5+R09wwxwBkekspvufkmcr2o7Y4PsANCEPmq5vK/IhMGanx6loavshV0l+LORjXT8w78WtQvDAYHZfbkLMzeIlh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support large folio swapin for shmem in the following patches, add large folio allocation for the new replacement folio in shmem_replace_folio(). Moreover large folios occupy N consecutive entries in the swap cache instead of using multi-index entries like the page cache, therefore we should replace each consecutive entries in the swap cache instead of using the shmem_replace_entry(). As well as updating statistics and folio reference count using the number of pages in the folio. Signed-off-by: Baolin Wang --- mm/shmem.c | 53 ++++++++++++++++++++++++++++++----------------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 4d7996962388..c0a9253f3a99 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1878,28 +1878,24 @@ static bool shmem_should_replace_folio(struct folio *folio, gfp_t gfp) static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, struct shmem_inode_info *info, pgoff_t index) { - struct folio *old, *new; - struct address_space *swap_mapping; - swp_entry_t entry; - pgoff_t swap_index; - int error; - - old = *foliop; - entry = old->swap; - swap_index = swap_cache_index(entry); - swap_mapping = swap_address_space(entry); + struct folio *new, *old = *foliop; + swp_entry_t entry = old->swap; + struct address_space *swap_mapping = swap_address_space(entry); + pgoff_t swap_index = swap_cache_index(entry); + XA_STATE(xas, &swap_mapping->i_pages, swap_index); + int nr_pages = folio_nr_pages(old); + int error = 0, i; /* * We have arrived here because our zones are constrained, so don't * limit chance of success by further cpuset and node constraints. */ gfp &= ~GFP_CONSTRAINT_MASK; - VM_BUG_ON_FOLIO(folio_test_large(old), old); - new = shmem_alloc_folio(gfp, 0, info, index); + new = shmem_alloc_folio(gfp, folio_order(old), info, index); if (!new) return -ENOMEM; - folio_get(new); + folio_ref_add(new, nr_pages); folio_copy(new, old); flush_dcache_folio(new); @@ -1909,18 +1905,24 @@ static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, new->swap = entry; folio_set_swapcache(new); - /* - * Our caller will very soon move newpage out of swapcache, but it's - * a nice clean interface for us to replace oldpage by newpage there. - */ + /* Swap cache still stores N entries instead of a high-order entry */ xa_lock_irq(&swap_mapping->i_pages); - error = shmem_replace_entry(swap_mapping, swap_index, old, new); + for (i = 0; i < nr_pages; i++) { + void *item = xas_store(&xas, new); + + if (item != old) { + error = -ENOENT; + break; + } + + xas_next(&xas); + } if (!error) { mem_cgroup_replace_folio(old, new); - __lruvec_stat_mod_folio(new, NR_FILE_PAGES, 1); - __lruvec_stat_mod_folio(new, NR_SHMEM, 1); - __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -1); - __lruvec_stat_mod_folio(old, NR_SHMEM, -1); + __lruvec_stat_mod_folio(new, NR_FILE_PAGES, nr_pages); + __lruvec_stat_mod_folio(new, NR_SHMEM, nr_pages); + __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -nr_pages); + __lruvec_stat_mod_folio(old, NR_SHMEM, -nr_pages); } xa_unlock_irq(&swap_mapping->i_pages); @@ -1940,7 +1942,12 @@ static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, old->private = NULL; folio_unlock(old); - folio_put_refs(old, 2); + /* + * The old folio are removed from swap cache, drop the 'nr_pages' + * reference, as well as one temporary reference getting from swap + * cache. + */ + folio_put_refs(old, nr_pages + 1); return error; } From patchwork Tue Jun 18 06:54:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13701843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41788C27C4F for ; Tue, 18 Jun 2024 06:55:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 91B8A6B0159; Tue, 18 Jun 2024 02:54:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8CBEE8D0001; Tue, 18 Jun 2024 02:54:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F3F36B015B; Tue, 18 Jun 2024 02:54:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4A56F6B0157 for ; Tue, 18 Jun 2024 02:54:41 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 07BD71604EC for ; Tue, 18 Jun 2024 06:54:41 +0000 (UTC) X-FDA: 82243096362.01.B9856AB Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by imf29.hostedemail.com (Postfix) with ESMTP id 0BA58120005 for ; Tue, 18 Jun 2024 06:54:38 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=gyAs4VwM; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf29.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718693674; a=rsa-sha256; cv=none; b=DFQxAVGMr8hN9ptIaUseSZu1K2dBftER3oawoXDD17oPpFetFyaSBp3vLjKRy1w4o1Ix6/ SioOqpqa4IBF7fjw+Jb92a+UoFMruSM8ZVQpd6/9XLVbVM1onOtJHEXVM0AP7gFvcXCfyI OnKr+4FsyhNiJUiMMu2KOnFx+k+tmjs= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=gyAs4VwM; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf29.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718693674; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BtIWGiPeiPtgyrkhSZFohxCr8LroRfij9iZfhgtdvvw=; b=fY7D21pov0MxVL2auQPelVwcqHPLomDKfIz4OzNsABIvwk3OpkRrvcU1yHxqZnM/PlMLTx xXfMTgiklCrxEPMmccugOeNLkDe/GejBq40sX51ZLCm/35FJk/fFU6Iq++f2cWvsw2HcSb EHKXPemQYLHqPzwhV8yzWLGt0tB6MzA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693676; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=BtIWGiPeiPtgyrkhSZFohxCr8LroRfij9iZfhgtdvvw=; b=gyAs4VwMbBvh5u3NgjIpZNLJC1Fo5FHERWRYo8JvDldXw1EP3eGTh9FovWARVvEbQpHsQLa54S8j6Bixdq2yockcs6r3ze1ZfxqmXGQ/p6erAZGqZ4tC6ECuWS/YilUVtYTG72BbDWGCdN6CyXf71gOOPFc4GDwxgqSyBsSN+No= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033068173054;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jFFMY_1718693675; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jFFMY_1718693675) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:35 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 8/9] mm: shmem: drop folio reference count using 'nr_pages' in shmem_delete_from_page_cache() Date: Tue, 18 Jun 2024 14:54:20 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 0BA58120005 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 4n33prpndsjtbbsr8p5cs8898x5j6zao X-HE-Tag: 1718693678-279823 X-HE-Meta: U2FsdGVkX198V3au8v/FifDFxGx5ohZVu5I/WIVcUtahepcKRjLphme54ye1TlovQTKsvgetHg/uctqMX+sSifc56Uv6xOCjrZfHPCNzSqMJGaI3++M0Wqe/qdI3mTyPiUWhqtOhivCOc/EebM77nRo4zHeGBY+Vc7mTeGLkMt/xzhqpE1xy7rccuPOsojh8gkw4uuGkLSiXpETj3gdiOPGGRYLg+aInNSpPrs0eOtnBPrLpcSNcLxMoFfPR2gxhTUiRuDdfkJhwTF6RbjxUwYUbvIbT9op7EzsJO6ksTw2xjVWh6oxfMg34a4DxBGhwujRJ2lF/WmM+BPvEMXkHh/Vw8EKb0e0N+aHA/WIFNAwsWvc7ZskpDla1m7s2rgn9bZCUMftTi/npY+v7N/0c9DYDV+hLgvjYcNcJFjXFhrYU1KBJ6/dl/iF3/kIvUBa1+N2MOYzTMNmuS7O0wLY4Bgb5wf2JKh0Yt16oH80NvV+XH3FYJS7Em1s1gt3s86YhSw0jpHPwQvViQqbY5H8yHl8K5H4yO73wQZbpCg6cjjQ2EpGu8MCuQ7iMEUes4FJWzmUC5IrvhMcSHhQQQ75Hs2Zw350v4e1NkJXRGwB2OYXMOM5JtQdfGQwLl6G5gHFgA7PGCjP1oNhmv3GCwBClTIMZJge1WDwvmf7uVarzjRa+IV7QOrhGRyo38wzYs+kxFrTh3KC/xmm2hG4zWRg1Iw//Qpeg/AvbEPLBjkO0xMOPBFnDbl2197c2mgJxP143i3zRgFAoZVCeOR88ELaltydvCJgIHifB1mvvKr4DhUqbFtxl6BDyDPr8wE7lYYG0GgBt3NLSs1lGI94a9S2HEjOKMjsCepvy0k/+KRsyR/SnQrPuoaM2fPQQwlCTLe+iKfMjN3mBZE8YUDQQaYiDBUfO4+PBQNVnWp/FhU7fQ9v5aBJIWz2JjZWtYdz5EuMqZ/qbZ3x6vT/X0TvVo/H w0vJ5/IY c35N0jcaB3W5wgwBzh/2bHpcRy9Ssib5j5uOcZRbZL5shNMNsgVFYA5yEtEUTck6SNoSweh69zymy9Ee2Bl5vWRw+fElLvgY8/xK2IoFpzAci3CPR1UZ9S49ohZ1W2K33Jht7JG7v7l0LpEb67/mcvVC3YK7TxwVVFqZ/mLVnKQAjEcRPEzy25CCjth61agCX51IubpDs/W3GifMFJHnZWiQB1DOsZ/1VKYyb6OHpjUmQuMvIZ2+ZWIK8yhbDvOqREztmLFa7gQCgZd61XHgYahkmv4RHOA2Vniyt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support large folio swapin/swapout for shmem in the following patches, drop the folio's reference count by the number of pages contained in the folio when a shmem folio is deleted from shmem pagecache after adding into swap cache. Signed-off-by: Baolin Wang --- mm/shmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index c0a9253f3a99..9a35ee7e7f40 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -831,7 +831,7 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap) __lruvec_stat_mod_folio(folio, NR_FILE_PAGES, -nr); __lruvec_stat_mod_folio(folio, NR_SHMEM, -nr); xa_unlock_irq(&mapping->i_pages); - folio_put(folio); + folio_put_refs(folio, nr); BUG_ON(error); } From patchwork Tue Jun 18 06:54:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13701844 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5103C27C4F for ; Tue, 18 Jun 2024 06:55:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1CA346B0172; Tue, 18 Jun 2024 02:54:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 150C86B0173; Tue, 18 Jun 2024 02:54:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6F596B0174; Tue, 18 Jun 2024 02:54:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B888C6B0172 for ; Tue, 18 Jun 2024 02:54:42 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 702C6140490 for ; Tue, 18 Jun 2024 06:54:42 +0000 (UTC) X-FDA: 82243096404.10.BECD7B9 Received: from out30-98.freemail.mail.aliyun.com (out30-98.freemail.mail.aliyun.com [115.124.30.98]) by imf26.hostedemail.com (Postfix) with ESMTP id 7C2B914000D for ; Tue, 18 Jun 2024 06:54:40 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="j/tpHpF6"; spf=pass (imf26.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.98 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718693677; a=rsa-sha256; cv=none; b=XMepzSgA33AuIzWztmFJOy7Sbe5PmxrvNOH2hdyN3/rvkKHvuhpwMlHXkluJnjJftnFl9X vlLD/xFYcS2wGOSai9OZNX+jFPnB2TNoVr3d/igMJmRnVfFw2KmsCSDq2dDcuVDZrxo9YA 83clnrxlKEsmqA4VIpLA3SzicKT1QPU= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="j/tpHpF6"; spf=pass (imf26.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.98 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718693677; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=X2IkWrVoIA4HwkKR/y14knPLhy9dqitMrcGqnXrDEBs=; b=CP/icvIAEIi6bpEQjlmpmYNYAxmKZsz50NzSOpVN4yWm+3aIWZrAu9Ff3NaBl1pGdhAo4I rTC4i+qqNvTwYrFioSf45jTp93LLZAV9INotk+P8sGMKSGFcIr+NIT9ba0SVrpYzZgTpZM Llv/ba6jbHxRcQCnj7/c+POWKBakYSo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693678; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=X2IkWrVoIA4HwkKR/y14knPLhy9dqitMrcGqnXrDEBs=; b=j/tpHpF65loG60eh+FQ8RHML/IKZYpV/JEf/Hx58eIP8cx3s14PxNksyLAxXDZHV66RxDHtvU64yjJqf5rVxSRL49c4gbYI5Ah9wNGB2P61OY4jEe9fJYbLSMDHxmydBJBSiG9krryj/mWAZYprvRPwyMMEKgLvU8y0aYTraplc= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R891e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045046011;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jH7nR_1718693676; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jH7nR_1718693676) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:36 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 9/9] mm: shmem: support large folio swap out Date: Tue, 18 Jun 2024 14:54:21 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: d9pfocuuqbmxdt9hg7wci43aiki713ac X-Rspamd-Queue-Id: 7C2B914000D X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1718693680-788027 X-HE-Meta: U2FsdGVkX1/sx/Yht/90EOTeHK42oCZsd6xftQg8WblVtR525rBs+QajQjqibXmVCIXotBtUrudcj/tH6XMq9tCbM3jQhvp5/Bo7KOeFIx0GzDk6awQSV0ytlpyiNnZUxRflC4WqdNGh3vdCJJ8PW7VF+wOr80PXox+toO6UNmx5JHECWitUCkYghxgEPGZUdcSjcXePBKEb1IOPO853hwL6tPPV/aPHYixB/Zq2me1d5v2UIC3P7oKUhPrBBR7Vga9gDLseYsvIPL+1c5bbIEMC7tnct1YBpE1ixNfBILLo+XZcN5ox4/TnMTcjjvpDc4J1VWUoJLcBd7CjPpJUlrfSZjyPn7c+ZukHlt78VDi60HdWsnxC6sImjp1lBUU+VeWNnqI1JwVqbeqDW4OPmvWoeRLlcywAYGvW8YIX5Z0bEKpTwk8lcdWJmQMRd8308rJuqVMStvkFvK8fGecPw8hrLugT5rce8ZggDIbDERIkkwFkBHf7v2s/uBRBZGpDvuefwxo+0/HdOVkN6jiOG9r5XsBMDcjsvpWaMwsjiLYLJOGck3XHGBEfH+pKSXXRBZNTEKJwtHIcwSNVeQeP65QxwpwmAPen7BOL3hRnxEPoc4/wFvnSmUEveSysBCvjU3qpW2aVANuYv8E9KUMHtABg5pNN1CEuL/wrbL6TjWm4E8AszDyymsXbX2GUCh8upV1XUVyIX35mZTumY/G0ros2E5lWA5j0t3OQBMtC7zA9706ByBniyOeMVmhB6BiukRNZoqtGJwv98nhVb+dcC1cT/XqV5d+UjXS7iciN8EaGn4EomPXX4PFdmaq8ER7SRwYl2dRTUpsAuxY0HNjXxeJ/KrXFMtSjV+9u5OTx8lhLnzmLIBD3s1sjMUeXs5ZjXPENaPFiejdw2ndrUx/737TCDNh3eZg8tazzPCDXcn7dviKkhhs2ZeXNhjuTT+AmrYZnlCu7O5lEr1lV46l ykMbOaHJ LMhQn9jh5BFpORguLDPBc33n9eq96cSkEGU4DSoFmv3T2X/sNT5EmjiZkuUyASIYjaAUyO4v2IILDfXLfpX1p8dpyB5DXFuGdZvVf7MVIK0W04UMXbxXs/autfkdYPu22hQj23YPidorao0FSyoO4jrhghNJ9g0vEamHVEXbDm2VEX48HKlmqQyvj1PpJHkUIALcQ4m2rTyFutfESkXyEnTbuaaZQC05qR3tli0fOswDBku4KufCNgAgYy/1tqTn+MInO/Sq/OgbkxVZRqI5rwYoHp0CE5JUlVdvQruxDTEp/u6OdKDzqwaz86kXv3GM/ylTX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Shmem will support large folio allocation [1] [2] to get a better performance, however, the memory reclaim still splits the precious large folios when trying to swap out shmem, which may lead to the memory fragmentation issue and can not take advantage of the large folio for shmeme. Moreover, the swap code already supports for swapping out large folio without split, hence this patch set supports the large folio swap out for shmem. Note the i915_gem_shmem driver still need to be split when swapping, thus add a new flag 'split_large_folio' for writeback_control to indicate spliting the large folio. [1] https://lore.kernel.org/all/cover.1717495894.git.baolin.wang@linux.alibaba.com/ [2] https://lore.kernel.org/all/20240515055719.32577-1-da.gomez@samsung.com/ Signed-off-by: Baolin Wang --- drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 1 + include/linux/writeback.h | 1 + mm/shmem.c | 3 +-- mm/vmscan.c | 14 ++++++++++++-- 4 files changed, 15 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c index c5e1c718a6d2..c66cb9c585e1 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c @@ -308,6 +308,7 @@ void __shmem_writeback(size_t size, struct address_space *mapping) .range_start = 0, .range_end = LLONG_MAX, .for_reclaim = 1, + .split_large_folio = 1, }; unsigned long i; diff --git a/include/linux/writeback.h b/include/linux/writeback.h index 112d806ddbe4..6f2599244ae0 100644 --- a/include/linux/writeback.h +++ b/include/linux/writeback.h @@ -63,6 +63,7 @@ struct writeback_control { unsigned range_cyclic:1; /* range_start is cyclic */ unsigned for_sync:1; /* sync(2) WB_SYNC_ALL writeback */ unsigned unpinned_netfs_wb:1; /* Cleared I_PINNING_NETFS_WB */ + unsigned split_large_folio:1; /* Split large folio for shmem writeback */ /* * When writeback IOs are bounced through async layers, only the diff --git a/mm/shmem.c b/mm/shmem.c index 9a35ee7e7f40..2c951d936fc1 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -775,7 +775,6 @@ static int shmem_add_to_page_cache(struct folio *folio, VM_BUG_ON_FOLIO(index != round_down(index, nr), folio); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(!folio_test_swapbacked(folio), folio); - VM_BUG_ON(expected && folio_test_large(folio)); folio_ref_add(folio, nr); folio->mapping = mapping; @@ -1462,7 +1461,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) * "force", drivers/gpu/drm/i915/gem/i915_gem_shmem.c gets huge pages, * and its shmem_writeback() needs them to be split when swapping. */ - if (folio_test_large(folio)) { + if (wbc->split_large_folio && folio_test_large(folio)) { /* Ensure the subpages are still dirty */ folio_test_set_dirty(folio); if (split_huge_page(page) < 0) diff --git a/mm/vmscan.c b/mm/vmscan.c index 61465f92283f..fd503506262d 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1245,8 +1245,12 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, if (!total_swap_pages) goto activate_locked; - /* Split shmem folio */ - if (split_folio_to_list(folio, folio_list)) + /* + * Only split shmem folio when CONFIG_THP_SWAP + * is not enabled. + */ + if (!IS_ENABLED(CONFIG_THP_SWAP) && + split_folio_to_list(folio, folio_list)) goto keep_locked; } @@ -1348,10 +1352,16 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, * starts and then write it out here. */ try_to_unmap_flush_dirty(); +try_pageout: switch (pageout(folio, mapping, &plug)) { case PAGE_KEEP: goto keep_locked; case PAGE_ACTIVATE: + if (shmem_mapping(mapping) && folio_test_large(folio) && + !split_folio_to_list(folio, folio_list)) { + nr_pages = 1; + goto try_pageout; + } goto activate_locked; case PAGE_SUCCESS: stat->nr_pageout += nr_pages;