From patchwork Thu Jul 4 11:24:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13723633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72EFAC30653 for ; Thu, 4 Jul 2024 11:25:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D65C6B00A4; Thu, 4 Jul 2024 07:25:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 63A606B00C1; Thu, 4 Jul 2024 07:25:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 090C96B00A4; Thu, 4 Jul 2024 07:25:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D394C6B00B3 for ; Thu, 4 Jul 2024 07:25:15 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8FBC5A3729 for ; Thu, 4 Jul 2024 11:25:15 +0000 (UTC) X-FDA: 82301838990.22.BBA72B9 Received: from out30-101.freemail.mail.aliyun.com (out30-101.freemail.mail.aliyun.com [115.124.30.101]) by imf07.hostedemail.com (Postfix) with ESMTP id 5424A4000E for ; Thu, 4 Jul 2024 11:25:12 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=nHMzrB0X; spf=pass (imf07.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.101 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720092281; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gnvmYA2eSyc6bAhukdfnEGSceMWwrl4WSlzxAF32JnQ=; b=X+DMIsnHqSe4JuLOg+4+V6Sw1wy1u1+VKgCibZqTDj7bPq3ixkbZqsNQ9qvdNt08TQahFr 3ckQI0BIaJNRC3hm2TXDMqi3kvBiE4cAVhwfZxAHoMDhr4HG/e67KC6Ri/wvOcC4qELWl6 iNaE3Wc0o7uJUqpHVrFNiJro1sztfoc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=nHMzrB0X; spf=pass (imf07.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.101 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720092281; a=rsa-sha256; cv=none; b=ILl7HO2axcorfCeO17D52cUfKNTj8rvF9Qnqfdn9kqm9PWJrancP8NOf3Sl4hspiEQXEnH oBDK9SzZyc54J2oE+JVjnYXcgFvbBl6YN4O/A8w7zr13IJdOl1bitUO7o51QGdi6TX0TxI GLt21G4uQdp25Hpk1vAJHRYEWDu6Vv4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1720092309; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=gnvmYA2eSyc6bAhukdfnEGSceMWwrl4WSlzxAF32JnQ=; b=nHMzrB0XAjd+FBvoo7NibQk5kHFIBPF4dPo2L0Esrf4ixMAvf6uMcCxfI22r162O0Da+h68QJC0sGU8d1QTwj41yQWRkXFRmij/Nnn3zM+TorHbmFKUcus8EfE/NjAqARP5aX8LaYo1DQ8EyrmWprM1sh8oauaS7FJNToZIS/oE= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067110;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W9qjOw7_1720092308; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W9qjOw7_1720092308) by smtp.aliyun-inc.com; Thu, 04 Jul 2024 19:25:08 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 01/10] mm: vmscan: add validation before spliting shmem large folio Date: Thu, 4 Jul 2024 19:24:50 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5424A4000E X-Stat-Signature: a9o7akfxesxo8zynb16r8upyf41f9yzu X-Rspam-User: X-HE-Tag: 1720092312-738842 X-HE-Meta: U2FsdGVkX18miM2aE1UKorgk8XpLsvP0eGg9PLhpcfN11jBE6m9ngVjpgiJg8G4j6mjaUF3gK2MjoIVmjg6M9X+kNWCCMiLYbFniQYqMZNSCW3ol3+pQGoa3siNSDeMIIjOqN2b8EE/8AzQ2ZqtNt/GDrB8VLzRF9ht8iZJQjIrnBm5P5fuMp0Jh1jK6fja+pM9pmbSsnIhD+0C7jQtMHwhJhTCji65w/Ci83x+Hh1lPim35Slcl5p/ak/eFcwW3UJFmDpmU+NHP8gedX4xIkA1jaS9Bc5Mr5+NLD9ecl4mEa1PIBFt7gE+Ci4FIIHMQc+rrC5nziWaSKAIZUA+jfULpx1XAhu/0OQWTg/zZp+YW1xYYra0x9jgWpwMv9EMAQXT5hKUnRs7tKZp9dt9ERjoFPrEaV12u5/uLmLVRTuq5ZsNVNsqQfmmSKgbHRdQSeNeupxDZGN0Kdjzyo+yhUSLwDj0+7tl2Erdzkr+VHXQ0Cpr1iSZLlqEKxHoRS9mO4hWyA/6RXO9GK4VbtkTyRQ4nKK760AkJV0sVRvirEauCU/4JY7SyS61DGQ31F/7XXHR/c87s6f3YzuWr5OtwgKcMtDT6mrH87IDif7C9VuVlV1NmsFxw/Tt4Rv6lzstVnv5akfwzQAm5v8udVVrBrusvKGovZgA2ZoFhk0i4+l0bs5K8G+dbFVgO/peKyQ3m0TjFxyXFh7omIImQenGJpQ0e4BO7r2KlotW+Ye8UCzIT2z+5MUKZkJ7Ra9Gd3Fe7pDM58QH7kUMgbPm/P/ei/6w9r/UhRfXy+UaH7sVfl+mhOaPL4G7iyKkXasLqJG+GF8FMkyqvGGtX9A+exZjSjPexk2YQzwZj/A4QXvnKTstMkMq7FA5Kbh99TOdQPhg0APKvqTG9SaWtBbIehky3OfDf77RIPP4pBTrh5QiVjieDEWr6HL50x4Bwn8GImglSwjcpw9DRlxZ+rPL9zrv nspXx0Fd 73q1wuwnZv3DWx0gaAVKRzhy6JmmWrLfitZCvyhitAR3kXbxfZ7uM02c33LAiuLJvslPRm0qzN5pnbeZgjVUolDhd4n9xTaWgmp+kiGP4Zf3HKaS3qR+7iBeUVB5ViVj031eBnK2NuGpUwaMDZkm0p+ohmd+2zu02yvWoGKsmZIkqInKXSnHfgJEPcj6jiqj5hjrRNJiytgqvhy/iRjqu/QF7t+cKB02+Dv9+WzXSoqtgICk1HC0uBMVANYciuhdQRZKjZtw99E67Ryc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Page reclaim will not scan anon LRU if no swap space, however MADV_PAGEOUT can still split shmem large folios even without a swap device. Thus add swap available space validation before spliting shmem large folio to avoid redundant split. Signed-off-by: Baolin Wang --- mm/vmscan.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/vmscan.c b/mm/vmscan.c index 1807e5d95dda..61465f92283f 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1237,6 +1237,14 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, } } else if (folio_test_swapbacked(folio) && folio_test_large(folio)) { + + /* + * Do not split shmem folio if no swap memory + * available. + */ + if (!total_swap_pages) + goto activate_locked; + /* Split shmem folio */ if (split_folio_to_list(folio, folio_list)) goto keep_locked; From patchwork Thu Jul 4 11:24:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13723632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00514C31D97 for ; Thu, 4 Jul 2024 11:25:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4421B6B00A7; Thu, 4 Jul 2024 07:25:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 269886B00CE; Thu, 4 Jul 2024 07:25:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F35E46B00BA; Thu, 4 Jul 2024 07:25:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id CA7586B00A7 for ; Thu, 4 Jul 2024 07:25:15 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 98B94A0DE6 for ; Thu, 4 Jul 2024 11:25:15 +0000 (UTC) X-FDA: 82301838990.13.446FCEE Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com [115.124.30.131]) by imf17.hostedemail.com (Postfix) with ESMTP id 7FB1A4000F for ; Thu, 4 Jul 2024 11:25:13 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=CmB2NiZI; spf=pass (imf17.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720092301; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NCJfA953k4EF8smVyxxG9pGEmrjgEHhQRz3PIQnFNcQ=; b=Hzf+9q04G1D+YUIvhObIm2x+8BQruqxfV02wRgqz0QTuj8qMEcoNHL6jocT8ftxCxdHeg+ X3koXqUwrRZ+/d/t6LrhIXQ8iYAJ1Aez1ZLCuQWH7b+GEcTwYPdDeiIFpGLIgiMSrLDK46 4UM/7tUVz4dX4/T+NoM7EWTo4J6LiQA= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=CmB2NiZI; spf=pass (imf17.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720092301; a=rsa-sha256; cv=none; b=54tJ/htWYdYRuh5KN2oBv6OnaTQyoaX+wefi7ovDr8qRv1D9HddF2pT+IA8e7tSSLHMt3h Nng/2+a416vPOZ1Dd9n/4QC3v6n2sGbhHltVhdsAQuvdh0ogZEQvslHkyneeY1Epoa/JJp 6xCSerc07v/2SDbI1yjBH2PgDnzkQts= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1720092310; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=NCJfA953k4EF8smVyxxG9pGEmrjgEHhQRz3PIQnFNcQ=; b=CmB2NiZItXef6EWV2TMeiCfbhbKGqc/KFF+XaUoxV9ZAQK3k2wAqXIRt8sYRtfLjNZ/MwIIFUnMIzrLNoETUcQ6NP+uOCeasRzymGJn0Z+R7ywtSOa24EDTgcnq5EOSJXD4jFKBi1kclYr41a12SD9xlJ08lp7tcADmk1U+N5RU= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067113;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W9qWV3G_1720092309; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W9qWV3G_1720092309) by smtp.aliyun-inc.com; Thu, 04 Jul 2024 19:25:09 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 02/10] mm: swap: extend swap_shmem_alloc() to support batch SWAP_MAP_SHMEM flag setting Date: Thu, 4 Jul 2024 19:24:51 +0800 Message-Id: <7da9021dcb4f59249d4738c4e6fea093bcbe5586.1720079976.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 7FB1A4000F X-Stat-Signature: ipethd3wr5euwrgcxwyco43ceoywwxp7 X-Rspam-User: X-HE-Tag: 1720092313-939262 X-HE-Meta: U2FsdGVkX1/3lnqj9N95E53c84ZdUI7/Q7gzqk15MJp2lFJ4idqyESZU0zWWmnT63WFil9ZDtU+xX0fDMSVrgFCSV/EJDeL/VtgEq3A2slGZvaSj6eqdS8QtkmUjzNJ8Os4FYConJLhv/9Frw6mRkOeI+WDrFMT2v8go6GlHkm0QO0BY/WFnl3jz4wfyV+wd5WoSWnoODJlNiwsU/B5wp8XZ7VHipRZzyAg8RsEMa7Pqvd64XgqUXWeC7Zp4iTmqb3hTwOjjx4Z4EHX4CbDtnDzp0JpwO3yJtUQjXe5bx4a6CXbKYIos62tsHRWnFZxsZjoI2aABgUajsykJVA8mvnOHAe7xVDnq8fiE1WzZ3KtRTO4jVr4dhNMnc8CxNK9NEv/hiBM6dc+lRN5e1sT0mRLe583OPObxrqc1Dzpo2jvsEMNAM9MQ8LC1nbacQjuybWcWOFh1KUvLd3khl9LRvfCKTV7v+wzs1BxP4ThYMdLa3mdIv0JNWUuDZKouGFD6o6Wd4rXdWDXBUQJvVEyAkXJzeKu4zHKCohdMl1S8xz8xKWuihucsJ9vW2b8oiUD3QCfem+7YteW49StgmRRO6kwNYh5QVMIVWw3+YI1EDuLPt7rxVX/wuFJCdqf7PyaN//EDoktgG8Sw34EHLBP5f5QDGU1KJaJCsRZ/OpxaZvYy/LpUJljKGCsNCKyt0UXbT4xh9csVqJQXYl+UtqLeaGFr3HTpqIoXTqF1YECIFw/nVaODacnj9sjBtiWFML+cIgwHVNcXdblxbBVDmjw68IE7NBcAxza+N2dG/WJ9/arBrAUz7EyikUUiX0Zo28ITwwjWdS2oCkQOQ4RZzGBR9C7xjy3czd6QjqwmTdYnecSIWiw/Ilyboay5i2o0aWFzQyC3w75VnlBMIXkoAfC4UHzuoSJNm9oH5VFaBnlYYWNtKL8QZE6TIlv4ncJsL6LvyjVxPfZibWWFEvpB5SI RWIA5mwF PE/8SSKu600j4heDDhsA8r/ePgJR3AsbBE6WWpKcpSVSTmFUiEA4z6rP75jknaiAlFAID9VANBbqLaQIntnsgcCYCS22OrFxTMKC3AfRtf0BXZ4bFvd/H1NjWf2Dw1UBQzCLAvPgPBm8dsF3Sk++AMVV91DmtVw2g6slXgva2T3y8BEK7+/ueoy9GOYOLE9IOQPX4BUZsnTnaG0R4sVO+jtn9KtF31qmFEYPaKtENEEoYKluSmb5lhDEReHB0IATnVGYDwIXxDS3iDgTO7foQmqcOT8epEjzEDRsl X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support shmem large folio swap operations, add a new parameter to swap_shmem_alloc() that allows batch SWAP_MAP_SHMEM flag setting for shmem swap entries. While we are at it, using folio_nr_pages() to get the number of pages of the folio as a preparation. Signed-off-by: Baolin Wang --- include/linux/swap.h | 4 +- mm/shmem.c | 6 ++- mm/swapfile.c | 98 +++++++++++++++++++++++--------------------- 3 files changed, 57 insertions(+), 51 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 483b760d49a4..8839bb2f2fa6 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -475,7 +475,7 @@ void put_swap_folio(struct folio *folio, swp_entry_t entry); extern swp_entry_t get_swap_page_of_type(int); extern int get_swap_pages(int n, swp_entry_t swp_entries[], int order); extern int add_swap_count_continuation(swp_entry_t, gfp_t); -extern void swap_shmem_alloc(swp_entry_t); +extern void swap_shmem_alloc(swp_entry_t, int); extern int swap_duplicate(swp_entry_t); extern int swapcache_prepare(swp_entry_t); extern void swap_free_nr(swp_entry_t entry, int nr_pages); @@ -542,7 +542,7 @@ static inline int add_swap_count_continuation(swp_entry_t swp, gfp_t gfp_mask) return 0; } -static inline void swap_shmem_alloc(swp_entry_t swp) +static inline void swap_shmem_alloc(swp_entry_t swp, int nr) { } diff --git a/mm/shmem.c b/mm/shmem.c index 13d139abe69a..6244b909b069 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1443,6 +1443,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb); swp_entry_t swap; pgoff_t index; + int nr_pages; /* * Our capabilities prevent regular writeback or sync from ever calling @@ -1475,6 +1476,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) } index = folio->index; + nr_pages = folio_nr_pages(folio); /* * This is somewhat ridiculous, but without plumbing a SWAP_MAP_FALLOC @@ -1527,8 +1529,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) if (add_to_swap_cache(folio, swap, __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN, NULL) == 0) { - shmem_recalc_inode(inode, 0, 1); - swap_shmem_alloc(swap); + shmem_recalc_inode(inode, 0, nr_pages); + swap_shmem_alloc(swap, nr_pages); shmem_delete_from_page_cache(folio, swp_to_radix_entry(swap)); mutex_unlock(&shmem_swaplist_mutex); diff --git a/mm/swapfile.c b/mm/swapfile.c index 2f5203aa2d2c..0cc8a5a72a2f 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -3387,62 +3387,58 @@ void si_swapinfo(struct sysinfo *val) * - swap-cache reference is requested but the entry is not used. -> ENOENT * - swap-mapped reference requested but needs continued swap count. -> ENOMEM */ -static int __swap_duplicate(swp_entry_t entry, unsigned char usage) +static int __swap_duplicate(struct swap_info_struct *p, unsigned long offset, + int nr, unsigned char usage) { - struct swap_info_struct *p; struct swap_cluster_info *ci; - unsigned long offset; unsigned char count; unsigned char has_cache; - int err; + int err, i; - p = swp_swap_info(entry); - - offset = swp_offset(entry); ci = lock_cluster_or_swap_info(p, offset); - count = p->swap_map[offset]; - - /* - * swapin_readahead() doesn't check if a swap entry is valid, so the - * swap entry could be SWAP_MAP_BAD. Check here with lock held. - */ - if (unlikely(swap_count(count) == SWAP_MAP_BAD)) { - err = -ENOENT; - goto unlock_out; - } - - has_cache = count & SWAP_HAS_CACHE; - count &= ~SWAP_HAS_CACHE; - err = 0; - - if (usage == SWAP_HAS_CACHE) { + for (i = 0; i < nr; i++) { + count = p->swap_map[offset + i]; - /* set SWAP_HAS_CACHE if there is no cache and entry is used */ - if (!has_cache && count) - has_cache = SWAP_HAS_CACHE; - else if (has_cache) /* someone else added cache */ - err = -EEXIST; - else /* no users remaining */ + /* + * swapin_readahead() doesn't check if a swap entry is valid, so the + * swap entry could be SWAP_MAP_BAD. Check here with lock held. + */ + if (unlikely(swap_count(count) == SWAP_MAP_BAD)) { err = -ENOENT; + break; + } - } else if (count || has_cache) { + has_cache = count & SWAP_HAS_CACHE; + count &= ~SWAP_HAS_CACHE; + err = 0; + + if (usage == SWAP_HAS_CACHE) { + /* set SWAP_HAS_CACHE if there is no cache and entry is used */ + if (!has_cache && count) + has_cache = SWAP_HAS_CACHE; + else if (has_cache) /* someone else added cache */ + err = -EEXIST; + else /* no users remaining */ + err = -ENOENT; + } else if (count || has_cache) { + if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) + count += usage; + else if ((count & ~COUNT_CONTINUED) > SWAP_MAP_MAX) + err = -EINVAL; + else if (swap_count_continued(p, offset + i, count)) + count = COUNT_CONTINUED; + else + err = -ENOMEM; + } else + err = -ENOENT; /* unused swap entry */ - if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) - count += usage; - else if ((count & ~COUNT_CONTINUED) > SWAP_MAP_MAX) - err = -EINVAL; - else if (swap_count_continued(p, offset, count)) - count = COUNT_CONTINUED; - else - err = -ENOMEM; - } else - err = -ENOENT; /* unused swap entry */ + if (err) + break; - if (!err) - WRITE_ONCE(p->swap_map[offset], count | has_cache); + WRITE_ONCE(p->swap_map[offset + i], count | has_cache); + } -unlock_out: unlock_cluster_or_swap_info(p, ci); return err; } @@ -3451,9 +3447,12 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage) * Help swapoff by noting that swap entry belongs to shmem/tmpfs * (in which case its reference count is never incremented). */ -void swap_shmem_alloc(swp_entry_t entry) +void swap_shmem_alloc(swp_entry_t entry, int nr) { - __swap_duplicate(entry, SWAP_MAP_SHMEM); + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); + + __swap_duplicate(p, offset, nr, SWAP_MAP_SHMEM); } /* @@ -3465,9 +3464,11 @@ void swap_shmem_alloc(swp_entry_t entry) */ int swap_duplicate(swp_entry_t entry) { + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); int err = 0; - while (!err && __swap_duplicate(entry, 1) == -ENOMEM) + while (!err && __swap_duplicate(p, offset, 1, 1) == -ENOMEM) err = add_swap_count_continuation(entry, GFP_ATOMIC); return err; } @@ -3482,7 +3483,10 @@ int swap_duplicate(swp_entry_t entry) */ int swapcache_prepare(swp_entry_t entry) { - return __swap_duplicate(entry, SWAP_HAS_CACHE); + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); + + return __swap_duplicate(p, offset, 1, SWAP_HAS_CACHE); } void swapcache_clear(struct swap_info_struct *si, swp_entry_t entry) From patchwork Thu Jul 4 11:24:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13723634 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C7E7C31D97 for ; Thu, 4 Jul 2024 11:25:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 304226B00B7; Thu, 4 Jul 2024 07:25:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2651B6B00CE; Thu, 4 Jul 2024 07:25:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 106D96B00D2; Thu, 4 Jul 2024 07:25:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DB0606B00CE for ; Thu, 4 Jul 2024 07:25:18 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7E100120E05 for ; Thu, 4 Jul 2024 11:25:17 +0000 (UTC) X-FDA: 82301839074.23.CB44BBD Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by imf25.hostedemail.com (Postfix) with ESMTP id 6F29AA001D for ; Thu, 4 Jul 2024 11:25:14 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Y5J1X28w; spf=pass (imf25.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720092303; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bjDtTTMdlmmldWh/OT1ZyYCpZPGPkV2gHwKUwOxuuNY=; b=bxASp7KomvKvd3ESdEqjTFK5aEM1qzc6fmhxcyeS5stchLcSzsHh+tgUOjYZOP2KF4kDbP 7QghIy+9+yEVRmSrY7bLxIcGQ15y9xJcwxIao0whYqy/NQDuR5IMDsTaKl4kGCqNcYXlFe 5oLFk9+OV8yrCsC2AM0fU3lwh8d9MKo= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Y5J1X28w; spf=pass (imf25.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720092303; a=rsa-sha256; cv=none; b=nrFlpAKU0sQTylfwpqjSPMgDT+7I1743EmKiC8EZw6rr20Ua5NOfyvzy5HKFldbe5VaPDD PmGxdR9q4+RKVhChSzAXhM5REs6d4ZP9URKr3sEvhfwYAep82D7xFtod3j38TTqSrfHza/ 9YRn9XrPHycyl8qnxZm3Wm7uIDNdfYM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1720092311; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=bjDtTTMdlmmldWh/OT1ZyYCpZPGPkV2gHwKUwOxuuNY=; b=Y5J1X28wKMNtRfaK7um5XNoGpN7tcnRpI0PlJk7nqlI0dinzvUwc/BehRUquKftt0VwdYoeOG77zRTPRdhS+Z5kq0hj1YvVBn/e85yYH9ih+WBMZuIbV9caufnSBv+UFIYYjymBRqnIIvwcXTv+3kFgg0Lc6dAO5S42h355UAPw= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045220184;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W9qjOx8_1720092310; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W9qjOx8_1720092310) by smtp.aliyun-inc.com; Thu, 04 Jul 2024 19:25:10 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 03/10] mm: shmem: extend shmem_partial_swap_usage() to support large folio swap Date: Thu, 4 Jul 2024 19:24:52 +0800 Message-Id: <16b2d5360b0a5588c21508714b1afa50c874f307.1720079976.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 6F29AA001D X-Stat-Signature: o4y7tx8wrzr34dxq4hkiu4nm77ad7kix X-HE-Tag: 1720092314-307874 X-HE-Meta: U2FsdGVkX1/5DA+p5qCH+hXny6MJu+0SMZXnMXtLVJ5/CD1FxrPs64QCaPcayvHO5bpZ0RtCDOYHNH99vZgVoU6Z99u90m7yMMPXSzBPSZV9vs91xdGW3hr5577N5GzuoPA6V5E1+F3qZKkXpu80Mz2i3WgrKoZAi4oTUTJALY9vhLgS3KxGRWHAUkDed9l4/zGR/54FhnX1IYoaefvDUm0da7lKB8zWlDZzPIA53hLbjs2H4nfRle73o17Lmn07KxYv1TSyMM4CMYRqj+/+lIr3taCpnacw2jbvdn36WqxpZXigOTR6N/Rs0J/5+ckSQ5fVZ7ZJmIjdU8f9KnSnAoPiuSEavTuKVfqdFqwMDBskHhUQKI5cL61sjo4IXFZ9GyROlWsdz+LHlTekQ/iqh32bckz+QFtwZFMtZhwDI/ff+cpqkdbyLr6hY1j40HlfUGcP73jhOHIotGaTPnZyuh9QoqLPtRJBbMKG+U05hz7UpKvf5swDM+tAVssPLhXR9HrAao9VbUC9lLZBhX3OToR4TUdluJXbtDpOJxpYZBNq6U0bWpjKvlew3/cRiGMdRS+LZTCjFcuZi+dMRyiIKeJApdHuth0JcAAx+H2QszZAXCDbhBEPYwJMeKCWMNpPp4+kmOeCL0JfhJvurlxQN52po+nRuWX99cQfR6G1DTZ6REiwb0tOVEOBaTROX6+xbqolF1S2JLVm5iA2FSyT/7YMyIrxOe1d0Tz0KzSKwlLJH55fcZfXajXVAigg8lHeIs1vAZLMJtAhG1HAzF3UbhNuNkaI1wzT7pE6E5gKWEvVkAPQiLjgYR0ED4mJyZcTxiVOUebkjkU9dh13f22CpeOZSDQSxWFRucAjerEiKs16V3ebCGqJ569hSLhinnDKCc/TWF9fPxlWCkHi8iX9ysP3VgMNA6m8yh22wWr0AhOnjk+X3LCsRXJyPD35abiqET9FFcLU/Zm3y0JGeFr EsV24wMb hpZDGQ6QQMbQae1J8EwfydWWo4cR1WrlUinEl+Gqy1ml/j6KbmAxFx7LPFaHpfxPc4FBGA8TV+PwtI3oRo0IEWrB6xtq5+iQBdeNbAbi9UUku3NVugmkxSXXJrGsw6MGWxI61+hR6nWuOtZlhstTsTwe008f0EftrOfDJ+oAniXcPxrmb131YD/hNUHutQmJApEiwMDVsl4kwmNBZXd4gn6x+ef7hGUNyD7OJACtFZZXo9cfxWodduHw6vrh7GdzPaVgTuEu2+FySAQBdt3SYFp7HXo7x/v4N90Fj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support shmem large folio swapout in the following patches, using xa_get_order() to get the order of the swap entry to calculate the swap usage of shmem. Signed-off-by: Baolin Wang --- mm/shmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index 6244b909b069..42bf3176fa6c 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -881,7 +881,7 @@ unsigned long shmem_partial_swap_usage(struct address_space *mapping, if (xas_retry(&xas, page)) continue; if (xa_is_value(page)) - swapped++; + swapped += 1 << xa_get_order(xas.xa, xas.xa_index); if (xas.xa_index == max) break; if (need_resched()) { From patchwork Thu Jul 4 11:24:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13723640 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E90CC30653 for ; Thu, 4 Jul 2024 11:25:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 155836B008C; Thu, 4 Jul 2024 07:25:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 106B86B00E5; Thu, 4 Jul 2024 07:25:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E09AD6B00E7; Thu, 4 Jul 2024 07:25:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AFA9F6B00E6 for ; Thu, 4 Jul 2024 07:25:25 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5EE87A3DD5 for ; Thu, 4 Jul 2024 11:25:25 +0000 (UTC) X-FDA: 82301839410.08.AF723DC Received: from out30-113.freemail.mail.aliyun.com (out30-113.freemail.mail.aliyun.com [115.124.30.113]) by imf17.hostedemail.com (Postfix) with ESMTP id 7F8F34002A for ; Thu, 4 Jul 2024 11:25:21 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=AunEbIZ1; spf=pass (imf17.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.113 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720092298; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EpkaKZeif3DR2KNud0UcpLjPquN1O7r8iKvlNR8OHHc=; b=qHcVUG4i6OmvU+SG6vxopwx1L8Jsk8AA8RNnu9NK3Jdx6NAJLQUWFcEPhm7eB5LvRpfUoM Jpj9C+/yqltyyMu0zrTT7EMcu7BO6Mlvh+u2FdYmC+a7f66z4g6PpgoqtI0tQa73P6EWT0 pemMkyLCdteZHHO84PZmIHAzjO2f5S0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720092298; a=rsa-sha256; cv=none; b=Him9sVytmcxZ62VaAi07MQVs0YzHiP67JHi9nYKEkyhet1b+I2KtZ1dBX2EaGb4YqOoMpY 1M35mt01MxgCw4kkJF6JBpAqqsucU6VUb+/3KzEj9w3OID+mkTUtJINzmzJ8Ad12FNJzk7 ecYe4o4GfD35yEGJTtb36j5hpxytqZg= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=AunEbIZ1; spf=pass (imf17.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.113 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1720092312; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=EpkaKZeif3DR2KNud0UcpLjPquN1O7r8iKvlNR8OHHc=; b=AunEbIZ1LHcy+89V68+o9NsXGHt8gVXfXQ9YNakBzzfXE9boyWYpLhbZHcI0P5RQpTAZ6yiGyso2Qee6/G68gi6Og83zjuYEUoQQ7NK0Tmr3V9xBMG2hpvms4EYz0fdHdYnrTr4t6gwSqCA6EHevW5CyQlwFX7qfUzxMDs9Uqxw= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067111;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W9qjOxX_1720092311; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W9qjOxX_1720092311) by smtp.aliyun-inc.com; Thu, 04 Jul 2024 19:25:11 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 04/10] mm: shmem: return number of pages beeing freed in shmem_free_swap Date: Thu, 4 Jul 2024 19:24:53 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: npweqtabds17hyxcrk3xgq9uabomf4tw X-Rspamd-Queue-Id: 7F8F34002A X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1720092321-195437 X-HE-Meta: U2FsdGVkX1+/sS/52/3MoeiPXBN0FcpUymXlIqjdpD9tdR/rbxc5+naF2ScscqNENAC/6VGnlFXyaebNIF+a2eyPj/zsQuRH/ABqzV41VQRVrElMksCYAMVSiIQFOKGD3o3Po1Cn5IUtE8WirTWS24zDJVKyRwJDDHllpltFXDYXhI8Brum91xB6mMJCIl52relFl4pmIlGdxtuaJ5Qh/iNQVk3jEtSFEo+kQX+ivbEpWb3XXUf9Kmu70nW5jPhV7j5JaFsukjnfdFnQcP5bSm2TgVGqja7oFS3YfheqpRn5pTaucmmM9Rmlg+c+eweh4cmWaufMhYFGxRu9NnjezNsgYZa3YRP3TNm4s4Z3+LubxOVk0/sesYIn8Wm0vrIy1cldxuxbLMSV07rZzQJ/yISXEOpAdbc9dov697uMqPR1wsEbWdy14YO5RShaIA6p9XcnTbfyP1hCrtozm8HdotqijAPgHb2frddUe+XHaJp165KvjCQXaku45wg1rILDNYUhK4v8u0Xvs3KHcv/pNOyZPv4ID8vRbqJYWaX64BZvFQCSpnfpXxrDEDLnKgSMS7Qe4B5KfZPRUBLvJfX9Dk4+viCMMNwJTYf6iqoJAyVhFJyLTqIooDLPorSzbHUB8RB951AYOTQkenGKyGqLgwPuw9Fg70Ofc+MuCrgrT7FcVAgQXfECdF4jZrxIYUMPmcAfxYwo2btkGcdbJIUNctL3rSRKo6ux+9UIilCEWA6bfPl8oO0F2bdbPDgLx10808mpXmOMQr4uuEHeipb/nXhA8pDMOZtzAnUzexUO2SDbf4e8uAzk6CjenWNeBKZlF90mIkiDFZ+7bkEGB7p0J2RYqBqAmQP4rdobWoduSTWnQuvHmdA7Y1Z+TX+DfYFUoynI9t90FOIHsKrwYMFq4yY+5uHUt1t/sx65jXBy+i54Rko3Kc/Fhg8Y0Xp/FnO0e9BoWU5alMEviHkcird bXntUhQN VvQrTgQwVM4YvMmTXdzAhUdoVFlqKVYI+FnVqoOjm7RL1DIHv/iJNUBnGx41zRCE3tGsqsAIFZfIx/fdJ6EyxJ0DxEmURZrdQuyvWj7WcZ/dMKdJLmE9w8Odpk9jVH6XYYg0USNu73P47Up/uhlsdo3kRiIyyqwk9z1I6r54Jf9Q5DSZLyMybD465ZB1XO4ZwwoxqZV46IAIwrQi77siz3qjUKoe4jHLIBxbmA4eYoJw7xgowDaDIUEvJnPOUJbqLAG+46VcqREhZWrCyDX8wS0bD4SQBCySpGq/Aw6RtE+g6iJo8YhA22uC7E3F/QCUcfgrPpcw834FtSu+1fYQEkDJLK/qEFhdXtMIhEgvBcdpAK9g1ESXySiyXtQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Daniel Gomez Both shmem_free_swap callers expect the number of pages being freed. In the large folios context, this needs to support larger values other than 0 (used as 1 page being freed) and -ENOENT (used as 0 pages being freed). In preparation for large folios adoption, make shmem_free_swap routine return the number of pages being freed. So, returning 0 in this context, means 0 pages being freed. While we are at it, changing to use free_swap_and_cache_nr() to free large order swap entry by Baolin Wang. Suggested-by: Matthew Wilcox Signed-off-by: Daniel Gomez Signed-off-by: Baolin Wang --- mm/shmem.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 42bf3176fa6c..5159d60456ed 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -847,18 +847,22 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap) } /* - * Remove swap entry from page cache, free the swap and its page cache. + * Remove swap entry from page cache, free the swap and its page cache. Returns + * the number of pages being freed. 0 means entry not found in XArray (0 pages + * being freed). */ -static int shmem_free_swap(struct address_space *mapping, - pgoff_t index, void *radswap) +static long shmem_free_swap(struct address_space *mapping, + pgoff_t index, void *radswap) { + int order = xa_get_order(&mapping->i_pages, index); void *old; old = xa_cmpxchg_irq(&mapping->i_pages, index, radswap, NULL, 0); if (old != radswap) - return -ENOENT; - free_swap_and_cache(radix_to_swp_entry(radswap)); - return 0; + return 0; + free_swap_and_cache_nr(radix_to_swp_entry(radswap), 1 << order); + + return 1 << order; } /* @@ -1010,7 +1014,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, if (xa_is_value(folio)) { if (unfalloc) continue; - nr_swaps_freed += !shmem_free_swap(mapping, + nr_swaps_freed += shmem_free_swap(mapping, indices[i], folio); continue; } @@ -1077,14 +1081,17 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, folio = fbatch.folios[i]; if (xa_is_value(folio)) { + long swaps_freed; + if (unfalloc) continue; - if (shmem_free_swap(mapping, indices[i], folio)) { + swaps_freed = shmem_free_swap(mapping, indices[i], folio); + if (!swaps_freed) { /* Swap was replaced by page: retry */ index = indices[i]; break; } - nr_swaps_freed++; + nr_swaps_freed += swaps_freed; continue; } From patchwork Thu Jul 4 11:24:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13723635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9194DC30653 for ; Thu, 4 Jul 2024 11:25:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 85C4F6B00D2; Thu, 4 Jul 2024 07:25:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E2976B00D8; Thu, 4 Jul 2024 07:25:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65CE96B00DC; Thu, 4 Jul 2024 07:25:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 440976B00D2 for ; Thu, 4 Jul 2024 07:25:20 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id EB9181202A1 for ; Thu, 4 Jul 2024 11:25:19 +0000 (UTC) X-FDA: 82301839158.20.1A10A93 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by imf17.hostedemail.com (Postfix) with ESMTP id B827440021 for ; Thu, 4 Jul 2024 11:25:17 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=GXt422Bu; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf17.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720092298; a=rsa-sha256; cv=none; b=rqf3/3H/bEbpDHhGfw3GSFNRJdKEouNo7nxQGhvgTVXjdRXBv3eAxFZKtgAaTbfwSatiw5 Nk7fSM4ty4pzrJwhqd3tCT+AM2NmLbdNx8mylnZET+Vwi7+5zCJkKSa4gEce8zOk4ucWNg Q/u43742GXGbXzvHmWdnX174Abqb3W8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=GXt422Bu; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf17.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720092298; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QyBoZMTgb7NS//TXwyienog6t9srSRphl+czPh5Cin4=; b=iR1W8rXIYLU5RLr4A5YCrWyBRLzHD6PobAKdapEksyc8iPKmFHYxwim5MSd/KBxMouCxla CfSigPOJkcUZApJDIk1W4c+GdLexv0DCEfjZMOoMKedIxZWXHPOab0IQS1cYUKq3cFtgy+ DRCKIrcMuLkqRshgkCicikW3h+nxKsE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1720092314; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=QyBoZMTgb7NS//TXwyienog6t9srSRphl+czPh5Cin4=; b=GXt422Bu3EY5MyVCQnrbMEBU6ujx9LD6o+YcorNPC+INTf81iwD4/T/vlPpZncXA9E2Kzs2q+z1ArfvA7ZUw7MtPFbvzyOWz1jcFHMxZGNOY9P/ovRM8912DCe5BQCLIAM1/WZ+jTO2brpXqkfszvBgCqeDev/wkr25Gs/dTd+g= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045046011;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W9qjnNQ_1720092312; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W9qjnNQ_1720092312) by smtp.aliyun-inc.com; Thu, 04 Jul 2024 19:25:12 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 05/10] mm: filemap: use xa_get_order() to get the swap entry order Date: Thu, 4 Jul 2024 19:24:54 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: B827440021 X-Stat-Signature: 4z5w68g7no44ins8e86ozcjbq79fbbof X-Rspam-User: X-HE-Tag: 1720092317-58601 X-HE-Meta: U2FsdGVkX1+G75MAuB2bAl9VJ5wuD2/NULkkmB2ZVAEQ9AHoYM7l6Nu1bPsANYRVIslXvyEGWSsOkxwCeaj4nVi61a4+a4tLGA+Bp82ejjc/Na3AmVTpnxpnKUPg/ewdk7MuNja0hwk0zBwQ5ZGf57wj6fhpzZk9M8h4u2sEDzw2k6W5yunvgJOm9ovcUler5gCXlmg0E+w0vcKmMIyv8gTkSHgVq0vJkSOGYXTfGEbBc3bzlr7aZGvOnjxlK3x58txa5VKptGFWLEI3gunU99mJlUnj8w1gUaiwY8oSX8tGbqBQU+3VE+fHL1fYMVh2oJIe3MxGX7ilCK5yoGUnQN6WpTS8VVH8Dq2HzGSSw/KZQ1HDaFOUUw+hwdr+hObUPgJYTddRcRvIFW1IKuCVqyHbexzE4S2pqk+ws1KY2ayyZ9dCJZ2k0m5SBLCylFZvhN0HaGJaDxZgPnwyOL+hpOdlofS0g4RUUpVA/K23RvIrW89QFFGnbFce8AwHM213n5TQgkshlbCPaLHKezTr6pvB92buK6rXnzjK4U3EEUbkzW1nL4IEEdh2VayOhigAjBa8zQaeCyCWb58t4j6kudJn0IEhizqhjyR/akqyBgkO+RlYjV1PgaelAbn//rCpAUGBxtQrZYKWQ5L2FSa0TUJ2Ley9FDSqbWBTiGJj1ZcQ2VsSgBcdc7nd9iRokA4yWOsbXKf2yRvLVLxUuWjZHGTsw83Pr2KXvJcBB+er9Jf2ZHTrVCse93c1V7zbRreEgTmnaoZfLpvH1zy2Pp5oBbZagqfatXHnLe8NANIUz+QHfHYAELDeOBqkqOi5ZRiYz//lkLzPzAwxXdGJ1B/xgxJFYcBPhj+R5CKue6MANu1E95mtngQbC1BKG4I4RBw9iIVt8VOFdmlGvOMGDEWnz87yukNkd28WelVCa4zpXWGQPmC4Ycm7Cjpz/ydpS3HVMeabY8fi3sCpn66I75/ bjGXeiSt ZRbT3qc0v9rdf/wHM3pOr5uUNMaRMlQvpWCrSoDnK8kIXE0u0KyDpZf+0dre2nVjwFEviGhvLT1aiSZWGtP2Uomm7qcXQI1SEUY68ixlwJijbrOh+0jXo7XJRvfkhKHwgU5pxiU8I/zUOgDAboJTTatTRNc3QYL1Ik4SW9lAKlQsAG+b1lcB7M28t+SE0Is9AZfXUCe0B65RM9EX/0v5/56LRduOMuymp3B8R7TORSRsmh1pIoOcAvQmHA+eu5xE2Wdj9DXoC+P46byjR+MsljTdUZQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the following patches, shmem will support the swap out of large folios, which means the shmem mappings may contain large order swap entries, so using xa_get_order() to get the folio order of the shmem swap entry to update the '*start' correctly. Signed-off-by: Baolin Wang --- mm/filemap.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 95d56cc788ef..c1a703eb8116 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2056,6 +2056,8 @@ unsigned find_get_entries(struct address_space *mapping, pgoff_t *start, folio = fbatch->folios[idx]; if (!xa_is_value(folio)) nr = folio_nr_pages(folio); + else + nr = 1 << xa_get_order(&mapping->i_pages, indices[idx]); *start = indices[idx] + nr; } return folio_batch_count(fbatch); @@ -2120,6 +2122,8 @@ unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start, folio = fbatch->folios[idx]; if (!xa_is_value(folio)) nr = folio_nr_pages(folio); + else + nr = 1 << xa_get_order(&mapping->i_pages, indices[idx]); *start = indices[idx] + nr; } return folio_batch_count(fbatch); From patchwork Thu Jul 4 11:24:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13723636 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42ABDC30653 for ; Thu, 4 Jul 2024 11:25:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BB7DC6B00DC; Thu, 4 Jul 2024 07:25:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AEE846B00DF; Thu, 4 Jul 2024 07:25:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91C9B6B00E0; Thu, 4 Jul 2024 07:25:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 71FC76B00DC for ; Thu, 4 Jul 2024 07:25:21 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id EF355C11D4 for ; Thu, 4 Jul 2024 11:25:20 +0000 (UTC) X-FDA: 82301839200.14.B2DD350 Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) by imf03.hostedemail.com (Postfix) with ESMTP id D1C1820012 for ; Thu, 4 Jul 2024 11:25:18 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=B7EEJvKz; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf03.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.110 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720092299; a=rsa-sha256; cv=none; b=cogdPN8mE+QrKjrYr0gxUojSCAwd8VNvQsdNA2vhluN5TlPxPosgVjx5tmlLVXvqnIS7Rk Y2i5X6BVwh1WPk3Hjrk7quG5kM6gYEjoYtE2Ui8gv75xkAYGeZlWAEODX7PV+NjzPup6Mo XEA/SGTkDPQvddy0FtwHbIivHQ2jmxc= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=B7EEJvKz; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf03.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.110 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720092299; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cw7XWKxljyeN4WJVgu0jE9S3cyW8I72GFf5V991lB4Y=; b=ZOYYlX9xvVSOty2pfp6RqR6QtKUFQpwKke8/F7EKox87szpckPgMaO9vqwefE0jzoRgSBg MXpXnMhFj/N6aMl1ZJ36D3cZP1vMzSpY75bTSVRxcPRO+Lu9KbkFBRV89Yrlby8BI7PhKM CTuvqFzcJULwrC9l2g+wBFocUwVmcpc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1720092316; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=cw7XWKxljyeN4WJVgu0jE9S3cyW8I72GFf5V991lB4Y=; b=B7EEJvKz/X883XfMGuFi9jG+UE8TcyudSYkIYIzSCzUj4Prl/bx6hMf3/ErL9rvba1KHDRrX5YPn333FhqxQQWGYSaYs49Gl9f60zGHW1Ak0fQUaL8Ut2xb6N3nF65CfHt4esy3VCbonkJzPDsYblZfuEtfjyLXAOh8QGjrwKuY= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033023225041;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W9qjnNh_1720092313; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W9qjnNh_1720092313) by smtp.aliyun-inc.com; Thu, 04 Jul 2024 19:25:14 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 06/10] mm: shmem: use swap_free_nr() to free shmem swap entries Date: Thu, 4 Jul 2024 19:24:55 +0800 Message-Id: <23ca2d1733c1a4e9f9781312a2dc21c8873b3bd5.1720079976.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: D1C1820012 X-Stat-Signature: 8311xj1d36si7kdze6kid96cbj6tf7o5 X-Rspam-User: X-HE-Tag: 1720092318-295901 X-HE-Meta: U2FsdGVkX1+DqjVdxYRQsHtwDWHZC1db2rlZWMV/TSMelW4/Sg1QVgiR3aM0x4uII5Ugl2u9QT5a+tkcpr0orZctTF0zaFc4y0EOZG5Er6V7DCJM7KpIMNtLB2V6qOd6kGRDmkNRCDa2Rsz1ebjait8oIhCxzoMG3Jkbvlg55SE7fv+ZI+zqRtEbjCBDjuRtYHNYPPXs9E46oD9HekWvvMfPlCGHm5MmDLukqHymNh4+WZWno45uerj85Xjpnnvh2GrU9qiKMX/ozii5kd5zNegOkO0f3Z+KsQ8+0TiSj4cSXXKKF95C3TFk6N0+dELM83bKLgcPeh9EPbiMu9K9bNx55ICCDgoJqQvg/pXUP8dwxLtuXiZzPknJOQ97O6RjyWqJEwGUDLQflo5QlffvQM7bFkzmclmqQ2Oel5+hdJx4hmLYhfA8/dcpl9piJ6Mpvlbc/jBbF4A88s0L3VM5bMzL+OEA5HHPgY0uPbYIkubC4xaNmojePAPjSZHiOp9v+oNwTn8sDUjPYfWu+fkemlY7icGqe6eRHPbt1HDkarSxIwMZVo0G30FVOhwogEonKz1uHRHepDZR8PnM/b/SbVz9KMteLPMcIEfbyGxhYRexCx6DN6qsqSKSKrdHH8Nsuk1fxymo8TJWkvKOoCWPLugCecwWgFWrRlf2w7eQcgfs45m6tOjRjC8tXdGvhJwlAWzwz6eIdApL/vhjn6zULufx2V0A0Oc04XgnL//Fd7W2Lm3lyv3txOeGKdnHmQe0kEG0MsJlnx/yaOcPosMw9389Nb/JyULKy2HyDq6phpWNEg5yNH7FzQYmixeWBsCzzYesU9B5qMKv9tPHfIkZqHCDs/HvikMrKilgnSFdBwjk792Jata8lG/M0EvHTVmQFvxrpX243Rqiqkk94Arv9ObIuyUCNrPOE89enBTAkTYiNxfIWizXom27fUAL4sSzXggQVXTDJNNcdDrXboH 14cUiV0e ERE9XPjUxW/zJtDifxIS9ECzk44nE5tyf4cTuyxM3LD8nYvD1rpPNPYTkq3XOt3EnfmWVBnKV48X/e6equb2CBItJvQ4q1g8x1/2dW+1gRCP/VdU20Sm3wEs65nJI8wxY+ln112KMGmoC7SqU9AG/hyxP1Qy5wbPsK3G2k5QCtTM8eemHj/TeI3eQrmityea6iXvxALezbVOXzWPUbrdQqu6tpXsfjFLPh8QyGzCn45gsexIwmHP9S17bHwrYSIOib0PU8VvUU1sWQWuwOlLssaI96+Jy+ztEPA8T X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As a preparation for supporting shmem large folio swapout, use swap_free_nr() to free some continuous swap entries of the shmem large folio when the large folio was swapped in from the swap cache. In addition, the index should also be round down to the number of pages when adding the swapin folio into the pagecache. Signed-off-by: Baolin Wang --- mm/shmem.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 5159d60456ed..c1b6cdc61619 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1961,6 +1961,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, struct address_space *mapping = inode->i_mapping; swp_entry_t swapin_error; void *old; + int nr_pages; swapin_error = make_poisoned_swp_entry(); old = xa_cmpxchg_irq(&mapping->i_pages, index, @@ -1969,6 +1970,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, if (old != swp_to_radix_entry(swap)) return; + nr_pages = folio_nr_pages(folio); folio_wait_writeback(folio); delete_from_swap_cache(folio); /* @@ -1976,8 +1978,8 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, * won't be 0 when inode is released and thus trigger WARN_ON(i_blocks) * in shmem_evict_inode(). */ - shmem_recalc_inode(inode, -1, -1); - swap_free(swap); + shmem_recalc_inode(inode, -nr_pages, -nr_pages); + swap_free_nr(swap, nr_pages); } /* @@ -1996,7 +1998,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, struct swap_info_struct *si; struct folio *folio = NULL; swp_entry_t swap; - int error; + int error, nr_pages; VM_BUG_ON(!*foliop || !xa_is_value(*foliop)); swap = radix_to_swp_entry(*foliop); @@ -2043,6 +2045,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, goto failed; } folio_wait_writeback(folio); + nr_pages = folio_nr_pages(folio); /* * Some architectures may have to restore extra metadata to the @@ -2056,19 +2059,20 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, goto failed; } - error = shmem_add_to_page_cache(folio, mapping, index, + error = shmem_add_to_page_cache(folio, mapping, + round_down(index, nr_pages), swp_to_radix_entry(swap), gfp); if (error) goto failed; - shmem_recalc_inode(inode, 0, -1); + shmem_recalc_inode(inode, 0, -nr_pages); if (sgp == SGP_WRITE) folio_mark_accessed(folio); delete_from_swap_cache(folio); folio_mark_dirty(folio); - swap_free(swap); + swap_free_nr(swap, nr_pages); put_swap_device(si); *foliop = folio; From patchwork Thu Jul 4 11:24:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13723639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D4C8C31D97 for ; Thu, 4 Jul 2024 11:25:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA6466B00E4; Thu, 4 Jul 2024 07:25:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A2CF56B00E5; Thu, 4 Jul 2024 07:25:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8AB1F6B00E6; Thu, 4 Jul 2024 07:25:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 6674B6B00E4 for ; Thu, 4 Jul 2024 07:25:25 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 250F1160D9D for ; Thu, 4 Jul 2024 11:25:25 +0000 (UTC) X-FDA: 82301839410.22.4AAC7A7 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by imf02.hostedemail.com (Postfix) with ESMTP id 0CB3F8001B for ; Thu, 4 Jul 2024 11:25:22 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=xYOmMn7I; spf=pass (imf02.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720092298; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JZrqpGDPBtXPHidCmEmeC3ZJ9JHyMn+COl4BmjCUDX8=; b=l7mRm2G8eM+JvNy9Qmc6HlaugFL2Wf8LREAiUzm7NO/tVFfYbROThLRYIaQiY25wZq6jeR Yi6qk0Wli3ZRpKwugJK2EngaLkY2mdaahY42AYq6pPkydgq8uALsEmvLMICIsfu8RjdY1F H1/RRtX9wcXAVz+QScAIOAwKHqwMX0Q= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720092298; a=rsa-sha256; cv=none; b=4yDzNtjRn27BwPURO3+S+INgWi4IWGT2f+VN245k6wyRmf6Uh7eSuQTKBxT/IV9jFrnOHP NJ3y/Hdb3e35ENKdC92cuOG4mRQfrsusvW2CIs7JHu9uTSsDGHG3ttFy7MlVeOSOTnoqDF 441M9GCA5abgljW0Vjz+Cj6zWV9/a6A= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=xYOmMn7I; spf=pass (imf02.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1720092316; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=JZrqpGDPBtXPHidCmEmeC3ZJ9JHyMn+COl4BmjCUDX8=; b=xYOmMn7IqJwvjMHo5oxtjlCcNa7bkIPKQaJrYJYo0D+ZB0r6Y8Fv2JVgI52Om5OHZzWVLUOFLn4d+ch/xJqEo3gZLS0FX8CKQq4krCi+dJsJfbuWinj8GNMg1k9U9wHfeKbUkT/NHhLFyl1EDL0UxSMgqMMiE7O5qbqA5Cb4EHw= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R331e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067111;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W9qjOyu_1720092314; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W9qjOyu_1720092314) by smtp.aliyun-inc.com; Thu, 04 Jul 2024 19:25:15 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 07/10] mm: shmem: support large folio allocation for shmem_replace_folio() Date: Thu, 4 Jul 2024 19:24:56 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: 8pigoymmby7geoixumf994asqo7b18ri X-Rspamd-Queue-Id: 0CB3F8001B X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1720092322-588683 X-HE-Meta: U2FsdGVkX18CvpxURsavzIN/9olhZxZ3EXTMsxr0Qe5SZVU/39ot5sq5bph9FmqSzIOqjrb+v0bzG9i8uT3i2yKOOvUgx4U3H7QXnJJgZ3f2sAiBuSFS3056YizuIUKvZp7cfJWLRCzZkPVs+Q4//SmCoUzDst5h2yILVQqtliGmw4vGPvrz9l+9AefKNw3cJeWWJ/Chk0jx42TDheIIX5qGNmb5lh5xy//Kj1haAi+1PW7EB7+jT0QvrUYe2Y1YaLkXjl+yAk2W5d9B7K3W4/IxQvLfdhW+ESFXsYWxlapXWpIGTVzTsQ2mPv2WryL2OJUF5hbyhx0bMNgpTsqfP8wXdFiKxGmGHXKF2KSkr2sCPR9q8Q1m/1288ZDt/fbqURCN+/URhUVxZJCdo1MH3L7lWfoo27uD2QmcK7row5+qmKjbg6SrT265zGYvRCL7a7Mc+NWeC5OJqlLoJCoBSeOfiRDU7f+YTt6zYHNiP6sPgw8lMNUPhyMLWBlG8jUY8QbhFHNQwkYfNKrzuyPWMdYqax2LmrKvB475wrIZZg4lOgUShZe1oIurAivMOyZpX4Ckh+GVkJN7JyUTH9194WqrFvkPUHB6QXvBHlfnMHuhRC6J4nLpzDgkK0HaVcMn9iAiEv0/3RVv9e+1G8KTcuYlbq8ad5F9pkVK0GVCL5Vy5hr4D0kiz9rpHt3A4APLArsakRUuO7bfNTRZkYFLGIhGZr6hxeBeODzRs9c0QmWgy33dxpWvygnx32oucOCP2SHZbIV8rm+VMabWmVCwimSWhOG+D1F0f/qkhJVpoV2uTyg51Ls17M8MBhoi3iqUhZjoCr9PXOnWTFkEu0AQ5oCXNz7AP9zGGAxf84NeAAOLDnVQ8x1m02N+EAXUVMquF4bXLd1mVIJVL0CVzwxO4w1Yrdw5qvYDIX0l5MP6T/fT/dhbNTNARI/yYtC5X1wTRiGZlNrjW7A8oeGvRn3 hOnMzLCZ N8Os8gkEK23ylKHNobu2E95m9GXpPOeqI9gezeSd8nUfyiNIbSEW7Airq1WB8Jkk0r8kTjKRE12sWZZQKilDXR6KCH0BENC/kqtZMTX8aAxr5wqQ85R/ZXiLHhv3kpWwbMtAw+qPki+3XB06bpgwEu547HQb6D44a/4brSQ2DCNedcKtNXvpFa3sXHd7J8UmOmRsmJ/+4E4QGBp2tJVe9E/YCl28MiM6LFRvRljN2P0SuZnACBVYpkwXhAPTWvopbyT64CSniEGblcY6vwk/FUdJRqIJuJ1qmBuSK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support large folio swapin for shmem in the following patches, add large folio allocation for the new replacement folio in shmem_replace_folio(). Moreover large folios occupy N consecutive entries in the swap cache instead of using multi-index entries like the page cache, therefore we should replace each consecutive entries in the swap cache instead of using the shmem_replace_entry(). As well as updating statistics and folio reference count using the number of pages in the folio. Signed-off-by: Baolin Wang --- mm/shmem.c | 54 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 31 insertions(+), 23 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index c1b6cdc61619..1718da7eefab 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1889,28 +1889,24 @@ static bool shmem_should_replace_folio(struct folio *folio, gfp_t gfp) static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, struct shmem_inode_info *info, pgoff_t index) { - struct folio *old, *new; - struct address_space *swap_mapping; - swp_entry_t entry; - pgoff_t swap_index; - int error; - - old = *foliop; - entry = old->swap; - swap_index = swap_cache_index(entry); - swap_mapping = swap_address_space(entry); + struct folio *new, *old = *foliop; + swp_entry_t entry = old->swap; + struct address_space *swap_mapping = swap_address_space(entry); + pgoff_t swap_index = swap_cache_index(entry); + XA_STATE(xas, &swap_mapping->i_pages, swap_index); + int nr_pages = folio_nr_pages(old); + int error = 0, i; /* * We have arrived here because our zones are constrained, so don't * limit chance of success by further cpuset and node constraints. */ gfp &= ~GFP_CONSTRAINT_MASK; - VM_BUG_ON_FOLIO(folio_test_large(old), old); - new = shmem_alloc_folio(gfp, 0, info, index); + new = shmem_alloc_folio(gfp, folio_order(old), info, index); if (!new) return -ENOMEM; - folio_get(new); + folio_ref_add(new, nr_pages); folio_copy(new, old); flush_dcache_folio(new); @@ -1920,18 +1916,25 @@ static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, new->swap = entry; folio_set_swapcache(new); - /* - * Our caller will very soon move newpage out of swapcache, but it's - * a nice clean interface for us to replace oldpage by newpage there. - */ + /* Swap cache still stores N entries instead of a high-order entry */ xa_lock_irq(&swap_mapping->i_pages); - error = shmem_replace_entry(swap_mapping, swap_index, old, new); + for (i = 0; i < nr_pages; i++) { + void *item = xas_load(&xas); + + if (item != old) { + error = -ENOENT; + break; + } + + xas_store(&xas, new); + xas_next(&xas); + } if (!error) { mem_cgroup_replace_folio(old, new); - __lruvec_stat_mod_folio(new, NR_FILE_PAGES, 1); - __lruvec_stat_mod_folio(new, NR_SHMEM, 1); - __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -1); - __lruvec_stat_mod_folio(old, NR_SHMEM, -1); + __lruvec_stat_mod_folio(new, NR_FILE_PAGES, nr_pages); + __lruvec_stat_mod_folio(new, NR_SHMEM, nr_pages); + __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -nr_pages); + __lruvec_stat_mod_folio(old, NR_SHMEM, -nr_pages); } xa_unlock_irq(&swap_mapping->i_pages); @@ -1951,7 +1954,12 @@ static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, old->private = NULL; folio_unlock(old); - folio_put_refs(old, 2); + /* + * The old folio are removed from swap cache, drop the 'nr_pages' + * reference, as well as one temporary reference getting from swap + * cache. + */ + folio_put_refs(old, nr_pages + 1); return error; } From patchwork Thu Jul 4 11:24:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13723637 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22D10C30653 for ; Thu, 4 Jul 2024 11:25:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B68716B00DF; Thu, 4 Jul 2024 07:25:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AF0A86B00E1; Thu, 4 Jul 2024 07:25:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91D2C6B00E2; Thu, 4 Jul 2024 07:25:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 675336B00DF for ; Thu, 4 Jul 2024 07:25:22 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 265691C0B97 for ; Thu, 4 Jul 2024 11:25:22 +0000 (UTC) X-FDA: 82301839284.09.67039F7 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by imf26.hostedemail.com (Postfix) with ESMTP id D01AA140003 for ; Thu, 4 Jul 2024 11:25:19 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=wGowpd2A; spf=pass (imf26.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720092308; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lRtF0SmyYSxOdTg4TDG7az1S/WCMj7v41FB3x6ABx2s=; b=yonb4qwOYdGnAkqd+i5aMBryj9Pey3oP/TFXpHH2aN8fAlq2hc9bGaTUa65p/DrRMMCxFE 6/ZpQSzEdS9jh8cF3T+EgKUEgywcMdC5Eq/b6JwOVCRlfxnut8QN3Z4vPDZa7aYlowgF1U FHQQG+/Ci3AbC2z1GRMFqHp+kCgogYE= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=wGowpd2A; spf=pass (imf26.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720092308; a=rsa-sha256; cv=none; b=i6+0Yyet4/Weem/or4HRbhuT9YNL/BxCYBkI82fa6cxfHMvDsU58BJORhLCFxNJ7A9ilx5 zB40zvl4xwekQ4SBZeGNaIvf3ML46jKiLjlV8i5OWLMpS+jyqz+L4/eHdKQkm8DxzYJdlB +IDRoNNr01pveVQrDr1wKTRfBDAFJ7E= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1720092317; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=lRtF0SmyYSxOdTg4TDG7az1S/WCMj7v41FB3x6ABx2s=; b=wGowpd2AnJSi+U10Xowe1qk1KCXBXaBMrx3kmgBsU6aHGBgfjW4o0nLlWb1vWZa67K3QVV4U7TrcdnduYixuFxrBR/M+reaOXXrgPWREoUht79mpK+LTbJizsFnT89NMqG65xIZDChpUunKTY+RJH8dYz5qgiUN6a3taE94zuBM= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R501e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033032014031;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W9qWV5d_1720092315; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W9qWV5d_1720092315) by smtp.aliyun-inc.com; Thu, 04 Jul 2024 19:25:16 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 08/10] mm: shmem: drop folio reference count using 'nr_pages' in shmem_delete_from_page_cache() Date: Thu, 4 Jul 2024 19:24:57 +0800 Message-Id: <871ca0ed707c6245c04751523f384d4832bed13a.1720079976.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: D01AA140003 X-Stat-Signature: fus4izc3cgumehr3riyqgn4zu8y455tz X-HE-Tag: 1720092319-30937 X-HE-Meta: U2FsdGVkX1+iKHKYr53SEM4ILK2AoLwEs5cN83HyhaCKgCE1tQEV3nrNWVuhBY4QJxjKRy8P00OQe1cgi/9urnYIg3qjc2Oa7JxtKN8nn9/T0PhHdKbW1KrZjzVEeSxjMe+bTS2lPauppo/1cp3/jpFF58x1sHUcQW3wU9wWPYrWxpOFlja6XT9Zj8ZjNTIqNe5ePpvrrCraG7Xz/cjGEyJd6ZqXl2JmQ9RA4pVI83aFAXix9Ga+uHYz3npXfHV49H/NbqwcrRvuFvwe/En2kl9ayFFT7c4/8ColiBWgu59o1K/NSDKhX5S40lgWOcsalIMedzBOPgZii86dmQ/P+4KILpPXCnwezKlva9gef0BbBaZ2m4iy9lBgeRluGym5LXUciPvDGWasqjMEuNqh6pE0PEHQNbgL/OZyMsToXp1j2LEapBxdUl5QgjlmD8ZfxKss2qRcp6ozpaQ8ZWwpSWYN4w06So1hy0MPcf8HMD09OIPFyqgTWMrfqn19k8ZXUEncqZUOKzwsQLOX+toGmA/qYrj7nAzW8JrJ7ZLYXYCNwoWNWcvq1X2XqIwFpZsju4qMDmOqz8goLmVfoLWYLTMM/d2ynAXy+M1uV62GAk40weEgdKx4g+ZrLKPFcg2+9hpYjf0bBYchNkZVH6+5Ww5FHRH/9seHjnCwVECMIX9e7YhlDN+M69FbUfT+5GlQJzFdALB60wtqEyVG7qwqXD8rh/0IL9ALPSfoQZuqWa8+DqlfMTAXqszUmbId3tzXRaycRul0Pp1o+MZcXLgeSaEL/ddjJYMv57PRCjtq3bOXKhNF5hRU7dDW90qAR4zUuvAnbsTj/FyBy6SA2FptgEPrthpNO5ntOuKaKTNaC9Bkp5Nm740CqsFZ/XAo6R7dFxuEl2McbR61eRmRpQ6ERkpLKF7YUCmXvjg/+nH0ixpNIVdvtIuHNtRiW7kXji8otUH87dpU2nzBqIUMPb4 /b3BcAO4 94sOtVrDEh0wPALB5fUojtn8czmznPKBLHMFU+27StEUYuWtQgpU+7MNbQcuxxnlqBNxBD+PnbVEZaA1CM7IXcMEUJeSyc9My0YGwAAtYn1EgorVaLSsVjN2R+XCM9vN/SxiucJHhAS87oD+4YKgicobMKXW2x/2hrRAG1rUhTG20BOBa+9YUxAIO7Kx8Xb7VVO13+MMHBvB9iB7dJkWtFu8xbE6fk5WgsYBmA5OnxRjTpaKkm5RV4eeqp8P/4IRteabnRyjR6c9OXKKdkaB0Fd3hk1nZtzrz2ix8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support large folio swapin/swapout for shmem in the following patches, drop the folio's reference count by the number of pages contained in the folio when a shmem folio is deleted from shmem pagecache after adding into swap cache. Signed-off-by: Baolin Wang --- mm/shmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index 1718da7eefab..eb030827f7fb 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -842,7 +842,7 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap) __lruvec_stat_mod_folio(folio, NR_FILE_PAGES, -nr); __lruvec_stat_mod_folio(folio, NR_SHMEM, -nr); xa_unlock_irq(&mapping->i_pages); - folio_put(folio); + folio_put_refs(folio, nr); BUG_ON(error); } From patchwork Thu Jul 4 11:24:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13723638 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05A55C38150 for ; Thu, 4 Jul 2024 11:25:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 54D1A6B00E1; Thu, 4 Jul 2024 07:25:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 461C66B00E2; Thu, 4 Jul 2024 07:25:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2DAB26B00E3; Thu, 4 Jul 2024 07:25:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 038AE6B00E1 for ; Thu, 4 Jul 2024 07:25:22 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id AC34F1202A1 for ; Thu, 4 Jul 2024 11:25:22 +0000 (UTC) X-FDA: 82301839284.19.E3E20F4 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by imf13.hostedemail.com (Postfix) with ESMTP id AF32A2000B for ; Thu, 4 Jul 2024 11:25:20 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="IrS06H/i"; spf=pass (imf13.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720092301; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8wAGNXuI1qwPyzOykQd8DO+mgTfhYkp9bBwCtZr1ZS0=; b=tvH1DVusRoFzgJHp7QnXVJ/qcTNX1QIj5abEJP7dn6gUd2xbORGYOnkNxxxnBTWa9qsbAC dCn3+RccDb2wRmap9cSZvkiUz7vy9PXE5O67Xbn6rxQbQPkftDOfx2PylQEe7GL8ByMBUT p2UjfmvGanFlnpyejLuj9nIrRQ3LaSQ= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="IrS06H/i"; spf=pass (imf13.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720092301; a=rsa-sha256; cv=none; b=LjazM8Jtt66f1guQ1TuU7TUmTywwEa+17tk3P+B7qp2qvg/1TP1AF1awZHQRUMGCWtjV8R jYh2TYqzU5fkckRYSfDTcjfT0yLoAvQKN7Ik3S5p1k36lGKNE3RYUfaIkcS0Ea89EAHsbj Nx3C5cAA++68JtNa5rpJ1C8YHSkAcDA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1720092318; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=8wAGNXuI1qwPyzOykQd8DO+mgTfhYkp9bBwCtZr1ZS0=; b=IrS06H/iF5itsGarec0zMq66fvnKviknZOMpv4ixZNbFt+hT1k9Mc9heQNTieGwAT6uEDaE9DleKwhrlRIZNQasOBa/b1G+gd51Bt9fmVKlEyY3ZLachUipr7SzDEYDAZuCRw/laQw7Joig4PDNedgwDF+tgO1GmOJb2ZH38SYg= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R111e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033023225041;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W9qWV5z_1720092316; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W9qWV5z_1720092316) by smtp.aliyun-inc.com; Thu, 04 Jul 2024 19:25:17 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 09/10] mm: shmem: split large entry if the swapin folio is not large Date: Thu, 4 Jul 2024 19:24:58 +0800 Message-Id: <7d831561b1daa14234c409cb1677a367d3ce5e0b.1720079976.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: i7dsy38z1x9cdi9naw81o5o6rwh58wi7 X-Rspam-User: X-Rspamd-Queue-Id: AF32A2000B X-Rspamd-Server: rspam02 X-HE-Tag: 1720092320-651773 X-HE-Meta: U2FsdGVkX19KPhEpRfjTNaewHhbHfbtHb/Kecd951Aad/ldtlJlJQBKCPBNkIUCe8P+OBzau03sy9zTbMtrGyXTycrbooy3qm9P1nyH75RorvN4risFyvA2twkKYrrhXj4OjZ0AF2pM5I9dL8GLRHhbuaTIa9PI2Bqpm5Elm64Nl6sgsfLmzClLh87LGQra5OP/9FEb7GBP8CZj/RfucAf0B1GTH1ROuj3oGsGi4YrbAHz/NxpnIF895yp0mNw72rxdHSFYOzS0AaZQn7XJMeqCq9naNNRB87jeBjCH1crnec1Si2UPcnmmuFiCT+1eSbK6uZ+0ScAv6ehU+XcEsHleU9blIu/PfqTIskKjFuWpoek7OLUwrdZWtJy7uD4NQDkQCqoT3tKfwpWy7DN3b4409+qCLaGPvx+US+GQnsk+OgkrhdMaX+TT7lGOKWg4e7/mCQGQ+yTs3l9ezC0z0QN2122mPZe7AvzXJFMJwz0njg/lOafMZxbUzj/NMQbdhEJfCxAoBHKSfddoWqcM9h8CBY/2C7jga1H1RCz4AkZjaGYVyBn+3Wk63VrzhAcnhkb0eEDrBLQO4J/1eAgHIj/vpbBO8sK+4jmM30Ca+CBiItias61ME1nH9BPrAcp414L0tooqk+NcOZyuqC8NaFwD1SX0Ojd0QQp6JnBCgKCQrREFvFoRE8KGEtqwNtPKKllc7h3Xva/rBY271vVHlJHj/rXnsNn+cyJCmcVYGr7cYC6a7sAVfpzEAjDbu+U2j1I3H8BRHcbeHbNNnarBbPGRptncxLRYR7FC6bkdY/8Cd6H1OLb5c99Ucy/xlTRmhW5P/GA7fr0oU8YGKMQ6FfdPNcb6/IPO95lGD4U3zzsxXmhlDdSJ9b0Zum1tQHtimyn+8bQK/VbWS3EHePupnsNjPnAmoTLnqp5VscSrW+w9CTjnnQevMp62LNKJlEau7kDE+nBWq0NtW+hOJHME jvJwQwGa 3Apm9KR/lpfAXwcVX19KJrjaYjQGJ2sjpIUXc+JfYlzd4HRP+tS3aP4JSbQNqFhRmO5gNMygcfDAX6KRZut2QIkoi6J0oU9/f6g3BigDqVCWPB+Uksb1EuqeuReoYtvAjgwbXSmbf8z7eZrUXCMFWF1FHvIX9H4mdwHYrmvyEpbeCXJyf1wAczcK5Ez8vU4IRk4rurnwNNraO88iLiulTFUmc82YFRN1wGtcTgF5qmrxZo6M= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now the swap device can only swap-in order 0 folio, even though a large folio is swapped out. This requires us to split the large entry previously saved in the shmem pagecache to support the swap in of small folios. Signed-off-by: Baolin Wang --- mm/shmem.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/mm/shmem.c b/mm/shmem.c index eb030827f7fb..b4468076f0e9 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1990,6 +1990,82 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, swap_free_nr(swap, nr_pages); } +static int shmem_split_large_entry(struct inode *inode, pgoff_t index, + swp_entry_t swap, int new_order, gfp_t gfp) +{ + struct address_space *mapping = inode->i_mapping; + XA_STATE_ORDER(xas, &mapping->i_pages, index, new_order); + void *alloced_shadow = NULL; + int alloced_order = 0, i; + + for (;;) { + int order = -1, split_order = 0; + void *old = NULL; + + xas_lock_irq(&xas); + old = xas_load(&xas); + if (!xa_is_value(old) || swp_to_radix_entry(swap) != old) { + xas_set_err(&xas, -EEXIST); + goto unlock; + } + + if (order == -1) + order = xas_get_order(&xas); + + /* Swap entry may have changed before we re-acquire the lock */ + if (alloced_order && + (old != alloced_shadow || order != alloced_order)) { + xas_destroy(&xas); + alloced_order = 0; + } + + /* Try to split large swap entry in pagecache */ + if (order > 0 && order > new_order) { + if (!alloced_order) { + split_order = order; + goto unlock; + } + xas_split(&xas, old, order); + + /* + * Re-set the swap entry after splitting, and the swap + * offset of the original large entry must be continuous. + */ + for (i = 0; i < 1 << order; i += (1 << new_order)) { + pgoff_t aligned_index = round_down(index, 1 << order); + swp_entry_t tmp; + + tmp = swp_entry(swp_type(swap), swp_offset(swap) + i); + __xa_store(&mapping->i_pages, aligned_index + i, + swp_to_radix_entry(tmp), 0); + } + } + +unlock: + xas_unlock_irq(&xas); + + /* split needed, alloc here and retry. */ + if (split_order) { + xas_split_alloc(&xas, old, split_order, gfp); + if (xas_error(&xas)) + goto error; + alloced_shadow = old; + alloced_order = split_order; + xas_reset(&xas); + continue; + } + + if (!xas_nomem(&xas, gfp)) + break; + } + +error: + if (xas_error(&xas)) + return xas_error(&xas); + + return alloced_order; +} + /* * Swap in the folio pointed to by *foliop. * Caller has to make sure that *foliop contains a valid swapped folio. @@ -2026,12 +2102,31 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, /* Look it up and read it in.. */ folio = swap_cache_get_folio(swap, NULL, 0); if (!folio) { + int split_order, offset; + /* Or update major stats only when swapin succeeds?? */ if (fault_type) { *fault_type |= VM_FAULT_MAJOR; count_vm_event(PGMAJFAULT); count_memcg_event_mm(fault_mm, PGMAJFAULT); } + + /* + * Now swap device can only swap in order 0 folio, then we + * should split the large swap entry stored in the pagecache + * if necessary. + */ + split_order = shmem_split_large_entry(inode, index, swap, 0, gfp); + if (split_order < 0) { + error = split_order; + goto failed; + } + + if (split_order > 0) { + offset = index - round_down(index, 1 << split_order); + swap = swp_entry(swp_type(swap), swp_offset(swap) + offset); + } + /* Here we actually start the io */ folio = shmem_swapin_cluster(swap, gfp, info, index); if (!folio) { From patchwork Thu Jul 4 11:24:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13723641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CB0AC3271E for ; Thu, 4 Jul 2024 11:25:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E4FF6B00E7; Thu, 4 Jul 2024 07:25:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 28BB96B00E8; Thu, 4 Jul 2024 07:25:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DE386B00E9; Thu, 4 Jul 2024 07:25:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E10996B00E7 for ; Thu, 4 Jul 2024 07:25:26 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8907CC11D0 for ; Thu, 4 Jul 2024 11:25:26 +0000 (UTC) X-FDA: 82301839452.20.15EAFE0 Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) by imf13.hostedemail.com (Postfix) with ESMTP id 8DA462001C for ; Thu, 4 Jul 2024 11:25:24 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=rahRraVq; spf=pass (imf13.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.110 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720092300; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dbFBoHDVwoZIsZWEIcvlBAvgPzWvo4rCLj+4qVnfJRg=; b=FQhv1ZGpeojqhrNiYwV3VufuxTeJPLb7rj/ZQcBDaQBfkf68ukHSOIWcvDa/TWNEmVOK6A YPDCo5OM74C0NKYBDS6o/cEO2nTVWThzPgu0XG7uQx7c21o29bSS6jhc6wyijpv1xCbG4b dSSoXzsOYYQmAf7ikuAP1xayxKuIgDQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720092300; a=rsa-sha256; cv=none; b=b6PolzxBoHDR2FPhlS6SWtFMeB7rBlDPzDEZ1z200SKyuOPDxTcHOU8fj2MLuVvqC8e9+L OJe70P3fg9WoaGDseLPXasvfJe9ixLQ4YWl8dhRs7g+RAElVVaZtrQnXg+nT0Fm0Skz13y CQEFXH0KnmvkdpmsnKkOoLMxrFzEPWE= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=rahRraVq; spf=pass (imf13.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.110 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1720092321; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=dbFBoHDVwoZIsZWEIcvlBAvgPzWvo4rCLj+4qVnfJRg=; b=rahRraVqHt0JvpZS3K8rdTtqhqlQd8VPlNtyDEvQc3rIWtNQQKzJiRkpGxDizbzu0dW/ZgObddbO175QUlviHudZedR5yNLYVkn2gy09R3Euh6zCS8zazvS9t0uUcBcmQx7q6LydK6cf3gTT3IulNOPrk4Yd7Un4DBQcqDF7QXI= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033023225041;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W9qjnPn_1720092317; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W9qjnPn_1720092317) by smtp.aliyun-inc.com; Thu, 04 Jul 2024 19:25:18 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 10/10] mm: shmem: support large folio swap out Date: Thu, 4 Jul 2024 19:24:59 +0800 Message-Id: <99a38bff0f562fe4844162f8abbe3840d1fcbbdc.1720079976.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 8DA462001C X-Stat-Signature: qn5sk8yix8cennw7fjxhmswiuas3a36w X-HE-Tag: 1720092324-870072 X-HE-Meta: U2FsdGVkX1/ekpadbSuLU0fqWffg3qfwrj1Csc8qDI7ZkLsZ2KP/3LnTSNwdqzX88e03M5miSwNIKeM/+TU41LkCS7ckCMIjF/LD/KwbRTrRdBrnXQM+2Q3mLViPylGiZHsrd22fD3msMn/arJiE18cDM7ThbJqJvsvLlish7liDA32xoHtepvOqmyJ0vhPjrmr4dh3FNBtSP1GjuIo7C+z+gGnHvMoWK7G6vSH8ehbNmag29gCd0/pfpdVNPDZVa5nVuaxEqzoa8IozOAivXM6qAB9YrsIlg/k/rpd/UhPfM6HG5+8Qpc7++6TQpSVVqHGmKbW1OfXPspd/QtxV/PgN8NztGwEMqOLFK291fdL6BDMh9njzvsCftihJA2WMwNA1ktn/j5g4PjzCrwAqWRf2/40aZilPSybL2RlhinqBG8mmZ65EdLEhWrMkfBFiPcusJiZOCoJU+5oBe5cpS4GVIBp9D+K7EZ0S6jiP4Ri3iyGSowJh/Qf/YXWxPPt8+yDcF11MrqUXXXZh/RFXRGvA8auDA0Jm9X/UKFqExOPRh58VjzT9jmkq+FFxIYr3phkK1DviR2jnqQI5V+j7P9JrL9LSpEQ7/KIYvmFXx9K2AHbCBOLAmop8lhbzgLvM4YN0JuCTLtAd/QwfqjXLd8i8OSoIGmpsGEAUPw3OGVSP8uMlFf/jLD9V/KVv/CiIvNMcXM7EG3m6wUeG41YV+GOx5yFktTxCzFkqDFuGxiMV4J3f9ipyg2mV1ggv1OuJzrZxgQ+nqmE+VsZY76S+qP4tALU+8n1j6meRcrLwsPSx23SreKEd2iYAOIcGCJRpWEFmwNBJVpTLAdab41tX40TbVJkc0IVeBqb8pKTTTDgbPFFuzTrvU0uvIMUt6ILNQbag7UnNS2PT0P93u1ewI4YRPLIFfCLzYFERqkRArPsT6iruXxYGWjX0Q1aS+Rg0JTxsdByp5CIfdSgyqnk UIMA8gNp MW9bMIRJIzO4/3DKLYUJga2scVhQszSBdab4vfHXHN28qrOfcyTR3Fz+tKmO5vbNc7whzej4So7pz3QzKoBxy6yJ1KXwNuRP0QAETorZ1fy9ndexgFVoUT5t6TQ/GDP+LVC8I/9KXk5rTW67Ak/xF9/RP20IBe0paW3YOGfOiMRfWGzphyHj2g/nBeUO6jrWuVQ7pFDabJO9dkhiyUlG3Ai7GF5zcuzPTfJtfIwr4JPvfBCmJpErspivflYrffPImDGdyw6SeQ9/JU3s1Gm7MIJIOnr33f88H8ztTe10pa24aLMy+kSF+bEAuUprmfVoUykKJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Shmem will support large folio allocation [1] [2] to get a better performance, however, the memory reclaim still splits the precious large folios when trying to swap out shmem, which may lead to the memory fragmentation issue and can not take advantage of the large folio for shmeme. Moreover, the swap code already supports for swapping out large folio without split, hence this patch set supports the large folio swap out for shmem. Note the i915_gem_shmem driver still need to be split when swapping, thus add a new flag 'split_large_folio' for writeback_control to indicate spliting the large folio. [1] https://lore.kernel.org/all/cover.1717495894.git.baolin.wang@linux.alibaba.com/ [2] https://lore.kernel.org/all/20240515055719.32577-1-da.gomez@samsung.com/ Signed-off-by: Baolin Wang --- drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 1 + include/linux/writeback.h | 1 + mm/shmem.c | 3 +-- mm/vmscan.c | 14 ++++++++++++-- 4 files changed, 15 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c index c5e1c718a6d2..c66cb9c585e1 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c @@ -308,6 +308,7 @@ void __shmem_writeback(size_t size, struct address_space *mapping) .range_start = 0, .range_end = LLONG_MAX, .for_reclaim = 1, + .split_large_folio = 1, }; unsigned long i; diff --git a/include/linux/writeback.h b/include/linux/writeback.h index 112d806ddbe4..6f2599244ae0 100644 --- a/include/linux/writeback.h +++ b/include/linux/writeback.h @@ -63,6 +63,7 @@ struct writeback_control { unsigned range_cyclic:1; /* range_start is cyclic */ unsigned for_sync:1; /* sync(2) WB_SYNC_ALL writeback */ unsigned unpinned_netfs_wb:1; /* Cleared I_PINNING_NETFS_WB */ + unsigned split_large_folio:1; /* Split large folio for shmem writeback */ /* * When writeback IOs are bounced through async layers, only the diff --git a/mm/shmem.c b/mm/shmem.c index b4468076f0e9..d93efcc64c3d 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -786,7 +786,6 @@ static int shmem_add_to_page_cache(struct folio *folio, VM_BUG_ON_FOLIO(index != round_down(index, nr), folio); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(!folio_test_swapbacked(folio), folio); - VM_BUG_ON(expected && folio_test_large(folio)); folio_ref_add(folio, nr); folio->mapping = mapping; @@ -1473,7 +1472,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) * "force", drivers/gpu/drm/i915/gem/i915_gem_shmem.c gets huge pages, * and its shmem_writeback() needs them to be split when swapping. */ - if (folio_test_large(folio)) { + if (wbc->split_large_folio && folio_test_large(folio)) { /* Ensure the subpages are still dirty */ folio_test_set_dirty(folio); if (split_huge_page(page) < 0) diff --git a/mm/vmscan.c b/mm/vmscan.c index 61465f92283f..fd503506262d 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1245,8 +1245,12 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, if (!total_swap_pages) goto activate_locked; - /* Split shmem folio */ - if (split_folio_to_list(folio, folio_list)) + /* + * Only split shmem folio when CONFIG_THP_SWAP + * is not enabled. + */ + if (!IS_ENABLED(CONFIG_THP_SWAP) && + split_folio_to_list(folio, folio_list)) goto keep_locked; } @@ -1348,10 +1352,16 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, * starts and then write it out here. */ try_to_unmap_flush_dirty(); +try_pageout: switch (pageout(folio, mapping, &plug)) { case PAGE_KEEP: goto keep_locked; case PAGE_ACTIVATE: + if (shmem_mapping(mapping) && folio_test_large(folio) && + !split_folio_to_list(folio, folio_list)) { + nr_pages = 1; + goto try_pageout; + } goto activate_locked; case PAGE_SUCCESS: stat->nr_pageout += nr_pages;