From patchwork Tue May 21 11:03:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13669302 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4FAEC25B74 for ; Tue, 21 May 2024 11:03:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D10716B009A; Tue, 21 May 2024 07:03:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C98A36B009B; Tue, 21 May 2024 07:03:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9D066B009C; Tue, 21 May 2024 07:03:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 857B06B009A for ; Tue, 21 May 2024 07:03:40 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 38F6EA1B89 for ; Tue, 21 May 2024 11:03:40 +0000 (UTC) X-FDA: 82142117400.25.546695F Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) by imf25.hostedemail.com (Postfix) with ESMTP id 0E349A0020 for ; Tue, 21 May 2024 11:03:37 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="mIdcj/Ol"; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf25.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.110 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716289418; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jRAtHbKubpnpADzITDXZT1b01GiilShGna80PhKpWSg=; b=7cMQ6nZs9YSIhTLdg1bYWf01sPNKKQKaXX1KrxP0e6XxZdnlOZpsZjWWA067DQE85ltHx9 YLNg0ePLGWZ/FBkya7ZbAKrLvvas1B3yrtAopP/pPkHo6vLz5Wcf05jeC3fB6vCmZB43Pv Yr6uZXIzHmQLvYyR5drhhV6PUDATkWk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716289418; a=rsa-sha256; cv=none; b=J7yXlyfJIoQnzKjFDlTx+LAPFGnYzMk7//zh1nlQciCuZDGPXLWc4Y73kv/rwxM2ofAha7 H3eC4ZOO7/TuEA6e3HzHDNWlNuY7a7MAFBuFCph5EfdVm4EmS2IdhWjd5qFvsBHzEusLz5 W/z2XIdSdeAH9WT4mCrAYuJ5QYVoHHs= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="mIdcj/Ol"; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf25.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.110 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1716289415; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=jRAtHbKubpnpADzITDXZT1b01GiilShGna80PhKpWSg=; b=mIdcj/OlS6r54xfMvfyidSJBugDf8yQ0/bVuXXQf9oFHrqI5G2jqWRFglnNdFnB32FKkJNGDHPA1IB5liez2bZeIVYEKpkQN2rlcHB5neqaNnX34Nu4ItiZYDe+cJHkTQvskqSAnKMqb0c8z2LlkIlaZTgNLMzLc+BfSTr2NKto= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R111e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067110;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W6xeTAt_1716289410; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W6xeTAt_1716289410) by smtp.aliyun-inc.com; Tue, 21 May 2024 19:03:31 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, ioworker0@gmail.com, hrisl@kernel.org, p.raghav@samsung.com, da.gomez@samsung.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 1/8] mm: fix shmem swapout statistic Date: Tue, 21 May 2024 19:03:11 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: sz6ztb5oudyoswyebi3xsgck8bk34grj X-Rspamd-Queue-Id: 0E349A0020 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1716289417-541954 X-HE-Meta: U2FsdGVkX1+iiuPn/g4fEyFAiEt4p3zRICVoYpiS7RpaNZN+MycfWHVAEvPmbWlKHnVY3nhl9+9WbICh39TJbJYToIpcay0iwPDq1UrWN83lPPYU+cbTdUpvjksGb+r3z/PHXZjoC3K60fOH27qzEL7zMXkgd2d5QvOiziLojfSK+Kz2Rl2E2lN5m+dULwhzD3ZGviBql0WHOrCxGi5ODkBPgPWlAO9ioZ3Nrx5oPMe0t5eGfHI+pvECVb1dOrlVo8EcjJVSc1e2ffDEICGzJLYsGc50IWOTakBfA2KyYlj17DK2xPRpwXPZIW+Oh7Lg/ocg1pqZb3cGuCFDUlV8c3UpwWyrVcrA05SvVchKUJtWDmNft3bVbn5H28GyFIqt+GB5rAg96XuyE/N+eum64Mxlmc2RKQFVu5ASgpKXogjLui+5qLvNMrPoXJ7LePpU/ZFGvQUnJc/TVfX1Krog9n53Bxslxt3YkJxK/7v07HzXGnf2VMem6g8DA2dVv/psaVkO6V+/tyVRCgF1aRe29LxGtZ+bw7/8OpWK1IOA6EQQRWvYX03cXL8vn9zyv3ZAEm9RsTXE57XEQUDw+mEcOAVWjNDJYT3x+oHueJOcPbjruPPrCYAP7TBfw70IXkW5nrY9doykueFRfYq4Q6q/d3eOPKTx14XXFwPoHjKiS8ylVt21uAWcjhNe0pSP7SKdCvpJpZA4lTMk15oxBetnP53eRt5R5rlbP7KuTDJSUJ4Lfe/LTo0yxKurqa/c+2CNEh6VKYHx3DgxS/EzidsW5Y6eEhY6FiDD7PndI1ZY9wrz+mqGb84V8fbxa1Tox8EbuLqO+7/zs87OffIaS45HrbXzgM1ScT8keUN+Y52Ml0aAO1GN6p2Gzg+x6/BbsmWmtQMWrwag2y3XZ73+nm1eVz80vdj1K3pVRp5FfU2G7doI84tgwkOQpMeX1Ji+OFV2sylmQJsSNt+tFpXkw8B MsGri0cO sBqycGh1+slFdcbHgaJP4SvMj2rOYrYVd0wBzvlmzpljwegWfQLEnXgaDxqVe0EG7X9rY46J2aiUxDqGJtrhUpfI1sF9opeHOWAWYTLTjw/14Dm6Um2+hcMP1n3pQwVjRl3XKRLBPLZU51hzmsiA/7GTFN/F1rAF21NYWVosJkLcnXC5JMeaIkbAymE92dGfxGD63tCI8i/iO+Nu82nI3LcAFK96oXYV+9+hvbDoBcnGCmak= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As we know, shmem not only supports the sharing of anonymous pages, but also the RAM-based temporary filesystem. Therefore, shmem swapouts should not be marked as anonymous swapout statistics. Fix it by adding folio_test_anon(). Fixes: d0f048ac39f6 ("mm: add per-order mTHP anon_swpout and anon_swpout_fallback counters") Signed-off-by: Baolin Wang --- mm/page_io.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/page_io.c b/mm/page_io.c index 46c603dddf04..b181b81f39e3 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -217,7 +217,9 @@ static inline void count_swpout_vm_event(struct folio *folio) count_memcg_folio_events(folio, THP_SWPOUT, 1); count_vm_event(THP_SWPOUT); } - count_mthp_stat(folio_order(folio), MTHP_STAT_ANON_SWPOUT); + + if (folio_test_anon(folio)) + count_mthp_stat(folio_order(folio), MTHP_STAT_ANON_SWPOUT); #endif count_vm_events(PSWPOUT, folio_nr_pages(folio)); } From patchwork Tue May 21 11:03:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13669301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE42BC25B7A for ; Tue, 21 May 2024 11:03:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34BC86B0095; Tue, 21 May 2024 07:03:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2ADE76B0096; Tue, 21 May 2024 07:03:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D8EE6B009A; Tue, 21 May 2024 07:03:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D974A6B0095 for ; Tue, 21 May 2024 07:03:39 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 91447A1175 for ; Tue, 21 May 2024 11:03:39 +0000 (UTC) X-FDA: 82142117358.25.37D39AB Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by imf29.hostedemail.com (Postfix) with ESMTP id 7F279120019 for ; Tue, 21 May 2024 11:03:37 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=lz3wmb9h; spf=pass (imf29.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716289417; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=emwz+iI9NQ7GxFXPBlPImuiv2PaM3+Taa8lvRwiumnM=; b=3+TL7LCauXkWOW5DUwOlrafE5jHy+U7LxwS9rLaZB5qkHbleCO4O6lkhnd9i1MNraN/FYX z02C81k/V8MoC6OQHtQko/xWcLvHOVQWPMAGT9HYftWp+dINinnOb4GwVQo7kHMafGEyZW oF5vuvoIgK0jkI6/0bKFL7WYcobGhtM= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=lz3wmb9h; spf=pass (imf29.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716289417; a=rsa-sha256; cv=none; b=R2B39GyVCZksU9KQvXuT39AwoaIVJvUskPj5C7W8iugAsip7Y+KP/9a9y/yY1NRflkSbZD 2AB511oPafp4X9cFrIfL1T1fr4sFb+dpjXXIChua/KClc97cYj+Kx7hTdpfUKUNw4QQdsR N8zx+m5Cmzt7n1YxVL1GzWNiQ9mrD2I= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1716289414; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=emwz+iI9NQ7GxFXPBlPImuiv2PaM3+Taa8lvRwiumnM=; b=lz3wmb9hM7NQm3xqYf8LPwVLZmpLvXu3uuGv0pvPh9RnGTU8FhSIHjQeHOmBJyr7IWGr9cAraPwBnK7LvvMbuxJNxcDCh0XiuLweDjD+LUCfhZ1XFVja4BLLP45UjsWr04fBiQEtCodoi18gUa21KVDxK1jeTH4cGU24+EPeKfg= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033022160150;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W6xl94l_1716289411; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W6xl94l_1716289411) by smtp.aliyun-inc.com; Tue, 21 May 2024 19:03:33 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, ioworker0@gmail.com, hrisl@kernel.org, p.raghav@samsung.com, da.gomez@samsung.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 2/8] mm: vmscan: add validation before spliting shmem large folio Date: Tue, 21 May 2024 19:03:12 +0800 Message-Id: <5c919e5002739d2f396918d11bde5711d4ae8182.1716285099.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: 4e6ocxcupucqfts4ou1e1h9n4fic68wa X-Rspamd-Queue-Id: 7F279120019 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1716289417-713021 X-HE-Meta: U2FsdGVkX18WpNjfuc1gVIjy+KexlfMjhz12AGGu6o+SQlCEKRnf49/SGitqYBCNfMXj9eN3ldFDfgXrK1BhCZLFGP1LeQuJEVIPzS36ZbS2rlwPhb9SQ3f2ANQ0G/CYMoEsncw6PCbWXcgZ1IhCWvzWl495g+2X8AiD8hWXlCmvROcXKhAwuRNt/oG0aK+oxmdsScZE/lGg32OtSkiaxDUNaBrtxXDZ2IzJnWrui2Idfe4DT7DnYXR+QPCmPdpciHs5FqRVz0rwJzAuapPCd9iOGcRCQB7ln9NPSuZEEvNddyHg6gFF0GBtRfhoyJZt0DkO245JctpVN1mzwYcgnCE4L5DGItmzbIZZbLMlO9ZiIwzKv9OGrZpdmPajBXXAIQQ3cKToltFLeBW4MZzPmOYcnvAVDcrvloCZtXiSYRmaDyr5dTIKvyz8zf++j2x0htLP1rhZcAvwVGxi/rxpUhGsOA8gWGwqttWZX/55KRfYdkT0n6GmRBHVg8NCm2ImqDj1waoYqtNurVqoaCiLpRvD0jAsfhVeQPek7EEnj5Zk1TEY8dN6PmL/40pw8+l3WhMzB0Cv9xerXWYxZ1dY1YvK396MM4frY2yW/nAWN4H2RC/jAvHlFMQi1yBCAU/4UbVkNyggxGu1j3JYYH9fingGRO9p/2bJDYn6jRaGBMzOiBjKwIVoGVRmfAlYv/HeuoGJVCquuMF5Lf1prGJtgwQq1fIxMSDmqL53mQiezLMfKTjRZZHWt+aESavqbXWuJiEEiaWMNSg+JENpvo3v+FEbhT/Qz7LtDuT/c+VVE9c4/bKVS37AbcDT1WiSRmPUNW7ykrvTrO9xQ9M9WmRWJ8OT14V1J5FSXFbVQvnpJ9yHvTA5UJ0iFV6ypJW4A3luPeDvaf8wvUtMnQjndQHg+yrDXbZwy7dmdmsAxgBjQgXoXhxUzFr2RKbePZwmkaUNWocyTrVn+VLgIfOVcPA I4DDVwxg JXcOOKqvh5ZEvRJYj4CpjzUM1vmjksm+7vCGCmYg4jVZFtKuXEmIi5WpmWLQWnmP7xfEa2hjAUL8YrfdSUjqO9Uicxp57f1SCAereRL+NIFmvHkeqBQdbW4QSJyB1xRV78XoVt0Fo2n5gjzgOybogNyVU4EaAUU4FlGiGNKGvQI03VR2zf+lUTgDpaxhLFBShUnS+s0Uwz+O/KC5JzxyeA4F2GVI9R4sI0phUsb/TgmGnLYf+LsY8nwa7GUZOg+k9XvWBg6AImG2wZDE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add swap available space validation before spliting shmem large folio to avoid redundant split, since we can not write shmem folio to the swap device in this case. Signed-off-by: Baolin Wang --- mm/vmscan.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/vmscan.c b/mm/vmscan.c index 6981a71c8ef0..bf11c0cbf12e 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1252,6 +1252,14 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, } } else if (folio_test_swapbacked(folio) && folio_test_large(folio)) { + + /* + * Do not split shmem folio if no swap memory + * available. + */ + if (!total_swap_pages) + goto activate_locked; + /* Split shmem folio */ if (split_folio_to_list(folio, folio_list)) goto keep_locked; From patchwork Tue May 21 11:03:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13669305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5391C25B74 for ; Tue, 21 May 2024 11:03:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 411486B009E; Tue, 21 May 2024 07:03:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3995F6B009F; Tue, 21 May 2024 07:03:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1EBDB6B00A0; Tue, 21 May 2024 07:03:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id EF2E56B009F for ; Tue, 21 May 2024 07:03:46 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 45232121176 for ; Tue, 21 May 2024 11:03:46 +0000 (UTC) X-FDA: 82142117652.21.CBBCCF5 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) by imf17.hostedemail.com (Postfix) with ESMTP id D149E40027 for ; Tue, 21 May 2024 11:03:41 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=QqaMVMxk; spf=pass (imf17.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.99 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716289423; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AnBwKR9CDgtaWp21L3C4wGJsMstG019ckRvWwRbPXDM=; b=4ey8Qw9vMUETZ6zrjtxo6HaiqQdB1NCon57zocNNndUKCvVOuXrS6HxQXIv/V15UGee3Eh awoPB/gjbiyLp8uNCRBPx0WdTAfx+tSCQq8VL94b+xCmgqBe6fzjy0HhXVs81/rkMQlTHv syXk8izF93/VsEkbGwFKsisQoPJOagY= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=QqaMVMxk; spf=pass (imf17.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.99 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716289423; a=rsa-sha256; cv=none; b=RBLS/0o4z5LmYiuDOWpQSuDP2CLdFhteTcYNctKUWYWC9l+XnWCRMDIPVMsK+ikKCr7SmI sLfcmEBUyIVdRNt0ncq/PV1jQ+4xYfdbU29CbznoX9za/RmDoDnmxT36EOWxk+d4O+2t9K rpNZf5O5VurSe5SnCwoDfr5N+tkM3l8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1716289416; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=AnBwKR9CDgtaWp21L3C4wGJsMstG019ckRvWwRbPXDM=; b=QqaMVMxkO39baTbt3EFevek7dyw2Du373ayknRz2U6NGfFkPoppI+sT+Vo17XfRqfZYGONl4Ug32Ot3vzD/bW2GBCz9ozGGuyYuHGuzLl/ISlhLj1d3vMmIynNhNEWB+WAgDUUi0tscqPg3NkRuNDKgpAP+3eB+ugQKwUJCCv+E= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R751e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033068173054;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W6xkOjU_1716289413; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W6xkOjU_1716289413) by smtp.aliyun-inc.com; Tue, 21 May 2024 19:03:34 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, ioworker0@gmail.com, hrisl@kernel.org, p.raghav@samsung.com, da.gomez@samsung.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 3/8] mm: swap: extend swap_shmem_alloc() to support batch SWAP_MAP_SHMEM flag setting Date: Tue, 21 May 2024 19:03:13 +0800 Message-Id: <3ded3d39f19250ed4f16a42f22121b5f2c2dfb5c.1716285099.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D149E40027 X-Stat-Signature: yhdrtha9dcrsmnjpn5dqycpxeaeodn6i X-HE-Tag: 1716289421-263960 X-HE-Meta: U2FsdGVkX1/5cKff7GC15cPr4lWzNVAev5gqPEtqf6Gjr+PnSHjHtK5kBfubbnHBPgvkRF7aqHJGeyAASVokbRauGN6+9z0CwlgZrxsntZg41TTq8ys9Qmz7ltd+HUHvMApBBxvtNgqWziPscaffYGxyzL4LzTw2AkNMnq21G+ecid1hbWmrtWV0ZNnKN8+pSzmVJEDcE87uJekscc770tWB/epem8GuRXA3ARH+5Bu6PDjmmAnp/wNaSIWa4Hm4tkcK1Tu4NKQx6m7EeNAVwO5C24KQBSKhXJ3RDH1K51rqE8yyx0bacSnDft/xv6V1LmcSCzH47ZcKTvG+xP3BMs3dP1jfKzSfYxzZ35ZW2PR33f0CY1qfTp+a4OZDmJTJzZeBrF5W+mICU/4bNc496G5C4n4+Iq0PmRxNpc4ByLVWjbbfs2EcEtvsjwdpR1DrnU2ydSmn4/D6396j16rbzMa1PqcxMUSUuRfmY9MRjPYF404iCmIriUFVOp7slL6te5N6rJMFM03iVhaQHvpjw7YaJXMKknqTLXXmSKMgGWxYPXvo7Y0mepybuj92xVayYXsZv0f0niNUrenrnrwTaBSt8jutBReDIlEHE/5RL05Wywnb6KFgQeXrA1hGP0qlClLwb6ZDJrODVuwicZEMg1n9K8FCvUsB+LqAoV8N7JtZQq0nAeJ767/lJrbECPJ4znZSca+Xc3x5WRap5DcaojP4bQqsjrpNphM4Sy76roKUGtiE1Jgs3gtUXg4kQIqJIJ94uBi0EYNpkbbd1c/qQbsQbcRO6RYebhu3RmmnHFqJU+dx0CXklhhW6vGZRp0CaTgVYhciP9T47HiMD++fxBcveJjedwx/S1ID59lcyZDjD3nwYecYEVrtYdFRBScyWuVnlQwfP9g9EsHucOir1/VtTCz/IjVH6ayIYG8d3tGOKB7KAsr6PJMi2of7c0K9QiPwLtR8EYc14M87YT1 EaixD6rj eyOZz0akqjd/rnvWO8yfW+mwtR4BrPmYpPPL6khDnR5V/jU/HL6nRU78sb/l6Bzm20DzZ3MXvpg4Qctp1bmsIaq5paQaSX9OC+oZn5W5I2e+LSZY7qVYjU+NYJFKayllsvvv0hjLoOnP8NhqniaP7xm935L+076x1nYCGGBnh9S3nCw1pRrbQ9xYM22JIQv3xlXegcqOQ09d5h0uO4ywRnMzw7aoPx+VSUfC7geQr9CCc0aceS3kdlPN1yFch2mg6mTjwFIovUPIcYS2Jw256vPN4nPt3yp369QMW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support shmem large folio swap operations, add a new parameter to swap_shmem_alloc() that allows batch SWAP_MAP_SHMEM flag setting for shmem swap entries. While we are at it, using folio_nr_pages() to get the number of pages of the folio as a preparation. Signed-off-by: Baolin Wang --- include/linux/swap.h | 4 +- mm/shmem.c | 6 ++- mm/swapfile.c | 98 +++++++++++++++++++++++--------------------- 3 files changed, 57 insertions(+), 51 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 48131b869a4d..78922922abbd 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -479,7 +479,7 @@ void put_swap_folio(struct folio *folio, swp_entry_t entry); extern swp_entry_t get_swap_page_of_type(int); extern int get_swap_pages(int n, swp_entry_t swp_entries[], int order); extern int add_swap_count_continuation(swp_entry_t, gfp_t); -extern void swap_shmem_alloc(swp_entry_t); +extern void swap_shmem_alloc(swp_entry_t, int); extern int swap_duplicate(swp_entry_t); extern int swapcache_prepare(swp_entry_t); extern void swap_free_nr(swp_entry_t entry, int nr_pages); @@ -546,7 +546,7 @@ static inline int add_swap_count_continuation(swp_entry_t swp, gfp_t gfp_mask) return 0; } -static inline void swap_shmem_alloc(swp_entry_t swp) +static inline void swap_shmem_alloc(swp_entry_t swp, int nr) { } diff --git a/mm/shmem.c b/mm/shmem.c index fd2cb2e73a21..daab124c3e61 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1433,6 +1433,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb); swp_entry_t swap; pgoff_t index; + int nr_pages; /* * Our capabilities prevent regular writeback or sync from ever calling @@ -1465,6 +1466,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) } index = folio->index; + nr_pages = folio_nr_pages(folio); /* * This is somewhat ridiculous, but without plumbing a SWAP_MAP_FALLOC @@ -1517,8 +1519,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) if (add_to_swap_cache(folio, swap, __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN, NULL) == 0) { - shmem_recalc_inode(inode, 0, 1); - swap_shmem_alloc(swap); + shmem_recalc_inode(inode, 0, nr_pages); + swap_shmem_alloc(swap, nr_pages); shmem_delete_from_page_cache(folio, swp_to_radix_entry(swap)); mutex_unlock(&shmem_swaplist_mutex); diff --git a/mm/swapfile.c b/mm/swapfile.c index 99e701620562..2f23b87ddcb3 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -3387,62 +3387,58 @@ void si_swapinfo(struct sysinfo *val) * - swap-cache reference is requested but the entry is not used. -> ENOENT * - swap-mapped reference requested but needs continued swap count. -> ENOMEM */ -static int __swap_duplicate(swp_entry_t entry, unsigned char usage) +static int __swap_duplicate(struct swap_info_struct *p, unsigned long offset, + int nr, unsigned char usage) { - struct swap_info_struct *p; struct swap_cluster_info *ci; - unsigned long offset; unsigned char count; unsigned char has_cache; - int err; + int err, i; - p = swp_swap_info(entry); - - offset = swp_offset(entry); ci = lock_cluster_or_swap_info(p, offset); - count = p->swap_map[offset]; - - /* - * swapin_readahead() doesn't check if a swap entry is valid, so the - * swap entry could be SWAP_MAP_BAD. Check here with lock held. - */ - if (unlikely(swap_count(count) == SWAP_MAP_BAD)) { - err = -ENOENT; - goto unlock_out; - } - - has_cache = count & SWAP_HAS_CACHE; - count &= ~SWAP_HAS_CACHE; - err = 0; - - if (usage == SWAP_HAS_CACHE) { + for (i = 0; i < nr; i++) { + count = p->swap_map[offset + i]; - /* set SWAP_HAS_CACHE if there is no cache and entry is used */ - if (!has_cache && count) - has_cache = SWAP_HAS_CACHE; - else if (has_cache) /* someone else added cache */ - err = -EEXIST; - else /* no users remaining */ + /* + * swapin_readahead() doesn't check if a swap entry is valid, so the + * swap entry could be SWAP_MAP_BAD. Check here with lock held. + */ + if (unlikely(swap_count(count) == SWAP_MAP_BAD)) { err = -ENOENT; + break; + } - } else if (count || has_cache) { + has_cache = count & SWAP_HAS_CACHE; + count &= ~SWAP_HAS_CACHE; + err = 0; + + if (usage == SWAP_HAS_CACHE) { + /* set SWAP_HAS_CACHE if there is no cache and entry is used */ + if (!has_cache && count) + has_cache = SWAP_HAS_CACHE; + else if (has_cache) /* someone else added cache */ + err = -EEXIST; + else /* no users remaining */ + err = -ENOENT; + } else if (count || has_cache) { + if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) + count += usage; + else if ((count & ~COUNT_CONTINUED) > SWAP_MAP_MAX) + err = -EINVAL; + else if (swap_count_continued(p, offset + i, count)) + count = COUNT_CONTINUED; + else + err = -ENOMEM; + } else + err = -ENOENT; /* unused swap entry */ - if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) - count += usage; - else if ((count & ~COUNT_CONTINUED) > SWAP_MAP_MAX) - err = -EINVAL; - else if (swap_count_continued(p, offset, count)) - count = COUNT_CONTINUED; - else - err = -ENOMEM; - } else - err = -ENOENT; /* unused swap entry */ + if (err) + break; - if (!err) - WRITE_ONCE(p->swap_map[offset], count | has_cache); + WRITE_ONCE(p->swap_map[offset + i], count | has_cache); + } -unlock_out: unlock_cluster_or_swap_info(p, ci); return err; } @@ -3451,9 +3447,12 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage) * Help swapoff by noting that swap entry belongs to shmem/tmpfs * (in which case its reference count is never incremented). */ -void swap_shmem_alloc(swp_entry_t entry) +void swap_shmem_alloc(swp_entry_t entry, int nr) { - __swap_duplicate(entry, SWAP_MAP_SHMEM); + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); + + __swap_duplicate(p, offset, nr, SWAP_MAP_SHMEM); } /* @@ -3465,9 +3464,11 @@ void swap_shmem_alloc(swp_entry_t entry) */ int swap_duplicate(swp_entry_t entry) { + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); int err = 0; - while (!err && __swap_duplicate(entry, 1) == -ENOMEM) + while (!err && __swap_duplicate(p, offset, 1, 1) == -ENOMEM) err = add_swap_count_continuation(entry, GFP_ATOMIC); return err; } @@ -3482,7 +3483,10 @@ int swap_duplicate(swp_entry_t entry) */ int swapcache_prepare(swp_entry_t entry) { - return __swap_duplicate(entry, SWAP_HAS_CACHE); + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); + + return __swap_duplicate(p, offset, 1, SWAP_HAS_CACHE); } void swapcache_clear(struct swap_info_struct *si, swp_entry_t entry) From patchwork Tue May 21 11:03:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13669304 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1873C25B74 for ; Tue, 21 May 2024 11:03:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CEB416B009D; Tue, 21 May 2024 07:03:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C71886B009E; Tue, 21 May 2024 07:03:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A50AC6B009F; Tue, 21 May 2024 07:03:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 817346B009D for ; Tue, 21 May 2024 07:03:46 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 188ED1C1E66 for ; Tue, 21 May 2024 11:03:46 +0000 (UTC) X-FDA: 82142117652.20.FCDE898 Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com [115.124.30.131]) by imf02.hostedemail.com (Postfix) with ESMTP id 055838000F for ; Tue, 21 May 2024 11:03:43 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=a0abYcVE; spf=pass (imf02.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716289424; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ArTvZKlaOes4XoFtzrKMEwLUH8CHPrOqCRPSPeGMt/g=; b=uDtQ6ZUvhPch8xetSaXjEzHRDa4LWnfP2W0HBM//Rnlt70JbVmOPCzhfQkgubWNpXGuWdi rUjxi/bkpEO0uVDsb12PZnqCpNFZLGrMpE1J/DEheK+Zesrd87xMXQw9PvRNgsQ6jKMkKe RaUeY0u1/4ALMLPr29xo+oO10srSN1Q= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716289424; a=rsa-sha256; cv=none; b=iENqIF4Re27da65yHuR+e/Zw5sq/czDHxgMIjgibCPxuPgDI80kqiUkU4W9qLbvGtj0OvF e5+QuAnp6j32VhGoqSxhy3biEju6Y2HdBeK7d6J+BYZwYfSaPEcAA5ofxjh89xGQxMNzSX ZPurR2rSwyGDnG8oQfzs0BtsSdx4JHw= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=a0abYcVE; spf=pass (imf02.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1716289417; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=ArTvZKlaOes4XoFtzrKMEwLUH8CHPrOqCRPSPeGMt/g=; b=a0abYcVElqGnhHB+0BarctgCnQwfetl3e4cTU2SDMdZg6QXiM3aff4nrNph168S8acADzcyo7xn/8G4ZLvhYctlWrNMdKetxd6kr96VgLnCFf5jjrqcrrspsbk4ewqh9OQuVcYFhvn5Z70qdgDnppgNdXGWqMk4KYkCj/NTA6KY= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R211e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045075189;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W6xeTCm_1716289415; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W6xeTCm_1716289415) by smtp.aliyun-inc.com; Tue, 21 May 2024 19:03:36 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, ioworker0@gmail.com, hrisl@kernel.org, p.raghav@samsung.com, da.gomez@samsung.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 4/8] mm: shmem: support large folio allocation for shmem_replace_folio() Date: Tue, 21 May 2024 19:03:14 +0800 Message-Id: <73d93136d8469658715388878c947e8952164c53.1716285099.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 9jjtpsie3e6x959uk783myb7ngam37o1 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 055838000F X-HE-Tag: 1716289423-590634 X-HE-Meta: U2FsdGVkX1/jgbvdR6dwd3a+ajguohq8X8JJfNVzouzVmua657ArLhaPUc6ZE0/VmWIePp2FYCYyNjUngWdXVvTF39bSghIos+VJGgyIGWU+Q0Hsbvifx0Mip4XidSduldlMfY0gzXwwA4rxOJqdQHXlMzOGnCmI8T/68DJ5NQoRryuj7syIlRC/Ep791JZklPDbARhLS9bwb64DwHRy6BMzERZ1etvgHz3fvzzN0hUF6hGzEdivQ/Lr9JBdPZZ24uFi3kw1AwFz2CGoz3s8WA4ZY0IAI74OEqzbBkGmXKvAmLURbDFl1gx8lC+NffsaLTYGJvb4kbYoCYnXKFvHo3P9tefrCKaDJlwzHmyO9l8UTd89PJi4dUGEXQNHm+zCG2FpPqXT3ufnR8OBp6xHGkC03M9iWiplfbRSRwABrOlYHSs8i7B2gvTR0s3mTjO9GonAFo9yW6UlQKqhXj/bwZx3ciYFbUiwA9uxBqE3JA5BAjqjeEenrqPDrhjvyFTaDVPfihTDgzBgiHwNs0aWmz+Ecdg4II+6KKsRa/Y5E8EAzeDkK+E6AyaVwt0a3vtb7miepkrmc74TAvXgXC9H+vvUYup62oa1YROowdl0ygDaLkhr8k588kdwsJlVaO8BtOOAHz5rSHcXH6Prs17i2er4P6pEJ72Um6sGUu4SjhVJfPD/3OHsG6eo06COyhf+Rof2l1Nd5caFY61f4VSRtVipG/+3HmDXbfshEdJ6+sqVPAYw1RXcweqOH5AfhpaXFHWTVq8ckUL1Hn+p7T0WcbMqnn2Bc2voUpfCK7dvF8EYAnGTkyqRz6nKqLNAqZGs1eG4E8HZwG3Oa3jXIyuJTBtfGrXnf/yupAe0Pxka3LI+7MrIUsS4RrimVkIthnz0V1ylvTW3QOWB8MEQjR/gCsNcgFx49JGNFuLqZqyK4gi0RjWnuHVU7bOcUzFpaJNZU1RQn8l8bWOzgx3gaN+ vlQqtJNm c6IDUABql1tqpL6sODzo2Xtnozu/n5pYX3q2e3j2cuICjP+ExB0yiKLYiH+nD1+rrP5FkFxeTrrk89kuYZPsij0tQtsVAGSlkrdU3EplHfHZq8D77fwCvvBDgroRc/oYvI6KWqi8qld56LET1c789usrvfEbQ1jYpCU7yJIuhKN5151BCa/0UoX4ZWivTJDy0ZrylPbxMg9gXZzT3gGUBOON21QmbjrHkSpOokkeeJ3aGn2V4d1z63fJ100QFuXCm5ZnEcM5ypZjixbmXsYMtZMX0s52bWvGl/x/6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support large folio swapin for shmem in the following patches, add large folio allocation for the new replacement folio in shmem_replace_folio(), as well as updating statistics using the number of pages in the folio. Signed-off-by: Baolin Wang --- mm/shmem.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index daab124c3e61..74821a7031b8 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1901,8 +1901,7 @@ static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, * limit chance of success by further cpuset and node constraints. */ gfp &= ~GFP_CONSTRAINT_MASK; - VM_BUG_ON_FOLIO(folio_test_large(old), old); - new = shmem_alloc_folio(gfp, info, index); + new = shmem_alloc_hugefolio(gfp, info, index, folio_order(old)); if (!new) return -ENOMEM; @@ -1923,11 +1922,13 @@ static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, xa_lock_irq(&swap_mapping->i_pages); error = shmem_replace_entry(swap_mapping, swap_index, old, new); if (!error) { + int nr_pages = folio_nr_pages(old); + mem_cgroup_migrate(old, new); - __lruvec_stat_mod_folio(new, NR_FILE_PAGES, 1); - __lruvec_stat_mod_folio(new, NR_SHMEM, 1); - __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -1); - __lruvec_stat_mod_folio(old, NR_SHMEM, -1); + __lruvec_stat_mod_folio(new, NR_FILE_PAGES, nr_pages); + __lruvec_stat_mod_folio(new, NR_SHMEM, nr_pages); + __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -nr_pages); + __lruvec_stat_mod_folio(old, NR_SHMEM, -nr_pages); } xa_unlock_irq(&swap_mapping->i_pages); From patchwork Tue May 21 11:03:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13669303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63E24C25B74 for ; Tue, 21 May 2024 11:03:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F2476B009C; Tue, 21 May 2024 07:03:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 87B846B009D; Tue, 21 May 2024 07:03:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F61D6B009E; Tue, 21 May 2024 07:03:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4D38A6B009C for ; Tue, 21 May 2024 07:03:45 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 0C50B1A10EC for ; Tue, 21 May 2024 11:03:45 +0000 (UTC) X-FDA: 82142117610.15.31C5B62 Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) by imf30.hostedemail.com (Postfix) with ESMTP id DE8E880015 for ; Tue, 21 May 2024 11:03:41 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=oVY+upUy; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf30.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.118 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716289422; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eLWM5LDYn4qmAcvClqJJz7Ry6ODVE2l8ecGnrwuiQN0=; b=E3j3/ORnW4vr30DqgpTJ6D2/Rz83sAK+f05VsR2TUuQoshMO4fvdMnDcaGqKjMcyP33hR3 VnhxpGhMgGkbaYHD8UPnBVkBDP8HqlMNzTXyHBLT4ivHmvdXiJ7jCzCTpTwOSrjrIkTHqQ qdyb1GKUbTCMXyXmyQkthjdYCZG+uM0= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=oVY+upUy; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf30.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.118 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716289422; a=rsa-sha256; cv=none; b=dcbkNHIBWvdJPvuYYLjtlWsg1jFbUdbrksX2lWgp5JWXA+Ey8X241ahO8MiQLr8WcNt3pv X2IRml+jrD5pcHDjulhsmlEijf3+ci+FDcvfgRfTmOfYjATRcmXtlgOO830uIrOkAIFR8j ivT6EiJFwNNQJFSneVvi2gUd/SI9fHA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1716289419; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=eLWM5LDYn4qmAcvClqJJz7Ry6ODVE2l8ecGnrwuiQN0=; b=oVY+upUyau3AQlXbvwgwJy9gBbrTKIB2NAPxsZlcumizQeO0ndU8eaqP9jB7AlXnTaLq8n1TFWXbA0y4ezsBb0BnZw0xAat0wdSTXX5eHwWeGlQGzdBxehKlA2v0SRDpqZhLOG93CSVGys6ziT/166ghuf8Z6WcD3acwgLqVjFU= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R261e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067111;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W6xeTDd_1716289416; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W6xeTDd_1716289416) by smtp.aliyun-inc.com; Tue, 21 May 2024 19:03:37 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, ioworker0@gmail.com, hrisl@kernel.org, p.raghav@samsung.com, da.gomez@samsung.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 5/8] mm: shmem: extend shmem_partial_swap_usage() to support large folio swap Date: Tue, 21 May 2024 19:03:15 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: DE8E880015 X-Stat-Signature: 51aqkx9n8jsrbntwu9zb1ircaa4usbxo X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1716289421-495668 X-HE-Meta: U2FsdGVkX19V0ZxWakA/CN2+GU7PQC0sl/saGmuw659JiJgCES+5iY+/QBXSAw395fHXij6qM7sUHRsdWXtlEPLVcxsCLDBhw47gVUeRaHq1ueVo14BjzIZBIAbAw8iN9F3l/prTjuvv8KSU3tUFtN01XtOc869So/qmCJ+NcNLtD2nuLDszl0gQyPFJc2qPbv7Qa69wz1L9DIM6/35pMD3nxu6MV+gj4jFG1rMO7s4xiP5oHsWtauPWMxG5AIyvMXxe9ZmoAEvutxc4+1QfCaGziBO4S/pcCkVEseErAe6rE0Fck+y0k/2SSRGu9+h0iDvr+NICanBHkY6ue9LgnWoPcdnSm2YWivVcqeOYJmihFOwcAeDZSOxAIPPysAwYP0r/BgONDLxRDcOTewWr2VGgEOzQaz6pBk8Hs7dAvr9QJTzbI+KCW5mnawLBVSkIi4zy8JQZCDvixoVt6OxGiugqXfdIquLjlOARFGcpx9Z0HIgPLyPOdAgBxZTGuCS1b1nHam9Za/xU4Dt6hVO1vsbBztiFaaPSRvg3dMwI9VyXXDmN27KOuvJFpOcsYwt/KlvK87fxQhynuG/N4GxohqiaEaBsb9ljgTYdPBAJf1hTf5SnzBYszzuXHOJiuRvRqH2Tqe1xs6NorRajArQ+UM9fjUv8HiSXSiOv0m4dXMqfNTYre45ZsXUdGhPC+zUsMur2hmhsTG7CKDfezx71gNEg9RobwwpodTdDp36kEJNsCL20q+5NUj4LKyLLo1lqXpOcgt2l3WJutTKs+dv6wzZ4bqPT0UMxQl0Dt+oX4SbSLYo7fNQwhOc8f3Z+BD4z7P9Vn+bzyVa0VUQujqUeE6pkvF2lfHyMMtEbiYzcQ+4sbcwjXJR/cq107SHx28aPLc//mKdZjIHada5IquESAIjQPlCTLmkk/hxQ/HOQ2hgUSP3c9SUvNaLTDgpmfplW7DOPpJLWU0tRFB9OQ8N o7KxUaCd y48Q1lZwfoS4RGlXPs81L2xFHLE1oxabnKzd1X+YFpQPjIuaVBYrd5Lo/If8R51C2wikldsKcRYB6OdiTIJ5kmr4Rz600WwCBD5zNpVADZCZT/xZJAt3UlBRPlLnktxaP9aqZ04yQbj2omWHD2GzKaD8kpGegqPmhl6hr8Qu4XX+FcoXtYjYqo7ycSZtmB/ZeU9lqyBHIb10HmHk8cRiCSPmsQflUAFDyOdtQobSGI6MsciYKaTUfzPDw4IYBwQ8joFRIrWeC3rPKL1FgQTyPUlBxvk/l6bCcp3Kh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support shmem large folio swapout in the following patches, using xa_get_order() to get the order of the swap entry to calculate the swap usage of shmem. Signed-off-by: Baolin Wang --- mm/shmem.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 74821a7031b8..bc099e8b9952 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -865,13 +865,16 @@ unsigned long shmem_partial_swap_usage(struct address_space *mapping, struct page *page; unsigned long swapped = 0; unsigned long max = end - 1; + int order; rcu_read_lock(); xas_for_each(&xas, page, max) { if (xas_retry(&xas, page)) continue; - if (xa_is_value(page)) - swapped++; + if (xa_is_value(page)) { + order = xa_get_order(xas.xa, xas.xa_index); + swapped += 1 << order; + } if (xas.xa_index == max) break; if (need_resched()) { From patchwork Tue May 21 11:03:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13669306 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04065C25B74 for ; Tue, 21 May 2024 11:03:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A79D26B009F; Tue, 21 May 2024 07:03:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9DAC66B00A0; Tue, 21 May 2024 07:03:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E0126B00A1; Tue, 21 May 2024 07:03:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4F5466B00A0 for ; Tue, 21 May 2024 07:03:47 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0382A1610A0 for ; Tue, 21 May 2024 11:03:46 +0000 (UTC) X-FDA: 82142117694.05.985FC7C Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by imf22.hostedemail.com (Postfix) with ESMTP id BB97BC0023 for ; Tue, 21 May 2024 11:03:44 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=KddY3vsg; spf=pass (imf22.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716289425; a=rsa-sha256; cv=none; b=Si/rZcqfCyV0vQvRf42YfFJZ3HmN8XUWW4Gq6NkUeucloTlkCI3dYvNuMs1E2vx9NMPdOB nlrWWfZ+UJqjeMl1cmt8PkNwIQJnJsOpI3gU3ykc00Piel+FxTqomaFUWi8O5N3oLVO0ae 7nH2i0my/l+lqFdL9s9EkDtvKfKjccE= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=KddY3vsg; spf=pass (imf22.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716289425; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RUK8yowxerwNkroF7zPanFQGgOdBeDeZsuZ9hXwq2Jg=; b=ZVccVOHEFkllwOvhPXiOOmpOPyicILL8T2rtX5W5QOsaJxaCvBCYlCnKN2N6UHFc4N5TSw YLSfKW1KLJsZ5VppW9DwH8OprBQNTowgDJzFGADjCcq0bpm/JOSBpHxzUVwMGHih4FSJAi eEN/29Bw/+LbXAIEMbMtjXu0hVEq4Mw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1716289422; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=RUK8yowxerwNkroF7zPanFQGgOdBeDeZsuZ9hXwq2Jg=; b=KddY3vsgh3ZyQny2pUAsR9BB+dk6jyCRv+BewdrEmAOlNwU8g94yc4AZDoijXwF63L7u9VpCYZBPEsUwECE520qFbwvqAsE9fmL4vfVx0pt9ULm7vNMvFtNf+ygls7ZxsBeBfi1t8EcJiGasKaTrpr0fjmlZ+3NETiWrpwvV8GI= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067112;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W6xkOmJ_1716289418; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W6xkOmJ_1716289418) by smtp.aliyun-inc.com; Tue, 21 May 2024 19:03:39 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, ioworker0@gmail.com, hrisl@kernel.org, p.raghav@samsung.com, da.gomez@samsung.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 6/8] mm: add new 'orders' parameter for find_get_entries() and find_lock_entries() Date: Tue, 21 May 2024 19:03:16 +0800 Message-Id: <88b28bb5929ff8dd23278974138c85c2226e333d.1716285099.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: BB97BC0023 X-Stat-Signature: 7o6xerddpdh3j6wu4i8tecm1nbrp8nzf X-HE-Tag: 1716289424-261696 X-HE-Meta: U2FsdGVkX1/oJVb39r+NDU/TYW07gNOc2V1xyGAQNujhCNYf2cZGwoidFGpx/wLQQLFhD63Eq8YWHlorLaiVZUE+1+DyLtpYHOKEI60jQqbkf4QK2FEJFWg8XdMiM05lIQ3bzE8YReYPCOFqsoPDpA6riVCI40tObbTU+LsrHrKp7uKcze3VRrhueUfz7KBzBzDzjVEGDmWn3r+37S/SBO+U5RjeCWR6ttA69x3WbbT/IrpQOKtP8jE5z9qFhNAaBZoxFsspPMDFL/Rm5Juddlil2VPc7Jpillukdd1+TON7jyFx5e5ZsvXhxG7874Ho6OWML7MiQei4Ab/gjp00B248ZZTSyPVuc6zuZu569BNEN/IcqzRSLE/rx7i8txSie30BPdhNLs8EHeszinsz0ICrUbgoTVYoR2xlyoOdQ0vFgLWIFVbGBdldVThFei4OMtESP/cJU1M1yToBRHnV+4sOykzcsEZzBAbRicKpug91QBtQ6PXs17VrWNo979K4heoQ3vha8ZUVIij4TyF7jUqU2sHemzw9qt0+y41JLBVR8Nk4ZhtJtZEDaa7eAPQupi3b7xr6RXc9oYKnwGp3zHI6dAvv5lAAUkiI+fyjiiEbr85gwM8DrrX6XyA5JBJFsOzKVUV6CFUNrclcNKnBEM1jOwBfLo/Ahf0VPH1I2Vo44HSk7iMgcH608xWt185IRbyz2PhNMfW7abohVmxdfFfNDRWHXxa/LUMxlNpDvY5U8h+r/FLrjz6aMlWRRY21xLys5PrrRpidgp5ZGtxuKYuEZgVHq4Vfu3l8ePUm2PCqjHA5YgGPKQiQBYmTeNqHLZZLVegdV2VHUfCSlZm43KzqiRg0HZtSTd1ne/mM5BIFRCj8eiXrUI9yCyVT+pOMtGJfC1LJJFUVQGGP4xcdcJErV1fcJ/WJW+dTzuJbcovvXaUHA2rw3Gkx3zw4PtEFqow4+P3cPVE8xHk9Drs TiDXQ7/+ z/LMhNrEkQRqp7XTH+r7iYyq9XLwZKANYA6Rg17/+oYmG5HC7W3WniHOdAbk4mK0W8kCG0fk1j8aclROXkQb04vQhYi+R1XzhmGw5yNSaOxQgO5GtfqY7ZwPF2TFGI0L5xdafSIJe+vTjyVubufDZWJZF8Zy1XEKgbLEN3TkDM3NVTz/W+RbambFi8orTcsEQy09d6rz55IX7lxKliYysTinoALaEMMtxOFJ2w6Og2mLzfmg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the following patches, shmem will support the swap out of large folios, which means the shmem mappings may contain large order swap entries, so an 'orders' array is added for find_get_entries() and find_lock_entries() to obtain the order size of shmem swap entries, which will help in the release of shmem large folio swap entries. Signed-off-by: Baolin Wang --- mm/filemap.c | 27 +++++++++++++++++++++++++-- mm/internal.h | 4 ++-- mm/shmem.c | 17 +++++++++-------- mm/truncate.c | 8 ++++---- 4 files changed, 40 insertions(+), 16 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index ec273b00ce5f..9d8544df5e4a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2036,14 +2036,24 @@ static inline struct folio *find_get_entry(struct xa_state *xas, pgoff_t max, * Return: The number of entries which were found. */ unsigned find_get_entries(struct address_space *mapping, pgoff_t *start, - pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices) + pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices, + int *orders) { XA_STATE(xas, &mapping->i_pages, *start); struct folio *folio; + int order; rcu_read_lock(); while ((folio = find_get_entry(&xas, end, XA_PRESENT)) != NULL) { indices[fbatch->nr] = xas.xa_index; + if (orders) { + if (!xa_is_value(folio)) + order = folio_order(folio); + else + order = xa_get_order(xas.xa, xas.xa_index); + + orders[fbatch->nr] = order; + } if (!folio_batch_add(fbatch, folio)) break; } @@ -2056,6 +2066,8 @@ unsigned find_get_entries(struct address_space *mapping, pgoff_t *start, folio = fbatch->folios[idx]; if (!xa_is_value(folio)) nr = folio_nr_pages(folio); + else if (orders) + nr = 1 << orders[idx]; *start = indices[idx] + nr; } return folio_batch_count(fbatch); @@ -2082,10 +2094,12 @@ unsigned find_get_entries(struct address_space *mapping, pgoff_t *start, * Return: The number of entries which were found. */ unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start, - pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices) + pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices, + int *orders) { XA_STATE(xas, &mapping->i_pages, *start); struct folio *folio; + int order; rcu_read_lock(); while ((folio = find_get_entry(&xas, end, XA_PRESENT))) { @@ -2099,9 +2113,16 @@ unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start, if (folio->mapping != mapping || folio_test_writeback(folio)) goto unlock; + if (orders) + order = folio_order(folio); VM_BUG_ON_FOLIO(!folio_contains(folio, xas.xa_index), folio); + } else if (orders) { + order = xa_get_order(xas.xa, xas.xa_index); } + + if (orders) + orders[fbatch->nr] = order; indices[fbatch->nr] = xas.xa_index; if (!folio_batch_add(fbatch, folio)) break; @@ -2120,6 +2141,8 @@ unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start, folio = fbatch->folios[idx]; if (!xa_is_value(folio)) nr = folio_nr_pages(folio); + else if (orders) + nr = 1 << orders[idx]; *start = indices[idx] + nr; } return folio_batch_count(fbatch); diff --git a/mm/internal.h b/mm/internal.h index 17b0a1824948..755df223cd3a 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -339,9 +339,9 @@ static inline void force_page_cache_readahead(struct address_space *mapping, } unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start, - pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices); + pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices, int *orders); unsigned find_get_entries(struct address_space *mapping, pgoff_t *start, - pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices); + pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices, int *orders); void filemap_free_folio(struct address_space *mapping, struct folio *folio); int truncate_inode_folio(struct address_space *mapping, struct folio *folio); bool truncate_inode_partial_folio(struct folio *folio, loff_t start, diff --git a/mm/shmem.c b/mm/shmem.c index bc099e8b9952..b3e39d9cf42c 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -840,14 +840,14 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap) * Remove swap entry from page cache, free the swap and its page cache. */ static int shmem_free_swap(struct address_space *mapping, - pgoff_t index, void *radswap) + pgoff_t index, void *radswap, int order) { void *old; old = xa_cmpxchg_irq(&mapping->i_pages, index, radswap, NULL, 0); if (old != radswap) return -ENOENT; - free_swap_and_cache(radix_to_swp_entry(radswap)); + free_swap_and_cache_nr(radix_to_swp_entry(radswap), 1 << order); return 0; } @@ -981,6 +981,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, pgoff_t end = (lend + 1) >> PAGE_SHIFT; struct folio_batch fbatch; pgoff_t indices[PAGEVEC_SIZE]; + int orders[PAGEVEC_SIZE]; struct folio *folio; bool same_folio; long nr_swaps_freed = 0; @@ -996,15 +997,15 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, folio_batch_init(&fbatch); index = start; while (index < end && find_lock_entries(mapping, &index, end - 1, - &fbatch, indices)) { + &fbatch, indices, orders)) { for (i = 0; i < folio_batch_count(&fbatch); i++) { folio = fbatch.folios[i]; if (xa_is_value(folio)) { if (unfalloc) continue; - nr_swaps_freed += !shmem_free_swap(mapping, - indices[i], folio); + if (!shmem_free_swap(mapping, indices[i], folio, orders[i])) + nr_swaps_freed += 1 << orders[i]; continue; } @@ -1058,7 +1059,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, cond_resched(); if (!find_get_entries(mapping, &index, end - 1, &fbatch, - indices)) { + indices, orders)) { /* If all gone or hole-punch or unfalloc, we're done */ if (index == start || end != -1) break; @@ -1072,12 +1073,12 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, if (xa_is_value(folio)) { if (unfalloc) continue; - if (shmem_free_swap(mapping, indices[i], folio)) { + if (shmem_free_swap(mapping, indices[i], folio, orders[i])) { /* Swap was replaced by page: retry */ index = indices[i]; break; } - nr_swaps_freed++; + nr_swaps_freed += 1 << orders[i]; continue; } diff --git a/mm/truncate.c b/mm/truncate.c index e99085bf3d34..514834045bc8 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -352,7 +352,7 @@ void truncate_inode_pages_range(struct address_space *mapping, folio_batch_init(&fbatch); index = start; while (index < end && find_lock_entries(mapping, &index, end - 1, - &fbatch, indices)) { + &fbatch, indices, NULL)) { truncate_folio_batch_exceptionals(mapping, &fbatch, indices); for (i = 0; i < folio_batch_count(&fbatch); i++) truncate_cleanup_folio(fbatch.folios[i]); @@ -392,7 +392,7 @@ void truncate_inode_pages_range(struct address_space *mapping, while (index < end) { cond_resched(); if (!find_get_entries(mapping, &index, end - 1, &fbatch, - indices)) { + indices, NULL)) { /* If all gone from start onwards, we're done */ if (index == start) break; @@ -496,7 +496,7 @@ unsigned long mapping_try_invalidate(struct address_space *mapping, int i; folio_batch_init(&fbatch); - while (find_lock_entries(mapping, &index, end, &fbatch, indices)) { + while (find_lock_entries(mapping, &index, end, &fbatch, indices, NULL)) { for (i = 0; i < folio_batch_count(&fbatch); i++) { struct folio *folio = fbatch.folios[i]; @@ -622,7 +622,7 @@ int invalidate_inode_pages2_range(struct address_space *mapping, folio_batch_init(&fbatch); index = start; - while (find_get_entries(mapping, &index, end, &fbatch, indices)) { + while (find_get_entries(mapping, &index, end, &fbatch, indices, NULL)) { for (i = 0; i < folio_batch_count(&fbatch); i++) { struct folio *folio = fbatch.folios[i]; From patchwork Tue May 21 11:03:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13669307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 578E2C25B7A for ; Tue, 21 May 2024 11:03:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D60E86B00A1; Tue, 21 May 2024 07:03:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE9C26B00A2; Tue, 21 May 2024 07:03:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B3C2D6B00A3; Tue, 21 May 2024 07:03:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8E6046B00A1 for ; Tue, 21 May 2024 07:03:48 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 45A1B1A0341 for ; Tue, 21 May 2024 11:03:48 +0000 (UTC) X-FDA: 82142117736.10.F8A3C25 Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com [115.124.30.131]) by imf03.hostedemail.com (Postfix) with ESMTP id 345B420013 for ; Tue, 21 May 2024 11:03:45 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=OTpGakiC; spf=pass (imf03.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716289426; a=rsa-sha256; cv=none; b=g/MbnOBeQbjTQNoi3gFKSo2l1Rn6n4SmGt6TZX6xWIe+MN6OTxlZ5wlWNgy/9CV60DZcMj uBfzgimILy4M757t+nZ9DigcLRcsxotIU6neyDWXOQs+v8Qe96Ko3ChOKDXeEZCyO1UHtz zdU9ZWKj2rZMBFVguqaHZ6cJ/vRYb7o= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=OTpGakiC; spf=pass (imf03.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716289426; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/V3DnBye2LjO7u5ouBgBU8nPrUqi/2Egdsz3zKACdwM=; b=hY+zVneU0yds3NZjYl3c6AP5BUvhxmlO+LAjk5nbedMFQg/nDkBLY/F94OLA44ivSKG04J 6i5BMEWamw52n2erRvSPqfaTJOieG/cXXYkLqGMNh4NHjitNKMn/2Rm/6cB+2wxZGGre9o 4IED5otOnAapMJdutB2U/oESOZOiVvw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1716289422; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=/V3DnBye2LjO7u5ouBgBU8nPrUqi/2Egdsz3zKACdwM=; b=OTpGakiC/UZJlMSyaFPQo432K8mpDsxrkIxOwJ8KJl6PCC9YC+3JbgAzLxJ+9sgngjaeWGiexkJRw3XTAxbMy4mawf/SKY4oW+MOxf5K3HAI5DIczcK73cuo7rVvh4zE825OhgSvsQQ7APsv4RzeElpcFn0MplmhGe/vX+IXW2k= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R931e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067113;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W6xl985_1716289419; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W6xl985_1716289419) by smtp.aliyun-inc.com; Tue, 21 May 2024 19:03:41 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, ioworker0@gmail.com, hrisl@kernel.org, p.raghav@samsung.com, da.gomez@samsung.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 7/8] mm: shmem: use swap_free_nr() to free shmem swap entries Date: Tue, 21 May 2024 19:03:17 +0800 Message-Id: <90fda314a9e966374fe837ae3ebd74708380479d.1716285099.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 345B420013 X-Stat-Signature: ukgc8p6xq35g6wbazj9gghnjo3szgirc X-HE-Tag: 1716289425-363609 X-HE-Meta: U2FsdGVkX18cP6/ZZzXEM6gcgyKlsAH9JPRnuuKxyn1tx9YLlzRBaEFNteCSRYtsS1yoiChAbKVBhP2ybo/Wnedptx6MeLPtLurEp3Cw9xasa7m0vACDb1Z3b7d3z6RkTpot6Kaji2+gD36BU+MUr1yyFuN6MBa00TIWNavsYplixfY/RYg3tRQrHkmcBQ3/OF6SIsctKb7+s+YR9za5YSE0yP8FhvRj647uGUj5NI03Cdpb9rBe2XQKPi09Nbuh2gJH2eWXvDx+zCW8OWES72FUVjED8VrkxTHkK+GWnqAiBqwTByfkIMMBrE6KxhEeAjIRWK9WUnGhuCWU8UoxIbeJJPJHJBZ6eu+eP+ONwH5ndJoCoVlUpdxbQlVIT4xFHUm7AYMTeeUJnszbAZdQu1OjMwSw3TmNVLjkzoGWKmhTglbRSSTq3gfE/0DBPPGmtgpcAw8hg1IrBcC5TtoETak+ieccMfF3K63Xh5CxtW8Gw+NN/IWsX8oONT3eJ/Di+RhXJnrBKqPZAvqpc3HKgu8x4rbZafFq242VvrtTbR2isgHSXfWEzM7+OnXd56QRo2qDSS1inVExQP3xNaEg84EyYUplBGsuSvcuYI3pwqIjvLZa2yrge5ibuAerik/opunwgaVrwIQHjWerd1LRgUVO9F8i2mwvb9+3LTjhk8x+I1YZHOTLKpAEmwbUqWbW3Kuhshw5N0kS9pR7ZMWYVIT3psLfd0jr0PKhIKdNzSm8fK/YUL5Gu49RoHe46ar0+JUa3Wkq2XS2pdBiphOAxiu4B+VSPVIHfm53oWZ4YJ/e8dV50GZDBxqiihedBjk/OdEk5IlkewMtJFxOPGsC4//e0hSukCthOBgmDKRyccgGrzr81hKKauMIOCjzRa7HCzVLBBztj1uaPKEYOL1Ec5V3BsWgC4gMFKGF1I+mtjvZ5jkEkvrF2crDkdHyXlgM8Y1/TOvYyJ7kPLOmDiE MBw5Wbgt BYtbB8txCa7anEv7zZ985A34QYxKSPwR6zP8qLYmNIl/LqbMsyLFoXtvzk3vt9ZT0WhW7opjFGSDna4cna2AfbpQalECv+tLjTNBq4v+DPweKF1edXLJDV2NOKgodLQm/cJP0EG5G73sFwThILIS/BsS5Zf4ZvDt5TKXSLwC6oJgVR2jYVRQPG2xAZ8ZH6mEItrGAZbYErh16B/dfbGZSszTOatMV3XC1ZsqOcx2k0kwjjJ+aVsbxkXTMCYiE8d/F1qtFVTNTYUl2lKi0Jm3gfd+gCk62JkbYQRrM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As a preparation for supporting shmem large folio swapout, use swap_free_nr() to free some continuous swap entries of the shmem large folio when the large folio was swapped in from the swap cache. Signed-off-by: Baolin Wang --- mm/shmem.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index b3e39d9cf42c..fdc71e14916c 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1962,6 +1962,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, struct address_space *mapping = inode->i_mapping; swp_entry_t swapin_error; void *old; + int nr_pages; swapin_error = make_poisoned_swp_entry(); old = xa_cmpxchg_irq(&mapping->i_pages, index, @@ -1970,6 +1971,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, if (old != swp_to_radix_entry(swap)) return; + nr_pages = folio_nr_pages(folio); folio_wait_writeback(folio); delete_from_swap_cache(folio); /* @@ -1977,8 +1979,8 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, * won't be 0 when inode is released and thus trigger WARN_ON(i_blocks) * in shmem_evict_inode(). */ - shmem_recalc_inode(inode, -1, -1); - swap_free(swap); + shmem_recalc_inode(inode, -nr_pages, -nr_pages); + swap_free_nr(swap, nr_pages); } /* @@ -1997,7 +1999,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, struct swap_info_struct *si; struct folio *folio = NULL; swp_entry_t swap; - int error; + int error, nr_pages; VM_BUG_ON(!*foliop || !xa_is_value(*foliop)); swap = radix_to_swp_entry(*foliop); @@ -2044,6 +2046,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, goto failed; } folio_wait_writeback(folio); + nr_pages = folio_nr_pages(folio); /* * Some architectures may have to restore extra metadata to the @@ -2062,14 +2065,14 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, if (error) goto failed; - shmem_recalc_inode(inode, 0, -1); + shmem_recalc_inode(inode, 0, -nr_pages); if (sgp == SGP_WRITE) folio_mark_accessed(folio); delete_from_swap_cache(folio); folio_mark_dirty(folio); - swap_free(swap); + swap_free_nr(swap, nr_pages); put_swap_device(si); *foliop = folio; From patchwork Tue May 21 11:03:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13669308 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4FB1C25B74 for ; Tue, 21 May 2024 11:04:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BDA666B00A2; Tue, 21 May 2024 07:03:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B3CC26B00A3; Tue, 21 May 2024 07:03:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98E976B00A4; Tue, 21 May 2024 07:03:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 733C66B00A2 for ; Tue, 21 May 2024 07:03:49 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 34CDA1411F1 for ; Tue, 21 May 2024 11:03:49 +0000 (UTC) X-FDA: 82142117778.22.85633F9 Received: from out30-119.freemail.mail.aliyun.com (out30-119.freemail.mail.aliyun.com [115.124.30.119]) by imf23.hostedemail.com (Postfix) with ESMTP id EDB59140009 for ; Tue, 21 May 2024 11:03:46 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=JSbbCpxS; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf23.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.119 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716289427; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3ZBjHevxDMMp9OTvEWxh5sKqkBiotmw21gIhPQdkVy8=; b=yf8WrbH2PES4X/L81R+S9lTYjUvimCgyZlHSqcgGGmtPVzyHKBewHXotKYvoCau4tfuYgK D+2eZ5thmYIqwgEeLUxKXHN4w/PovECPR07+LJFcmYy+NWA22GEe3RhZFz0j3VpvlYXDTw QRF5kRdrcXsUEgjkJoxoDRH2L607p/M= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=JSbbCpxS; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf23.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.119 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716289427; a=rsa-sha256; cv=none; b=hzabbSLMmHlf+CQKEQPg46dWTagPR/LI4k1GtKwluMQTFKNzFF0+ZxMLBcvBXAL0y1W+S8 D8zYOZPWXTbyHbdbVJ+4G+YKOgoMsFlclD4sksfPik6qFfhaS0e4BLy/0EqV/jsDqvnPuc JCBw9Y4H2uAJOI+l6PFIpbPWTKywgmg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1716289424; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=3ZBjHevxDMMp9OTvEWxh5sKqkBiotmw21gIhPQdkVy8=; b=JSbbCpxSuHCmM8c8wQVf7vJ92cG/QxZ9CgqZn8gfNjkl1dkF+01Sx0t1M9gEimdaz/cssXeJpAmw+ZMx9yede5bRvO9tEThPVavcjSG6um/Kt7J7zxhOoFk/YhXXH2wXEuFfm0wf/lMDZgu6zO65tXQohpf0OVhk/2reNUSXYHo= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R551e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067110;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W6xl98k_1716289421; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W6xl98k_1716289421) by smtp.aliyun-inc.com; Tue, 21 May 2024 19:03:42 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, ioworker0@gmail.com, hrisl@kernel.org, p.raghav@samsung.com, da.gomez@samsung.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 8/8] mm: shmem: support large folio swap out Date: Tue, 21 May 2024 19:03:18 +0800 Message-Id: <1f50ac5f9dfa69c3c7cc57440eae2b1728178cca.1716285099.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: EDB59140009 X-Stat-Signature: rjt6h8zw8aegmghbqmowpx7ytc9oc5z9 X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1716289426-320394 X-HE-Meta: U2FsdGVkX19pCiUhUd+oWzBYNvpAMqRZl30ESmdYktZ0xfkjOQJF3q7LvE68bXml+FANuINRDkzTMDN941bhMEOrOj5Cr8pYllJ/mfxVpLYuYyeR4PuQ4rtjtuoz6Rr+sd+aoL+TzJYYuTYS2C+qxXD/gXtI4ohyNBUtxqqMdZDNSXFHgy9WPzodd+mADrhct2IwmVbgRCWZGJzQqJyQlTjQfGjZXRlQIkD51kJ+ZOkP93MtZydFniP3RoR8qUxNS9AhsOT/LZ0kXEcPoRQsFadMOHLM0XZIyTMXGeyv0iaJFMOdgzHyWLhb3UdmrRCT0FJuozyJLlN8X86ksIzLAfYTLdmMZNlfjou9bqwVr6uH/PQiZzU+3KcQVAApxp0go/wp+TDonMLKyp/BXTYZK6ZajI/v9ULIncv5iSenZD5PbGPdU4O7TIju59hUhFN1tR2+o2HcOYG8gZ5uKt2/LmiVTnynd+Mt921Mv9+dcZVE65sNRisOMk3aW0Yj3UM1AUKEUia6uPrHVI/zkZR1TkCqL8rMHfkF4pyqXQ1f3mU7tFu5RAWkGbYvxIqQXUovL58/GTTVHZ8IjXz3DGbTzj/mSZi/CJuH7ZQq/hcw+xl/kPxskOU44RF0EyB6RvsK70Xjpam5G9JYOZ3Npeqry35wix9D3ze65EPJSzB0bx5ei1y8R2zSE0XWKuEdU78dEl1RHCrzWh0cycH9i29yxucU8RXvTb54u8+l18YBnoh4hZY00eD4DTov4cNoX+56B7UZts50wKqEcylQABIlyGWGQ/WN8IR1KN1Yw5SoHa2IwIOcZM8Vid003VXl6II+Z5UFO30SCJBDLFiYA1P1KafoNIdbbBdirZ+fiU84Demfm6KCrugMtGcM5DD5zjeOnZ4VTOZn/nFype2yBmG3LBPJ4AKDmF4md0DVjgM4VtVbFy9XNBqDLk+7OAHifKK2MmXxgP2yc2IxHJC/OuT 207r/QSa ecyUMSoIk36xGtgDiYrqTge4WsD/IqDWYLzaHdcc9+RH+7KzJSkjGm0Br7SS9Nu6a6OFlQAERZpQ1BhjHvzO1fIF1DOEIFrPZRcl3vj0CXveY5paKsTIGItDI240IVmswqyqEHj7MTS2yU4z0KqoIzhSiXmPyjwoy/1Pwjh5yLRO2QeyQwtpJqBvvoNaR9ziIvGhflC+QCdr1lFKiiGdqzIVZ572GXN7QPmriVHOFfm1EKEkfHH8e4lHt+euyqkZqA4CcUkTQ9w22+h5GyyhCPOs50EO/iBsAAm+lg/tf02u+rOsXMpNFVL9zrcBHs92pVz9q X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Shmem will support large folio allocation [1] [2] to get a better performance, however, the memory reclaim still splits the precious large folios when trying to swap out shmem, which may lead to the memory fragmentation issue and can not take advantage of the large folio for shmeme. Moreover, the swap code already supports for swapping out large folio without split, hence this patch set supports the large folio swap out for shmem. Note the i915_gem_shmem driver still need to be split when swapping, thus add a new flag 'split_large_folio' for writeback_control to indicate spliting the large folio. [1] https://lore.kernel.org/all/cover.1715571279.git.baolin.wang@linux.alibaba.com/ [2] https://lore.kernel.org/all/20240515055719.32577-1-da.gomez@samsung.com/ Signed-off-by: Baolin Wang --- drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 1 + include/linux/writeback.h | 1 + mm/shmem.c | 3 +-- mm/vmscan.c | 14 ++++++++++++-- 4 files changed, 15 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c index 38b72d86560f..968274be14ef 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c @@ -308,6 +308,7 @@ void __shmem_writeback(size_t size, struct address_space *mapping) .range_start = 0, .range_end = LLONG_MAX, .for_reclaim = 1, + .split_large_folio = 1, }; unsigned long i; diff --git a/include/linux/writeback.h b/include/linux/writeback.h index 112d806ddbe4..6f2599244ae0 100644 --- a/include/linux/writeback.h +++ b/include/linux/writeback.h @@ -63,6 +63,7 @@ struct writeback_control { unsigned range_cyclic:1; /* range_start is cyclic */ unsigned for_sync:1; /* sync(2) WB_SYNC_ALL writeback */ unsigned unpinned_netfs_wb:1; /* Cleared I_PINNING_NETFS_WB */ + unsigned split_large_folio:1; /* Split large folio for shmem writeback */ /* * When writeback IOs are bounced through async layers, only the diff --git a/mm/shmem.c b/mm/shmem.c index fdc71e14916c..6645169aa913 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -776,7 +776,6 @@ static int shmem_add_to_page_cache(struct folio *folio, VM_BUG_ON_FOLIO(index != round_down(index, nr), folio); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(!folio_test_swapbacked(folio), folio); - VM_BUG_ON(expected && folio_test_large(folio)); folio_ref_add(folio, nr); folio->mapping = mapping; @@ -1460,7 +1459,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) * "force", drivers/gpu/drm/i915/gem/i915_gem_shmem.c gets huge pages, * and its shmem_writeback() needs them to be split when swapping. */ - if (folio_test_large(folio)) { + if (wbc->split_large_folio && folio_test_large(folio)) { /* Ensure the subpages are still dirty */ folio_test_set_dirty(folio); if (split_huge_page(page) < 0) diff --git a/mm/vmscan.c b/mm/vmscan.c index bf11c0cbf12e..856286e84d62 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1260,8 +1260,12 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, if (!total_swap_pages) goto activate_locked; - /* Split shmem folio */ - if (split_folio_to_list(folio, folio_list)) + /* + * Only split shmem folio when CONFIG_THP_SWAP + * is not enabled. + */ + if (!IS_ENABLED(CONFIG_THP_SWAP) && + split_folio_to_list(folio, folio_list)) goto keep_locked; } @@ -1363,10 +1367,16 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, * starts and then write it out here. */ try_to_unmap_flush_dirty(); +try_pageout: switch (pageout(folio, mapping, &plug)) { case PAGE_KEEP: goto keep_locked; case PAGE_ACTIVATE: + if (shmem_mapping(mapping) && folio_test_large(folio) && + !split_folio_to_list(folio, folio_list)) { + nr_pages = 1; + goto try_pageout; + } goto activate_locked; case PAGE_SUCCESS: stat->nr_pageout += nr_pages;