From patchwork Tue Aug 20 09:49:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13769793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22243C5320E for ; Tue, 20 Aug 2024 09:49:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 116026B0092; Tue, 20 Aug 2024 05:49:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 09CE16B0095; Tue, 20 Aug 2024 05:49:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E57D66B0096; Tue, 20 Aug 2024 05:49:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id BA5AE6B0092 for ; Tue, 20 Aug 2024 05:49:48 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 66549A199C for ; Tue, 20 Aug 2024 09:49:48 +0000 (UTC) X-FDA: 82472152056.14.69BC8F7 Received: from out30-98.freemail.mail.aliyun.com (out30-98.freemail.mail.aliyun.com [115.124.30.98]) by imf12.hostedemail.com (Postfix) with ESMTP id 7983540008 for ; Tue, 20 Aug 2024 09:49:45 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=wJmhRB14; spf=pass (imf12.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.98 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724147346; a=rsa-sha256; cv=none; b=Z0lUaiuhx6knR4ooD+NihiOKQnWr85oCJBXTXpAbnb8nzv0uySbnZg6QfFqCbC4edrDkc6 i4svUyCdhkH8Etx3OOjx2OR8m+wsJuPe396nU3rLFmwyCTmDZVLh+LhmDuU4raHoluRqu7 li1C0wy4WcXeJE87nyYiwFWRidZ2b/k= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=wJmhRB14; spf=pass (imf12.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.98 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724147346; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NJsMChbPBN7mMZhuJpQhE2p+c0ZuMVKGW4qyVg+x7+o=; b=YtX4VQAxVtpc+xngIvR2SSo2XDzVHqDGDd+XjJTnSAE5H1mUqV/yfbSjIoi09rsrHEYXGE uvFrrIlo7FBV1s/v04bJfC1HBC7vQmu5pk6+Y59CWrN+T9uXHh0d++7qfyOC3gWd6zCYIb VAubWS+dzGpiNqCqu2ZyuYOplKYOrrA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724147382; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=NJsMChbPBN7mMZhuJpQhE2p+c0ZuMVKGW4qyVg+x7+o=; b=wJmhRB14xcARWJwD7SHr9erwUF2rLJNN5/0AWSJAmWPX/vqIw2T38d64wa/ia/F8RslRZXd3UGBdBDF5qY72nUvMQ/+xFNxJgvFNSzFoZm+xvsfQWYOed1uO2S+qRpjluXz+Y4G+YzCs6jmTYedX8BltDhLES7dpI3H1676/Lo4= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WDILeC9_1724147380) by smtp.aliyun-inc.com; Tue, 20 Aug 2024 17:49:41 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: hughd@google.com, willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/5] mm: khugepaged: expand the is_refcount_suitable() to support file folios Date: Tue, 20 Aug 2024 17:49:13 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: 9m9wzr7stz1fiww1xun5zz9c8b6jtr5b X-Rspamd-Queue-Id: 7983540008 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1724147385-257243 X-HE-Meta: U2FsdGVkX197WIlIeLsDrGjMebNEwLImM3XPrbp2LoVIiSzHXnTJY7icm+QG0otFWSoryQYH+YGRft/i7xx7Ovoe2E75U4Iup5sZplusVAd7k7CGCzcUrXYAHXgLBb4FpJOKxE+Rnky2WjHFC+v7k3/uufvra99KAVIpw5aU4H6hjqNVkM4bhm3uq8kPuHOxzC73edjSGIEAP8kDUQM4+ShOGKUzH8n1BMAojXPJG72dvR2iVnQddmBY94AaqmNsOGH4gaFGgwnrkoJdCtJdQPnQ+Dn8dirHvEv6grKVL2S3jo4ReClCBHWsfMgxXRTCdO6o9WVSEuohKuJrLC0Z/dpvXkfHLJQ5E4+j6cYhGE82zrF1LryqxCCo63ekhMJpHF+IU79dGkkwuDITV9noK3iOySFw1HVTUtU8gEadcuZkP4+hEbK8NiziGBymy26LPArevyxV8yE83QCcdN1p0On+mwaiBKdotl+UIJjKGpxS4eS1SPT99JTk/ROr23YKWNkrD9fQLQYnI8KUwzol8o3wcBp0XZ2bVIZqIJYsrT8OfwMKRULrPZorhKmPeuso91erVOlCAep6qwUsB5LrQ88aKJ79vr81Jqsqt6d9iLceF3uM3tUzHx1ilKznvYLkCnqtrFD1HKxnN9zIpxVyu7UOGjUFa3q/bViYT4tcWwa11XyfTw7VfqQ85+59++UwoMHlDET/AYR7Evfxq1LsiJ68EOADWB9aVhIgOprrTzD5pYsTV+uxZ8g2yu9rsEnVyL8DuRb4QOmggUkwoaeRzIrllYT8QP7HLm1HWRirVSOoF+Yu7GpX7j7AE0K0j6hO0wtW7pKbK93g9onQnYClmObOgatsMrsBxQpUt69AyEoz0lOsy73a0CegdpBxtJ7e5vG22qxCkbyf906ONpSb+CYL7omjAGC9IrCaAfcTcSMy+9Nbz7822svefEoY8UT/fQNr88eA/8KmMwvYX8i GZajWPgx AYK5GRmT2+IkE6jOMYDuFzx3BbTzlu4hFcDRfZNCJ/oa9fj6xru7rB24EzeYy9h7vZJePdG2JvyDTHP24rdmzczinK9DzpnxfIedh2uOHBzWcRbyaSovmSGQPI6sZb9TWU4WtTEfIO9YpH0bCFExg8X4cwL1msleGUVwPquMuuFWwbz8RT2x8m3iAwJ06ACu3EImz99NHhJsu6KBdMZBQ9fWMzo4zU4J1RIsIVLo4VLV5E4NHI6ENRsZcyQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Expand the is_refcount_suitable() to support reference checks for file folios, as preparation for supporting shmem mTHP collapse. Signed-off-by: Baolin Wang Acked-by: David Hildenbrand --- mm/khugepaged.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index cdd1d8655a76..76f05215ee87 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -546,12 +546,14 @@ static void release_pte_pages(pte_t *pte, pte_t *_pte, static bool is_refcount_suitable(struct folio *folio) { - int expected_refcount; + int expected_refcount = folio_mapcount(folio); - expected_refcount = folio_mapcount(folio); - if (folio_test_swapcache(folio)) + if (!folio_test_anon(folio) || folio_test_swapcache(folio)) expected_refcount += folio_nr_pages(folio); + if (folio_test_private(folio)) + expected_refcount++; + return folio_ref_count(folio) == expected_refcount; } @@ -2285,8 +2287,7 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr, break; } - if (folio_ref_count(folio) != - 1 + folio_mapcount(folio) + folio_test_private(folio)) { + if (!is_refcount_suitable(folio)) { result = SCAN_PAGE_COUNT; break; } From patchwork Tue Aug 20 09:49:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13769792 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB5D9C531DF for ; Tue, 20 Aug 2024 09:49:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B0026B008C; Tue, 20 Aug 2024 05:49:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 160DF6B0092; Tue, 20 Aug 2024 05:49:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1AAC6B0095; Tue, 20 Aug 2024 05:49:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D5E146B008C for ; Tue, 20 Aug 2024 05:49:47 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8E0AC81914 for ; Tue, 20 Aug 2024 09:49:47 +0000 (UTC) X-FDA: 82472152014.05.D7FE1F6 Received: from out30-119.freemail.mail.aliyun.com (out30-119.freemail.mail.aliyun.com [115.124.30.119]) by imf25.hostedemail.com (Postfix) with ESMTP id 7744FA0016 for ; Tue, 20 Aug 2024 09:49:45 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=sPsGyMCq; spf=pass (imf25.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.119 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724147308; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BdoRq8oN4jMPChy+X0I/pW/sjGhNsrwJxvZFVW1ygPQ=; b=cQxtRq+44EV1kscQ/D1ZRVUToeYFzAHfNZDnQc1/4FGycO0oG07kkb7/vXOfQtCuXdE1JC C8AVT6RCBbmWZQvUwqOJvu+grVPdO77XS6YqRG6qAMSDYSKpMO5ATQdL5GLd6DOZjuzzPT OPAU7gL7Bj3uGIdJ4w17vT0kLdG+aIQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724147308; a=rsa-sha256; cv=none; b=Ho4Sda6ZHpa+sQ/6zgLNvvMgS15qRpwjPeC5OXxsplURZcqCBzMXE4NV9CyOQv1+AOJF5R MbkP+9FJwpPo678VJYri7pcp5MZgpoJxriw+K4IMl/YEK5lp9QPRx/RM2ZFTSR7319/k0Y J5Qj30AQ4GMQlZIGud9YrqoiBmhwm9o= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=sPsGyMCq; spf=pass (imf25.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.119 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724147382; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=BdoRq8oN4jMPChy+X0I/pW/sjGhNsrwJxvZFVW1ygPQ=; b=sPsGyMCqgR0Ot6D70+V+k9rXsJ3UFsAhX3+FRJr7Pebn8hCfJ3KF+FRGJ2IF6ZPvI8CpzHQYfPEqB+qYLQe8kf6X0+Q0jDlYIlrW9RWWqAGD/bMO+/NKVK8iYTasnZNEmJUKvoZqgqpnlX07wsqPd6MT5wx0z0vWEX5tW20ME20= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WDIChL0_1724147381) by smtp.aliyun-inc.com; Tue, 20 Aug 2024 17:49:42 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: hughd@google.com, willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/5] mm: khugepaged: use the number of pages in the folio to check the reference count Date: Tue, 20 Aug 2024 17:49:14 +0800 Message-Id: <9ea49262308de28957596cc6e8edc2d3a4f54659.1724140601.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 7744FA0016 X-Stat-Signature: ripqothdxyaa1nb57t7rc9szmzmu4war X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1724147385-603532 X-HE-Meta: U2FsdGVkX1/Tu9+mCKGYvVYTiIyDtf10W3UhvwZKs9WDv0RumB5XAn0lqOOwvSsDlVtKZ6aP1RvoKPnbrLSrG86L5kmk1xHppR96JQB0lzPBr1L58oUgYWMQ1I1B1ncoRFcOpCKIMfrr3fWqIujL4VqqB7dissQ3PfiSrRfGB5w0d31JrQDfDTvFXOkUOMddPNAIWLY3El8lYzkTEWGVmXlg4/YLq+g6UH8gHRoAb5NPNWwhqIU0eR5eWGTY6K7oSyfR5wdevC6tBLIT7kq8qsypDg/EToSrC2l6XAgPkNvDtGHlifzF+PeeyjsbWmXUjY9sE8BgLDDgzmvR6disluDFPQ7rj0Y4X0mn3N6uZDukuVLkgJK+a4bOKvBXcupTLn2vl8f2S8Cw7DqGBAP8fQeoy4SuHN4dRgQVqwh5Ws37MJmu+O/9UX4TscdnIjmrcTHz9QFQMa0exLQ6Wr7kgzG7BdQEkzDoj0eXRP6vRMChjKUCnIrKawTDBSLB5CJSHTAENFyUC84a2bCr6vNwnnwvLd8RET5p6fG1CpLbN9m2j105KHHGdeWksGi5HimsntLtTnsUtONEHHRDBIOUQG+xBO3gbsMN+QiU0YdYd7M8u1ZDF/dHLfiZQkS/vKCOG5+iOSaMJBSt5JB1AaJMycHagATnQ//n5lIlBhGgw20MoV2C13Ce0G4zmikvYK/Z4H+OOPEOIlHfi+iQ1JeXAn0CEIqWUDPYlVFyzS8A92eYx/q+dNykTUpn0CEdqhupxRLJ2gvhP9GQpSVUycmDniX4uQnYrOeHvgKXlp6qyis1jMgKYCQB8KiaPZCSyGqc/xdKQyxDK0S5sXWcE0KVdQnIR6sk2iQvyAmHs3kR4Uu8Mlfa9eJKjt0AiSU3aKpR6IDo0lACpz1sKTRpGmqGwe8kDr+o6RuLWratqM5MenMo3q1fVBDC4I0Dewx93RJBxv9RdTGNfcrFKcomnRs CPCFBUNG UXcP/68obcko/ZvAdBUTknqy1yRZD3q/Nv6z/0o6uJRXtH4YTXYOthvzh4wztvSU8bAXXwRfZ6QFe4NAwDhFnof/yYd8imxSf4nyTa14oEYZFQLR3Ezpb9bqGtEf/ytmzN+8hcW2CCskmBuLGvjNTIxJjCGjkuu5ple/yKlB8QB7MOV3v3U/F3uAeG84yLIHMBsXwJGN/mV0GMo3lnQspP7iDFBsNBoVxKOx83SLZ38zOkCAKNqGO6TH8+oESMHCbDqGxjeYRoTzCgWl3UYXqgNBM/g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use the number of pages in the folio to check the reference count as preparation for supporting shmem mTHP collapse. Signed-off-by: Baolin Wang Acked-by: David Hildenbrand --- mm/khugepaged.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 76f05215ee87..e015c94eba09 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1988,9 +1988,9 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, VM_BUG_ON_FOLIO(folio != xa_load(xas.xa, index), folio); /* - * We control three references to the folio: + * We control 2 + nr_pages references to the folio: * - we hold a pin on it; - * - one reference from page cache; + * - nr_pages reference from page cache; * - one from lru_isolate_folio; * If those are the only references, then any new usage * of the folio will have to fetch it from the page @@ -1998,7 +1998,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, * truncate, so any new usage will be blocked until we * unlock folio after collapse/during rollback. */ - if (folio_ref_count(folio) != 3) { + if (folio_ref_count(folio) != 2 + folio_nr_pages(folio)) { result = SCAN_PAGE_COUNT; xas_unlock_irq(&xas); folio_putback_lru(folio); @@ -2181,7 +2181,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, folio_clear_active(folio); folio_clear_unevictable(folio); folio_unlock(folio); - folio_put_refs(folio, 3); + folio_put_refs(folio, 2 + folio_nr_pages(folio)); } goto out; From patchwork Tue Aug 20 09:49:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13769794 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AA80C3DA4A for ; Tue, 20 Aug 2024 09:49:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 66D366B0096; Tue, 20 Aug 2024 05:49:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F9456B0099; Tue, 20 Aug 2024 05:49:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 421126B0096; Tue, 20 Aug 2024 05:49:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0A3886B0098 for ; Tue, 20 Aug 2024 05:49:49 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A2140A1A2D for ; Tue, 20 Aug 2024 09:49:48 +0000 (UTC) X-FDA: 82472152056.09.AF8F152 Received: from out30-100.freemail.mail.aliyun.com (out30-100.freemail.mail.aliyun.com [115.124.30.100]) by imf19.hostedemail.com (Postfix) with ESMTP id AB9161A0005 for ; Tue, 20 Aug 2024 09:49:46 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="RWmL/4ZY"; spf=pass (imf19.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.100 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724147308; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0UzIUq6FZARxgmMF/NPpanXZh/m/Lu7W5+1J/nDWXrI=; b=Bws/JsDaVg1qsWc0nt9LxlUo/UPRRiraNgu2wKjippRaSGUk31tA9w45EtNW70boP6/C5h RHoeLsg5FBckPwQBqZ0Q3aTCTBywetQWK+FbAmGtMoCYWM8zASRpM7aDNocvPD4KZhq/nE gpeSrpb3weQg6yt739jdFxpqufaLQ4A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724147308; a=rsa-sha256; cv=none; b=vzxsHqqX1eFZH22+DAbf8iddUk3QUxHrVeW/04MzBpTI1ZFd5UboAHz4nqcxlRcIKq7gcQ QbgDej5H1QPz4vmI+7hyHhvZ3u7slF/T+c0YGykNf/yXhsDmgoqeuxKUPI1dTeTM7J6j/X 4OibsdRS8gxSWLOCFfUNABFaXIbQloY= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="RWmL/4ZY"; spf=pass (imf19.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.100 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724147384; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=0UzIUq6FZARxgmMF/NPpanXZh/m/Lu7W5+1J/nDWXrI=; b=RWmL/4ZYqLu4isjVJqF065APyI1B4CaYn8dLMNrtRGAgX3aPdxj/78ecjeJxkd7hU8f9vAJq8oqDoMJ2AE7P5+uPFg6xqT0BTPaocAw3k+VK/1zfJPkHqKgAmUYPtfq8rSNO3EK0NZmwhHY6i1pehEI5xkVqoW62W4E4h7/lmoU= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WDIEwjB_1724147382) by smtp.aliyun-inc.com; Tue, 20 Aug 2024 17:49:43 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: hughd@google.com, willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/5] mm: khugepaged: support shmem mTHP copy Date: Tue, 20 Aug 2024 17:49:15 +0800 Message-Id: <222d615b7c837eabb47a238126c5fdeff8aa5283.1724140601.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: AB9161A0005 X-Stat-Signature: yo7jpkw5h315go3jg34yz1i8m6swj4ae X-HE-Tag: 1724147386-806538 X-HE-Meta: U2FsdGVkX19T5AUzU5tcWvkn0RyrOKw4xF1UiUBrZFe+HLUAUKFzUlS0GdIqIb5dSYrg1wgOzek4b1q4urKVSKgM40ef7Ys7IWC1aYRtcQ+JCVtRohoKgqcVT+wFWaVkRCR0K9+ODUN209I9UrmX6csKKV0TtcgOzTguskIZtlGCwBxlYmwzN5louCEfuFuh4KfbDO+cAwHXdEmDQ7DmO/XkB9wWviONWqqmJteKdZ9EFS1QaFtONWOWjvMc8u+EufsefqCSr0S1VOVKwXBQqo3rUVqKTzOBooqGGQZIeiwE8nljbW1a0jLGsIfGIIiuJtb4ZfetITHE99S2PHG13oYklT1MKtQjeMOGCAW8yCab2WD3/sqPhf0+6MJD+tQ9GG38eDN2LpXYevlRTcuRQno8361tDFBeV80SIoPBUaTY46zmabRfvNVUxII6pn7HqTJectWM6gUCg3PQb4I3q6NWdJvicfhxJp5k2SiemHw3ESI/edh0Fk0MuV6IO5eRGiJjBh/FbCdl7fDskzfcqgWOVXeF3v1BZUtIUkyt9wRlby3RUyttqbF+ZaSJAUvCnbq+wRTg1cdrD5ldBob1jekqTr2T4kEHwPS14mE1pwLNBpu9xmUQa/TGIa9XZxqiCYS1mq09XNafy5yIPStc0vjZ+FKMFFLIe25PYxMuei8feb7RSUoYrzhR45HqCqgwoc0qOUykmREw/9IXLeUyf2pDr3oI577qWrDwn+qojlHvv/bnWiRNznJ3/6nhZ7uKfRiRPZqZe9gh83mIzIgzZdvW2M4yvZhj+2+rzTh7n3ZOU2/UcBNDAwiYb+K1f8fhKoqnHV1NAJ4KBPgQsGnbKUpz/ra6xZILwsdJBqpRv7rJhT28zree3TDFjDgGzrR2peFL3zlX4Xd0AHqyv2ifqqIR6a88QsLKj7OkNItdqyOrvGSJUehrTGLsvSCtytHorlKF1GLGNSJcgENhUUX a4cQuksZ sNfmMqUTOHlMn0jsu/f5GBi4KxrtbkGWKnXHiCY86xh0U+u8LgVMl6YoV1j7XWnl1qE2jJXjRh7EGLr2h7rdxE8b3kupkzD5ikFMbd6MUoOb9v0qvaP4GdIoPNNJLL3GZPKWwv1497z8x183OLSNbUIagyrWV275uAZazHxnAo9HSFTqTA8IV0RjOfgDCJu8yXLu0RoxQm+CdmXD0hZmYChr/kGSDAANCxfdu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Iterate each subpage in the large folio to copy, as preparation for supporting shmem mTHP collapse. Signed-off-by: Baolin Wang --- mm/khugepaged.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index e015c94eba09..4996f7487c13 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2056,17 +2056,22 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, index = start; dst = folio_page(new_folio, 0); list_for_each_entry(folio, &pagelist, lru) { + int i, nr_pages = folio_nr_pages(folio); + while (index < folio->index) { clear_highpage(dst); index++; dst++; } - if (copy_mc_highpage(dst, folio_page(folio, 0)) > 0) { - result = SCAN_COPY_MC; - goto rollback; + + for (i = 0; i < nr_pages; i++) { + if (copy_mc_highpage(dst, folio_page(folio, i)) > 0) { + result = SCAN_COPY_MC; + goto rollback; + } + index++; + dst++; } - index++; - dst++; } while (index < end) { clear_highpage(dst); From patchwork Tue Aug 20 09:49:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13769795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28DAAC3DA4A for ; Tue, 20 Aug 2024 09:49:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A7AF6B0089; Tue, 20 Aug 2024 05:49:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 55A096B0098; Tue, 20 Aug 2024 05:49:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 335C76B0099; Tue, 20 Aug 2024 05:49:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0C9B96B0089 for ; Tue, 20 Aug 2024 05:49:50 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B2ED3A1A2F for ; Tue, 20 Aug 2024 09:49:49 +0000 (UTC) X-FDA: 82472152098.01.6D798AB Received: from out30-98.freemail.mail.aliyun.com (out30-98.freemail.mail.aliyun.com [115.124.30.98]) by imf23.hostedemail.com (Postfix) with ESMTP id B6C63140010 for ; Tue, 20 Aug 2024 09:49:47 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=SsH6Xp0A; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf23.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.98 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724147300; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bSL4r/U095xkuorzmW1ZSPtc/nQFSKSzXorZ/e7zvxM=; b=2kZWMSmO7OY3c1WLy0zhpGlJbG6KGv8CVM66h9WAeGVgskBTc4cj4BDFEJJhPjXEPx9Hei gFJnuLDWxcbc5nvz4w7X2AWR/zPBRIQ7sSzj3EQAt65XrSvyYqrTGKNkzfB6XxjEAEUHNY GOkLVrFUkbAUZzzsapnjE7iXi1a8o7k= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724147300; a=rsa-sha256; cv=none; b=zeEMBXVkkjFo4VgHv3uMtkDxBU20WHi9BAzLk92qjj88XMt+A6i29qUmAaF+ZX0DxmqotM aknUSdXbIGmFoTkZTBSnnqYInaf2FlJVWjAzCOMyNdRHhyDPitjNfm+L5QV/nivDBvt3CN xtpJ9aVbXb/6ojUEp9h9TSwcDnJ97f4= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=SsH6Xp0A; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf23.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.98 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724147385; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=bSL4r/U095xkuorzmW1ZSPtc/nQFSKSzXorZ/e7zvxM=; b=SsH6Xp0ANjTMUbvjUlCGHD9737tCl3RrODOXO0jUNvMS89yDsDYZPRDZmsr/okem1m3HuxwFFo1J8C+RW5QF+aBOfq9jX+q7ouu40GEeM7vOv61N/O1zRpSl/4GNUD/ajERzTOOgkUosju1iuambrYjNsUqvbcrkxfxf/XO1Jtc= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WDIChLj_1724147383) by smtp.aliyun-inc.com; Tue, 20 Aug 2024 17:49:44 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: hughd@google.com, willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 4/5] mm: khugepaged: support shmem mTHP collapse Date: Tue, 20 Aug 2024 17:49:16 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B6C63140010 X-Stat-Signature: ek9k1p1fc85g48tgq1kaheghdq65chui X-Rspam-User: X-HE-Tag: 1724147387-47424 X-HE-Meta: U2FsdGVkX19e83ZU97v30sNwA2KADO9smdnuJHl61k5y+9Ad0nBgNFJL+1cnVgHKlGDYt3BZIiQ+DHBAchTZb78xrxQ66dEOYPQ1MXOXrFLq/ohoZBbaPzEOHgsmDCqIBVYTOE1ZU1dGIdeiNr3aEb8zGvGUOoGqN/Q5r600mL6ycxqqgIdhfejgWcAJn6XfE5Qtk2xyyoEGo9KwdjF8VIsWTJsZ3K7gVgsZvNpGGVhbxkAWujACZIJeyuvDjB2XvkpCIr4+FsvehkmyCOo//mEz7fLN3NnkyPB283cARbGVA/wWpQHIco5hzhVmZYnyMxhBUHnk4Dn5cTivwoWnjkFpkpqU2hbBpdIMP1Q9LsLn3WZVxFW0gkriRlelrKxFiNVgALmm9Ccnp40d9sCmBuBojsSA2GF0a7XnAvviDjWmuK4NvQs4KqJ9hWVl19iQh7yBjZL5LwZugcuCsgJVXU44JgXe8NHHMd3JaAuClQNThkFyi9NFqTrRNLSyUMQzoI+2HAgWieEckZ+yiTbVDbVXIeSQCveNHz2s+yECvxchMxYcXLoeU3ttBZsLpmxRXYROyyPf1hFA9J/+9QzKUnKO968IQUcdRRxYMRf4S+LpjyTU/gOY7FiUzJWuoewvuRNffMzFKjAwTin0A5P979Fk3WRdwB8DjdYVbX1EvrSEv6ukKkMv0uS5S4nUEN1QEgQr0ZAJaVDlER+rWe+7OcW/vg6/3F5rZdD5ONQbA0jccZeeErvFq0EyUYF5mllDBD05l08bE5wI62dGDV3R57ytN/qJu5lu/lsmM2CLS5+gQW3icCQI8FA4NYHEH0GFgpA94oiXHQuacKaMXZDCPsilUqphG3S/HqsW+mayTvnUOJnLZBNlBdQsvSsXgROnUAiPB54C1CGJsd7KgX1R0dfdqJJKEcLtaJLPIoFBYvCYmMlJzD0hqGDlXpvnZeIdHANcx67fxbSPwIiXghC FL0Tcvx/ LQubACJt5bqMgC14JUsOPYmvKvO9oa1GeaGSrBJqLejOf9WbJ+RrxF1avaYjL5VPDLnRDVQCpDh1fnXb21SDgkCLdlGPVa0CyWbdv9vwx9GiXkN/bSuj496bBunrFbW242Q3NOZMcHpNY4adJp3F0etshaS2rjx8KaJnmg3xvkRP6oBzfizHGZNuq3IO+o5AfoGDv14DTRh8H171+sc6p19Dz18Yirc+S65aZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Shmem already supports the allocation of mTHP, but khugepaged does not yet support collapsing mTHP folios. Now khugepaged is ready to support mTHP, and this patch enables the collapse of shmem mTHP. Signed-off-by: Baolin Wang --- mm/khugepaged.c | 28 +++++++++++----------------- 1 file changed, 11 insertions(+), 17 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 4996f7487c13..4a83c40d9053 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1843,7 +1843,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, } } while (1); - for (index = start; index < end; index++) { + for (index = start; index < end;) { xas_set(&xas, index); folio = xas_load(&xas); @@ -1862,6 +1862,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, } } nr_none++; + index++; continue; } @@ -1943,12 +1944,10 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, * we locked the first folio, then a THP might be there already. * This will be discovered on the first iteration. */ - if (folio_test_large(folio)) { - result = folio_order(folio) == HPAGE_PMD_ORDER && - folio->index == start - /* Maybe PMD-mapped */ - ? SCAN_PTE_MAPPED_HUGEPAGE - : SCAN_PAGE_COMPOUND; + if (folio_order(folio) == HPAGE_PMD_ORDER && + folio->index == start) { + /* Maybe PMD-mapped */ + result = SCAN_PTE_MAPPED_HUGEPAGE; goto out_unlock; } @@ -2009,6 +2008,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, * Accumulate the folios that are being collapsed. */ list_add_tail(&folio->lru, &pagelist); + index += folio_nr_pages(folio); continue; out_unlock: folio_unlock(folio); @@ -2261,16 +2261,10 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr, continue; } - /* - * TODO: khugepaged should compact smaller compound pages - * into a PMD sized page - */ - if (folio_test_large(folio)) { - result = folio_order(folio) == HPAGE_PMD_ORDER && - folio->index == start - /* Maybe PMD-mapped */ - ? SCAN_PTE_MAPPED_HUGEPAGE - : SCAN_PAGE_COMPOUND; + if (folio_order(folio) == HPAGE_PMD_ORDER && + folio->index == start) { + /* Maybe PMD-mapped */ + result = SCAN_PTE_MAPPED_HUGEPAGE; /* * For SCAN_PTE_MAPPED_HUGEPAGE, further processing * by the caller won't touch the page cache, and so From patchwork Tue Aug 20 09:49:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13769796 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95474C3DA4A for ; Tue, 20 Aug 2024 09:50:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 114B66B009D; Tue, 20 Aug 2024 05:50:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C4626B009E; Tue, 20 Aug 2024 05:50:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECD886B009F; Tue, 20 Aug 2024 05:50:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CB7AD6B009D for ; Tue, 20 Aug 2024 05:50:02 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 530AFC1903 for ; Tue, 20 Aug 2024 09:50:02 +0000 (UTC) X-FDA: 82472152644.01.BB8A2B1 Received: from out199-4.us.a.mail.aliyun.com (out199-4.us.a.mail.aliyun.com [47.90.199.4]) by imf11.hostedemail.com (Postfix) with ESMTP id 0C63B40014 for ; Tue, 20 Aug 2024 09:49:59 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=kbGM7uKb; spf=pass (imf11.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 47.90.199.4 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724147361; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NETcifOYIb+Yf9TxOqDeY39PaKApxrCiukqpjtRmh74=; b=a6tZyhNIy/K8diAP2GeSAoegpxVJYPNNK53NdHmp/Ux1F4R79Qgda1x2BWdRggLGGwf8EO Rc9g9jhG/dXPF+xJbtCEbR8UBDbrMzfOqjbwsWRXklXBgwAsFIdyEYj2ac5U1QxyigdID+ WU7IN5KG2kzVymV2JORIUsMMqnKHGS0= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=kbGM7uKb; spf=pass (imf11.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 47.90.199.4 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724147361; a=rsa-sha256; cv=none; b=sFBUBQc2A2wlT9rIsg7Ikiy3C49SJP/aynpg1RToK+nKGAZjn9EKvw0ACfuEsCarLLmF4g yehKoYZaXeFRXtQP8Ar/oXlKZlpZaavQlJFzZ8oEpaUwgNBqpuj0Pe23BINy6bsgEfhIIc tR0+kcd2nJ1DM8LZ8lJTCu9BG7lmzPQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724147386; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=NETcifOYIb+Yf9TxOqDeY39PaKApxrCiukqpjtRmh74=; b=kbGM7uKbxeQJwLDFnmM7YJxxLdAEaMfd4WAuliMyaYQcTe81bEVZJxQUKgAl0LcAX7On5qhUuqMDcQ1T/eRcxkbxJY5AgOF4QLqOBoeYI5l5AyiFCO+nbwUAcwTPfxyFpMfs/fJ5XDyU7FaTssjFE6txPv4FSB7aFTIWMsB+hl8= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WDIEwkS_1724147384) by smtp.aliyun-inc.com; Tue, 20 Aug 2024 17:49:45 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: hughd@google.com, willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 5/5] selftests: mm: support shmem mTHP collapse testing Date: Tue, 20 Aug 2024 17:49:17 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 0C63B40014 X-Stat-Signature: ohb9qfe67q4j1tc9hu93nc1r7set1j4w X-HE-Tag: 1724147399-164004 X-HE-Meta: U2FsdGVkX1+K9C+G+03KZqG3A1xDABloUHMyosg3XPNBemepHZBqeljq05i/AQDqdBtQjsp6envBnAqKkLSoOXsHAUxsnItqts8hmB7QpPSpdRD5ui1Uus/Gl+Butcc9sf7tfvGUtZddFncbZFlHp76+1UEoioj+8oIFrQ6kALhzv4uFAOVyJeROPAi+uVd+4MOUvtMk01vKvIZ7gd+5C3IlNJPaib/8iSGPS7VwuXFiCqrUhdm5DwKAsZEpJbFFb7Xe0lA6mKSy7VtMA0wfeZlei8eRgtZCTbJ3CfZuQp55QSNtu9m5R6Vp1+sVimXp2wfA1ilHbP1MrY2YXk2GYyj+flYGeIMCPkrDlMcYwsm/S4NfkRDHCPqSqNmwvSsUUGgdyB75HwCPSZQYU93jHXshAWVMRKVDldAWkEqoHOToP7kA+jnGZGmw6PA54Syua2//KkN/7nNlu2FVjV4CLl6Y2JcEpBJWLFs7bwIjz7fsxj/Nz9ErlWmzB+DeoscQ+9wyFGlv0qaALP846I4OnbYvbAOACpptC/sChz9afl5hHUDgdIYI1H4sZH8/AVVe/SGXXb90fwpaAW/WAkWwL8uTnCsaYNvhooTqg21F+ei524pdxyACOyHSCJRKdj9rPL0xdLJdJmNih0JpkC5ge/UqGuITbKo2FU5RzZDilfLSl7dIEEz87CFGKIzpfYJL0k8H/NkLDizd7EcPfqebAC9pSJi6bwzGeJetyAg36zoLGHuUTQKWKWw84C+641RRtPKDDMyf0WyulABF7upP4ZMWljBbKgKzcAWD4MviHXSYWbd3LydGzJ69xFkhNjvUmLCHWw9HuyIsHnrqFHd5WRxfZlkolozUfgWJ5KgPsjQWI2IlZBzw1ynukmKYmjt1xTvKoyYRKijQAQwCI9Edj5odzzTc1Rc3BetGdZk/VG0ctajdjG8TgWGvawJf/968e97X5zaHDBXm2R4kdCb sqp8pbSQ 71uREJRsL8/tD1imbxWwVYhTld67TE8zokZOB1zxU1iKajlJtAt1alVJBNfwAPMF8lJV8hxRJK+q3WgwnvFE1pT6eQ4Lrn1g0VhCs1Jxa8dFi6f8BBOVadPeB+3Li4/OiDWiHekAsJgJpyVaAowFwqcJIdv8Oj7JXAfsHBj8HWfsaQKAGvY7e2zMrNOE8GCa3OwvE7+28V8brsUJIArKL8D6RXBT5QhSqlalWxvY3GgdBAo2sPu0fs8Nm51YElkRyDIFAT2n0qPcnYcQVA1wKL85Ivw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add shmem mTHP collpase testing. Similar to the anonymous page, users can use the '-s' parameter to specify the shmem mTHP size for testing. Signed-off-by: Baolin Wang --- tools/testing/selftests/mm/khugepaged.c | 4 +- tools/testing/selftests/mm/thp_settings.c | 46 ++++++++++++++++++++--- tools/testing/selftests/mm/thp_settings.h | 9 ++++- 3 files changed, 51 insertions(+), 8 deletions(-) diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 829320a519e7..56d4480e8d3c 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -1095,7 +1095,7 @@ static void usage(void) fprintf(stderr, "\n\tSupported Options:\n"); fprintf(stderr, "\t\t-h: This help message.\n"); fprintf(stderr, "\t\t-s: mTHP size, expressed as page order.\n"); - fprintf(stderr, "\t\t Defaults to 0. Use this size for anon allocations.\n"); + fprintf(stderr, "\t\t Defaults to 0. Use this size for anon or shmem allocations.\n"); exit(1); } @@ -1209,6 +1209,8 @@ int main(int argc, char **argv) default_settings.khugepaged.pages_to_scan = hpage_pmd_nr * 8; default_settings.hugepages[hpage_pmd_order].enabled = THP_INHERIT; default_settings.hugepages[anon_order].enabled = THP_ALWAYS; + default_settings.shmem_hugepages[hpage_pmd_order].enabled = SHMEM_INHERIT; + default_settings.shmem_hugepages[anon_order].enabled = SHMEM_ALWAYS; save_settings(); thp_push_settings(&default_settings); diff --git a/tools/testing/selftests/mm/thp_settings.c b/tools/testing/selftests/mm/thp_settings.c index a4163438108e..577eaab6266f 100644 --- a/tools/testing/selftests/mm/thp_settings.c +++ b/tools/testing/selftests/mm/thp_settings.c @@ -33,10 +33,11 @@ static const char * const thp_defrag_strings[] = { }; static const char * const shmem_enabled_strings[] = { + "never", "always", "within_size", "advise", - "never", + "inherit", "deny", "force", NULL @@ -200,6 +201,7 @@ void thp_write_num(const char *name, unsigned long num) void thp_read_settings(struct thp_settings *settings) { unsigned long orders = thp_supported_orders(); + unsigned long shmem_orders = thp_shmem_supported_orders(); char path[PATH_MAX]; int i; @@ -234,12 +236,24 @@ void thp_read_settings(struct thp_settings *settings) settings->hugepages[i].enabled = thp_read_string(path, thp_enabled_strings); } + + for (i = 0; i < NR_ORDERS; i++) { + if (!((1 << i) & shmem_orders)) { + settings->shmem_hugepages[i].enabled = SHMEM_NEVER; + continue; + } + snprintf(path, PATH_MAX, "hugepages-%ukB/shmem_enabled", + (getpagesize() >> 10) << i); + settings->shmem_hugepages[i].enabled = + thp_read_string(path, shmem_enabled_strings); + } } void thp_write_settings(struct thp_settings *settings) { struct khugepaged_settings *khugepaged = &settings->khugepaged; unsigned long orders = thp_supported_orders(); + unsigned long shmem_orders = thp_shmem_supported_orders(); char path[PATH_MAX]; int enabled; int i; @@ -271,6 +285,15 @@ void thp_write_settings(struct thp_settings *settings) enabled = settings->hugepages[i].enabled; thp_write_string(path, thp_enabled_strings[enabled]); } + + for (i = 0; i < NR_ORDERS; i++) { + if (!((1 << i) & shmem_orders)) + continue; + snprintf(path, PATH_MAX, "hugepages-%ukB/shmem_enabled", + (getpagesize() >> 10) << i); + enabled = settings->shmem_hugepages[i].enabled; + thp_write_string(path, shmem_enabled_strings[enabled]); + } } struct thp_settings *thp_current_settings(void) @@ -324,17 +347,18 @@ void thp_set_read_ahead_path(char *path) dev_queue_read_ahead_path[sizeof(dev_queue_read_ahead_path) - 1] = '\0'; } -unsigned long thp_supported_orders(void) +static unsigned long __thp_supported_orders(bool is_shmem) { unsigned long orders = 0; char path[PATH_MAX]; char buf[256]; - int ret; - int i; + int ret, i; + char anon_dir[] = "enabled"; + char shmem_dir[] = "shmem_enabled"; for (i = 0; i < NR_ORDERS; i++) { - ret = snprintf(path, PATH_MAX, THP_SYSFS "hugepages-%ukB/enabled", - (getpagesize() >> 10) << i); + ret = snprintf(path, PATH_MAX, THP_SYSFS "hugepages-%ukB/%s", + (getpagesize() >> 10) << i, is_shmem ? shmem_dir : anon_dir); if (ret >= PATH_MAX) { printf("%s: Pathname is too long\n", __func__); exit(EXIT_FAILURE); @@ -347,3 +371,13 @@ unsigned long thp_supported_orders(void) return orders; } + +unsigned long thp_supported_orders(void) +{ + return __thp_supported_orders(false); +} + +unsigned long thp_shmem_supported_orders(void) +{ + return __thp_supported_orders(true); +} diff --git a/tools/testing/selftests/mm/thp_settings.h b/tools/testing/selftests/mm/thp_settings.h index 71cbff05f4c7..876235a23460 100644 --- a/tools/testing/selftests/mm/thp_settings.h +++ b/tools/testing/selftests/mm/thp_settings.h @@ -22,10 +22,11 @@ enum thp_defrag { }; enum shmem_enabled { + SHMEM_NEVER, SHMEM_ALWAYS, SHMEM_WITHIN_SIZE, SHMEM_ADVISE, - SHMEM_NEVER, + SHMEM_INHERIT, SHMEM_DENY, SHMEM_FORCE, }; @@ -46,6 +47,10 @@ struct khugepaged_settings { unsigned long pages_to_scan; }; +struct shmem_hugepages_settings { + enum shmem_enabled enabled; +}; + struct thp_settings { enum thp_enabled thp_enabled; enum thp_defrag thp_defrag; @@ -54,6 +59,7 @@ struct thp_settings { struct khugepaged_settings khugepaged; unsigned long read_ahead_kb; struct hugepages_settings hugepages[NR_ORDERS]; + struct shmem_hugepages_settings shmem_hugepages[NR_ORDERS]; }; int read_file(const char *path, char *buf, size_t buflen); @@ -76,5 +82,6 @@ void thp_save_settings(void); void thp_set_read_ahead_path(char *path); unsigned long thp_supported_orders(void); +unsigned long thp_shmem_supported_orders(void); #endif /* __THP_SETTINGS_H__ */