From patchwork Mon Aug 19 08:14:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13768056 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB2B6C5320E for ; Mon, 19 Aug 2024 08:14:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B6B56B0093; Mon, 19 Aug 2024 04:14:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 93E3A6B008C; Mon, 19 Aug 2024 04:14:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DE1F6B0092; Mon, 19 Aug 2024 04:14:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5AD176B008A for ; Mon, 19 Aug 2024 04:14:53 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 05B6441060 for ; Mon, 19 Aug 2024 08:14:53 +0000 (UTC) X-FDA: 82468284066.10.F3EE162 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) by imf29.hostedemail.com (Postfix) with ESMTP id 9708C120018 for ; Mon, 19 Aug 2024 08:14:50 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="Acp9cb/D"; spf=pass (imf29.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.99 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724055276; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wXFLqHbVSHxFKCZqqifJmzb9vdz6whYXmBhNU0imZD0=; b=CrzWnvSTFma4JFQit/rCYzoMYXi3M6ur4kQ/vGGi5pGdfu1WnyouTXI7FeEEvxfNTRarxv 9YI1YbKWqA7jMBpmPZ3u4194v2tSI+0CAXa794v8h1E9kHy2LCSho3LHnHLzJkNiJdOH44 RykR5OULj+EuV847HtAz8yCA/0PtkUU= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="Acp9cb/D"; spf=pass (imf29.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.99 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724055276; a=rsa-sha256; cv=none; b=Ev9PFwK4oDghpuB+KHVeJegPMx6JHKcqzaDblNAAMxQGvZSn+RFKSDVTQjk3bA2eTSpEqA gS4Czagdbt9Wuq0HRNUKHe7N+CE8yjC5oncQEhZgWoEYHxIiQQ6Axw6/igdUuSHqvHnXj+ uKwoElAJTFCEwnEEwWW8XIZbGBH6pZM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724055286; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=wXFLqHbVSHxFKCZqqifJmzb9vdz6whYXmBhNU0imZD0=; b=Acp9cb/DupVdSrhmtniMMNOLRLSPf+tSK5diilQexTp3OG0WHtJP4plCBcbvWXNNrM3oZCgEsPEzHUvZJU5Q309IXP02+C7IO0sKc/Vd70KZtfipB86kabQGLmtesDMvceYXA8tjZKoJp9VXFu/H5+f+C8S7/+7qlvGUJttNaxc= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WD8W7Ls_1724055285) by smtp.aliyun-inc.com; Mon, 19 Aug 2024 16:14:46 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: hughd@google.com, willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/5] mm: khugepaged: expand the is_refcount_suitable() to support file folios Date: Mon, 19 Aug 2024 16:14:02 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: qhenr1hwusszau5mhw7sgm9fjraddwjp X-Rspamd-Queue-Id: 9708C120018 X-Rspamd-Server: rspam11 X-HE-Tag: 1724055290-894937 X-HE-Meta: U2FsdGVkX19bczBeOEUR90xeXWUzah4mMjIbelywzVx4HiLenxoGgngaETlc9qhXkmmPsZdBFX6Q9EpbZuPCmmn9llWHkLrpM7j71rW2wQ269vrE3p/zlYMsHYLzZyCbl6s4RgxF5aq7m1nSmVkzME5Mf2oIz0L6uNoRsAvGRCDel3wms0r5nct87Ipzqwj3+D534ANUegpG2j9yXjtjEwNWQpJ5+cQLPNv1Jg/iyS4nXeaMBXca8ZXeMatKKKNVNqyHYKrxARU5I1TQAf6dXvGytwCibZ5D+TRRAL3a6gh5I5x5OGbRbFZ7GLKDPOyGxINOxX8npiQGMdwi77Ccq103NH/3iCYXcxzpUP/d9qOR+27K+CFNq0HGLRIjqNko63pkW6XYDA7dK3wsvaerA5B80DZURqzhiwPieOhzua5y93GVm85nLE6RbYi6feLowXh297tVYU5gpFFsbg7gLh9HkgGEXDeKcRkEzUwLajMdDLPnfN9SR9M35AsWW8MYtnfWsl9ziSqnrwio5ZlgOVC9t7EhT59A/mt1A02zjtYV8671uXhIUkFahYkBgiH5uKHVe2WbK+Zo+kFdI4QjVPGg5xDAi8NSqEbJOm7Jn11fmYyDIyx0RXgGtkOoQAuRSX7EObgsCPIs9H5pdHZ88o5WsR1+U1qFgj4ctYzwFZenoPpPC2C6LNQxcfdCvoct1ODFpfV1WJFz+Y6VodIqfSqdkVfDKS72qvZ7LJANau9toI0SeORCG/mc9jJd7FtgjCLBSjhQINikFlPgDkrOpBbh1U27HBrAIku2DllPsvlTzd9SCZ8m7njVPlUGcZlaglZhix8Sa1kuRR95s6VMicGGvSjxl2whMGJmGLUiNSqsldSF6PMUFcc4iWgsx9D///j6GGkvncgTk8NmbLd7MKVaq6C74q/+AJ+6i9MLUD8QepPAu9YM2kn0k6sYDWVSGP6gK/aPgJb3dai3AvO Lo5XgCc4 0ZtS7DiunVm0PJyI5kTuv09/0fqDKq5P2HgHgV2qQ7H2fwbkqGjj2NhO/sxinaA1RK02BGF/DmW8tRlZp1azLtWXG3e9MZ7MBfaWqy13UzcPKQ+dtIKjVttNQcWV/cANxh3F6R9SB41GSwQ31WBa4bCI5XWHw5jqD8HdTJxsZdwB1NBJag8D8QcW8NMstSG8cDuFn+APNOpzf5mqhk8pTge8BdJ/SwYmxGkDCQpj3v26jzLpiqQ7C4V/YREIitzu8YuYoBHghgE8REjfRtgF2wA+eyqUwUtTv58/dPeKPECDFVDQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Expand the is_refcount_suitable() to support reference checks for file folios, as preparation for supporting shmem mTHP collapse. Signed-off-by: Baolin Wang --- mm/khugepaged.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index cdd1d8655a76..f11b4f172e61 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -549,8 +549,14 @@ static bool is_refcount_suitable(struct folio *folio) int expected_refcount; expected_refcount = folio_mapcount(folio); - if (folio_test_swapcache(folio)) + if (folio_test_anon(folio)) { + expected_refcount += folio_test_swapcache(folio) ? + folio_nr_pages(folio) : 0; + } else { expected_refcount += folio_nr_pages(folio); + if (folio_test_private(folio)) + expected_refcount++; + } return folio_ref_count(folio) == expected_refcount; } @@ -2285,8 +2291,7 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr, break; } - if (folio_ref_count(folio) != - 1 + folio_mapcount(folio) + folio_test_private(folio)) { + if (!is_refcount_suitable(folio)) { result = SCAN_PAGE_COUNT; break; } From patchwork Mon Aug 19 08:14:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13768057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C060C3DA4A for ; Mon, 19 Aug 2024 08:14:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C21466B0083; Mon, 19 Aug 2024 04:14:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B210E6B0092; Mon, 19 Aug 2024 04:14:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B4216B008A; Mon, 19 Aug 2024 04:14:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 774F46B0083 for ; Mon, 19 Aug 2024 04:14:53 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1E2A4C1056 for ; Mon, 19 Aug 2024 08:14:53 +0000 (UTC) X-FDA: 82468284066.06.0D413A6 Received: from out30-100.freemail.mail.aliyun.com (out30-100.freemail.mail.aliyun.com [115.124.30.100]) by imf13.hostedemail.com (Postfix) with ESMTP id DE03320016 for ; Mon, 19 Aug 2024 08:14:50 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=AsTaN93z; spf=pass (imf13.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.100 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724055214; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mYH71MdwFDoWK+6jyn77RluPuidTuLdZqV2Iscl7EWk=; b=3rj1NkAFtbAkwr6BgHmN0QE5oSCEqwrte339kzSVtQLTO+pcVTx46veN3gf06yqFTj4DyX KV31lXj5xR8x/VsqnYMiYxDqLbIXZ/yGkBLUUgPtpuoCURih3dF64OTiF6tPeBMV7feqxd /mmD+sPloIAXHJwpkMSPLvvOpwjHtNo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724055214; a=rsa-sha256; cv=none; b=TAyetosNGUw2JEKqy2Aw1P0JyevrSVelo9ljdhUMDtWSbpS2XYuRAmHcl4fa4pvrKW21fV U6ofZIF86RYyJbJFJyxfdUMRsYtOnMG4R2KlN5b84sKhXFsjWL43ORSyWK/tV+lcZr2pxj jeXQqt9VZGLDGvuRxGe2j2gqkMZZfGc= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=AsTaN93z; spf=pass (imf13.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.100 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724055288; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=mYH71MdwFDoWK+6jyn77RluPuidTuLdZqV2Iscl7EWk=; b=AsTaN93zYybvHAy47unvLcLrxgtIGesYoZoB9oBIpS8vu6HwCaWfFKH3TntpCk8rDAuE3FSC0NGDtLZ5t1CMOMrYbZvOfeQIGa0drX/re7vr51eVHZ6oAEKTMbe66ppLDwgooSLIeUnl9zWupe2Gof/8RPnAuOYZ2Y2cX7c5noE= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WD8W7MM_1724055286) by smtp.aliyun-inc.com; Mon, 19 Aug 2024 16:14:47 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: hughd@google.com, willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/5] mm: khugepaged: use the number of pages in the folio to check the reference count Date: Mon, 19 Aug 2024 16:14:03 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: DE03320016 X-Stat-Signature: hoimp4in1rbdbyx6ncw5oi3a7abnko8x X-HE-Tag: 1724055290-761079 X-HE-Meta: U2FsdGVkX18YT5QKFiwQKDGu504lXMlIrLibZ8Y3HXkM/izyd7zRfGzgOqnM3zUazu1MWUDdlGZ5Yoo7nn7rFD1wi7SeThFKUaKZlVLIYtw14A5y4V+PoN7vst59DN9I61Ae8quKo/jhmIAyAemoWrZr4tdn0j6yOpjyFfAGFeRV9qnnf/YedM04nrhgZavlBZNv8c8i9XxUdUGYqvYyPdGnNTnuXCycHaqdF7KUHIlDGHtnTQ5+PqkBbzDfsRjx8TPNAyouJbEJ1UNsBsP1Om/8v8uUgU56rsiwp4ousvgeENf3w0h0mSLRiue/jWKfigRllH/Qzn8CZTD4cG5MZzDrH9hUcrBNqG8B+JU65u/zERT7ryVGy2jCo+a6LGdfVcAMk/JAuXYM+fwOeCMQ3UK/wujszZuIxfXxGZbU4PzN0nXryP2SV5VOb4QEhLjbPcAJMd3mZO4KR/icYGjZojyCPfzXL86Yc31OR028FCXEeDqYxFgWJTM8MhEoHXaYcerGiV6sP/N/jjlpz8BuB3AwZ9wH/vx+c7bVlCSpKPhY7yBR4IuP5yu0Kfus26xK8gB3+bPgEbvbHR9ir9dBid0GbFgi/t6MlTXzvKngByz3Bv896DMxyn9B238+ns1YoZt/iIUYvSvttqhlH7+FfpXwALEPbAPpM+pUaj+/N04D2XEESh12oLM8VpysTo5NoQ4/p95wdfIKdS7HCtGFTv+gxDOMd1Pu0u4ZvG6yzBy4oix6mnfAwzpJEgs4ZWrWBCm6UzXKA8MNG8Xm4uSn+TEqgUDJvchwEEaqVwa4vOXY/Flye6seVXcd4zT0NEJIpgyBvXsCAB1Ol4LuP3YLBe3artOS5rL54TDlcmmyUygMLytyYwY6C8Z9VVp5fNwvkgc9Oys1kporrdY6lrIPhu3vbyL6GUeL0r8kPONicbD0e5g8ZHeQKwVoGGRYca0mLRo/MfN5DdZE0vevBiS sLQdCsMR 6Tcs8me0AF6dM9YtcUYmLXHXxibWs31py7VdfSOuJ8m/8u4/NVDPJHSTcC5Och7D3IjtaSaMin68biyVFGOVZYu/VjGRxojMsuelSgXZzUR8/UJlFZtnfH1xmqS1pFJrTcFEU0wQ/K7NVqiShxz2BOtglWFo8SReV/1oQaFIWXmWaTguttvw51U6KIOxfnkxidGxthsnc4Qt9ANivHdwjZAjhWT6Us2n9wBC8l8kwn7WWOcQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use the number of pages in the folio to check the reference count as preparation for supporting shmem mTHP collapse. Signed-off-by: Baolin Wang Acked-by: David Hildenbrand --- mm/khugepaged.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f11b4f172e61..60d95f08610c 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1994,7 +1994,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, /* * We control three references to the folio: * - we hold a pin on it; - * - one reference from page cache; + * - nr_pages reference from page cache; * - one from lru_isolate_folio; * If those are the only references, then any new usage * of the folio will have to fetch it from the page @@ -2002,7 +2002,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, * truncate, so any new usage will be blocked until we * unlock folio after collapse/during rollback. */ - if (folio_ref_count(folio) != 3) { + if (folio_ref_count(folio) != 2 + folio_nr_pages(folio)) { result = SCAN_PAGE_COUNT; xas_unlock_irq(&xas); folio_putback_lru(folio); @@ -2185,7 +2185,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, folio_clear_active(folio); folio_clear_unevictable(folio); folio_unlock(folio); - folio_put_refs(folio, 3); + folio_put_refs(folio, 2 + folio_nr_pages(folio)); } goto out; From patchwork Mon Aug 19 08:14:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13768060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B707DC52D7C for ; Mon, 19 Aug 2024 08:15:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 87D5E6B0098; Mon, 19 Aug 2024 04:15:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 82D576B0099; Mon, 19 Aug 2024 04:15:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D1A06B009A; Mon, 19 Aug 2024 04:15:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4CB1D6B0098 for ; Mon, 19 Aug 2024 04:15:02 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0AC55A1084 for ; Mon, 19 Aug 2024 08:15:02 +0000 (UTC) X-FDA: 82468284444.15.0872451 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by imf16.hostedemail.com (Postfix) with ESMTP id E506118001D for ; Mon, 19 Aug 2024 08:14:58 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=ZBd2VAQ6; spf=pass (imf16.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724055224; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xzFqhooDuGLHWOHvpjrYWo9DgZTyuxAAfwMQUt6kp8M=; b=QAsFzkumGxb6EBBUXdxwGHqqiCKEZDHzMfXtYAQr2OJE65bzUn/BVONMk1HbiXh4qtggVs Z0yYErj27QRkc5F0cgsegOZiAJDpN+7kOAm8KOQxcgwEkpdZeodKndxct5tWSHExdp4ZdN OGCumonvFwr5FXcwbt+lXuZAoFx1zj0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724055224; a=rsa-sha256; cv=none; b=tOT2u/aV9QMq6gcLftCBdxCPKCmByny5ERoRSPfJeVAMjDi4eU207bQIyBmHKuEM6U7ama 2bh4UWIVPYNZyMquveg52OwzAyIEY8YxGTk09EqhAOaCJFqvsRf+VZlfhlFRVg2ziKm75/ QchleVM363a375jdzBnY2IA9/pPxMN4= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=ZBd2VAQ6; spf=pass (imf16.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724055289; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=xzFqhooDuGLHWOHvpjrYWo9DgZTyuxAAfwMQUt6kp8M=; b=ZBd2VAQ6mVAeLthLMAGrLwrGf9+g3Nq5KIwokq/ZYTkixUtBzoBUaYqOcqcN65UQLMWqAeAxMq5HLbqwfvjx0kkVLfsAP51Tj9ILed/pjB00moXwyVbHmrFUJAn0tmDdOHxb3CswTguMXjKcmFc1qqIkb/T6Z1Gg6li/Iu4hlU8= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WD9CrIU_1724055287) by smtp.aliyun-inc.com; Mon, 19 Aug 2024 16:14:48 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: hughd@google.com, willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/5] mm: khugepaged: support shmem mTHP copy Date: Mon, 19 Aug 2024 16:14:04 +0800 Message-Id: <938e8cc5a15f841fdd343d324d8a6dd989cd52e0.1724054125.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: E506118001D X-Stat-Signature: zjquf7u61fcufbgmdowmsaqzpzq56arb X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1724055298-614742 X-HE-Meta: U2FsdGVkX19jHbEZJTv5B8F56IB0jhlKTcoY6oCpvJxiNsbFO3sJW6a6PHUcMXoMsJo0NFNWxEOAn3Q6RUfs2jxUU7V4UNFeGnv+WiZhuK32hv7omCsR905Kqff6SW0066BP88o1RqsMYFqsynBP+RZ2RbyXbmcumGBPmtgD7rlCyww6+m4wr/BVtwx/z9hTp2uTNsqpCeFtxisXSRl0HkNPpY9ZOMolE0ZDGhhRpJVQeEyUyJ/r8ZBBMWKMZLwOzyRAagdFW5GtcYuZw8hyoOSLpMkGQnn6Ih8mD+MAxdTOv+KjKQFgrn2w3uGZb2sHAoWoMjxrN+dRyUS+pRge1kf/PvccaNMZlGK5zzMwKrjb0keSCeich/ELJgz/Nq96Pl11xtZA3aq3C2j3QHiYD795AjYAIh9c0Y3teBlIRRKf5zLUggfMOqqCYPR+diyjBpFpj5hK4qKkF1zgQJtRvS/rZ+uWFbLc8b7N7OQlftEzOFRyvFyF7kgXa6Vxea7U2BRpgV222k6u6XHe2V/4JJPKP/d+7pS/0wauOd26JDoSzCqTVe71PGNnmA0z0S8EHzYgNCCJ+CYrNvQlkpPq7LFTHyHOcCqD1PythQ9efmNEZkWz1169EZtgsSalg0I1ndJMs6scj5RwGJkpBujtQvURcvkcR3NJn/2efofEamxjRgoCgwrcX7PNpNaaC4fv2ce7qgXMFxrC3vCOD/HYv3Q9ibf0kElNSj6C+3LC1qT2qyOd0ZBKEg1enTNrpnmLdFE6HWbr5Zcwa+HDwu0mjs1ejcmlHajklXlF3myyMXu4UNNDVbsRogtRRGhclyEBTXae0ynxpQ+DHCF2+3Kzop5UYoIBxoxQvEx/35cYjIRpUeQriW/K08vtKFiEmV+hU2ELuJSp/Ui5bwpj1M1pm6ktePlsB7p0eT6cCnwJip8xgWgl8sSEU9J0U9g2J0zJf+x1xjbV9tyyb5Ol18d wboSlq/D 0JIb9bPfESuddfmLX+k2ntCVzbOpUEmfvHxdzwNtLL1jCFPVcCod0fOeg8jbaLCeusICDtxMAzw/KAbxYGkT73aWt/Q+637u1ZdpeZNml7N8UvbsU4TlyXxRUGctkqF2gwE/+hajr3f6tLWRaRr1JXoRs9ay4vc2ou4qteGDiA8Xfqgn1jBNhyjZcAI/oyHbjyUjd0iCAO39ZQmc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Iterate each subpage in the large folio to copy, as preparation for supporting shmem mTHP collapse. Signed-off-by: Baolin Wang --- mm/khugepaged.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 60d95f08610c..91ee672db202 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2060,17 +2060,22 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, index = start; dst = folio_page(new_folio, 0); list_for_each_entry(folio, &pagelist, lru) { + int i, nr_pages = folio_nr_pages(folio); + while (index < folio->index) { clear_highpage(dst); index++; dst++; } - if (copy_mc_highpage(dst, folio_page(folio, 0)) > 0) { - result = SCAN_COPY_MC; - goto rollback; + + for (i = 0; i < nr_pages; i++) { + if (copy_mc_highpage(dst, folio_page(folio, i)) > 0) { + result = SCAN_COPY_MC; + goto rollback; + } + index++; + dst++; } - index++; - dst++; } while (index < end) { clear_highpage(dst); From patchwork Mon Aug 19 08:14:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13768058 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C48B8C5320E for ; Mon, 19 Aug 2024 08:14:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0D7F6B008C; Mon, 19 Aug 2024 04:14:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E97496B0095; Mon, 19 Aug 2024 04:14:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D35D76B0092; Mon, 19 Aug 2024 04:14:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A6FA96B008A for ; Mon, 19 Aug 2024 04:14:55 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 62900A10A7 for ; Mon, 19 Aug 2024 08:14:55 +0000 (UTC) X-FDA: 82468284150.04.422A1F4 Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) by imf16.hostedemail.com (Postfix) with ESMTP id 2F244180007 for ; Mon, 19 Aug 2024 08:14:52 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=A3z7YfkG; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf16.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.124 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724055230; a=rsa-sha256; cv=none; b=C/QPc8qSJAX8Br4RTJHBxj/SHPnvETKk+HkmRc4ynlmW8VK1AorHzRoXnW6vTSXtACmyaR gMGrvx4WMfgq4To/TF+8I+qKn/PJk1a9MDIMp5N3Jyz9wuneny5MaNRGcXQqCc+RugABFA u4T8647vvGK0xMUqEa80nsOD3+gM4bY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=A3z7YfkG; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf16.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.124 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724055230; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6V7YyBIVTByMizq9TfzYvtX9oniYv5CXu0xU9TzqEDc=; b=Ay8kDodW2oYzKAJ9AjK51ZSuU5bt15vTy85PDi+C3Q7fAWVaIwJ1G1+k8aCHG+v7KfSgx/ qJh2A2a3BglOpOL7PqJbrxCh7wVIqBk7EZvaXnR6zJTmRowQdk8SEmcqo8xYfdVir7pA/e SzrLxGqJa2woT+Y6m9d0toY7zksk5hE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724055290; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=6V7YyBIVTByMizq9TfzYvtX9oniYv5CXu0xU9TzqEDc=; b=A3z7YfkGm4x0DCib5UTsQjmy4e6ieTd746j6dyJd5ERlOW2GpyFPk1mS23yQcpEn1E+f3t+YPUaFmKYgQLITAcBTF2/+gSXAtxM8TPBb+5hrjlar3Kfn5WuXSpLO3/ZkPPyFEcC2/icWr0kpb16+nVo2oTBENXb2HjmK6zTxcl8= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WD9CrIt_1724055288) by smtp.aliyun-inc.com; Mon, 19 Aug 2024 16:14:49 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: hughd@google.com, willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/5] mm: khugepaged: support shmem mTHP collapse Date: Mon, 19 Aug 2024 16:14:05 +0800 Message-Id: <8a2a87affac65f2ecdcea9917b5ae7a261689676.1724054125.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2F244180007 X-Stat-Signature: 76xxhqa6wewmnxru53c97me3gx3wxgfm X-Rspam-User: X-HE-Tag: 1724055292-469048 X-HE-Meta: U2FsdGVkX19w02PSYJ3eIOYhaeQl+0E8JSyHoIWHH5PTNa4Q7A0qLp1IENxpDcmtso34Ga8k5CurVD7ClTZENeCA8+W8eIRR1c11vcdoABilD7boWND/z9J18e7rdtdv1klmqbZCIXqOLMHkW/iuf/B2Eri+TcY69HoXgdiGE/wWWQjsOoKktFd4vk5bQoIfBOnIQHe5si/cuKtRLoX9oz1+TfK7x1DL+5YzDV+8qK8Y5AP79LqIgzSUmdLRXTinihUmuIHlF6Fd6W6RothXOOthc3wPIH+eQEOjibuhEoSBGtWrUsBEmyuKtZIIaTQwQlM0xhj5uAPeL+rA6gEM50zyzNlTsMW62Gffps44V7FI8v7rvRxWfwuq0xT6jnGq/oek+Zs6SktdEJK5mU1Ed92TxZg0C12TCl2XH+08GEGUwJxzYqM1iDykHehf5yyLSeLh29v+9z6yhWFHuyqJm/OCqCecfPNc8sAIifCNG0zejMkMs65WB3zI0Jlrfz6rt3vQCDsQIe4sG3giAAV5UqckJP0Z/ynUcwcJWtOYn7rRZaPzsnCh/Q7K2Su9iAKn4xhcQRCYchF+QxU0o8Y8Qy3SNc6Qt8Vgg4Qco1Nrko0XbHr2oM9AIxBM/jx7Ub8YVxjUD01tLXCf5XEyDBmOgNHiy/k1KgRiHYN1OfCUVp3+5+e87ZIGF/aIwte8SLV3f0Kct60yn3vBgxWZGcxDtGe415MXJmW0bGXnTxwWTAwXnQH7VpArXYjvJhk3JiQa+8T5n5FcOMivlhZ8WKZV16Htd7/7YzUlvn9kL389pHdbj5TjJjcesCB1F5HW3XDm4xLuMuy+PVrO8+nZFW0YBWjHwFVlOEINqvWpsSyVatEMpIhmAemLsUCfA/dUOP/GmpUG22I+whs8UpaobTvjS64x0Wo48o6n0a+C6GDGIM5q1MRLymrs5S5PqWvrBWrpMk7jiUhjW/qCcJiJmZ5 s8acm9xV xzXZqOYf7EZAQiB9KYQVfL+zl3oho2twwACcFNEh6PgTuYhfr+l3LREMtU9g4fQh2Op4UXCSHYrNZkT3jw3BHblVPZZWd5WVb1Usfz/4YGk18bm+uql128QRkDKvLcczIHk+WMX+EOlRtaaO0xSqfGFksNKLfeMqfQ3XqnAzxpJe9Fr5bJnKKZSrmn3fYWNvfQMddiQyNre2sfWvvG+bPgA5p3EeHgLEW+QS0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Shmem already supports the allocation of mTHP, but khugepaged does not yet support collapsing mTHP folios. Now khugepaged is ready to support mTHP, and this patch enables the collapse of shmem mTHP. Signed-off-by: Baolin Wang --- mm/khugepaged.c | 28 +++++++++++----------------- 1 file changed, 11 insertions(+), 17 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 91ee672db202..4b35239b5e46 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1847,7 +1847,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, } } while (1); - for (index = start; index < end; index++) { + for (index = start; index < end;) { xas_set(&xas, index); folio = xas_load(&xas); @@ -1866,6 +1866,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, } } nr_none++; + index++; continue; } @@ -1947,12 +1948,10 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, * we locked the first folio, then a THP might be there already. * This will be discovered on the first iteration. */ - if (folio_test_large(folio)) { - result = folio_order(folio) == HPAGE_PMD_ORDER && - folio->index == start - /* Maybe PMD-mapped */ - ? SCAN_PTE_MAPPED_HUGEPAGE - : SCAN_PAGE_COMPOUND; + if (folio_order(folio) == HPAGE_PMD_ORDER && + folio->index == start) { + /* Maybe PMD-mapped */ + result = SCAN_PTE_MAPPED_HUGEPAGE; goto out_unlock; } @@ -2013,6 +2012,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, * Accumulate the folios that are being collapsed. */ list_add_tail(&folio->lru, &pagelist); + index += folio_nr_pages(folio); continue; out_unlock: folio_unlock(folio); @@ -2265,16 +2265,10 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr, continue; } - /* - * TODO: khugepaged should compact smaller compound pages - * into a PMD sized page - */ - if (folio_test_large(folio)) { - result = folio_order(folio) == HPAGE_PMD_ORDER && - folio->index == start - /* Maybe PMD-mapped */ - ? SCAN_PTE_MAPPED_HUGEPAGE - : SCAN_PAGE_COMPOUND; + if (folio_order(folio) == HPAGE_PMD_ORDER && + folio->index == start) { + /* Maybe PMD-mapped */ + result = SCAN_PTE_MAPPED_HUGEPAGE; /* * For SCAN_PTE_MAPPED_HUGEPAGE, further processing * by the caller won't touch the page cache, and so From patchwork Mon Aug 19 08:14:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13768059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FDC8C52D7C for ; Mon, 19 Aug 2024 08:15:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D6636B0095; Mon, 19 Aug 2024 04:15:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 436E36B0096; Mon, 19 Aug 2024 04:15:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2AF2B6B0098; Mon, 19 Aug 2024 04:15:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0B0346B0095 for ; Mon, 19 Aug 2024 04:15:00 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B92921C4C1A for ; Mon, 19 Aug 2024 08:14:59 +0000 (UTC) X-FDA: 82468284318.01.20E9E28 Received: from out30-113.freemail.mail.aliyun.com (out30-113.freemail.mail.aliyun.com [115.124.30.113]) by imf16.hostedemail.com (Postfix) with ESMTP id B0CA118000E for ; Mon, 19 Aug 2024 08:14:56 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=pIaJgWaL; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf16.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.113 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724055241; a=rsa-sha256; cv=none; b=DpHSMUVwNs0uWqBDrByZdJwqyavRH/yuDQih9CE8eZQe5Q2mWTV4WL3PfrQb0TF+ejd6Z6 vZf+wqLajLjDHJmYj3/P742lmUgSvBJTwSC3/gyryioc9haUKJbkgHmERV2f5VFIm1WgCu em3Z9mYgQOiot+J+hwBFXuYjQIZLKow= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=pIaJgWaL; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf16.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.113 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724055241; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NETcifOYIb+Yf9TxOqDeY39PaKApxrCiukqpjtRmh74=; b=gdiMhSr9CWuEkkgyNrCFPYAXca4PLn9QlPXLT+JByV/0d0QG3jOfXFYuvRQ8voIvP3gxjQ ISUoFrv5qM3erqnRLzKg3wYXrXEPflGHMU+BMg+Eg/Ycewg6T7yjTkMGT6+qZ++cqJQljl SN2NYZHgUez+J6fBSYqZC4UPgYPvJYY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724055291; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=NETcifOYIb+Yf9TxOqDeY39PaKApxrCiukqpjtRmh74=; b=pIaJgWaLYPkSjsM2fSFfdx9rwuEUuQksJTQJKmAOtW4DrFZ3yHH2Fm2VAiYFObD3N/1xw9Au/tL61a17zqxEyQ1QhA3GVXQYT1nX//vMXfywYv/SNA7FS7JItQbAVQ8ixpGFMOnL0XoSvuKFvOF1pBk+PNVZ5poJh4Dmoexd/6M= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WD9CrJE_1724055289) by smtp.aliyun-inc.com; Mon, 19 Aug 2024 16:14:50 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: hughd@google.com, willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/5] selftests: mm: support shmem mTHP collapse testing Date: Mon, 19 Aug 2024 16:14:06 +0800 Message-Id: <12ec6a8b5fe51cd17db1a82c3ffa5006d4b22de3.1724054125.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: B0CA118000E X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: fttg6xotkrtz8dmn57d78mnbi8comrsn X-HE-Tag: 1724055296-874144 X-HE-Meta: U2FsdGVkX19kpwylxz25cQbGOUXGrHv7I/IC8SH2ueDhcxxM9qMyY1RFWUsyZFJ+nWUkoezbauIAczir9vXOCTakH7R6AkIbRlYVbWHBM3QH+6llNud5HuwL/9+WZ194uPNXamAnD4Cq67US/eOJ7TOK1IXL8FlKt7DtrLUHXEl9KUmwvSCflH7O5useWAMF1PusJ4EQ8pRStgkHWJt1lKptFhUFkGLLZD3gEwBPRjda4pMHBIUfpsmMajEolDiiX0Jz0wGtLUxHgc7IR+7PEOofcbXx/DZqc5sPhcRjg7uLo6xLXwn5hzDbsG/cPJHx4d13/1KZof/8Ur4CnSFJCahhHOeBWSxRjvfGjXSLWEc1GvHlJ1EduHH44ToxKQxnDo0accBffW0p3V97g2OgXB8LL1MeOeoHdCQ9XSqupdCr7S27KrWvZeUhVmEyIkP+B4Rj2WiWKv+FldaTjsVeAtPhUrn6q1II3tcrvCyLXYTMDgR3tD+nkrUCFmk6XP52j+wfDA12WH8JO7AKk/ea4VJWnCN2xsCWJBGH3SZqS5I4ToxBkFICHpK8UjZSf7rV7/VuMNMM+op3qBUyROB27lNvax8H1VvhiZPwCg7QFVVUG11fz1QXl4Tip1NHzSz1uGWU8ga7FzTAtFLTzBqWZYS/GG0mwLSX/YCd6B3EKPpanPjqEr9vd1Hpfxm6icLynsihqm5QRdJSgFTDFHsihdSjPewno+xqoXTdaIt9kIBGBSRsQ1UBj1BIjNngA3c5ZvTTHnG2XEfvVd3BKcoqFdp+FHAeBdL90m+6tOzSB/ibVlxxTJ9HLYBA17deImbew17bY+m7JD3yaDrD8LWFR8YJBnNzwivi20Jiekeu2Cfx2LhDVct24voZHoazhyqwXtWMgkjhadWYO2P8cXAdM0jvrmCf0FCangtk2sAVwlxGkxrMGeSQcMg5Uq9izf890OhwWITlwV6r4gUfl9p +GtejCPS ZjczOoe1My+0kH7uWMcuSkYQ/KZnYmTCYMVftCwnCkAlBc/sp6uBprDBpUoZcA6AbH+DHokHzRz8R+F8a+EuyBjon2vUEZIl0nS3Ym0U0vxvOJ7E5ZNNxeP4YYQylm99hVhkBhiPtc8Of8mPfFkzJMVnroan6/HFSKc/JjzVuNGb80x6myNTizli8n0BNw37YyFqy6GxNYh8vnEk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add shmem mTHP collpase testing. Similar to the anonymous page, users can use the '-s' parameter to specify the shmem mTHP size for testing. Signed-off-by: Baolin Wang --- tools/testing/selftests/mm/khugepaged.c | 4 +- tools/testing/selftests/mm/thp_settings.c | 46 ++++++++++++++++++++--- tools/testing/selftests/mm/thp_settings.h | 9 ++++- 3 files changed, 51 insertions(+), 8 deletions(-) diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 829320a519e7..56d4480e8d3c 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -1095,7 +1095,7 @@ static void usage(void) fprintf(stderr, "\n\tSupported Options:\n"); fprintf(stderr, "\t\t-h: This help message.\n"); fprintf(stderr, "\t\t-s: mTHP size, expressed as page order.\n"); - fprintf(stderr, "\t\t Defaults to 0. Use this size for anon allocations.\n"); + fprintf(stderr, "\t\t Defaults to 0. Use this size for anon or shmem allocations.\n"); exit(1); } @@ -1209,6 +1209,8 @@ int main(int argc, char **argv) default_settings.khugepaged.pages_to_scan = hpage_pmd_nr * 8; default_settings.hugepages[hpage_pmd_order].enabled = THP_INHERIT; default_settings.hugepages[anon_order].enabled = THP_ALWAYS; + default_settings.shmem_hugepages[hpage_pmd_order].enabled = SHMEM_INHERIT; + default_settings.shmem_hugepages[anon_order].enabled = SHMEM_ALWAYS; save_settings(); thp_push_settings(&default_settings); diff --git a/tools/testing/selftests/mm/thp_settings.c b/tools/testing/selftests/mm/thp_settings.c index a4163438108e..577eaab6266f 100644 --- a/tools/testing/selftests/mm/thp_settings.c +++ b/tools/testing/selftests/mm/thp_settings.c @@ -33,10 +33,11 @@ static const char * const thp_defrag_strings[] = { }; static const char * const shmem_enabled_strings[] = { + "never", "always", "within_size", "advise", - "never", + "inherit", "deny", "force", NULL @@ -200,6 +201,7 @@ void thp_write_num(const char *name, unsigned long num) void thp_read_settings(struct thp_settings *settings) { unsigned long orders = thp_supported_orders(); + unsigned long shmem_orders = thp_shmem_supported_orders(); char path[PATH_MAX]; int i; @@ -234,12 +236,24 @@ void thp_read_settings(struct thp_settings *settings) settings->hugepages[i].enabled = thp_read_string(path, thp_enabled_strings); } + + for (i = 0; i < NR_ORDERS; i++) { + if (!((1 << i) & shmem_orders)) { + settings->shmem_hugepages[i].enabled = SHMEM_NEVER; + continue; + } + snprintf(path, PATH_MAX, "hugepages-%ukB/shmem_enabled", + (getpagesize() >> 10) << i); + settings->shmem_hugepages[i].enabled = + thp_read_string(path, shmem_enabled_strings); + } } void thp_write_settings(struct thp_settings *settings) { struct khugepaged_settings *khugepaged = &settings->khugepaged; unsigned long orders = thp_supported_orders(); + unsigned long shmem_orders = thp_shmem_supported_orders(); char path[PATH_MAX]; int enabled; int i; @@ -271,6 +285,15 @@ void thp_write_settings(struct thp_settings *settings) enabled = settings->hugepages[i].enabled; thp_write_string(path, thp_enabled_strings[enabled]); } + + for (i = 0; i < NR_ORDERS; i++) { + if (!((1 << i) & shmem_orders)) + continue; + snprintf(path, PATH_MAX, "hugepages-%ukB/shmem_enabled", + (getpagesize() >> 10) << i); + enabled = settings->shmem_hugepages[i].enabled; + thp_write_string(path, shmem_enabled_strings[enabled]); + } } struct thp_settings *thp_current_settings(void) @@ -324,17 +347,18 @@ void thp_set_read_ahead_path(char *path) dev_queue_read_ahead_path[sizeof(dev_queue_read_ahead_path) - 1] = '\0'; } -unsigned long thp_supported_orders(void) +static unsigned long __thp_supported_orders(bool is_shmem) { unsigned long orders = 0; char path[PATH_MAX]; char buf[256]; - int ret; - int i; + int ret, i; + char anon_dir[] = "enabled"; + char shmem_dir[] = "shmem_enabled"; for (i = 0; i < NR_ORDERS; i++) { - ret = snprintf(path, PATH_MAX, THP_SYSFS "hugepages-%ukB/enabled", - (getpagesize() >> 10) << i); + ret = snprintf(path, PATH_MAX, THP_SYSFS "hugepages-%ukB/%s", + (getpagesize() >> 10) << i, is_shmem ? shmem_dir : anon_dir); if (ret >= PATH_MAX) { printf("%s: Pathname is too long\n", __func__); exit(EXIT_FAILURE); @@ -347,3 +371,13 @@ unsigned long thp_supported_orders(void) return orders; } + +unsigned long thp_supported_orders(void) +{ + return __thp_supported_orders(false); +} + +unsigned long thp_shmem_supported_orders(void) +{ + return __thp_supported_orders(true); +} diff --git a/tools/testing/selftests/mm/thp_settings.h b/tools/testing/selftests/mm/thp_settings.h index 71cbff05f4c7..876235a23460 100644 --- a/tools/testing/selftests/mm/thp_settings.h +++ b/tools/testing/selftests/mm/thp_settings.h @@ -22,10 +22,11 @@ enum thp_defrag { }; enum shmem_enabled { + SHMEM_NEVER, SHMEM_ALWAYS, SHMEM_WITHIN_SIZE, SHMEM_ADVISE, - SHMEM_NEVER, + SHMEM_INHERIT, SHMEM_DENY, SHMEM_FORCE, }; @@ -46,6 +47,10 @@ struct khugepaged_settings { unsigned long pages_to_scan; }; +struct shmem_hugepages_settings { + enum shmem_enabled enabled; +}; + struct thp_settings { enum thp_enabled thp_enabled; enum thp_defrag thp_defrag; @@ -54,6 +59,7 @@ struct thp_settings { struct khugepaged_settings khugepaged; unsigned long read_ahead_kb; struct hugepages_settings hugepages[NR_ORDERS]; + struct shmem_hugepages_settings shmem_hugepages[NR_ORDERS]; }; int read_file(const char *path, char *buf, size_t buflen); @@ -76,5 +82,6 @@ void thp_save_settings(void); void thp_set_read_ahead_path(char *path); unsigned long thp_supported_orders(void); +unsigned long thp_shmem_supported_orders(void); #endif /* __THP_SETTINGS_H__ */