From patchwork Tue Jun 4 04:24:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13684615 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DC8BC27C52 for ; Tue, 4 Jun 2024 04:22:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72A2B6B00A3; Tue, 4 Jun 2024 00:22:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DA876B00A4; Tue, 4 Jun 2024 00:22:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 579D46B00A5; Tue, 4 Jun 2024 00:22:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3612B6B00A3 for ; Tue, 4 Jun 2024 00:22:04 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E9895A0B90 for ; Tue, 4 Jun 2024 04:22:03 +0000 (UTC) X-FDA: 82191908526.22.BFAAB7B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf16.hostedemail.com (Postfix) with ESMTP id 51611180004 for ; Tue, 4 Jun 2024 04:22:02 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=n1n6Rqkd; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717474922; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=D25PVgx888IOPs+nLo4v1cjbsQQlK7jqAMxF2FdVb28=; b=bkhmwb3OJ8JTr3IZ7z9HEeukwSUYb0Lx8+uv73ueoyBBFOrb2rOvYCsG46AvMV5rJcv9og 3QK7vcXmU674ceDvIbnFGvXYuDkvCf5keW5/TCRLm+GDgTtpk4+JxUMZcdCqqvMbN0UIZ9 PyCBwQmAomGgPJgqVqH2h8f6L7O0WTg= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=n1n6Rqkd; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717474922; a=rsa-sha256; cv=none; b=1KkYVQAXeEPizaQTTWTe6r2SrquMG7dR0Rbb1nYKs+WTgeUfdOuL6HPXakl0+t/AS9jzHw f6FcKzg0QeeDpu9DwjvsmLNJKpB8xUY8KCL0ZNaBpqlQIJ1Tv7N3k5l5N1RO8ErYI7caLT 09wHzXK8trppQ67EiGE0g5SgmTqExrg= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 9BA5C61203; Tue, 4 Jun 2024 04:22:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F02A6C2BBFC; Tue, 4 Jun 2024 04:21:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717474921; bh=zYmT3lwFo6rWfb+tk0qEdUJ1sHLo1w0Ku5xx7RwN91E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=n1n6RqkdZBqCVuUxiRYbqCsvhoBbYlLF6Tvbv/GcWyfrKGI0NaYyHr/QY8N+fCXeo ZyqzSABKskTK19EVEKFZ0dPpOIG49//2tmiIEI5wj1RjvoGrfxWZMwUvYk3U1fMLZq M0sQEx6nVa6TrVMytPlCJK79PonvX/LOvVR9KZwKxF8cWxEHsRVrel6+djm0sUbqfy l+sZVswL5p6CcvKVP6t8yEMZlFgtcRTcEBMqW8CS1+bnZjfS9f6CSOaJL/axF4ejz7 RRoM2ihQmnQSyii+KyuXtFT5x+qw/CElR9Gf/KgRbdPbJIcCsvuGxA0QwKM0IhZkAE bgeMB1Y+dWEFw== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, izik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, chrisw@sous-sol.org, hughd@google.com, david@redhat.com Cc: "Alex Shi (tencent)" Subject: [PATCH 10/10] m/ksm: use folio in ksm scan path Date: Tue, 4 Jun 2024 12:24:52 +0800 Message-ID: <20240604042454.2012091-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240604042454.2012091-1-alexs@kernel.org> References: <20240604042454.2012091-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 51611180004 X-Stat-Signature: u1zurooa1ew6mfsbtxb165cqyqxnepze X-HE-Tag: 1717474922-260956 X-HE-Meta: U2FsdGVkX1/pC+n9y7hs2i/62qFjR2z7v3w9HAgT1VgM3OydPOmVsVOOfpzG0c1H+5GVLoH57jIlYFof9G4uVZtdRdgrDGAFiIbkS8Gbep3DHsBXvclLoiIqoQp70SEuKazz1RMhR6sK/9Vhk6B1oQAHhoMsBpOgHqJFW6+8ohP5Wtd8hqgK+lTzWdECJZh2sBMag8lXGkWrOem/qCnIxK+ITFxt80Yjk+4HGxjD2B/2CdyZM23YvTUedydw3diB/0UwDOWYEniybxATymYj3Lu67u4710uDmbfG3J4v4eNA423E/9nGvV6EWOYU690uN2TOqxsgXBpph3mQqWbJ9dCBGKTyjRBOdXy1Aw6d0cTkvPx4+aolVNfIN3W6j7i+Rt5aB9qLyz/zQa0Tp8xw5gsqRbFKxxZIb4qI2EZtZG4dX53Y9/dValduSmw0Q3dAlnTXRe/Xxt/9hWgS0mwK23yjPT6LSWffvlaI3Icwz/TAbUE3ZvkqxX6NYaCZXvb6Z8TIyvHn0ejGzNeXbptrgBpTcezojQ3kbMnMpgILREs+fBWE+ety9QgAxVK+h/MTjmGwTQ5a6R6QQ2yZDhQelX+1jiOn46he+YMLP7XhPOA8GMXYlOeg2c9ezp1/zsjYU/hV6sqO/P22YC8WKUmyr02Y7vdyGgr6EylTvHlLnmCuuiEjcPqGKoh076vrIbr8vt/ar/ITKzGjyMK/h4OMN4JVPgrWSwoRKf12yRsfKU3atEiarmV8aOHycmw8ksEK9CVmxWwPCn9w2ilN32txw0opQtjZbFgU5l6nZeSEZjw4hEOB5/gUV8xq55zqOUSG9rQhOJc7KInoQaTt+KRnm7cl4jyWMo8+d2U7/U8CPPOAPzpjdZ/LV3PbVQe7lU+cYqWyhBmWSK7vd2BRCFJR8HOXEz6S6hYfF+iwYqoQkjZ3NlCAcOvVJKZ+bsMBM0J0zp+LGSkbkkZnpVKWbSy 4Hg84OuC Odynhrirr7/kZO0JXvOvcscNnpOo+r3jrxWqvU23LzylncBxXymuL+PjKDjr87BC6uhNXSS6oJ7T8BBZudc9ty7JCtcInSXghzYA8KpDzblANf2E/1eAsK5MEzj8GK3ZGoL8MdHV9xGScSeYFotSM9/7Wh8c9aoh2f2kLp64Tl6tkeUfOHg/ZEwo+XF5wt9Su+3BN/G7uCaRX/tnoH5/dqZVa9JI5+CWEuxspddOiv2VsTSI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Now we could safely use folio in ksm_do_scan path to save a few compound checks. The 'page' still used since follow_page function still no folio version. Signed-off-by: Alex Shi (tencent) --- mm/ksm.c | 47 +++++++++++++++++++++++++---------------------- 1 file changed, 25 insertions(+), 22 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index dc2b5e6a9659..57904b3b18dc 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -2304,7 +2304,7 @@ static void stable_tree_append(struct ksm_rmap_item *rmap_item, * @page: the page that we are searching identical page to. * @rmap_item: the reverse mapping into the virtual address of this page */ -static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_item) +static void cmp_and_merge_page(struct folio *folio, struct ksm_rmap_item *rmap_item) { struct mm_struct *mm = rmap_item->mm; struct ksm_rmap_item *tree_rmap_item; @@ -2314,9 +2314,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite unsigned int checksum; int err; bool max_page_sharing_bypass = false; - struct folio *folio; - folio = page_folio(page); stable_node = folio_stable_node(folio); if (stable_node) { if (stable_node->head != &migrate_nodes && @@ -2518,12 +2516,12 @@ static unsigned int skip_age(rmap_age_t age) } /* - * Determines if a page should be skipped for the current scan. + * Determines if a folio should be skipped for the current scan. * - * @page: page to check - * @rmap_item: associated rmap_item of page + * @folio: folio to check + * @rmap_item: associated rmap_item of folio */ -static bool should_skip_rmap_item(struct page *page, +static bool should_skip_rmap_item(struct folio *folio, struct ksm_rmap_item *rmap_item) { rmap_age_t age; @@ -2536,7 +2534,7 @@ static bool should_skip_rmap_item(struct page *page, * will essentially ignore them, but we still have to process them * properly. */ - if (PageKsm(page)) + if (folio_test_ksm(folio)) return false; age = rmap_item->age; @@ -2566,7 +2564,7 @@ static bool should_skip_rmap_item(struct page *page, return true; } -static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) +static struct ksm_rmap_item *scan_get_next_rmap_item(struct folio **folio) { struct mm_struct *mm; struct ksm_mm_slot *mm_slot; @@ -2655,36 +2653,41 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) ksm_scan.address = vma->vm_end; while (ksm_scan.address < vma->vm_end) { + struct page *page; + if (ksm_test_exit(mm)) break; - *page = follow_page(vma, ksm_scan.address, FOLL_GET); - if (IS_ERR_OR_NULL(*page)) { + page = follow_page(vma, ksm_scan.address, FOLL_GET); + if (IS_ERR_OR_NULL(page)) { ksm_scan.address += PAGE_SIZE; cond_resched(); continue; } - VM_WARN_ON(PageTail(*page)); - nr = compound_nr(*page); - if (is_zone_device_page(*page)) + *folio = page_folio(page); + VM_WARN_ON(PageTail(page)); + nr = folio_nr_pages(*folio); + + if (folio_is_zone_device(*folio)) goto next_page; - if (PageAnon(*page)) { + + if (folio_test_anon(*folio)) { rmap_item = get_next_rmap_item(mm_slot, ksm_scan.rmap_list, ksm_scan.address); if (rmap_item) { ksm_scan.rmap_list = &rmap_item->rmap_list; - if (should_skip_rmap_item(*page, rmap_item)) + if (should_skip_rmap_item(*folio, rmap_item)) goto next_page; ksm_scan.address += nr * PAGE_SIZE; } else - put_page(*page); + folio_put(*folio); mmap_read_unlock(mm); return rmap_item; } next_page: - put_page(*page); + folio_put(*folio); ksm_scan.address += nr * PAGE_SIZE; cond_resched(); } @@ -2755,16 +2758,16 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) static void ksm_do_scan(unsigned int scan_npages) { struct ksm_rmap_item *rmap_item; - struct page *page; + struct folio *folio; unsigned int npages = scan_npages; while (npages-- && likely(!freezing(current))) { cond_resched(); - rmap_item = scan_get_next_rmap_item(&page); + rmap_item = scan_get_next_rmap_item(&folio); if (!rmap_item) return; - cmp_and_merge_page(page, rmap_item); - put_page(page); + cmp_and_merge_page(folio, rmap_item); + folio_put(folio); } ksm_pages_scanned += scan_npages - npages;