From patchwork Fri Mar 11 09:01:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: bibo mao X-Patchwork-Id: 12777693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C40AC433EF for ; Fri, 11 Mar 2022 09:01:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A1DC18D0002; Fri, 11 Mar 2022 04:01:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CDE48D0001; Fri, 11 Mar 2022 04:01:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BE018D0002; Fri, 11 Mar 2022 04:01:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 7891A8D0001 for ; Fri, 11 Mar 2022 04:01:28 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4AEA91255 for ; Fri, 11 Mar 2022 09:01:28 +0000 (UTC) X-FDA: 79231511856.08.D1A2372 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by imf20.hostedemail.com (Postfix) with ESMTP id 48A5E1C002E for ; Fri, 11 Mar 2022 09:01:24 +0000 (UTC) Received: from localhost.localdomain (unknown [10.2.5.185]) by mail.loongson.cn (Coremail) with SMTP id AQAAf9Dx7xPgDytiQD0HAA--.2826S2; Fri, 11 Mar 2022 17:01:20 +0800 (CST) From: Bibo Mao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH] mm/khugepaged: sched to numa node when collapse huge page Date: Fri, 11 Mar 2022 04:01:19 -0500 Message-Id: <20220311090119.2412738-1-maobibo@loongson.cn> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-CM-TRANSID: AQAAf9Dx7xPgDytiQD0HAA--.2826S2 X-Coremail-Antispam: 1UD129KBjvJXoW7Zw47ZFW8uw1UXFW8GFWxZwb_yoW8CryUpF WUtw4UGrWUJr1vgr1Iqan8AryFqr1kJFWktw1fAas7t3s0qr4FgFy5Za15A34UJFWkGFW3 ArWavrn09r48J3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnUUvcSsGvfC2KfnxnUUI43ZEXa7xR_UUUUUUUUU== X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 48A5E1C002E X-Stat-Signature: 57xebebxghoxrirs8wi343hdkb1u7fs6 Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn; dmarc=none X-HE-Tag: 1646989284-933533 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: collapse huge page is slow, specially when khugepaged daemon runs on different numa node with that of huge page. It suffers from huge page copying across nodes, also cache is not used for target node. With this patch, khugepaged daemon switches to the same numa node with huge page. It saves copying time and makes use of local cache better. Signed-off-by: Bibo Mao --- mm/khugepaged.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 131492fd1148..460c285dc974 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -116,6 +116,7 @@ struct khugepaged_scan { struct list_head mm_head; struct mm_slot *mm_slot; unsigned long address; + int node; }; static struct khugepaged_scan khugepaged_scan = { @@ -1066,6 +1067,7 @@ static void collapse_huge_page(struct mm_struct *mm, struct vm_area_struct *vma; struct mmu_notifier_range range; gfp_t gfp; + const struct cpumask *cpumask; VM_BUG_ON(address & ~HPAGE_PMD_MASK); @@ -1079,6 +1081,13 @@ static void collapse_huge_page(struct mm_struct *mm, * that. We will recheck the vma after taking it again in write mode. */ mmap_read_unlock(mm); + + /* sched to specified node before huage page memory copy */ + cpumask = cpumask_of_node(node); + if ((khugepaged_scan.node != node) && !cpumask_empty(cpumask)) { + set_cpus_allowed_ptr(current, cpumask); + khugepaged_scan.node = node; + } new_page = khugepaged_alloc_page(hpage, gfp, node); if (!new_page) { result = SCAN_ALLOC_HUGE_PAGE_FAIL; @@ -2380,6 +2389,7 @@ int start_stop_khugepaged(void) kthread_stop(khugepaged_thread); khugepaged_thread = NULL; } + khugepaged_scan.node = NUMA_NO_NODE; set_recommended_min_free_kbytes(); fail: mutex_unlock(&khugepaged_mutex);