From patchwork Tue Mar 1 08:53:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 12764365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D149C433EF for ; Tue, 1 Mar 2022 08:54:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 36C8C8D0006; Tue, 1 Mar 2022 03:54:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 315B18D0001; Tue, 1 Mar 2022 03:54:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B7C48D0006; Tue, 1 Mar 2022 03:54:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0227.hostedemail.com [216.40.44.227]) by kanga.kvack.org (Postfix) with ESMTP id 0DF628D0001 for ; Tue, 1 Mar 2022 03:54:02 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C7232180AC32C for ; Tue, 1 Mar 2022 08:54:01 +0000 (UTC) X-FDA: 79195205082.30.E3EDFE9 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by imf21.hostedemail.com (Postfix) with ESMTP id 0224B1C000A for ; Tue, 1 Mar 2022 08:54:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646124841; x=1677660841; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Is7PELDrfcn7EzisRyWYLhNpJlmmAB/6lqn+KL4wNd4=; b=PygOH9sTW/DLzgAtaKESDzwNjgR+sNIN9ba+ryI7IOK0gxSXN38oF/2Y j/iv0pfntsFiJU/4oc4K/IuQO94BnWbg1+o37/fusA4McN+7bdLjxZjpM wZrNwxhMq8FEFC3GA5R/bMtNIK6Gs1Jmh/l5stFF/G1byzox3azd18oPr ulp/81PIvmMnGOx4RSSUf4kGYN8pD/WFF3C/fMk9paWJbbFYG1NMNgtWV gS4X2/W+P8dTg3Lj53X2M6HSooika0FXLy0GHe0CzPRM9VRNA9tztDsiH IEDi1vrFwSk+MqV4dFMqYOnq6Y348cVEPC1VCwrS3rG1UFVUvIm4M18i+ A==; X-IronPort-AV: E=McAfee;i="6200,9189,10272"; a="240503383" X-IronPort-AV: E=Sophos;i="5.90,145,1643702400"; d="scan'208";a="240503383" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Mar 2022 00:53:58 -0800 X-IronPort-AV: E=Sophos;i="5.90,145,1643702400"; d="scan'208";a="550627144" Received: from yhuang6-desk2.sh.intel.com ([10.239.13.11]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Mar 2022 00:53:52 -0800 From: Huang Ying To: Peter Zijlstra , Mel Gorman , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Feng Tang , Huang Ying , Dave Hansen , Baolin Wang , Johannes Weiner , Oscar Salvador , Michal Hocko , Rik van Riel , Yang Shi , Zi Yan , Wei Xu , Shakeel Butt , zhongjiang-ali Subject: [PATCH -V14 3/3] memory tiering: skip to scan fast memory Date: Tue, 1 Mar 2022 16:53:29 +0800 Message-Id: <20220301085329.3210428-4-ying.huang@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220301085329.3210428-1-ying.huang@intel.com> References: <20220301085329.3210428-1-ying.huang@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 0224B1C000A X-Stat-Signature: eusmcak53xqchtkgir9p176yjjouf7fu Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=PygOH9sT; spf=none (imf21.hostedemail.com: domain of ying.huang@intel.com has no SPF policy when checking 134.134.136.20) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-HE-Tag: 1646124840-437276 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If the NUMA balancing isn't used to optimize the page placement among sockets but only among memory types, the hot pages in the fast memory node couldn't be migrated (promoted) to anywhere. So it's unnecessary to scan the pages in the fast memory node via changing their PTE/PMD mapping to be PROT_NONE. So that the page faults could be avoided too. In the test, if only the memory tiering NUMA balancing mode is enabled, the number of the NUMA balancing hint faults for the DRAM node is reduced to almost 0 with the patch. While the benchmark score doesn't change visibly. Signed-off-by: "Huang, Ying" Suggested-by: Dave Hansen Tested-by: Baolin Wang Reviewed-by: Baolin Wang Acked-by: Johannes Weiner Reviewed-by: Oscar Salvador Cc: Andrew Morton Cc: Michal Hocko Cc: Rik van Riel Cc: Mel Gorman Cc: Peter Zijlstra Cc: Yang Shi Cc: Zi Yan Cc: Wei Xu Cc: Shakeel Butt Cc: zhongjiang-ali Cc: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org Reviewed-by: Yang Shi --- mm/huge_memory.c | 30 +++++++++++++++++++++--------- mm/mprotect.c | 13 ++++++++++++- 2 files changed, 33 insertions(+), 10 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 406a3c28c026..9ce126cb0cfd 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include @@ -1766,17 +1767,28 @@ int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, } #endif - /* - * Avoid trapping faults against the zero page. The read-only - * data is likely to be read-cached on the local CPU and - * local/remote hits to the zero page are not interesting. - */ - if (prot_numa && is_huge_zero_pmd(*pmd)) - goto unlock; + if (prot_numa) { + struct page *page; + /* + * Avoid trapping faults against the zero page. The read-only + * data is likely to be read-cached on the local CPU and + * local/remote hits to the zero page are not interesting. + */ + if (is_huge_zero_pmd(*pmd)) + goto unlock; - if (prot_numa && pmd_protnone(*pmd)) - goto unlock; + if (pmd_protnone(*pmd)) + goto unlock; + page = pmd_page(*pmd); + /* + * Skip scanning top tier node if normal numa + * balancing is disabled + */ + if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_NORMAL) && + node_is_toptier(page_to_nid(page))) + goto unlock; + } /* * In case prot_numa, we are under mmap_read_lock(mm). It's critical * to not clear pmd intermittently to avoid race with MADV_DONTNEED diff --git a/mm/mprotect.c b/mm/mprotect.c index 0138dfcdb1d8..2fe03e695c81 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -83,6 +84,7 @@ static unsigned long change_pte_range(struct vm_area_struct *vma, pmd_t *pmd, */ if (prot_numa) { struct page *page; + int nid; /* Avoid TLB flush if possible */ if (pte_protnone(oldpte)) @@ -109,7 +111,16 @@ static unsigned long change_pte_range(struct vm_area_struct *vma, pmd_t *pmd, * Don't mess with PTEs if page is already on the node * a single-threaded process is running on. */ - if (target_node == page_to_nid(page)) + nid = page_to_nid(page); + if (target_node == nid) + continue; + + /* + * Skip scanning top tier node if normal numa + * balancing is disabled + */ + if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_NORMAL) && + node_is_toptier(nid)) continue; }