From patchwork Tue Aug 25 00:23:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 11734497 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3ABA8913 for ; Tue, 25 Aug 2020 00:24:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 11F432074D for ; Tue, 25 Aug 2020 00:24:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 11F432074D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 038086B0024; Mon, 24 Aug 2020 20:24:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F29C38D0003; Mon, 24 Aug 2020 20:24:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3FD86B0027; Mon, 24 Aug 2020 20:24:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id CDB726B0024 for ; Mon, 24 Aug 2020 20:24:46 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 918A88248047 for ; Tue, 25 Aug 2020 00:24:46 +0000 (UTC) X-FDA: 77187195372.27.value78_540ba9627057 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 6E2153D663 for ; Tue, 25 Aug 2020 00:24:46 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,ying.huang@intel.com,,RULES_HIT:30034:30054:30064:30070,0,RBL:134.134.136.20:@intel.com:.lbl8.mailshell.net-62.50.0.100 64.95.201.95;04y8m7dyy4wxxu145dapgc77eo31tycbfw9t9aamigegsdzrz1ypnxgnxxt4af4.zgyehohfo7csbrumihhe1bqe7ohkbjuwe7sj5jpf5azbif16jc7kimpx387mac3.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: value78_540ba9627057 X-Filterd-Recvd-Size: 6128 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Tue, 25 Aug 2020 00:24:45 +0000 (UTC) IronPort-SDR: ZcEJd23A0QDL/glftYXgXQjQy1Qxicl06W+RGVNFGywzDJYExGYz8FZheHJ9fjt+gOH7Br2S8k qNE7DrbF8PQw== X-IronPort-AV: E=McAfee;i="6000,8403,9723"; a="143794322" X-IronPort-AV: E=Sophos;i="5.76,350,1592895600"; d="scan'208";a="143794322" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Aug 2020 17:24:45 -0700 IronPort-SDR: MtGg5mOs/IDkLKaqt+QPQJD6f31cfW72KzhVgxXzfncPKSnpMAA/Dt9Nbm4beW7eNzyPpi55wm EROedV1onnHw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,350,1592895600"; d="scan'208";a="443428049" Received: from huiyao-mobl2.ccr.corp.intel.com (HELO yhuang-mobile.ccr.corp.intel.com) ([10.254.214.197]) by orsmga004.jf.intel.com with ESMTP; 24 Aug 2020 17:24:41 -0700 From: Huang Ying To: Peter Zijlstra Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , Dave Hansen , Andrew Morton , Michal Hocko , Rik van Riel , Mel Gorman , Ingo Molnar , Dan Williams Subject: [RFC -V3 2/5] autonuma, memory tiering: Skip to scan fast memory Date: Tue, 25 Aug 2020 08:23:51 +0800 Message-Id: <20200825002354.17038-3-ying.huang@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200825002354.17038-1-ying.huang@intel.com> References: <20200825002354.17038-1-ying.huang@intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6E2153D663 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If the AutoNUMA isn't used to optimize the page placement among sockets but only among memory types, the hot pages in the fast memory node couldn't be migrated (promoted) to anywhere. So it's unnecessary to scan the pages in the fast memory node via changing their PTE/PMD mapping to be PROT_NONE. So that the page faults could be avoided too. In the test, if only the memory tiering AutoNUMA mode is enabled, the number of the AutoNUMA hint faults for the DRAM node is reduced to almost 0 with the patch. While the benchmark score doesn't change visibly. Signed-off-by: "Huang, Ying" Suggested-by: Dave Hansen Cc: Andrew Morton Cc: Michal Hocko Cc: Rik van Riel Cc: Mel Gorman Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Dan Williams Cc: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org --- include/linux/node.h | 5 +++++ mm/huge_memory.c | 30 +++++++++++++++++++++--------- mm/mprotect.c | 13 ++++++++++++- 3 files changed, 38 insertions(+), 10 deletions(-) diff --git a/include/linux/node.h b/include/linux/node.h index f7a539390c81..ac0e7a45edff 100644 --- a/include/linux/node.h +++ b/include/linux/node.h @@ -189,4 +189,9 @@ static inline int next_demotion_node(int node) } #endif +static inline bool node_is_toptier(int node) +{ + return node_state(node, N_CPU); +} + #endif /* _LINUX_NODE_H_ */ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 78c84bee7e29..7d5db965a48c 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include @@ -1821,17 +1822,28 @@ int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, } #endif - /* - * Avoid trapping faults against the zero page. The read-only - * data is likely to be read-cached on the local CPU and - * local/remote hits to the zero page are not interesting. - */ - if (prot_numa && is_huge_zero_pmd(*pmd)) - goto unlock; + if (prot_numa) { + struct page *page; + /* + * Avoid trapping faults against the zero page. The read-only + * data is likely to be read-cached on the local CPU and + * local/remote hits to the zero page are not interesting. + */ + if (is_huge_zero_pmd(*pmd)) + goto unlock; - if (prot_numa && pmd_protnone(*pmd)) - goto unlock; + if (pmd_protnone(*pmd)) + goto unlock; + page = pmd_page(*pmd); + /* + * Skip scanning top tier node if normal numa + * balancing is disabled + */ + if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_NORMAL) && + node_is_toptier(page_to_nid(page))) + goto unlock; + } /* * In case prot_numa, we are under mmap_read_lock(mm). It's critical * to not clear pmd intermittently to avoid race with MADV_DONTNEED diff --git a/mm/mprotect.c b/mm/mprotect.c index ce8b8a5eacbb..8abec0c267fa 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -83,6 +84,7 @@ static unsigned long change_pte_range(struct vm_area_struct *vma, pmd_t *pmd, */ if (prot_numa) { struct page *page; + int nid; /* Avoid TLB flush if possible */ if (pte_protnone(oldpte)) @@ -109,7 +111,16 @@ static unsigned long change_pte_range(struct vm_area_struct *vma, pmd_t *pmd, * Don't mess with PTEs if page is already on the node * a single-threaded process is running on. */ - if (target_node == page_to_nid(page)) + nid = page_to_nid(page); + if (target_node == nid) + continue; + + /* + * Skip scanning top tier node if normal numa + * balancing is disabled + */ + if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_NORMAL) && + node_is_toptier(nid)) continue; }