From patchwork Thu Nov 26 05:32:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 11932685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A672EC63777 for ; Thu, 26 Nov 2020 05:35:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1D913207BC for ; Thu, 26 Nov 2020 05:35:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1D913207BC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A4B4F6B0070; Thu, 26 Nov 2020 00:35:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9D5BE6B0071; Thu, 26 Nov 2020 00:35:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C48D6B0072; Thu, 26 Nov 2020 00:35:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0110.hostedemail.com [216.40.44.110]) by kanga.kvack.org (Postfix) with ESMTP id 776D26B0070 for ; Thu, 26 Nov 2020 00:35:00 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 372EF824999B for ; Thu, 26 Nov 2020 05:35:00 +0000 (UTC) X-FDA: 77525455560.27.slope64_5c018742737c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 1A3E83D663 for ; Thu, 26 Nov 2020 05:35:00 +0000 (UTC) X-HE-Tag: slope64_5c018742737c X-Filterd-Recvd-Size: 9613 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Nov 2020 05:34:59 +0000 (UTC) IronPort-SDR: od+GxwzdGC8aBSa6Kib3ShF2mL2BzY/Z8X6GpFuqnhq+mNmGvPzFIZqtKoCY4UbS/uAjyiETlo qXJr4UV/hmfg== X-IronPort-AV: E=McAfee;i="6000,8403,9816"; a="190391006" X-IronPort-AV: E=Sophos;i="5.78,371,1599548400"; d="scan'208";a="190391006" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Nov 2020 21:34:57 -0800 IronPort-SDR: qWKpFP5TIso9GcjYB/lNNHf09IO0dE2DMngnHwBTfsb04fffNCRPvbaGK/dTVme6PJ7EcRyvHE Rab/LrtXIARw== X-IronPort-AV: E=Sophos;i="5.78,371,1599548400"; d="scan'208";a="537180143" Received: from yhuang-mobile.sh.intel.com ([10.238.5.184]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Nov 2020 21:34:53 -0800 From: Huang Ying To: Peter Zijlstra Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Matthew Wilcox (Oracle)" , Rafael Aquini , Andrew Morton , Ingo Molnar , Mel Gorman , Rik van Riel , Johannes Weiner , Dave Hansen , Andi Kleen , Michal Hocko , David Rientjes , linux-api@vger.kernel.org, "Huang, Ying" Subject: [PATCH -V6 3/3] NOT kernel/numactl: Support to enable Linux kernel NUMA balancing Date: Thu, 26 Nov 2020 13:32:19 +0800 Message-Id: <20201126053219.234793-4-ying.huang@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126053219.234793-1-ying.huang@intel.com> References: <20201126053219.234793-1-ying.huang@intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Huang Ying A new API: numa_set_membind_balancing() is added to libnuma. It is same as numa_set_membind() except that the Linux kernel NUMA balancing will be enabled for the task if the feature is supported by the kernel. At the same time, a new option: --balancing (-b) is added to numactl. Which can be used before the memory policy options in the command line. With it, the Linux kernel NUMA balancing will be enabled for the process if the feature is supported by the kernel. Signed-off-by: "Huang, Ying" --- libnuma.c | 14 ++++++++++++++ numa.3 | 15 +++++++++++++++ numa.h | 4 ++++ numactl.8 | 9 +++++++++ numactl.c | 17 ++++++++++++++--- numaif.h | 3 +++ versions.ldscript | 8 ++++++++ 7 files changed, 67 insertions(+), 3 deletions(-) diff --git a/libnuma.c b/libnuma.c index 88f479b..f073c50 100644 --- a/libnuma.c +++ b/libnuma.c @@ -1064,6 +1064,20 @@ numa_set_membind_v2(struct bitmask *bmp) make_internal_alias(numa_set_membind_v2); +void +numa_set_membind_balancing(struct bitmask *bmp) +{ + /* MPOL_F_NUMA_BALANCING: ignore if unsupported */ + if (set_mempolicy(MPOL_BIND | MPOL_F_NUMA_BALANCING, + bmp->maskp, bmp->size + 1) < 0) { + if (errno == EINVAL) { + errno = 0; + numa_set_membind_v2(bmp); + } else + numa_error("set_mempolicy"); + } +} + /* * copy a bitmask map body to a numa.h nodemask_t structure */ diff --git a/numa.3 b/numa.3 index 3e18098..c44ee50 100644 --- a/numa.3 +++ b/numa.3 @@ -80,6 +80,8 @@ numa \- NUMA policy library .br .BI "void numa_set_membind(struct bitmask *" nodemask ); .br +.BI "void numa_set_membind_balancing(struct bitmask *" nodemask ); +.br .B struct bitmask *numa_get_membind(void); .sp .BI "void *numa_alloc_onnode(size_t " size ", int " node ); @@ -538,6 +540,19 @@ that contains nodes other than those in the mask returned by .IR numa_get_mems_allowed () will result in an error. +.BR numa_set_membind_balancing () +sets the memory allocation mask and enable the Linux kernel NUMA +balancing for the task if the feature is supported by the kernel. The +task will only allocate memory from the nodes set in +.IR nodemask . +Passing an empty +.I nodemask +or a +.I nodemask +that contains nodes other than those in the mask returned by +.IR numa_get_mems_allowed () +will result in an error. + .BR numa_get_membind () returns the mask of nodes from which memory can currently be allocated. If the returned mask is equal to diff --git a/numa.h b/numa.h index bd1d676..5d8543a 100644 --- a/numa.h +++ b/numa.h @@ -192,6 +192,10 @@ void numa_set_localalloc(void); /* Only allocate memory from the nodes set in mask. 0 to turn off */ void numa_set_membind(struct bitmask *nodemask); +/* Only allocate memory from the nodes set in mask. Optimize page + placement with Linux kernel NUMA balancing if possible. 0 to turn off */ +void numa_set_membind_balancing(struct bitmask *bmp); + /* Return current membind */ struct bitmask *numa_get_membind(void); diff --git a/numactl.8 b/numactl.8 index f3bb22b..109dd8f 100644 --- a/numactl.8 +++ b/numactl.8 @@ -25,6 +25,8 @@ numactl \- Control NUMA policy for processes or shared memory [ .B \-\-all ] [ +.B \-\-balancing +] [ .B \-\-interleave nodes ] [ .B \-\-preferred node @@ -168,6 +170,9 @@ but if memory cannot be allocated there fall back to other nodes. This option takes only a single node number. Relative notation may be used. .TP +.B \-\-balancing, \-b +Enable Linux kernel NUMA balancing for the process if it is supported by kernel. +.TP .B \-\-show, \-s Show NUMA policy settings of the current process. .TP @@ -278,6 +283,10 @@ numactl \-\-cpunodebind=0 \-\-membind=0,1 -- process -l Run process as above, but with an option (-l) that would be confused with a numactl option. +numactl \-\-cpunodebind=0 \-\-balancing \-\-membind=0,1 process +Run process on node 0 with memory allocated on node 0 and 1. Optimize the +page placement with Linux kernel NUMA balancing mechanism if possible. + numactl \-\-cpunodebind=netdev:eth0 \-\-membind=netdev:eth0 network-server Run network-server on the node of network device eth0 with its memory also in the same node. diff --git a/numactl.c b/numactl.c index df9dbcb..5a9d2df 100644 --- a/numactl.c +++ b/numactl.c @@ -45,6 +45,7 @@ struct option opts[] = { {"membind", 1, 0, 'm'}, {"show", 0, 0, 's' }, {"localalloc", 0,0, 'l'}, + {"balancing", 0, 0, 'b'}, {"hardware", 0,0,'H' }, {"shm", 1, 0, 'S'}, @@ -65,9 +66,10 @@ struct option opts[] = { void usage(void) { fprintf(stderr, - "usage: numactl [--all | -a] [--interleave= | -i ] [--preferred= | -p ]\n" - " [--physcpubind= | -C ] [--cpunodebind= | -N ]\n" - " [--membind= | -m ] [--localalloc | -l] command args ...\n" + "usage: numactl [--all | -a] [--balancing | -b] [--interleave= | -i ]\n" + " [--preferred= | -p ] [--physcpubind= | -C ]\n" + " [--cpunodebind= | -N ] [--membind= | -m ]\n" + " [--localalloc | -l] command args ...\n" " numactl [--show | -s]\n" " numactl [--hardware | -H]\n" " numactl [--length | -l ] [--offset | -o ] [--shmmode | -M ]\n" @@ -90,6 +92,8 @@ void usage(void) "all numbers and ranges can be made cpuset-relative with +\n" "the old --cpubind argument is deprecated.\n" "use --cpunodebind or --physcpubind instead\n" + "use --balancing | -b to enable Linux kernel NUMA balancing\n" + "for the process if it is supported by kernel\n" " can have g (GB), m (MB) or k (KB) suffixes\n"); exit(1); } @@ -338,6 +342,7 @@ int do_dump = 0; int shmattached = 0; int did_node_cpu_parse = 0; int parse_all = 0; +int numa_balancing = 0; char *shmoption; void check_cpubind(int flag) @@ -431,6 +436,10 @@ int main(int ac, char **av) nopolicy(); hardware(); exit(0); + case 'b': /* --balancing */ + nopolicy(); + numa_balancing = 1; + break; case 'i': /* --interleave */ checknuma(); if (parse_all) @@ -507,6 +516,8 @@ int main(int ac, char **av) numa_set_bind_policy(1); if (shmfd >= 0) { numa_tonodemask_memory(shmptr, shmlen, mask); + } else if (numa_balancing) { + numa_set_membind_balancing(mask); } else { numa_set_membind(mask); } diff --git a/numaif.h b/numaif.h index 91aa230..32c12c3 100644 --- a/numaif.h +++ b/numaif.h @@ -29,6 +29,9 @@ extern long move_pages(int pid, unsigned long count, #define MPOL_LOCAL 4 #define MPOL_MAX 5 +/* Flags for set_mempolicy, specified in mode */ +#define MPOL_F_NUMA_BALANCING (1 << 13) /* Optimize with NUMA balancing if possible */ + /* Flags for get_mem_policy */ #define MPOL_F_NODE (1<<0) /* return next il node or node of address */ /* Warning: MPOL_F_NODE is unsupported and diff --git a/versions.ldscript b/versions.ldscript index 23074a0..358eeeb 100644 --- a/versions.ldscript +++ b/versions.ldscript @@ -146,3 +146,11 @@ libnuma_1.4 { local: *; } libnuma_1.3; + +# New interface for membind with NUMA balancing optimization +libnuma_1.5 { + global: + numa_set_membind_balancing; + local: + *; +} libnuma_1.4;