From patchwork Sun Jun 21 07:36:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 11616467 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D461013B1 for ; Sun, 21 Jun 2020 07:36:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AB43A23D99 for ; Sun, 21 Jun 2020 07:36:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB43A23D99 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3ABE28D001B; Sun, 21 Jun 2020 03:36:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 335528D0008; Sun, 21 Jun 2020 03:36:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D4008D001B; Sun, 21 Jun 2020 03:36:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id 053A28D0008 for ; Sun, 21 Jun 2020 03:36:51 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id BB3998248068 for ; Sun, 21 Jun 2020 07:36:50 +0000 (UTC) X-FDA: 76952412180.02.bells29_6312b2426e28 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id 8E89060000AAE18D for ; Sun, 21 Jun 2020 07:36:50 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,feng.tang@intel.com,,RULES_HIT:30054:30064:30070,0,RBL:192.55.52.136:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: bells29_6312b2426e28 X-Filterd-Recvd-Size: 2946 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Sun, 21 Jun 2020 07:36:49 +0000 (UTC) IronPort-SDR: 0BRd7hKW3oaH0Aqv7Qs8Un1RZB1akmenDFz5iBUiQDAj4g6lL0M5sw6esyqi6JbQwTV+eTGQid Vte3FWXjw4kg== X-IronPort-AV: E=McAfee;i="6000,8403,9658"; a="123177661" X-IronPort-AV: E=Sophos;i="5.75,262,1589266800"; d="scan'208";a="123177661" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jun 2020 00:36:48 -0700 IronPort-SDR: EWcJttH9zBV9fz+andEDlkAPwbjTsMiyjxbEBLq+u9gHmTABVxHEjYZs1X3J7bOS6zubNgyUMn WnxGskZQcKrA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,262,1589266800"; d="scan'208";a="478102511" Received: from shbuild999.sh.intel.com ([10.239.146.107]) by fmsmga005.fm.intel.com with ESMTP; 21 Jun 2020 00:36:45 -0700 From: Feng Tang To: Andrew Morton , Michal Hocko , Johannes Weiner , Matthew Wilcox , Mel Gorman , Kees Cook , Luis Chamberlain , Iurii Zaikin , andi.kleen@intel.com, tim.c.chen@intel.com, dave.hansen@intel.com, ying.huang@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Feng Tang Subject: [PATCH v5 1/3] proc/meminfo: avoid open coded reading of vm_committed_as Date: Sun, 21 Jun 2020 15:36:38 +0800 Message-Id: <1592725000-73486-2-git-send-email-feng.tang@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592725000-73486-1-git-send-email-feng.tang@intel.com> References: <1592725000-73486-1-git-send-email-feng.tang@intel.com> X-Rspamd-Queue-Id: 8E89060000AAE18D X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the existing vm_memory_committed() instead, which is also convenient for future change. Signed-off-by: Feng Tang Acked-by: Michal Hocko Cc: Matthew Wilcox (Oracle) Cc: Johannes Weiner Cc: Mel Gorman Cc: Kees Cook Cc: Andi Kleen Cc: Tim Chen Cc: Dave Hansen Cc: Huang Ying --- fs/proc/meminfo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 38ea95f..f262bff 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -41,7 +41,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) si_meminfo(&i); si_swapinfo(&i); - committed = percpu_counter_read_positive(&vm_committed_as); + committed = vm_memory_committed(); cached = global_node_page_state(NR_FILE_PAGES) - total_swapcache_pages() - i.bufferram; From patchwork Sun Jun 21 07:36:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 11616469 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 816721392 for ; Sun, 21 Jun 2020 07:36:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 57AFD23B86 for ; Sun, 21 Jun 2020 07:36:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 57AFD23B86 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 557C18D001C; Sun, 21 Jun 2020 03:36:56 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4E0898D0008; Sun, 21 Jun 2020 03:36:56 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A87F8D001C; Sun, 21 Jun 2020 03:36:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0065.hostedemail.com [216.40.44.65]) by kanga.kvack.org (Postfix) with ESMTP id 242BC8D0008 for ; Sun, 21 Jun 2020 03:36:56 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D04642C07 for ; Sun, 21 Jun 2020 07:36:55 +0000 (UTC) X-FDA: 76952412390.17.uncle59_031631126e28 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id A8B16180D0184 for ; Sun, 21 Jun 2020 07:36:55 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,feng.tang@intel.com,,RULES_HIT:30046:30054:30064:30075,0,RBL:192.55.52.136:@intel.com:.lbl8.mailshell.net-64.95.201.95 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: uncle59_031631126e28 X-Filterd-Recvd-Size: 4023 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Sun, 21 Jun 2020 07:36:54 +0000 (UTC) IronPort-SDR: uOOh1WgdVzHP1d3r47b/XX9pYLkHFdR0l23DUpDo95CxVVZ9aBMAZmYWqv1ccbHUCrO1XRX5UL 8Pt+J8nZeRWw== X-IronPort-AV: E=McAfee;i="6000,8403,9658"; a="123177663" X-IronPort-AV: E=Sophos;i="5.75,262,1589266800"; d="scan'208";a="123177663" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jun 2020 00:36:52 -0700 IronPort-SDR: Eoxe27Qgas/5AxcjOz+orM3yoyS4wCnzWFHVAf5HL3mtPx9gks+PcISKWPj172HZBsG/O9VbkT H9kkx+X0isFA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,262,1589266800"; d="scan'208";a="478102527" Received: from shbuild999.sh.intel.com ([10.239.146.107]) by fmsmga005.fm.intel.com with ESMTP; 21 Jun 2020 00:36:48 -0700 From: Feng Tang To: Andrew Morton , Michal Hocko , Johannes Weiner , Matthew Wilcox , Mel Gorman , Kees Cook , Luis Chamberlain , Iurii Zaikin , andi.kleen@intel.com, tim.c.chen@intel.com, dave.hansen@intel.com, ying.huang@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Feng Tang , "K. Y. Srinivasan" , Haiyang Zhang Subject: [PATCH v5 2/3] mm/util.c: make vm_memory_committed() more accurate Date: Sun, 21 Jun 2020 15:36:39 +0800 Message-Id: <1592725000-73486-3-git-send-email-feng.tang@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592725000-73486-1-git-send-email-feng.tang@intel.com> References: <1592725000-73486-1-git-send-email-feng.tang@intel.com> X-Rspamd-Queue-Id: A8B16180D0184 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: percpu_counter_sum_positive() will provide more accurate info. As with percpu_counter_read_positive(), in worst case the deviation could be 'batch * nr_cpus', which is totalram_pages/256 for now, and will be more when the batch gets enlarged. Its time cost is about 800 nanoseconds on a 2C/4T platform and 2~3 microseconds on a 2S/36C/72T Skylake server in normal case, and in worst case where vm_committed_as's spinlock is under severe contention, it costs 30~40 microseconds for the 2S/36C/72T Skylake sever, which should be fine for its only two users: /proc/meminfo and HyperV balloon driver's status trace per second. Signed-off-by: Feng Tang Acked-by: Michal Hocko # for /proc/meminfo Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Matthew Wilcox (Oracle) Cc: Johannes Weiner Cc: Mel Gorman Cc: Andi Kleen Cc: Tim Chen Cc: Dave Hansen Cc: Huang Ying --- mm/util.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/util.c b/mm/util.c index c63c8e4..1c9d097 100644 --- a/mm/util.c +++ b/mm/util.c @@ -787,10 +787,15 @@ struct percpu_counter vm_committed_as ____cacheline_aligned_in_smp; * balancing memory across competing virtual machines that are hosted. * Several metrics drive this policy engine including the guest reported * memory commitment. + * + * The time cost of this is very low for small platforms, and for big + * platform like a 2S/36C/72T Skylake server, in worst case where + * vm_committed_as's spinlock is under severe contention, the time cost + * could be about 30~40 microseconds. */ unsigned long vm_memory_committed(void) { - return percpu_counter_read_positive(&vm_committed_as); + return percpu_counter_sum_positive(&vm_committed_as); } EXPORT_SYMBOL_GPL(vm_memory_committed); From patchwork Sun Jun 21 07:36:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 11616471 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3885A138C for ; Sun, 21 Jun 2020 07:37:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0EB3F23B86 for ; Sun, 21 Jun 2020 07:36:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0EB3F23B86 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B4BC78D001D; Sun, 21 Jun 2020 03:36:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AD5B08D0008; Sun, 21 Jun 2020 03:36:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99E4F8D001D; Sun, 21 Jun 2020 03:36:57 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0068.hostedemail.com [216.40.44.68]) by kanga.kvack.org (Postfix) with ESMTP id 83A828D0008 for ; Sun, 21 Jun 2020 03:36:57 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 3F91A8248068 for ; Sun, 21 Jun 2020 07:36:57 +0000 (UTC) X-FDA: 76952412474.14.coil48_07180b026e28 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 194C818229818 for ; Sun, 21 Jun 2020 07:36:57 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,feng.tang@intel.com,,RULES_HIT:30034:30054:30064:30070,0,RBL:192.55.52.136:@intel.com:.lbl8.mailshell.net-64.95.201.95 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1,LUA_SUMMARY:none X-HE-Tag: coil48_07180b026e28 X-Filterd-Recvd-Size: 7601 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Sun, 21 Jun 2020 07:36:55 +0000 (UTC) IronPort-SDR: 070AKy7rUw8q+f661LFy+lz9opIz6d/WxwDzhBnFKxYx2Or6GiMnvtEGMk7jKRUsMbcA8+K0x7 dlRDn9RZ8yEw== X-IronPort-AV: E=McAfee;i="6000,8403,9658"; a="123177665" X-IronPort-AV: E=Sophos;i="5.75,262,1589266800"; d="scan'208";a="123177665" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jun 2020 00:36:55 -0700 IronPort-SDR: fKe89FpcbZ7ysZDzI2TjwVwce1DW6iyYrXNJXfV1zY1fHS0oUOlxxHyH1Tauy2Lv5iNq9o++OX c19rlnb5u3Ew== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,262,1589266800"; d="scan'208";a="478102550" Received: from shbuild999.sh.intel.com ([10.239.146.107]) by fmsmga005.fm.intel.com with ESMTP; 21 Jun 2020 00:36:52 -0700 From: Feng Tang To: Andrew Morton , Michal Hocko , Johannes Weiner , Matthew Wilcox , Mel Gorman , Kees Cook , Luis Chamberlain , Iurii Zaikin , andi.kleen@intel.com, tim.c.chen@intel.com, dave.hansen@intel.com, ying.huang@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Feng Tang Subject: [PATCH v5 3/3] mm: adjust vm_committed_as_batch according to vm overcommit policy Date: Sun, 21 Jun 2020 15:36:40 +0800 Message-Id: <1592725000-73486-4-git-send-email-feng.tang@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592725000-73486-1-git-send-email-feng.tang@intel.com> References: <1592725000-73486-1-git-send-email-feng.tang@intel.com> X-Rspamd-Queue-Id: 194C818229818 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When checking a performance change for will-it-scale scalability mmap test [1], we found very high lock contention for spinlock of percpu counter 'vm_committed_as': 94.14% 0.35% [kernel.kallsyms] [k] _raw_spin_lock_irqsave 48.21% _raw_spin_lock_irqsave;percpu_counter_add_batch;__vm_enough_memory;mmap_region;do_mmap; 45.91% _raw_spin_lock_irqsave;percpu_counter_add_batch;__do_munmap; Actually this heavy lock contention is not always necessary. The 'vm_committed_as' needs to be very precise when the strict OVERCOMMIT_NEVER policy is set, which requires a rather small batch number for the percpu counter. So keep 'batch' number unchanged for strict OVERCOMMIT_NEVER policy, and lift it to 64X for OVERCOMMIT_ALWAYS and OVERCOMMIT_GUESS policies. Also add a sysctl handler to adjust it when the policy is reconfigured. Benchmark with the same testcase in [1] shows 53% improvement on a 8C/16T desktop, and 2097%(20X) on a 4S/72C/144T server. We tested with test platforms in 0day (server, desktop and laptop), and 80%+ platforms shows improvements with that test. And whether it shows improvements depends on if the test mmap size is bigger than the batch number computed. And if the lift is 16X, 1/3 of the platforms will show improvements, though it should help the mmap/unmap usage generally, as Michal Hocko mentioned: : I believe that there are non-synthetic worklaods which would benefit from : a larger batch. E.g. large in memory databases which do large mmaps : during startups from multiple threads. [1] https://lore.kernel.org/lkml/20200305062138.GI5972@shao2-debian/ Link: http://lkml.kernel.org/r/1589611660-89854-4-git-send-email-feng.tang@intel.com Signed-off-by: Feng Tang Acked-by: Michal Hocko Cc: Matthew Wilcox (Oracle) Cc: Johannes Weiner Cc: Mel Gorman Cc: Kees Cook Cc: Andi Kleen Cc: Tim Chen Cc: Dave Hansen Cc: Huang Ying Reported-by: kernel test robot Reported-by: kernel test robot Signed-off-by: Feng Tang --- include/linux/mm.h | 2 ++ include/linux/mman.h | 4 ++++ kernel/sysctl.c | 2 +- mm/mm_init.c | 18 ++++++++++++++---- mm/util.c | 12 ++++++++++++ 5 files changed, 33 insertions(+), 5 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index e6ff54a..d00facb 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -206,6 +206,8 @@ int overcommit_ratio_handler(struct ctl_table *, int, void *, size_t *, loff_t *); int overcommit_kbytes_handler(struct ctl_table *, int, void *, size_t *, loff_t *); +int overcommit_policy_handler(struct ctl_table *, int, void *, size_t *, + loff_t *); #define nth_page(page,n) pfn_to_page(page_to_pfn((page)) + (n)) diff --git a/include/linux/mman.h b/include/linux/mman.h index 4b08e9c..91c93c1 100644 --- a/include/linux/mman.h +++ b/include/linux/mman.h @@ -57,8 +57,12 @@ extern struct percpu_counter vm_committed_as; #ifdef CONFIG_SMP extern s32 vm_committed_as_batch; +extern void mm_compute_batch(void); #else #define vm_committed_as_batch 0 +static inline void mm_compute_batch(void) +{ +} #endif unsigned long vm_memory_committed(void); diff --git a/kernel/sysctl.c b/kernel/sysctl.c index 40180cd..10dcc06 100644 --- a/kernel/sysctl.c +++ b/kernel/sysctl.c @@ -2650,7 +2650,7 @@ static struct ctl_table vm_table[] = { .data = &sysctl_overcommit_memory, .maxlen = sizeof(sysctl_overcommit_memory), .mode = 0644, - .proc_handler = proc_dointvec_minmax, + .proc_handler = overcommit_policy_handler, .extra1 = SYSCTL_ZERO, .extra2 = &two, }, diff --git a/mm/mm_init.c b/mm/mm_init.c index 435e5f7..c5a6fb1 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "internal.h" #ifdef CONFIG_DEBUG_MEMORY_INIT @@ -144,14 +145,23 @@ EXPORT_SYMBOL_GPL(mm_kobj); #ifdef CONFIG_SMP s32 vm_committed_as_batch = 32; -static void __meminit mm_compute_batch(void) +void mm_compute_batch(void) { u64 memsized_batch; s32 nr = num_present_cpus(); s32 batch = max_t(s32, nr*2, 32); - - /* batch size set to 0.4% of (total memory/#cpus), or max int32 */ - memsized_batch = min_t(u64, (totalram_pages()/nr)/256, 0x7fffffff); + unsigned long ram_pages = totalram_pages(); + + /* + * For policy of OVERCOMMIT_NEVER, set batch size to 0.4% + * of (total memory/#cpus), and lift it to 25% for other + * policies to easy the possible lock contention for percpu_counter + * vm_committed_as, while the max limit is INT_MAX + */ + if (sysctl_overcommit_memory == OVERCOMMIT_NEVER) + memsized_batch = min_t(u64, ram_pages/nr/256, INT_MAX); + else + memsized_batch = min_t(u64, ram_pages/nr/4, INT_MAX); vm_committed_as_batch = max_t(s32, memsized_batch, batch); } diff --git a/mm/util.c b/mm/util.c index 1c9d097..52ed9c1 100644 --- a/mm/util.c +++ b/mm/util.c @@ -746,6 +746,18 @@ int overcommit_ratio_handler(struct ctl_table *table, int write, void *buffer, return ret; } +int overcommit_policy_handler(struct ctl_table *table, int write, void *buffer, + size_t *lenp, loff_t *ppos) +{ + int ret; + + ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos); + if (ret == 0 && write) + mm_compute_batch(); + + return ret; +} + int overcommit_kbytes_handler(struct ctl_table *table, int write, void *buffer, size_t *lenp, loff_t *ppos) {