From patchwork Fri May 29 01:06:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 11577399 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60C45912 for ; Fri, 29 May 2020 01:06:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 370A1206E2 for ; Fri, 29 May 2020 01:06:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 370A1206E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 06E3B800B6; Thu, 28 May 2020 21:06:21 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F37A180010; Thu, 28 May 2020 21:06:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4B41800B6; Thu, 28 May 2020 21:06:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id CE50280010 for ; Thu, 28 May 2020 21:06:20 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 9005F180AD807 for ; Fri, 29 May 2020 01:06:20 +0000 (UTC) X-FDA: 76867965720.09.hill06_3733e16222717 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id 79B7E180AD820 for ; Fri, 29 May 2020 01:06:20 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,feng.tang@intel.com,,RULES_HIT:30054:30064:30070,0,RBL:192.55.52.115:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: hill06_3733e16222717 X-Filterd-Recvd-Size: 2430 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Fri, 29 May 2020 01:06:19 +0000 (UTC) IronPort-SDR: iWtUpNkSq3jlYVV1IVMqXBKyqlfyienosNO12pjFYM9hmaz9fcuH8Zd9415dQOWc0ha+CpSPjk XsPyGeE1dh4g== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 May 2020 18:06:18 -0700 IronPort-SDR: p/zjETwGibrawIJEt9ooi+19bx4ElL9Nj2HegGeipzELO9cqLKCPwvIUNIprMkwiQDETIPG/1g /dY9oPKxpkrQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,446,1583222400"; d="scan'208";a="443187395" Received: from shbuild999.sh.intel.com ([10.239.146.107]) by orsmga005.jf.intel.com with ESMTP; 28 May 2020 18:06:15 -0700 From: Feng Tang To: Andrew Morton , Michal Hocko , Johannes Weiner , Matthew Wilcox , Mel Gorman , Kees Cook , Qian Cai , andi.kleen@intel.com, tim.c.chen@intel.com, dave.hansen@intel.com, ying.huang@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Feng Tang Subject: [PATCH v4 1/4] proc/meminfo: avoid open coded reading of vm_committed_as Date: Fri, 29 May 2020 09:06:07 +0800 Message-Id: <1590714370-67182-2-git-send-email-feng.tang@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1590714370-67182-1-git-send-email-feng.tang@intel.com> References: <1590714370-67182-1-git-send-email-feng.tang@intel.com> X-Rspamd-Queue-Id: 79B7E180AD820 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the existing vm_memory_committed() instead, which is also convenient for future change. Signed-off-by: Feng Tang Acked-by: Michal Hocko --- fs/proc/meminfo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index b030d8b..e3d14ee 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -41,7 +41,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) si_meminfo(&i); si_swapinfo(&i); - committed = percpu_counter_read_positive(&vm_committed_as); + committed = vm_memory_committed(); cached = global_node_page_state(NR_FILE_PAGES) - total_swapcache_pages() - i.bufferram; From patchwork Fri May 29 01:06:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 11577401 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3841B912 for ; Fri, 29 May 2020 01:06:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0D9CB206E2 for ; Fri, 29 May 2020 01:06:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D9CB206E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 219A7800B7; Thu, 28 May 2020 21:06:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1C83080010; Thu, 28 May 2020 21:06:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DE98800B7; Thu, 28 May 2020 21:06:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0225.hostedemail.com [216.40.44.225]) by kanga.kvack.org (Postfix) with ESMTP id EA80480010 for ; Thu, 28 May 2020 21:06:23 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B0DFA248F for ; Fri, 29 May 2020 01:06:23 +0000 (UTC) X-FDA: 76867965846.21.debt42_37b8a1965e41f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 9BECE180442C0 for ; Fri, 29 May 2020 01:06:23 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,feng.tang@intel.com,,RULES_HIT:30046:30054:30064:30075,0,RBL:192.55.52.115:@intel.com:.lbl8.mailshell.net-64.95.201.95 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1,LUA_SUMMARY:none X-HE-Tag: debt42_37b8a1965e41f X-Filterd-Recvd-Size: 2855 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Fri, 29 May 2020 01:06:22 +0000 (UTC) IronPort-SDR: 20RkHJ7gYymi7pqsRYo2a6bs/fJV6h1W55OdxLWWFpt/HCn3TH5oY0VL4dB1s9cgTeqoMNfnup zEt4kOqIk3Xg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 May 2020 18:06:22 -0700 IronPort-SDR: XQoQpnrS+4xFfOoDQX7PM+NvMtEzY0KYWFvbVJVP6f3UZ1aedMbPSk0G7zOAjoXi/aB9lLwgtq o6qbvV0VJD8w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,446,1583222400"; d="scan'208";a="443187402" Received: from shbuild999.sh.intel.com ([10.239.146.107]) by orsmga005.jf.intel.com with ESMTP; 28 May 2020 18:06:19 -0700 From: Feng Tang To: Andrew Morton , Michal Hocko , Johannes Weiner , Matthew Wilcox , Mel Gorman , Kees Cook , Qian Cai , andi.kleen@intel.com, tim.c.chen@intel.com, dave.hansen@intel.com, ying.huang@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Feng Tang Subject: [PATCH v4 2/4] mm/util.c: make vm_memory_committed() more accurate Date: Fri, 29 May 2020 09:06:08 +0800 Message-Id: <1590714370-67182-3-git-send-email-feng.tang@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1590714370-67182-1-git-send-email-feng.tang@intel.com> References: <1590714370-67182-1-git-send-email-feng.tang@intel.com> X-Rspamd-Queue-Id: 9BECE180442C0 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: percpu_counter_sum_positive() will provide more accurate info. As with percpu_counter_read_positive(), in worst case the deviation could be 'batch * nr_cpus', which is totalram_pages/256 for now, and will be more when the batch gets enlarged. Its time cost is about 800 nanoseconds on a 2C/4T platform and 2~3 microseconds on a 2S/36C/72T server in normal case, and in worst case where vm_committed_as's spinlock is under severe contention, it costs 30~40 microseconds for the 2S/36C/72T sever, which should be fine for its only two users: /proc/meminfo and HyperV balloon driver's status trace per second. Signed-off-by: Feng Tang Acked-by: Michal Hocko --- mm/util.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/util.c b/mm/util.c index 9b3be03..3c7a08c 100644 --- a/mm/util.c +++ b/mm/util.c @@ -790,7 +790,7 @@ struct percpu_counter vm_committed_as ____cacheline_aligned_in_smp; */ unsigned long vm_memory_committed(void) { - return percpu_counter_read_positive(&vm_committed_as); + return percpu_counter_sum_positive(&vm_committed_as); } EXPORT_SYMBOL_GPL(vm_memory_committed); From patchwork Fri May 29 01:06:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 11577403 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B820912 for ; Fri, 29 May 2020 01:06:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4812B206E2 for ; Fri, 29 May 2020 01:06:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4812B206E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6AEF8800B8; Thu, 28 May 2020 21:06:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 65D8280010; Thu, 28 May 2020 21:06:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 573D9800B8; Thu, 28 May 2020 21:06:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0019.hostedemail.com [216.40.44.19]) by kanga.kvack.org (Postfix) with ESMTP id 404D680010 for ; Thu, 28 May 2020 21:06:28 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id E4F9C8245571 for ; Fri, 29 May 2020 01:06:27 +0000 (UTC) X-FDA: 76867966014.19.pull53_384649ce53126 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id C62EA1ACC27 for ; Fri, 29 May 2020 01:06:27 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,feng.tang@intel.com,,RULES_HIT:30054:30064:30070,0,RBL:192.55.52.115:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: pull53_384649ce53126 X-Filterd-Recvd-Size: 2971 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Fri, 29 May 2020 01:06:26 +0000 (UTC) IronPort-SDR: PKqChP5RqBPLmz6B6psOZ+PqVFoS1jMKviI2m2jRKov43ybCOoQeZFN2WGPA5a8kiux9TcSCnA dLoLCxoRm2+g== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 May 2020 18:06:26 -0700 IronPort-SDR: t9036NUhHUGwWveLlQIds2EuWb519n1VXvhfA4Y1WdfrsMoWv1eWHhzwMauuq1L5xjXAGhainG 8jkg7yyatc3A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,446,1583222400"; d="scan'208";a="443187421" Received: from shbuild999.sh.intel.com ([10.239.146.107]) by orsmga005.jf.intel.com with ESMTP; 28 May 2020 18:06:22 -0700 From: Feng Tang To: Andrew Morton , Michal Hocko , Johannes Weiner , Matthew Wilcox , Mel Gorman , Kees Cook , Qian Cai , andi.kleen@intel.com, tim.c.chen@intel.com, dave.hansen@intel.com, ying.huang@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Feng Tang , Konstantin Khlebnikov Subject: [PATCH v4 3/4] mm/util.c: remove the VM_WARN_ONCE for vm_committed_as underflow check Date: Fri, 29 May 2020 09:06:09 +0800 Message-Id: <1590714370-67182-4-git-send-email-feng.tang@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1590714370-67182-1-git-send-email-feng.tang@intel.com> References: <1590714370-67182-1-git-send-email-feng.tang@intel.com> X-Rspamd-Queue-Id: C62EA1ACC27 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As is explained by Michal Hocko: : Looking at the history, this has been added by 82f71ae4a2b8 : ("mm: catch memory commitment underflow") to have a safety check : for issues which have been fixed. There doesn't seem to be any bug : reports mentioning this splat since then so it is likely just : spending cycles for a hot path (yes many people run with DEBUG_VM) : without a strong reason. Signed-off-by: Feng Tang Cc: Konstantin Khlebnikov Cc: Qian Cai Cc: Michal Hocko Cc: Andi Kleen --- mm/util.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/mm/util.c b/mm/util.c index 3c7a08c..fe63271 100644 --- a/mm/util.c +++ b/mm/util.c @@ -814,14 +814,6 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin) { long allowed; - /* - * A transient decrease in the value is unlikely, so no need - * READ_ONCE() for vm_committed_as.count. - */ - VM_WARN_ONCE(data_race(percpu_counter_read(&vm_committed_as) < - -(s64)vm_committed_as_batch * num_online_cpus()), - "memory commitment underflow"); - vm_acct_memory(pages); /* From patchwork Fri May 29 01:06:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 11577405 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B3E72913 for ; Fri, 29 May 2020 01:06:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 80B42206E2 for ; Fri, 29 May 2020 01:06:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 80B42206E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A2E69800B9; Thu, 28 May 2020 21:06:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9DF3B80010; Thu, 28 May 2020 21:06:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A671800B9; Thu, 28 May 2020 21:06:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id 7157580010 for ; Thu, 28 May 2020 21:06:31 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 3230D180AD820 for ; Fri, 29 May 2020 01:06:31 +0000 (UTC) X-FDA: 76867966182.07.fight62_38bf73df65e2c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id F33A61803FFCF for ; Fri, 29 May 2020 01:06:30 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,feng.tang@intel.com,,RULES_HIT:30034:30054:30064:30070,0,RBL:192.55.52.115:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: fight62_38bf73df65e2c X-Filterd-Recvd-Size: 7459 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Fri, 29 May 2020 01:06:29 +0000 (UTC) IronPort-SDR: xV9r6ZO/uw8I91h0SNaX7h7ev92edyn8HGPwHBpQYsVI/nOArIUTjHx8p0CTJ023vsqi6wgV1H o7bKTqcejpbw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 May 2020 18:06:29 -0700 IronPort-SDR: RxXsvuJCMcDX8IEzpOUSMFasbltQTH5HRxJ9mhwLY3p5ixHUaP3fzc7TY7mYV8oqtworpBpMFZ kW1YSkdx7yqw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,446,1583222400"; d="scan'208";a="443187429" Received: from shbuild999.sh.intel.com ([10.239.146.107]) by orsmga005.jf.intel.com with ESMTP; 28 May 2020 18:06:26 -0700 From: Feng Tang To: Andrew Morton , Michal Hocko , Johannes Weiner , Matthew Wilcox , Mel Gorman , Kees Cook , Qian Cai , andi.kleen@intel.com, tim.c.chen@intel.com, dave.hansen@intel.com, ying.huang@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Feng Tang Subject: [PATCH v4 4/4] mm: adjust vm_committed_as_batch according to vm overcommit policy Date: Fri, 29 May 2020 09:06:10 +0800 Message-Id: <1590714370-67182-5-git-send-email-feng.tang@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1590714370-67182-1-git-send-email-feng.tang@intel.com> References: <1590714370-67182-1-git-send-email-feng.tang@intel.com> X-Rspamd-Queue-Id: F33A61803FFCF X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When checking a performance change for will-it-scale scalability mmap test [1], we found very high lock contention for spinlock of percpu counter 'vm_committed_as': 94.14% 0.35% [kernel.kallsyms] [k] _raw_spin_lock_irqsave 48.21% _raw_spin_lock_irqsave;percpu_counter_add_batch;__vm_enough_memory;mmap_region;do_mmap; 45.91% _raw_spin_lock_irqsave;percpu_counter_add_batch;__do_munmap; Actually this heavy lock contention is not always necessary. The 'vm_committed_as' needs to be very precise when the strict OVERCOMMIT_NEVER policy is set, which requires a rather small batch number for the percpu counter. So keep 'batch' number unchanged for strict OVERCOMMIT_NEVER policy, and lift it to 64X for OVERCOMMIT_ALWAYS and OVERCOMMIT_GUESS policies. Also add a sysctl handler to adjust it when the policy is reconfigured. Benchmark with the same testcase in [1] shows 53% improvement on a 8C/16T desktop, and 2097%(20X) on a 4S/72C/144T server. We tested with test platforms in 0day (server, desktop and laptop), and 80%+ platforms shows improvements with that test. And whether it shows improvements depends on if the test mmap size is bigger than the batch number computed. And if the lift is 16X, 1/3 of the platforms will show improvements, though it should help the mmap/unmap usage generally, as Michal Hocko mentioned: : I believe that there are non-synthetic worklaods which would benefit from : a larger batch. E.g. large in memory databases which do large mmaps : during startups from multiple threads. [1] https://lore.kernel.org/lkml/20200305062138.GI5972@shao2-debian/ Link: http://lkml.kernel.org/r/1589611660-89854-4-git-send-email-feng.tang@intel.com Signed-off-by: Feng Tang Cc: Michal Hocko Cc: Matthew Wilcox (Oracle) Cc: Johannes Weiner Cc: Mel Gorman Cc: Kees Cook Cc: Andi Kleen Cc: Tim Chen Cc: Dave Hansen Cc: Huang Ying Signed-off-by: Andrew Morton Acked-by: Michal Hocko --- include/linux/mm.h | 2 ++ include/linux/mman.h | 4 ++++ kernel/sysctl.c | 2 +- mm/mm_init.c | 18 ++++++++++++++---- mm/util.c | 12 ++++++++++++ 5 files changed, 33 insertions(+), 5 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 573947c..c2efea6 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -206,6 +206,8 @@ int overcommit_ratio_handler(struct ctl_table *, int, void *, size_t *, loff_t *); int overcommit_kbytes_handler(struct ctl_table *, int, void *, size_t *, loff_t *); +int overcommit_policy_handler(struct ctl_table *, int, void *, size_t *, + loff_t *); #define nth_page(page,n) pfn_to_page(page_to_pfn((page)) + (n)) diff --git a/include/linux/mman.h b/include/linux/mman.h index 4b08e9c..91c93c1 100644 --- a/include/linux/mman.h +++ b/include/linux/mman.h @@ -57,8 +57,12 @@ extern struct percpu_counter vm_committed_as; #ifdef CONFIG_SMP extern s32 vm_committed_as_batch; +extern void mm_compute_batch(void); #else #define vm_committed_as_batch 0 +static inline void mm_compute_batch(void) +{ +} #endif unsigned long vm_memory_committed(void); diff --git a/kernel/sysctl.c b/kernel/sysctl.c index db1ce7a..9456c86 100644 --- a/kernel/sysctl.c +++ b/kernel/sysctl.c @@ -2650,7 +2650,7 @@ static struct ctl_table vm_table[] = { .data = &sysctl_overcommit_memory, .maxlen = sizeof(sysctl_overcommit_memory), .mode = 0644, - .proc_handler = proc_dointvec_minmax, + .proc_handler = overcommit_policy_handler, .extra1 = SYSCTL_ZERO, .extra2 = &two, }, diff --git a/mm/mm_init.c b/mm/mm_init.c index 435e5f7..c5a6fb1 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "internal.h" #ifdef CONFIG_DEBUG_MEMORY_INIT @@ -144,14 +145,23 @@ EXPORT_SYMBOL_GPL(mm_kobj); #ifdef CONFIG_SMP s32 vm_committed_as_batch = 32; -static void __meminit mm_compute_batch(void) +void mm_compute_batch(void) { u64 memsized_batch; s32 nr = num_present_cpus(); s32 batch = max_t(s32, nr*2, 32); - - /* batch size set to 0.4% of (total memory/#cpus), or max int32 */ - memsized_batch = min_t(u64, (totalram_pages()/nr)/256, 0x7fffffff); + unsigned long ram_pages = totalram_pages(); + + /* + * For policy of OVERCOMMIT_NEVER, set batch size to 0.4% + * of (total memory/#cpus), and lift it to 25% for other + * policies to easy the possible lock contention for percpu_counter + * vm_committed_as, while the max limit is INT_MAX + */ + if (sysctl_overcommit_memory == OVERCOMMIT_NEVER) + memsized_batch = min_t(u64, ram_pages/nr/256, INT_MAX); + else + memsized_batch = min_t(u64, ram_pages/nr/4, INT_MAX); vm_committed_as_batch = max_t(s32, memsized_batch, batch); } diff --git a/mm/util.c b/mm/util.c index fe63271..580d268 100644 --- a/mm/util.c +++ b/mm/util.c @@ -746,6 +746,18 @@ int overcommit_ratio_handler(struct ctl_table *table, int write, void *buffer, return ret; } +int overcommit_policy_handler(struct ctl_table *table, int write, void *buffer, + size_t *lenp, loff_t *ppos) +{ + int ret; + + ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos); + if (ret == 0 && write) + mm_compute_batch(); + + return ret; +} + int overcommit_kbytes_handler(struct ctl_table *table, int write, void *buffer, size_t *lenp, loff_t *ppos) {