From patchwork Tue Jun 25 03:13:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhiqiang Liu X-Patchwork-Id: 11014487 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A83C76 for ; Tue, 25 Jun 2019 03:14:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0DF128A89 for ; Tue, 25 Jun 2019 03:14:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CE97228AB1; Tue, 25 Jun 2019 03:14:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EEE1E28A89 for ; Tue, 25 Jun 2019 03:14:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727060AbfFYDOd (ORCPT ); Mon, 24 Jun 2019 23:14:33 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:19072 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726372AbfFYDOd (ORCPT ); Mon, 24 Jun 2019 23:14:33 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id 7C6E9EDA485226346D06; Tue, 25 Jun 2019 11:14:30 +0800 (CST) Received: from [127.0.0.1] (10.184.225.177) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.439.0; Tue, 25 Jun 2019 11:14:21 +0800 To: , , Kees Cook , Eric Dumazet CC: Andrew Morton , , , , , , , , , , , , "wangxiaogang (F)" , "Zhoukang (A)" , Mingfangsen , From: Zhiqiang Liu Subject: [PATCH next v2] softirq: enable MAX_SOFTIRQ_TIME tuning with sysctl, max_softirq_time_msecs Message-ID: <53770380-053e-70b6-f75e-a0e00bf35c30@huawei.com> Date: Tue, 25 Jun 2019 11:13:58 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 X-Originating-IP: [10.184.225.177] X-CFilter-Loop: Reflected Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Zhiqiang liu In __do_softirq func, MAX_SOFTIRQ_TIME was set to 2ms via experimentation by commit c10d73671 ("softirq: reduce latencies") in 2013, which was designed to reduce latencies for various network workloads. The key reason is that the maximum number of microseconds in one NAPI polling cycle in net_rx_action func was set to 2 jiffies, so different HZ settting will lead to different latencies. However, commit 7acf8a1e8 ("Replace 2 jiffies with sysctl netdev_budget_usecs to enable softirq tuning") adopts netdev_budget_usecs to tun maximum number of microseconds in one NAPI polling cycle. So the latencies of net_rx_action can be controlled by sysadmins to copy with hardware changes over time. Correspondingly, the MAX_SOFTIRQ_TIME should be able to be tunned by sysadmins, who knows best about hardware performance, for excepted tradeoff between latence and fairness. Here, we add sysctl variable max_softirq_time_msecs to replace MAX_SOFTIRQ_TIME with 2ms default value. Note: max_softirq_time_msecs will be coverted to jiffies, and any budget value will be rounded up to the next jiffies, which relates to CONFIG_HZ. The time accuracy of jiffies will result in a certain difference between the setting jiffies of max_softirq_time_msecs and the actual value, which is in one jiffies range. Signed-off-by: Zhiqiang liu --- Documentation/sysctl/kernel.txt | 17 +++++++++++++++++ kernel/softirq.c | 8 +++++--- kernel/sysctl.c | 9 +++++++++ 3 files changed, 31 insertions(+), 3 deletions(-) diff --git a/Documentation/sysctl/kernel.txt b/Documentation/sysctl/kernel.txt index f0c86fbb3b48..23b36393f150 100644 --- a/Documentation/sysctl/kernel.txt +++ b/Documentation/sysctl/kernel.txt @@ -44,6 +44,7 @@ show up in /proc/sys/kernel: - kexec_load_disabled - kptr_restrict - l2cr [ PPC only ] +- max_softirq_time_msecs - modprobe ==> Documentation/debugging-modules.txt - modules_disabled - msg_next_id [ sysv ipc ] @@ -445,6 +446,22 @@ This flag controls the L2 cache of G3 processor boards. If ============================================================== +max_softirq_time_msecs: + +Maximum number of milliseconds to break the loop of restarting softirq +processing for at most MAX_SOFTIRQ_RESTART times in __do_softirq(). +max_softirq_time_msecs will be coverted to jiffies, and any budget +value will be rounded up to the next jiffies, which relates to CONFIG_HZ. +The time accuracy of jiffies will result in a certain difference +between the setting jiffies of max_softirq_time_msecs and the actual +value, which is in one jiffies range. + +max_softirq_time_msecs is a non-negative integer value, and setting +negative value is meaningless and will return error. +Default: 2 + +============================================================== + modules_disabled: A toggle value indicating if modules are allowed to be loaded diff --git a/kernel/softirq.c b/kernel/softirq.c index a6b81c6b6bff..1e456db70093 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -199,7 +199,8 @@ EXPORT_SYMBOL(__local_bh_enable_ip); /* * We restart softirq processing for at most MAX_SOFTIRQ_RESTART times, - * but break the loop if need_resched() is set or after 2 ms. + * but break the loop if need_resched() is set or after + * max_softirq_time_msecs msecs. * The MAX_SOFTIRQ_TIME provides a nice upper bound in most cases, but in * certain cases, such as stop_machine(), jiffies may cease to * increment and so we need the MAX_SOFTIRQ_RESTART limit as @@ -210,7 +211,7 @@ EXPORT_SYMBOL(__local_bh_enable_ip); * we want to handle softirqs as soon as possible, but they * should not be able to lock up the box. */ -#define MAX_SOFTIRQ_TIME msecs_to_jiffies(2) +unsigned int __read_mostly max_softirq_time_msecs = 2; #define MAX_SOFTIRQ_RESTART 10 #ifdef CONFIG_TRACE_IRQFLAGS @@ -248,7 +249,8 @@ static inline void lockdep_softirq_end(bool in_hardirq) { } asmlinkage __visible void __softirq_entry __do_softirq(void) { - unsigned long end = jiffies + MAX_SOFTIRQ_TIME; + unsigned long end = jiffies + + msecs_to_jiffies(max_softirq_time_msecs); unsigned long old_flags = current->flags; int max_restart = MAX_SOFTIRQ_RESTART; struct softirq_action *h; diff --git a/kernel/sysctl.c b/kernel/sysctl.c index 1beca96fb625..96ff292ce7f6 100644 --- a/kernel/sysctl.c +++ b/kernel/sysctl.c @@ -118,6 +118,7 @@ extern unsigned int sysctl_nr_open_min, sysctl_nr_open_max; #ifndef CONFIG_MMU extern int sysctl_nr_trim_pages; #endif +extern unsigned int max_softirq_time_msecs; /* Constants used for minimum and maximum */ #ifdef CONFIG_LOCKUP_DETECTOR @@ -1276,6 +1277,14 @@ static struct ctl_table kern_table[] = { .extra2 = &one, }, #endif + { + .procname = "max_softirq_time_msecs", + .data = &max_softirq_time_msecs, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = &zero, + }, { } };