From patchwork Fri Nov 13 14:02:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 11903667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0956BC388F7 for ; Fri, 13 Nov 2020 14:27:08 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ADF3020715 for ; Fri, 13 Nov 2020 14:27:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="slH3woGp"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="jv2EEflp"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="9fpm5+mi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ADF3020715 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linutronix.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:Subject:To:From:Date: Message-Id:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Owner; bh=L/DjqG4fou2BMcegYRu2mydQ+aeJaQsTK+MSuZIwARI=; b=slH3woGpKqhLrB87j+KIZhEPg Cs1L0QlRBss3CKi70hD3zsPhr9VGLKhyBqAzjgrdnC4wqxlrjyL7oPzGFahEmxGwRyC34gH5ADQGj is/U97XkRywDjZiQO324VRQSVMuBoSu5B3HZIJSFpuVmzV8J9dJqBNKNzBqk7VVCP2sG6uvfnxrQg wueirnymAUZKicGhtyb9Pn3owYm0c0C19Alxys0a37uVZ1ss1S9NQqg2P67jsRq+UPmjk0KWC0F/O K/cRZqQvfHlDH2SEgJIrHJPuutSMYjjt2aPUdPWajY9cGbUz3kr0S+he6urcCQ0f54i0M1fQsYkHn Wqcj2j2vw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kda25-00079x-CP; Fri, 13 Nov 2020 14:26:33 +0000 Received: from galois.linutronix.de ([2a0a:51c0:0:12e:550::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kda05-0006KO-Cq; Fri, 13 Nov 2020 14:24:37 +0000 Message-Id: <20201113141733.974214480@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1605277467; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: references:references; bh=0UTeqwKc93bYeYTKZ9eNkJ81wNDHGHkvSAl8j8+Yj5Q=; b=jv2EEflphEE3CebcE72q1RNivrgP3cY6CABEy6b2ZgZLZDSvbL+oeMi5wEZugM4xGCuqu7 iA1kifAro0LVNSxg7SiHdTE4/VJI8sHguJCBvTJoX3AfamxiI8P7XO1BHBKk/yTWSS/Lhn SwwaIxFQ2VxUKsJMvA4DG6XeplBDsVoiQFOT/ceQWIaU/pKJr7zagt96VnJrFHNQxNvihF kAEReD5i/Fx180ZHUtX/TWNbSqTV/8OF/HEfMjOnDHgs4pz85Q5Rg1CD63euT6nGQbFfsi EN8ORThOKeZm4WBytizPIltx9HExbUBbfnT3JCYNTd8gjAileLVA9vl7kainFw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1605277467; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: references:references; bh=0UTeqwKc93bYeYTKZ9eNkJ81wNDHGHkvSAl8j8+Yj5Q=; b=9fpm5+miuJxrT+jze0aGitkbirAatyttDigNLtfzZs+AU5LOUy0Evsyi9vnWdaCR1KDuYI Vmu6Q7I4ArxJjlBQ== Date: Fri, 13 Nov 2020 15:02:18 +0100 From: Thomas Gleixner To: LKML Subject: [patch 11/19] softirq: Move related code into one section References: <20201113140207.499353218@linutronix.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201113_092429_840624_E1FF3CF1 X-CRM114-Status: GOOD ( 16.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Zyngier , Rich Felker , Catalin Marinas , Paul McKenney , Arnd Bergmann , linux-sh@vger.kernel.org, Peter Zijlstra , Richard Weinberger , Frederic Weisbecker , Valentin Schneider , Jeff Dike , Russell King , Yoshinori Sato , "James E.J. Bottomley" , linux-parisc@vger.kernel.org, Helge Deller , linux-um@lists.infradead.org, Will Deacon , Sebastian Andrzej Siewior , linux-arm-kernel@lists.infradead.org, Anton Ivanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To prepare for adding a RT aware variant of softirq serialization and processing move related code into one section so the necessary #ifdeffery is reduced to one. Signed-off-by: Thomas Gleixner Reviewed-by: Frederic Weisbecker --- kernel/softirq.c | 107 +++++++++++++++++++++++++++---------------------------- 1 file changed, 54 insertions(+), 53 deletions(-) --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -92,6 +92,13 @@ static bool ksoftirqd_running(unsigned l !__kthread_should_park(tsk); } +#ifdef CONFIG_TRACE_IRQFLAGS +DEFINE_PER_CPU(int, hardirqs_enabled); +DEFINE_PER_CPU(int, hardirq_context); +EXPORT_PER_CPU_SYMBOL_GPL(hardirqs_enabled); +EXPORT_PER_CPU_SYMBOL_GPL(hardirq_context); +#endif + /* * preempt_count and SOFTIRQ_OFFSET usage: * - preempt_count is changed by SOFTIRQ_OFFSET on entering or leaving @@ -102,17 +109,11 @@ static bool ksoftirqd_running(unsigned l * softirq and whether we just have bh disabled. */ +#ifdef CONFIG_TRACE_IRQFLAGS /* - * This one is for softirq.c-internal use, - * where hardirqs are disabled legitimately: + * This is for softirq.c-internal use, where hardirqs are disabled + * legitimately: */ -#ifdef CONFIG_TRACE_IRQFLAGS - -DEFINE_PER_CPU(int, hardirqs_enabled); -DEFINE_PER_CPU(int, hardirq_context); -EXPORT_PER_CPU_SYMBOL_GPL(hardirqs_enabled); -EXPORT_PER_CPU_SYMBOL_GPL(hardirq_context); - void __local_bh_disable_ip(unsigned long ip, unsigned int cnt) { unsigned long flags; @@ -203,6 +204,50 @@ void __local_bh_enable_ip(unsigned long } EXPORT_SYMBOL(__local_bh_enable_ip); +static inline void invoke_softirq(void) +{ + if (ksoftirqd_running(local_softirq_pending())) + return; + + if (!force_irqthreads) { +#ifdef CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK + /* + * We can safely execute softirq on the current stack if + * it is the irq stack, because it should be near empty + * at this stage. + */ + __do_softirq(); +#else + /* + * Otherwise, irq_exit() is called on the task stack that can + * be potentially deep already. So call softirq in its own stack + * to prevent from any overrun. + */ + do_softirq_own_stack(); +#endif + } else { + wakeup_softirqd(); + } +} + +asmlinkage __visible void do_softirq(void) +{ + __u32 pending; + unsigned long flags; + + if (in_interrupt()) + return; + + local_irq_save(flags); + + pending = local_softirq_pending(); + + if (pending && !ksoftirqd_running(pending)) + do_softirq_own_stack(); + + local_irq_restore(flags); +} + /* * We restart softirq processing for at most MAX_SOFTIRQ_RESTART times, * but break the loop if need_resched() is set or after 2 ms. @@ -327,24 +372,6 @@ asmlinkage __visible void __softirq_entr current_restore_flags(old_flags, PF_MEMALLOC); } -asmlinkage __visible void do_softirq(void) -{ - __u32 pending; - unsigned long flags; - - if (in_interrupt()) - return; - - local_irq_save(flags); - - pending = local_softirq_pending(); - - if (pending && !ksoftirqd_running(pending)) - do_softirq_own_stack(); - - local_irq_restore(flags); -} - /** * irq_enter_rcu - Enter an interrupt context with RCU watching */ @@ -371,32 +398,6 @@ void irq_enter(void) irq_enter_rcu(); } -static inline void invoke_softirq(void) -{ - if (ksoftirqd_running(local_softirq_pending())) - return; - - if (!force_irqthreads) { -#ifdef CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK - /* - * We can safely execute softirq on the current stack if - * it is the irq stack, because it should be near empty - * at this stage. - */ - __do_softirq(); -#else - /* - * Otherwise, irq_exit() is called on the task stack that can - * be potentially deep already. So call softirq in its own stack - * to prevent from any overrun. - */ - do_softirq_own_stack(); -#endif - } else { - wakeup_softirqd(); - } -} - static inline void tick_irq_exit(void) { #ifdef CONFIG_NO_HZ_COMMON