From patchwork Wed Feb 9 15:35:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 12740393 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CB6D7C433EF for ; Wed, 9 Feb 2022 15:39:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=M+vAk/iXnqhK/hZAAdG+p10aASBkuMfB7yLXQdF9zEs=; b=N7VSpsnUg0zN3K G9zRl47g3V8lYy0LGX0gUDjJ2iSLKlB7OpHZuvpzVHSh8kJM/BxQAdETK96nK2q/18vGv0oo2pR7r 8fMBgX2xV5Y/U2zCqNa5EjaSVIDagkIFuBySOIA5tlO6psvB0pPvWCE+Y8NmAU9oD0lNYYOL/WJbi iSUwSGOY03asVVHuw826Z0XbFKaWxRcnnYCWaphqRbtpJE1VXw9ebfxRQdQBdNOa5Q2UwBD1qORD4 lu1mKL8iMq176WwLqU8RruDrQ0mS7/4rQrSELXTTv1XL7pAGQj59GcDDXpqrXMxUnybvQ1xnz+4jc AQWPlymn3vBs/4Mp/Rlw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nHp2x-000clI-Um; Wed, 09 Feb 2022 15:38:20 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nHp0U-000bkv-Vp for linux-arm-kernel@lists.infradead.org; Wed, 09 Feb 2022 15:35:52 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 38F1E1FB; Wed, 9 Feb 2022 07:35:43 -0800 (PST) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A71F43F73B; Wed, 9 Feb 2022 07:35:40 -0800 (PST) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, frederic@kernel.org, james.morse@arm.com, joey.gouly@arm.com, juri.lelli@redhat.com, linux-kernel@vger.kernel.org, luto@kernel.org, mark.rutland@arm.com, mingo@redhat.com, peterz@infradead.org, tglx@linutronix.de, valentin.schneider@arm.com, will@kernel.org Subject: [PATCH v3 0/7] arm64 / sched/preempt: support PREEMPT_DYNAMIC with static keys Date: Wed, 9 Feb 2022 15:35:28 +0000 Message-Id: <20220209153535.818830-1-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220209_073547_110207_2BB40758 X-CRM114-Status: GOOD ( 12.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This series enables PREEMPT_DYNAMIC on arm64. To do so, it adds a new mechanism allowing the preemption functions to be enabled/disabled using static keys rather than static calls, with architectures selecting whether they use static calls or static keys. With non-inline static calls, each function call results in a call to the (out-of-line) trampoline which either tail-calls its associated callee or performs an early return. The key idea is that where we're only enabling/disabling a single callee, we can inline this trampoline into the start of the callee, using a static key to decide whether to return early, and leaving the remaining codegen to the compiler. The overhead should be similar to (and likely lower than) using a static call trampoline. Since most codegen is up to the compiler, we sidestep a number of implementation pain-points (e.g. things like CFI should "just work" as well as they do for any other functions). The bulk of the diffstat for kernel/sched/core.c is shuffling the PREEMPT_DYNAMIC code later in the file, and the actual additions are fairly trivial. I've given this very light build+boot testing so far. Since v1 [1]: * Rework Kconfig text to be clearer * Rework arm64 entry code * Clarify commit messages. Since v2 [2]: * Add missing includes * Always provide prototype for preempt_schedule() * Always provide prototype for preempt_schedule_notrace() * Fix __cond_resched() to default to disabled * Fix might_resched() to default to disabled * Clarify example in commit message [1] https://lore.kernel.org/r/20211109172408.49641-1-mark.rutland@arm.com/ [2] https://lore.kernel.org/r/20220204150557.434610-1-mark.rutland@arm.com/ Mark Rutland (7): sched/preempt: move PREEMPT_DYNAMIC logic later sched/preempt: refactor sched_dynamic_update() sched/preempt: simplify irqentry_exit_cond_resched() callers sched/preempt: decouple HAVE_PREEMPT_DYNAMIC from GENERIC_ENTRY sched/preempt: add PREEMPT_DYNAMIC using static keys arm64: entry: centralize premeption decision arm64: support PREEMPT_DYNAMIC arch/Kconfig | 37 +++- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/preempt.h | 19 +- arch/arm64/kernel/entry-common.c | 28 ++- arch/x86/Kconfig | 2 +- arch/x86/include/asm/preempt.h | 10 +- include/linux/entry-common.h | 15 +- include/linux/kernel.h | 7 +- include/linux/sched.h | 10 +- kernel/entry/common.c | 23 +- kernel/sched/core.c | 347 ++++++++++++++++++------------- 11 files changed, 327 insertions(+), 172 deletions(-) Acked-by: Frederic Weisbecker Acked-by: Ard Biesheuvel