From patchwork Mon Feb 14 16:52:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 12745904 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6F97BC433F5 for ; Mon, 14 Feb 2022 16:53:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=iimTmC2oRKLS3FQ8spu8RDm64TBC8xQuuS2aC+igVGM=; b=xwpteATgI9/2wE 8smJ3URxzCQtxONtiLHylh+pLsG+DYZejlVd6F/nidw99pzHRUR6FOHid0bHxqFZlESz6C/ZB4i1T zsAyV99xArtHUlZHjIYxuObbIi5DmENpYjy4kwm3rFUProatoDo+LQzZ4YxEbHSqs2xKnqvCfuzZj EO0+I4U7Sb0geZ5qxnLUgW1v/TNzLyyOJH9T3oLljpg3zO+1LfOE5Mc45G3jiSA5XTC/p4zntNMI7 0oov1hfOEAB6JMz9DSXREsnuVZ68yHwqCYRVugH9mBMQ+glo0noAl7H53+6BJV47ERYRWcXK3E3G0 2KhJDS7E8HtmUQiZVzvA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nJeaY-00GAbL-Gq; Mon, 14 Feb 2022 16:52:35 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nJeaQ-00GAYo-4g for linux-arm-kernel@lists.infradead.org; Mon, 14 Feb 2022 16:52:28 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4E35213D5; Mon, 14 Feb 2022 08:52:25 -0800 (PST) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4C1C03F70D; Mon, 14 Feb 2022 08:52:23 -0800 (PST) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, frederic@kernel.org, james.morse@arm.com, joey.gouly@arm.com, juri.lelli@redhat.com, linux-kernel@vger.kernel.org, luto@kernel.org, mark.rutland@arm.com, mingo@redhat.com, peterz@infradead.org, tglx@linutronix.de, valentin.schneider@arm.com, will@kernel.org Subject: [PATCH v4 0/7] arm64 / sched/preempt: support PREEMPT_DYNAMIC with static keys Date: Mon, 14 Feb 2022 16:52:09 +0000 Message-Id: <20220214165216.2231574-1-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220214_085226_340099_0A06187B X-CRM114-Status: GOOD ( 13.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This series enables PREEMPT_DYNAMIC on arm64. To do so, it adds a new mechanism allowing the preemption functions to be enabled/disabled using static keys rather than static calls, with architectures selecting whether they use static calls or static keys. With non-inline static calls, each function call results in a call to the (out-of-line) trampoline which either tail-calls its associated callee or performs an early return. The key idea is that where we're only enabling/disabling a single callee, we can inline this trampoline into the start of the callee, using a static key to decide whether to return early, and leaving the remaining codegen to the compiler. The overhead should be similar to (and likely lower than) using a static call trampoline. Since most codegen is up to the compiler, we sidestep a number of implementation pain-points (e.g. things like CFI should "just work" as well as they do for any other functions). The bulk of the diffstat for kernel/sched/core.c is shuffling the PREEMPT_DYNAMIC code later in the file, and the actual additions are fairly trivial. I've given this very light build+boot testing so far. Since v1 [1]: * Rework Kconfig text to be clearer * Rework arm64 entry code * Clarify commit messages. Since v2 [2]: * Add missing includes * Always provide prototype for preempt_schedule() * Always provide prototype for preempt_schedule_notrace() * Fix __cond_resched() to default to disabled * Fix might_resched() to default to disabled * Clarify example in commit message Since v3 [3]: * Add acks (including those missing from v3) * Fix Kconfig dependencies for jump labels * Don't default-enable PREEMPT_DYNAMIC when using static keys * Clarify generated assembly examples [1] https://lore.kernel.org/r/20211109172408.49641-1-mark.rutland@arm.com/ [2] https://lore.kernel.org/r/20220204150557.434610-1-mark.rutland@arm.com/ [3] https://lore.kernel.org/r/20220209153535.818830-1-mark.rutland@arm.com/ Thanks, Mark. Mark Rutland (7): sched/preempt: move PREEMPT_DYNAMIC logic later sched/preempt: refactor sched_dynamic_update() sched/preempt: simplify irqentry_exit_cond_resched() callers sched/preempt: decouple HAVE_PREEMPT_DYNAMIC from GENERIC_ENTRY sched/preempt: add PREEMPT_DYNAMIC using static keys arm64: entry: centralize preemption decision arm64: support PREEMPT_DYNAMIC arch/Kconfig | 37 +++- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/preempt.h | 19 +- arch/arm64/kernel/entry-common.c | 28 ++- arch/x86/Kconfig | 2 +- arch/x86/include/asm/preempt.h | 10 +- include/linux/entry-common.h | 15 +- include/linux/kernel.h | 7 +- include/linux/sched.h | 10 +- kernel/Kconfig.preempt | 3 +- kernel/entry/common.c | 23 +- kernel/sched/core.c | 347 ++++++++++++++++++------------- 12 files changed, 329 insertions(+), 173 deletions(-) Acked-by: Frederic Weisbecker