From patchwork Tue Dec 5 17:11:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10093497 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D66D960348 for ; Tue, 5 Dec 2017 17:34:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C1A3529140 for ; Tue, 5 Dec 2017 17:34:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B65932972C; Tue, 5 Dec 2017 17:34:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 28F1629140 for ; Tue, 5 Dec 2017 17:34:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=QtoEQZu8f2lxYn8LKzKmxrkCHJGkrjP2/ZbhoVtdg/Q=; b=bMIXARJov9EnuzXXyrIn7mPYUY DFc8B3EAvp4sWT/ooZkaC4l+rZFCxZYLasUYN0HTAfi9sPIl6e/MlAcH6aORcwrw2AJwkD59yagDb qZfuNmxUOE1/sFlhwqof7J065mjGFNV+uvar3MOrBnVLhW9DXw9Rpu69iomaIoIa7/uBeObVjMyLY Qmo2z4xHMCeErknrZd+8i45RfHJ0bkb0Vseqr/QQ/1UHL/MOaabTW52HfIIaJCag4ucWelQagKtAu JnSdZEwSlrpgui7/mf6I7yHgKBxELPcY4g0SdobfJYkSTx1cUYsRBIFsnw6ybTGBSVfoQf7MJ5DoC 64F+tq9w==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1eMH6y-0006oD-PF; Tue, 05 Dec 2017 17:34:28 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1eMH6w-0006nI-Gv for linux-arm-kernel@bombadil.infradead.org; Tue, 05 Dec 2017 17:34:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=JBaDJUY/aBUFpCEhsd0a5rur5UCWwAfixcIuIfZWQOQ=; b=Rn7irdkVR9Pf2q2H/pM18jVOJ EV7wntKLF+9US5GmGXEKxZVi+vZ3BLAH9DxyZ8omnK1kSR1uOa+SkNpMD1atz2MywNpVcYcth8qDy 6VabXU9hXvC5RBL2AiZvhD8muAE48LINQMqZKGxS3cNouMcvn9kxJP1lZt2T8dlwdIc8OfaJcuxdZ UtE/r8MMX4iTRaFjUH203fEkwmC3LqG/VRv39FSGHRdtIQT2r3BqqSJnjuM+bE9o6KE0rkTceB416 tMgR81EbNLExQ5EP/b2BgX6aPye6vbICnYc350OssFHKNkQdag9ydll33/Qm5A8zJBFG3dEy/XVGV IpF0hsygQ==; Received: from foss.arm.com ([217.140.101.70]) by merlin.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1eMGld-0000JK-Lg for linux-arm-kernel@lists.infradead.org; Tue, 05 Dec 2017 17:12:26 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 26E2D15BE; Tue, 5 Dec 2017 09:12:06 -0800 (PST) Received: from e112298-lin.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A1A503F246; Tue, 5 Dec 2017 09:12:04 -0800 (PST) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 2/6] arm64: alternative: Apply alternatives early in boot process Date: Tue, 5 Dec 2017 17:11:48 +0000 Message-Id: <1512493912-37478-3-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1512493912-37478-1-git-send-email-julien.thierry@arm.com> References: <1512493912-37478-1-git-send-email-julien.thierry@arm.com> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, Catalin Marinas , Will Deacon , james.morse@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Daniel Thompson Currently alternatives are applied very late in the boot process (and a long time after we enable scheduling). Some alternative sequences, such as those that alter the way CPU context is stored, must be applied much earlier in the boot sequence. Introduce apply_alternatives_early() to allow some alternatives to be applied immediately after we detect the CPU features of the boot CPU. Signed-off-by: Daniel Thompson Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon --- arch/arm64/include/asm/alternative.h | 1 + arch/arm64/kernel/alternative.c | 39 +++++++++++++++++++++++++++++++++--- arch/arm64/kernel/smp.c | 6 ++++++ 3 files changed, 43 insertions(+), 3 deletions(-) -- 1.9.1 diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h index 4a85c69..1fc1cdb 100644 --- a/arch/arm64/include/asm/alternative.h +++ b/arch/arm64/include/asm/alternative.h @@ -20,6 +20,7 @@ struct alt_instr { u8 alt_len; /* size of new instruction(s), <= orig_len */ }; +void __init apply_alternatives_early(void); void __init apply_alternatives_all(void); void apply_alternatives(void *start, size_t length); diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index 6dd0a3a3..78051d4 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -28,6 +28,18 @@ #include #include +/* + * early-apply features are detected using only the boot CPU and checked on + * secondary CPUs startup, even then, + * These early-apply features should only include features where we must + * patch the kernel very early in the boot process. + * + * Note that the cpufeature logic *must* be made aware of early-apply + * features to ensure they are reported as enabled without waiting + * for other CPUs to boot. + */ +#define EARLY_APPLY_FEATURE_MASK BIT(ARM64_HAS_SYSREG_GIC_CPUIF) + #define __ALT_PTR(a,f) ((void *)&(a)->f + (a)->f) #define ALT_ORIG_PTR(a) __ALT_PTR(a, orig_offset) #define ALT_REPL_PTR(a) __ALT_PTR(a, alt_offset) @@ -105,7 +117,8 @@ static u32 get_alt_insn(struct alt_instr *alt, __le32 *insnptr, __le32 *altinsnp return insn; } -static void __apply_alternatives(void *alt_region, bool use_linear_alias) +static void __apply_alternatives(void *alt_region, bool use_linear_alias, + unsigned long feature_mask) { struct alt_instr *alt; struct alt_region *region = alt_region; @@ -115,6 +128,9 @@ static void __apply_alternatives(void *alt_region, bool use_linear_alias) u32 insn; int i, nr_inst; + if ((BIT(alt->cpufeature) & feature_mask) == 0) + continue; + if (!cpus_have_cap(alt->cpufeature)) continue; @@ -138,6 +154,21 @@ static void __apply_alternatives(void *alt_region, bool use_linear_alias) } /* + * This is called very early in the boot process (directly after we run + * a feature detect on the boot CPU). No need to worry about other CPUs + * here. + */ +void apply_alternatives_early(void) +{ + struct alt_region region = { + .begin = (struct alt_instr *)__alt_instructions, + .end = (struct alt_instr *)__alt_instructions_end, + }; + + __apply_alternatives(®ion, true, EARLY_APPLY_FEATURE_MASK); +} + +/* * We might be patching the stop_machine state machine, so implement a * really simple polling protocol here. */ @@ -156,7 +187,9 @@ static int __apply_alternatives_multi_stop(void *unused) isb(); } else { BUG_ON(patched); - __apply_alternatives(®ion, true); + + __apply_alternatives(®ion, true, ~EARLY_APPLY_FEATURE_MASK); + /* Barriers provided by the cache flushing */ WRITE_ONCE(patched, 1); } @@ -177,5 +210,5 @@ void apply_alternatives(void *start, size_t length) .end = start + length, }; - __apply_alternatives(®ion, false); + __apply_alternatives(®ion, false, -1); } diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 551eb07..37361b5 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -453,6 +453,12 @@ void __init smp_prepare_boot_cpu(void) * cpuinfo_store_boot_cpu() above. */ update_cpu_errata_workarounds(); + /* + * We now know enough about the boot CPU to apply the + * alternatives that cannot wait until interrupt handling + * and/or scheduling is enabled. + */ + apply_alternatives_early(); } static u64 __init of_get_cpu_mpidr(struct device_node *dn)