From patchwork Wed Sep 26 13:56:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10616037 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D63C415E8 for ; Wed, 26 Sep 2018 14:01:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D08C52B0C5 for ; Wed, 26 Sep 2018 14:01:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CE3F52B0BC; Wed, 26 Sep 2018 14:01:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 055942B0C0 for ; Wed, 26 Sep 2018 14:01:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=t4xceN1NhKsElvfp4M8UzpioCLgvtDbcqgLAcfLNoxQ=; b=WEwc8AjEezBAUrtYaH/tQG6Q3C UUje1lIMo4Tlk6wuTiAcI/HAHgMQ4xNLKmSTXrXywFGgCEhYT2sl0EAwu2xh7v34vH3sxP5FeXsEx 5KgMgMnySikT2d/FKzHIJGBIl9Yo7vxs+oH+rnb/ulWbc9ydWJLMjaTFk7jw9UJgeEvQVmz/OhxPB anqQAf7j4ZKmj1b8hM+hWnjE3hlAvllwqu9zTPAeUroVtTQDLlK+901zLYc5Mbxiv6XTCa8cy7xgc GuKDZtAKW3HWpBz025UIxkBNvxyYLeVkFgZLV+/z7Ca5anEcg6OW3FW1wLCITAIX8R4C4VMbbqtvu 6j04MxVw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AN5-0001R7-SM; Wed, 26 Sep 2018 14:00:56 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AJM-00076U-K3 for linux-arm-kernel@lists.infradead.org; Wed, 26 Sep 2018 13:57:41 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3E27315AD; Wed, 26 Sep 2018 06:56:32 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B25813F5B3; Wed, 26 Sep 2018 06:56:30 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 1/7] arm64: Add static check for pt_regs alignment Date: Wed, 26 Sep 2018 14:56:18 +0100 Message-Id: <1537970184-44348-2-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> References: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180926_065704_712098_2D9CA2BD X-CRM114-Status: GOOD ( 11.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, james.morse@arm.com, Dave.Martin@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The pt_regs structure's size is required to be a multiple of 16 to maintain the stack alignment upon kernel entries. Add a static check to ensure this is the case. Signed-off-by: Julien Thierry Acked-by: Ard Biesheuvel --- arch/arm64/include/asm/ptrace.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 177b851..cc17fad 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -229,6 +229,8 @@ static inline u64 regs_get_register(struct pt_regs *regs, unsigned int offset) WARN_ON(offset & 7); + BUILD_BUG_ON((sizeof (struct pt_regs) % 16 != 0)); + offset >>= 3; switch (offset) { case 0 ... 30: From patchwork Wed Sep 26 13:56:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10616039 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 06D48913 for ; Wed, 26 Sep 2018 14:01:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E9C0F2B045 for ; Wed, 26 Sep 2018 14:01:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B1F202B137; Wed, 26 Sep 2018 14:01:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 980612B0ED for ; Wed, 26 Sep 2018 14:01:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=nIMw9aiq1D7rxjDgWUSlzw6YFdptRC7RHzmo+lHdtKU=; b=HzyZ9uHiB9NzenW/N0zJOXRZNT tVeq2UYSYWrK/n4TAG5nySs/KJYMiQyduOX9/Ab9fEYshDypvGFoN90PNoE4EyXyBLED8yHQbtPmB +ungPZ/DyOEQY2wnN2cDqSkBbYbps+MPJzmHZEDHJfJFfAZ9ssx1Zd3thoGSJ9yPji/CaSS3DvdAb 8iOerWGguMb/v/YV902HhPIeJSeEIQlfI+o34lx1Es6WHcKVjcONexXB+oY7g6Sxt5X8HrY7379Q+ G6LZVHICKTSKrxUSOIaAC+DqxGZP0RQEEwngA0FINwTF9n/o3KmZzAEt4G542yEAMxQjxb4mSBzqV DtJSU2Wg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5ANu-0001un-H7; Wed, 26 Sep 2018 14:01:46 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AJM-00076X-LF for linux-arm-kernel@lists.infradead.org; Wed, 26 Sep 2018 13:57:42 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0084C15BE; Wed, 26 Sep 2018 06:56:34 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7D8D73F5B3; Wed, 26 Sep 2018 06:56:32 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 2/7] arm64: sdei: Always use sdei stack for sdei events Date: Wed, 26 Sep 2018 14:56:19 +0100 Message-Id: <1537970184-44348-3-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> References: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180926_065704_739396_594BED5B X-CRM114-Status: GOOD ( 16.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, james.morse@arm.com, Dave.Martin@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP SDEI events can occur at any point, including when the stack pointer is not aligned. SP could be modified to respect alignment 16-byte alignement, but there is a need to deal with code using SP as scratch register. Always reserve the SDEI stacks to handle its events. Since stack allocation relies on VMAPed stacks, lets make SDEI depend on VMAP_STACK. Signed-off-by: Julien Thierry --- arch/arm64/include/asm/sdei.h | 2 -- arch/arm64/kernel/entry.S | 2 -- arch/arm64/kernel/sdei.c | 23 ++++++++--------------- drivers/firmware/Kconfig | 1 + 4 files changed, 9 insertions(+), 19 deletions(-) diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h index ffe47d7..149dffe 100644 --- a/arch/arm64/include/asm/sdei.h +++ b/arch/arm64/include/asm/sdei.h @@ -46,8 +46,6 @@ asmlinkage unsigned long __sdei_handler(struct pt_regs *regs, static inline bool on_sdei_stack(unsigned long sp, struct stack_info *info) { - if (!IS_ENABLED(CONFIG_VMAP_STACK)) - return false; if (!IS_ENABLED(CONFIG_ARM_SDE_INTERFACE)) return false; if (in_nmi()) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 09dbea22..fc5842b 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -1187,7 +1187,6 @@ ENTRY(__sdei_asm_handler) mov x19, x1 -#ifdef CONFIG_VMAP_STACK /* * entry.S may have been using sp as a scratch register, find whether * this is a normal or critical event and switch to the appropriate @@ -1201,7 +1200,6 @@ ENTRY(__sdei_asm_handler) 2: mov x6, #SDEI_STACK_SIZE add x5, x5, x6 mov sp, x5 -#endif /* * We may have interrupted userspace, or a guest, or exit-from or diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 5ba4465..2e2fb0b 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -20,21 +20,16 @@ unsigned long sdei_exit_mode; /* - * VMAP'd stacks checking for stack overflow on exception using sp as a scratch - * register, meaning SDEI has to switch to its own stack. We need two stacks as - * a critical event may interrupt a normal event that has just taken a - * synchronous exception, and is using sp as scratch register. For a critical - * event interrupting a normal event, we can't reliably tell if we were on the - * sdei stack. + * SDEI events could occur at a time where SP_EL1 is misaligned or invalid, a + * solution is to give SDEI its own stack. We need two stacks as a critical + * event may interrupt a normal event that has just taken a synchronous + * exception, and is using sp as scratch register. For a critical event + * interrupting a normal event, we can't reliably tell if we were on the sdei + * stack. * For now, we allocate stacks when the driver is probed. */ -DECLARE_PER_CPU(unsigned long *, sdei_stack_normal_ptr); -DECLARE_PER_CPU(unsigned long *, sdei_stack_critical_ptr); - -#ifdef CONFIG_VMAP_STACK DEFINE_PER_CPU(unsigned long *, sdei_stack_normal_ptr); DEFINE_PER_CPU(unsigned long *, sdei_stack_critical_ptr); -#endif static void _free_sdei_stack(unsigned long * __percpu *ptr, int cpu) { @@ -150,10 +145,8 @@ unsigned long sdei_arch_get_entry_point(int conduit) return 0; } - if (IS_ENABLED(CONFIG_VMAP_STACK)) { - if (init_sdei_stacks()) - return 0; - } + if (init_sdei_stacks()) + return 0; sdei_exit_mode = (conduit == CONDUIT_HVC) ? SDEI_EXIT_HVC : SDEI_EXIT_SMC; diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index 6e83880..c63df31 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -85,6 +85,7 @@ config ARM_SCPI_POWER_DOMAIN config ARM_SDE_INTERFACE bool "ARM Software Delegated Exception Interface (SDEI)" depends on ARM64 + depends on VMAP_STACK help The Software Delegated Exception Interface (SDEI) is an ARM standard for registering callbacks from the platform firmware From patchwork Wed Sep 26 13:56:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10616033 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B740A15A6 for ; Wed, 26 Sep 2018 13:59:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B1FFA2AF61 for ; Wed, 26 Sep 2018 13:59:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A5CF22AFAC; Wed, 26 Sep 2018 13:59:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A4C632AF61 for ; Wed, 26 Sep 2018 13:58:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2YaosrEQp+2W+bQV/Pr5scobQ+jg/F56vegQiKaxLj4=; b=svxBtBD15rvBnKm28YoTIckMjk UTnEX6dOKW3ikiJlZv6xQ0WkbvReOEh1XZ8ELqTpk5u+FKDSnx9oqFDkVPKAPZV93qN03p6Hdb4ak 0T8qsEK2psEMwLytTr8l1KB76sxuDGFHGgVr4JlNYPqRtcUpuqe8XdwH6pP4GMYAylMMPKkfYItxZ 9jLlSah2emg/5XMu7eZRoTBJD5zHtDcfDu6k/VzqYpzfwYeTMM5hqgmQ0VWqz+hj+Mr1+alZ9GLdb cU/e+T/BAppGzRx+PiQeH+lfWjjcj09ugIHT0edfj2tkybt4Wyj4+hn/1NFQxzrQU8di7syEysRQh PL8FPXJw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AL1-0007qP-54; Wed, 26 Sep 2018 13:58:47 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AJM-00077N-A8 for linux-arm-kernel@lists.infradead.org; Wed, 26 Sep 2018 13:57:35 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BFE0E15BF; Wed, 26 Sep 2018 06:56:35 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3FDAB3F5B3; Wed, 26 Sep 2018 06:56:34 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 3/7] arm64: Align stack when taking exception from EL1 Date: Wed, 26 Sep 2018 14:56:20 +0100 Message-Id: <1537970184-44348-4-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> References: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180926_065704_388848_7709B788 X-CRM114-Status: GOOD ( 11.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, james.morse@arm.com, Dave.Martin@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Arm64 SP needs to be aligned to 16 bytes before being used as base address for loads and stores. When taking some valid exceptions from EL1 (e.g. irq, dbg, data abort), there is no guarantee that SP_EL1 was aligned when taking the exception. Pad the stack on EL1 entries when misaligned. Signed-off-by: Julien Thierry --- arch/arm64/include/asm/assembler.h | 9 +++++++++ arch/arm64/kernel/entry.S | 32 ++++++++++++++++++++++++++++++-- 2 files changed, 39 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 0bcc98d..a0a5415 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -701,4 +701,13 @@ .Lyield_out_\@ : .endm +/* + * Echange content of register xt with sp. + */ + .macro xchg_sp xt + add sp, sp, \xt // sp' = sp + xt + sub \xt, sp, \xt // xt' = sp' - xt = sp + xt - xt = sp + sub sp, sp, \xt // sp'' = sp' - xt' = sp + xt - sp = xt + .endm + #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index fc5842b..8fb66e4 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -59,6 +59,19 @@ .endr .endm + .macro force_stack_align + xchg_sp x0 + str x1, [x0] // store x1 far away from S_SP + + // aligned_sp[S_SP] = old_sp + bic x1, x0, #0xf // align down to 16-byte + str x0, [x1, #S_SP] + + ldr x1, [x0] + bic x0, x0, #0xf // x0 = aligned_sp + xchg_sp x0 + .endm + /* * Bad Abort numbers *----------------- @@ -158,6 +171,10 @@ alternative_cb_end .if \regsize == 32 mov w0, w0 // zero upper 32 bits of x0 .endif + .if \el != 0 + force_stack_align + .endif + stp x0, x1, [sp, #16 * 0] stp x2, x3, [sp, #16 * 1] stp x4, x5, [sp, #16 * 2] @@ -184,7 +201,8 @@ alternative_cb_end apply_ssbd 1, x22, x23 .else - add x21, sp, #S_FRAME_SIZE + ldr x21, [sp, #S_SP] + add x21, x21, #S_FRAME_SIZE // adjust stored sp get_thread_info tsk /* Save the task's original addr_limit and set USER_DS */ ldr x20, [tsk, #TSK_TI_ADDR_LIMIT] @@ -327,7 +345,6 @@ alternative_else_nop_endif msr elr_el1, x21 // set up the return data msr spsr_el1, x22 - ldp x0, x1, [sp, #16 * 0] ldp x2, x3, [sp, #16 * 1] ldp x4, x5, [sp, #16 * 2] ldp x6, x7, [sp, #16 * 3] @@ -343,7 +360,18 @@ alternative_else_nop_endif ldp x26, x27, [sp, #16 * 13] ldp x28, x29, [sp, #16 * 14] ldr lr, [sp, #S_LR] + + /* Restore x0, x1 and sp */ + .if \el != 0 + mov x1, sp + ldr x0, [sp, #S_SP] + mov sp, x0 + ldp x0, x1, [x1, #16 * 0] + .else + ldp x0, x1, [sp, #16 * 0] add sp, sp, #S_FRAME_SIZE // restore sp + .endif + /* * ARCH_HAS_MEMBARRIER_SYNC_CORE rely on eret context synchronization * when returning from IPI handler, and when returning to user-space. From patchwork Wed Sep 26 13:56:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10616035 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1BF69913 for ; Wed, 26 Sep 2018 14:00:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 161D52B01D for ; Wed, 26 Sep 2018 14:00:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 13E212AFEE; Wed, 26 Sep 2018 14:00:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1EBF52AFBF for ; Wed, 26 Sep 2018 14:00:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=XebD3Dkh6K4/QuUxeCAFUApMe2ULhSsXXa2D5zTZMC8=; b=sDIVHGCiDFQEF0cXpDwLS805IE jBkhIgSumrHgPNlAFXFr3kA2mwCRGP2fwjGFa6XYV362Qi7WPZh2ofPaCAqypUWl4Fa5eqjB8jvFM GkZaGX/jQfCxENQDiG7u2z3nSe0AWEvz1y1QApGyGPLOU6YQI2+VOpgBjLk7OU/3AMAQQTO6310gT lSWitFN/Aa977DXiZrEvGKj58MCJgN7dbAntzzAuJ4PUrR+9eLcDCvs8TDS8zMKUVniZSfhkBW1o2 aBvmFm3X3Q7iILrsbtzNoErzY/JIOxrSjDwZgGUV+8ykprYqEWie5l9dNGHDVvDuAVJxJyVBZoAUa ZNLF8rGg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AMH-0008FW-7y; Wed, 26 Sep 2018 14:00:05 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AJM-00077q-Dw for linux-arm-kernel@lists.infradead.org; Wed, 26 Sep 2018 13:57:38 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8B35C1688; Wed, 26 Sep 2018 06:56:37 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0AD323F5B3; Wed, 26 Sep 2018 06:56:35 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 4/7] arm64: Add fast-path for stack alignment Date: Wed, 26 Sep 2018 14:56:21 +0100 Message-Id: <1537970184-44348-5-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> References: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180926_065704_545743_13B8DA98 X-CRM114-Status: GOOD ( 11.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, james.morse@arm.com, Dave.Martin@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Avoiding stack alignment for already aligned stack can give a small performance boost. Branch out of the aligning code when the stack is known to be aligned. Signed-off-by: Julien Thierry Acked-by: Will Deacon --- arch/arm64/kernel/entry.S | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 8fb66e4..bd8d52c 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -61,6 +61,10 @@ .macro force_stack_align xchg_sp x0 + + tst x0, #0xf + b.eq 0f + str x1, [x0] // store x1 far away from S_SP // aligned_sp[S_SP] = old_sp @@ -69,6 +73,11 @@ ldr x1, [x0] bic x0, x0, #0xf // x0 = aligned_sp + b 1f + +0: + str x0, [x0, #S_SP] +1: xchg_sp x0 .endm From patchwork Wed Sep 26 13:56:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10616045 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A3F9913 for ; Wed, 26 Sep 2018 14:12:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 338362B059 for ; Wed, 26 Sep 2018 14:12:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 274D12B060; Wed, 26 Sep 2018 14:12:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7D53A2B059 for ; Wed, 26 Sep 2018 14:12:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=dPa45UGpY8bDCXPTC9j1A8XttCDYqZ+DXyfcrauipyU=; b=GF3NrGaeEDvihxlBoWvzq5VBx9 5a0Ljn5GKrTt6zqFWUV3Vb+rCLEcTralRWi9Yc1ZwWYpajsEzYCBvOdx/Vo1PbOu1iIJP7JF8EPAd sgwV/PDCYqNIL0TYk974DyMSQIMEsyfJyhE+G8u7u15c/kFN1anZ3hvJ8Gy6fQ6FsI4ZA89eDbeMs ZmObdRegv06x5nOxuEwUkkQNHvdmiS7XLwzP0I+Iv6Y3v7t3e6YfDtSbQqBtmfkIshmElLGfw4Is5 cJpNkdLoI7pB/JWK/ewz7npO2GN3D5+wQGKO7TRLOVL8oj57vZPkE+7ttFMjoqaSgQpl7JlLRvrHa lY8r5pew==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AXp-0007jQ-1S; Wed, 26 Sep 2018 14:12:01 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AK9-0007W4-9J for linux-arm-kernel@lists.infradead.org; Wed, 26 Sep 2018 13:58:10 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7A8B51684; Wed, 26 Sep 2018 06:56:39 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id CAB963F5B3; Wed, 26 Sep 2018 06:56:37 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 5/7] arm64: Do not apply BP hardening for hyp entries from EL2 Date: Wed, 26 Sep 2018 14:56:22 +0100 Message-Id: <1537970184-44348-6-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> References: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180926_065753_384622_9A801E35 X-CRM114-Status: GOOD ( 12.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, kvmarm@lists.cs.columbia.edu, james.morse@arm.com, Dave.Martin@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP When an EL2 entry of __kvm_hyp_vector is taken, it means an entry from a lower EL was previously taken to exit the guest. Taking that lower EL entry already applied BP hardening if it was needed, so there is no need to do it again. Only apply BP hardening for exception coming from lower EL. Signed-off-by: Julien Thierry Cc: kvmarm@lists.cs.columbia.edu --- arch/arm64/kernel/cpu_errata.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) -- 1.9.1 diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index dec1089..9db5ecc 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -103,10 +103,16 @@ static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start, void *dst = lm_alias(__bp_harden_hyp_vecs_start + slot * SZ_2K); int i; - for (i = 0; i < SZ_2K; i += 0x80) + /* + * Only overwrite hyp entries for exceptions from lower EL. + * Exception vection vector is 2K bytes, first 1K bytes concern + * exception from EL2 (EL2t, EL2h), last 1K concert lower exception + * levels (ELx-64bits, ELx-32bits). + */ + for (i = SZ_1K; i < SZ_2K; i += 0x80) memcpy(dst + i, hyp_vecs_start, hyp_vecs_end - hyp_vecs_start); - __flush_icache_range((uintptr_t)dst, (uintptr_t)dst + SZ_2K); + __flush_icache_range((uintptr_t)dst + SZ_1K, (uintptr_t)dst + SZ_2K); } static void __install_bp_hardening_cb(bp_hardening_cb_t fn, From patchwork Wed Sep 26 13:56:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10616041 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 671E2913 for ; Wed, 26 Sep 2018 14:02:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6340B2B107 for ; Wed, 26 Sep 2018 14:02:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6135E2B10B; Wed, 26 Sep 2018 14:02:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A34632B11D for ; Wed, 26 Sep 2018 14:02:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=m0o3sy6TjYmodRr6m+w5ilk73TpCyM2PEQ2tAq2/eSQ=; b=KaetgI9ERnbX+1OeWgIo2JL3Jk 6CQhBxnhzmOKPYi6DFdi/zBp5fX+fReI7RVgjMIqgXMrxo97dt6RBiQPzycNBYuksJgmaIJ1oqfPw JUd4LWSz7ZbtWG9NJbzXUFVp5LrL5BypDyq01dycNDGQK2xz0X+RXBfB9dbZQ+VOcQRzNIlng98oW tGznNvb77K0bmWXPuiA7oxlu3OvlkCUxnOUcElphgmbsD/Blzss4gPzQMDBkvTpfvKKBKh4aCJp/Z tPgrf9l4kl/ieFWDdtqSFw/nX1ZgA1EY+myZCzZSp1/2+HaUTdu5qxvI7Y4mgeVrOEpyK1He0xmL1 CvfJhkXA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AOe-0002TB-1G; Wed, 26 Sep 2018 14:02:32 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AK9-0007W3-9K for linux-arm-kernel@lists.infradead.org; Wed, 26 Sep 2018 13:58:10 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6A116168F; Wed, 26 Sep 2018 06:56:41 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BA40A3F5B3; Wed, 26 Sep 2018 06:56:39 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 6/7] arm64: Do not apply vector harderning for hyp entries from EL2 Date: Wed, 26 Sep 2018 14:56:23 +0100 Message-Id: <1537970184-44348-7-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> References: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180926_065753_374843_85C42AE1 X-CRM114-Status: GOOD ( 11.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, kvmarm@lists.cs.columbia.edu, james.morse@arm.com, Dave.Martin@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP When an EL2 entry of __kvm_hyp_vector is taken, it means an entry from a lower EL was previously taken to exit the guest. Taking that lower EL entry already applied vector hardening if needed, so there is no need to do it again. Only apply vector hardening for exception coming from lower EL. Signed-off-by: Julien Thierry Cc: kvmarm@lists.cs.columbia.edu --- arch/arm64/kvm/hyp/hyp-entry.S | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) -- 1.9.1 diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 24b4fba..da31386 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -257,7 +257,15 @@ ENTRY(__kvm_hyp_vector) ENDPROC(__kvm_hyp_vector) #ifdef CONFIG_KVM_INDIRECT_VECTORS -.macro hyp_ventry +.macro hyp_el2_ventry + .align 7 +1: b __kvm_hyp_vector + (1b - 0b) + .rept 31 + nop + .endr +.endm + +.macro hyp_el1_ventry .align 7 1: .rept 27 nop @@ -290,8 +298,11 @@ alternative_cb_end .macro generate_vectors 0: - .rept 16 - hyp_ventry + .rept 8 + hyp_el2_ventry + .endr + .rept 8 + hyp_el1_ventry .endr .org 0b + SZ_2K // Safety measure .endm From patchwork Wed Sep 26 13:56:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10616043 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7AAEE913 for ; Wed, 26 Sep 2018 14:04:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7402728B26 for ; Wed, 26 Sep 2018 14:04:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 67A7A29BB2; Wed, 26 Sep 2018 14:04:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B3A3328B26 for ; Wed, 26 Sep 2018 14:03:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=9qElvygRK57PinZ8QhGfCJprkyd3XFS141yugK9jqG0=; b=dvgi7YbD4ytYtAI47jCm4sIyXO BW0kpAUgaVBqVMMqQd4XuS+wmttUBZDoYpAfi6bfZ0N3Yk3y5g4daQLpBFss0SZ/QsGtntt49DhPt oWiwxmgqPlm+vjs54qCLdpIF5s1gbgNLPOinK4iDKAmMQAeBD+g2VHSSoH8V8U6uwY86rTjfKoOeY vtuVX+JNfwWTTu4Co9f40nYyf2tl/IhrII8AKXfhOArJGr0L4q0dk+VpEgua3MelnmrGAl1YjyBJD jljFg7B5rbwt7Dm9H+8EUNn+4/6cG34e1sKODZyLUFUnnUPSM0DLy2xpED9Q1if5bw47W9uK2PqvJ oNDiHiQw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5APs-0002v1-FF; Wed, 26 Sep 2018 14:03:48 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AKL-0007XX-2n for linux-arm-kernel@lists.infradead.org; Wed, 26 Sep 2018 13:58:24 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5985216A3; Wed, 26 Sep 2018 06:56:43 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A9B403F5B3; Wed, 26 Sep 2018 06:56:41 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 7/7] arm64: kvm: Align stack for exception coming from EL2 Date: Wed, 26 Sep 2018 14:56:24 +0100 Message-Id: <1537970184-44348-8-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> References: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180926_065805_193571_B57F60A9 X-CRM114-Status: GOOD ( 12.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, kvmarm@lists.cs.columbia.edu, james.morse@arm.com, Dave.Martin@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP SP needs to be 16-bytes aligned before accessing memory through it. When handling exceptions from EL2, there is no guarantee that SP is already aligned. Ensure SP is aligned upon entries from EL2. Signed-off-by: Julien Thierry Cc: kvmarm@lists.cs.columbia.edu --- arch/arm64/kvm/hyp/hyp-entry.S | 63 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 52 insertions(+), 11 deletions(-) -- 1.9.1 diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index da31386..f611072 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -184,6 +184,10 @@ el2_error: ccmp x0, x1, #4, ne b.ne __hyp_panic mov x0, #(1 << ARM_EXIT_WITH_SERROR_BIT) + + /* Restore the stack */ + ldr x1, [sp] + mov sp, x1 eret ENTRY(__hyp_do_panic) @@ -221,29 +225,66 @@ ENDPROC(\label) .align 11 -.macro valid_vect target +/* + * Aligns the stack and stores the old sp value. This is what the stack + * looks like after this code: + * + * +--------+ + * | | + * |--------|--> old_sp (upon entry) + * | x1 |--> (only used for local save/restore) + * |--------|--> old_sp - 8 + * | |--> padding + * |--------|--> aligned_sp + 8 + * | old_sp | + * |--------|--> sp = aligned_sp + * | | + * +--------+ + */ +.macro align_sp_el2 + xchg_sp x0 + str x1, [x0, #-8] // save x1 + mov x1, x0 // x1 = old_sp + sub x0, x0, #0x10 // save space for x1 + old_sp + bic x0, x0, #0xf // align down to 16-bytes + xchg_sp x0 + str x1, [sp] // save old_sp + ldr x1, [x1, #-8] +.endm + +.macro valid_vect target, el=1 .align 7 + .if \el == 2 + align_sp_el2 + .endif + stp x0, x1, [sp, #-16]! b \target .endm -.macro invalid_vect target +.macro invalid_vect target, el=1 .align 7 + .if \el == 2 + align_sp_el2 + .endif + b \target + .if \el == 1 ldp x0, x1, [sp], #16 b \target + .endif .endm ENTRY(__kvm_hyp_vector) - invalid_vect el2t_sync_invalid // Synchronous EL2t - invalid_vect el2t_irq_invalid // IRQ EL2t - invalid_vect el2t_fiq_invalid // FIQ EL2t - invalid_vect el2t_error_invalid // Error EL2t - - invalid_vect el2h_sync_invalid // Synchronous EL2h - invalid_vect el2h_irq_invalid // IRQ EL2h - invalid_vect el2h_fiq_invalid // FIQ EL2h - valid_vect el2_error // Error EL2h + invalid_vect el2t_sync_invalid, 2 // Synchronous EL2t + invalid_vect el2t_irq_invalid, 2 // IRQ EL2t + invalid_vect el2t_fiq_invalid, 2 // FIQ EL2t + invalid_vect el2t_error_invalid, 2 // Error EL2t + + invalid_vect el2h_sync_invalid, 2 // Synchronous EL2h + invalid_vect el2h_irq_invalid, 2 // IRQ EL2h + invalid_vect el2h_fiq_invalid, 2 // FIQ EL2h + valid_vect el2_error, 2 // Error EL2h valid_vect el1_sync // Synchronous 64-bit EL1 valid_vect el1_irq // IRQ 64-bit EL1