From patchwork Wed Sep 26 13:56:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10616033 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B740A15A6 for ; Wed, 26 Sep 2018 13:59:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B1FFA2AF61 for ; Wed, 26 Sep 2018 13:59:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A5CF22AFAC; Wed, 26 Sep 2018 13:59:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A4C632AF61 for ; Wed, 26 Sep 2018 13:58:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2YaosrEQp+2W+bQV/Pr5scobQ+jg/F56vegQiKaxLj4=; b=svxBtBD15rvBnKm28YoTIckMjk UTnEX6dOKW3ikiJlZv6xQ0WkbvReOEh1XZ8ELqTpk5u+FKDSnx9oqFDkVPKAPZV93qN03p6Hdb4ak 0T8qsEK2psEMwLytTr8l1KB76sxuDGFHGgVr4JlNYPqRtcUpuqe8XdwH6pP4GMYAylMMPKkfYItxZ 9jLlSah2emg/5XMu7eZRoTBJD5zHtDcfDu6k/VzqYpzfwYeTMM5hqgmQ0VWqz+hj+Mr1+alZ9GLdb cU/e+T/BAppGzRx+PiQeH+lfWjjcj09ugIHT0edfj2tkybt4Wyj4+hn/1NFQxzrQU8di7syEysRQh PL8FPXJw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AL1-0007qP-54; Wed, 26 Sep 2018 13:58:47 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g5AJM-00077N-A8 for linux-arm-kernel@lists.infradead.org; Wed, 26 Sep 2018 13:57:35 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BFE0E15BF; Wed, 26 Sep 2018 06:56:35 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3FDAB3F5B3; Wed, 26 Sep 2018 06:56:34 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 3/7] arm64: Align stack when taking exception from EL1 Date: Wed, 26 Sep 2018 14:56:20 +0100 Message-Id: <1537970184-44348-4-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> References: <1537970184-44348-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180926_065704_388848_7709B788 X-CRM114-Status: GOOD ( 11.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, james.morse@arm.com, Dave.Martin@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Arm64 SP needs to be aligned to 16 bytes before being used as base address for loads and stores. When taking some valid exceptions from EL1 (e.g. irq, dbg, data abort), there is no guarantee that SP_EL1 was aligned when taking the exception. Pad the stack on EL1 entries when misaligned. Signed-off-by: Julien Thierry --- arch/arm64/include/asm/assembler.h | 9 +++++++++ arch/arm64/kernel/entry.S | 32 ++++++++++++++++++++++++++++++-- 2 files changed, 39 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 0bcc98d..a0a5415 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -701,4 +701,13 @@ .Lyield_out_\@ : .endm +/* + * Echange content of register xt with sp. + */ + .macro xchg_sp xt + add sp, sp, \xt // sp' = sp + xt + sub \xt, sp, \xt // xt' = sp' - xt = sp + xt - xt = sp + sub sp, sp, \xt // sp'' = sp' - xt' = sp + xt - sp = xt + .endm + #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index fc5842b..8fb66e4 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -59,6 +59,19 @@ .endr .endm + .macro force_stack_align + xchg_sp x0 + str x1, [x0] // store x1 far away from S_SP + + // aligned_sp[S_SP] = old_sp + bic x1, x0, #0xf // align down to 16-byte + str x0, [x1, #S_SP] + + ldr x1, [x0] + bic x0, x0, #0xf // x0 = aligned_sp + xchg_sp x0 + .endm + /* * Bad Abort numbers *----------------- @@ -158,6 +171,10 @@ alternative_cb_end .if \regsize == 32 mov w0, w0 // zero upper 32 bits of x0 .endif + .if \el != 0 + force_stack_align + .endif + stp x0, x1, [sp, #16 * 0] stp x2, x3, [sp, #16 * 1] stp x4, x5, [sp, #16 * 2] @@ -184,7 +201,8 @@ alternative_cb_end apply_ssbd 1, x22, x23 .else - add x21, sp, #S_FRAME_SIZE + ldr x21, [sp, #S_SP] + add x21, x21, #S_FRAME_SIZE // adjust stored sp get_thread_info tsk /* Save the task's original addr_limit and set USER_DS */ ldr x20, [tsk, #TSK_TI_ADDR_LIMIT] @@ -327,7 +345,6 @@ alternative_else_nop_endif msr elr_el1, x21 // set up the return data msr spsr_el1, x22 - ldp x0, x1, [sp, #16 * 0] ldp x2, x3, [sp, #16 * 1] ldp x4, x5, [sp, #16 * 2] ldp x6, x7, [sp, #16 * 3] @@ -343,7 +360,18 @@ alternative_else_nop_endif ldp x26, x27, [sp, #16 * 13] ldp x28, x29, [sp, #16 * 14] ldr lr, [sp, #S_LR] + + /* Restore x0, x1 and sp */ + .if \el != 0 + mov x1, sp + ldr x0, [sp, #S_SP] + mov sp, x0 + ldp x0, x1, [x1, #16 * 0] + .else + ldp x0, x1, [sp, #16 * 0] add sp, sp, #S_FRAME_SIZE // restore sp + .endif + /* * ARCH_HAS_MEMBARRIER_SYNC_CORE rely on eret context synchronization * when returning from IPI handler, and when returning to user-space.