From patchwork Sat Apr 18 04:36:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11496279 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 70AE815AB for ; Sat, 18 Apr 2020 04:36:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 55A06221F7 for ; Sat, 18 Apr 2020 04:36:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725873AbgDREgM (ORCPT ); Sat, 18 Apr 2020 00:36:12 -0400 Received: from mga18.intel.com ([134.134.136.126]:25561 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725440AbgDREgL (ORCPT ); Sat, 18 Apr 2020 00:36:11 -0400 IronPort-SDR: BWGCN3ScX6U8Ir3A2I4tyFU3BN9YUSN1R9wZ/mXRduHb2LyT/3fvdqw5Na3Ta8FSzRxaNDQFcq 9yPwZFRc+rBQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Apr 2020 21:36:10 -0700 IronPort-SDR: V8TqS/88gUU76Byj0DpVrMtMxY49AknoyGFtn+c0wwTCCS9G7jNKGFu1TWsZn6+1M026hAcBl8 /kogwlWdsu+g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,397,1580803200"; d="scan'208";a="289459977" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.202]) by fmsmga002.fm.intel.com with ESMTP; 17 Apr 2020 21:36:10 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: Nathaniel McCallum , Cedric Xing , Jethro Beekman , Andy Lutomirski , linux-sgx@vger.kernel.org Subject: [PATCH for_v29 v3 1/2] x86/sgx: vdso: Make __vdso_sgx_enter_enclave() callable from C code Date: Fri, 17 Apr 2020 21:36:08 -0700 Message-Id: <20200418043609.29406-2-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: <20200418043609.29406-1-sean.j.christopherson@intel.com> References: <20200418043609.29406-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Make __vdso_sgx_enter_enclave() callable from C by preserving %rbx and taking @leaf in %rcx instead of %rax. Being able to invoke the vDSO from C reduces the overhead of runtimes that are tightly coupled with their enclaves, e.g. that can rely on the enclave to save and restore non-volatile registers, as the runtime doesn't need an assembly wrapper to preserve non-volatile registers and/or shuffle stack arguments. Note, both %rcx and %rbx are consumed by EENTER/ERESUME, i.e. consuming them doesn't violate the primary tenet of __vdso_sgx_enter_enclave() that "thou shalt not restrict how information is exchanged between an enclave and its host process". Define a typedef for the __vdso_sgx_enter_enclave() prototype and move the entire function comment from the assembly code to the uAPI header, dropping the kernel doc hack along the way. Update the selftest to pass EENTER to the vDSO wrapper via %ecx instead of hardcoding it to make the wrapper compatible with the new vDSO calling convention. Suggested-by: Nathaniel McCallum Cc: Cedric Xing Cc: Jethro Beekman Cc: Andy Lutomirski Cc: linux-sgx@vger.kernel.org Signed-off-by: Sean Christopherson --- arch/x86/entry/vdso/vsgx_enter_enclave.S | 64 ++---------------------- arch/x86/include/uapi/asm/sgx.h | 61 ++++++++++++++++++++++ tools/testing/selftests/sgx/call.S | 1 - tools/testing/selftests/sgx/defines.h | 1 + tools/testing/selftests/sgx/main.c | 2 +- tools/testing/selftests/sgx/main.h | 2 +- 6 files changed, 68 insertions(+), 63 deletions(-) diff --git a/arch/x86/entry/vdso/vsgx_enter_enclave.S b/arch/x86/entry/vdso/vsgx_enter_enclave.S index 34cee2b0ef09..be7e467e1efb 100644 --- a/arch/x86/entry/vdso/vsgx_enter_enclave.S +++ b/arch/x86/entry/vdso/vsgx_enter_enclave.S @@ -15,66 +15,6 @@ .code64 .section .text, "ax" -/** - * __vdso_sgx_enter_enclave() - Enter an SGX enclave - * @leaf: ENCLU leaf, must be EENTER or ERESUME - * @tcs: TCS, must be non-NULL - * @e: Optional struct sgx_enclave_exception instance - * @handler: Optional enclave exit handler - * - * **Important!** __vdso_sgx_enter_enclave() is **NOT** compliant with the - * x86-64 ABI, i.e. cannot be called from standard C code. - * - * Input ABI: - * @leaf %eax - * @tcs 8(%rsp) - * @e 0x10(%rsp) - * @handler 0x18(%rsp) - * - * Output ABI: - * @ret %eax - * - * All general purpose registers except RAX, RBX and RCX are passed as-is to - * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are - * loaded with @leaf, asynchronous exit pointer, and @tcs respectively. - * - * RBP and the stack are used to anchor __vdso_sgx_enter_enclave() to the - * pre-enclave state, e.g. to retrieve @e and @handler after an enclave exit. - * All other registers are available for use by the enclave and its runtime, - * e.g. an enclave can push additional data onto the stack (and modify RSP) to - * pass information to the optional exit handler (see below). - * - * Most exceptions reported on ENCLU, including those that occur within the - * enclave, are fixed up and reported synchronously instead of being delivered - * via a standard signal. Debug Exceptions (#DB) and Breakpoints (#BP) are - * never fixed up and are always delivered via standard signals. On synchrously - * reported exceptions, -EFAULT is returned and details about the exception are - * recorded in @e, the optional sgx_enclave_exception struct. - - * If an exit handler is provided, the handler will be invoked on synchronous - * exits from the enclave and for all synchronously reported exceptions. In - * latter case, @e is filled prior to invoking the handler. - * - * The exit handler's return value is interpreted as follows: - * >0: continue, restart __vdso_sgx_enter_enclave() with @ret as @leaf - * 0: success, return @ret to the caller - * <0: error, return @ret to the caller - * - * The exit handler may transfer control, e.g. via longjmp() or C++ exception, - * without returning to __vdso_sgx_enter_enclave(). - * - * Return: - * 0 on success, - * -EINVAL if ENCLU leaf is not allowed, - * -EFAULT if an exception occurs on ENCLU or within the enclave - * -errno for all other negative values returned by the userspace exit handler - */ -#ifdef SGX_KERNEL_DOC -/* C-style function prototype to coerce kernel-doc into parsing the comment. */ -int __vdso_sgx_enter_enclave(int leaf, void *tcs, - struct sgx_enclave_exception *e, - sgx_enclave_exit_handler_t handler); -#endif SYM_FUNC_START(__vdso_sgx_enter_enclave) /* Prolog */ .cfi_startproc @@ -83,7 +23,10 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) .cfi_rel_offset %rbp, 0 mov %rsp, %rbp .cfi_def_cfa_register %rbp + push %rbx + .cfi_rel_offset %rbx, -8 + mov %ecx, %eax .Lenter_enclave: /* EENTER <= leaf <= ERESUME */ cmp $EENTER, %eax @@ -109,6 +52,7 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) jne .Linvoke_userspace_handler .Lout: + pop %rbx leave .cfi_def_cfa %rsp, 8 ret diff --git a/arch/x86/include/uapi/asm/sgx.h b/arch/x86/include/uapi/asm/sgx.h index e196cfd44b70..3760e5d5dc0c 100644 --- a/arch/x86/include/uapi/asm/sgx.h +++ b/arch/x86/include/uapi/asm/sgx.h @@ -111,4 +111,65 @@ typedef int (*sgx_enclave_exit_handler_t)(long rdi, long rsi, long rdx, void *tcs, int ret, struct sgx_enclave_exception *e); +/** + * __vdso_sgx_enter_enclave() - Enter an SGX enclave + * @rdi: Pass-through value for RDI + * @rsi: Pass-through value for RSI + * @rdx: Pass-through value for RDX + * @leaf: ENCLU leaf, must be EENTER or ERESUME + * @r8: Pass-through value for R8 + * @r9: Pass-through value for R9 + * @tcs: TCS, must be non-NULL + * @e: Optional struct sgx_enclave_exception instance + * @handler: Optional enclave exit handler + * + * NOTE: __vdso_sgx_enter_enclave() does not ensure full compliance with the + * x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. Except for + * non-volatile general purpose registers, preserving/setting state in + * accordance with the x86-64 ABI is the responsibility of the enclave and its + * runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code + * without careful consideration by both the enclave and its runtime. + * + * All general purpose registers except RAX, RBX and RCX are passed as-is to + * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are + * loaded with @leaf, asynchronous exit pointer, and @tcs respectively. + * + * RBP and the stack are used to anchor __vdso_sgx_enter_enclave() to the + * pre-enclave state, e.g. to retrieve @e and @handler after an enclave exit. + * All other registers are available for use by the enclave and its runtime, + * e.g. an enclave can push additional data onto the stack (and modify RSP) to + * pass information to the optional exit handler (see below). + * + * Most exceptions reported on ENCLU, including those that occur within the + * enclave, are fixed up and reported synchronously instead of being delivered + * via a standard signal. Debug Exceptions (#DB) and Breakpoints (#BP) are + * never fixed up and are always delivered via standard signals. On synchrously + * reported exceptions, -EFAULT is returned and details about the exception are + * recorded in @e, the optional sgx_enclave_exception struct. + + * If an exit handler is provided, the handler will be invoked on synchronous + * exits from the enclave and for all synchronously reported exceptions. In + * latter case, @e is filled prior to invoking the handler. + * + * The exit handler's return value is interpreted as follows: + * >0: continue, restart __vdso_sgx_enter_enclave() with @ret as @leaf + * 0: success, return @ret to the caller + * <0: error, return @ret to the caller + * + * The exit handler may transfer control, e.g. via longjmp() or C++ exception, + * without returning to __vdso_sgx_enter_enclave(). + * + * Return: + * 0 on success, + * -EINVAL if ENCLU leaf is not allowed, + * -EFAULT if an exception occurs on ENCLU or within the enclave + * -errno for all other negative values returned by the userspace exit handler + */ +typedef int (*vdso_sgx_enter_enclave_t)(unsigned long rdi, unsigned long rsi, + unsigned long rdx, unsigned int leaf, + unsigned long r8, unsigned long r9, + void *tcs, + struct sgx_enclave_exception *e, + sgx_enclave_exit_handler_t handler); + #endif /* _UAPI_ASM_X86_SGX_H */ diff --git a/tools/testing/selftests/sgx/call.S b/tools/testing/selftests/sgx/call.S index c37f55a607c8..77131e83db42 100644 --- a/tools/testing/selftests/sgx/call.S +++ b/tools/testing/selftests/sgx/call.S @@ -37,7 +37,6 @@ sgx_call_vdso: .cfi_adjust_cfa_offset 8 push 0x48(%rsp) .cfi_adjust_cfa_offset 8 - mov $2, %eax call *eenter(%rip) add $0x20, %rsp .cfi_adjust_cfa_offset -0x20 diff --git a/tools/testing/selftests/sgx/defines.h b/tools/testing/selftests/sgx/defines.h index 1802cace7527..be8969922804 100644 --- a/tools/testing/selftests/sgx/defines.h +++ b/tools/testing/selftests/sgx/defines.h @@ -15,6 +15,7 @@ #define __packed __attribute__((packed)) #include "../../../../arch/x86/kernel/cpu/sgx/arch.h" +#include "../../../../arch/x86/include/asm/enclu.h" #include "../../../../arch/x86/include/uapi/asm/sgx.h" #endif /* DEFINES_H */ diff --git a/tools/testing/selftests/sgx/main.c b/tools/testing/selftests/sgx/main.c index 9238cce47f77..f6bb40f22884 100644 --- a/tools/testing/selftests/sgx/main.c +++ b/tools/testing/selftests/sgx/main.c @@ -171,7 +171,7 @@ int main(int argc, char *argv[], char *envp[]) eenter = addr + eenter_sym->st_value; - sgx_call_vdso((void *)&MAGIC, &result, 0, NULL, NULL, NULL, + sgx_call_vdso((void *)&MAGIC, &result, 0, EENTER, NULL, NULL, (void *)encl.encl_base, &exception, NULL); if (result != MAGIC) goto err; diff --git a/tools/testing/selftests/sgx/main.h b/tools/testing/selftests/sgx/main.h index 6e1ae292bd25..999422cc7343 100644 --- a/tools/testing/selftests/sgx/main.h +++ b/tools/testing/selftests/sgx/main.h @@ -32,7 +32,7 @@ bool encl_load(const char *path, struct encl *encl); bool encl_measure(struct encl *encl); bool encl_build(struct encl *encl); -int sgx_call_vdso(void *rdi, void *rsi, long rdx, void *rcx, void *r8, void *r9, +int sgx_call_vdso(void *rdi, void *rsi, long rdx, u32 leaf, void *r8, void *r9, void *tcs, struct sgx_enclave_exception *ei, void *cb); #endif /* MAIN_H */ From patchwork Sat Apr 18 04:36:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11496283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01D411871 for ; Sat, 18 Apr 2020 04:36:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8D93221F7 for ; Sat, 18 Apr 2020 04:36:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725858AbgDREgM (ORCPT ); Sat, 18 Apr 2020 00:36:12 -0400 Received: from mga18.intel.com ([134.134.136.126]:25561 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725440AbgDREgM (ORCPT ); Sat, 18 Apr 2020 00:36:12 -0400 IronPort-SDR: 8pbUu6gTzmw2iOQaCO6JjorA+j17yhGw/gLkkdtWPyHllB/n8MaMKK92XilgLb8c8oNn0QZ5gS Dbx9rUwT5y4g== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Apr 2020 21:36:11 -0700 IronPort-SDR: uep3E1Dq6EVBReWcaEsc0VrciWjoDr9ZljvYgxoaGEajFwdCfYlY90ec/zE782+0LkN0d60LDL jboSvTNoCkiQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,397,1580803200"; d="scan'208";a="289459980" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.202]) by fmsmga002.fm.intel.com with ESMTP; 17 Apr 2020 21:36:10 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: Nathaniel McCallum , Cedric Xing , Jethro Beekman , Andy Lutomirski , linux-sgx@vger.kernel.org Subject: [PATCH for_v29 v3 2/2] selftests/sgx: Add selftest to invoke __vsgx_enter_enclave() from C Date: Fri, 17 Apr 2020 21:36:09 -0700 Message-Id: <20200418043609.29406-3-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: <20200418043609.29406-1-sean.j.christopherson@intel.com> References: <20200418043609.29406-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Add a selftest to call __vsgx_enter_enclave() from C. Stop clearing non-volatile registers in the enclave's trampoline code to avoid clobbering the untrusted runtime's state when the vDSO is called from C. Suggested-by: Jarkko Sakkinen Signed-off-by: Sean Christopherson --- tools/testing/selftests/sgx/main.c | 18 +++++++++++++++--- .../selftests/sgx/test_encl_bootstrap.S | 6 +----- 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/tools/testing/selftests/sgx/main.c b/tools/testing/selftests/sgx/main.c index f6bb40f22884..5394b2f6af8e 100644 --- a/tools/testing/selftests/sgx/main.c +++ b/tools/testing/selftests/sgx/main.c @@ -19,7 +19,7 @@ #include "main.h" static const uint64_t MAGIC = 0x1122334455667788ULL; -void *eenter; +vdso_sgx_enter_enclave_t eenter; struct vdso_symtab { Elf64_Sym *elf_symtab; @@ -173,15 +173,27 @@ int main(int argc, char *argv[], char *envp[]) sgx_call_vdso((void *)&MAGIC, &result, 0, EENTER, NULL, NULL, (void *)encl.encl_base, &exception, NULL); - if (result != MAGIC) + if (result != MAGIC) { + printf("FAIL: sgx_call_vdso(), expected: 0x%lx, got: 0x%lx\n", + MAGIC, result); goto err; + } + + /* Invoke the vDSO directly. */ + result = 0; + eenter((unsigned long)&MAGIC, (unsigned long)&result, 0, EENTER, 0, 0, + (void *)encl.encl_base, &exception, NULL); + if (result != MAGIC) { + printf("FAIL: eenter(), expected: 0x%lx, got: 0x%lx\n", + MAGIC, result); + goto err; + } printf("SUCCESS\n"); encl_delete(&encl); exit(0); err: - printf("FAILURE\n"); encl_delete(&encl); exit(1); } diff --git a/tools/testing/selftests/sgx/test_encl_bootstrap.S b/tools/testing/selftests/sgx/test_encl_bootstrap.S index 6a5d734cbf16..6836ea86126e 100644 --- a/tools/testing/selftests/sgx/test_encl_bootstrap.S +++ b/tools/testing/selftests/sgx/test_encl_bootstrap.S @@ -54,7 +54,7 @@ encl_entry: pop %rbx # pop the enclave base address - # Clear GPRs. + /* Clear volatile GPRs, except RAX (EEXIT leaf). */ xor %rcx, %rcx xor %rdx, %rdx xor %rdi, %rdi @@ -63,10 +63,6 @@ encl_entry: xor %r9, %r9 xor %r10, %r10 xor %r11, %r11 - xor %r12, %r12 - xor %r13, %r13 - xor %r14, %r14 - xor %r15, %r15 # Reset status flags. add %rdx, %rdx # OF = SF = AF = CF = 0; ZF = PF = 1