From patchwork Thu Mar 19 01:11:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11446363 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4A58913 for ; Thu, 19 Mar 2020 01:11:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C3FD020767 for ; Thu, 19 Mar 2020 01:11:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727064AbgCSBLh (ORCPT ); Wed, 18 Mar 2020 21:11:37 -0400 Received: from mga18.intel.com ([134.134.136.126]:30444 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726596AbgCSBLh (ORCPT ); Wed, 18 Mar 2020 21:11:37 -0400 IronPort-SDR: YfXOqAFN+d3qe+41pMaDPOrkioKsrOAyRd/pipfD6QOgsYHh2VCSaRWoVgZt4xYiwTUekEGFd9 8hWtif4BdlTw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2020 18:11:35 -0700 IronPort-SDR: HtnpJ9CJljuLvjsYx9P/hz/8hPbQRh+qWd1vaArtvLcTMflNvEomWJSzrOfdf1HmBhwdAmdA3X 0sruqjI62nwg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,569,1574150400"; d="scan'208";a="324358693" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.202]) by orsmga001.jf.intel.com with ESMTP; 18 Mar 2020 18:11:35 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: Nathaniel McCallum , Cedric Xing , Jethro Beekman , Andy Lutomirski , linux-sgx@vger.kernel.org Subject: [PATCH for_v29 1/8] x86/sgx: vdso: Remove an incorrect statement the enter enclave comment Date: Wed, 18 Mar 2020 18:11:23 -0700 Message-Id: <20200319011130.8556-2-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200319011130.8556-1-sean.j.christopherson@intel.com> References: <20200319011130.8556-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Remove the statement about the userspace exit handler being required to clean up the untrusted stack, the vDSO unconditionally restores %rsp to its value at the time of EEXIT. Reported-by: Nathaniel McCallum Signed-off-by: Sean Christopherson Acked-by: Jarkko Sakkinen --- arch/x86/entry/vdso/vsgx_enter_enclave.S | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/x86/entry/vdso/vsgx_enter_enclave.S b/arch/x86/entry/vdso/vsgx_enter_enclave.S index 94a8e5f99961..22a22e0774d8 100644 --- a/arch/x86/entry/vdso/vsgx_enter_enclave.S +++ b/arch/x86/entry/vdso/vsgx_enter_enclave.S @@ -59,10 +59,8 @@ * 0: success, return @ret to the caller * <0: error, return @ret to the caller * - * The userspace exit handler is responsible for unwinding the stack, e.g. to - * pop @e, u_rsp and @tcs, prior to returning to __vdso_sgx_enter_enclave(). - * The exit handler may also transfer control, e.g. via longjmp() or a C++ - * exception, without returning to __vdso_sgx_enter_enclave(). + * The exit handler may transfer control, e.g. via longjmp() or C++ exception, + * without returning to __vdso_sgx_enter_enclave(). * * Return: * 0 on success, From patchwork Thu Mar 19 01:11:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11446357 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5D2C4139A for ; Thu, 19 Mar 2020 01:11:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3DD5120767 for ; Thu, 19 Mar 2020 01:11:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726912AbgCSBLg (ORCPT ); Wed, 18 Mar 2020 21:11:36 -0400 Received: from mga18.intel.com ([134.134.136.126]:30443 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726623AbgCSBLg (ORCPT ); Wed, 18 Mar 2020 21:11:36 -0400 IronPort-SDR: rtmpQDcwoAeDWnTqqUFrLYPEYlCOXMD+5miOg5nP5M9lxNpiWu+dJjl8D7kHs68YD6gdxtUIfS zBCVQUa1F7qg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2020 18:11:35 -0700 IronPort-SDR: piS94afKj2luCC2wuRzCM2qMGMqSd8yIJK3fbaAFtRidC6vdyPj4LFaggSc7mab+XafDUUaFSP DfMiHIgf5kgA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,569,1574150400"; d="scan'208";a="324358695" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.202]) by orsmga001.jf.intel.com with ESMTP; 18 Mar 2020 18:11:35 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: Nathaniel McCallum , Cedric Xing , Jethro Beekman , Andy Lutomirski , linux-sgx@vger.kernel.org Subject: [PATCH for_v29 2/8] x86/sgx: vdso: Make the %rsp fixup on return from handler relative Date: Wed, 18 Mar 2020 18:11:24 -0700 Message-Id: <20200319011130.8556-3-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200319011130.8556-1-sean.j.christopherson@intel.com> References: <20200319011130.8556-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Modify the %rsp fixup after returning from the exit handler to be relative instead of absolute to avoid clobbering any %rsp adjustments made by the exit handler, e.g. if the exit handler modifies the stack prior to re-entering the enclave. Reported-by: Nathaniel McCallum Signed-off-by: Sean Christopherson --- I'm on the fence as to whether or not this is a good idea. It's not super painful, but it's not exactly standard/obvious code. Part of me thinks its a bug to not let the exit handler manipulate %rsp, the other part of me thinks it's straight up crazy :-) arch/x86/entry/vdso/vsgx_enter_enclave.S | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/vdso/vsgx_enter_enclave.S b/arch/x86/entry/vdso/vsgx_enter_enclave.S index 22a22e0774d8..14f07d5e47ae 100644 --- a/arch/x86/entry/vdso/vsgx_enter_enclave.S +++ b/arch/x86/entry/vdso/vsgx_enter_enclave.S @@ -137,8 +137,9 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) /* Pass the untrusted RSP (at exit) to the callback via %rcx. */ mov %rsp, %rcx - /* Save the untrusted RSP in %rbx (non-volatile register). */ + /* Save the untrusted RSP offset in %rbx (non-volatile register). */ mov %rsp, %rbx + and $0xf, %rbx /* * Align stack per x86_64 ABI. Note, %rsp needs to be 16-byte aligned @@ -159,8 +160,8 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) mov 0x20(%rbp), %rax call .Lretpoline - /* Restore %rsp to its post-exit value. */ - mov %rbx, %rsp + /* Undo the post-exit %rsp adjustment. */ + lea 0x20(%rsp,%rbx), %rsp /* * If the return from callback is zero or negative, return immediately, From patchwork Thu Mar 19 01:11:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11446371 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC59B139A for ; Thu, 19 Mar 2020 01:11:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A6BAD20767 for ; Thu, 19 Mar 2020 01:11:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726913AbgCSBLi (ORCPT ); Wed, 18 Mar 2020 21:11:38 -0400 Received: from mga18.intel.com ([134.134.136.126]:30444 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726623AbgCSBLi (ORCPT ); Wed, 18 Mar 2020 21:11:38 -0400 IronPort-SDR: BPZ1aPCSBEKTV6oUVwKCja2P07GhY660SDjzTVQnmKAPrvZunEah/YT0MxymqKl/25aqaRDRJn 8xo8nv1xFQOg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2020 18:11:35 -0700 IronPort-SDR: 2OMASfcLXZySzdeklV+BTJ25+hSfvPWnSoV2ImPGxVo+7DGk99DdWg9PNcFcUq6NnJzCyXg+mo 26q5UP8Z/oAw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,569,1574150400"; d="scan'208";a="324358697" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.202]) by orsmga001.jf.intel.com with ESMTP; 18 Mar 2020 18:11:35 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: Nathaniel McCallum , Cedric Xing , Jethro Beekman , Andy Lutomirski , linux-sgx@vger.kernel.org Subject: [PATCH for_v29 3/8] x86/sgx: vdso: Make __vdso_sgx_enter_enclave() callable from C code Date: Wed, 18 Mar 2020 18:11:25 -0700 Message-Id: <20200319011130.8556-4-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200319011130.8556-1-sean.j.christopherson@intel.com> References: <20200319011130.8556-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Make __vdso_sgx_enter_enclave() callable from C by preserving %rbx and taking @leaf in %rcx instead of %rax. Being able to invoke the vDSO from C reduces the overhead of runtimes that are tightly coupled with their enclaves, e.g. that can rely on the enclave to save and restore non-volatile registers, as the runtime doesn't need an assembly wrapper to preserve non-volatile registers and/or shuffle stack arguments. Note, both %rcx and %rbx are consumed by EENTER/ERESUME, i.e. consuming them doesn't violate the primary tenet of __vdso_sgx_enter_enclave() that "thou shalt not restrict how information is exchanged between an enclave and its host process". Suggested-by: Nathaniel McCallum Cc: Cedric Xing Cc: Jethro Beekman Cc: Andy Lutomirski Cc: linux-sgx@vger.kernel.org Signed-off-by: Sean Christopherson --- Cedric, please go to town on this, you're much better at the userspace stack and debugging interactions, e.g. I don't think the CFA directives need updating, but my knowledge of that stuff is skeeetchy. Note, if the previous patch to make the %rsp restoration relative is dropped, then %rbx should be restored relative to %rbp instead of %rsp. arch/x86/entry/vdso/vsgx_enter_enclave.S | 33 ++++++++++++++---------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/arch/x86/entry/vdso/vsgx_enter_enclave.S b/arch/x86/entry/vdso/vsgx_enter_enclave.S index 14f07d5e47ae..7a0565476a29 100644 --- a/arch/x86/entry/vdso/vsgx_enter_enclave.S +++ b/arch/x86/entry/vdso/vsgx_enter_enclave.S @@ -16,25 +16,25 @@ /** * __vdso_sgx_enter_enclave() - Enter an SGX enclave + * @rdi: Pass-through value for RDI + * @rsi: Pass-through value for RSI + * @rdx: Pass-through value for RDX * @leaf: ENCLU leaf, must be EENTER or ERESUME + * @r8: Pass-through value for R8 + * @r9: Pass-through value for R9 * @tcs: TCS, must be non-NULL * @e: Optional struct sgx_enclave_exception instance * @handler: Optional enclave exit handler * - * **Important!** __vdso_sgx_enter_enclave() is **NOT** compliant with the - * x86-64 ABI, i.e. cannot be called from standard C code. - * - * Input ABI: - * @leaf %eax - * @tcs 8(%rsp) - * @e 0x10(%rsp) - * @handler 0x18(%rsp) - * - * Output ABI: - * @ret %eax + * **Important!** __vdso_sgx_enter_enclave() does not ensure full compliance + * with the x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. + * Except for non-volatile general purpose registers, preserving/setting state + * in accordance with the x86-64 ABI is the responsibility of the enclave and + * its runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code + * without careful consideration by both the enclave and its runtime. * * All general purpose registers except RAX, RBX and RCX are passed as-is to - * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are + * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are * loaded with @leaf, asynchronous exit pointer, and @tcs respectively. * * RBP and the stack are used to anchor __vdso_sgx_enter_enclave() to the @@ -70,9 +70,11 @@ */ #ifdef SGX_KERNEL_DOC /* C-style function prototype to coerce kernel-doc into parsing the comment. */ -int __vdso_sgx_enter_enclave(int leaf, void *tcs, +int __vdso_sgx_enter_enclave(unsigned long rdi, unsigned long rsi, + unsigned long rdx, unsigned int leaf, + unsigned long r8, unsigned long r9, void *tcs, struct sgx_enclave_exception *e, - sgx_enclave_exit_handler_t handler); + sgx_enclave_exit_handler_t handler) #endif SYM_FUNC_START(__vdso_sgx_enter_enclave) /* Prolog */ @@ -82,7 +84,9 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) .cfi_rel_offset %rbp, 0 mov %rsp, %rbp .cfi_def_cfa_register %rbp + push %rbx + mov %ecx, %eax .Lenter_enclave: /* EENTER <= leaf <= ERESUME */ cmp $0x2, %eax @@ -108,6 +112,7 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) jne .Linvoke_userspace_handler .Lout: + pop %rbx leave .cfi_def_cfa %rsp, 8 ret From patchwork Thu Mar 19 01:11:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11446365 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1CF15139A for ; Thu, 19 Mar 2020 01:11:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 05F9020767 for ; Thu, 19 Mar 2020 01:11:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727065AbgCSBLh (ORCPT ); Wed, 18 Mar 2020 21:11:37 -0400 Received: from mga18.intel.com ([134.134.136.126]:30443 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726913AbgCSBLh (ORCPT ); Wed, 18 Mar 2020 21:11:37 -0400 IronPort-SDR: U62jlWlubNGUZIAD69A+0Hit70aQBnysWOA0ZyJBZZEtjvs9oBLLU5XTQlHFrrcmgExifo/P/1 6+BTJ7voJpjQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2020 18:11:35 -0700 IronPort-SDR: 0ES7UTbe/DeGXSPUbISzejCmouYIymuWwlV7Hi66ufLDXoSxplI0ZUTJoFQ92d2/BVTFAWycP6 4Y31iHzFpzUA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,569,1574150400"; d="scan'208";a="324358700" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.202]) by orsmga001.jf.intel.com with ESMTP; 18 Mar 2020 18:11:35 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: Nathaniel McCallum , Cedric Xing , Jethro Beekman , Andy Lutomirski , linux-sgx@vger.kernel.org Subject: [PATCH for_v29 4/8] x86/sgx: vdso: Define a typedef for __vdso_sgx_enter_enclave Date: Wed, 18 Mar 2020 18:11:26 -0700 Message-Id: <20200319011130.8556-5-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200319011130.8556-1-sean.j.christopherson@intel.com> References: <20200319011130.8556-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Define a typedef for the __vdso_sgx_enter_enclave() prototype and move the entire function comment from the assembly code to the uAPI header, dropping the kernel doc hack along the way. Signed-off-by: Sean Christopherson --- arch/x86/entry/vdso/vsgx_enter_enclave.S | 62 ------------------------ arch/x86/include/uapi/asm/sgx.h | 61 +++++++++++++++++++++++ 2 files changed, 61 insertions(+), 62 deletions(-) diff --git a/arch/x86/entry/vdso/vsgx_enter_enclave.S b/arch/x86/entry/vdso/vsgx_enter_enclave.S index 7a0565476a29..46f088927c1d 100644 --- a/arch/x86/entry/vdso/vsgx_enter_enclave.S +++ b/arch/x86/entry/vdso/vsgx_enter_enclave.S @@ -14,68 +14,6 @@ .code64 .section .text, "ax" -/** - * __vdso_sgx_enter_enclave() - Enter an SGX enclave - * @rdi: Pass-through value for RDI - * @rsi: Pass-through value for RSI - * @rdx: Pass-through value for RDX - * @leaf: ENCLU leaf, must be EENTER or ERESUME - * @r8: Pass-through value for R8 - * @r9: Pass-through value for R9 - * @tcs: TCS, must be non-NULL - * @e: Optional struct sgx_enclave_exception instance - * @handler: Optional enclave exit handler - * - * **Important!** __vdso_sgx_enter_enclave() does not ensure full compliance - * with the x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. - * Except for non-volatile general purpose registers, preserving/setting state - * in accordance with the x86-64 ABI is the responsibility of the enclave and - * its runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code - * without careful consideration by both the enclave and its runtime. - * - * All general purpose registers except RAX, RBX and RCX are passed as-is to - * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are - * loaded with @leaf, asynchronous exit pointer, and @tcs respectively. - * - * RBP and the stack are used to anchor __vdso_sgx_enter_enclave() to the - * pre-enclave state, e.g. to retrieve @e and @handler after an enclave exit. - * All other registers are available for use by the enclave and its runtime, - * e.g. an enclave can push additional data onto the stack (and modify RSP) to - * pass information to the optional exit handler (see below). - * - * Most exceptions reported on ENCLU, including those that occur within the - * enclave, are fixed up and reported synchronously instead of being delivered - * via a standard signal. Debug Exceptions (#DB) and Breakpoints (#BP) are - * never fixed up and are always delivered via standard signals. On synchrously - * reported exceptions, -EFAULT is returned and details about the exception are - * recorded in @e, the optional sgx_enclave_exception struct. - - * If an exit handler is provided, the handler will be invoked on synchronous - * exits from the enclave and for all synchronously reported exceptions. In - * latter case, @e is filled prior to invoking the handler. - * - * The exit handler's return value is interpreted as follows: - * >0: continue, restart __vdso_sgx_enter_enclave() with @ret as @leaf - * 0: success, return @ret to the caller - * <0: error, return @ret to the caller - * - * The exit handler may transfer control, e.g. via longjmp() or C++ exception, - * without returning to __vdso_sgx_enter_enclave(). - * - * Return: - * 0 on success, - * -EINVAL if ENCLU leaf is not allowed, - * -EFAULT if an exception occurs on ENCLU or within the enclave - * -errno for all other negative values returned by the userspace exit handler - */ -#ifdef SGX_KERNEL_DOC -/* C-style function prototype to coerce kernel-doc into parsing the comment. */ -int __vdso_sgx_enter_enclave(unsigned long rdi, unsigned long rsi, - unsigned long rdx, unsigned int leaf, - unsigned long r8, unsigned long r9, void *tcs, - struct sgx_enclave_exception *e, - sgx_enclave_exit_handler_t handler) -#endif SYM_FUNC_START(__vdso_sgx_enter_enclave) /* Prolog */ .cfi_startproc diff --git a/arch/x86/include/uapi/asm/sgx.h b/arch/x86/include/uapi/asm/sgx.h index e196cfd44b70..8ca9ef7ea50a 100644 --- a/arch/x86/include/uapi/asm/sgx.h +++ b/arch/x86/include/uapi/asm/sgx.h @@ -111,4 +111,65 @@ typedef int (*sgx_enclave_exit_handler_t)(long rdi, long rsi, long rdx, void *tcs, int ret, struct sgx_enclave_exception *e); +/** + * __vdso_sgx_enter_enclave() - Enter an SGX enclave + * @rdi: Pass-through value for RDI + * @rsi: Pass-through value for RSI + * @rdx: Pass-through value for RDX + * @leaf: ENCLU leaf, must be EENTER or ERESUME + * @r8: Pass-through value for R8 + * @r9: Pass-through value for R9 + * @tcs: TCS, must be non-NULL + * @e: Optional struct sgx_enclave_exception instance + * @handler: Optional enclave exit handler + * + * **Important!** __vdso_sgx_enter_enclave() does not ensure full compliance + * with the x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. + * Except for non-volatile general purpose registers, preserving/setting state + * in accordance with the x86-64 ABI is the responsibility of the enclave and + * its runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code + * without careful consideration by both the enclave and its runtime. + * + * All general purpose registers except RAX, RBX and RCX are passed as-is to + * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are + * loaded with @leaf, asynchronous exit pointer, and @tcs respectively. + * + * RBP and the stack are used to anchor __vdso_sgx_enter_enclave() to the + * pre-enclave state, e.g. to retrieve @e and @handler after an enclave exit. + * All other registers are available for use by the enclave and its runtime, + * e.g. an enclave can push additional data onto the stack (and modify RSP) to + * pass information to the optional exit handler (see below). + * + * Most exceptions reported on ENCLU, including those that occur within the + * enclave, are fixed up and reported synchronously instead of being delivered + * via a standard signal. Debug Exceptions (#DB) and Breakpoints (#BP) are + * never fixed up and are always delivered via standard signals. On synchrously + * reported exceptions, -EFAULT is returned and details about the exception are + * recorded in @e, the optional sgx_enclave_exception struct. + + * If an exit handler is provided, the handler will be invoked on synchronous + * exits from the enclave and for all synchronously reported exceptions. In + * latter case, @e is filled prior to invoking the handler. + * + * The exit handler's return value is interpreted as follows: + * >0: continue, restart __vdso_sgx_enter_enclave() with @ret as @leaf + * 0: success, return @ret to the caller + * <0: error, return @ret to the caller + * + * The exit handler may transfer control, e.g. via longjmp() or C++ exception, + * without returning to __vdso_sgx_enter_enclave(). + * + * Return: + * 0 on success, + * -EINVAL if ENCLU leaf is not allowed, + * -EFAULT if an exception occurs on ENCLU or within the enclave + * -errno for all other negative values returned by the userspace exit handler + */ +typedef int (*vdso_sgx_enter_enclave_t)(unsigned long rdi, unsigned long rsi, + unsigned long rdx, unsigned int leaf, + unsigned long r8, unsigned long r9, + void *tcs, + struct sgx_enclave_exception *e, + sgx_enclave_exit_handler_t handler); + #endif /* _UAPI_ASM_X86_SGX_H */ From patchwork Thu Mar 19 01:11:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11446361 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6D411893 for ; Thu, 19 Mar 2020 01:11:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8700E20767 for ; Thu, 19 Mar 2020 01:11:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726971AbgCSBLh (ORCPT ); Wed, 18 Mar 2020 21:11:37 -0400 Received: from mga18.intel.com ([134.134.136.126]:30444 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726623AbgCSBLh (ORCPT ); Wed, 18 Mar 2020 21:11:37 -0400 IronPort-SDR: Cy0608T70gX7jlXdSrT7/rqLLS5ZcTeToRfs8CzsIJ/gafc5uhoBMMHOum79AE8yB0BqLjp3CI wY1n6+usieAA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2020 18:11:35 -0700 IronPort-SDR: HxpzM6pecHEK0M+u7tmK3mfqMdkAQPx2Ofs+4GExzE4msQ+lfAZWngzUlYT+AkhIr7FT7mi0Ag KmZlcWbvxK/A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,569,1574150400"; d="scan'208";a="324358703" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.202]) by orsmga001.jf.intel.com with ESMTP; 18 Mar 2020 18:11:35 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: Nathaniel McCallum , Cedric Xing , Jethro Beekman , Andy Lutomirski , linux-sgx@vger.kernel.org Subject: [PATCH for_v29 5/8] selftests/x86: sgx: Zero out @result before invoking vDSO sub-test Date: Wed, 18 Mar 2020 18:11:27 -0700 Message-Id: <20200319011130.8556-6-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200319011130.8556-1-sean.j.christopherson@intel.com> References: <20200319011130.8556-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Zero out @result before running the vDSO sub-test, otherwise the vDSO could fail completely and the selftest would be none the wiser, e.g. it doesn't explicitly check the return value. Signed-off-by: Sean Christopherson --- tools/testing/selftests/x86/sgx/main.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/x86/sgx/main.c b/tools/testing/selftests/x86/sgx/main.c index d97cc3cf0093..c9c37d2bbec8 100644 --- a/tools/testing/selftests/x86/sgx/main.c +++ b/tools/testing/selftests/x86/sgx/main.c @@ -366,6 +366,7 @@ int main(int argc, char *argv[], char *envp[]) printf("Input: 0x%lx\n", MAGIC); + result = 0; sgx_call_vdso((void *)&MAGIC, &result, 0, NULL, NULL, NULL, (void *)secs.base, &exception, NULL); if (result != MAGIC) { From patchwork Thu Mar 19 01:11:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11446359 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E13F14DD for ; Thu, 19 Mar 2020 01:11:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6766520767 for ; Thu, 19 Mar 2020 01:11:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726934AbgCSBLh (ORCPT ); Wed, 18 Mar 2020 21:11:37 -0400 Received: from mga18.intel.com ([134.134.136.126]:30443 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726596AbgCSBLh (ORCPT ); Wed, 18 Mar 2020 21:11:37 -0400 IronPort-SDR: dgxi80c2uPLU4hM1TbES6z+Clfwa4M0O3Eq7m7AluTJkCOK1aKNy45BfhqYV93vgLPDFn3eCbZ 80LSS6ZjYK3g== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2020 18:11:35 -0700 IronPort-SDR: IJIs2yg1Qp3yBQgTSzimVGXO0Y2rfVpmEzPd3FPKwEs5Yg2M5ets1gYGbC2nEKXyxmyIxHuMv9 HY42WbmKiT0w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,569,1574150400"; d="scan'208";a="324358706" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.202]) by orsmga001.jf.intel.com with ESMTP; 18 Mar 2020 18:11:35 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: Nathaniel McCallum , Cedric Xing , Jethro Beekman , Andy Lutomirski , linux-sgx@vger.kernel.org Subject: [PATCH for_v29 6/8] selftests/x86: sgx: Pass EENTER to vDSO wrapper instead of hardcoding Date: Wed, 18 Mar 2020 18:11:28 -0700 Message-Id: <20200319011130.8556-7-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200319011130.8556-1-sean.j.christopherson@intel.com> References: <20200319011130.8556-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Pass EENTER to the vDSO wrapper via %ecx instead of hardcoding it to make the wrapper compatible with the new vDSO calling convention. Signed-off-by: Sean Christopherson --- tools/testing/selftests/x86/sgx/main.c | 2 +- tools/testing/selftests/x86/sgx/sgx_call.S | 1 - tools/testing/selftests/x86/sgx/sgx_call.h | 2 +- 3 files changed, 2 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/x86/sgx/main.c b/tools/testing/selftests/x86/sgx/main.c index c9c37d2bbec8..9b89946a976b 100644 --- a/tools/testing/selftests/x86/sgx/main.c +++ b/tools/testing/selftests/x86/sgx/main.c @@ -367,7 +367,7 @@ int main(int argc, char *argv[], char *envp[]) printf("Input: 0x%lx\n", MAGIC); result = 0; - sgx_call_vdso((void *)&MAGIC, &result, 0, NULL, NULL, NULL, + sgx_call_vdso((void *)&MAGIC, &result, 0, 2, NULL, NULL, (void *)secs.base, &exception, NULL); if (result != MAGIC) { fprintf(stderr, "0x%lx != 0x%lx\n", result, MAGIC); diff --git a/tools/testing/selftests/x86/sgx/sgx_call.S b/tools/testing/selftests/x86/sgx/sgx_call.S index e71f44f7a995..79b4c80e4a1c 100644 --- a/tools/testing/selftests/x86/sgx/sgx_call.S +++ b/tools/testing/selftests/x86/sgx/sgx_call.S @@ -48,7 +48,6 @@ sgx_call_vdso: .cfi_adjust_cfa_offset 8 push 0x48(%rsp) .cfi_adjust_cfa_offset 8 - mov $2, %eax call *eenter(%rip) add $0x20, %rsp .cfi_adjust_cfa_offset -0x20 diff --git a/tools/testing/selftests/x86/sgx/sgx_call.h b/tools/testing/selftests/x86/sgx/sgx_call.h index a4072c5ecce7..02f1ef292823 100644 --- a/tools/testing/selftests/x86/sgx/sgx_call.h +++ b/tools/testing/selftests/x86/sgx/sgx_call.h @@ -8,7 +8,7 @@ void sgx_call_eenter(void *rdi, void *rsi, void *entry); -int sgx_call_vdso(void *rdi, void *rsi, long rdx, void *rcx, void *r8, void *r9, +int sgx_call_vdso(void *rdi, void *rsi, long rdx, long rcx, void *r8, void *r9, void *tcs, struct sgx_enclave_exception *ei, void *cb); #endif /* SGX_CALL_H */ From patchwork Thu Mar 19 01:11:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11446367 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 755AF16C1 for ; Thu, 19 Mar 2020 01:11:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5506020767 for ; Thu, 19 Mar 2020 01:11:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727068AbgCSBLi (ORCPT ); Wed, 18 Mar 2020 21:11:38 -0400 Received: from mga18.intel.com ([134.134.136.126]:30443 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726596AbgCSBLi (ORCPT ); Wed, 18 Mar 2020 21:11:38 -0400 IronPort-SDR: LGq0gtfA33QRUUD9U6iA6ldaxKq3n0DJHN62iJGeX+dY4M9tfXGoDXsSctLmu5PWc+9pbKM2/S hecqaNKtKohQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2020 18:11:35 -0700 IronPort-SDR: diW1A1DvCt7/yloVqOfJBVhaBO7tTzNw6ED+GyVaG/KvCdYTBw1uZI6XKuPyxY42keOa928Qky tGlpSPUiILAA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,569,1574150400"; d="scan'208";a="324358709" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.202]) by orsmga001.jf.intel.com with ESMTP; 18 Mar 2020 18:11:35 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: Nathaniel McCallum , Cedric Xing , Jethro Beekman , Andy Lutomirski , linux-sgx@vger.kernel.org Subject: [PATCH for_v29 7/8] selftests/x86: sgx: Stop clobbering non-volatile registers Date: Wed, 18 Mar 2020 18:11:29 -0700 Message-Id: <20200319011130.8556-8-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200319011130.8556-1-sean.j.christopherson@intel.com> References: <20200319011130.8556-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Stop clearing non-volatile registers in the enclave's trampoline code, there are no secrets to preserve, and not clobbering the registers makes the enclave compatible with calling the vDSO from C. Signed-off-by: Sean Christopherson --- tools/testing/selftests/x86/sgx/encl_bootstrap.S | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/tools/testing/selftests/x86/sgx/encl_bootstrap.S b/tools/testing/selftests/x86/sgx/encl_bootstrap.S index 3a1479f1cdcf..173e76fa6da6 100644 --- a/tools/testing/selftests/x86/sgx/encl_bootstrap.S +++ b/tools/testing/selftests/x86/sgx/encl_bootstrap.S @@ -40,7 +40,7 @@ encl_entry: pop %rbx # pop the enclave base address - # Clear GPRs. + /* Clear volatile GPRs, except RAX (EEXIT leaf). */ xor %rcx, %rcx xor %rdx, %rdx xor %rdi, %rdi @@ -49,10 +49,6 @@ encl_entry: xor %r9, %r9 xor %r10, %r10 xor %r11, %r11 - xor %r12, %r12 - xor %r13, %r13 - xor %r14, %r14 - xor %r15, %r15 # Reset status flags. add %rdx, %rdx # OF = SF = AF = CF = 0; ZF = PF = 1 From patchwork Thu Mar 19 01:11:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11446369 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9D80A913 for ; Thu, 19 Mar 2020 01:11:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7E31820767 for ; Thu, 19 Mar 2020 01:11:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726596AbgCSBLi (ORCPT ); Wed, 18 Mar 2020 21:11:38 -0400 Received: from mga18.intel.com ([134.134.136.126]:30443 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726913AbgCSBLi (ORCPT ); Wed, 18 Mar 2020 21:11:38 -0400 IronPort-SDR: aGp0qTQICgNXu6BQvP4a4CF10UiLDQD1h+DD1Y8M98rM/sIxbhoOj3tdCJjHp+W7fdJIXG5yri fireJ5ODJZ+Q== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2020 18:11:35 -0700 IronPort-SDR: 7okiq+RXu8UsQ1TppwEuevfqqfkt9lGnq4qHhhplZvB79Rr26gGnPblUGt5pnipzbXLXDy3xN5 6sEN6vlnvo4Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,569,1574150400"; d="scan'208";a="324358712" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.202]) by orsmga001.jf.intel.com with ESMTP; 18 Mar 2020 18:11:35 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: Nathaniel McCallum , Cedric Xing , Jethro Beekman , Andy Lutomirski , linux-sgx@vger.kernel.org Subject: [PATCH for_v29 8/8] selftests/x86: Add selftest to invoke __vsgx_enter_enclave() from C Date: Wed, 18 Mar 2020 18:11:30 -0700 Message-Id: <20200319011130.8556-9-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200319011130.8556-1-sean.j.christopherson@intel.com> References: <20200319011130.8556-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Add a selftest to call __vsgx_enter_enclave() from C. Suggested-by: Jarkko Sakkinen Signed-off-by: Sean Christopherson --- tools/testing/selftests/x86/sgx/main.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/x86/sgx/main.c b/tools/testing/selftests/x86/sgx/main.c index 9b89946a976b..bf64fa857983 100644 --- a/tools/testing/selftests/x86/sgx/main.c +++ b/tools/testing/selftests/x86/sgx/main.c @@ -21,7 +21,7 @@ #define PAGE_SIZE 4096 static const uint64_t MAGIC = 0x1122334455667788ULL; -void *eenter; +vdso_sgx_enter_enclave_t eenter; struct vdso_symtab { Elf64_Sym *elf_symtab; @@ -376,5 +376,17 @@ int main(int argc, char *argv[], char *envp[]) printf("Output: 0x%lx\n", result); + printf("Input: 0x%lx\n", MAGIC); + result = 0; + + /* Invoke the vDSO directly. */ + eenter((unsigned long)&MAGIC, (unsigned long)&result, 0, 2, 0, 0, + (void *)secs.base, &exception, NULL); + if (result != MAGIC) { + fprintf(stderr, "0x%lx != 0x%lx\n", result, MAGIC); + exit(1); + } + printf("Output: 0x%lx\n", result); + exit(0); }