[v22,07/24] x86/sgx: Add wrappers for ENCLS leaf functions
diff mbox series

Message ID 20190903142655.21943-8-jarkko.sakkinen@linux.intel.com
State New
Headers show
Series
  • Intel SGX foundations
Related show

Commit Message

Jarkko Sakkinen Sept. 3, 2019, 2:26 p.m. UTC
ENCLS is a ring 0 instruction that contains a set of leaf functions for
managing enclaves [1]. Enclaves SGX hosted measured and signed software
entities, which are protected by asserting the outside memory accesses and
memory encryption.

Add a two-layer macro system along with an encoding scheme to allow
wrappers to return trap numbers along ENCLS-specific error codes. The
bottom layer of the macro system splits between the leafs that return an
error code and those that do not. The second layer generates the correct
input/output annotations based on the number of operands for each leaf
function.

[1] Intel SDM: 36.6 ENCLAVE INSTRUCTIONS AND INTEL®

Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
---
 arch/x86/kernel/cpu/sgx/Makefile |   1 +
 arch/x86/kernel/cpu/sgx/encls.c  |  24 +++
 arch/x86/kernel/cpu/sgx/encls.h  | 244 +++++++++++++++++++++++++++++++
 3 files changed, 269 insertions(+)
 create mode 100644 arch/x86/kernel/cpu/sgx/Makefile
 create mode 100644 arch/x86/kernel/cpu/sgx/encls.c
 create mode 100644 arch/x86/kernel/cpu/sgx/encls.h

Comments

Borislav Petkov Oct. 4, 2019, 9:45 a.m. UTC | #1
On Tue, Sep 03, 2019 at 05:26:38PM +0300, Jarkko Sakkinen wrote:
> ENCLS is a ring 0 instruction that contains a set of leaf functions for
> managing enclaves [1]. Enclaves SGX hosted measured and signed software
> entities, which are protected by asserting the outside memory accesses and
> memory encryption.
> 
> Add a two-layer macro system along with an encoding scheme to allow
> wrappers to return trap numbers along ENCLS-specific error codes. The
> bottom layer of the macro system splits between the leafs that return an
> error code and those that do not. The second layer generates the correct
> input/output annotations based on the number of operands for each leaf
> function.
> 
> [1] Intel SDM: 36.6 ENCLAVE INSTRUCTIONS AND INTEL®
> 
> Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>

This SOB needs to come...

> Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>

<--- ... here.

> +/**
> + * ENCLS_FAULT_FLAG - flag signifying an ENCLS return code is a trapnr
> + *
> + * ENCLS has its own (positive value) error codes and also generates
> + * ENCLS specific #GP and #PF faults.  And the ENCLS values get munged
> + * with system error codes as everything percolates back up the stack.
> + * Unfortunately (for us), we need to precisely identify each unique
> + * error code, e.g. the action taken if EWB fails varies based on the
> + * type of fault and on the exact SGX error code, i.e. we can't simply
> + * convert all faults to -EFAULT.
> + *
> + * To make all three error types coexist, we set bit 30 to identify an
> + * ENCLS fault.  Bit 31 (technically bits N:31) is used to differentiate
> + * between positive (faults and SGX error codes) and negative (system
> + * error codes) values.
> + */
> +#define ENCLS_FAULT_FLAG 0x40000000

BIT(30)

> +
> +/**
> + * Retrieve the encoded trapnr from the specified return code.
> + */
> +#define ENCLS_TRAPNR(r) ((r) & ~ENCLS_FAULT_FLAG)
> +
> +/* Issue a WARN() about an ENCLS leaf. */
> +#define ENCLS_WARN(r, name) {						\
> +	do {								\
> +		int _r = (r);						\
> +		WARN(_r, "sgx: %s returned %d (0x%x)\n", (name), _r,	\
> +		     _r);						\

Let that line stick out a bit:

		WARN(_r, "sgx: %s returned %d (0x%x)\n", (name), _r, _r); \

> +	} while (0);							\
> +}
> +
> +/**
> + * encls_faulted() - Check if ENCLS leaf function faulted
> + * @ret:	the return value of an ENCLS leaf function call
> + *
> + * Return: true if the fault flag is set
> + */
> +static inline bool encls_faulted(int ret)
> +{
> +	return (ret & ENCLS_FAULT_FLAG) != 0;

	return ret & ENCLS_FAULT_FLAG;
Jarkko Sakkinen Oct. 4, 2019, 6:56 p.m. UTC | #2
On Fri, Oct 04, 2019 at 11:45:13AM +0200, Borislav Petkov wrote:
> On Tue, Sep 03, 2019 at 05:26:38PM +0300, Jarkko Sakkinen wrote:
> > ENCLS is a ring 0 instruction that contains a set of leaf functions for
> > managing enclaves [1]. Enclaves SGX hosted measured and signed software
> > entities, which are protected by asserting the outside memory accesses and
> > memory encryption.
> > 
> > Add a two-layer macro system along with an encoding scheme to allow
> > wrappers to return trap numbers along ENCLS-specific error codes. The
> > bottom layer of the macro system splits between the leafs that return an
> > error code and those that do not. The second layer generates the correct
> > input/output annotations based on the number of operands for each leaf
> > function.
> > 
> > [1] Intel SDM: 36.6 ENCLAVE INSTRUCTIONS AND INTEL®
> > 
> > Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
> 
> This SOB needs to come...
> 
> > Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
> > Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> 
> <--- ... here.

This issue might persists in a few commits. I'll go through all of
them.

> 
> > +/**
> > + * ENCLS_FAULT_FLAG - flag signifying an ENCLS return code is a trapnr
> > + *
> > + * ENCLS has its own (positive value) error codes and also generates
> > + * ENCLS specific #GP and #PF faults.  And the ENCLS values get munged
> > + * with system error codes as everything percolates back up the stack.
> > + * Unfortunately (for us), we need to precisely identify each unique
> > + * error code, e.g. the action taken if EWB fails varies based on the
> > + * type of fault and on the exact SGX error code, i.e. we can't simply
> > + * convert all faults to -EFAULT.
> > + *
> > + * To make all three error types coexist, we set bit 30 to identify an
> > + * ENCLS fault.  Bit 31 (technically bits N:31) is used to differentiate
> > + * between positive (faults and SGX error codes) and negative (system
> > + * error codes) values.
> > + */
> > +#define ENCLS_FAULT_FLAG 0x40000000
> 
> BIT(30)
> 
> > +
> > +/**
> > + * Retrieve the encoded trapnr from the specified return code.
> > + */
> > +#define ENCLS_TRAPNR(r) ((r) & ~ENCLS_FAULT_FLAG)
> > +
> > +/* Issue a WARN() about an ENCLS leaf. */
> > +#define ENCLS_WARN(r, name) {						\
> > +	do {								\
> > +		int _r = (r);						\
> > +		WARN(_r, "sgx: %s returned %d (0x%x)\n", (name), _r,	\
> > +		     _r);						\
> 
> Let that line stick out a bit:
> 
> 		WARN(_r, "sgx: %s returned %d (0x%x)\n", (name), _r, _r); \
> 
> > +	} while (0);							\
> > +}
> > +
> > +/**
> > + * encls_faulted() - Check if ENCLS leaf function faulted
> > + * @ret:	the return value of an ENCLS leaf function call
> > + *
> > + * Return: true if the fault flag is set
> > + */
> > +static inline bool encls_faulted(int ret)
> > +{
> > +	return (ret & ENCLS_FAULT_FLAG) != 0;
> 
> 	return ret & ENCLS_FAULT_FLAG;

Great, thanks once more for great review comments. Highly appreciated.

/Jarkko
Sean Christopherson Oct. 8, 2019, 4:04 a.m. UTC | #3
On Fri, Oct 04, 2019 at 11:45:13AM +0200, Borislav Petkov wrote:
> On Tue, Sep 03, 2019 at 05:26:38PM +0300, Jarkko Sakkinen wrote:
> > +/**
> > + * ENCLS_FAULT_FLAG - flag signifying an ENCLS return code is a trapnr
> > + *
> > + * ENCLS has its own (positive value) error codes and also generates
> > + * ENCLS specific #GP and #PF faults.  And the ENCLS values get munged
> > + * with system error codes as everything percolates back up the stack.
> > + * Unfortunately (for us), we need to precisely identify each unique
> > + * error code, e.g. the action taken if EWB fails varies based on the
> > + * type of fault and on the exact SGX error code, i.e. we can't simply
> > + * convert all faults to -EFAULT.
> > + *
> > + * To make all three error types coexist, we set bit 30 to identify an
> > + * ENCLS fault.  Bit 31 (technically bits N:31) is used to differentiate
> > + * between positive (faults and SGX error codes) and negative (system
> > + * error codes) values.
> > + */
> > +#define ENCLS_FAULT_FLAG 0x40000000
> 
> BIT(30)

This is intentionally open coded so that it can be stringified in asm.
Alternatively, the asm could use the raw value or a different define.  Is
there a third option?

#define __encls_ret_N(rax, inputs...)				\
	({							\
	int ret;						\
	asm volatile(						\
	"1: .byte 0x0f, 0x01, 0xcf;\n\t"			\
	"2:\n"							\
	".section .fixup,\"ax\"\n"				\
	"3: orl $"__stringify(ENCLS_FAULT_FLAG)",%%eax\n"	\  <----
	"   jmp 2b\n"						\
	".previous\n"						\
	_ASM_EXTABLE_FAULT(1b, 3b)				\
	: "=a"(ret)						\
	: "a"(rax), inputs					\
	: "memory", "cc");					\
	ret;							\
	})
Borislav Petkov Oct. 8, 2019, 7:18 a.m. UTC | #4
On Mon, Oct 07, 2019 at 09:04:05PM -0700, Sean Christopherson wrote:
> > BIT(30)
> 
> This is intentionally open coded so that it can be stringified in asm.

It stringifies just fine with the BIT() macro too:

# 187 "arch/x86/kernel/cpu/sgx/encls.h" 1
        1: .byte 0x0f, 0x01, 0xcf;
        2:
.section .fixup,"ax"
3: orl $((((1UL))) << (30)),%eax
   jmp 2b
.previous

and the resulting object:

Disassembly of section .fixup:

0000000000000000 <.fixup>:
   0:   0d 00 00 00 40          or     $0x40000000,%eax
   5:   e9 00 00 00 00          jmpq   a <__addressable_sgx_free_page107+0x2>
Sean Christopherson Oct. 8, 2019, 1:35 p.m. UTC | #5
On Tue, Oct 08, 2019 at 09:18:45AM +0200, Borislav Petkov wrote:
> On Mon, Oct 07, 2019 at 09:04:05PM -0700, Sean Christopherson wrote:
> > > BIT(30)
> > 
> > This is intentionally open coded so that it can be stringified in asm.
> 
> It stringifies just fine with the BIT() macro too:
> 
> # 187 "arch/x86/kernel/cpu/sgx/encls.h" 1
>         1: .byte 0x0f, 0x01, 0xcf;
>         2:
> .section .fixup,"ax"
> 3: orl $((((1UL))) << (30)),%eax
>    jmp 2b
> .previous
> 
> and the resulting object:
> 
> Disassembly of section .fixup:
> 
> 0000000000000000 <.fixup>:
>    0:   0d 00 00 00 40          or     $0x40000000,%eax
>    5:   e9 00 00 00 00          jmpq   a <__addressable_sgx_free_page107+0x2>

Hmm, I get assembler errors using gcc 5.4.0

  linux/arch/x86/kernel/cpu/sgx/encls.h: Assembler messages:
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: junk `UL)))<<(30))' after expression
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: junk `UL)))<<(30))' after expression
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: missing ')'
  linux/arch/x86/kernel/cpu/sgx/encls.h:207: Error: junk `UL)))<<(30))' after expression
  linux/scripts/Makefile.build:265: recipe for target 'arch/x86/kernel/cpu/sgx/encls.o' failed
Borislav Petkov Oct. 8, 2019, 2:56 p.m. UTC | #6
On Tue, Oct 08, 2019 at 06:35:11AM -0700, Sean Christopherson wrote:
> Hmm, I get assembler errors using gcc 5.4.0

Crap. 8.3 eats this just fine. And I guess we can accomodate old gccs
by having BIT() evaluate the enclosing UL() macro into its __ASSEMBLY__
variant but it ain't worth the trouble.

Patch
diff mbox series

diff --git a/arch/x86/kernel/cpu/sgx/Makefile b/arch/x86/kernel/cpu/sgx/Makefile
new file mode 100644
index 000000000000..4432d935894e
--- /dev/null
+++ b/arch/x86/kernel/cpu/sgx/Makefile
@@ -0,0 +1 @@ 
+obj-y += encls.o
diff --git a/arch/x86/kernel/cpu/sgx/encls.c b/arch/x86/kernel/cpu/sgx/encls.c
new file mode 100644
index 000000000000..1b492c15a2b8
--- /dev/null
+++ b/arch/x86/kernel/cpu/sgx/encls.c
@@ -0,0 +1,24 @@ 
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+// Copyright(c) 2016-19 Intel Corporation.
+
+#include <asm/cpufeature.h>
+#include <asm/traps.h>
+#include "encls.h"
+#include "sgx.h"
+
+/**
+ * encls_failed() - Check if an ENCLS leaf function failed
+ * @ret:	the return value of an ENCLS leaf function call
+ *
+ * Check if an ENCLS leaf function failed. This is a condition where the leaf
+ * function causes a fault that is not caused by an EPCM conflict.
+ *
+ * Return: true if there was a fault other than an EPCM conflict
+ */
+bool encls_failed(int ret)
+{
+	int epcm_trapnr = boot_cpu_has(X86_FEATURE_SGX2) ?
+			  X86_TRAP_PF : X86_TRAP_GP;
+
+	return encls_faulted(ret) && ENCLS_TRAPNR(ret) != epcm_trapnr;
+}
diff --git a/arch/x86/kernel/cpu/sgx/encls.h b/arch/x86/kernel/cpu/sgx/encls.h
new file mode 100644
index 000000000000..aea3b9d09936
--- /dev/null
+++ b/arch/x86/kernel/cpu/sgx/encls.h
@@ -0,0 +1,244 @@ 
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+#ifndef _X86_ENCLS_H
+#define _X86_ENCLS_H
+
+#include <linux/bitops.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/rwsem.h>
+#include <linux/types.h>
+#include <asm/asm.h>
+#include "arch.h"
+
+/**
+ * ENCLS_FAULT_FLAG - flag signifying an ENCLS return code is a trapnr
+ *
+ * ENCLS has its own (positive value) error codes and also generates
+ * ENCLS specific #GP and #PF faults.  And the ENCLS values get munged
+ * with system error codes as everything percolates back up the stack.
+ * Unfortunately (for us), we need to precisely identify each unique
+ * error code, e.g. the action taken if EWB fails varies based on the
+ * type of fault and on the exact SGX error code, i.e. we can't simply
+ * convert all faults to -EFAULT.
+ *
+ * To make all three error types coexist, we set bit 30 to identify an
+ * ENCLS fault.  Bit 31 (technically bits N:31) is used to differentiate
+ * between positive (faults and SGX error codes) and negative (system
+ * error codes) values.
+ */
+#define ENCLS_FAULT_FLAG 0x40000000
+
+/**
+ * Retrieve the encoded trapnr from the specified return code.
+ */
+#define ENCLS_TRAPNR(r) ((r) & ~ENCLS_FAULT_FLAG)
+
+/* Issue a WARN() about an ENCLS leaf. */
+#define ENCLS_WARN(r, name) {						\
+	do {								\
+		int _r = (r);						\
+		WARN(_r, "sgx: %s returned %d (0x%x)\n", (name), _r,	\
+		     _r);						\
+	} while (0);							\
+}
+
+/**
+ * encls_faulted() - Check if ENCLS leaf function faulted
+ * @ret:	the return value of an ENCLS leaf function call
+ *
+ * Return: true if the fault flag is set
+ */
+static inline bool encls_faulted(int ret)
+{
+	return (ret & ENCLS_FAULT_FLAG) != 0;
+}
+
+/**
+ * encls_returned_code() - Check if an ENCLS leaf function returned a code
+ * @ret:	the return value of an ENCLS leaf function call
+ *
+ * Check if an ENCLS leaf function returned an error or information code.
+ *
+ * Return: true if there was a fault other than an EPCM conflict
+ */
+static inline bool encls_returned_code(int ret)
+{
+	return !encls_faulted(ret) && ret;
+}
+
+bool encls_failed(int ret);
+
+/**
+ * __encls_ret_N - encode an ENCLS leaf that returns an error code in EAX
+ * @rax:	leaf number
+ * @inputs:	asm inputs for the leaf
+ *
+ * Emit assembly for an ENCLS leaf that returns an error code, e.g. EREMOVE.
+ * And because SGX isn't complex enough as it is, leafs that return an error
+ * code also modify flags.
+ *
+ * Return:
+ *	0 on success,
+ *	SGX error code on failure
+ */
+#define __encls_ret_N(rax, inputs...)				\
+	({							\
+	int ret;						\
+	asm volatile(						\
+	"1: .byte 0x0f, 0x01, 0xcf;\n\t"			\
+	"2:\n"							\
+	".section .fixup,\"ax\"\n"				\
+	"3: orl $"__stringify(ENCLS_FAULT_FLAG)",%%eax\n"	\
+	"   jmp 2b\n"						\
+	".previous\n"						\
+	_ASM_EXTABLE_FAULT(1b, 3b)				\
+	: "=a"(ret)						\
+	: "a"(rax), inputs					\
+	: "memory", "cc");					\
+	ret;							\
+	})
+
+#define __encls_ret_1(rax, rcx)		\
+	({				\
+	__encls_ret_N(rax, "c"(rcx));	\
+	})
+
+#define __encls_ret_2(rax, rbx, rcx)		\
+	({					\
+	__encls_ret_N(rax, "b"(rbx), "c"(rcx));	\
+	})
+
+#define __encls_ret_3(rax, rbx, rcx, rdx)			\
+	({							\
+	__encls_ret_N(rax, "b"(rbx), "c"(rcx), "d"(rdx));	\
+	})
+
+/**
+ * __encls_N - encode an ENCLS leaf that doesn't return an error code
+ * @rax:	leaf number
+ * @rbx_out:	optional output variable
+ * @inputs:	asm inputs for the leaf
+ *
+ * Emit assembly for an ENCLS leaf that does not return an error code,
+ * e.g. ECREATE.  Leaves without error codes either succeed or fault.
+ * @rbx_out is an optional parameter for use by EDGBRD, which returns
+ * the the requested value in RBX.
+ *
+ * Return:
+ *   0 on success,
+ *   trapnr with ENCLS_FAULT_FLAG set on fault
+ */
+#define __encls_N(rax, rbx_out, inputs...)			\
+	({							\
+	int ret;						\
+	asm volatile(						\
+	"1: .byte 0x0f, 0x01, 0xcf;\n\t"			\
+	"   xor %%eax,%%eax;\n"					\
+	"2:\n"							\
+	".section .fixup,\"ax\"\n"				\
+	"3: orl $"__stringify(ENCLS_FAULT_FLAG)",%%eax\n"	\
+	"   jmp 2b\n"						\
+	".previous\n"						\
+	_ASM_EXTABLE_FAULT(1b, 3b)				\
+	: "=a"(ret), "=b"(rbx_out)				\
+	: "a"(rax), inputs					\
+	: "memory");						\
+	ret;							\
+	})
+
+#define __encls_2(rax, rbx, rcx)				\
+	({							\
+	unsigned long ign_rbx_out;				\
+	__encls_N(rax, ign_rbx_out, "b"(rbx), "c"(rcx));	\
+	})
+
+#define __encls_1_1(rax, data, rcx)			\
+	({						\
+	unsigned long rbx_out;				\
+	int ret = __encls_N(rax, rbx_out, "c"(rcx));	\
+	if (!ret)					\
+		data = rbx_out;				\
+	ret;						\
+	})
+
+static inline int __ecreate(struct sgx_pageinfo *pginfo, void *secs)
+{
+	return __encls_2(SGX_ECREATE, pginfo, secs);
+}
+
+static inline int __eextend(void *secs, void *addr)
+{
+	return __encls_2(SGX_EEXTEND, secs, addr);
+}
+
+static inline int __eadd(struct sgx_pageinfo *pginfo, void *addr)
+{
+	return __encls_2(SGX_EADD, pginfo, addr);
+}
+
+static inline int __einit(void *sigstruct, struct sgx_einittoken *einittoken,
+			  void *secs)
+{
+	return __encls_ret_3(SGX_EINIT, sigstruct, secs, einittoken);
+}
+
+static inline int __eremove(void *addr)
+{
+	return __encls_ret_1(SGX_EREMOVE, addr);
+}
+
+static inline int __edbgwr(void *addr, unsigned long *data)
+{
+	return __encls_2(SGX_EDGBWR, *data, addr);
+}
+
+static inline int __edbgrd(void *addr, unsigned long *data)
+{
+	return __encls_1_1(SGX_EDGBRD, *data, addr);
+}
+
+static inline int __etrack(void *addr)
+{
+	return __encls_ret_1(SGX_ETRACK, addr);
+}
+
+static inline int __eldu(struct sgx_pageinfo *pginfo, void *addr,
+			 void *va)
+{
+	return __encls_ret_3(SGX_ELDU, pginfo, addr, va);
+}
+
+static inline int __eblock(void *addr)
+{
+	return __encls_ret_1(SGX_EBLOCK, addr);
+}
+
+static inline int __epa(void *addr)
+{
+	unsigned long rbx = SGX_PAGE_TYPE_VA;
+
+	return __encls_2(SGX_EPA, rbx, addr);
+}
+
+static inline int __ewb(struct sgx_pageinfo *pginfo, void *addr,
+			void *va)
+{
+	return __encls_ret_3(SGX_EWB, pginfo, addr, va);
+}
+
+static inline int __eaug(struct sgx_pageinfo *pginfo, void *addr)
+{
+	return __encls_2(SGX_EAUG, pginfo, addr);
+}
+
+static inline int __emodpr(struct sgx_secinfo *secinfo, void *addr)
+{
+	return __encls_ret_2(SGX_EMODPR, secinfo, addr);
+}
+
+static inline int __emodt(struct sgx_secinfo *secinfo, void *addr)
+{
+	return __encls_ret_2(SGX_EMODT, secinfo, addr);
+}
+
+#endif /* _X86_ENCLS_H */