| Message ID | 20220705183648.3739111-1-dave.hansen@linux.intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | x86/sgx: Allow enclaves to use Asynchrounous Exit Notification | expand |
On Tue, Jul 05, 2022 at 11:36:48AM -0700, Dave Hansen wrote: > Short Version: > > Allow enclaves to use the new Asynchronous EXit (AEX) > notification mechanism. This mechanism lets enclaves run a > handler after an AEX event. These handlers can run mitigations > for things like SGX-Step[1]. > > AEX Notify will be made available both on upcoming processors and > on some older processors through microcode updates. > > Long Version: > > == SGX Attribute Background == > > The SGX architecture includes a list of SGX "attributes". These > attributes ensure consistency and transparency around specific > enclave features. > > As a simple example, the "DEBUG" attribute allows an enclave to > be debugged, but also destroys virtually all of SGX security. > Using attributes, enclaves can know that they are being debugged. > Attributes also affect enclave attestation so an enclave can, for > instance, be denied access to secrets while it is being debugged. > > The kernel keeps a list of known attributes and will only > initialize enclaves that use a known set of attributes. This > kernel policy eliminates the chance that a new SGX attribute > could cause undesired effects. > > For example, imagine a new attribute was added called > "PROVISIONKEY2" that provided similar functionality to > "PROVISIIONKEY". A kernel policy that allowed indiscriminate use > of unknown attributes and thus PROVISIONKEY2 would undermine the > existing kernel policy which limits use of PROVISIONKEY enclaves. > > == AEX Notify Background == > > "Intel Architecture Instruction Set Extensions and Future > Features - Version 45" is out[2]. There is a new chapter: > > Asynchronous Enclave Exit Notify and the EDECCSSA User Leaf Function. > > Enclaves exit can be either synchronous and consensual (EEXIT for > instance) or asynchronous (on an interrupt or fault). The > asynchronous ones can evidently be exploited to single step > enclaves[1], on top of which other naughty things can be built. > > AEX Notify will be made available both on upcoming processors and > on some older processors through microcode updates. > > == The Problem == > > These attacks are currently entirely opaque to the enclave since > the hardware does the save/restore under the covers. The > Asynchronous Enclave Exit Notify (AEX Notify) mechanism provides > enclaves an ability to detect and mitigate potential exposure to > these kinds of attacks. > > == The Solution == > > Define the new attribute value for AEX Notification. Ensure the > attribute is cleared from the list reserved attributes which > allows it to be used in enclaves. > > I just built this and ran it to make sure there were no obvious > regressions since I do not have the hardware to test it handy. > Tested-by's would be much appreciated. Is this available on recent ucode updates e.g. for Icelake or Geminilake? > > 1. https://github.com/jovanbulck/sgx-step > 2. https://cdrdv2.intel.com/v1/dl/getContent/671368?explicitVersion=true > > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> > Cc: Jarkko Sakkinen <jarkko@kernel.org> > Cc: Andy Lutomirski <luto@kernel.org> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: Ingo Molnar <mingo@redhat.com> > Cc: Borislav Petkov <bp@alien8.de> > Cc: x86@kernel.org > Cc: "H. Peter Anvin" <hpa@zytor.com> > Cc: linux-sgx@vger.kernel.org > Cc: linux-kernel@vger.kernel.org > --- > arch/x86/include/asm/sgx.h | 25 ++++++++++++++++++------- > 1 file changed, 18 insertions(+), 7 deletions(-) > > diff --git a/arch/x86/include/asm/sgx.h b/arch/x86/include/asm/sgx.h > index 3f9334ef67cd..f7328d8efd83 100644 > --- a/arch/x86/include/asm/sgx.h > +++ b/arch/x86/include/asm/sgx.h > @@ -110,17 +110,28 @@ enum sgx_miscselect { > * %SGX_ATTR_EINITTOKENKEY: Allow to use token signing key that is used to > * sign cryptographic tokens that can be passed to > * EINIT as an authorization to run an enclave. > + * %SGX_ATTR_ASYNC_EXIT_NOTIFY: Allow enclaves to be notified after an > + * asynchronous exit has occurred. > */ > enum sgx_attribute { > - SGX_ATTR_INIT = BIT(0), > - SGX_ATTR_DEBUG = BIT(1), > - SGX_ATTR_MODE64BIT = BIT(2), > - SGX_ATTR_PROVISIONKEY = BIT(4), > - SGX_ATTR_EINITTOKENKEY = BIT(5), > - SGX_ATTR_KSS = BIT(7), > + SGX_ATTR_INIT = BIT(0), > + SGX_ATTR_DEBUG = BIT(1), > + SGX_ATTR_MODE64BIT = BIT(2), > + /* BIT(3) is reserved */ > + SGX_ATTR_PROVISIONKEY = BIT(4), > + SGX_ATTR_EINITTOKENKEY = BIT(5), > + /* BIT(6) is for CET */ > + SGX_ATTR_KSS = BIT(7), > + /* BIT(8) is reserved */ > + /* BIT(9) is reserved */ > + SGX_ATTR_ASYNC_EXIT_NOTIFY = BIT(10), > }; > > -#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | BIT_ULL(6) | GENMASK_ULL(63, 8)) > +#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | \ > + BIT_ULL(6) | \ > + BIT_ULL(8) | \ > + BIT_ULL(9) | \ > + GENMASK_ULL(63, 11)) > > /** > * struct sgx_secs - SGX Enclave Control Structure (SECS) > -- > 2.34.1 > BR, Jarkko
On Mon, Jul 11, 2022 at 05:34:48AM +0300, Jarkko Sakkinen wrote: > On Tue, Jul 05, 2022 at 11:36:48AM -0700, Dave Hansen wrote: > > Short Version: > > > > Allow enclaves to use the new Asynchronous EXit (AEX) > > notification mechanism. This mechanism lets enclaves run a > > handler after an AEX event. These handlers can run mitigations > > for things like SGX-Step[1]. > > > > AEX Notify will be made available both on upcoming processors and > > on some older processors through microcode updates. > > > > Long Version: > > > > == SGX Attribute Background == > > > > The SGX architecture includes a list of SGX "attributes". These > > attributes ensure consistency and transparency around specific > > enclave features. > > > > As a simple example, the "DEBUG" attribute allows an enclave to > > be debugged, but also destroys virtually all of SGX security. > > Using attributes, enclaves can know that they are being debugged. > > Attributes also affect enclave attestation so an enclave can, for > > instance, be denied access to secrets while it is being debugged. > > > > The kernel keeps a list of known attributes and will only > > initialize enclaves that use a known set of attributes. This > > kernel policy eliminates the chance that a new SGX attribute > > could cause undesired effects. > > > > For example, imagine a new attribute was added called > > "PROVISIONKEY2" that provided similar functionality to > > "PROVISIIONKEY". A kernel policy that allowed indiscriminate use > > of unknown attributes and thus PROVISIONKEY2 would undermine the > > existing kernel policy which limits use of PROVISIONKEY enclaves. > > > > == AEX Notify Background == > > > > "Intel Architecture Instruction Set Extensions and Future > > Features - Version 45" is out[2]. There is a new chapter: > > > > Asynchronous Enclave Exit Notify and the EDECCSSA User Leaf Function. > > > > Enclaves exit can be either synchronous and consensual (EEXIT for > > instance) or asynchronous (on an interrupt or fault). The > > asynchronous ones can evidently be exploited to single step > > enclaves[1], on top of which other naughty things can be built. > > > > AEX Notify will be made available both on upcoming processors and > > on some older processors through microcode updates. > > > > == The Problem == > > > > These attacks are currently entirely opaque to the enclave since > > the hardware does the save/restore under the covers. The > > Asynchronous Enclave Exit Notify (AEX Notify) mechanism provides > > enclaves an ability to detect and mitigate potential exposure to > > these kinds of attacks. > > > > == The Solution == > > > > Define the new attribute value for AEX Notification. Ensure the > > attribute is cleared from the list reserved attributes which > > allows it to be used in enclaves. > > > > I just built this and ran it to make sure there were no obvious > > regressions since I do not have the hardware to test it handy. > > Tested-by's would be much appreciated. > > Is this available on recent ucode updates e.g. for Icelake > or Geminilake? I mean it would not take me long to upgrade our exception handling flow to this. Then I can run our full test suite on it. But this will of course require ucode update for Icelake. AEX Notify will actually just simplify everything. We kind of simulate "AEX Notify" already with EENTER to execute in-enclave exception handler before doing actual ERESUME. BR, Jarkko
Hi Dave I think you need add this change in arch/x86/kernel/cpu/sgx/ioctl.c, inside sgx_encl_create function: - encl->attributes_mask = SGX_ATTR_DEBUG | SGX_ATTR_MODE64BIT | SGX_ATTR_KSS; + encl->attributes_mask = SGX_ATTR_DEBUG | SGX_ATTR_MODE64BIT | SGX_ATTR_KSS | SGX_ATTR_ASYNC_EXIT_NOTIFY; Otherwise, EINIT ioctl fails with this new attribute set in enclave secs. Thanks Haitao On Tue, 05 Jul 2022 13:36:48 -0500, Dave Hansen <dave.hansen@linux.intel.com> wrote: > Short Version: > > Allow enclaves to use the new Asynchronous EXit (AEX) > notification mechanism. This mechanism lets enclaves run a > handler after an AEX event. These handlers can run mitigations > for things like SGX-Step[1]. > > AEX Notify will be made available both on upcoming processors and > on some older processors through microcode updates. > > Long Version: > > == SGX Attribute Background == > > The SGX architecture includes a list of SGX "attributes". These > attributes ensure consistency and transparency around specific > enclave features. > > As a simple example, the "DEBUG" attribute allows an enclave to > be debugged, but also destroys virtually all of SGX security. > Using attributes, enclaves can know that they are being debugged. > Attributes also affect enclave attestation so an enclave can, for > instance, be denied access to secrets while it is being debugged. > > The kernel keeps a list of known attributes and will only > initialize enclaves that use a known set of attributes. This > kernel policy eliminates the chance that a new SGX attribute > could cause undesired effects. > > For example, imagine a new attribute was added called > "PROVISIONKEY2" that provided similar functionality to > "PROVISIIONKEY". A kernel policy that allowed indiscriminate use > of unknown attributes and thus PROVISIONKEY2 would undermine the > existing kernel policy which limits use of PROVISIONKEY enclaves. > > == AEX Notify Background == > > "Intel Architecture Instruction Set Extensions and Future > Features - Version 45" is out[2]. There is a new chapter: > > Asynchronous Enclave Exit Notify and the EDECCSSA User Leaf Function. > > Enclaves exit can be either synchronous and consensual (EEXIT for > instance) or asynchronous (on an interrupt or fault). The > asynchronous ones can evidently be exploited to single step > enclaves[1], on top of which other naughty things can be built. > > AEX Notify will be made available both on upcoming processors and > on some older processors through microcode updates. > > == The Problem == > > These attacks are currently entirely opaque to the enclave since > the hardware does the save/restore under the covers. The > Asynchronous Enclave Exit Notify (AEX Notify) mechanism provides > enclaves an ability to detect and mitigate potential exposure to > these kinds of attacks. > > == The Solution == > > Define the new attribute value for AEX Notification. Ensure the > attribute is cleared from the list reserved attributes which > allows it to be used in enclaves. > > I just built this and ran it to make sure there were no obvious > regressions since I do not have the hardware to test it handy. > Tested-by's would be much appreciated. > > 1. https://github.com/jovanbulck/sgx-step > 2. https://cdrdv2.intel.com/v1/dl/getContent/671368?explicitVersion=true > > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> > Cc: Jarkko Sakkinen <jarkko@kernel.org> > Cc: Andy Lutomirski <luto@kernel.org> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: Ingo Molnar <mingo@redhat.com> > Cc: Borislav Petkov <bp@alien8.de> > Cc: x86@kernel.org > Cc: "H. Peter Anvin" <hpa@zytor.com> > Cc: linux-sgx@vger.kernel.org > Cc: linux-kernel@vger.kernel.org > --- > arch/x86/include/asm/sgx.h | 25 ++++++++++++++++++------- > 1 file changed, 18 insertions(+), 7 deletions(-) > > diff --git a/arch/x86/include/asm/sgx.h b/arch/x86/include/asm/sgx.h > index 3f9334ef67cd..f7328d8efd83 100644 > --- a/arch/x86/include/asm/sgx.h > +++ b/arch/x86/include/asm/sgx.h > @@ -110,17 +110,28 @@ enum sgx_miscselect { > * %SGX_ATTR_EINITTOKENKEY: Allow to use token signing key that is used > to > * sign cryptographic tokens that can be passed to > * EINIT as an authorization to run an enclave. > + * %SGX_ATTR_ASYNC_EXIT_NOTIFY: Allow enclaves to be notified after an > + * asynchronous exit has occurred. > */ > enum sgx_attribute { > - SGX_ATTR_INIT = BIT(0), > - SGX_ATTR_DEBUG = BIT(1), > - SGX_ATTR_MODE64BIT = BIT(2), > - SGX_ATTR_PROVISIONKEY = BIT(4), > - SGX_ATTR_EINITTOKENKEY = BIT(5), > - SGX_ATTR_KSS = BIT(7), > + SGX_ATTR_INIT = BIT(0), > + SGX_ATTR_DEBUG = BIT(1), > + SGX_ATTR_MODE64BIT = BIT(2), > + /* BIT(3) is reserved */ > + SGX_ATTR_PROVISIONKEY = BIT(4), > + SGX_ATTR_EINITTOKENKEY = BIT(5), > + /* BIT(6) is for CET */ > + SGX_ATTR_KSS = BIT(7), > + /* BIT(8) is reserved */ > + /* BIT(9) is reserved */ > + SGX_ATTR_ASYNC_EXIT_NOTIFY = BIT(10), > }; > -#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | BIT_ULL(6) | > GENMASK_ULL(63, 8)) > +#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | \ > + BIT_ULL(6) | \ > + BIT_ULL(8) | \ > + BIT_ULL(9) | \ > + GENMASK_ULL(63, 11)) > /** > * struct sgx_secs - SGX Enclave Control Structure (SECS)
On 7/10/22 19:34, Jarkko Sakkinen wrote: > Is this available on recent ucode updates e.g. for Icelake > or Geminilake? I do not believe it is currently publicly available. It *really* needs some @intel.com Tested-by's.
On 7/14/22 06:19, Haitao Huang wrote: > > I think you need add this change in arch/x86/kernel/cpu/sgx/ioctl.c, > inside sgx_encl_create function: > > - encl->attributes_mask = SGX_ATTR_DEBUG | SGX_ATTR_MODE64BIT | > SGX_ATTR_KSS; > + encl->attributes_mask = SGX_ATTR_DEBUG | SGX_ATTR_MODE64BIT | > SGX_ATTR_KSS | SGX_ATTR_ASYNC_EXIT_NOTIFY; > > Otherwise, EINIT ioctl fails with this new attribute set in enclave secs. Makes sense. I'll add it there. I also just realized we need to update arch/x86/kvm/cpuid.c as well. It keeps a complete list of guest-available SGX attributes.
Oh, and, btw... I don't have any hardware that can actually test this patch. It can't be applied until I have an actual tested-by from someone with the hardware.
On Tue, 19 Jul 2022 12:54:41 -0500, Dave Hansen <dave.hansen@intel.com> wrote: > Oh, and, btw... I don't have any hardware that can actually test this > patch. It can't be applied until I have an actual tested-by from > someone with the hardware. I can ask someone in our team to test and report back results with Tested-by when you post updated patch. Thanks Haitao
On Tue, Jul 19, 2022 at 10:54:41AM -0700, Dave Hansen wrote: > Oh, and, btw... I don't have any hardware that can actually test this > patch. It can't be applied until I have an actual tested-by from > someone with the hardware. ... I bet that NUC7's will never get a patch BR, Jarkko
diff --git a/arch/x86/include/asm/sgx.h b/arch/x86/include/asm/sgx.h index 3f9334ef67cd..f7328d8efd83 100644 --- a/arch/x86/include/asm/sgx.h +++ b/arch/x86/include/asm/sgx.h @@ -110,17 +110,28 @@ enum sgx_miscselect { * %SGX_ATTR_EINITTOKENKEY: Allow to use token signing key that is used to * sign cryptographic tokens that can be passed to * EINIT as an authorization to run an enclave. + * %SGX_ATTR_ASYNC_EXIT_NOTIFY: Allow enclaves to be notified after an + * asynchronous exit has occurred. */ enum sgx_attribute { - SGX_ATTR_INIT = BIT(0), - SGX_ATTR_DEBUG = BIT(1), - SGX_ATTR_MODE64BIT = BIT(2), - SGX_ATTR_PROVISIONKEY = BIT(4), - SGX_ATTR_EINITTOKENKEY = BIT(5), - SGX_ATTR_KSS = BIT(7), + SGX_ATTR_INIT = BIT(0), + SGX_ATTR_DEBUG = BIT(1), + SGX_ATTR_MODE64BIT = BIT(2), + /* BIT(3) is reserved */ + SGX_ATTR_PROVISIONKEY = BIT(4), + SGX_ATTR_EINITTOKENKEY = BIT(5), + /* BIT(6) is for CET */ + SGX_ATTR_KSS = BIT(7), + /* BIT(8) is reserved */ + /* BIT(9) is reserved */ + SGX_ATTR_ASYNC_EXIT_NOTIFY = BIT(10), }; -#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | BIT_ULL(6) | GENMASK_ULL(63, 8)) +#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | \ + BIT_ULL(6) | \ + BIT_ULL(8) | \ + BIT_ULL(9) | \ + GENMASK_ULL(63, 11)) /** * struct sgx_secs - SGX Enclave Control Structure (SECS)
Short Version: Allow enclaves to use the new Asynchronous EXit (AEX) notification mechanism. This mechanism lets enclaves run a handler after an AEX event. These handlers can run mitigations for things like SGX-Step[1]. AEX Notify will be made available both on upcoming processors and on some older processors through microcode updates. Long Version: == SGX Attribute Background == The SGX architecture includes a list of SGX "attributes". These attributes ensure consistency and transparency around specific enclave features. As a simple example, the "DEBUG" attribute allows an enclave to be debugged, but also destroys virtually all of SGX security. Using attributes, enclaves can know that they are being debugged. Attributes also affect enclave attestation so an enclave can, for instance, be denied access to secrets while it is being debugged. The kernel keeps a list of known attributes and will only initialize enclaves that use a known set of attributes. This kernel policy eliminates the chance that a new SGX attribute could cause undesired effects. For example, imagine a new attribute was added called "PROVISIONKEY2" that provided similar functionality to "PROVISIIONKEY". A kernel policy that allowed indiscriminate use of unknown attributes and thus PROVISIONKEY2 would undermine the existing kernel policy which limits use of PROVISIONKEY enclaves. == AEX Notify Background == "Intel Architecture Instruction Set Extensions and Future Features - Version 45" is out[2]. There is a new chapter: Asynchronous Enclave Exit Notify and the EDECCSSA User Leaf Function. Enclaves exit can be either synchronous and consensual (EEXIT for instance) or asynchronous (on an interrupt or fault). The asynchronous ones can evidently be exploited to single step enclaves[1], on top of which other naughty things can be built. AEX Notify will be made available both on upcoming processors and on some older processors through microcode updates. == The Problem == These attacks are currently entirely opaque to the enclave since the hardware does the save/restore under the covers. The Asynchronous Enclave Exit Notify (AEX Notify) mechanism provides enclaves an ability to detect and mitigate potential exposure to these kinds of attacks. == The Solution == Define the new attribute value for AEX Notification. Ensure the attribute is cleared from the list reserved attributes which allows it to be used in enclaves. I just built this and ran it to make sure there were no obvious regressions since I do not have the hardware to test it handy. Tested-by's would be much appreciated. 1. https://github.com/jovanbulck/sgx-step 2. https://cdrdv2.intel.com/v1/dl/getContent/671368?explicitVersion=true Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Cc: Jarkko Sakkinen <jarkko@kernel.org> Cc: Andy Lutomirski <luto@kernel.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: Borislav Petkov <bp@alien8.de> Cc: x86@kernel.org Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: linux-sgx@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- arch/x86/include/asm/sgx.h | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)