| Message ID | 20171113194528.28557-12-jarkko.sakkinen@linux.intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Mon, 2017-11-13 at 21:45 +0200, Jarkko Sakkinen wrote: > Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com> > --- > Documentation/index.rst | 1 + > Documentation/x86/intel_sgx.rst | 131 > ++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 132 insertions(+) > create mode 100644 Documentation/x86/intel_sgx.rst > > diff --git a/Documentation/index.rst b/Documentation/index.rst > index cb7f1ba5b3b1..ccfebc260e04 100644 > --- a/Documentation/index.rst > +++ b/Documentation/index.rst > @@ -86,6 +86,7 @@ implementation. > :maxdepth: 2 > > sh/index > + x86/index > > Korean translations > ------------------- > diff --git a/Documentation/x86/intel_sgx.rst > b/Documentation/x86/intel_sgx.rst > new file mode 100644 > index 000000000000..34bcf6a2a495 > --- /dev/null > +++ b/Documentation/x86/intel_sgx.rst > @@ -0,0 +1,131 @@ > +=================== > +Intel(R) SGX driver > +=================== > + > +Introduction > +============ > + > +Intel(R) SGX is a set of CPU instructions that can be used by > applications to > +set aside private regions of code and data. The code outside the > enclave is > +disallowed to access the memory inside the enclave by the CPU access > control. > +In a way you can think that SGX provides inverted sandbox. It > protects the > +application from a malicious host. > + > +There is a new hardware unit in the processor called Memory > Encryption Engine > +(MEE) starting from the Skylake microarchitecture. BIOS can define > one or many > +MEE regions that can hold enclave data by configuring them with > PRMRR registers. > + > +The MEE automatically encrypts the data leaving the processor > package to the MEE > +regions. The data is encrypted using a random key whose life-time is > exactly one > +power cycle. Not sure whether you should talk about MEE staff here. They are not in SDM and (thus) may potentially be changed in the future. > + > +You can tell if your CPU supports SGX by looking into > ``/proc/cpuinfo``: > + > + ``cat /proc/cpuinfo | grep sgx`` > + > +Enclave data types > +================== > + > +SGX defines new data types to maintain information about the > enclaves and their > +security properties. > + > +The following data structures exist in MEE regions: > + > +* **Enclave Page Cache (EPC):** memory pages for protected code and > data > +* **Enclave Page Cache Map (EPCM):** meta-data for each EPC page > + > +The Enclave Page Cache holds following types of pages: > + > +* **SGX Enclave Control Structure (SECS)**: meta-data defining the > global > + properties of an enclave such as range of addresses it can access. > +* **Regular (REG):** containing code and data for the enclave. > +* **Thread Control Structure (TCS):** defines an entry point for a > hardware > + thread to enter into the enclave. The enclave can only be entered > through > + these entry points. > +* **Version Array (VA)**: an EPC page receives a unique 8 byte > version number > + when it is swapped, which is then stored into a VA page. A VA page > can hold up > + to 512 version numbers. > + > +Launch control > +============== > + > +For launching an enclave, two structures must be provided for > ENCLS(EINIT): > + > +1. **SIGSTRUCT:** a signed measurement of the enclave binary. > +2. **EINITTOKEN:** the measurement, the public key of the signer and > various > + enclave attributes. This structure contains a MAC of its contents > using > + hardware derived symmetric key called *launch key*. > + > +The hardware platform contains a root key pair for signing the > SIGTRUCT > +for a *launch enclave* that is able to acquire the *launch key* for > +creating EINITTOKEN's for other enclaves. For the launch enclave > +EINITTOKEN is not needed because it is signed with the private root > key. > + > +There are two feature control bits associate with launch control > + > +* **IA32_FEATURE_CONTROL[0]**: locks down the feature control > register > +* **IA32_FEATURE_CONTROL[17]**: allow runtime reconfiguration of > + IA32_SGXLEPUBKEYHASHn MSRs that define MRSIGNER hash for the > launch > + enclave. Essentially they define a signing key that does not > require > + EINITTOKEN to be let run. > + > +The BIOS can configure IA32_SGXLEPUBKEYHASHn MSRs before feature > control > +register is locked. > + > +It could be tempting to implement launch control by writing the MSRs > +every time when an enclave is launched. This does not scale because > for > +generic case because BIOS might lock down the MSRs before handover > to > +the OS. > + > +Debug enclaves > +-------------- > + > +Enclave can be set as a *debug enclave* of which memory can be read > or written > +by using the ENCLS(EDBGRD) and ENCLS(EDBGWR) opcodes. The Intel > provided launch > +enclave provides them always a valid EINITTOKEN and therefore they > are a low > +hanging fruit way to try out SGX. > + > +Virtualization > +============== > + > +Launch control > +-------------- > + > +The values for IA32_SGXLEPUBKEYHASHn MSRs cannot be emulated for a > virtual > +machine guest. It would easily seem feasible to hold virtual values > for these > +MSRs, trap ENCLS(EINIT) and use the host LE to generate a token when > a guest LE > +is initialized. IMHO, this statement doesn't make sense to me because of: - Guest LE doesn't need a token. Guest LE is nothing different from Host LE from HW's point of view. - Host LE won't (and shouldn't) generate token for any enclave from guest. Guset LE generates token for enclaves in the guest, just like host LE generates token for enclaves in the host (but not guest). - In fact, theoretically KVM guest can still run LE and other enclaves without entire host SGX driver. There's no dependency between host LE and guest enclaves. > + > +However, looking at the pseudo code of ENCLS(EINIT) from the SDM > there is a > +constraint that the instruction will fail if > ATTRIBUTES.EINITTOKENKEY is set > +(the documentation does not tell the reason why the constraint > exists but it > +exists). Don't understand what you mean. This statement in SDM has nothing to do with virtualization. It's just description of hehavior of EINIT. > + > +Thus, only when the MSRs are left unlocked before handover to the OS > the > +setting of these MSRs can be supported for VM guests. IMHO you can just remove this "virtualization" section entirely as your first upstreaming driver won't consider virtualization at all. Thanks, -Kai > + > +Suspend and resume > +------------------ > + > +If the host suspends and resumes, the enclave memory for the VM > guest could > +become invalid. This can make ENCLS leaf operations suddenly fail. > + > +The driver has a graceful fallback mechanism to manage this > situation. If any of > +the ENCLS leaf operations fail, the driver will fallback by kicking > threads out > +of the enclave, removing the TCS entries and marking enclave as > invalid. After > +this no new pages can be allocated for the enclave and no entry can > be done. > + > +SGX uapi > +======== > + > +.. kernel-doc:: drivers/platform/x86/intel_sgx_ioctl.c > + :functions: sgx_ioc_enclave_create > + sgx_ioc_enclave_add_page > + sgx_ioc_enclave_init > + > +.. kernel-doc:: arch/x86/include/uapi/asm/sgx.h > + > +References > +========== > + > +* System Programming Manual: 39.1.4 Intel® SGX Launch Control > Configuration > _______________________________________________ > intel-sgx-kernel-dev mailing list > intel-sgx-kernel-dev@lists.01.org > https://lists.01.org/mailman/listinfo/intel-sgx-kernel-dev
On Tue, Nov 14, 2017 at 04:01:05PM +1300, Kai Huang wrote: > Not sure whether you should talk about MEE staff here. They are not in > SDM and (thus) may potentially be changed in the future. The use of MEE is documented in many places like https://eprint.iacr.org/2016/204.pdf I see no reason not to talk about MEE. > IMHO you can just remove this "virtualization" section entirely as your > first upstreaming driver won't consider virtualization at all. > > Thanks, > -Kai Makes sense. /Jarkko
On Tue, Nov 14, 2017 at 09:36:47AM +0100, Borislav Petkov wrote: > > +* **IA32_FEATURE_CONTROL[0]**: locks down the feature control register > > +* **IA32_FEATURE_CONTROL[17]**: allow runtime reconfiguration of > > + IA32_SGXLEPUBKEYHASHn MSRs that define MRSIGNER hash for the launch > > + enclave. Essentially they define a signing key that does not require > > + EINITTOKEN to be let run. > [SNIP] > > What does that mean exactly? > > OEM vendor BIOS can control how many enclaves user can launch and what > signing key is loaded and lock down the feature control register so that > no other signing keys are loaded? > > Or am I misreading this? Pre-boot firmware could potentially configure the root key hash for the enclave that signs launch tokens for other enclaves i.e. the launch enclave that is built and signed during the kbuild. VMM/Hypervisor could potentially simulate such enviroment for guests. In these cases IA32_FEATURE_CONTROL[17] would be zeroed before locking the feature control, which would mean that the kernel could not write new values with wrmsr for the root key hash. You could use this to write policy inside the LE. The current LE is really simple "pass-through everything" but it could be refined to be something more non-trivial. The question is whether we want to allow this or not. If the answer is no, a check can be added to the driver initialization code whether 17 is set, and if not, it driver would fail to initialize. I'm not too dilated to any direction concerning this question. I just wrote a driver that is robust. This is where I need your feedback to do the right thing. > -- > Regards/Gruss, > Boris. PS. I'll add a proper commit message. Sorry for not doing that before. /Jarkko
On Tue, 2017-11-14 at 21:47 +0200, Jarkko Sakkinen wrote: > On Tue, Nov 14, 2017 at 04:01:05PM +1300, Kai Huang wrote: > > Not sure whether you should talk about MEE staff here. They are not > > in > > SDM and (thus) may potentially be changed in the future. > > The use of MEE is documented in many places like > > https://eprint.iacr.org/2016/204.pdf > > I see no reason not to talk about MEE. Sure. Thanks for pointing out the paper. > > > IMHO you can just remove this "virtualization" section entirely as > > your > > first upstreaming driver won't consider virtualization at all. > > > > Thanks, > > -Kai > > Makes sense. Thanks. Thanks, -Kai > > /Jarkko
On Tue, Nov 14, 2017 at 10:53:27PM +0100, Borislav Petkov wrote: > On Tue, Nov 14, 2017 at 10:49:48PM +0200, Jarkko Sakkinen wrote: > > Pre-boot firmware could potentially configure the root key hash for the > > enclave that signs launch tokens for other enclaves i.e. the launch > > enclave that is built and signed during the kbuild. > > So how about firmware doesn't do anything and the machine owner decide > what enclaves get launched and what key hashes to load for a change? > I.e., let the owner really own the hardware she paid money for. > > Or are we doing encrypted enclaves but then the firmware vendor can look > inside too? > > -- > Regards/Gruss, > Boris. Firmware cannot access the memory inside an enclave. CPU asserts every memory access coming outside the enclave. /Jarkko
On Mon, Nov 20, 2017 at 11:42:56PM +0100, Borislav Petkov wrote: > On Tue, Nov 21, 2017 at 12:37:41AM +0200, Jarkko Sakkinen wrote: > > Firmware cannot access the memory inside an enclave. CPU asserts every > > memory access coming outside the enclave. > > But "firmware could potentially configure the root key hash for the > enclave." How about the owner configures the root key hash instead? > Along with deciding whether to lock down the feature control register or In potential deployments of SGX, the owner could do this either in the firmware level or OS level depending whether the MSRs are configured as writable in the feature control. One option would be to have a config flag to decide whether to require MSRs to be writable or not. /Jarkko
On Fri, Nov 17, 2017 at 01:43:10PM -0800, Darren Hart wrote: > This series will need to be updated per the comments received so far, as > well as with commit messages and a complete Cc list *per patch* giving > all required parties an opportunity to review. > > With respect to the obvious security nature of this series, who from the > kernel security folks are going to be reviewing this? > security@kernel.org? I think it would make sense to CC this to linux-security module. > Cc updated for this thread, and specifically the question regarding > location below: > > > Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com> > > --- > > Documentation/index.rst | 1 + > > Documentation/x86/intel_sgx.rst | 131 ++++++++++++++++++++++++++++++++++++++++ > > 2 files changed, 132 insertions(+) > > create mode 100644 Documentation/x86/intel_sgx.rst > > > > ... > > > diff --git a/Documentation/x86/intel_sgx.rst b/Documentation/x86/intel_sgx.rst > > new file mode 100644 > > index 000000000000..34bcf6a2a495 > > --- /dev/null > > +++ b/Documentation/x86/intel_sgx.rst > > @@ -0,0 +1,131 @@ > > +=================== > > +Intel(R) SGX driver > > +=================== > > + > > +Introduction > > +============ > > + > > +Intel(R) SGX is a set of CPU instructions that can be used by applications to > > +set aside private regions of code and data. The code outside the enclave is > > +disallowed to access the memory inside the enclave by the CPU access control. > > +In a way you can think that SGX provides inverted sandbox. It protects the > > +application from a malicious host. > > + > > +There is a new hardware unit in the processor called Memory Encryption Engine > > +(MEE) starting from the Skylake microarchitecture. BIOS can define one or many > > +MEE regions that can hold enclave data by configuring them with PRMRR registers. > > + > > +The MEE automatically encrypts the data leaving the processor package to the MEE > > +regions. The data is encrypted using a random key whose life-time is exactly one > > +power cycle. > > + > > +You can tell if your CPU supports SGX by looking into ``/proc/cpuinfo``: > > + > > + ``cat /proc/cpuinfo | grep sgx`` > > Is SGX considered architectural or not? A quick search of the SDM > includes it in Volume 3: > > Volume 3: Includes the full system programming guide, parts 1, 2, 3, and > 4. Describes the operating-system support environment of Intel® 64 and > IA-32 architectures, including: memory management, protection, task > management, interrupt and exception handling, multi-processor support, > thermal and power management features, debugging, performance > monitoring, system management mode, virtual machine extensions (VMX) > instructions, Intel® Virtualization Technology (Intel® VT), and Intel® > Software Guard Extensions (Intel® SGX). > > https://software.intel.com/en-us/articles/intel-sdm > > Depending on the answer, this impacts whether this belongs in > drivers/platform/x86 or arch/x86/platform per our recent agreement with > Thomas. > > Thomas, Mingo, HPA, do you wish to see this organized/located > differently than it is here in v5? The code is made easily relocatable. I just wanted to keep it as an encapsulated driver up until I hear the maintainer feedback. I'll submit v6 with code otherwise fixed according to the feedback that I've heard up until that point and relocate it in v7 based on your feedback. > > +Launch control > > +============== > > + > > +For launching an enclave, two structures must be provided for ENCLS(EINIT): > > + > > +1. **SIGSTRUCT:** a signed measurement of the enclave binary. > > +2. **EINITTOKEN:** the measurement, the public key of the signer and various > > + enclave attributes. This structure contains a MAC of its contents using > > + hardware derived symmetric key called *launch key*. > > + > > +The hardware platform contains a root key pair for signing the SIGTRUCT > > +for a *launch enclave* that is able to acquire the *launch key* for > > +creating EINITTOKEN's for other enclaves. For the launch enclave > > +EINITTOKEN is not needed because it is signed with the private root key. > > + > > +There are two feature control bits associate with launch control > > Nit: missing colon at the end of the line above ^ Yes. Thanks for spotting that out :-) /Jarkko
diff --git a/Documentation/index.rst b/Documentation/index.rst index cb7f1ba5b3b1..ccfebc260e04 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -86,6 +86,7 @@ implementation. :maxdepth: 2 sh/index + x86/index Korean translations ------------------- diff --git a/Documentation/x86/intel_sgx.rst b/Documentation/x86/intel_sgx.rst new file mode 100644 index 000000000000..34bcf6a2a495 --- /dev/null +++ b/Documentation/x86/intel_sgx.rst @@ -0,0 +1,131 @@ +=================== +Intel(R) SGX driver +=================== + +Introduction +============ + +Intel(R) SGX is a set of CPU instructions that can be used by applications to +set aside private regions of code and data. The code outside the enclave is +disallowed to access the memory inside the enclave by the CPU access control. +In a way you can think that SGX provides inverted sandbox. It protects the +application from a malicious host. + +There is a new hardware unit in the processor called Memory Encryption Engine +(MEE) starting from the Skylake microarchitecture. BIOS can define one or many +MEE regions that can hold enclave data by configuring them with PRMRR registers. + +The MEE automatically encrypts the data leaving the processor package to the MEE +regions. The data is encrypted using a random key whose life-time is exactly one +power cycle. + +You can tell if your CPU supports SGX by looking into ``/proc/cpuinfo``: + + ``cat /proc/cpuinfo | grep sgx`` + +Enclave data types +================== + +SGX defines new data types to maintain information about the enclaves and their +security properties. + +The following data structures exist in MEE regions: + +* **Enclave Page Cache (EPC):** memory pages for protected code and data +* **Enclave Page Cache Map (EPCM):** meta-data for each EPC page + +The Enclave Page Cache holds following types of pages: + +* **SGX Enclave Control Structure (SECS)**: meta-data defining the global + properties of an enclave such as range of addresses it can access. +* **Regular (REG):** containing code and data for the enclave. +* **Thread Control Structure (TCS):** defines an entry point for a hardware + thread to enter into the enclave. The enclave can only be entered through + these entry points. +* **Version Array (VA)**: an EPC page receives a unique 8 byte version number + when it is swapped, which is then stored into a VA page. A VA page can hold up + to 512 version numbers. + +Launch control +============== + +For launching an enclave, two structures must be provided for ENCLS(EINIT): + +1. **SIGSTRUCT:** a signed measurement of the enclave binary. +2. **EINITTOKEN:** the measurement, the public key of the signer and various + enclave attributes. This structure contains a MAC of its contents using + hardware derived symmetric key called *launch key*. + +The hardware platform contains a root key pair for signing the SIGTRUCT +for a *launch enclave* that is able to acquire the *launch key* for +creating EINITTOKEN's for other enclaves. For the launch enclave +EINITTOKEN is not needed because it is signed with the private root key. + +There are two feature control bits associate with launch control + +* **IA32_FEATURE_CONTROL[0]**: locks down the feature control register +* **IA32_FEATURE_CONTROL[17]**: allow runtime reconfiguration of + IA32_SGXLEPUBKEYHASHn MSRs that define MRSIGNER hash for the launch + enclave. Essentially they define a signing key that does not require + EINITTOKEN to be let run. + +The BIOS can configure IA32_SGXLEPUBKEYHASHn MSRs before feature control +register is locked. + +It could be tempting to implement launch control by writing the MSRs +every time when an enclave is launched. This does not scale because for +generic case because BIOS might lock down the MSRs before handover to +the OS. + +Debug enclaves +-------------- + +Enclave can be set as a *debug enclave* of which memory can be read or written +by using the ENCLS(EDBGRD) and ENCLS(EDBGWR) opcodes. The Intel provided launch +enclave provides them always a valid EINITTOKEN and therefore they are a low +hanging fruit way to try out SGX. + +Virtualization +============== + +Launch control +-------------- + +The values for IA32_SGXLEPUBKEYHASHn MSRs cannot be emulated for a virtual +machine guest. It would easily seem feasible to hold virtual values for these +MSRs, trap ENCLS(EINIT) and use the host LE to generate a token when a guest LE +is initialized. + +However, looking at the pseudo code of ENCLS(EINIT) from the SDM there is a +constraint that the instruction will fail if ATTRIBUTES.EINITTOKENKEY is set +(the documentation does not tell the reason why the constraint exists but it +exists). + +Thus, only when the MSRs are left unlocked before handover to the OS the +setting of these MSRs can be supported for VM guests. + +Suspend and resume +------------------ + +If the host suspends and resumes, the enclave memory for the VM guest could +become invalid. This can make ENCLS leaf operations suddenly fail. + +The driver has a graceful fallback mechanism to manage this situation. If any of +the ENCLS leaf operations fail, the driver will fallback by kicking threads out +of the enclave, removing the TCS entries and marking enclave as invalid. After +this no new pages can be allocated for the enclave and no entry can be done. + +SGX uapi +======== + +.. kernel-doc:: drivers/platform/x86/intel_sgx_ioctl.c + :functions: sgx_ioc_enclave_create + sgx_ioc_enclave_add_page + sgx_ioc_enclave_init + +.. kernel-doc:: arch/x86/include/uapi/asm/sgx.h + +References +========== + +* System Programming Manual: 39.1.4 Intel® SGX Launch Control Configuration
Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com> --- Documentation/index.rst | 1 + Documentation/x86/intel_sgx.rst | 131 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 132 insertions(+) create mode 100644 Documentation/x86/intel_sgx.rst