Message ID | 20241008211727.49088-1-dorjoychy111@gmail.com (mailing list archive) |
---|---|
Headers | show |
Series | AWS Nitro Enclave emulation support | expand |
On 08.10.24 23:17, Dorjoy Chowdhury wrote: > This is v8 submission for AWS Nitro Enclave emulation in QEMU. From the QEMU side > the implementation for nitro enclaves is complete. v7 is at: > https://lore.kernel.org/qemu-devel/20240922094441.23802-1-dorjoychy111@gmail.com/T/#t > > Changes in v8: > - create_default_memdev related changes have been made into a separate commit > - PCR states are now saved during migration > - some changes in virtio-nsm.c for making the implementation similar to real > NSM in AWS which include returning the same error, ignoring unknown properties in > nested map etc. I got a chance to test real NSM in AWS and I prepared a script to > test various NSM operations and error conditions which I am maintaining a git repo > here: https://github.com/dorjoy03/nsm-api . The README has some details for testing. > > Changes in v7: > - rebased with master as the first 3 crypto patches got in master now > - had to change some QCRYPTO_HASH_ALG refs to QCRYPTO_HASH_ALGO (with an 'O') > as that got changed in master > - had to re-do the libvirt-ci update patch as I got a conflict from master > - renamed handle_* functions in virtio-nsm.c to be not camel case > > Changes in v6: > - updated MAINTAINERS in the commit that introduced eif.c, eif.h files > - used iov_to_buf and iov_from_buf in virtio-nsm.c handle_input > - used g_new0 for iovec_list instead of g_malloc > - updated documentation as now the vsock backend work[6] has been merged in > rust-vmm's vhost-device-vsock > > Changes in v5: > - bunch of use of glib utilities like g_autofree, g_memdup2, GList etc > - updated libvirt-ci and added libcbor dependency > - files in this patch are built under libcbor and gnutls dependency check now and > libcbor dependency has been moved to root meson.build file > - separated and re-ordered commits as suggested > - user_data and nonce are added as null to attestation when empty and payload_map_size is fixed > - variables in eif.c have been shortened for readability > > Changes in v4: > - fixed error_setv assertion failed. I could not reproduce this but I think > this was happening because I did not set Error *err = NULL in x86_load_eif > - qemu_cbor.. helpers moved to a separate file now > - libcbor version requirement reduced from 0.8.0 to 0.7.0 > - replaced GChecksum uses with qcrypto apis > - timestamp multiplied by 1000 in virtio-nsm > - user_data and nonce are now included in attestation even when they are empty > - added x509-utils in crypto > - added G_CHECKSUM_SHA384 support in hash-glib.c > - PCR3 and PCR4 can be set from nitro-enclave machine options. I did not add > the options for virtio-nsm device though. I think the PCR states are set by > machines so it made sense to add the options for machine only. > > Changes in v3: > - Support for virtio-nsm device > - The EIF related logic has been removed from microvm.c i.e., the logic is > contained in enclave related code > - For vsock emulation in nitro-enclave, now vhost-user-vsock is being used > instead of vhost-vsock (more details in the cover-letter below) > - updated documentation accordingly > > Changes in v2: > - moved eif.c and eif.h files from hw/i386 to hw/core > > Hi, > > Hope everyone is doing well. This is a patch series adding AWS Nitro Enclave[1] > emulation support in QEMU. Alexander Graf is mentoring me on this work. I have > a gitlab branch where you can view the patches in the gitlab web UI for each commit: > https://gitlab.com/dorjoy03/qemu/-/tree/nitro-enclave-emulation > > AWS nitro enclaves is an Amazon EC2[2] feature that allows creating isolated > execution environments, called enclaves, from Amazon EC2 instances, which are > used for processing highly sensitive data. Enclaves have no persistent storage > and no external networking. The enclave VMs are based on Firecracker microvm > and have a vhost-vsock device for communication with the parent EC2 instance > that spawned it and a Nitro Secure Module (NSM) device for cryptographic > attestation. The parent instance VM always has CID 3 while the enclave VM gets > a dynamic CID. The enclave VMs can communicate with the parent instance over > various ports to CID 3, for example, the init process inside an enclave sends a > heartbeat to port 9000 upon boot, expecting a heartbeat reply, letting the > parent instance know that the enclave VM has successfully booted. > > From inside an EC2 instance, nitro-cli[3] is used to spawn an enclave VM using > an EIF (Enclave Image Format)[4] file. EIF files can be built using nitro-cli > as well. The EIF specification can be found in the README of the github > aws-nitro-enclaves-image-format repository[4]. An EIF file contains the kernel, > cmdline and ramdisk(s) in different sections which are used to boot the enclave > VM. > > Adding nitro enclave emulation support in QEMU will make the life of AWS Nitro > Enclave users easier as they will be able to test their EIF images locally > without having to run real nitro enclaves which can be difficult for debugging > due to its roots in security. This will also make quick prototyping easier. > > In QEMU, the new nitro-enclave machine type is implemented based on the microvm > machine type similar to how AWS Nitro Enclaves are based on Firecracker microvm. > > The vsock emulation support is added using vhost-user-vsock device. This is > needed as nitro VMs always talk to parent VM (CID 3) but there is no support for > sibling VM communication in vhost-vsock. So to run nitro-enclave, a process that > does vsock emulation in user-space like vhost-device-vsock[5] from rust-vmm must > be run. I am working on adding proxying using vsock (right now it uses unix > domain socket) to the host machine in vhost-device-vsock which I have already > posted a PR[6] in rust-vmm repo. This will allow users to run the necessary parent > VM applications in the host machine instead of a separate VM with CID 3. Update: > this has been merged now. > > A new device virtio-nsm support has been added to QEMU. This device is built-into > the nitro-enclave VM. The virtio-nsm spec can be found here[7]. > > For local testing you need to generate a hello.eif image by first building > nitro-cli locally[8]. Then you can use nitro-cli to build a hello.eif image[9]. > More details about testing can be found in the docs/system/i386/nitro-enclave.rst > file. Reviewed-by: Alexander Graf <graf@amazon.com> Alex Amazon Web Services Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B Sitz: Berlin Ust-ID: DE 365 538 597
Ping This patch series has been reviewed by Alex. I am not sure if it needs more review. If not, maybe this can be picked up for merging. Thanks! Regards, Dorjoy
On Wed, Oct 16, 2024 at 7:58 PM Dorjoy Chowdhury <dorjoychy111@gmail.com> wrote: > > Ping > > This patch series has been reviewed by Alex. I am not sure if it needs > more review. If not, maybe this can be picked up for merging. Thanks! > Gentle ping. This patch series has been reviewed by Alex and there hasn't been any more reviews. it would be great if this could be picked up for merging. Thanks! Regards, Dorjoy
On 10/23/24 16:27, Dorjoy Chowdhury wrote: > On Wed, Oct 16, 2024 at 7:58 PM Dorjoy Chowdhury <dorjoychy111@gmail.com> wrote: >> >> Ping >> >> This patch series has been reviewed by Alex. I am not sure if it needs >> more review. If not, maybe this can be picked up for merging. Thanks! >> > > Gentle ping. > > This patch series has been reviewed by Alex and there hasn't been any > more reviews. it would be great if this could be picked up for > merging. Thanks! Hi, sorry about the delay -- the patches failed CI and I didn't have much time to investigate until now. The issues are basically: 1) some rST syntax errors 2) failures on non-Linux due to lack of VHOST_USER 3) failures on 32-bit due to uint64_t/long mismatch. While fixing (2) I also moved the dependency on libcbor and gnutls from meson to Kconfig, and added --enable-libcbor to configure. I also split hw/core/eif.c to a separate symbol, just to simplify reproducing the 32-bit failure on the right commit. And finally, VIRTIO_NSM should default to no (the nitro-enclave machne takes care of selecting it). No big deal; it's easier done than described. See attached patch for the differences. Paolo
Hi Paolo, On Wed, Oct 30, 2024 at 1:32 AM Paolo Bonzini <pbonzini@redhat.com> wrote: > > On 10/23/24 16:27, Dorjoy Chowdhury wrote: > > On Wed, Oct 16, 2024 at 7:58 PM Dorjoy Chowdhury <dorjoychy111@gmail.com> wrote: > >> > >> Ping > >> > >> This patch series has been reviewed by Alex. I am not sure if it needs > >> more review. If not, maybe this can be picked up for merging. Thanks! > >> > > > > Gentle ping. > > > > This patch series has been reviewed by Alex and there hasn't been any > > more reviews. it would be great if this could be picked up for > > merging. Thanks! > > Hi, > > sorry about the delay -- the patches failed CI and I didn't have much > time to investigate until now. > > The issues are basically: > > 1) some rST syntax errors > > 2) failures on non-Linux due to lack of VHOST_USER > > 3) failures on 32-bit due to uint64_t/long mismatch. > > > While fixing (2) I also moved the dependency on libcbor and gnutls from > meson to Kconfig, and added --enable-libcbor to configure. I also split > hw/core/eif.c to a separate symbol, just to simplify reproducing the > 32-bit failure on the right commit. > > And finally, VIRTIO_NSM should default to no (the nitro-enclave machne > takes care of selecting it). > > No big deal; it's easier done than described. See attached patch for > the differences. > Thanks for fixing. The attached patch looks great to me. I just have one suggestion. Now that the CONFIG_* symbols have the dependencies listed explicitly in the Kconfig files, maybe we don't need the explicit dependencies in the meson.build files? For example, the following line in hw/core/meson.build file: system_ss.add(when: 'CONFIG_EIF', if_true: [files('eif.c'), zlib, libcbor, gnutls]) can be changed to: system_ss.add(when: 'CONFIG_EIF', if_true: [files('eif.c')]) I am not sure if zlib is a required dependency for QEMU, probably not needed to be listed above as well. I am just guessing. Same goes for files added in hw/virtio/meson.build. Thanks! Regards, Dorjoy
On Tue, Oct 29, 2024 at 9:08 PM Dorjoy Chowdhury <dorjoychy111@gmail.com> wrote: > Thanks for fixing. The attached patch looks great to me. I just have > one suggestion. Now that the CONFIG_* symbols have the dependencies > listed explicitly in the Kconfig files, maybe we don't need the > explicit dependencies in the meson.build files? For example, the > following line in hw/core/meson.build file: > system_ss.add(when: 'CONFIG_EIF', if_true: [files('eif.c'), zlib, > libcbor, gnutls]) > can be changed to: > system_ss.add(when: 'CONFIG_EIF', if_true: [files('eif.c')]) > > I am not sure if zlib is a required dependency for QEMU, probably not > needed to be listed above as well. I am just guessing. No, because the dependencies are not automatically added to all compiler and linker commands. Having them in the "add" call lets the compiler find include files and the linker add the dependency to the executable. As an aside, if foo.found() system_ss.add(files('x.c')) endif can be written system_ss.add(when: foo, if_true: files('x.c')) and "when:" supports multiple entries. But in this case it's okay to put it only in "if_true", since the dependency is handled in the Kconfig files and guaranteed to be present. Paolo
On Wed, Oct 30, 2024, 1:03 PM Paolo Bonzini <pbonzini@redhat.com> wrote: > On Tue, Oct 29, 2024 at 9:08 PM Dorjoy Chowdhury <dorjoychy111@gmail.com> > wrote: > > Thanks for fixing. The attached patch looks great to me. I just have > > one suggestion. Now that the CONFIG_* symbols have the dependencies > > listed explicitly in the Kconfig files, maybe we don't need the > > explicit dependencies in the meson.build files? For example, the > > following line in hw/core/meson.build file: > > system_ss.add(when: 'CONFIG_EIF', if_true: [files('eif.c'), zlib, > > libcbor, gnutls]) > > can be changed to: > > system_ss.add(when: 'CONFIG_EIF', if_true: [files('eif.c')]) > > > > I am not sure if zlib is a required dependency for QEMU, probably not > > needed to be listed above as well. I am just guessing. > > No, because the dependencies are not automatically added to all > compiler and linker commands. Having them in the "add" call lets the > compiler find include files and the linker add the dependency to the > executable. > > As an aside, > > if foo.found() > system_ss.add(files('x.c')) > endif > > can be written > > system_ss.add(when: foo, if_true: files('x.c')) > > and "when:" supports multiple entries. But in this case it's okay to > put it only in "if_true", since the dependency is handled in the > Kconfig files and guaranteed to be present. > Understood. All good then. Thanks! Regards, Dorjoy
Hey Paolo, On 29.10.24 20:32, Paolo Bonzini wrote: > > On 10/23/24 16:27, Dorjoy Chowdhury wrote: >> On Wed, Oct 16, 2024 at 7:58 PM Dorjoy Chowdhury >> <dorjoychy111@gmail.com> wrote: >>> >>> Ping >>> >>> This patch series has been reviewed by Alex. I am not sure if it needs >>> more review. If not, maybe this can be picked up for merging. Thanks! >>> >> >> Gentle ping. >> >> This patch series has been reviewed by Alex and there hasn't been any >> more reviews. it would be great if this could be picked up for >> merging. Thanks! > > Hi, > > sorry about the delay -- the patches failed CI and I didn't have much > time to investigate until now. > > The issues are basically: > > 1) some rST syntax errors > > 2) failures on non-Linux due to lack of VHOST_USER > > 3) failures on 32-bit due to uint64_t/long mismatch. > > > While fixing (2) I also moved the dependency on libcbor and gnutls from > meson to Kconfig, and added --enable-libcbor to configure. I also split > hw/core/eif.c to a separate symbol, just to simplify reproducing the > 32-bit failure on the right commit. > > And finally, VIRTIO_NSM should default to no (the nitro-enclave machne > takes care of selecting it). > > No big deal; it's easier done than described. See attached patch for > the differences. Thanks a bunch for looking deeply and fixing the issues above :). I suppose the next step is to post v9 with your changes included? Alex Amazon Web Services Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B Sitz: Berlin Ust-ID: DE 365 538 597
On 10/30/24 09:43, Alexander Graf wrote: >> Hi, >> >> sorry about the delay -- the patches failed CI and I didn't have much >> time to investigate until now. >> >> The issues are basically: >> >> 1) some rST syntax errors >> >> 2) failures on non-Linux due to lack of VHOST_USER >> >> 3) failures on 32-bit due to uint64_t/long mismatch. >> >> While fixing (2) I also moved the dependency on libcbor and gnutls from >> meson to Kconfig, and added --enable-libcbor to configure. I also split >> hw/core/eif.c to a separate symbol, just to simplify reproducing the >> 32-bit failure on the right commit. >> >> And finally, VIRTIO_NSM should default to no (the nitro-enclave machne >> takes care of selecting it). >> >> No big deal; it's easier done than described. See attached patch for >> the differences. > > Thanks a bunch for looking deeply and fixing the issues above :). I > suppose the next step is to post v9 with your changes included? No, I can see that it wasn't clear but I've queued it. :) Paolo
On Wed, Oct 30, 2024, 5:31 PM Paolo Bonzini <pbonzini@redhat.com> wrote: > On 10/30/24 09:43, Alexander Graf wrote: > >> Hi, > >> > >> sorry about the delay -- the patches failed CI and I didn't have much > >> time to investigate until now. > >> > >> The issues are basically: > >> > >> 1) some rST syntax errors > >> > >> 2) failures on non-Linux due to lack of VHOST_USER > >> > >> 3) failures on 32-bit due to uint64_t/long mismatch. > >> > >> While fixing (2) I also moved the dependency on libcbor and gnutls from > >> meson to Kconfig, and added --enable-libcbor to configure. I also split > >> hw/core/eif.c to a separate symbol, just to simplify reproducing the > >> 32-bit failure on the right commit. > >> > >> And finally, VIRTIO_NSM should default to no (the nitro-enclave machne > >> takes care of selecting it). > >> > >> No big deal; it's easier done than described. See attached patch for > >> the differences. > > > > Thanks a bunch for looking deeply and fixing the issues above :). I > > suppose the next step is to post v9 with your changes included? > > No, I can see that it wasn't clear but I've queued it. :) > Thank you for fixing the errors and queueing this, Paolo. Regards, Dorjoy