| Message ID | 20230106075330.3662549-4-clg@kaod.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | s390x/pv: Improve protected VM support | expand |
On 1/6/23 08:53, Cédric Le Goater wrote: > From: Cédric Le Goater <clg@redhat.com> > > If a secure kernel is started in a non-protected VM, the OS will hang > during boot without giving a proper error message to the user. Most of the time you see nothing in the console because libvirt is too slow. If you start the VM in paused mode, attach a console and then resume it, then you'll see a nice error message. > > Perform the checks on Confidential Guest support at runtime with an > helper called from the service call switching the guest to protected > mode. If we don't have PV support then the subcodes >=8 are a specification exception so this is never executed AFAIK. > if (env->psw.mask & PSW_MASK_PSTATE) { > s390_program_interrupt(env, PGM_PRIVILEGED, ra); > @@ -176,6 +177,12 @@ out: > return; > } > > + if (!s390_pv_check(&local_err)) { > + error_report_err(local_err); > + env->regs[r1 + 1] = DIAG_308_RC_INVAL_FOR_PV; > + return; > + } > + > s390_ipl_reset_request(cs, S390_RESET_PV); > break; > default:
On 1/9/23 10:04, Janosch Frank wrote: > On 1/6/23 08:53, Cédric Le Goater wrote: >> From: Cédric Le Goater <clg@redhat.com> >> >> If a secure kernel is started in a non-protected VM, the OS will hang >> during boot without giving a proper error message to the user. > > Most of the time you see nothing in the console because libvirt is too slow. If you start the VM in paused mode, attach a console and then resume it, then you'll see a nice error message. If you wait long enough, the VM fails to mount / and falls into the dracut initrams. >> Perform the checks on Confidential Guest support at runtime with an >> helper called from the service call switching the guest to protected >> mode. > > If we don't have PV support then the subcodes >=8 are a specification exception > so this is never executed AFAIK. It is. The test on huge page size was added just above this change. Thanks, C. > >> if (env->psw.mask & PSW_MASK_PSTATE) { >> s390_program_interrupt(env, PGM_PRIVILEGED, ra); >> @@ -176,6 +177,12 @@ out: >> return; >> } >> + if (!s390_pv_check(&local_err)) { >> + error_report_err(local_err); >> + env->regs[r1 + 1] = DIAG_308_RC_INVAL_FOR_PV; >> + return; >> + } >> + >> s390_ipl_reset_request(cs, S390_RESET_PV); >> break; >> default: >
On 1/9/23 10:27, Cédric Le Goater wrote: > On 1/9/23 10:04, Janosch Frank wrote: >> On 1/6/23 08:53, Cédric Le Goater wrote: >>> From: Cédric Le Goater <clg@redhat.com> >>> >>> If a secure kernel is started in a non-protected VM, the OS will hang >>> during boot without giving a proper error message to the user. >> >> Most of the time you see nothing in the console because libvirt is too slow. If you start the VM in paused mode, attach a console and then resume it, then you'll see a nice error message. > > If you wait long enough, the VM fails to mount / and falls into the dracut > initrams. I have the feeling that we're not talking about the same thing here. A PV VM always starts out as a non-PV VM and is put into PV mode via two diag308 subcodes (8 & 10). ALL PV subcodes (8 - 10) are spec exceptions if the host isn't enabled for PV. There is no way to run a secure image in a non-PV environment. What's being run is a non-secure bootloader that initiates the switch into secure mode. Either the switch fails and we return with DIAG_308_RC_INVAL_FOR_PV to the non-PV bootloader or we start running in PV mode and __never__ return to the bootloader without a reboot. > >>> Perform the checks on Confidential Guest support at runtime with an >>> helper called from the service call switching the guest to protected >>> mode. >> >> If we don't have PV support then the subcodes >=8 are a specification exception >> so this is never executed AFAIK. > > It is. The test on huge page size was added just above this change. And the huge page test only applies if the PV feature is in the cpumodel which also means that the host is PV enabled since it's based on the capability: if (subcode >= DIAG308_PV_SET && !s390_has_feat(S390_FEAT_UNPACK)) { s390_program_interrupt(env, PGM_SPECIFICATION, ra); return; } > > Thanks, > > C. > >> >>> if (env->psw.mask & PSW_MASK_PSTATE) { >>> s390_program_interrupt(env, PGM_PRIVILEGED, ra); >>> @@ -176,6 +177,12 @@ out: >>> return; >>> } >>> + if (!s390_pv_check(&local_err)) { >>> + error_report_err(local_err); >>> + env->regs[r1 + 1] = DIAG_308_RC_INVAL_FOR_PV; >>> + return; >>> + } >>> + >>> s390_ipl_reset_request(cs, S390_RESET_PV); >>> break; >>> default: >> >
On 1/9/23 10:49, Janosch Frank wrote: > On 1/9/23 10:27, Cédric Le Goater wrote: >> On 1/9/23 10:04, Janosch Frank wrote: >>> On 1/6/23 08:53, Cédric Le Goater wrote: >>>> From: Cédric Le Goater <clg@redhat.com> >>>> >>>> If a secure kernel is started in a non-protected VM, the OS will hang >>>> during boot without giving a proper error message to the user. >>> >>> Most of the time you see nothing in the console because libvirt is too slow. If you start the VM in paused mode, attach a console and then resume it, then you'll see a nice error message. >> >> If you wait long enough, the VM fails to mount / and falls into the dracut >> initrams. > > I have the feeling that we're not talking about the same thing here.> > A PV VM always starts out as a non-PV VM and is put into PV mode via two diag308 subcodes (8 & 10). ALL PV subcodes (8 - 10) are spec exceptions if the host isn't enabled for PV. The corner case this patch is trying to address is for a PV-enabled host, a secure enabled OS and !PV-enabled QEMU. Please run this command on a secure disk image : qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=<file>,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio and tell me what you get. Thanks, C.
On 1/9/23 14:30, Cédric Le Goater wrote: > On 1/9/23 10:49, Janosch Frank wrote: >> On 1/9/23 10:27, Cédric Le Goater wrote: >>> On 1/9/23 10:04, Janosch Frank wrote: >>>> On 1/6/23 08:53, Cédric Le Goater wrote: >>>>> From: Cédric Le Goater <clg@redhat.com> >>>>> >>>>> If a secure kernel is started in a non-protected VM, the OS will hang >>>>> during boot without giving a proper error message to the user. >>>> >>>> Most of the time you see nothing in the console because libvirt is too slow. If you start the VM in paused mode, attach a console and then resume it, then you'll see a nice error message. >>> >>> If you wait long enough, the VM fails to mount / and falls into the dracut >>> initrams. >> >> I have the feeling that we're not talking about the same thing here.> > > A PV VM always starts out as a non-PV VM and is put into PV mode via two diag308 subcodes (8 & 10). ALL PV subcodes (8 - 10) are spec exceptions if the host isn't enabled for PV. > > The corner case this patch is trying to address is for a PV-enabled host, > a secure enabled OS and !PV-enabled QEMU. > > Please run this command on a secure disk image : > > qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=<file>,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio > > and tell me what you get. > qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=u2204.qcow2,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio LOADPARM=[ ] Using virtio-blk. Using SCSI scheme. ............................................................................................................................. Secure unpack facility is not available > Thanks, > > C.
On 1/9/23 14:45, Janosch Frank wrote: > On 1/9/23 14:30, Cédric Le Goater wrote: >> On 1/9/23 10:49, Janosch Frank wrote: >>> On 1/9/23 10:27, Cédric Le Goater wrote: >>>> On 1/9/23 10:04, Janosch Frank wrote: >>>>> On 1/6/23 08:53, Cédric Le Goater wrote: >>>>>> From: Cédric Le Goater <clg@redhat.com> >>>>>> >>>>>> If a secure kernel is started in a non-protected VM, the OS will hang >>>>>> during boot without giving a proper error message to the user. >>>>> >>>>> Most of the time you see nothing in the console because libvirt is too slow. If you start the VM in paused mode, attach a console and then resume it, then you'll see a nice error message. >>>> >>>> If you wait long enough, the VM fails to mount / and falls into the dracut >>>> initrams. >>> >>> I have the feeling that we're not talking about the same thing here.> >> > A PV VM always starts out as a non-PV VM and is put into PV mode via two diag308 subcodes (8 & 10). ALL PV subcodes (8 - 10) are spec exceptions if the host isn't enabled for PV. >> >> The corner case this patch is trying to address is for a PV-enabled host, >> a secure enabled OS and !PV-enabled QEMU. >> >> Please run this command on a secure disk image : >> >> qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=<file>,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio >> >> and tell me what you get. >> > > qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=u2204.qcow2,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio > LOADPARM=[ ] > Using virtio-blk. > Using SCSI scheme. > ............................................................................................................................. > Secure unpack facility is not available Yes. That's with a !PV-enabled host. Correct ? Can you try with prot_virt=1 on the host please ? Thanks, C.
On 1/9/23 14:53, Cédric Le Goater wrote: > On 1/9/23 14:45, Janosch Frank wrote: >> On 1/9/23 14:30, Cédric Le Goater wrote: >>> On 1/9/23 10:49, Janosch Frank wrote: >>>> On 1/9/23 10:27, Cédric Le Goater wrote: >>>>> On 1/9/23 10:04, Janosch Frank wrote: >>>>>> On 1/6/23 08:53, Cédric Le Goater wrote: >>>>>>> From: Cédric Le Goater <clg@redhat.com> >>>>>>> >>>>>>> If a secure kernel is started in a non-protected VM, the OS will hang >>>>>>> during boot without giving a proper error message to the user. >>>>>> >>>>>> Most of the time you see nothing in the console because libvirt is too slow. If you start the VM in paused mode, attach a console and then resume it, then you'll see a nice error message. >>>>> >>>>> If you wait long enough, the VM fails to mount / and falls into the dracut >>>>> initrams. >>>> >>>> I have the feeling that we're not talking about the same thing here.> >>> > A PV VM always starts out as a non-PV VM and is put into PV mode via two diag308 subcodes (8 & 10). ALL PV subcodes (8 - 10) are spec exceptions if the host isn't enabled for PV. >>> >>> The corner case this patch is trying to address is for a PV-enabled host, >>> a secure enabled OS and !PV-enabled QEMU. >>> >>> Please run this command on a secure disk image : >>> >>> qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=<file>,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio >>> >>> and tell me what you get. >>> >> >> qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=u2204.qcow2,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio >> LOADPARM=[ ] >> Using virtio-blk. >> Using SCSI scheme. >> ............................................................................................................................. >> Secure unpack facility is not available > > Yes. That's with a !PV-enabled host. Correct ? > > Can you try with prot_virt=1 on the host please ? With prot_virt=1 it boots until it doesn't find the file system (at least if you give it a bit more memory than the standard 256MB): qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=u2204.qcow2,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio -m 4096 [Linux boot stuff] ALERT! /dev/disk/by-path/ccw-0.0.0000-part1 does not exist. Dropping to a shell!
On 1/9/23 15:31, Janosch Frank wrote: > On 1/9/23 14:53, Cédric Le Goater wrote: >> On 1/9/23 14:45, Janosch Frank wrote: >>> On 1/9/23 14:30, Cédric Le Goater wrote: >>>> On 1/9/23 10:49, Janosch Frank wrote: >>>>> On 1/9/23 10:27, Cédric Le Goater wrote: >>>>>> On 1/9/23 10:04, Janosch Frank wrote: >>>>>>> On 1/6/23 08:53, Cédric Le Goater wrote: >>>>>>>> From: Cédric Le Goater <clg@redhat.com> >>>>>>>> >>>>>>>> If a secure kernel is started in a non-protected VM, the OS will hang >>>>>>>> during boot without giving a proper error message to the user. >>>>>>> >>>>>>> Most of the time you see nothing in the console because libvirt is too slow. If you start the VM in paused mode, attach a console and then resume it, then you'll see a nice error message. >>>>>> >>>>>> If you wait long enough, the VM fails to mount / and falls into the dracut >>>>>> initrams. >>>>> >>>>> I have the feeling that we're not talking about the same thing here.> >>>> > A PV VM always starts out as a non-PV VM and is put into PV mode via two diag308 subcodes (8 & 10). ALL PV subcodes (8 - 10) are spec exceptions if the host isn't enabled for PV. >>>> >>>> The corner case this patch is trying to address is for a PV-enabled host, >>>> a secure enabled OS and !PV-enabled QEMU. >>>> >>>> Please run this command on a secure disk image : >>>> >>>> qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=<file>,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio >>>> >>>> and tell me what you get. >>>> >>> >>> qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=u2204.qcow2,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio >>> LOADPARM=[ ] >>> Using virtio-blk. >>> Using SCSI scheme. >>> ............................................................................................................................. >>> Secure unpack facility is not available >> >> Yes. That's with a !PV-enabled host. Correct ? >> >> Can you try with prot_virt=1 on the host please ? > > With prot_virt=1 it boots until it doesn't find the file system (at > least if you give it a bit more memory than the standard 256MB): > > qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive > file=u2204.qcow2,if=virtio,format=qcow2 -nographic -nodefaults -serial > mon:stdio -m 4096 > [Linux boot stuff] > ALERT! /dev/disk/by-path/ccw-0.0.0000-part1 does not exist. Dropping > to a shell! > I.e. it boots into secure mode just fine. Now if you add iommu_platform=true to the device then it'll even boot from disk. So we'd rather need an error message if you attach a device without the iommu being set to true. The whole topic of PV iommu problems has a few windings which I don't fully want to bring to electronic paper right now. Either you start a secure guest that has devices with manual iommu entries or you go the launch security route and let libvirt/qemu handle it for you.
On 1/9/23 15:52, Janosch Frank wrote: > On 1/9/23 15:31, Janosch Frank wrote: >> On 1/9/23 14:53, Cédric Le Goater wrote: >>> On 1/9/23 14:45, Janosch Frank wrote: >>>> On 1/9/23 14:30, Cédric Le Goater wrote: >>>>> On 1/9/23 10:49, Janosch Frank wrote: >>>>>> On 1/9/23 10:27, Cédric Le Goater wrote: >>>>>>> On 1/9/23 10:04, Janosch Frank wrote: >>>>>>>> On 1/6/23 08:53, Cédric Le Goater wrote: >>>>>>>>> From: Cédric Le Goater <clg@redhat.com> >>>>>>>>> >>>>>>>>> If a secure kernel is started in a non-protected VM, the OS will hang >>>>>>>>> during boot without giving a proper error message to the user. >>>>>>>> >>>>>>>> Most of the time you see nothing in the console because libvirt is too slow. If you start the VM in paused mode, attach a console and then resume it, then you'll see a nice error message. >>>>>>> >>>>>>> If you wait long enough, the VM fails to mount / and falls into the dracut >>>>>>> initrams. >>>>>> >>>>>> I have the feeling that we're not talking about the same thing here.> >>>>> > A PV VM always starts out as a non-PV VM and is put into PV mode via two diag308 subcodes (8 & 10). ALL PV subcodes (8 - 10) are spec exceptions if the host isn't enabled for PV. >>>>> >>>>> The corner case this patch is trying to address is for a PV-enabled host, >>>>> a secure enabled OS and !PV-enabled QEMU. >>>>> >>>>> Please run this command on a secure disk image : >>>>> >>>>> qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=<file>,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio >>>>> >>>>> and tell me what you get. >>>>> >>>> >>>> qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive file=u2204.qcow2,if=virtio,format=qcow2 -nographic -nodefaults -serial mon:stdio >>>> LOADPARM=[ ] >>>> Using virtio-blk. >>>> Using SCSI scheme. >>>> ............................................................................................................................. >>>> Secure unpack facility is not available >>> >>> Yes. That's with a !PV-enabled host. Correct ? >>> >>> Can you try with prot_virt=1 on the host please ? >> >> With prot_virt=1 it boots until it doesn't find the file system (at >> least if you give it a bit more memory than the standard 256MB): >> >> qemu-system-s390x -M s390-ccw-virtio -accel kvm -drive >> file=u2204.qcow2,if=virtio,format=qcow2 -nographic -nodefaults -serial >> mon:stdio -m 4096 >> [Linux boot stuff] >> ALERT! /dev/disk/by-path/ccw-0.0.0000-part1 does not exist. Dropping >> to a shell! >> > > I.e. it boots into secure mode just fine. > > Now if you add iommu_platform=true to the device then it'll even boot from disk. > > So we'd rather need an error message if you attach a device without the iommu being set to true. The whole topic of PV iommu problems has a few windings which I don't fully want to bring to electronic paper right now. > > Either you start a secure guest that has devices with manual iommu entries or you go the launch security route and let libvirt/qemu handle it for you. aaahh. So "-machine confidential-guest-support=pv0 -object s390-pv-guest,id=pv0" is equivalent to "-global virtio-device.iommu_platform=true". I should have looked closer :/ Thanks for the clarification, C.
diff --git a/include/hw/s390x/pv.h b/include/hw/s390x/pv.h index 9360aa1091..ca7dac2e20 100644 --- a/include/hw/s390x/pv.h +++ b/include/hw/s390x/pv.h @@ -55,6 +55,7 @@ int kvm_s390_dump_init(void); int kvm_s390_dump_cpu(S390CPU *cpu, void *buff); int kvm_s390_dump_mem_state(uint64_t addr, size_t len, void *dest); int kvm_s390_dump_completion_data(void *buff); +bool s390_pv_check(Error **errp); #else /* CONFIG_KVM */ static inline bool s390_is_pv(void) { return false; } static inline int s390_pv_query_info(void) { return 0; } @@ -75,6 +76,7 @@ static inline int kvm_s390_dump_cpu(S390CPU *cpu, void *buff) { return 0; } static inline int kvm_s390_dump_mem_state(uint64_t addr, size_t len, void *dest) { return 0; } static inline int kvm_s390_dump_completion_data(void *buff) { return 0; } +static inline bool s390_pv_check(Error **errp) { return false; } #endif /* CONFIG_KVM */ int s390_pv_kvm_init(ConfidentialGuestSupport *cgs, Error **errp); diff --git a/hw/s390x/pv.c b/hw/s390x/pv.c index d53ef8fd38..13c6116076 100644 --- a/hw/s390x/pv.c +++ b/hw/s390x/pv.c @@ -327,6 +327,19 @@ int s390_pv_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) return 0; } +bool s390_pv_check(Error **errp) +{ + MachineState *ms = MACHINE(qdev_get_machine()); + + if (!ms->cgs) { + error_setg(errp, "Protected VM is started without Confidential" + " Guest support"); + return false; + } + + return s390_pv_guest_check(ms->cgs, errp); +} + OBJECT_DEFINE_TYPE_WITH_INTERFACES(S390PVGuest, s390_pv_guest, S390_PV_GUEST, diff --git a/target/s390x/diag.c b/target/s390x/diag.c index 76b01dcd68..9b16e25930 100644 --- a/target/s390x/diag.c +++ b/target/s390x/diag.c @@ -79,6 +79,7 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) uint64_t addr = env->regs[r1]; uint64_t subcode = env->regs[r3]; IplParameterBlock *iplb; + Error *local_err = NULL; if (env->psw.mask & PSW_MASK_PSTATE) { s390_program_interrupt(env, PGM_PRIVILEGED, ra); @@ -176,6 +177,12 @@ out: return; } + if (!s390_pv_check(&local_err)) { + error_report_err(local_err); + env->regs[r1 + 1] = DIAG_308_RC_INVAL_FOR_PV; + return; + } + s390_ipl_reset_request(cs, S390_RESET_PV); break; default: