| Message ID | 20220606203325.110625-21-mjrosato@linux.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | KVM: s390: enable zPCI for interpretive execution | expand |
On 06/06/2022 22.33, Matthew Rosato wrote: > The KVM_S390_ZPCI_OP ioctl provides a mechanism for managing > hardware-assisted virtualization features for s390x zPCI passthrough. > Add the first 2 operations, which can be used to enable/disable > the specified device for Adapter Event Notification interpretation. > > Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com> > --- > Documentation/virt/kvm/api.rst | 47 +++++++++++++++++++ > arch/s390/kvm/kvm-s390.c | 16 +++++++ > arch/s390/kvm/pci.c | 85 ++++++++++++++++++++++++++++++++++ > arch/s390/kvm/pci.h | 2 + > include/uapi/linux/kvm.h | 31 +++++++++++++ > 5 files changed, 181 insertions(+) Reviewed-by: Thomas Huth <thuth@redhat.com>
On 6/6/22 22:33, Matthew Rosato wrote: > The KVM_S390_ZPCI_OP ioctl provides a mechanism for managing > hardware-assisted virtualization features for s390x zPCI passthrough. > Add the first 2 operations, which can be used to enable/disable > the specified device for Adapter Event Notification interpretation. > > Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com> Despite I really do not like passing the FH in the structure. I think that if I admit that then this LGTM. Acked-by: Pierre Morel <pmorel@linux.ibm.com> > --- > Documentation/virt/kvm/api.rst | 47 +++++++++++++++++++ > arch/s390/kvm/kvm-s390.c | 16 +++++++ > arch/s390/kvm/pci.c | 85 ++++++++++++++++++++++++++++++++++ > arch/s390/kvm/pci.h | 2 + > include/uapi/linux/kvm.h | 31 +++++++++++++ > 5 files changed, 181 insertions(+) > > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst > index 11e00a46c610..2eb604769dce 100644 > --- a/Documentation/virt/kvm/api.rst > +++ b/Documentation/virt/kvm/api.rst > @@ -5802,6 +5802,53 @@ of CPUID leaf 0xD on the host. > > This ioctl injects an event channel interrupt directly to the guest vCPU. > > +4.136 KVM_S390_ZPCI_OP > +-------------------- > + > +:Capability: KVM_CAP_S390_ZPCI_OP > +:Architectures: s390 > +:Type: vm ioctl > +:Parameters: struct kvm_s390_zpci_op (in) > +:Returns: 0 on success, <0 on error > + > +Used to manage hardware-assisted virtualization features for zPCI devices. > + > +Parameters are specified via the following structure:: > + > + struct kvm_s390_zpci_op { > + /* in */ > + __u32 fh; /* target device */ > + __u8 op; /* operation to perform */ > + __u8 pad[3]; > + union { > + /* for KVM_S390_ZPCIOP_REG_AEN */ > + struct { > + __u64 ibv; /* Guest addr of interrupt bit vector */ > + __u64 sb; /* Guest addr of summary bit */ > + __u32 flags; > + __u32 noi; /* Number of interrupts */ > + __u8 isc; /* Guest interrupt subclass */ > + __u8 sbo; /* Offset of guest summary bit vector */ > + __u16 pad; > + } reg_aen; > + __u64 reserved[8]; > + } u; > + }; > + > +The type of operation is specified in the "op" field. > +KVM_S390_ZPCIOP_REG_AEN is used to register the VM for adapter event > +notification interpretation, which will allow firmware delivery of adapter > +events directly to the vm, with KVM providing a backup delivery mechanism; > +KVM_S390_ZPCIOP_DEREG_AEN is used to subsequently disable interpretation of > +adapter event notifications. > + > +The target zPCI function must also be specified via the "fh" field. For the > +KVM_S390_ZPCIOP_REG_AEN operation, additional information to establish firmware > +delivery must be provided via the "reg_aen" struct. > + > +The "pad" and "reserved" fields may be used for future extensions and should be > +set to 0s by userspace. > + > 5. The kvm_run structure > ======================== > > diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c > index 4758bb731199..f214e0fc62ed 100644 > --- a/arch/s390/kvm/kvm-s390.c > +++ b/arch/s390/kvm/kvm-s390.c > @@ -618,6 +618,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) > case KVM_CAP_S390_PROTECTED: > r = is_prot_virt_host(); > break; > + case KVM_CAP_S390_ZPCI_OP: > + r = kvm_s390_pci_interp_allowed(); > + break; > default: > r = 0; > } > @@ -2629,6 +2632,19 @@ long kvm_arch_vm_ioctl(struct file *filp, > r = -EFAULT; > break; > } > + case KVM_S390_ZPCI_OP: { > + struct kvm_s390_zpci_op args; > + > + r = -EINVAL; > + if (!IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM)) > + break; > + if (copy_from_user(&args, argp, sizeof(args))) { > + r = -EFAULT; > + break; > + } > + r = kvm_s390_pci_zpci_op(kvm, &args); > + break; > + } > default: > r = -ENOTTY; > } > diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c > index 24211741deb0..4946fb7757d6 100644 > --- a/arch/s390/kvm/pci.c > +++ b/arch/s390/kvm/pci.c > @@ -585,6 +585,91 @@ void kvm_s390_pci_clear_list(struct kvm *kvm) > spin_unlock(&kvm->arch.kzdev_list_lock); > } > > +static struct zpci_dev *get_zdev_from_kvm_by_fh(struct kvm *kvm, u32 fh) > +{ > + struct zpci_dev *zdev = NULL; > + struct kvm_zdev *kzdev; > + > + spin_lock(&kvm->arch.kzdev_list_lock); > + list_for_each_entry(kzdev, &kvm->arch.kzdev_list, entry) { > + if (kzdev->zdev->fh == fh) { > + zdev = kzdev->zdev; > + break; > + } > + } > + spin_unlock(&kvm->arch.kzdev_list_lock); > + > + return zdev; > +} > + > +static int kvm_s390_pci_zpci_reg_aen(struct zpci_dev *zdev, > + struct kvm_s390_zpci_op *args) > +{ > + struct zpci_fib fib = {}; > + bool hostflag; > + > + fib.fmt0.aibv = args->u.reg_aen.ibv; > + fib.fmt0.isc = args->u.reg_aen.isc; > + fib.fmt0.noi = args->u.reg_aen.noi; > + if (args->u.reg_aen.sb != 0) { > + fib.fmt0.aisb = args->u.reg_aen.sb; > + fib.fmt0.aisbo = args->u.reg_aen.sbo; > + fib.fmt0.sum = 1; > + } else { > + fib.fmt0.aisb = 0; > + fib.fmt0.aisbo = 0; > + fib.fmt0.sum = 0; > + } > + > + hostflag = !(args->u.reg_aen.flags & KVM_S390_ZPCIOP_REGAEN_HOST); > + return kvm_s390_pci_aif_enable(zdev, &fib, hostflag); > +} > + > +int kvm_s390_pci_zpci_op(struct kvm *kvm, struct kvm_s390_zpci_op *args) > +{ > + struct kvm_zdev *kzdev; > + struct zpci_dev *zdev; > + int r; > + > + zdev = get_zdev_from_kvm_by_fh(kvm, args->fh); > + if (!zdev) > + return -ENODEV; > + > + mutex_lock(&zdev->kzdev_lock); > + mutex_lock(&kvm->lock); > + > + kzdev = zdev->kzdev; > + if (!kzdev) { > + r = -ENODEV; > + goto out; > + } > + if (kzdev->kvm != kvm) { > + r = -EPERM; > + goto out; > + } > + > + switch (args->op) { > + case KVM_S390_ZPCIOP_REG_AEN: > + /* Fail on unknown flags */ > + if (args->u.reg_aen.flags & ~KVM_S390_ZPCIOP_REGAEN_HOST) { > + r = -EINVAL; > + break; > + } > + r = kvm_s390_pci_zpci_reg_aen(zdev, args); > + break; > + case KVM_S390_ZPCIOP_DEREG_AEN: > + r = kvm_s390_pci_aif_disable(zdev, false); > + break; > + default: > + r = -EINVAL; > + } > + > +out: > + mutex_unlock(&kvm->lock); > + mutex_unlock(&zdev->kzdev_lock); > + return r; > +} > + > int kvm_s390_pci_init(void) > { > aift = kzalloc(sizeof(struct zpci_aift), GFP_KERNEL); > diff --git a/arch/s390/kvm/pci.h b/arch/s390/kvm/pci.h > index fb2b91b76e0c..0351382e990f 100644 > --- a/arch/s390/kvm/pci.h > +++ b/arch/s390/kvm/pci.h > @@ -59,6 +59,8 @@ void kvm_s390_pci_aen_exit(void); > void kvm_s390_pci_init_list(struct kvm *kvm); > void kvm_s390_pci_clear_list(struct kvm *kvm); > > +int kvm_s390_pci_zpci_op(struct kvm *kvm, struct kvm_s390_zpci_op *args); > + > int kvm_s390_pci_init(void); > void kvm_s390_pci_exit(void); > > diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h > index 5088bd9f1922..52cc12d53022 100644 > --- a/include/uapi/linux/kvm.h > +++ b/include/uapi/linux/kvm.h > @@ -1157,6 +1157,7 @@ struct kvm_ppc_resize_hpt { > #define KVM_CAP_VM_TSC_CONTROL 214 > #define KVM_CAP_SYSTEM_EVENT_DATA 215 > #define KVM_CAP_ARM_SYSTEM_SUSPEND 216 > +#define KVM_CAP_S390_ZPCI_OP 217 > > #ifdef KVM_CAP_IRQ_ROUTING > > @@ -2118,4 +2119,34 @@ struct kvm_stats_desc { > /* Available with KVM_CAP_XSAVE2 */ > #define KVM_GET_XSAVE2 _IOR(KVMIO, 0xcf, struct kvm_xsave) > > +/* Available with KVM_CAP_S390_ZPCI_OP */ > +#define KVM_S390_ZPCI_OP _IOW(KVMIO, 0xd1, struct kvm_s390_zpci_op) > + > +struct kvm_s390_zpci_op { > + /* in */ > + __u32 fh; /* target device */ > + __u8 op; /* operation to perform */ > + __u8 pad[3]; > + union { > + /* for KVM_S390_ZPCIOP_REG_AEN */ > + struct { > + __u64 ibv; /* Guest addr of interrupt bit vector */ > + __u64 sb; /* Guest addr of summary bit */ > + __u32 flags; > + __u32 noi; /* Number of interrupts */ > + __u8 isc; /* Guest interrupt subclass */ > + __u8 sbo; /* Offset of guest summary bit vector */ > + __u16 pad; > + } reg_aen; > + __u64 reserved[8]; > + } u; > +}; > + > +/* types for kvm_s390_zpci_op->op */ > +#define KVM_S390_ZPCIOP_REG_AEN 0 > +#define KVM_S390_ZPCIOP_DEREG_AEN 1 > + > +/* flags for kvm_s390_zpci_op->u.reg_aen.flags */ > +#define KVM_S390_ZPCIOP_REGAEN_HOST (1 << 0) > + > #endif /* __LINUX_KVM_H */ >
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 11e00a46c610..2eb604769dce 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -5802,6 +5802,53 @@ of CPUID leaf 0xD on the host. This ioctl injects an event channel interrupt directly to the guest vCPU. +4.136 KVM_S390_ZPCI_OP +-------------------- + +:Capability: KVM_CAP_S390_ZPCI_OP +:Architectures: s390 +:Type: vm ioctl +:Parameters: struct kvm_s390_zpci_op (in) +:Returns: 0 on success, <0 on error + +Used to manage hardware-assisted virtualization features for zPCI devices. + +Parameters are specified via the following structure:: + + struct kvm_s390_zpci_op { + /* in */ + __u32 fh; /* target device */ + __u8 op; /* operation to perform */ + __u8 pad[3]; + union { + /* for KVM_S390_ZPCIOP_REG_AEN */ + struct { + __u64 ibv; /* Guest addr of interrupt bit vector */ + __u64 sb; /* Guest addr of summary bit */ + __u32 flags; + __u32 noi; /* Number of interrupts */ + __u8 isc; /* Guest interrupt subclass */ + __u8 sbo; /* Offset of guest summary bit vector */ + __u16 pad; + } reg_aen; + __u64 reserved[8]; + } u; + }; + +The type of operation is specified in the "op" field. +KVM_S390_ZPCIOP_REG_AEN is used to register the VM for adapter event +notification interpretation, which will allow firmware delivery of adapter +events directly to the vm, with KVM providing a backup delivery mechanism; +KVM_S390_ZPCIOP_DEREG_AEN is used to subsequently disable interpretation of +adapter event notifications. + +The target zPCI function must also be specified via the "fh" field. For the +KVM_S390_ZPCIOP_REG_AEN operation, additional information to establish firmware +delivery must be provided via the "reg_aen" struct. + +The "pad" and "reserved" fields may be used for future extensions and should be +set to 0s by userspace. + 5. The kvm_run structure ======================== diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index 4758bb731199..f214e0fc62ed 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -618,6 +618,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_S390_PROTECTED: r = is_prot_virt_host(); break; + case KVM_CAP_S390_ZPCI_OP: + r = kvm_s390_pci_interp_allowed(); + break; default: r = 0; } @@ -2629,6 +2632,19 @@ long kvm_arch_vm_ioctl(struct file *filp, r = -EFAULT; break; } + case KVM_S390_ZPCI_OP: { + struct kvm_s390_zpci_op args; + + r = -EINVAL; + if (!IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM)) + break; + if (copy_from_user(&args, argp, sizeof(args))) { + r = -EFAULT; + break; + } + r = kvm_s390_pci_zpci_op(kvm, &args); + break; + } default: r = -ENOTTY; } diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c index 24211741deb0..4946fb7757d6 100644 --- a/arch/s390/kvm/pci.c +++ b/arch/s390/kvm/pci.c @@ -585,6 +585,91 @@ void kvm_s390_pci_clear_list(struct kvm *kvm) spin_unlock(&kvm->arch.kzdev_list_lock); } +static struct zpci_dev *get_zdev_from_kvm_by_fh(struct kvm *kvm, u32 fh) +{ + struct zpci_dev *zdev = NULL; + struct kvm_zdev *kzdev; + + spin_lock(&kvm->arch.kzdev_list_lock); + list_for_each_entry(kzdev, &kvm->arch.kzdev_list, entry) { + if (kzdev->zdev->fh == fh) { + zdev = kzdev->zdev; + break; + } + } + spin_unlock(&kvm->arch.kzdev_list_lock); + + return zdev; +} + +static int kvm_s390_pci_zpci_reg_aen(struct zpci_dev *zdev, + struct kvm_s390_zpci_op *args) +{ + struct zpci_fib fib = {}; + bool hostflag; + + fib.fmt0.aibv = args->u.reg_aen.ibv; + fib.fmt0.isc = args->u.reg_aen.isc; + fib.fmt0.noi = args->u.reg_aen.noi; + if (args->u.reg_aen.sb != 0) { + fib.fmt0.aisb = args->u.reg_aen.sb; + fib.fmt0.aisbo = args->u.reg_aen.sbo; + fib.fmt0.sum = 1; + } else { + fib.fmt0.aisb = 0; + fib.fmt0.aisbo = 0; + fib.fmt0.sum = 0; + } + + hostflag = !(args->u.reg_aen.flags & KVM_S390_ZPCIOP_REGAEN_HOST); + return kvm_s390_pci_aif_enable(zdev, &fib, hostflag); +} + +int kvm_s390_pci_zpci_op(struct kvm *kvm, struct kvm_s390_zpci_op *args) +{ + struct kvm_zdev *kzdev; + struct zpci_dev *zdev; + int r; + + zdev = get_zdev_from_kvm_by_fh(kvm, args->fh); + if (!zdev) + return -ENODEV; + + mutex_lock(&zdev->kzdev_lock); + mutex_lock(&kvm->lock); + + kzdev = zdev->kzdev; + if (!kzdev) { + r = -ENODEV; + goto out; + } + if (kzdev->kvm != kvm) { + r = -EPERM; + goto out; + } + + switch (args->op) { + case KVM_S390_ZPCIOP_REG_AEN: + /* Fail on unknown flags */ + if (args->u.reg_aen.flags & ~KVM_S390_ZPCIOP_REGAEN_HOST) { + r = -EINVAL; + break; + } + r = kvm_s390_pci_zpci_reg_aen(zdev, args); + break; + case KVM_S390_ZPCIOP_DEREG_AEN: + r = kvm_s390_pci_aif_disable(zdev, false); + break; + default: + r = -EINVAL; + } + +out: + mutex_unlock(&kvm->lock); + mutex_unlock(&zdev->kzdev_lock); + return r; +} + int kvm_s390_pci_init(void) { aift = kzalloc(sizeof(struct zpci_aift), GFP_KERNEL); diff --git a/arch/s390/kvm/pci.h b/arch/s390/kvm/pci.h index fb2b91b76e0c..0351382e990f 100644 --- a/arch/s390/kvm/pci.h +++ b/arch/s390/kvm/pci.h @@ -59,6 +59,8 @@ void kvm_s390_pci_aen_exit(void); void kvm_s390_pci_init_list(struct kvm *kvm); void kvm_s390_pci_clear_list(struct kvm *kvm); +int kvm_s390_pci_zpci_op(struct kvm *kvm, struct kvm_s390_zpci_op *args); + int kvm_s390_pci_init(void); void kvm_s390_pci_exit(void); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 5088bd9f1922..52cc12d53022 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1157,6 +1157,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_VM_TSC_CONTROL 214 #define KVM_CAP_SYSTEM_EVENT_DATA 215 #define KVM_CAP_ARM_SYSTEM_SUSPEND 216 +#define KVM_CAP_S390_ZPCI_OP 217 #ifdef KVM_CAP_IRQ_ROUTING @@ -2118,4 +2119,34 @@ struct kvm_stats_desc { /* Available with KVM_CAP_XSAVE2 */ #define KVM_GET_XSAVE2 _IOR(KVMIO, 0xcf, struct kvm_xsave) +/* Available with KVM_CAP_S390_ZPCI_OP */ +#define KVM_S390_ZPCI_OP _IOW(KVMIO, 0xd1, struct kvm_s390_zpci_op) + +struct kvm_s390_zpci_op { + /* in */ + __u32 fh; /* target device */ + __u8 op; /* operation to perform */ + __u8 pad[3]; + union { + /* for KVM_S390_ZPCIOP_REG_AEN */ + struct { + __u64 ibv; /* Guest addr of interrupt bit vector */ + __u64 sb; /* Guest addr of summary bit */ + __u32 flags; + __u32 noi; /* Number of interrupts */ + __u8 isc; /* Guest interrupt subclass */ + __u8 sbo; /* Offset of guest summary bit vector */ + __u16 pad; + } reg_aen; + __u64 reserved[8]; + } u; +}; + +/* types for kvm_s390_zpci_op->op */ +#define KVM_S390_ZPCIOP_REG_AEN 0 +#define KVM_S390_ZPCIOP_DEREG_AEN 1 + +/* flags for kvm_s390_zpci_op->u.reg_aen.flags */ +#define KVM_S390_ZPCIOP_REGAEN_HOST (1 << 0) + #endif /* __LINUX_KVM_H */
The KVM_S390_ZPCI_OP ioctl provides a mechanism for managing hardware-assisted virtualization features for s390x zPCI passthrough. Add the first 2 operations, which can be used to enable/disable the specified device for Adapter Event Notification interpretation. Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com> --- Documentation/virt/kvm/api.rst | 47 +++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 16 +++++++ arch/s390/kvm/pci.c | 85 ++++++++++++++++++++++++++++++++++ arch/s390/kvm/pci.h | 2 + include/uapi/linux/kvm.h | 31 +++++++++++++ 5 files changed, 181 insertions(+)