| Message ID | 1536781396-13601-1-git-send-email-akrowiak@linux.vnet.ibm.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | guest dedicated crypto adapters | expand |
FWIW, assuming that review comments for v10 will be addresses in v11, I plan to add the upcoming v11 to a topic branch and will merge that after 2 or 3 days after v11 in kvms390/next. In the future the vfio-ap driver will likely be handled by Martins s390 tree, so I guess it makes sense for him to merge the upcoming topic branch as well. I will coordinate with Martin. Christian On 09/12/2018 09:42 PM, Tony Krowiak wrote: > From: Tony Krowiak <akrowiak@linux.ibm.com> > > Notes: > ===== > > Patches 1-2 (by David) are posted with this series because they are not > currently available in our master branch, upon which this series is based, > and because this series is dependent upon them. > > This patch series works with the v8 QEMU patches. > > Abstract: > ======== > > On s390, we have cryptographic coprocessor cards, which are modeled on > Linux as devices on the AP bus. Each card can be partitioned into domains > which can be thought of as a set of hardware registers for processing > crypto commands. Crypto commands are sent to a specific domain within a > card is via a queue which is identified as a (card,domain) tuple. We model > this something like the following (assuming we have access to cards 3 and > 4 and domains 1 and 2): > > AP -> card3 -> queue (3,1) > -> queue (3,2) > -> card4 -> queue (4,1) > -> queue (4,2) > > If we want to virtualize this, we can use a feature provided by the > hardware. We basically attach a satellite control block to our main > hardware virtualization control block and the hardware takes care of > most of the rest. > > For this control block, we don't specify explicit tuples, but a list of > cards and a list of domains. The guest will get access to the cross > product. > > Because of this, we need to take care that the lists provided to > different guests don't overlap; i.e., we need to enforce sane > configurations. Otherwise, one guest may get access to things like > secret keys for another guest. > > The idea of this patch set is to introduce a new device, the matrix > device. This matrix device hangs off a different root and acts as the > parent node for mdev devices. > > If you now want to give the tuples (4,1) and (4,2), you need to do the > following: > > - Make sure the queues (4,1) and (4,2) belong to vfio_ap (see patches > #5 and #6) > - Create the mediated device. > - Assign card 4 and domains 1 and 2 to the mediated device > - Optionally activate the mediated device. > > QEMU will now simply consume the mediated device and things should work. > > For a complete description of the architecture and concepts underlying > the design, see the Documentation/s390/vfio-ap.txt file included with this > patch set. > > v9 => v10 Change log: > =================== > * Replaced statically allocated with dynamically allocated matrix device > * Made changes to drivers/iommu/Kconfig and arch/s390/Kconfig to fix the > dependency chain so that make menuconfig can be used to configure VFIO_AP > * Added KVM device attributes to enable/disable hw interpretation of AP > instructions from userspace. > * Return more meaningful error values from mediated matrix device > assignment sysfs interfaces > * No longer enforcing convention that the ADM is a superset of the AQM at > for guest level 2 > * Broke 2 vSIE patches into several to make it easier to review them > * Reworked vSIE patches to handle various CRYCB formats > v8 => v9 Change log: > =================== > * Removed /sys/devices/virtual/misc/vfio_ap device and restored > /sys/devices/vfio_ap/matrix device as parent of mediated matrix devices > * Return boolean from ap_configuration_available() in ap.h instead of 0 or > an error. > * Miscellaneous changes due to review comments > > v7 => v8 Change log: > =================== > * Removed the AP bus gained the ability to designate queues as 'used by > host' or as 'used by alternate driver(s)'. > * Removed 'activate' attribute from mediated device. > * Do consistency checking during device assignment: > 1. Verify that APQNs assigned to the mediated device are bound to the > VFIO AP device driver > 2. Verify that no APQN assigned to the mediated matrix device is assigned > to any other mediated matrix device. > * The attributes of a mediated matrix device that is in use by a guest can > not be changed - i.e., no device assignment/unassignment allowed > * A mediated matrix device that is in use by a guest can not be removed. > * Removed all printk logging from VFIO AP driver; allowing return codes > from interfaces to describe the error. > * Reworked the handling of the CRYCB in vSIE based upon patches introduced > by David in the mainline. > > v6 => v7 Change log: > =================== > * The AP bus gained the ability to designate queues as 'used by host' > or as 'used by alternate driver(s)'. This allows us to authorise access > (via the CRYCB) to queues that are not currently bound to the vfio_ap > driver. If a vfio_ap owned queue diss- and reapears it's guaranteed > to get bound back to the vfio_ap driver. > * The mediated device gained an 'activate' attribute. Sharing conflicts are > checked on activation now. If the device was not activated, the mdev > open still implies activation. An active ap_matrix_mdev device claims > it's resources -- an inactive does not. > * An active ap_matrix_mdev device can not be removed. An ap_matrix_mdev > that is hooked up with a guest can not be deactivated. > * An active ap_matrix_mdev device rejects assign_* and deassign_* > operations. Thus changing the CRYCB masks of a guest in order to > accomplys certain hotplug scenarios is planned, but not supported yet. In > previous versions it was possible to do those operations on a ap_matrix_mdev > that is hooked up to a guest, but the changes would take effect on the next > mdev_open. > * Synchronisation was reworked. > * The sysfs path of the parent device changed from /sys/devices/vfio_ap/matrix/ > to /sys/devices/virtual/misc/vfio_ap/. The parent device is a misc > device now. > * The severity for most of the messages were reduced form error to > warning. > * We are not as thick headed about the zapq as we used to be in v6. > > v5 => v6 Change log: > =================== > * Added VSIE support - thanks to Pierre Morel > * Added VFIO_DEVICE_RESET ioctl > * Zeroizing AP queues when mediated device released and when > VFIO_DEVICE_RESET ioctl is invoked > * Removed /arch/s390/kvm/kvm-ap.c and arch/s390/include/asm/kvm-ap.h and > moved guest matrix configuration into vfio driver > * Removed temporary interfaces to be supplied by AP bus > * Made the variable that keeps track of mdev instance count an atomic_t > type > * Removed code iterating through vm_list to determine if another guest has > a queue .... not keep a list of matrix_mdev devices and verify against > that list. Removes the need for the kvm_lock. > * Added a sysfs attribute for the mediated matrix device to display the > matrix contained in the guest's CRYCB if a guest is using the mdev. > > > v4 => v5 Change log: > =================== > * Verify AP queues bound to driver in mediated device open callback, prior > to configuring the matrix in the CRYCB > * Implement VFIO_DEVICE_RESET ioctl > * Zeroize queues on guest entry and exit > * Removed vnet from all email IBM email addresses referenced > * Add synchronization in mdev create/remove and open/release. > > v4 => v5 Change log: > =================== > * Added code to mdev open callback to ensure not more than one vfio-ap > device can be opened by a guest. > * Interpret AP instructions by default > * Removed patch implementing interface to enable/disable AP interpretation, > since that will now be done by default > * Removed patch to reset crypto attributes for ALL vcpus. That will be > submitted as a single patch since it will not be needed in this series - > i.e., it was called from the interface to enable/disable AP instructions > * All code for initializing crypto for a guest has been moved back to > kvm-s390.c, kvm_s390_crypto_init(kvm) function > * Maintaining a module reference count for the vfio_ap module so it is not > removed while a guest with AP devices is running. > > > > v3 => v4 Change log: > =================== > * Resolved issue with enabling ZCRYPT when KVM is enabled by using > #ifdef ZCRYPT in relevant functions > * Added patch with a new function for resetting the crypto attributes > for all vcpus to resolve the issue raised with running vcpus getting out > of sync. > * Removed KVM_S390_VM_CRYPTO_INTERPRET_AP: Setting interpretive exec mode > from vfio_ap driver when mdev device is opened. > > > v2 => v3 Change log: > =================== > * Set APIE in VCPU setup function > * Renamed patch 13/15: > KVM: s390: Configure the guest's CRYCB > KVM: s390: Configure the guest's AP devices > * Fixed problem with building arch/s390/kvm/kvm-ap.c when CONFIG_ZCRYPT > not selected > * Removed patch introducing VSIE support for AP pending further > investigation > * Initialized AP maximum mask sizes - i.e., APM, AQM and ADM - from info > returned from PQAP(QCI) function > * Introduced a new device attribute to the KVM_S390_VM_CRYPTO attribute > group for setting a flag via the KVM_SET_DEVICE_ATTR ioctl to indicate > whether ECA_APIE should be set or not. The flag is used in the > kvm_s390_vcpu_crypto_setup() function to set ECA_APIE in the SIE block. > > > v1 => v2 Change log: > =================== > * Added documentation vfio-ap.txt > * Renamed vfio_ap_matrix module and device driver to vfio_ap > * Use device core device list instead of maintaining list of matrix > devices in driver > * Added VSIE support for AP > * Create matrix device before registering VFIO AP device driver with the > AP bus > * Renamed the following files in drivers/s390/crypto: > * vfio_ap_matrix.drv -> vfio_ap_drv > * vfio_ap_matrix_private.h -> vfio_ap_private.h > * vfio_ap_matrix_ops.c -> vfio_ap_ops.c > * arch/s390/include/asm/kvm/ap-matrix-config.h > * Renamed to kvm-ap.h > * Changed the data type of the bit mask fields for the matrix structure > to unsigned long and create them with DECLARE_BITMAP > * Changed #define prefixes from AP_MATRIX to KVM_AP > * Changed function and structure prefixes from ap_matrix to kvm_ap > * Added function interface to check if AP Extended Addressing (APXA) > facility is installedCRYCB_FORMAT_MASK > * Added function interface to get the maximum ID for AP mask type > * Added function interface to set the AP execution mode > * arch/s390/kvm/ap-matrix-config.c > * Renamed to kvm-ap.c > * Changed function prefixes from ap_matrix to kvm_ap > * Added function to check if AP Extended Addressing (APXA) facility is > installed > * Added function to get the maximum ID for AP mask type > * Added function to set the AP execution mode > * Added a boolean parameter to the functions that retrieve the APM, AQM > and ADM bit mask fields from the CRYCB. If true, then the function > will clear the bits in the mask before returning a reference to it > * Added validation to verify that APM, AQM and ADM bits that are set do > not exceed the maximum ID value allowed > * > * arch/s390/include/asm/kvm_host.h > * Changed defined for ECA_AP to ECA_APIE - interpretive execution mode > * Added a flag to struct kvm_s390_crypto to indicate whether the > KVM_S390_VM_CPU_FEAT_AP CPU model feature for AP facilities is set > * Added two CPU facilities features to set STFLE.12 and STFLE.15 > * arch/s390/kvm/kvm-s390.c > * Added initialization for new KVM_S390_VM_CPU_FEAT_AP CPU model feature > * Removed kvm_s390_apxa_installed() function > * Changed call to kvm_s390_apxa_installed() which has been removed to a > call to new kvm_ap_apxa_installed() function. > * Added code to kvm_s390_vcpu_crypto_setup() to set the new CPU model > feature flag in the kvm_s390_crypto structure > * Added CRYCB_FORMAT_MASK to mask CRYCBD > * arch/s390/tools/gen_facilities.c > * Added STFLE.12 and STFLE.15 to struct facility _def > * drivers/s390/crypto/vfio_ap_matrix_private.h > * Changed name of file to vfio_ap.private.h > * Changed #define prefixes from VFIO_AP_MATRIX to VFIO_AP > * struct ap_matrix: removed list fields and locks > * struct vfio_ap_queue: removed list field > * Renamed functions ap_matrix_mdev_register and ap_matrix_mdev_unregister > to vfio_ap_mdev_register and vfio_ap_mdev_unregister respectively > * drivers/s390/crypto/vfio_ap_matrix_drv.c > * Renamed file to drivers/s390/crypto/vfio_ap_drv.c > * Changed all #define, structure and function prefixes to vfio_ap > * probe function > * Changed root device name for the matrix device to vfio_ap: > i.e., /sys/devices/vfio_ap/matrix > * No longer storing the AP queue device in a list, it is retrievable via > the device core > * Removed unnecessary check whether matrix device exists > * Store the vfio_ap_queue structure in the private field of the ap_queue > structure rather than using list interface > * remove function > * Retrieve vfio_ap_queue structure from the struct ap_queue private > data rather than from a list > * Removed unnecesary check > * drivers/s390/crypto/vfio_ap_matrix_ops.c > * Renamed file to vfio_ap_ops.c > * Changed #define prefixes from AP_MATRIX to VFIO_AP > * Changed function name prefixes from ap_matrix to vfio_ap > * Removed ioctl to configure the CRYCB > * create function > * Removed ap_matrix_mdev_find_by_uuid() function - function is provided > by mdev core > * Removed available_instances verification, provided by mdev core > * Removed check to see if mediated device exists, handled by mdev core > * notifier function > * Configuring matrix here instead of via ioctl > * Set interpretive execution mode for all VCPUs > * Removed R/O attributes to display adapters and domains > * Added an R/O attribute to display the matrix > * assign_control_domain mdev attribute: > * Removed check to see if the domain is installed on the linux host > * Added check to verify the control domain ID does not exceed the max > value > * assign_adapter mdev attribute: > * Added check to verify the adapter ID does not exceed the max > value > * If any APQNs configured for the mediated matrix device that > have an APID matching the adapter ID being assigned are not > bound to the vfio_ap device driver then it is assumed that the APQN is > bound to another driver and assignment will fail > * assign_domain mdev attribute: > * Added check to verify the domain ID does not exceed the max > value > * If any APQNs configured for the mediated matrix device that > have an APQI matching the domain ID being assigned are not > bound to the vfio_ap device driver then it is assumed that the APQN is > bound to another driver and assignment will fail > * tools/arch/s390/include/uapi/asm/kvm.h > * removed KVM_S390_VM_CPU_FEAT_AP feature definition > > David Hildenbrand (2): > KVM: s390: vsie: simulate VCPU SIE entry/exit > KVM: s390: introduce and use KVM_REQ_VSIE_RESTART > > Pierre Morel (9): > KVM: s390: Clear Crypto Control Block when using vSIE > KVM: s390: vsie: Do the CRYCB validation first > KVM: s390: vsie: Make use of CRYCB FORMAT2 clear > KVM: s390: vsie: Allow CRYCB FORMAT-2 > KVM: s390: vsie: allow CRYCB FORMAT-1 > KVM: s390: vsie: allow CRYCB FORMAT-0 > KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-1 > KVM: s390: vsie: allow guest FORMAT-1 CRYCB on host FORMAT-2 > KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-2 > > Tony Krowiak (15): > KVM: s390: refactor crypto initialization > s390: vfio-ap: base implementation of VFIO AP device driver > s390: vfio-ap: register matrix device with VFIO mdev framework > s390: vfio-ap: sysfs interfaces to configure adapters > s390: vfio-ap: sysfs interfaces to configure domains > s390: vfio-ap: sysfs interfaces to configure control domains > s390: vfio-ap: sysfs interface to view matrix mdev matrix > KVM: s390: interfaces to clear CRYCB masks > s390: vfio-ap: implement mediated device open callback > s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl > s390: vfio-ap: zeroize the AP queues > s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl > KVM: s390: device attrs to enable/disable AP interpretation > KVM: s390: CPU model support for AP virtualization > s390: doc: detailed specifications for AP virtualization > > Documentation/s390/vfio-ap.txt | 778 ++++++++++++++++++++++++++ > MAINTAINERS | 12 + > arch/s390/Kconfig | 11 + > arch/s390/include/asm/kvm_host.h | 7 + > arch/s390/include/uapi/asm/kvm.h | 3 + > arch/s390/kvm/kvm-s390.c | 134 +++-- > arch/s390/kvm/kvm-s390.h | 1 + > arch/s390/kvm/vsie.c | 208 +++++++- > arch/s390/tools/gen_facilities.c | 2 + > drivers/iommu/Kconfig | 8 + > drivers/s390/crypto/Makefile | 4 + > drivers/s390/crypto/vfio_ap_drv.c | 153 ++++++ > drivers/s390/crypto/vfio_ap_ops.c | 973 +++++++++++++++++++++++++++++++++ > drivers/s390/crypto/vfio_ap_private.h | 88 +++ > include/uapi/linux/vfio.h | 2 + > 15 files changed, 2324 insertions(+), 60 deletions(-) > create mode 100644 Documentation/s390/vfio-ap.txt > create mode 100644 drivers/s390/crypto/vfio_ap_drv.c > create mode 100644 drivers/s390/crypto/vfio_ap_ops.c > create mode 100644 drivers/s390/crypto/vfio_ap_private.h >
On Wed, 12 Sep 2018 15:42:50 -0400 Tony Krowiak <akrowiak@linux.vnet.ibm.com> wrote: > Abstract: > ======== > > On s390, we have cryptographic coprocessor cards, which are modeled on > Linux as devices on the AP bus. Each card can be partitioned into domains > which can be thought of as a set of hardware registers for processing > crypto commands. Crypto commands are sent to a specific domain within a > card is via a queue which is identified as a (card,domain) tuple. We model > this something like the following (assuming we have access to cards 3 and > 4 and domains 1 and 2): > > AP -> card3 -> queue (3,1) > -> queue (3,2) > -> card4 -> queue (4,1) > -> queue (4,2) > > If we want to virtualize this, we can use a feature provided by the > hardware. We basically attach a satellite control block to our main > hardware virtualization control block and the hardware takes care of > most of the rest. > > For this control block, we don't specify explicit tuples, but a list of > cards and a list of domains. The guest will get access to the cross > product. > > Because of this, we need to take care that the lists provided to > different guests don't overlap; i.e., we need to enforce sane > configurations. Otherwise, one guest may get access to things like > secret keys for another guest. > > The idea of this patch set is to introduce a new device, the matrix > device. This matrix device hangs off a different root and acts as the > parent node for mdev devices. > > If you now want to give the tuples (4,1) and (4,2), you need to do the > following: > > - Make sure the queues (4,1) and (4,2) belong to vfio_ap (see patches > #5 and #6) > - Create the mediated device. > - Assign card 4 and domains 1 and 2 to the mediated device > - Optionally activate the mediated device. > > QEMU will now simply consume the mediated device and things should work. > > For a complete description of the architecture and concepts underlying > the design, see the Documentation/s390/vfio-ap.txt file included with this > patch set. I did not spot anything major, and if v11 addresses the issues raised by various reviewers I don't see why it should not be merged (interface looks sane). I skipped looking at the vsie stuff, though ;)
On Mon, 24 Sep 2018 12:10:37 +0200 Christian Borntraeger <borntraeger@de.ibm.com> wrote: > FWIW, assuming that review comments for v10 will be addresses in v11, I plan to > add the upcoming v11 to a topic branch and will merge that after 2 or 3 days after > v11 in kvms390/next. As said before, I think this looks sane enough to merge with the requested changes. > In the future the vfio-ap driver will likely be handled by > Martins s390 tree, so I guess it makes sense for him to merge the upcoming > topic branch as well. I will coordinate with Martin. I assume future patches will still be posted to kvm + linux-s390 for review?
On 09/24/2018 07:49 AM, Cornelia Huck wrote: > On Wed, 12 Sep 2018 15:42:50 -0400 > Tony Krowiak <akrowiak@linux.vnet.ibm.com> wrote: > >> Abstract: >> ======== >> >> On s390, we have cryptographic coprocessor cards, which are modeled on >> Linux as devices on the AP bus. Each card can be partitioned into domains >> which can be thought of as a set of hardware registers for processing >> crypto commands. Crypto commands are sent to a specific domain within a >> card is via a queue which is identified as a (card,domain) tuple. We model >> this something like the following (assuming we have access to cards 3 and >> 4 and domains 1 and 2): >> >> AP -> card3 -> queue (3,1) >> -> queue (3,2) >> -> card4 -> queue (4,1) >> -> queue (4,2) >> >> If we want to virtualize this, we can use a feature provided by the >> hardware. We basically attach a satellite control block to our main >> hardware virtualization control block and the hardware takes care of >> most of the rest. >> >> For this control block, we don't specify explicit tuples, but a list of >> cards and a list of domains. The guest will get access to the cross >> product. >> >> Because of this, we need to take care that the lists provided to >> different guests don't overlap; i.e., we need to enforce sane >> configurations. Otherwise, one guest may get access to things like >> secret keys for another guest. >> >> The idea of this patch set is to introduce a new device, the matrix >> device. This matrix device hangs off a different root and acts as the >> parent node for mdev devices. >> >> If you now want to give the tuples (4,1) and (4,2), you need to do the >> following: >> >> - Make sure the queues (4,1) and (4,2) belong to vfio_ap (see patches >> #5 and #6) >> - Create the mediated device. >> - Assign card 4 and domains 1 and 2 to the mediated device >> - Optionally activate the mediated device. >> >> QEMU will now simply consume the mediated device and things should work. >> >> For a complete description of the architecture and concepts underlying >> the design, see the Documentation/s390/vfio-ap.txt file included with this >> patch set. > > I did not spot anything major, and if v11 addresses the issues raised > by various reviewers I don't see why it should not be merged (interface > looks sane). I skipped looking at the vsie stuff, though ;) I think David has that covered. >
On 09/24/2018 07:53 AM, Cornelia Huck wrote: > On Mon, 24 Sep 2018 12:10:37 +0200 > Christian Borntraeger <borntraeger@de.ibm.com> wrote: > >> FWIW, assuming that review comments for v10 will be addresses in v11, I plan to >> add the upcoming v11 to a topic branch and will merge that after 2 or 3 days after >> v11 in kvms390/next. > > As said before, I think this looks sane enough to merge with the > requested changes. > >> In the future the vfio-ap driver will likely be handled by >> Martins s390 tree, so I guess it makes sense for him to merge the upcoming >> topic branch as well. I will coordinate with Martin. > > I assume future patches will still be posted to kvm + linux-s390 for > review? Affirmative >
On 09/24/2018 06:10 AM, Christian Borntraeger wrote: > FWIW, assuming that review comments for v10 will be addresses in v11, I plan to > add the upcoming v11 to a topic branch and will merge that after 2 or 3 days after > v11 in kvms390/next. In the future the vfio-ap driver will likely be handled by > Martins s390 tree, so I guess it makes sense for him to merge the upcoming > topic branch as well. I will coordinate with Martin. > > Christian As soon as I see that review comments have stopped coming, I'll post v11 .... probably later this week. It will include all changes accepted from review. > > > On 09/12/2018 09:42 PM, Tony Krowiak wrote: >> From: Tony Krowiak <akrowiak@linux.ibm.com> >> >> Notes: >> ===== >> >> Patches 1-2 (by David) are posted with this series because they are not >> currently available in our master branch, upon which this series is based, >> and because this series is dependent upon them. >> >> This patch series works with the v8 QEMU patches. >> >> Abstract: >> ======== >> >> On s390, we have cryptographic coprocessor cards, which are modeled on >> Linux as devices on the AP bus. Each card can be partitioned into domains >> which can be thought of as a set of hardware registers for processing >> crypto commands. Crypto commands are sent to a specific domain within a >> card is via a queue which is identified as a (card,domain) tuple. We model >> this something like the following (assuming we have access to cards 3 and >> 4 and domains 1 and 2): >> >> AP -> card3 -> queue (3,1) >> -> queue (3,2) >> -> card4 -> queue (4,1) >> -> queue (4,2) >> >> If we want to virtualize this, we can use a feature provided by the >> hardware. We basically attach a satellite control block to our main >> hardware virtualization control block and the hardware takes care of >> most of the rest. >> >> For this control block, we don't specify explicit tuples, but a list of >> cards and a list of domains. The guest will get access to the cross >> product. >> >> Because of this, we need to take care that the lists provided to >> different guests don't overlap; i.e., we need to enforce sane >> configurations. Otherwise, one guest may get access to things like >> secret keys for another guest. >> >> The idea of this patch set is to introduce a new device, the matrix >> device. This matrix device hangs off a different root and acts as the >> parent node for mdev devices. >> >> If you now want to give the tuples (4,1) and (4,2), you need to do the >> following: >> >> - Make sure the queues (4,1) and (4,2) belong to vfio_ap (see patches >> #5 and #6) >> - Create the mediated device. >> - Assign card 4 and domains 1 and 2 to the mediated device >> - Optionally activate the mediated device. >> >> QEMU will now simply consume the mediated device and things should work. >> >> For a complete description of the architecture and concepts underlying >> the design, see the Documentation/s390/vfio-ap.txt file included with this >> patch set. >> >> v9 => v10 Change log: >> =================== >> * Replaced statically allocated with dynamically allocated matrix device >> * Made changes to drivers/iommu/Kconfig and arch/s390/Kconfig to fix the >> dependency chain so that make menuconfig can be used to configure VFIO_AP >> * Added KVM device attributes to enable/disable hw interpretation of AP >> instructions from userspace. >> * Return more meaningful error values from mediated matrix device >> assignment sysfs interfaces >> * No longer enforcing convention that the ADM is a superset of the AQM at >> for guest level 2 >> * Broke 2 vSIE patches into several to make it easier to review them >> * Reworked vSIE patches to handle various CRYCB formats >> v8 => v9 Change log: >> =================== >> * Removed /sys/devices/virtual/misc/vfio_ap device and restored >> /sys/devices/vfio_ap/matrix device as parent of mediated matrix devices >> * Return boolean from ap_configuration_available() in ap.h instead of 0 or >> an error. >> * Miscellaneous changes due to review comments >> >> v7 => v8 Change log: >> =================== >> * Removed the AP bus gained the ability to designate queues as 'used by >> host' or as 'used by alternate driver(s)'. >> * Removed 'activate' attribute from mediated device. >> * Do consistency checking during device assignment: >> 1. Verify that APQNs assigned to the mediated device are bound to the >> VFIO AP device driver >> 2. Verify that no APQN assigned to the mediated matrix device is assigned >> to any other mediated matrix device. >> * The attributes of a mediated matrix device that is in use by a guest can >> not be changed - i.e., no device assignment/unassignment allowed >> * A mediated matrix device that is in use by a guest can not be removed. >> * Removed all printk logging from VFIO AP driver; allowing return codes >> from interfaces to describe the error. >> * Reworked the handling of the CRYCB in vSIE based upon patches introduced >> by David in the mainline. >> >> v6 => v7 Change log: >> =================== >> * The AP bus gained the ability to designate queues as 'used by host' >> or as 'used by alternate driver(s)'. This allows us to authorise access >> (via the CRYCB) to queues that are not currently bound to the vfio_ap >> driver. If a vfio_ap owned queue diss- and reapears it's guaranteed >> to get bound back to the vfio_ap driver. >> * The mediated device gained an 'activate' attribute. Sharing conflicts are >> checked on activation now. If the device was not activated, the mdev >> open still implies activation. An active ap_matrix_mdev device claims >> it's resources -- an inactive does not. >> * An active ap_matrix_mdev device can not be removed. An ap_matrix_mdev >> that is hooked up with a guest can not be deactivated. >> * An active ap_matrix_mdev device rejects assign_* and deassign_* >> operations. Thus changing the CRYCB masks of a guest in order to >> accomplys certain hotplug scenarios is planned, but not supported yet. In >> previous versions it was possible to do those operations on a ap_matrix_mdev >> that is hooked up to a guest, but the changes would take effect on the next >> mdev_open. >> * Synchronisation was reworked. >> * The sysfs path of the parent device changed from /sys/devices/vfio_ap/matrix/ >> to /sys/devices/virtual/misc/vfio_ap/. The parent device is a misc >> device now. >> * The severity for most of the messages were reduced form error to >> warning. >> * We are not as thick headed about the zapq as we used to be in v6. >> >> v5 => v6 Change log: >> =================== >> * Added VSIE support - thanks to Pierre Morel >> * Added VFIO_DEVICE_RESET ioctl >> * Zeroizing AP queues when mediated device released and when >> VFIO_DEVICE_RESET ioctl is invoked >> * Removed /arch/s390/kvm/kvm-ap.c and arch/s390/include/asm/kvm-ap.h and >> moved guest matrix configuration into vfio driver >> * Removed temporary interfaces to be supplied by AP bus >> * Made the variable that keeps track of mdev instance count an atomic_t >> type >> * Removed code iterating through vm_list to determine if another guest has >> a queue .... not keep a list of matrix_mdev devices and verify against >> that list. Removes the need for the kvm_lock. >> * Added a sysfs attribute for the mediated matrix device to display the >> matrix contained in the guest's CRYCB if a guest is using the mdev. >> >> >> v4 => v5 Change log: >> =================== >> * Verify AP queues bound to driver in mediated device open callback, prior >> to configuring the matrix in the CRYCB >> * Implement VFIO_DEVICE_RESET ioctl >> * Zeroize queues on guest entry and exit >> * Removed vnet from all email IBM email addresses referenced >> * Add synchronization in mdev create/remove and open/release. >> >> v4 => v5 Change log: >> =================== >> * Added code to mdev open callback to ensure not more than one vfio-ap >> device can be opened by a guest. >> * Interpret AP instructions by default >> * Removed patch implementing interface to enable/disable AP interpretation, >> since that will now be done by default >> * Removed patch to reset crypto attributes for ALL vcpus. That will be >> submitted as a single patch since it will not be needed in this series - >> i.e., it was called from the interface to enable/disable AP instructions >> * All code for initializing crypto for a guest has been moved back to >> kvm-s390.c, kvm_s390_crypto_init(kvm) function >> * Maintaining a module reference count for the vfio_ap module so it is not >> removed while a guest with AP devices is running. >> >> >> >> v3 => v4 Change log: >> =================== >> * Resolved issue with enabling ZCRYPT when KVM is enabled by using >> #ifdef ZCRYPT in relevant functions >> * Added patch with a new function for resetting the crypto attributes >> for all vcpus to resolve the issue raised with running vcpus getting out >> of sync. >> * Removed KVM_S390_VM_CRYPTO_INTERPRET_AP: Setting interpretive exec mode >> from vfio_ap driver when mdev device is opened. >> >> >> v2 => v3 Change log: >> =================== >> * Set APIE in VCPU setup function >> * Renamed patch 13/15: >> KVM: s390: Configure the guest's CRYCB >> KVM: s390: Configure the guest's AP devices >> * Fixed problem with building arch/s390/kvm/kvm-ap.c when CONFIG_ZCRYPT >> not selected >> * Removed patch introducing VSIE support for AP pending further >> investigation >> * Initialized AP maximum mask sizes - i.e., APM, AQM and ADM - from info >> returned from PQAP(QCI) function >> * Introduced a new device attribute to the KVM_S390_VM_CRYPTO attribute >> group for setting a flag via the KVM_SET_DEVICE_ATTR ioctl to indicate >> whether ECA_APIE should be set or not. The flag is used in the >> kvm_s390_vcpu_crypto_setup() function to set ECA_APIE in the SIE block. >> >> >> v1 => v2 Change log: >> =================== >> * Added documentation vfio-ap.txt >> * Renamed vfio_ap_matrix module and device driver to vfio_ap >> * Use device core device list instead of maintaining list of matrix >> devices in driver >> * Added VSIE support for AP >> * Create matrix device before registering VFIO AP device driver with the >> AP bus >> * Renamed the following files in drivers/s390/crypto: >> * vfio_ap_matrix.drv -> vfio_ap_drv >> * vfio_ap_matrix_private.h -> vfio_ap_private.h >> * vfio_ap_matrix_ops.c -> vfio_ap_ops.c >> * arch/s390/include/asm/kvm/ap-matrix-config.h >> * Renamed to kvm-ap.h >> * Changed the data type of the bit mask fields for the matrix structure >> to unsigned long and create them with DECLARE_BITMAP >> * Changed #define prefixes from AP_MATRIX to KVM_AP >> * Changed function and structure prefixes from ap_matrix to kvm_ap >> * Added function interface to check if AP Extended Addressing (APXA) >> facility is installedCRYCB_FORMAT_MASK >> * Added function interface to get the maximum ID for AP mask type >> * Added function interface to set the AP execution mode >> * arch/s390/kvm/ap-matrix-config.c >> * Renamed to kvm-ap.c >> * Changed function prefixes from ap_matrix to kvm_ap >> * Added function to check if AP Extended Addressing (APXA) facility is >> installed >> * Added function to get the maximum ID for AP mask type >> * Added function to set the AP execution mode >> * Added a boolean parameter to the functions that retrieve the APM, AQM >> and ADM bit mask fields from the CRYCB. If true, then the function >> will clear the bits in the mask before returning a reference to it >> * Added validation to verify that APM, AQM and ADM bits that are set do >> not exceed the maximum ID value allowed >> * >> * arch/s390/include/asm/kvm_host.h >> * Changed defined for ECA_AP to ECA_APIE - interpretive execution mode >> * Added a flag to struct kvm_s390_crypto to indicate whether the >> KVM_S390_VM_CPU_FEAT_AP CPU model feature for AP facilities is set >> * Added two CPU facilities features to set STFLE.12 and STFLE.15 >> * arch/s390/kvm/kvm-s390.c >> * Added initialization for new KVM_S390_VM_CPU_FEAT_AP CPU model feature >> * Removed kvm_s390_apxa_installed() function >> * Changed call to kvm_s390_apxa_installed() which has been removed to a >> call to new kvm_ap_apxa_installed() function. >> * Added code to kvm_s390_vcpu_crypto_setup() to set the new CPU model >> feature flag in the kvm_s390_crypto structure >> * Added CRYCB_FORMAT_MASK to mask CRYCBD >> * arch/s390/tools/gen_facilities.c >> * Added STFLE.12 and STFLE.15 to struct facility _def >> * drivers/s390/crypto/vfio_ap_matrix_private.h >> * Changed name of file to vfio_ap.private.h >> * Changed #define prefixes from VFIO_AP_MATRIX to VFIO_AP >> * struct ap_matrix: removed list fields and locks >> * struct vfio_ap_queue: removed list field >> * Renamed functions ap_matrix_mdev_register and ap_matrix_mdev_unregister >> to vfio_ap_mdev_register and vfio_ap_mdev_unregister respectively >> * drivers/s390/crypto/vfio_ap_matrix_drv.c >> * Renamed file to drivers/s390/crypto/vfio_ap_drv.c >> * Changed all #define, structure and function prefixes to vfio_ap >> * probe function >> * Changed root device name for the matrix device to vfio_ap: >> i.e., /sys/devices/vfio_ap/matrix >> * No longer storing the AP queue device in a list, it is retrievable via >> the device core >> * Removed unnecessary check whether matrix device exists >> * Store the vfio_ap_queue structure in the private field of the ap_queue >> structure rather than using list interface >> * remove function >> * Retrieve vfio_ap_queue structure from the struct ap_queue private >> data rather than from a list >> * Removed unnecesary check >> * drivers/s390/crypto/vfio_ap_matrix_ops.c >> * Renamed file to vfio_ap_ops.c >> * Changed #define prefixes from AP_MATRIX to VFIO_AP >> * Changed function name prefixes from ap_matrix to vfio_ap >> * Removed ioctl to configure the CRYCB >> * create function >> * Removed ap_matrix_mdev_find_by_uuid() function - function is provided >> by mdev core >> * Removed available_instances verification, provided by mdev core >> * Removed check to see if mediated device exists, handled by mdev core >> * notifier function >> * Configuring matrix here instead of via ioctl >> * Set interpretive execution mode for all VCPUs >> * Removed R/O attributes to display adapters and domains >> * Added an R/O attribute to display the matrix >> * assign_control_domain mdev attribute: >> * Removed check to see if the domain is installed on the linux host >> * Added check to verify the control domain ID does not exceed the max >> value >> * assign_adapter mdev attribute: >> * Added check to verify the adapter ID does not exceed the max >> value >> * If any APQNs configured for the mediated matrix device that >> have an APID matching the adapter ID being assigned are not >> bound to the vfio_ap device driver then it is assumed that the APQN is >> bound to another driver and assignment will fail >> * assign_domain mdev attribute: >> * Added check to verify the domain ID does not exceed the max >> value >> * If any APQNs configured for the mediated matrix device that >> have an APQI matching the domain ID being assigned are not >> bound to the vfio_ap device driver then it is assumed that the APQN is >> bound to another driver and assignment will fail >> * tools/arch/s390/include/uapi/asm/kvm.h >> * removed KVM_S390_VM_CPU_FEAT_AP feature definition >> >> David Hildenbrand (2): >> KVM: s390: vsie: simulate VCPU SIE entry/exit >> KVM: s390: introduce and use KVM_REQ_VSIE_RESTART >> >> Pierre Morel (9): >> KVM: s390: Clear Crypto Control Block when using vSIE >> KVM: s390: vsie: Do the CRYCB validation first >> KVM: s390: vsie: Make use of CRYCB FORMAT2 clear >> KVM: s390: vsie: Allow CRYCB FORMAT-2 >> KVM: s390: vsie: allow CRYCB FORMAT-1 >> KVM: s390: vsie: allow CRYCB FORMAT-0 >> KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-1 >> KVM: s390: vsie: allow guest FORMAT-1 CRYCB on host FORMAT-2 >> KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-2 >> >> Tony Krowiak (15): >> KVM: s390: refactor crypto initialization >> s390: vfio-ap: base implementation of VFIO AP device driver >> s390: vfio-ap: register matrix device with VFIO mdev framework >> s390: vfio-ap: sysfs interfaces to configure adapters >> s390: vfio-ap: sysfs interfaces to configure domains >> s390: vfio-ap: sysfs interfaces to configure control domains >> s390: vfio-ap: sysfs interface to view matrix mdev matrix >> KVM: s390: interfaces to clear CRYCB masks >> s390: vfio-ap: implement mediated device open callback >> s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl >> s390: vfio-ap: zeroize the AP queues >> s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl >> KVM: s390: device attrs to enable/disable AP interpretation >> KVM: s390: CPU model support for AP virtualization >> s390: doc: detailed specifications for AP virtualization >> >> Documentation/s390/vfio-ap.txt | 778 ++++++++++++++++++++++++++ >> MAINTAINERS | 12 + >> arch/s390/Kconfig | 11 + >> arch/s390/include/asm/kvm_host.h | 7 + >> arch/s390/include/uapi/asm/kvm.h | 3 + >> arch/s390/kvm/kvm-s390.c | 134 +++-- >> arch/s390/kvm/kvm-s390.h | 1 + >> arch/s390/kvm/vsie.c | 208 +++++++- >> arch/s390/tools/gen_facilities.c | 2 + >> drivers/iommu/Kconfig | 8 + >> drivers/s390/crypto/Makefile | 4 + >> drivers/s390/crypto/vfio_ap_drv.c | 153 ++++++ >> drivers/s390/crypto/vfio_ap_ops.c | 973 +++++++++++++++++++++++++++++++++ >> drivers/s390/crypto/vfio_ap_private.h | 88 +++ >> include/uapi/linux/vfio.h | 2 + >> 15 files changed, 2324 insertions(+), 60 deletions(-) >> create mode 100644 Documentation/s390/vfio-ap.txt >> create mode 100644 drivers/s390/crypto/vfio_ap_drv.c >> create mode 100644 drivers/s390/crypto/vfio_ap_ops.c >> create mode 100644 drivers/s390/crypto/vfio_ap_private.h >>
From: Tony Krowiak <akrowiak@linux.ibm.com> Notes: ===== Patches 1-2 (by David) are posted with this series because they are not currently available in our master branch, upon which this series is based, and because this series is dependent upon them. This patch series works with the v8 QEMU patches. Abstract: ======== On s390, we have cryptographic coprocessor cards, which are modeled on Linux as devices on the AP bus. Each card can be partitioned into domains which can be thought of as a set of hardware registers for processing crypto commands. Crypto commands are sent to a specific domain within a card is via a queue which is identified as a (card,domain) tuple. We model this something like the following (assuming we have access to cards 3 and 4 and domains 1 and 2): AP -> card3 -> queue (3,1) -> queue (3,2) -> card4 -> queue (4,1) -> queue (4,2) If we want to virtualize this, we can use a feature provided by the hardware. We basically attach a satellite control block to our main hardware virtualization control block and the hardware takes care of most of the rest. For this control block, we don't specify explicit tuples, but a list of cards and a list of domains. The guest will get access to the cross product. Because of this, we need to take care that the lists provided to different guests don't overlap; i.e., we need to enforce sane configurations. Otherwise, one guest may get access to things like secret keys for another guest. The idea of this patch set is to introduce a new device, the matrix device. This matrix device hangs off a different root and acts as the parent node for mdev devices. If you now want to give the tuples (4,1) and (4,2), you need to do the following: - Make sure the queues (4,1) and (4,2) belong to vfio_ap (see patches #5 and #6) - Create the mediated device. - Assign card 4 and domains 1 and 2 to the mediated device - Optionally activate the mediated device. QEMU will now simply consume the mediated device and things should work. For a complete description of the architecture and concepts underlying the design, see the Documentation/s390/vfio-ap.txt file included with this patch set. v9 => v10 Change log: =================== * Replaced statically allocated with dynamically allocated matrix device * Made changes to drivers/iommu/Kconfig and arch/s390/Kconfig to fix the dependency chain so that make menuconfig can be used to configure VFIO_AP * Added KVM device attributes to enable/disable hw interpretation of AP instructions from userspace. * Return more meaningful error values from mediated matrix device assignment sysfs interfaces * No longer enforcing convention that the ADM is a superset of the AQM at for guest level 2 * Broke 2 vSIE patches into several to make it easier to review them * Reworked vSIE patches to handle various CRYCB formats v8 => v9 Change log: =================== * Removed /sys/devices/virtual/misc/vfio_ap device and restored /sys/devices/vfio_ap/matrix device as parent of mediated matrix devices * Return boolean from ap_configuration_available() in ap.h instead of 0 or an error. * Miscellaneous changes due to review comments v7 => v8 Change log: =================== * Removed the AP bus gained the ability to designate queues as 'used by host' or as 'used by alternate driver(s)'. * Removed 'activate' attribute from mediated device. * Do consistency checking during device assignment: 1. Verify that APQNs assigned to the mediated device are bound to the VFIO AP device driver 2. Verify that no APQN assigned to the mediated matrix device is assigned to any other mediated matrix device. * The attributes of a mediated matrix device that is in use by a guest can not be changed - i.e., no device assignment/unassignment allowed * A mediated matrix device that is in use by a guest can not be removed. * Removed all printk logging from VFIO AP driver; allowing return codes from interfaces to describe the error. * Reworked the handling of the CRYCB in vSIE based upon patches introduced by David in the mainline. v6 => v7 Change log: =================== * The AP bus gained the ability to designate queues as 'used by host' or as 'used by alternate driver(s)'. This allows us to authorise access (via the CRYCB) to queues that are not currently bound to the vfio_ap driver. If a vfio_ap owned queue diss- and reapears it's guaranteed to get bound back to the vfio_ap driver. * The mediated device gained an 'activate' attribute. Sharing conflicts are checked on activation now. If the device was not activated, the mdev open still implies activation. An active ap_matrix_mdev device claims it's resources -- an inactive does not. * An active ap_matrix_mdev device can not be removed. An ap_matrix_mdev that is hooked up with a guest can not be deactivated. * An active ap_matrix_mdev device rejects assign_* and deassign_* operations. Thus changing the CRYCB masks of a guest in order to accomplys certain hotplug scenarios is planned, but not supported yet. In previous versions it was possible to do those operations on a ap_matrix_mdev that is hooked up to a guest, but the changes would take effect on the next mdev_open. * Synchronisation was reworked. * The sysfs path of the parent device changed from /sys/devices/vfio_ap/matrix/ to /sys/devices/virtual/misc/vfio_ap/. The parent device is a misc device now. * The severity for most of the messages were reduced form error to warning. * We are not as thick headed about the zapq as we used to be in v6. v5 => v6 Change log: =================== * Added VSIE support - thanks to Pierre Morel * Added VFIO_DEVICE_RESET ioctl * Zeroizing AP queues when mediated device released and when VFIO_DEVICE_RESET ioctl is invoked * Removed /arch/s390/kvm/kvm-ap.c and arch/s390/include/asm/kvm-ap.h and moved guest matrix configuration into vfio driver * Removed temporary interfaces to be supplied by AP bus * Made the variable that keeps track of mdev instance count an atomic_t type * Removed code iterating through vm_list to determine if another guest has a queue .... not keep a list of matrix_mdev devices and verify against that list. Removes the need for the kvm_lock. * Added a sysfs attribute for the mediated matrix device to display the matrix contained in the guest's CRYCB if a guest is using the mdev. v4 => v5 Change log: =================== * Verify AP queues bound to driver in mediated device open callback, prior to configuring the matrix in the CRYCB * Implement VFIO_DEVICE_RESET ioctl * Zeroize queues on guest entry and exit * Removed vnet from all email IBM email addresses referenced * Add synchronization in mdev create/remove and open/release. v4 => v5 Change log: =================== * Added code to mdev open callback to ensure not more than one vfio-ap device can be opened by a guest. * Interpret AP instructions by default * Removed patch implementing interface to enable/disable AP interpretation, since that will now be done by default * Removed patch to reset crypto attributes for ALL vcpus. That will be submitted as a single patch since it will not be needed in this series - i.e., it was called from the interface to enable/disable AP instructions * All code for initializing crypto for a guest has been moved back to kvm-s390.c, kvm_s390_crypto_init(kvm) function * Maintaining a module reference count for the vfio_ap module so it is not removed while a guest with AP devices is running. v3 => v4 Change log: =================== * Resolved issue with enabling ZCRYPT when KVM is enabled by using #ifdef ZCRYPT in relevant functions * Added patch with a new function for resetting the crypto attributes for all vcpus to resolve the issue raised with running vcpus getting out of sync. * Removed KVM_S390_VM_CRYPTO_INTERPRET_AP: Setting interpretive exec mode from vfio_ap driver when mdev device is opened. v2 => v3 Change log: =================== * Set APIE in VCPU setup function * Renamed patch 13/15: KVM: s390: Configure the guest's CRYCB KVM: s390: Configure the guest's AP devices * Fixed problem with building arch/s390/kvm/kvm-ap.c when CONFIG_ZCRYPT not selected * Removed patch introducing VSIE support for AP pending further investigation * Initialized AP maximum mask sizes - i.e., APM, AQM and ADM - from info returned from PQAP(QCI) function * Introduced a new device attribute to the KVM_S390_VM_CRYPTO attribute group for setting a flag via the KVM_SET_DEVICE_ATTR ioctl to indicate whether ECA_APIE should be set or not. The flag is used in the kvm_s390_vcpu_crypto_setup() function to set ECA_APIE in the SIE block. v1 => v2 Change log: =================== * Added documentation vfio-ap.txt * Renamed vfio_ap_matrix module and device driver to vfio_ap * Use device core device list instead of maintaining list of matrix devices in driver * Added VSIE support for AP * Create matrix device before registering VFIO AP device driver with the AP bus * Renamed the following files in drivers/s390/crypto: * vfio_ap_matrix.drv -> vfio_ap_drv * vfio_ap_matrix_private.h -> vfio_ap_private.h * vfio_ap_matrix_ops.c -> vfio_ap_ops.c * arch/s390/include/asm/kvm/ap-matrix-config.h * Renamed to kvm-ap.h * Changed the data type of the bit mask fields for the matrix structure to unsigned long and create them with DECLARE_BITMAP * Changed #define prefixes from AP_MATRIX to KVM_AP * Changed function and structure prefixes from ap_matrix to kvm_ap * Added function interface to check if AP Extended Addressing (APXA) facility is installedCRYCB_FORMAT_MASK * Added function interface to get the maximum ID for AP mask type * Added function interface to set the AP execution mode * arch/s390/kvm/ap-matrix-config.c * Renamed to kvm-ap.c * Changed function prefixes from ap_matrix to kvm_ap * Added function to check if AP Extended Addressing (APXA) facility is installed * Added function to get the maximum ID for AP mask type * Added function to set the AP execution mode * Added a boolean parameter to the functions that retrieve the APM, AQM and ADM bit mask fields from the CRYCB. If true, then the function will clear the bits in the mask before returning a reference to it * Added validation to verify that APM, AQM and ADM bits that are set do not exceed the maximum ID value allowed * * arch/s390/include/asm/kvm_host.h * Changed defined for ECA_AP to ECA_APIE - interpretive execution mode * Added a flag to struct kvm_s390_crypto to indicate whether the KVM_S390_VM_CPU_FEAT_AP CPU model feature for AP facilities is set * Added two CPU facilities features to set STFLE.12 and STFLE.15 * arch/s390/kvm/kvm-s390.c * Added initialization for new KVM_S390_VM_CPU_FEAT_AP CPU model feature * Removed kvm_s390_apxa_installed() function * Changed call to kvm_s390_apxa_installed() which has been removed to a call to new kvm_ap_apxa_installed() function. * Added code to kvm_s390_vcpu_crypto_setup() to set the new CPU model feature flag in the kvm_s390_crypto structure * Added CRYCB_FORMAT_MASK to mask CRYCBD * arch/s390/tools/gen_facilities.c * Added STFLE.12 and STFLE.15 to struct facility _def * drivers/s390/crypto/vfio_ap_matrix_private.h * Changed name of file to vfio_ap.private.h * Changed #define prefixes from VFIO_AP_MATRIX to VFIO_AP * struct ap_matrix: removed list fields and locks * struct vfio_ap_queue: removed list field * Renamed functions ap_matrix_mdev_register and ap_matrix_mdev_unregister to vfio_ap_mdev_register and vfio_ap_mdev_unregister respectively * drivers/s390/crypto/vfio_ap_matrix_drv.c * Renamed file to drivers/s390/crypto/vfio_ap_drv.c * Changed all #define, structure and function prefixes to vfio_ap * probe function * Changed root device name for the matrix device to vfio_ap: i.e., /sys/devices/vfio_ap/matrix * No longer storing the AP queue device in a list, it is retrievable via the device core * Removed unnecessary check whether matrix device exists * Store the vfio_ap_queue structure in the private field of the ap_queue structure rather than using list interface * remove function * Retrieve vfio_ap_queue structure from the struct ap_queue private data rather than from a list * Removed unnecesary check * drivers/s390/crypto/vfio_ap_matrix_ops.c * Renamed file to vfio_ap_ops.c * Changed #define prefixes from AP_MATRIX to VFIO_AP * Changed function name prefixes from ap_matrix to vfio_ap * Removed ioctl to configure the CRYCB * create function * Removed ap_matrix_mdev_find_by_uuid() function - function is provided by mdev core * Removed available_instances verification, provided by mdev core * Removed check to see if mediated device exists, handled by mdev core * notifier function * Configuring matrix here instead of via ioctl * Set interpretive execution mode for all VCPUs * Removed R/O attributes to display adapters and domains * Added an R/O attribute to display the matrix * assign_control_domain mdev attribute: * Removed check to see if the domain is installed on the linux host * Added check to verify the control domain ID does not exceed the max value * assign_adapter mdev attribute: * Added check to verify the adapter ID does not exceed the max value * If any APQNs configured for the mediated matrix device that have an APID matching the adapter ID being assigned are not bound to the vfio_ap device driver then it is assumed that the APQN is bound to another driver and assignment will fail * assign_domain mdev attribute: * Added check to verify the domain ID does not exceed the max value * If any APQNs configured for the mediated matrix device that have an APQI matching the domain ID being assigned are not bound to the vfio_ap device driver then it is assumed that the APQN is bound to another driver and assignment will fail * tools/arch/s390/include/uapi/asm/kvm.h * removed KVM_S390_VM_CPU_FEAT_AP feature definition David Hildenbrand (2): KVM: s390: vsie: simulate VCPU SIE entry/exit KVM: s390: introduce and use KVM_REQ_VSIE_RESTART Pierre Morel (9): KVM: s390: Clear Crypto Control Block when using vSIE KVM: s390: vsie: Do the CRYCB validation first KVM: s390: vsie: Make use of CRYCB FORMAT2 clear KVM: s390: vsie: Allow CRYCB FORMAT-2 KVM: s390: vsie: allow CRYCB FORMAT-1 KVM: s390: vsie: allow CRYCB FORMAT-0 KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-1 KVM: s390: vsie: allow guest FORMAT-1 CRYCB on host FORMAT-2 KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-2 Tony Krowiak (15): KVM: s390: refactor crypto initialization s390: vfio-ap: base implementation of VFIO AP device driver s390: vfio-ap: register matrix device with VFIO mdev framework s390: vfio-ap: sysfs interfaces to configure adapters s390: vfio-ap: sysfs interfaces to configure domains s390: vfio-ap: sysfs interfaces to configure control domains s390: vfio-ap: sysfs interface to view matrix mdev matrix KVM: s390: interfaces to clear CRYCB masks s390: vfio-ap: implement mediated device open callback s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl s390: vfio-ap: zeroize the AP queues s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl KVM: s390: device attrs to enable/disable AP interpretation KVM: s390: CPU model support for AP virtualization s390: doc: detailed specifications for AP virtualization Documentation/s390/vfio-ap.txt | 778 ++++++++++++++++++++++++++ MAINTAINERS | 12 + arch/s390/Kconfig | 11 + arch/s390/include/asm/kvm_host.h | 7 + arch/s390/include/uapi/asm/kvm.h | 3 + arch/s390/kvm/kvm-s390.c | 134 +++-- arch/s390/kvm/kvm-s390.h | 1 + arch/s390/kvm/vsie.c | 208 +++++++- arch/s390/tools/gen_facilities.c | 2 + drivers/iommu/Kconfig | 8 + drivers/s390/crypto/Makefile | 4 + drivers/s390/crypto/vfio_ap_drv.c | 153 ++++++ drivers/s390/crypto/vfio_ap_ops.c | 973 +++++++++++++++++++++++++++++++++ drivers/s390/crypto/vfio_ap_private.h | 88 +++ include/uapi/linux/vfio.h | 2 + 15 files changed, 2324 insertions(+), 60 deletions(-) create mode 100644 Documentation/s390/vfio-ap.txt create mode 100644 drivers/s390/crypto/vfio_ap_drv.c create mode 100644 drivers/s390/crypto/vfio_ap_ops.c create mode 100644 drivers/s390/crypto/vfio_ap_private.h