From patchwork Wed Mar 13 16:04:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 10851453 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 87EF2139A for ; Wed, 13 Mar 2019 16:06:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7014C2A087 for ; Wed, 13 Mar 2019 16:06:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6E3092A0B4; Wed, 13 Mar 2019 16:06:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C7CD72A087 for ; Wed, 13 Mar 2019 16:05:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726733AbfCMQFT (ORCPT ); Wed, 13 Mar 2019 12:05:19 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:52496 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725876AbfCMQFO (ORCPT ); Wed, 13 Mar 2019 12:05:14 -0400 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x2DG4I9O099876 for ; Wed, 13 Mar 2019 12:05:12 -0400 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 2r749aajq2-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 13 Mar 2019 12:05:12 -0400 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 13 Mar 2019 16:05:10 -0000 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 13 Mar 2019 16:05:08 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x2DG562d14155930 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 13 Mar 2019 16:05:06 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5A12DA404D; Wed, 13 Mar 2019 16:05:06 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E8930A4040; Wed, 13 Mar 2019 16:05:05 +0000 (GMT) Received: from morel-ThinkPad-W530.boeblingen.de.ibm.com (unknown [9.152.224.140]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 13 Mar 2019 16:05:05 +0000 (GMT) From: Pierre Morel To: borntraeger@de.ibm.com Cc: alex.williamson@redhat.com, cohuck@redhat.com, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, frankja@linux.ibm.com, akrowiak@linux.ibm.com, pasic@linux.ibm.com, david@redhat.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, freude@linux.ibm.com, mimu@linux.ibm.com Subject: [PATCH v5 1/7] s390: ap: kvm: add PQAP interception for AQIC Date: Wed, 13 Mar 2019 17:04:58 +0100 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> References: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 19031316-0012-0000-0000-000003022B93 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19031316-0013-0000-0000-000021395277 Message-Id: <1552493104-30510-2-git-send-email-pmorel@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-03-13_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903130114 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We prepare the interception of the PQAP/AQIC instruction for the case the AQIC facility is enabled in the guest. We add a callback inside the KVM arch structure for s390 for a VFIO driver to handle a specific response to the PQAP instruction with the AQIC command and only this command. The preceding behavior for other commands should not change. We inject the correct exceptions from inside KVM for the case the callback is not initialized, which happens when the vfio_ap driver is not loaded. It is the duty of the vfio_driver to setup a pqap callback inside the crypto structure. If the callback has been setup we call it. If not we setup an answer considering that no queue is available for the guest when no callback has been setup. We do consider the responsability of the driver to always initialize the PQAP callback if it defines queues by initializing the CRYCB for a guest. Signed-off-by: Pierre Morel --- arch/s390/include/asm/kvm_host.h | 8 +++++ arch/s390/kvm/priv.c | 62 +++++++++++++++++++++++++++++++++++ drivers/s390/crypto/vfio_ap_private.h | 2 ++ 3 files changed, 72 insertions(+) diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h index a496276..624460b 100644 --- a/arch/s390/include/asm/kvm_host.h +++ b/arch/s390/include/asm/kvm_host.h @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -721,8 +722,15 @@ struct kvm_s390_cpu_model { unsigned short ibc; }; +struct kvm_s390_module_hook { + int (*hook)(struct kvm_vcpu *vcpu); + void *data; + struct module *owner; +}; + struct kvm_s390_crypto { struct kvm_s390_crypto_cb *crycb; + struct kvm_s390_module_hook *pqap_hook; __u32 crycbd; __u8 aes_kw; __u8 dea_kw; diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c index 8679bd7..72f683a 100644 --- a/arch/s390/kvm/priv.c +++ b/arch/s390/kvm/priv.c @@ -27,6 +27,7 @@ #include #include #include +#include #include "gaccess.h" #include "kvm-s390.h" #include "trace.h" @@ -592,6 +593,65 @@ static int handle_io_inst(struct kvm_vcpu *vcpu) } } +/* + * handle_pqap: Handling pqap interception + * @vcpu: the vcpu having issue the pqap instruction + * + * We now support PQAP/AQIC instructions and we need to correctly + * answer the guest even if no dedicated driver's hook is available. + * + * The intercepting code calls a dedicated callback for this instruction + * if a driver did register one in the CRYPTO satellite of the + * SIE block. + * + * For PQAP/AQIC instructions only, verify privilege and specifications. + * + * If no callback available, the queues are not available, return this to + * the caller. + * Else return the value returned by the callback. + */ +static int handle_pqap(struct kvm_vcpu *vcpu) +{ + uint8_t fc; + struct ap_queue_status status = {}; + int ret; + /* Verify that the AP instruction are available */ + if (!ap_instructions_available()) + return -EOPNOTSUPP; + /* Verify that the guest is allowed to use AP instructions */ + if (!(vcpu->arch.sie_block->eca & ECA_APIE)) + return -EOPNOTSUPP; + /* Verify that the function code is AQIC */ + fc = vcpu->run->s.regs.gprs[0] >> 24; + /* We do not want to change the behavior we had before this patch*/ + if (fc != 0x03) + return -EOPNOTSUPP; + + /* PQAP instructions are allowed for guest kernel only */ + if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE) + return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP); + /* AQIC instruction is allowed only if facility 65 is available */ + if (!test_kvm_facility(vcpu->kvm, 65)) + return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION); + /* Verify that the hook callback is registered and call it */ + if (vcpu->kvm->arch.crypto.pqap_hook) { + if (!try_module_get(vcpu->kvm->arch.crypto.pqap_hook->owner)) + return -EOPNOTSUPP; + ret = vcpu->kvm->arch.crypto.pqap_hook->hook(vcpu); + module_put(vcpu->kvm->arch.crypto.pqap_hook->owner); + return ret; + } + /* + * It is the duty of the vfio_driver to register a hook + * If it does not and we get an exception on AQIC we must + * guess that there is no vfio_ap_driver at all and no one + * to handle the guests's CRYCB and the CRYCB is empty. + */ + status.response_code = 0x01; + memcpy(&vcpu->run->s.regs.gprs[1], &status, sizeof(status)); + return 0; +} + static int handle_stfl(struct kvm_vcpu *vcpu) { int rc; @@ -878,6 +938,8 @@ int kvm_s390_handle_b2(struct kvm_vcpu *vcpu) return handle_sthyi(vcpu); case 0x7d: return handle_stsi(vcpu); + case 0xaf: + return handle_pqap(vcpu); case 0xb1: return handle_stfl(vcpu); case 0xb2: diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h index 76b7f98..a910be1 100644 --- a/drivers/s390/crypto/vfio_ap_private.h +++ b/drivers/s390/crypto/vfio_ap_private.h @@ -16,6 +16,7 @@ #include #include #include +#include #include "ap_bus.h" @@ -81,6 +82,7 @@ struct ap_matrix_mdev { struct ap_matrix matrix; struct notifier_block group_notifier; struct kvm *kvm; + struct kvm_s390_module_hook pqap_hook; }; extern int vfio_ap_mdev_register(void); From patchwork Wed Mar 13 16:04:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 10851455 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98408139A for ; Wed, 13 Mar 2019 16:06:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 838D829E5D for ; Wed, 13 Mar 2019 16:06:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 819402A07E; Wed, 13 Mar 2019 16:06:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 128262A07F for ; Wed, 13 Mar 2019 16:06:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726991AbfCMQGB (ORCPT ); Wed, 13 Mar 2019 12:06:01 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:59964 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726708AbfCMQFS (ORCPT ); Wed, 13 Mar 2019 12:05:18 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x2DG4MgM144812 for ; Wed, 13 Mar 2019 12:05:16 -0400 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 2r73jtd6xa-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 13 Mar 2019 12:05:16 -0400 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 13 Mar 2019 16:05:13 -0000 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 13 Mar 2019 16:05:09 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x2DG57PN21168232 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 13 Mar 2019 16:05:08 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DC953A4059; Wed, 13 Mar 2019 16:05:07 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 67FBEA4040; Wed, 13 Mar 2019 16:05:07 +0000 (GMT) Received: from morel-ThinkPad-W530.boeblingen.de.ibm.com (unknown [9.152.224.140]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 13 Mar 2019 16:05:07 +0000 (GMT) From: Pierre Morel To: borntraeger@de.ibm.com Cc: alex.williamson@redhat.com, cohuck@redhat.com, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, frankja@linux.ibm.com, akrowiak@linux.ibm.com, pasic@linux.ibm.com, david@redhat.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, freude@linux.ibm.com, mimu@linux.ibm.com Subject: [PATCH v5 2/7] s390: ap: new vfio_ap_queue structure Date: Wed, 13 Mar 2019 17:04:59 +0100 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> References: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 19031316-0012-0000-0000-000003022B94 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19031316-0013-0000-0000-000021395278 Message-Id: <1552493104-30510-3-git-send-email-pmorel@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-03-13_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903130114 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The AP interruptions are assigned on a queue basis and the GISA structure is handled on a VM basis, so that we need to add a structure we can retrieve from both side holding the information we need to handle PQAP/AQIC interception and setup the GISA. Since we can not add more information to the ap_device we add a new structure vfio_ap_queue, to hold per queue information useful to handle interruptions and set it as driver's data of the standard ap_queue device. Usually, the device and the mediated device are linked together but in the vfio_ap driver design we have a bunch of "sub" devices (the ap_queue devices) belonging to the mediated device. Linking these structure to the mediated device it is assigned to, with the help of the vfio_ap_queue structure will help us to retrieve the AP devices associated with the mediated devices during the mediated device operations. ------------ ------------- | AP queue |--> | AP_vfio_q |<---- ------------ ------^------ | --------------- | <--->| matrix_mdev | ------------ ------v------ | --------------- | AP queue |--> | AP_vfio_q |----- ------------ ------------- The vfio_ap_queue device will hold the following entries: - apqn: AP queue number (defined here) - isc : Interrupt subclass (defined later) - nib : notification information byte (defined later) - list: a list_head entry allowing to link this structure to a matrix mediated device it is assigned to. The vfio_ap_queue structure is allocated when the vfio_ap_driver is probed and added as driver data to the ap_queue device. It is free on remove. The structure is linked to the matrix_dev host device at the probe of the device building some kind of free list for the matrix mediated devices. When the vfio_queue is associated to a matrix mediated device, during assign_adapter or assign_domain, the vfio_ap_queue device is linked to this matrix mediated device and unlinked when dissociated. Queuing the devices on a list of free devices and testing the matrix_mdev pointer to the associated matrix allow us to know if the queue is associated to the matrix device and associated or not to a mediated device. Signed-off-by: Pierre Morel --- drivers/s390/crypto/vfio_ap_drv.c | 31 ++- drivers/s390/crypto/vfio_ap_ops.c | 390 +++++++++++++++++----------------- drivers/s390/crypto/vfio_ap_private.h | 7 + 3 files changed, 234 insertions(+), 194 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_drv.c b/drivers/s390/crypto/vfio_ap_drv.c index e9824c3..df6f21a 100644 --- a/drivers/s390/crypto/vfio_ap_drv.c +++ b/drivers/s390/crypto/vfio_ap_drv.c @@ -40,14 +40,42 @@ static struct ap_device_id ap_queue_ids[] = { MODULE_DEVICE_TABLE(vfio_ap, ap_queue_ids); +/** + * vfio_ap_queue_dev_probe: + * + * Allocate a vfio_ap_queue structure and associate it + * with the device as driver_data. + */ static int vfio_ap_queue_dev_probe(struct ap_device *apdev) { + struct vfio_ap_queue *q; + + q = kzalloc(sizeof(*q), GFP_KERNEL); + if (!q) + return -ENOMEM; + dev_set_drvdata(&apdev->device, q); + q->apqn = to_ap_queue(&apdev->device)->qid; + INIT_LIST_HEAD(&q->list); + mutex_lock(&matrix_dev->lock); + list_add(&q->list, &matrix_dev->free_list); + mutex_unlock(&matrix_dev->lock); return 0; } +/** + * vfio_ap_queue_dev_remove: + * + * Free the associated vfio_ap_queue structure + */ static void vfio_ap_queue_dev_remove(struct ap_device *apdev) { - /* Nothing to do yet */ + struct vfio_ap_queue *q; + + q = dev_get_drvdata(&apdev->device); + mutex_lock(&matrix_dev->lock); + list_del(&q->list); + mutex_unlock(&matrix_dev->lock); + kfree(q); } static void vfio_ap_matrix_dev_release(struct device *dev) @@ -107,6 +135,7 @@ static int vfio_ap_matrix_dev_create(void) matrix_dev->device.bus = &matrix_bus; matrix_dev->device.release = vfio_ap_matrix_dev_release; matrix_dev->vfio_ap_drv = &vfio_ap_drv; + INIT_LIST_HEAD(&matrix_dev->free_list); ret = device_register(&matrix_dev->device); if (ret) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index 900b9cf..485595c 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -24,6 +24,48 @@ #define VFIO_AP_MDEV_TYPE_HWVIRT "passthrough" #define VFIO_AP_MDEV_NAME_HWVIRT "VFIO AP Passthrough Device" +/** + * vfio_ap_get_queue: Retrieve a queue with a specific APQN from a list + * @apqn: The queue APQN + * + * Retrieve a queue with a specific APQN from the list of the + * devices associated with a list. + * + * Returns the pointer to the associated vfio_ap_queue + */ +struct vfio_ap_queue *vfio_ap_get_queue(int apqn, struct list_head *l) +{ + struct vfio_ap_queue *q; + + list_for_each_entry(q, l, list) + if (q->apqn == apqn) + return q; + return NULL; +} + +static int vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q) +{ + struct ap_queue_status status; + int retry = 1; + + do { + status = ap_zapq(q->apqn); + switch (status.response_code) { + case AP_RESPONSE_NORMAL: + return 0; + case AP_RESPONSE_RESET_IN_PROGRESS: + case AP_RESPONSE_BUSY: + msleep(20); + break; + default: + /* things are really broken, give up */ + return -EIO; + } + } while (retry--); + + return -EBUSY; +} + static void vfio_ap_matrix_init(struct ap_config_info *info, struct ap_matrix *matrix) { @@ -45,6 +87,7 @@ static int vfio_ap_mdev_create(struct kobject *kobj, struct mdev_device *mdev) return -ENOMEM; } + INIT_LIST_HEAD(&matrix_mdev->qlist); vfio_ap_matrix_init(&matrix_dev->info, &matrix_mdev->matrix); mdev_set_drvdata(mdev, matrix_mdev); mutex_lock(&matrix_dev->lock); @@ -113,162 +156,178 @@ static struct attribute_group *vfio_ap_mdev_type_groups[] = { NULL, }; -struct vfio_ap_queue_reserved { - unsigned long *apid; - unsigned long *apqi; - bool reserved; -}; +static void vfio_ap_free_queue(int apqn, struct ap_matrix_mdev *matrix_mdev) +{ + struct vfio_ap_queue *q; + + q = vfio_ap_get_queue(apqn, &matrix_mdev->qlist); + if (!q) + return; + q->matrix_mdev = NULL; + vfio_ap_mdev_reset_queue(q); + list_move(&q->list, &matrix_dev->free_list); +} /** - * vfio_ap_has_queue - * - * @dev: an AP queue device - * @data: a struct vfio_ap_queue_reserved reference - * - * Flags whether the AP queue device (@dev) has a queue ID containing the APQN, - * apid or apqi specified in @data: - * - * - If @data contains both an apid and apqi value, then @data will be flagged - * as reserved if the APID and APQI fields for the AP queue device matches + * vfio_ap_put_all_domains: * - * - If @data contains only an apid value, @data will be flagged as - * reserved if the APID field in the AP queue device matches + * @matrix_mdev: the matrix mediated device for which we want to associate + * all available queues with a given apqi. + * @apid: The apid which associated with all defined APQI of the + * mediated device will define a AP queue. * - * - If @data contains only an apqi value, @data will be flagged as - * reserved if the APQI field in the AP queue device matches - * - * Returns 0 to indicate the input to function succeeded. Returns -EINVAL if - * @data does not contain either an apid or apqi. + * We remove the queue from the list of queues associated with the + * mediated device and put them back to the free list of the matrix + * device and clear the matrix_mdev pointer. */ -static int vfio_ap_has_queue(struct device *dev, void *data) +static void vfio_ap_put_all_domains(struct ap_matrix_mdev *matrix_mdev, + int apid) { - struct vfio_ap_queue_reserved *qres = data; - struct ap_queue *ap_queue = to_ap_queue(dev); - ap_qid_t qid; - unsigned long id; + int apqi, apqn; - if (qres->apid && qres->apqi) { - qid = AP_MKQID(*qres->apid, *qres->apqi); - if (qid == ap_queue->qid) - qres->reserved = true; - } else if (qres->apid && !qres->apqi) { - id = AP_QID_CARD(ap_queue->qid); - if (id == *qres->apid) - qres->reserved = true; - } else if (!qres->apid && qres->apqi) { - id = AP_QID_QUEUE(ap_queue->qid); - if (id == *qres->apqi) - qres->reserved = true; - } else { - return -EINVAL; + for_each_set_bit_inv(apqi, matrix_mdev->matrix.aqm, AP_DOMAINS) { + apqn = AP_MKQID(apid, apqi); + vfio_ap_free_queue(apqn, matrix_mdev); } - - return 0; } /** - * vfio_ap_verify_queue_reserved - * - * @matrix_dev: a mediated matrix device - * @apid: an AP adapter ID - * @apqi: an AP queue index - * - * Verifies that the AP queue with @apid/@apqi is reserved by the VFIO AP device - * driver according to the following rules: - * - * - If both @apid and @apqi are not NULL, then there must be an AP queue - * device bound to the vfio_ap driver with the APQN identified by @apid and - * @apqi + * vfio_ap_put_all_cards: * - * - If only @apid is not NULL, then there must be an AP queue device bound - * to the vfio_ap driver with an APQN containing @apid + * @matrix_mdev: the matrix mediated device for which we want to associate + * all available queues with a given apqi. + * @apqi: The apqi which associated with all defined APID of the + * mediated device will define a AP queue. * - * - If only @apqi is not NULL, then there must be an AP queue device bound - * to the vfio_ap driver with an APQN containing @apqi - * - * Returns 0 if the AP queue is reserved; otherwise, returns -EADDRNOTAVAIL. + * We remove the queue from the list of queues associated with the + * mediated device and put them back to the free list of the matrix + * device and clear the matrix_mdev pointer. */ -static int vfio_ap_verify_queue_reserved(unsigned long *apid, - unsigned long *apqi) +static void vfio_ap_put_all_cards(struct ap_matrix_mdev *matrix_mdev, int apqi) { - int ret; - struct vfio_ap_queue_reserved qres; + int apid, apqn; + + for_each_set_bit_inv(apid, matrix_mdev->matrix.apm, AP_DEVICES) { + apqn = AP_MKQID(apid, apqi); + vfio_ap_free_queue(apqn, matrix_mdev); + } +} - qres.apid = apid; - qres.apqi = apqi; - qres.reserved = false; +static void move_and_set(struct list_head *src, struct list_head *dst, + struct ap_matrix_mdev *matrix_mdev) +{ + struct vfio_ap_queue *q, *qtmp; - ret = driver_for_each_device(&matrix_dev->vfio_ap_drv->driver, NULL, - &qres, vfio_ap_has_queue); - if (ret) - return ret; + list_for_each_entry_safe(q, qtmp, src, list) { + list_move(&q->list, dst); + q->matrix_mdev = matrix_mdev; + } +} - if (qres.reserved) - return 0; +static int vfio_ap_queue_match(struct device *dev, void *data) +{ + struct ap_queue *ap; - return -EADDRNOTAVAIL; + ap = to_ap_queue(dev); + return ap->qid == *(int *)data; } -static int -vfio_ap_mdev_verify_queues_reserved_for_apid(struct ap_matrix_mdev *matrix_mdev, - unsigned long apid) +static struct vfio_ap_queue *vfio_ap_find_queue(int apqn) { - int ret; - unsigned long apqi; - unsigned long nbits = matrix_mdev->matrix.aqm_max + 1; + struct device *dev; - if (find_first_bit_inv(matrix_mdev->matrix.aqm, nbits) >= nbits) - return vfio_ap_verify_queue_reserved(&apid, NULL); + dev = driver_find_device(&matrix_dev->vfio_ap_drv->driver, NULL, + &apqn, vfio_ap_queue_match); + return dev ? dev_get_drvdata(dev) : NULL; +} - for_each_set_bit_inv(apqi, matrix_mdev->matrix.aqm, nbits) { - ret = vfio_ap_verify_queue_reserved(&apid, &apqi); - if (ret) - return ret; +/** + * vfio_ap_get_all_domains: + * + * @matrix_mdev: the matrix mediated device for which we want to associate + * all available queues with a given apqi. + * @apqi: The apqi which associated with all defined APID of the + * mediated device will define a AP queue. + * + * We define a local list to put all queues we find on the matrix driver + * device list when associating the apqi with all already defined apid for + * this matrix mediated device. + * + * If we can get all the devices we roll them to the mediated device list + * If we get errors we unroll them to the free list. + */ +static int vfio_ap_get_all_domains(struct ap_matrix_mdev *matrix_mdev, int apid) +{ + int apqi, apqn; + int ret = 0; + struct vfio_ap_queue *q; + struct list_head q_list; + + INIT_LIST_HEAD(&q_list); + + for_each_set_bit_inv(apqi, matrix_mdev->matrix.aqm, AP_DOMAINS) { + apqn = AP_MKQID(apid, apqi); + q = vfio_ap_find_queue(apqn); + if (!q) { + ret = -EADDRNOTAVAIL; + goto rewind; + } + if (q->matrix_mdev) { + ret = -EADDRINUSE; + goto rewind; + } + list_move(&q->list, &q_list); } - + move_and_set(&q_list, &matrix_mdev->qlist, matrix_mdev); return 0; +rewind: + move_and_set(&q_list, &matrix_dev->free_list, NULL); + return ret; } - /** - * vfio_ap_mdev_verify_no_sharing + * vfio_ap_get_all_cards: * - * Verifies that the APQNs derived from the cross product of the AP adapter IDs - * and AP queue indexes comprising the AP matrix are not configured for another - * mediated device. AP queue sharing is not allowed. + * @matrix_mdev: the matrix mediated device for which we want to associate + * all available queues with a given apqi. + * @apqi: The apqi which associated with all defined APID of the + * mediated device will define a AP queue. * - * @matrix_mdev: the mediated matrix device + * We define a local list to put all queues we find on the matrix device + * free list when associating the apqi with all already defined apid for + * this matrix mediated device. * - * Returns 0 if the APQNs are not shared, otherwise; returns -EADDRINUSE. + * If we can get all the devices we roll them to the mediated device list + * If we get errors we unroll them to the free list. */ -static int vfio_ap_mdev_verify_no_sharing(struct ap_matrix_mdev *matrix_mdev) +static int vfio_ap_get_all_cards(struct ap_matrix_mdev *matrix_mdev, int apqi) { - struct ap_matrix_mdev *lstdev; - DECLARE_BITMAP(apm, AP_DEVICES); - DECLARE_BITMAP(aqm, AP_DOMAINS); - - list_for_each_entry(lstdev, &matrix_dev->mdev_list, node) { - if (matrix_mdev == lstdev) - continue; - - memset(apm, 0, sizeof(apm)); - memset(aqm, 0, sizeof(aqm)); - - /* - * We work on full longs, as we can only exclude the leftover - * bits in non-inverse order. The leftover is all zeros. - */ - if (!bitmap_and(apm, matrix_mdev->matrix.apm, - lstdev->matrix.apm, AP_DEVICES)) - continue; - - if (!bitmap_and(aqm, matrix_mdev->matrix.aqm, - lstdev->matrix.aqm, AP_DOMAINS)) - continue; - - return -EADDRINUSE; + int apid, apqn; + int ret = 0; + struct vfio_ap_queue *q; + struct list_head q_list; + struct ap_matrix_mdev *tmp = NULL; + + INIT_LIST_HEAD(&q_list); + + for_each_set_bit_inv(apid, matrix_mdev->matrix.apm, AP_DEVICES) { + apqn = AP_MKQID(apid, apqi); + q = vfio_ap_find_queue(apqn); + if (!q) { + ret = -EADDRNOTAVAIL; + goto rewind; + } + if (q->matrix_mdev) { + ret = -EADDRINUSE; + goto rewind; + } + list_move(&q->list, &q_list); } - + tmp = matrix_mdev; + move_and_set(&q_list, &matrix_mdev->qlist, matrix_mdev); return 0; +rewind: + move_and_set(&q_list, &matrix_dev->free_list, NULL); + return ret; } /** @@ -330,21 +389,15 @@ static ssize_t assign_adapter_store(struct device *dev, */ mutex_lock(&matrix_dev->lock); - ret = vfio_ap_mdev_verify_queues_reserved_for_apid(matrix_mdev, apid); + ret = vfio_ap_get_all_domains(matrix_mdev, apid); if (ret) goto done; set_bit_inv(apid, matrix_mdev->matrix.apm); - ret = vfio_ap_mdev_verify_no_sharing(matrix_mdev); - if (ret) - goto share_err; - ret = count; goto done; -share_err: - clear_bit_inv(apid, matrix_mdev->matrix.apm); done: mutex_unlock(&matrix_dev->lock); @@ -391,32 +444,13 @@ static ssize_t unassign_adapter_store(struct device *dev, mutex_lock(&matrix_dev->lock); clear_bit_inv((unsigned long)apid, matrix_mdev->matrix.apm); + vfio_ap_put_all_domains(matrix_mdev, apid); mutex_unlock(&matrix_dev->lock); return count; } static DEVICE_ATTR_WO(unassign_adapter); -static int -vfio_ap_mdev_verify_queues_reserved_for_apqi(struct ap_matrix_mdev *matrix_mdev, - unsigned long apqi) -{ - int ret; - unsigned long apid; - unsigned long nbits = matrix_mdev->matrix.apm_max + 1; - - if (find_first_bit_inv(matrix_mdev->matrix.apm, nbits) >= nbits) - return vfio_ap_verify_queue_reserved(NULL, &apqi); - - for_each_set_bit_inv(apid, matrix_mdev->matrix.apm, nbits) { - ret = vfio_ap_verify_queue_reserved(&apid, &apqi); - if (ret) - return ret; - } - - return 0; -} - /** * assign_domain_store * @@ -471,21 +505,15 @@ static ssize_t assign_domain_store(struct device *dev, mutex_lock(&matrix_dev->lock); - ret = vfio_ap_mdev_verify_queues_reserved_for_apqi(matrix_mdev, apqi); + ret = vfio_ap_get_all_cards(matrix_mdev, apqi); if (ret) goto done; set_bit_inv(apqi, matrix_mdev->matrix.aqm); - ret = vfio_ap_mdev_verify_no_sharing(matrix_mdev); - if (ret) - goto share_err; - ret = count; goto done; -share_err: - clear_bit_inv(apqi, matrix_mdev->matrix.aqm); done: mutex_unlock(&matrix_dev->lock); @@ -533,6 +561,7 @@ static ssize_t unassign_domain_store(struct device *dev, mutex_lock(&matrix_dev->lock); clear_bit_inv((unsigned long)apqi, matrix_mdev->matrix.aqm); + vfio_ap_put_all_cards(matrix_mdev, apqi); mutex_unlock(&matrix_dev->lock); return count; @@ -790,49 +819,22 @@ static int vfio_ap_mdev_group_notifier(struct notifier_block *nb, return NOTIFY_OK; } -static int vfio_ap_mdev_reset_queue(unsigned int apid, unsigned int apqi, - unsigned int retry) -{ - struct ap_queue_status status; - - do { - status = ap_zapq(AP_MKQID(apid, apqi)); - switch (status.response_code) { - case AP_RESPONSE_NORMAL: - return 0; - case AP_RESPONSE_RESET_IN_PROGRESS: - case AP_RESPONSE_BUSY: - msleep(20); - break; - default: - /* things are really broken, give up */ - return -EIO; - } - } while (retry--); - - return -EBUSY; -} - static int vfio_ap_mdev_reset_queues(struct mdev_device *mdev) { int ret; int rc = 0; - unsigned long apid, apqi; struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); + struct vfio_ap_queue *q; - for_each_set_bit_inv(apid, matrix_mdev->matrix.apm, - matrix_mdev->matrix.apm_max + 1) { - for_each_set_bit_inv(apqi, matrix_mdev->matrix.aqm, - matrix_mdev->matrix.aqm_max + 1) { - ret = vfio_ap_mdev_reset_queue(apid, apqi, 1); - /* - * Regardless whether a queue turns out to be busy, or - * is not operational, we need to continue resetting - * the remaining queues. - */ - if (ret) - rc = ret; - } + list_for_each_entry(q, &matrix_mdev->qlist, list) { + ret = vfio_ap_mdev_reset_queue(q); + /* + * Regardless whether a queue turns out to be busy, or + * is not operational, we need to continue resetting + * the remaining queues but notice the last error code. + */ + if (ret) + rc = ret; } return rc; @@ -868,10 +870,10 @@ static void vfio_ap_mdev_release(struct mdev_device *mdev) if (matrix_mdev->kvm) kvm_arch_crypto_clear_masks(matrix_mdev->kvm); + matrix_mdev->kvm = NULL; vfio_ap_mdev_reset_queues(mdev); vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, &matrix_mdev->group_notifier); - matrix_mdev->kvm = NULL; module_put(THIS_MODULE); } @@ -905,7 +907,9 @@ static ssize_t vfio_ap_mdev_ioctl(struct mdev_device *mdev, ret = vfio_ap_mdev_get_device_info(arg); break; case VFIO_DEVICE_RESET: + mutex_lock(&matrix_dev->lock); ret = vfio_ap_mdev_reset_queues(mdev); + mutex_unlock(&matrix_dev->lock); break; default: ret = -EOPNOTSUPP; diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h index a910be1..3e6940c 100644 --- a/drivers/s390/crypto/vfio_ap_private.h +++ b/drivers/s390/crypto/vfio_ap_private.h @@ -40,6 +40,7 @@ struct ap_matrix_dev { atomic_t available_instances; struct ap_config_info info; struct list_head mdev_list; + struct list_head free_list; struct mutex lock; struct ap_driver *vfio_ap_drv; }; @@ -83,9 +84,15 @@ struct ap_matrix_mdev { struct notifier_block group_notifier; struct kvm *kvm; struct kvm_s390_module_hook pqap_hook; + struct list_head qlist; }; extern int vfio_ap_mdev_register(void); extern void vfio_ap_mdev_unregister(void); +struct vfio_ap_queue { + struct list_head list; + struct ap_matrix_mdev *matrix_mdev; + int apqn; +}; #endif /* _VFIO_AP_PRIVATE_H_ */ From patchwork Wed Mar 13 16:05:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 10851459 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A1823139A for ; Wed, 13 Mar 2019 16:06:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DC2329E5D for ; Wed, 13 Mar 2019 16:06:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C2592A06E; Wed, 13 Mar 2019 16:06:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 62FCB29E5D for ; Wed, 13 Mar 2019 16:06:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726691AbfCMQGX (ORCPT ); Wed, 13 Mar 2019 12:06:23 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:33788 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726719AbfCMQFR (ORCPT ); Wed, 13 Mar 2019 12:05:17 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x2DG4I5I077089 for ; Wed, 13 Mar 2019 12:05:16 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0b-001b2d01.pphosted.com with ESMTP id 2r747w2qdq-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 13 Mar 2019 12:05:16 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 13 Mar 2019 16:05:14 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 13 Mar 2019 16:05:11 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x2DG59uZ48234616 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 13 Mar 2019 16:05:09 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 59625A405D; Wed, 13 Mar 2019 16:05:09 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E9A24A4055; Wed, 13 Mar 2019 16:05:08 +0000 (GMT) Received: from morel-ThinkPad-W530.boeblingen.de.ibm.com (unknown [9.152.224.140]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 13 Mar 2019 16:05:08 +0000 (GMT) From: Pierre Morel To: borntraeger@de.ibm.com Cc: alex.williamson@redhat.com, cohuck@redhat.com, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, frankja@linux.ibm.com, akrowiak@linux.ibm.com, pasic@linux.ibm.com, david@redhat.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, freude@linux.ibm.com, mimu@linux.ibm.com Subject: [PATCH v5 3/7] vfio: ap: register IOMMU VFIO notifier Date: Wed, 13 Mar 2019 17:05:00 +0100 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> References: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 19031316-0028-0000-0000-00000353939D X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19031316-0029-0000-0000-000024121C99 Message-Id: <1552493104-30510-4-git-send-email-pmorel@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-03-13_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903130114 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP To be able to use the VFIO interface to facilitate the mediated device memory pinning/unpinning we need to register a notifier for IOMMU. While we will start to pin one guest page for the interrupt indicator byte, this is still ok with ballooning as this page will never be used by the guest virtio-balloon driver. So the pinned page will never be freed. And even a broken guest does so, that would not impact the host as the original page is still in control by vfio. Signed-off-by: Pierre Morel Reviewed-by: Cornelia Huck --- drivers/s390/crypto/vfio_ap_ops.c | 53 ++++++++++++++++++++++++++++++++--- drivers/s390/crypto/vfio_ap_private.h | 2 ++ 2 files changed, 51 insertions(+), 4 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index 485595c..0f8952c23 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -757,6 +757,36 @@ static const struct attribute_group *vfio_ap_mdev_attr_groups[] = { }; /** + * vfio_ap_mdev_iommu_notifier: IOMMU notifier callback + * + * @nb: The notifier block + * @action: Action to be taken + * @data: data associated with the request + * + * For an UNMAP request, unpin the guest IOVA (the NIB guest address we + * pinned before). Other requests are ignored. + * + */ +static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct ap_matrix_mdev *matrix_mdev; + + matrix_mdev = container_of(nb, struct ap_matrix_mdev, iommu_notifier); + + if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) { + struct vfio_iommu_type1_dma_unmap *unmap = data; + unsigned long g_pfn = unmap->iova >> PAGE_SHIFT; + + vfio_unpin_pages(mdev_dev(matrix_mdev->mdev), &g_pfn, 1); + return NOTIFY_OK; + } + + return NOTIFY_DONE; +} + + +/** * vfio_ap_mdev_set_kvm * * @matrix_mdev: a mediated matrix device @@ -855,12 +885,25 @@ static int vfio_ap_mdev_open(struct mdev_device *mdev) ret = vfio_register_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, &events, &matrix_mdev->group_notifier); - if (ret) { - module_put(THIS_MODULE); - return ret; - } + if (ret) + goto err_group; + + matrix_mdev->iommu_notifier.notifier_call = vfio_ap_mdev_iommu_notifier; + events = VFIO_IOMMU_NOTIFY_DMA_UNMAP; + + ret = vfio_register_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, + &events, &matrix_mdev->iommu_notifier); + if (ret) + goto err_iommu; return 0; + +err_iommu: + vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, + &matrix_mdev->group_notifier); +err_group: + module_put(THIS_MODULE); + return ret; } static void vfio_ap_mdev_release(struct mdev_device *mdev) @@ -872,6 +915,8 @@ static void vfio_ap_mdev_release(struct mdev_device *mdev) matrix_mdev->kvm = NULL; vfio_ap_mdev_reset_queues(mdev); + vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, + &matrix_mdev->iommu_notifier); vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, &matrix_mdev->group_notifier); module_put(THIS_MODULE); diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h index 3e6940c..4a287c8 100644 --- a/drivers/s390/crypto/vfio_ap_private.h +++ b/drivers/s390/crypto/vfio_ap_private.h @@ -82,9 +82,11 @@ struct ap_matrix_mdev { struct list_head node; struct ap_matrix matrix; struct notifier_block group_notifier; + struct notifier_block iommu_notifier; struct kvm *kvm; struct kvm_s390_module_hook pqap_hook; struct list_head qlist; + struct mdev_device *mdev; }; extern int vfio_ap_mdev_register(void); From patchwork Wed Mar 13 16:05:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 10851449 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BBFEB1515 for ; Wed, 13 Mar 2019 16:05:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A735B2A088 for ; Wed, 13 Mar 2019 16:05:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A4D192A081; Wed, 13 Mar 2019 16:05:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17A5E2A0AC for ; Wed, 13 Mar 2019 16:05:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726767AbfCMQFV (ORCPT ); Wed, 13 Mar 2019 12:05:21 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:59684 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726699AbfCMQFT (ORCPT ); Wed, 13 Mar 2019 12:05:19 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x2DG4I0I008122 for ; Wed, 13 Mar 2019 12:05:18 -0400 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0b-001b2d01.pphosted.com with ESMTP id 2r73htn9pw-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 13 Mar 2019 12:05:17 -0400 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 13 Mar 2019 16:05:15 -0000 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 13 Mar 2019 16:05:12 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x2DG5AbS61014154 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 13 Mar 2019 16:05:11 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C9E45A405E; Wed, 13 Mar 2019 16:05:10 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 66B67A4055; Wed, 13 Mar 2019 16:05:10 +0000 (GMT) Received: from morel-ThinkPad-W530.boeblingen.de.ibm.com (unknown [9.152.224.140]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 13 Mar 2019 16:05:10 +0000 (GMT) From: Pierre Morel To: borntraeger@de.ibm.com Cc: alex.williamson@redhat.com, cohuck@redhat.com, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, frankja@linux.ibm.com, akrowiak@linux.ibm.com, pasic@linux.ibm.com, david@redhat.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, freude@linux.ibm.com, mimu@linux.ibm.com Subject: [PATCH v5 4/7] s390: ap: setup relation betwen KVM and mediated device Date: Wed, 13 Mar 2019 17:05:01 +0100 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> References: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 19031316-0008-0000-0000-000002CC2DBC X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19031316-0009-0000-0000-00002238531B Message-Id: <1552493104-30510-5-git-send-email-pmorel@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-03-13_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903130114 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When the mediated device is open we setup the relation with KVM unset it when the mediated device is released. We ensure KVM is present on opening of the mediated device. We ensure that KVM survives the mediated device, and establish a direct link from KVM to the mediated device to simplify the relationship. Signed-off-by: Pierre Morel --- drivers/s390/crypto/vfio_ap_ops.c | 80 ++++++++++++++++++++++----------------- 1 file changed, 46 insertions(+), 34 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index 0f8952c23..6b559ca 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -790,7 +790,6 @@ static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb, * vfio_ap_mdev_set_kvm * * @matrix_mdev: a mediated matrix device - * @kvm: reference to KVM instance * * Verifies no other mediated matrix device has @kvm and sets a reference to * it in @matrix_mdev->kvm. @@ -798,53 +797,39 @@ static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb, * Return 0 if no other mediated matrix device has a reference to @kvm; * otherwise, returns an -EPERM. */ -static int vfio_ap_mdev_set_kvm(struct ap_matrix_mdev *matrix_mdev, - struct kvm *kvm) +static int vfio_ap_mdev_set_kvm(struct ap_matrix_mdev *matrix_mdev) { - struct ap_matrix_mdev *m; - mutex_lock(&matrix_dev->lock); + if (matrix_mdev->kvm->arch.crypto.pqap_hook) + goto err_unlock; - list_for_each_entry(m, &matrix_dev->mdev_list, node) { - if ((m != matrix_mdev) && (m->kvm == kvm)) { - mutex_unlock(&matrix_dev->lock); - return -EPERM; - } - } + if (!matrix_mdev->kvm->arch.crypto.crycbd) + goto err_unlock; - matrix_mdev->kvm = kvm; - mutex_unlock(&matrix_dev->lock); + matrix_mdev->kvm->arch.crypto.pqap_hook = &matrix_mdev->pqap_hook; + kvm_arch_crypto_set_masks(matrix_mdev->kvm, matrix_mdev->matrix.apm, + matrix_mdev->matrix.aqm, + matrix_mdev->matrix.adm); + kvm_get_kvm(matrix_mdev->kvm); + mutex_unlock(&matrix_dev->lock); return 0; + +err_unlock: + mutex_unlock(&matrix_dev->lock); + return -EPERM; } static int vfio_ap_mdev_group_notifier(struct notifier_block *nb, unsigned long action, void *data) { - int ret; struct ap_matrix_mdev *matrix_mdev; if (action != VFIO_GROUP_NOTIFY_SET_KVM) return NOTIFY_OK; matrix_mdev = container_of(nb, struct ap_matrix_mdev, group_notifier); - - if (!data) { - matrix_mdev->kvm = NULL; - return NOTIFY_OK; - } - - ret = vfio_ap_mdev_set_kvm(matrix_mdev, data); - if (ret) - return NOTIFY_DONE; - - /* If there is no CRYCB pointer, then we can't copy the masks */ - if (!matrix_mdev->kvm->arch.crypto.crycbd) - return NOTIFY_DONE; - - kvm_arch_crypto_set_masks(matrix_mdev->kvm, matrix_mdev->matrix.apm, - matrix_mdev->matrix.aqm, - matrix_mdev->matrix.adm); + matrix_mdev->kvm = data; return NOTIFY_OK; } @@ -888,6 +873,12 @@ static int vfio_ap_mdev_open(struct mdev_device *mdev) if (ret) goto err_group; + /* We do not support opening the mediated device without KVM */ + if (!matrix_mdev->kvm) { + ret = -ENODEV; + goto err_group; + } + matrix_mdev->iommu_notifier.notifier_call = vfio_ap_mdev_iommu_notifier; events = VFIO_IOMMU_NOTIFY_DMA_UNMAP; @@ -896,8 +887,15 @@ static int vfio_ap_mdev_open(struct mdev_device *mdev) if (ret) goto err_iommu; + ret = vfio_ap_mdev_set_kvm(matrix_mdev); + if (ret) + goto err_kvm; + return 0; +err_kvm: + vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, + &matrix_mdev->iommu_notifier); err_iommu: vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, &matrix_mdev->group_notifier); @@ -906,19 +904,33 @@ static int vfio_ap_mdev_open(struct mdev_device *mdev) return ret; } -static void vfio_ap_mdev_release(struct mdev_device *mdev) +static int vfio_ap_mdev_unset_kvm(struct ap_matrix_mdev *matrix_mdev) { - struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); + struct kvm *kvm = matrix_mdev->kvm; if (matrix_mdev->kvm) kvm_arch_crypto_clear_masks(matrix_mdev->kvm); - + vfio_ap_mdev_reset_queues(matrix_mdev->mdev); + matrix_mdev->kvm->arch.crypto.pqap_hook = NULL; matrix_mdev->kvm = NULL; + + kvm_put_kvm(kvm); + return 0; +} + +static void vfio_ap_mdev_release(struct mdev_device *mdev) +{ + struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); + + mutex_lock(&matrix_dev->lock); + vfio_ap_mdev_reset_queues(mdev); + vfio_ap_mdev_unset_kvm(matrix_mdev); vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, &matrix_mdev->iommu_notifier); vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, &matrix_mdev->group_notifier); + mutex_unlock(&matrix_dev->lock); module_put(THIS_MODULE); } From patchwork Wed Mar 13 16:05:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 10851451 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61380139A for ; Wed, 13 Mar 2019 16:05:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4C5DE28D74 for ; Wed, 13 Mar 2019 16:05:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4A3FA2A0B4; Wed, 13 Mar 2019 16:05:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D29E2A07B for ; Wed, 13 Mar 2019 16:05:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726805AbfCMQFz (ORCPT ); Wed, 13 Mar 2019 12:05:55 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:60318 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726314AbfCMQFU (ORCPT ); Wed, 13 Mar 2019 12:05:20 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x2DG4Mak144764 for ; Wed, 13 Mar 2019 12:05:19 -0400 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 2r73jtd715-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 13 Mar 2019 12:05:18 -0400 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 13 Mar 2019 16:05:16 -0000 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 13 Mar 2019 16:05:13 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x2DG5CEa23789670 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 13 Mar 2019 16:05:12 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4E9E5A4040; Wed, 13 Mar 2019 16:05:12 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D7456A404D; Wed, 13 Mar 2019 16:05:11 +0000 (GMT) Received: from morel-ThinkPad-W530.boeblingen.de.ibm.com (unknown [9.152.224.140]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 13 Mar 2019 16:05:11 +0000 (GMT) From: Pierre Morel To: borntraeger@de.ibm.com Cc: alex.williamson@redhat.com, cohuck@redhat.com, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, frankja@linux.ibm.com, akrowiak@linux.ibm.com, pasic@linux.ibm.com, david@redhat.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, freude@linux.ibm.com, mimu@linux.ibm.com Subject: [PATCH v5 5/7] s390: ap: implement PAPQ AQIC interception in kernel Date: Wed, 13 Mar 2019 17:05:02 +0100 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> References: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 19031316-0008-0000-0000-000002CC2DBE X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19031316-0009-0000-0000-00002238531D Message-Id: <1552493104-30510-6-git-send-email-pmorel@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-03-13_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=4 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903130114 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We register the AP PQAP instruction hook during the open of the mediated device. And unregister it on release. In the AP PQAP instruction hook, if we receive a demand to enable IRQs, - we retrieve the vfio_ap_queue based on the APQN we receive in REG1, - we retrieve the page of the guest address, (NIB), from register REG2 - we the mediated device to use the VFIO pinning infratrsucture to pin the page of the guest address, - we retrieve the pointer to KVM to register the guest ISC and retrieve the host ISC - finaly we activate GISA If we receive a demand to disable IRQs, - we deactivate GISA - unregister from the GIB - unping the NIB Signed-off-by: Pierre Morel --- drivers/s390/crypto/ap_bus.h | 1 + drivers/s390/crypto/vfio_ap_drv.c | 2 + drivers/s390/crypto/vfio_ap_ops.c | 204 +++++++++++++++++++++++++++++++++- drivers/s390/crypto/vfio_ap_private.h | 6 + 4 files changed, 210 insertions(+), 3 deletions(-) diff --git a/drivers/s390/crypto/ap_bus.h b/drivers/s390/crypto/ap_bus.h index d0059ea..9a4fd96 100644 --- a/drivers/s390/crypto/ap_bus.h +++ b/drivers/s390/crypto/ap_bus.h @@ -43,6 +43,7 @@ static inline int ap_test_bit(unsigned int *ptr, unsigned int nr) #define AP_RESPONSE_BUSY 0x05 #define AP_RESPONSE_INVALID_ADDRESS 0x06 #define AP_RESPONSE_OTHERWISE_CHANGED 0x07 +#define AP_RESPONSE_INVALID_GISA 0x08 #define AP_RESPONSE_Q_FULL 0x10 #define AP_RESPONSE_NO_PENDING_REPLY 0x10 #define AP_RESPONSE_INDEX_TOO_BIG 0x11 diff --git a/drivers/s390/crypto/vfio_ap_drv.c b/drivers/s390/crypto/vfio_ap_drv.c index df6f21a..796e73d4 100644 --- a/drivers/s390/crypto/vfio_ap_drv.c +++ b/drivers/s390/crypto/vfio_ap_drv.c @@ -55,6 +55,8 @@ static int vfio_ap_queue_dev_probe(struct ap_device *apdev) return -ENOMEM; dev_set_drvdata(&apdev->device, q); q->apqn = to_ap_queue(&apdev->device)->qid; + q->a_isc = VFIO_AP_ISC_INVALID; + q->p_isc = VFIO_AP_ISC_INVALID; INIT_LIST_HEAD(&q->list); mutex_lock(&matrix_dev->lock); list_add(&q->list, &matrix_dev->free_list); diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index 6b559ca..dc3454f 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -66,6 +66,194 @@ static int vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q) return -EBUSY; } +/** + * vfio_ap_free_irq: + * @q: The vfio_ap_queue + * + * Unpin the guest NIB + * Unregister the ISC from the GIB alert + * Clear the vfio_ap_queue intern fields + */ +static void vfio_ap_free_irq(struct vfio_ap_queue *q) +{ + if (!q) + return; + if (q->a_pfn) + vfio_unpin_pages(mdev_dev(q->matrix_mdev->mdev), &q->a_pfn, 1); + if (q->a_isc != VFIO_AP_ISC_INVALID) + kvm_s390_gisc_unregister(q->matrix_mdev->kvm, q->a_isc); + q->a_pfn = 0; + q->p_pfn = 0; + q->a_isc = VFIO_AP_ISC_INVALID; + q->p_isc = VFIO_AP_ISC_INVALID; +} + +/** + * vfio_ap_clrirq: Disable Interruption for a APQN + * + * @dev: the device associated with the ap_queue + * @q: the vfio_ap_queue holding AQIC parameters + * + * Issue the host side PQAP/AQIC + * On success: unpin the NIB saved in *q and unregister from GIB + * interface + * + * Return the ap_queue_status returned by the ap_aqic() + */ +static struct ap_queue_status vfio_ap_clrirq(struct vfio_ap_queue *q) +{ + struct ap_qirq_ctrl aqic_gisa = {}; + struct ap_queue_status status; + + status = ap_aqic(q->apqn, aqic_gisa, NULL); + if (!status.response_code) + vfio_ap_free_irq(q); + + return status; +} + +/** + * vfio_ap_setirq: Enable Interruption for a APQN + * + * @dev: the device associated with the ap_queue + * @q: the vfio_ap_queue holding AQIC parameters + * + * Pin the NIB saved in *q + * Register the guest ISC to GIB interface and retrieve the + * host ISC to issue the host side PQAP/AQIC + * + * Response.status may be set to following Response Code in case of error: + * - AP_RESPONSE_INVALID_ADDRESS: vfio_pin_pages failed + * - AP_RESPONSE_OTHERWISE_CHANGED: Hypervizor GISA internal error + * + * Otherwise return the ap_queue_status returned by the ap_aqic() + */ +static struct ap_queue_status vfio_ap_setirq(struct vfio_ap_queue *q) +{ + struct ap_qirq_ctrl aqic_gisa = {}; + struct ap_queue_status status = {}; + struct kvm_s390_gisa *gisa; + struct kvm *kvm; + unsigned long h_nib, h_pfn; + int ret; + + kvm = q->matrix_mdev->kvm; + gisa = kvm->arch.gisa_int.origin; + + q->a_pfn = q->a_nib >> PAGE_SHIFT; + ret = vfio_pin_pages(mdev_dev(q->matrix_mdev->mdev), &q->a_pfn, 1, + IOMMU_READ | IOMMU_WRITE, &h_pfn); + switch (ret) { + case 1: + break; + case -EINVAL: + case -E2BIG: + status.response_code = AP_RESPONSE_INVALID_ADDRESS; + /* Fallthrough */ + default: + return status; + } + + h_nib = (h_pfn << PAGE_SHIFT) | (q->a_nib & ~PAGE_MASK); + aqic_gisa.gisc = q->a_isc; + aqic_gisa.isc = kvm_s390_gisc_register(kvm, q->a_isc); + aqic_gisa.ir = 1; + aqic_gisa.gisa = gisa->next_alert >> 4; + + status = ap_aqic(q->apqn, aqic_gisa, (void *)h_nib); + switch (status.response_code) { + case AP_RESPONSE_NORMAL: + /* See if we did clear older IRQ configuration */ + if (q->p_pfn) + vfio_unpin_pages(mdev_dev(q->matrix_mdev->mdev), + &q->p_pfn, 1); + if (q->p_isc != VFIO_AP_ISC_INVALID) + kvm_s390_gisc_unregister(kvm, q->p_isc); + q->p_pfn = q->a_pfn; + q->p_isc = q->a_isc; + break; + case AP_RESPONSE_OTHERWISE_CHANGED: + /* We could not modify IRQ setings: clear new configuration */ + vfio_unpin_pages(mdev_dev(q->matrix_mdev->mdev), &q->a_pfn, 1); + kvm_s390_gisc_unregister(kvm, q->a_isc); + break; + case AP_RESPONSE_INVALID_GISA: + status.response_code = AP_RESPONSE_INVALID_ADDRESS; + default: /* Fall Through */ + pr_warn("%s: apqn %04x: response: %02x\n", __func__, q->apqn, + status.response_code); + vfio_ap_free_irq(q); + break; + } + + return status; +} + +/** + * handle_pqap: PQAP instruction callback + * + * @vcpu: The vcpu on which we received the PQAP instruction + * + * Get the general register contents to initialize internal variables. + * REG[0]: APQN + * REG[1]: IR and ISC + * REG[2]: NIB + * + * Response.status may be set to following Response Code: + * - AP_RESPONSE_Q_NOT_AVAIL: if the queue is not available + * - AP_RESPONSE_DECONFIGURED: if the queue is not configured + * - AP_RESPONSE_NORMAL (0) : in case of successs + * Check vfio_ap_setirq() and vfio_ap_clrirq() for other possible RC. + * We take the matrix_dev lock to ensure serialization on queues and + * mediated device access. + * + * Return 0 if we could handle the request inside KVM. + * otherwise, returns -EOPNOTSUPP to let QEMU handle the fault. + */ +static int handle_pqap(struct kvm_vcpu *vcpu) +{ + uint64_t status; + uint16_t apqn; + struct vfio_ap_queue *q; + struct ap_queue_status qstatus = { + .response_code = AP_RESPONSE_Q_NOT_AVAIL, }; + struct ap_matrix_mdev *matrix_mdev; + + /* If we do not use the AIV facility just go to userland */ + if (!(vcpu->arch.sie_block->eca & ECA_AIV)) + return -EOPNOTSUPP; + + apqn = vcpu->run->s.regs.gprs[0] & 0xffff; + mutex_lock(&matrix_dev->lock); + matrix_mdev = container_of(vcpu->kvm->arch.crypto.pqap_hook, + struct ap_matrix_mdev, pqap_hook); + if (!matrix_mdev) + goto out_unlock; + q = vfio_ap_get_queue(apqn, &matrix_mdev->qlist); + if (!q) + goto out_noqueue; + + status = vcpu->run->s.regs.gprs[1]; + + /* If IR bit(16) is set we enable the interrupt */ + if ((status >> (63 - 16)) & 0x01) { + q->a_isc = status & 0x07; + q->a_nib = vcpu->run->s.regs.gprs[2]; + qstatus = vfio_ap_setirq(q); + if (qstatus.response_code) { + q->a_nib = 0; + q->a_isc = VFIO_AP_ISC_INVALID; + } + } else + qstatus = vfio_ap_clrirq(q); + +out_noqueue: + memcpy(&vcpu->run->s.regs.gprs[1], &qstatus, sizeof(qstatus)); +out_unlock: + mutex_unlock(&matrix_dev->lock); + return 0; +} + static void vfio_ap_matrix_init(struct ap_config_info *info, struct ap_matrix *matrix) { @@ -88,8 +276,11 @@ static int vfio_ap_mdev_create(struct kobject *kobj, struct mdev_device *mdev) } INIT_LIST_HEAD(&matrix_mdev->qlist); + matrix_mdev->mdev = mdev; vfio_ap_matrix_init(&matrix_dev->info, &matrix_mdev->matrix); mdev_set_drvdata(mdev, matrix_mdev); + matrix_mdev->pqap_hook.hook = handle_pqap; + matrix_mdev->pqap_hook.owner = THIS_MODULE; mutex_lock(&matrix_dev->lock); list_add(&matrix_mdev->node, &matrix_dev->mdev_list); mutex_unlock(&matrix_dev->lock); @@ -100,11 +291,17 @@ static int vfio_ap_mdev_create(struct kobject *kobj, struct mdev_device *mdev) static int vfio_ap_mdev_remove(struct mdev_device *mdev) { struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); + struct vfio_ap_queue *q, *qtmp; if (matrix_mdev->kvm) return -EBUSY; mutex_lock(&matrix_dev->lock); + list_for_each_entry_safe(q, qtmp, &matrix_mdev->qlist, list) { + q->matrix_mdev = NULL; + vfio_ap_mdev_reset_queue(q); + list_move(&q->list, &matrix_dev->free_list); + } list_del(&matrix_mdev->node); mutex_unlock(&matrix_dev->lock); @@ -756,7 +953,7 @@ static const struct attribute_group *vfio_ap_mdev_attr_groups[] = { NULL }; -/** + /* * vfio_ap_mdev_iommu_notifier: IOMMU notifier callback * * @nb: The notifier block @@ -776,9 +973,10 @@ static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb, if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) { struct vfio_iommu_type1_dma_unmap *unmap = data; - unsigned long g_pfn = unmap->iova >> PAGE_SHIFT; + unsigned long pfn = unmap->iova >> PAGE_SHIFT; - vfio_unpin_pages(mdev_dev(matrix_mdev->mdev), &g_pfn, 1); + if (matrix_mdev->mdev) + vfio_unpin_pages(mdev_dev(matrix_mdev->mdev), &pfn, 1); return NOTIFY_OK; } diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h index 4a287c8..968d8aa 100644 --- a/drivers/s390/crypto/vfio_ap_private.h +++ b/drivers/s390/crypto/vfio_ap_private.h @@ -95,6 +95,12 @@ extern void vfio_ap_mdev_unregister(void); struct vfio_ap_queue { struct list_head list; struct ap_matrix_mdev *matrix_mdev; + unsigned long a_nib; + unsigned long a_pfn; + unsigned long p_pfn; int apqn; +#define VFIO_AP_ISC_INVALID 0xff + unsigned char a_isc; + unsigned char p_isc; }; #endif /* _VFIO_AP_PRIVATE_H_ */ From patchwork Wed Mar 13 16:05:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 10851457 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9D958139A for ; Wed, 13 Mar 2019 16:06:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8A17329C50 for ; Wed, 13 Mar 2019 16:06:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8858F29E5D; Wed, 13 Mar 2019 16:06:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F15692A05D for ; Wed, 13 Mar 2019 16:06:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726316AbfCMQGU (ORCPT ); Wed, 13 Mar 2019 12:06:20 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:33032 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725893AbfCMQGI (ORCPT ); Wed, 13 Mar 2019 12:06:08 -0400 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x2DG60eN077142 for ; Wed, 13 Mar 2019 12:06:07 -0400 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0b-001b2d01.pphosted.com with ESMTP id 2r748r2jth-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 13 Mar 2019 12:06:05 -0400 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 13 Mar 2019 16:05:18 -0000 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 13 Mar 2019 16:05:15 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x2DG5Ddd19857654 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 13 Mar 2019 16:05:13 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BF826A4053; Wed, 13 Mar 2019 16:05:13 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5B84EA4051; Wed, 13 Mar 2019 16:05:13 +0000 (GMT) Received: from morel-ThinkPad-W530.boeblingen.de.ibm.com (unknown [9.152.224.140]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 13 Mar 2019 16:05:13 +0000 (GMT) From: Pierre Morel To: borntraeger@de.ibm.com Cc: alex.williamson@redhat.com, cohuck@redhat.com, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, frankja@linux.ibm.com, akrowiak@linux.ibm.com, pasic@linux.ibm.com, david@redhat.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, freude@linux.ibm.com, mimu@linux.ibm.com Subject: [PATCH v5 6/7] s390: ap: Cleanup on removing the AP device Date: Wed, 13 Mar 2019 17:05:03 +0100 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> References: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 19031316-0012-0000-0000-000003022B96 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19031316-0013-0000-0000-000021395279 Message-Id: <1552493104-30510-7-git-send-email-pmorel@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-03-13_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903130114 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When a AP device is remove, clear the queue's APID bit in the guest CRYCB. to be sure that the guest will not access the AP queue anymore. Then we clear the interruptions and reset the AP device properly. Signed-off-by: Pierre Morel --- drivers/s390/crypto/vfio_ap_drv.c | 36 +++++++++++++++++++++++++++++++++++ drivers/s390/crypto/vfio_ap_ops.c | 16 +++++++++++++--- drivers/s390/crypto/vfio_ap_private.h | 3 +++ 3 files changed, 52 insertions(+), 3 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_drv.c b/drivers/s390/crypto/vfio_ap_drv.c index 796e73d4..850ba6e 100644 --- a/drivers/s390/crypto/vfio_ap_drv.c +++ b/drivers/s390/crypto/vfio_ap_drv.c @@ -5,6 +5,7 @@ * Copyright IBM Corp. 2018 * * Author(s): Tony Krowiak + * Pierre Morel */ #include @@ -12,6 +13,8 @@ #include #include #include +#include +#include #include "vfio_ap_private.h" #define VFIO_AP_ROOT_NAME "vfio_ap" @@ -65,6 +68,33 @@ static int vfio_ap_queue_dev_probe(struct ap_device *apdev) } /** + * vfio_ap_update_crycb + * @q: A pointer to the queue being removed + * + * We clear the APID of the queue, making this queue unusable for the guest. + * After this function we can reset the queue without to fear a race with + * the guest to access the queue again. + * We do not fear race with the host as we still get the device. + */ +static void vfio_ap_update_crycb(struct vfio_ap_queue *q) +{ + struct ap_matrix_mdev *matrix_mdev = q->matrix_mdev; + + if (!matrix_mdev) + return; + + clear_bit_inv(AP_QID_CARD(q->apqn), matrix_mdev->matrix.apm); + + if (!matrix_mdev->kvm) + return; + + kvm_arch_crypto_set_masks(matrix_mdev->kvm, + matrix_mdev->matrix.apm, + matrix_mdev->matrix.aqm, + matrix_mdev->matrix.adm); +} + +/** * vfio_ap_queue_dev_remove: * * Free the associated vfio_ap_queue structure @@ -74,7 +104,13 @@ static void vfio_ap_queue_dev_remove(struct ap_device *apdev) struct vfio_ap_queue *q; q = dev_get_drvdata(&apdev->device); + if (!q) + return; + mutex_lock(&matrix_dev->lock); + vfio_ap_update_crycb(q); + vfio_ap_mdev_reset_queue(q); + vfio_ap_free_irq(q); list_del(&q->list); mutex_unlock(&matrix_dev->lock); kfree(q); diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index dc3454f..dc3caa7 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -43,15 +43,22 @@ struct vfio_ap_queue *vfio_ap_get_queue(int apqn, struct list_head *l) return NULL; } -static int vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q) +int vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q) { struct ap_queue_status status; - int retry = 1; + int retry = 20; do { status = ap_zapq(q->apqn); switch (status.response_code) { case AP_RESPONSE_NORMAL: + while (!status.queue_empty && retry--) { + msleep(20); + status = ap_tapq(q->apqn, NULL); + } + if (retry <= 0) + pr_warn("%s: queue 0x%04x not empty\n", + __func__, q->apqn); return 0; case AP_RESPONSE_RESET_IN_PROGRESS: case AP_RESPONSE_BUSY: @@ -74,7 +81,7 @@ static int vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q) * Unregister the ISC from the GIB alert * Clear the vfio_ap_queue intern fields */ -static void vfio_ap_free_irq(struct vfio_ap_queue *q) +void vfio_ap_free_irq(struct vfio_ap_queue *q) { if (!q) return; @@ -300,6 +307,7 @@ static int vfio_ap_mdev_remove(struct mdev_device *mdev) list_for_each_entry_safe(q, qtmp, &matrix_mdev->qlist, list) { q->matrix_mdev = NULL; vfio_ap_mdev_reset_queue(q); + vfio_ap_free_irq(q); list_move(&q->list, &matrix_dev->free_list); } list_del(&matrix_mdev->node); @@ -362,6 +370,7 @@ static void vfio_ap_free_queue(int apqn, struct ap_matrix_mdev *matrix_mdev) return; q->matrix_mdev = NULL; vfio_ap_mdev_reset_queue(q); + vfio_ap_free_irq(q); list_move(&q->list, &matrix_dev->free_list); } @@ -1041,6 +1050,7 @@ static int vfio_ap_mdev_reset_queues(struct mdev_device *mdev) list_for_each_entry(q, &matrix_mdev->qlist, list) { ret = vfio_ap_mdev_reset_queue(q); + vfio_ap_free_irq(q); /* * Regardless whether a queue turns out to be busy, or * is not operational, we need to continue resetting diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h index 968d8aa..9fe580b 100644 --- a/drivers/s390/crypto/vfio_ap_private.h +++ b/drivers/s390/crypto/vfio_ap_private.h @@ -4,6 +4,7 @@ * * Author(s): Tony Krowiak * Halil Pasic + * Pierre Morel * * Copyright IBM Corp. 2018 */ @@ -103,4 +104,6 @@ struct vfio_ap_queue { unsigned char a_isc; unsigned char p_isc; }; +void vfio_ap_free_irq(struct vfio_ap_queue *q); +int vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q); #endif /* _VFIO_AP_PRIVATE_H_ */ From patchwork Wed Mar 13 16:05:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 10851447 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 363FE1515 for ; Wed, 13 Mar 2019 16:05:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 237602A08A for ; Wed, 13 Mar 2019 16:05:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 20EBA2A0A6; Wed, 13 Mar 2019 16:05:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 74F3C2A0D5 for ; Wed, 13 Mar 2019 16:05:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726868AbfCMQFZ (ORCPT ); Wed, 13 Mar 2019 12:05:25 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:33884 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726720AbfCMQFY (ORCPT ); Wed, 13 Mar 2019 12:05:24 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x2DG4Xbn105959 for ; Wed, 13 Mar 2019 12:05:24 -0400 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 2r73aux3r6-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 13 Mar 2019 12:05:22 -0400 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 13 Mar 2019 16:05:20 -0000 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 13 Mar 2019 16:05:16 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x2DG5F6j47054998 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 13 Mar 2019 16:05:15 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 44644A4051; Wed, 13 Mar 2019 16:05:15 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CC9F4A4040; Wed, 13 Mar 2019 16:05:14 +0000 (GMT) Received: from morel-ThinkPad-W530.boeblingen.de.ibm.com (unknown [9.152.224.140]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 13 Mar 2019 16:05:14 +0000 (GMT) From: Pierre Morel To: borntraeger@de.ibm.com Cc: alex.williamson@redhat.com, cohuck@redhat.com, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, frankja@linux.ibm.com, akrowiak@linux.ibm.com, pasic@linux.ibm.com, david@redhat.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, freude@linux.ibm.com, mimu@linux.ibm.com Subject: [PATCH v5 7/7] s390: ap: kvm: Enable PQAP/AQIC facility for the guest Date: Wed, 13 Mar 2019 17:05:04 +0100 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> References: <1552493104-30510-1-git-send-email-pmorel@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 19031316-0012-0000-0000-000003022B99 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19031316-0013-0000-0000-00002139527C Message-Id: <1552493104-30510-8-git-send-email-pmorel@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-03-13_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=842 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903130114 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP AP Queue Interruption Control (AQIC) facility gives the guest the possibility to control interruption for the Cryptographic Adjunct Processor queues. Signed-off-by: Pierre Morel Reviewed-by: Tony Krowiak --- arch/s390/tools/gen_facilities.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/s390/tools/gen_facilities.c b/arch/s390/tools/gen_facilities.c index 04c5f07..be4b826 100644 --- a/arch/s390/tools/gen_facilities.c +++ b/arch/s390/tools/gen_facilities.c @@ -111,6 +111,7 @@ static struct facility_def facility_defs[] = { .bits = (int[]){ 12, /* AP Query Configuration Information */ 15, /* AP Facilities Test */ + 65, /* AP Queue Interruption Control */ 156, /* etoken facility */ -1 /* END */ }