From patchwork Wed Sep 12 19:42:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Krowiak X-Patchwork-Id: 10598209 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0B731921 for ; Wed, 12 Sep 2018 19:44:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ECD722AA7C for ; Wed, 12 Sep 2018 19:44:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E0F412AA99; Wed, 12 Sep 2018 19:44:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 379CE2AA7C for ; Wed, 12 Sep 2018 19:44:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728361AbeIMAuE (ORCPT ); Wed, 12 Sep 2018 20:50:04 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:50432 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728254AbeIMAuD (ORCPT ); Wed, 12 Sep 2018 20:50:03 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w8CJYPlA104834 for ; Wed, 12 Sep 2018 15:44:01 -0400 Received: from e36.co.us.ibm.com (e36.co.us.ibm.com [32.97.110.154]) by mx0a-001b2d01.pphosted.com with ESMTP id 2mf7btmmr6-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 12 Sep 2018 15:44:00 -0400 Received: from localhost by e36.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 12 Sep 2018 13:44:00 -0600 Received: from b03cxnp07028.gho.boulder.ibm.com (9.17.130.15) by e36.co.us.ibm.com (192.168.1.136) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 12 Sep 2018 13:43:55 -0600 Received: from b03ledav003.gho.boulder.ibm.com (b03ledav003.gho.boulder.ibm.com [9.17.130.234]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w8CJhqSX30474366 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 12 Sep 2018 12:43:53 -0700 Received: from b03ledav003.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CBD866A057; Wed, 12 Sep 2018 13:43:52 -0600 (MDT) Received: from b03ledav003.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0377E6A051; Wed, 12 Sep 2018 13:43:49 -0600 (MDT) Received: from localhost.localdomain (unknown [9.80.213.181]) by b03ledav003.gho.boulder.ibm.com (Postfix) with ESMTPS; Wed, 12 Sep 2018 13:43:49 -0600 (MDT) From: Tony Krowiak To: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: freude@de.ibm.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, borntraeger@de.ibm.com, cohuck@redhat.com, kwankhede@nvidia.com, bjsdjshi@linux.vnet.ibm.com, pbonzini@redhat.com, alex.williamson@redhat.com, pmorel@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, mjrosato@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, thuth@redhat.com, pasic@linux.vnet.ibm.com, berrange@redhat.com, fiuczy@linux.vnet.ibm.com, buendgen@de.ibm.com, akrowiak@linux.vnet.ibm.com, frankja@linux.ibm.com, Tony Krowiak Subject: [PATCH v10 07/26] s390: vfio-ap: sysfs interfaces to configure domains Date: Wed, 12 Sep 2018 15:42:57 -0400 X-Mailer: git-send-email 1.7.1 In-Reply-To: <1536781396-13601-1-git-send-email-akrowiak@linux.vnet.ibm.com> References: <1536781396-13601-1-git-send-email-akrowiak@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18091219-0020-0000-0000-00000E640D5F X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00009709; HX=3.00000242; KW=3.00000007; PH=3.00000004; SC=3.00000266; SDB=6.01087371; UDB=6.00561488; IPR=6.00867383; MB=3.00023256; MTD=3.00000008; XFM=3.00000015; UTC=2018-09-12 19:43:58 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18091219-0021-0000-0000-000063013519 Message-Id: <1536781396-13601-8-git-send-email-akrowiak@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-09-12_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1809120195 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Tony Krowiak Introduces two new sysfs attributes for the VFIO mediated matrix device for assigning AP domains to and unassigning AP domains from a mediated matrix device. The IDs of the AP domains assigned to the mediated matrix device will be stored in an AP queue mask (AQM). The bits in the AQM, from most significant to least significant bit, correspond to AP queue index (APQI) 0 to 255 (note that an APQI is synonymous with with a domain ID). On some systems, the maximum allowable domain number may be less than 255 - depending upon the host's AP configuration - and assignment may be rejected if the input domain ID exceeds the limit. When a domain is assigned, the bit corresponding to the APQI will be set in the AQM. Likewise, when a domain is unassigned, the bit corresponding to the APQI will be cleared from the AQM. In order to successfully assign a domain, the APQNs derived from the domain ID being assigned and the adapter numbers of all adapters previously assigned: 1. Must be bound to the vfio_ap device driver. 2. Must not be assigned to any other mediated matrix device. If there are no adapters assigned to the mdev, then there must be an AP queue bound to the vfio_ap device driver with an APQN containing the domain ID (i.e., APQI), otherwise all adapters subsequently assigned will fail because there will be no AP queues bound with an APQN containing the APQI. Assigning or un-assigning an AP domain will also be rejected if a guest using the mediated matrix device is running. The relevant sysfs structures are: /sys/devices/vfio_ap/matrix/ ...... [mdev_supported_types] ......... [vfio_ap-passthrough] ............ [devices] ...............[$uuid] .................. assign_domain .................. unassign_domain To assign a domain to the $uuid mediated matrix device, write the domain's ID to the assign_domain file. To unassign a domain, write the domain's ID to the unassign_domain file. The ID is specified using conventional semantics: If it begins with 0x, the number will be parsed as a hexadecimal (case insensitive) number; if it begins with 0, it will be parsed as an octal number; otherwise, it will be parsed as a decimal number. For example, to assign domain 173 (0xad) to the mediated matrix device $uuid: echo 173 > assign_domain or echo 0255 > assign_domain or echo 0xad > assign_domain To unassign domain 173 (0xad): echo 173 > unassign_domain or echo 0255 > unassign_domain or echo 0xad > unassign_domain Signed-off-by: Tony Krowiak Reviewed-by: Halil Pasic Tested-by: Michael Mueller Tested-by: Farhan Ali Tested-by: Pierre Morel Signed-off-by: Christian Borntraeger Reviewed-by: Cornelia Huck --- drivers/s390/crypto/vfio_ap_ops.c | 138 ++++++++++++++++++++++++++++++++++++- 1 files changed, 137 insertions(+), 1 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index 1d4f890..a687624 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -385,10 +385,146 @@ static ssize_t unassign_adapter_store(struct device *dev, } DEVICE_ATTR_WO(unassign_adapter); +static int +vfio_ap_mdev_verify_queues_reserved_for_apqi(struct ap_matrix_mdev *matrix_mdev, + unsigned long apqi) +{ + int ret; + unsigned long apid; + unsigned long nbits = matrix_mdev->matrix.apm_max + 1; + + if (find_first_bit_inv(matrix_mdev->matrix.apm, nbits) >= nbits) + return vfio_ap_verify_queue_reserved(NULL, &apqi); + + for_each_set_bit_inv(apid, matrix_mdev->matrix.apm, nbits) { + ret = vfio_ap_verify_queue_reserved(&apid, &apqi); + if (ret) + return ret; + } + + return 0; +} + +/** + * assign_domain_store + * + * @dev: the matrix device + * @attr: the mediated matrix device's assign_domain attribute + * @buf: a buffer containing the AP queue index (APQI) of the domain to + * be assigned + * @count: the number of bytes in @buf + * + * Parses the APQI from @buf and sets the corresponding bit in the mediated + * matrix device's AQM. + * + * Returns the number of bytes processed if the APQI is valid; otherwise returns + * one of the following errors: + * + * 1. -EINVAL + * The APQI is not a valid number + * + * 2. -ENODEV + * The APQI exceeds the maximum value configured for the system + * + * 3. -EADDRNOTAVAIL + * An APQN derived from the cross product of the APQI being assigned + * and the APIDs previously assigned is not bound to the vfio_ap device + * driver; or, if no APIDs have yet been assigned, the APQI is not + * contained in an APQN bound to the vfio_ap device driver. + * + * 4. -EADDRINUSE + * An APQN derived from the cross product of the APQI being assigned + * and the APIDs previously assigned is being used by another mediated + * matrix device + */ +static ssize_t assign_domain_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + int ret; + unsigned long apqi; + struct mdev_device *mdev = mdev_from_dev(dev); + struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); + unsigned long max_apqi = matrix_mdev->matrix.aqm_max; + + ret = kstrtoul(buf, 0, &apqi); + if (ret) + return ret; + if (apqi > max_apqi) + return -ENODEV; + + mutex_lock(&matrix_dev->lock); + + ret = vfio_ap_mdev_verify_queues_reserved_for_apqi(matrix_mdev, apqi); + if (ret) + goto done; + + set_bit_inv(apqi, matrix_mdev->matrix.aqm); + + ret = vfio_ap_mdev_verify_no_sharing(matrix_mdev); + if (ret) + goto share_err; + + ret = count; + goto done; + +share_err: + clear_bit_inv(apqi, matrix_mdev->matrix.aqm); +done: + mutex_unlock(&matrix_dev->lock); + + return ret; +} +DEVICE_ATTR_WO(assign_domain); + + +/** + * unassign_domain_store + * + * @dev: the matrix device + * @attr: the mediated matrix device's unassign_domain attribute + * @buf: a buffer containing the AP queue index (APQI) of the domain to + * be unassigned + * @count: the number of bytes in @buf + * + * Parses the APQI from @buf and clears the corresponding bit in the + * mediated matrix device's AQM. + * + * Returns the number of bytes processed if the APQI is valid; otherwise, + * returns one of the following errors: + * -EINVAL if the APQI is not a number + * -ENODEV if the APQI exceeds the maximum value configured for the system + */ +static ssize_t unassign_domain_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + int ret; + unsigned long apqi; + struct mdev_device *mdev = mdev_from_dev(dev); + struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); + + ret = kstrtoul(buf, 0, &apqi); + if (ret) + return ret; + + if (apqi > matrix_mdev->matrix.aqm_max) + return -ENODEV; + + mutex_lock(&matrix_dev->lock); + clear_bit_inv((unsigned long)apqi, matrix_mdev->matrix.aqm); + mutex_unlock(&matrix_dev->lock); + + return count; +} +DEVICE_ATTR_WO(unassign_domain); + static struct attribute *vfio_ap_mdev_attrs[] = { &dev_attr_assign_adapter.attr, &dev_attr_unassign_adapter.attr, - NULL + &dev_attr_assign_domain.attr, + &dev_attr_unassign_domain.attr, + NULL, }; static struct attribute_group vfio_ap_mdev_attr_group = {