From patchwork Thu Oct 26 15:54:51 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Krowiak X-Patchwork-Id: 10028593 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A87A66022E for ; Thu, 26 Oct 2017 15:55:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 990EC28DE7 for ; Thu, 26 Oct 2017 15:55:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8D5D828E52; Thu, 26 Oct 2017 15:55:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C5D6B28DE7 for ; Thu, 26 Oct 2017 15:55:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932485AbdJZPzm (ORCPT ); Thu, 26 Oct 2017 11:55:42 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:33350 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932396AbdJZPzL (ORCPT ); Thu, 26 Oct 2017 11:55:11 -0400 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v9QFrB8k060040 for ; Thu, 26 Oct 2017 11:55:11 -0400 Received: from e13.ny.us.ibm.com (e13.ny.us.ibm.com [129.33.205.203]) by mx0b-001b2d01.pphosted.com with ESMTP id 2dugvx6w1m-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Thu, 26 Oct 2017 11:55:10 -0400 Received: from localhost by e13.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 26 Oct 2017 11:55:10 -0400 Received: from b01cxnp22033.gho.pok.ibm.com (9.57.198.23) by e13.ny.us.ibm.com (146.89.104.200) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 26 Oct 2017 11:55:06 -0400 Received: from b01ledav003.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v9QFt5uW41812056; Thu, 26 Oct 2017 15:55:05 GMT Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3BEFEB2058; Thu, 26 Oct 2017 11:52:19 -0400 (EDT) Received: from localhost.localdomain (unknown [9.60.75.228]) by b01ledav003.gho.pok.ibm.com (Postfix) with ESMTPS id CDE00B2065; Thu, 26 Oct 2017 11:52:18 -0400 (EDT) From: Tony Krowiak To: linux-s390@vger.kernel.org, qemu-s390x@nongnu.org, kvm@vger.kernel.org Cc: freude@de.ibm.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, borntraeger@de.ibm.com, cohuck@redhat.com, kwankhede@nvidia.com, bjsdjshi@linux.vnet.ibm.com, pbonzini@redhat.com, alex.williamson@redhat.com, pmorel@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, mjrosato@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, thuth@redhat.com, pasic@linux.vnet.ibm.com, Tony Krowiak Subject: [RFC 2/5] s390x/vfio: ap-matrix: Introduce VFIO AP Matrix device Date: Thu, 26 Oct 2017 11:54:51 -0400 X-Mailer: git-send-email 1.7.1 In-Reply-To: <1509033294-4945-1-git-send-email-akrowiak@linux.vnet.ibm.com> References: <1509033294-4945-1-git-send-email-akrowiak@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 17102615-0008-0000-0000-00000296B32F X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007955; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000239; SDB=6.00936790; UDB=6.00472099; IPR=6.00717052; BA=6.00005660; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017727; XFM=3.00000015; UTC=2017-10-26 15:55:09 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17102615-0009-0000-0000-0000372035EB Message-Id: <1509033294-4945-3-git-send-email-akrowiak@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-10-26_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1710260209 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Introduces a VFIO based AP matrix device. This device will establish a communication pathway to the VFIO AP Matrix kernel device driver via a mediated AP matrix device file descriptor. This communication pathway will be used to: 1. Signal the VFIO AP matrix device driver via the VFIO_AP_MATRIX_CONFIGURE ioctl to configure the AP matrix for the KVM guest. The device driver will set the bits in the APM, AQM and ADM fields of the CRYCB referenced by the KVM guest's SIE state description according to the AP matrix configuration specified for the mediated AP matrix device's sysfs attribute files. The AP matrix configuration will be returned to the guest from the ioctl call to notify the KVM guest about the AP resources to which the guest has access. 2. Pass intercepted AP instructions to the VFIO AP Matrix driver for execution on an AP adapter assigned to the LPAR in which the linux host is running. Signed-off-by: Tony Krowiak --- default-configs/s390x-softmmu.mak | 1 + hw/vfio/Makefile.objs | 1 + hw/vfio/ap-matrix.c | 179 +++++++++++++++++++++++++++++++++++++ include/hw/vfio/vfio-common.h | 1 + linux-headers/linux/vfio.h | 28 +++++- 5 files changed, 207 insertions(+), 3 deletions(-) create mode 100644 hw/vfio/ap-matrix.c diff --git a/default-configs/s390x-softmmu.mak b/default-configs/s390x-softmmu.mak index 444bf16..1fe53ea 100644 --- a/default-configs/s390x-softmmu.mak +++ b/default-configs/s390x-softmmu.mak @@ -8,3 +8,4 @@ CONFIG_S390_FLIC=y CONFIG_S390_FLIC_KVM=$(CONFIG_KVM) CONFIG_VFIO_CCW=$(CONFIG_LINUX) CONFIG_WDT_DIAG288=y +CONFIG_VFIO_AP_MATRIX=$(CONFIG_LINUX) diff --git a/hw/vfio/Makefile.objs b/hw/vfio/Makefile.objs index c3ab909..47e482f 100644 --- a/hw/vfio/Makefile.objs +++ b/hw/vfio/Makefile.objs @@ -6,4 +6,5 @@ obj-$(CONFIG_SOFTMMU) += platform.o obj-$(CONFIG_VFIO_XGMAC) += calxeda-xgmac.o obj-$(CONFIG_VFIO_AMD_XGBE) += amd-xgbe.o obj-$(CONFIG_SOFTMMU) += spapr.o +obj-$(CONFIG_VFIO_AP_MATRIX) += ap-matrix.o endif diff --git a/hw/vfio/ap-matrix.c b/hw/vfio/ap-matrix.c new file mode 100644 index 0000000..eeaa439 --- /dev/null +++ b/hw/vfio/ap-matrix.c @@ -0,0 +1,179 @@ +/* + * VFIO based AP matrix device assignment + * + * Copyright 2017 IBM Corp. + * Author(s): Tony Krowiak + * + * This work is licensed under the terms of the GNU GPL, version 2 or(at + * your option) any version. See the COPYING file in the top-level + * directory. + */ + +#include +#include +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "hw/sysbus.h" +#include "hw/vfio/vfio.h" +#include "hw/vfio/vfio-common.h" +#include "hw/s390x/s390-ap-matrix.h" +#include "hw/s390x/ap-matrix-device.h" +#include "qemu/error-report.h" + +#define TYPE_VFIO_AP_MATRIX_DEVICE "vfio-ap-matrix" +#define AP_MATRIX_SYSFSDEV_PROP_NAME "sysfsdev" + +typedef struct VFIOAPMatrixDevice { + S390APMatrixDevice apmdev; + VFIODevice vdev; +} VFIOAPMatrixDevice; + +static void vfio_ap_matrix_compute_needs_reset(VFIODevice *vdev) +{ + vdev->needs_reset = false; +} + +/* + * We don't need vfio_hot_reset_multi and vfio_eoi operations for + * vfio-ap-matrix device now. + */ +struct VFIODeviceOps vfio_ap_matrix_ops = { + .vfio_compute_needs_reset = vfio_ap_matrix_compute_needs_reset, +}; + +static void vfio_ap_matrix_reset(DeviceState *dev) +{ + APMatrixDevice *apmdev = DO_UPCAST(APMatrixDevice, parent_obj, dev); + S390APMatrixDevice *sapmdev = DO_UPCAST(S390APMatrixDevice, parent_obj, + apmdev); + VFIOAPMatrixDevice *vapmdev = DO_UPCAST(VFIOAPMatrixDevice, apmdev, + sapmdev); + + ioctl(vapmdev->vdev.fd, VFIO_DEVICE_RESET); +} + +static void vfio_put_device(VFIOAPMatrixDevice *apmdev) +{ + g_free(apmdev->vdev.name); + vfio_put_base_device(&apmdev->vdev); +} + +static VFIOGroup *vfio_ap_matrix_get_group(VFIOAPMatrixDevice *vapmdev, + Error **errp) +{ + char *tmp, group_path[PATH_MAX]; + ssize_t len; + int groupid; + + tmp = g_strdup_printf("%s/iommu_group", vapmdev->vdev.sysfsdev); + len = readlink(tmp, group_path, sizeof(group_path)); + g_free(tmp); + + if (len <= 0 || len >= sizeof(group_path)) { + error_setg(errp, "%s: no iommu_group found for %s", + TYPE_VFIO_AP_MATRIX_DEVICE, vapmdev->vdev.sysfsdev); + return NULL; + } + + group_path[len] = 0; + + if (sscanf(basename(group_path), "%d", &groupid) != 1) { + error_setg(errp, "vfio: failed to read %s", group_path); + return NULL; + } + + return vfio_get_group(groupid, &address_space_memory, errp); +} + +static void vfio_ap_matrix_realize(DeviceState *dev, Error **errp) +{ + VFIODevice *vbasedev; + VFIOGroup *vfio_group; + APMatrixDevice *apmdev = DO_UPCAST(APMatrixDevice, parent_obj, dev); + S390APMatrixDevice *sapmdev = DO_UPCAST(S390APMatrixDevice, parent_obj, + apmdev); + VFIOAPMatrixDevice *vapmdev = DO_UPCAST(VFIOAPMatrixDevice, apmdev, + sapmdev); + char *mdevid; + Error *local_err = NULL; + + vfio_group = vfio_ap_matrix_get_group(vapmdev, &local_err); + if (!vfio_group) { + goto out_err; + } + + vapmdev->vdev.ops = &vfio_ap_matrix_ops; + vapmdev->vdev.type = VFIO_DEVICE_TYPE_AP_MATRIX; + mdevid = basename(vapmdev->vdev.sysfsdev); + vapmdev->vdev.name = g_strdup_printf("%s-%s", TYPE_VFIO_AP_MATRIX_DEVICE, + mdevid); + vapmdev->vdev.dev = dev; + QLIST_FOREACH(vbasedev, &vfio_group->device_list, next) { + if (strcmp(vbasedev->name, vapmdev->vdev.name) == 0) { + error_setg(&local_err, + "%s: AP matrix device %s has already been realized", + TYPE_VFIO_AP_MATRIX_DEVICE, vapmdev->vdev.name); + goto out_device_err; + } + } + + if (vfio_get_device(vfio_group, mdevid, &vapmdev->vdev, &local_err)) { + goto out_device_err; + } + + return; + +out_device_err: + vfio_put_group(vfio_group); +out_err: + error_propagate(errp, local_err); +} + +static void vfio_ap_matrix_unrealize(DeviceState *dev, Error **errp) +{ + APMatrixDevice *apm_dev = DO_UPCAST(APMatrixDevice, parent_obj, dev); + S390APMatrixDevice *apmdev = DO_UPCAST(S390APMatrixDevice, parent_obj, + apm_dev); + VFIOAPMatrixDevice *vapmdev = DO_UPCAST(VFIOAPMatrixDevice, apmdev, apmdev); + VFIOGroup *group = vapmdev->vdev.group; + + vfio_put_device(vapmdev); + vfio_put_group(group); +} + +static Property vfio_ap_matrix_properties[] = { + DEFINE_PROP_STRING(AP_MATRIX_SYSFSDEV_PROP_NAME, VFIOAPMatrixDevice, + vdev.sysfsdev), + DEFINE_PROP_END_OF_LIST(), +}; + +static const VMStateDescription vfio_ap_matrix_vmstate = { + .name = TYPE_VFIO_AP_MATRIX_DEVICE, + .unmigratable = 1, +}; + +static void vfio_ap_matrix_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + + dc->props = vfio_ap_matrix_properties; + dc->vmsd = &vfio_ap_matrix_vmstate; + dc->desc = "VFIO-based AP matrix assignment"; + dc->realize = vfio_ap_matrix_realize; + dc->unrealize = vfio_ap_matrix_unrealize; + dc->reset = vfio_ap_matrix_reset; +} + +static const TypeInfo vfio_ap_matrix_info = { + .name = TYPE_VFIO_AP_MATRIX_DEVICE, + .parent = TYPE_S390_AP_MATRIX_DEVICE, + .instance_size = sizeof(VFIOAPMatrixDevice), + .class_init = vfio_ap_matrix_class_init, +}; + +static void register_vfio_ap_matrix_type(void) +{ + type_register_static(&vfio_ap_matrix_info); +} + +type_init(register_vfio_ap_matrix_type) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index f3a2ac9..7cdc1f3 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -46,6 +46,7 @@ enum { VFIO_DEVICE_TYPE_PCI = 0, VFIO_DEVICE_TYPE_PLATFORM = 1, VFIO_DEVICE_TYPE_CCW = 2, + VFIO_DEVICE_TYPE_AP_MATRIX = 3, }; typedef struct VFIOMmap { diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index 4e7ab4c..2d96c57 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -8,8 +8,8 @@ * it under the terms of the GNU General Public License version 2 as * published by the Free Software Foundation. */ -#ifndef VFIO_H -#define VFIO_H +#ifndef _UAPIVFIO_H +#define _UAPIVFIO_H #include #include @@ -199,6 +199,7 @@ struct vfio_device_info { #define VFIO_DEVICE_FLAGS_PLATFORM (1 << 2) /* vfio-platform device */ #define VFIO_DEVICE_FLAGS_AMBA (1 << 3) /* vfio-amba device */ #define VFIO_DEVICE_FLAGS_CCW (1 << 4) /* vfio-ccw device */ +#define VFIO_DEVICE_FLAGS_AP_MATRIX (1 << 5) /* vfio-ap-matrix device */ __u32 num_regions; /* Max region index + 1 */ __u32 num_irqs; /* Max IRQ index + 1 */ }; @@ -214,6 +215,7 @@ struct vfio_device_info { #define VFIO_DEVICE_API_PLATFORM_STRING "vfio-platform" #define VFIO_DEVICE_API_AMBA_STRING "vfio-amba" #define VFIO_DEVICE_API_CCW_STRING "vfio-ccw" +#define VFIO_DEVICE_API_AP_MATRIX_STRING "vfio-ap-matrix" /** * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8, @@ -714,6 +716,26 @@ struct vfio_iommu_spapr_tce_remove { }; #define VFIO_IOMMU_SPAPR_TCE_REMOVE _IO(VFIO_TYPE, VFIO_BASE + 20) +/** + * VFIO_AP_MATRIX_CONFIGURE _IO(VFIO_TYPE, VFIO_BASE + 21 + * + * Configure the AP matrix for a KVM guest. + */ +#define VFIO_AP_MATRIX_CONFIGURE _IO(VFIO_TYPE, VFIO_BASE + 21) + +#define VFIO_AP_MATRIX_MASK_INDICES 4 +#define VFIO_AP_MATTRIX_MASK_BYTES (VFIO_AP_MATRIX_MASK_INDICES * \ + sizeof(__u64)) +#define VFIO_AP_MATRIX_MASK_BITS (VFIO_AP_MATTRIX_MASK_BYTES * 8) + +struct vfio_ap_matrix_config { + __u32 argsz; + __u32 flags; + /* out */ + __u64 apm[VFIO_AP_MATRIX_MASK_INDICES]; + __u64 aqm[VFIO_AP_MATRIX_MASK_INDICES]; + __u64 adm[VFIO_AP_MATRIX_MASK_INDICES]; +}; /* ***************************************************************** */ -#endif /* VFIO_H */ +#endif /* _UAPIVFIO_H */