From patchwork Thu Jan 12 07:19:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dong Jia Shi X-Patchwork-Id: 9512209 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1E156601E7 for ; Thu, 12 Jan 2017 07:20:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 10237285ED for ; Thu, 12 Jan 2017 07:20:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 050352861D; Thu, 12 Jan 2017 07:20:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5C1E0285ED for ; Thu, 12 Jan 2017 07:20:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750829AbdALHUY (ORCPT ); Thu, 12 Jan 2017 02:20:24 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:51608 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750825AbdALHUW (ORCPT ); Thu, 12 Jan 2017 02:20:22 -0500 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id v0C7Iteq066967 for ; Thu, 12 Jan 2017 02:20:21 -0500 Received: from e34.co.us.ibm.com (e34.co.us.ibm.com [32.97.110.152]) by mx0b-001b2d01.pphosted.com with ESMTP id 27x1rcper7-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Thu, 12 Jan 2017 02:20:21 -0500 Received: from localhost by e34.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 12 Jan 2017 00:20:20 -0700 Received: from d03dlp02.boulder.ibm.com (9.17.202.178) by e34.co.us.ibm.com (192.168.1.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 12 Jan 2017 00:20:18 -0700 Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by d03dlp02.boulder.ibm.com (Postfix) with ESMTP id E22DD3E4003F; Thu, 12 Jan 2017 00:20:17 -0700 (MST) Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v0C7KGjt9961972; Thu, 12 Jan 2017 00:20:16 -0700 Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A4BE26E038; Thu, 12 Jan 2017 00:20:16 -0700 (MST) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP id 77C316E041; Thu, 12 Jan 2017 00:20:14 -0700 (MST) From: Dong Jia Shi To: kvm@vger.kernel.org, linux-s390@vger.kernel.org, qemu-devel@nongnu.org Cc: bjsdjshi@linux.vnet.ibm.com, renxiaof@linux.vnet.ibm.com, cornelia.huck@de.ibm.com, borntraeger@de.ibm.com, agraf@suse.com, alex.williamson@redhat.com, pmorel@linux.vnet.ibm.com, pasic@linux.vnet.ibm.com, wkywang@linux.vnet.ibm.com Subject: [PATCH RFC v2 06/15] vfio: ccw: register vfio_ccw to the mediated device framework Date: Thu, 12 Jan 2017 08:19:38 +0100 X-Mailer: git-send-email 2.8.4 In-Reply-To: <20170112071947.98071-1-bjsdjshi@linux.vnet.ibm.com> References: <20170112071947.98071-1-bjsdjshi@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 17011207-0016-0000-0000-0000059D56B4 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00006418; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000199; SDB=6.00806504; UDB=6.00392482; IPR=6.00583835; BA=6.00005049; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00013900; XFM=3.00000011; UTC=2017-01-12 07:20:20 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17011207-0017-0000-0000-00003630A09A Message-Id: <20170112071947.98071-7-bjsdjshi@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-01-12_04:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1612050000 definitions=main-1701120098 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP To make vfio support subchannel devices, we need to leverage the mediated device framework to create a mediated device for the subchannel device. This registers the subchannel device to the mediated device framework during probe to enable mediated device creation. Signed-off-by: Dong Jia Shi Reviewed-by: Pierre Morel --- arch/s390/Kconfig | 2 +- drivers/s390/cio/Makefile | 2 +- drivers/s390/cio/vfio_ccw_drv.c | 10 ++- drivers/s390/cio/vfio_ccw_ops.c | 149 ++++++++++++++++++++++++++++++++++++ drivers/s390/cio/vfio_ccw_private.h | 9 +++ 5 files changed, 169 insertions(+), 3 deletions(-) create mode 100644 drivers/s390/cio/vfio_ccw_ops.c diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig index b920df8..32008b8 100644 --- a/arch/s390/Kconfig +++ b/arch/s390/Kconfig @@ -673,7 +673,7 @@ config EADM_SCH config VFIO_CCW def_tristate n prompt "Support for VFIO-CCW subchannels" - depends on S390_CCW_IOMMU && VFIO + depends on S390_CCW_IOMMU && VFIO_MDEV help This driver allows usage of VFIO-CCW subchannels. diff --git a/drivers/s390/cio/Makefile b/drivers/s390/cio/Makefile index 1bec279..b0586b2 100644 --- a/drivers/s390/cio/Makefile +++ b/drivers/s390/cio/Makefile @@ -18,5 +18,5 @@ obj-$(CONFIG_CCWGROUP) += ccwgroup.o qdio-objs := qdio_main.o qdio_thinint.o qdio_debug.o qdio_setup.o obj-$(CONFIG_QDIO) += qdio.o -vfio_ccw-objs += vfio_ccw_drv.o vfio_ccw_cp.o +vfio_ccw-objs += vfio_ccw_drv.o vfio_ccw_cp.o vfio_ccw_ops.o obj-$(CONFIG_VFIO_CCW) += vfio_ccw.o diff --git a/drivers/s390/cio/vfio_ccw_drv.c b/drivers/s390/cio/vfio_ccw_drv.c index 5759d2a..ef34b15 100644 --- a/drivers/s390/cio/vfio_ccw_drv.c +++ b/drivers/s390/cio/vfio_ccw_drv.c @@ -23,7 +23,7 @@ /* * Helpers */ -static int vfio_ccw_sch_quiesce(struct subchannel *sch) +int vfio_ccw_sch_quiesce(struct subchannel *sch) { struct vfio_ccw_private *private = dev_get_drvdata(&sch->dev); DECLARE_COMPLETION_ONSTACK(completion); @@ -156,8 +156,14 @@ static int vfio_ccw_sch_probe(struct subchannel *sch) if (ret) goto out_disable; + ret = vfio_ccw_mdev_reg(sch); + if (ret) + goto out_rm_group; + return 0; +out_rm_group: + sysfs_remove_group(&sch->dev.kobj, &vfio_subchannel_attr_group); out_disable: cio_disable_subchannel(sch); out_free: @@ -172,6 +178,8 @@ static int vfio_ccw_sch_remove(struct subchannel *sch) vfio_ccw_sch_quiesce(sch); + vfio_ccw_mdev_unreg(sch); + sysfs_remove_group(&sch->dev.kobj, &vfio_subchannel_attr_group); dev_set_drvdata(&sch->dev, NULL); diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c new file mode 100644 index 0000000..6031a10 --- /dev/null +++ b/drivers/s390/cio/vfio_ccw_ops.c @@ -0,0 +1,149 @@ +/* + * Physical device callbacks for vfio_ccw + * + * Copyright IBM Corp. 2017 + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License (version 2 only) + * as published by the Free Software Foundation. + * + * Author(s): Dong Jia Shi + * Xiao Feng Ren + */ + +#include +#include + +#include "vfio_ccw_private.h" + +#define MAX_INSTANCES 1 +static int available_instances = MAX_INSTANCES; + +static int vfio_ccw_mdev_notifier(struct notifier_block *nb, + unsigned long action, + void *data) +{ + struct vfio_ccw_private *private = + container_of(nb, struct vfio_ccw_private, nb); + + if (!private) + return NOTIFY_STOP; + + /* + * TODO: + * Vendor drivers MUST unpin pages in response to an + * invalidation. + */ + if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) + return NOTIFY_BAD; + + return NOTIFY_DONE; +} + +static ssize_t name_show(struct kobject *kobj, struct device *dev, char *buf) +{ + return sprintf(buf, "I/O subchannel (Non-QDIO)\n"); +} +MDEV_TYPE_ATTR_RO(name); + +static ssize_t device_api_show(struct kobject *kobj, struct device *dev, + char *buf) +{ + return sprintf(buf, "%s\n", VFIO_DEVICE_API_CCW_STRING); +} +MDEV_TYPE_ATTR_RO(device_api); + +static ssize_t available_instances_show(struct kobject *kobj, + struct device *dev, char *buf) +{ + return sprintf(buf, "%d\n", available_instances); +} +MDEV_TYPE_ATTR_RO(available_instances); + +static struct attribute *mdev_types_attrs[] = { + &mdev_type_attr_name.attr, + &mdev_type_attr_device_api.attr, + &mdev_type_attr_available_instances.attr, + NULL, +}; + +static struct attribute_group mdev_type_group = { + .name = "io", + .attrs = mdev_types_attrs, +}; + +struct attribute_group *mdev_type_groups[] = { + &mdev_type_group, + NULL, +}; + +static int vfio_ccw_mdev_create(struct kobject *kobj, struct mdev_device *mdev) +{ + struct vfio_ccw_private *private = dev_get_drvdata(mdev->parent->dev); + + /* Only support one mediated device for each physical subchannel. */ + if (private->mdev) + return -EPERM; + + private->mdev = mdev; + available_instances--; + + return 0; +} + +static int vfio_ccw_mdev_remove(struct mdev_device *mdev) +{ + struct vfio_ccw_private *private = dev_get_drvdata(mdev->parent->dev); + struct subchannel *sch; + int ret; + + sch = private->sch; + ret = vfio_ccw_sch_quiesce(sch); + if (ret) + return ret; + ret = cio_enable_subchannel(sch, (u32)(unsigned long)sch); + if (ret) + return ret; + + private->mdev = NULL; + available_instances++; + + return 0; +} + +static int vfio_ccw_mdev_open(struct mdev_device *mdev) +{ + struct vfio_ccw_private *private = dev_get_drvdata(mdev->parent->dev); + unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP; + + private->nb.notifier_call = vfio_ccw_mdev_notifier; + + return vfio_register_notifier(&mdev->dev, VFIO_IOMMU_NOTIFY, + &events, &private->nb); +} + +void vfio_ccw_mdev_release(struct mdev_device *mdev) +{ + struct vfio_ccw_private *private = dev_get_drvdata(mdev->parent->dev); + + vfio_unregister_notifier(&mdev->dev, VFIO_IOMMU_NOTIFY, &private->nb); +} + +static const struct parent_ops vfio_ccw_mdev_ops = { + .owner = THIS_MODULE, + .supported_type_groups = mdev_type_groups, + .create = vfio_ccw_mdev_create, + .remove = vfio_ccw_mdev_remove, + .open = vfio_ccw_mdev_open, + .release = vfio_ccw_mdev_release, +}; + +int vfio_ccw_mdev_reg(struct subchannel *sch) +{ + return mdev_register_device(&sch->dev, &vfio_ccw_mdev_ops); +} + +void vfio_ccw_mdev_unreg(struct subchannel *sch) +{ + mdev_unregister_device(&sch->dev); +} diff --git a/drivers/s390/cio/vfio_ccw_private.h b/drivers/s390/cio/vfio_ccw_private.h index 1617c96..4cd6657 100644 --- a/drivers/s390/cio/vfio_ccw_private.h +++ b/drivers/s390/cio/vfio_ccw_private.h @@ -20,10 +20,19 @@ * struct vfio_ccw_private * @sch: pointor to the subchannel * @completion: synchronization helper of the I/O completion + * @mdev: pointor to the mediated device + * @nb: notifier for vfio events */ struct vfio_ccw_private { struct subchannel *sch; struct completion *completion; + struct mdev_device *mdev; + struct notifier_block nb; } __aligned(8); +extern int vfio_ccw_mdev_reg(struct subchannel *sch); +extern void vfio_ccw_mdev_unreg(struct subchannel *sch); + +extern int vfio_ccw_sch_quiesce(struct subchannel *sch); + #endif