From patchwork Mon Mar 23 17:19:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diana Madalina Craciun X-Patchwork-Id: 11453597 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56C54139A for ; Mon, 23 Mar 2020 17:20:31 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 323DD20735 for ; Mon, 23 Mar 2020 17:20:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Lpp1c+lh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 323DD20735 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oss.nxp.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=4QnRfo4HAJ6XWRY+Pff1QA5zCfJh+qkAV3INJ4DfrC8=; b=Lpp1c+lh42jQz0xo/vXRwSVvyW 4Hbtx/4ytg/DKeM5pwCqGh7a4U9uNcbWSBdRAeaQSnaioBYCQrWwsj9byVh83W6OP6huwyAgITX/N nGu6fzsJaCGt/TvesnGiFNQR/lQ7NZf7LPlOFU2iNhxZjeCiuBSsKdEEgP9r8hKPHXfNZwxEi8h/o JFSEMrTQ+WZg8hJNvPFpdVN73f0FcL3ng4AsVXDkVLqdgGMxIBnCNhL3lUw7cA+6w7OzN3LMyYuz1 031+Y6TL5KQRHoJdsr+V2RJB9b2AMCziHuVOEeSJVKNyDTD0dUiOLqfXy7sdSLUeZ7PLwf8o9OFNc kmx14IVQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQkT-0005k5-Se; Mon, 23 Mar 2020 17:20:25 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQjb-0003n4-6s for linux-arm-kernel@lists.infradead.org; Mon, 23 Mar 2020 17:19:34 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 5B4711A14DE; Mon, 23 Mar 2020 18:19:27 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 4C6CB1A14D4; Mon, 23 Mar 2020 18:19:27 +0100 (CET) Received: from fsr-ub1864-111.ea.freescale.net (fsr-ub1864-111.ea.freescale.net [10.171.82.141]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id E8F832035C; Mon, 23 Mar 2020 18:19:26 +0100 (CET) From: Diana Craciun To: kvm@vger.kernel.org, alex.williamson@redhat.com, laurentiu.tudor@nxp.com, linux-arm-kernel@lists.infradead.org, bharatb.yadav@gmail.com Subject: [PATCH 1/9] vfio/fsl-mc: Add VFIO framework skeleton for fsl-mc devices Date: Mon, 23 Mar 2020 19:19:03 +0200 Message-Id: <20200323171911.27178-2-diana.craciun@oss.nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200323171911.27178-1-diana.craciun@oss.nxp.com> References: <20200323171911.27178-1-diana.craciun@oss.nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200323_101931_530951_ECAEC259 X-CRM114-Status: GOOD ( 21.04 ) X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [92.121.34.13 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bharat Bhushan , linux-kernel@vger.kernel.org, Diana Craciun MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org From: Bharat Bhushan DPAA2 (Data Path Acceleration Architecture) consists in mechanisms for processing Ethernet packets, queue management, accelerators, etc. The Management Complex (mc) is a hardware entity that manages the DPAA2 hardware resources. It provides an object-based abstraction for software drivers to use the DPAA2 hardware. The MC mediates operations such as create, discover, destroy of DPAA2 objects. The MC provides memory-mapped I/O command interfaces (MC portals) which DPAA2 software drivers use to operate on DPAA2 objects. A DPRC is a container object that holds other types of DPAA2 objects. Each object in the DPRC is a Linux device and bound to a driver. The MC-bus driver is a platform driver (different from PCI or platform bus). The DPRC driver does runtime management of a bus instance. It performs the initial scan of the DPRC and handles changes in the DPRC configuration (adding/removing objects). All objects inside a container share the same hardware isolation context, meaning that only an entire DPRC can be assigned to a virtual machine. When a container is assigned to a virtual machine, all the objects within that container are assigned to that virtual machine. The DPRC container assigned to the virtual machine is not allowed to change contents (add/remove objects) by the guest. The restriction is set by the host and enforced by the mc hardware. The DPAA2 objects can be directly assigned to the guest. However the MC portals (the memory mapped command interface to the MC) need to be emulated because there are commands that configure the interrupts and the isolation IDs which are virtual in the guest. Example: echo vfio-fsl-mc > /sys/bus/fsl-mc/devices/dprc.2/driver_override echo dprc.2 > /sys/bus/fsl-mc/drivers/vfio-fsl-mc/bind The dprc.2 is bound to the VFIO driver and all the objects within dprc.2 are going to be bound to the VFIO driver. This patch adds the infrastructure for VFIO support for fsl-mc devices. Subsequent patches will add support for binding and secure assigning these devices using VFIO. More details about the DPAA2 objects can be found here: Documentation/networking/device_drivers/freescale/dpaa2/overview.rst Signed-off-by: Bharat Bhushan Signed-off-by: Diana Craciun --- MAINTAINERS | 6 + drivers/vfio/Kconfig | 1 + drivers/vfio/Makefile | 1 + drivers/vfio/fsl-mc/Kconfig | 9 ++ drivers/vfio/fsl-mc/Makefile | 2 + drivers/vfio/fsl-mc/vfio_fsl_mc.c | 161 ++++++++++++++++++++++ drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 14 ++ include/uapi/linux/vfio.h | 1 + 8 files changed, 195 insertions(+) create mode 100644 drivers/vfio/fsl-mc/Kconfig create mode 100644 drivers/vfio/fsl-mc/Makefile create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc.c create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc_private.h diff --git a/MAINTAINERS b/MAINTAINERS index cc1d18cb5d18..fc547e6f5bf8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -17566,6 +17566,12 @@ F: drivers/vfio/ F: include/linux/vfio.h F: include/uapi/linux/vfio.h +VFIO FSL-MC DRIVER +M: Diana Craciun +L: kvm@vger.kernel.org +S: Maintained +F: drivers/vfio/fsl-mc/ + VFIO MEDIATED DEVICE DRIVERS M: Kirti Wankhede L: kvm@vger.kernel.org diff --git a/drivers/vfio/Kconfig b/drivers/vfio/Kconfig index fd17db9b432f..5533df91b257 100644 --- a/drivers/vfio/Kconfig +++ b/drivers/vfio/Kconfig @@ -47,4 +47,5 @@ menuconfig VFIO_NOIOMMU source "drivers/vfio/pci/Kconfig" source "drivers/vfio/platform/Kconfig" source "drivers/vfio/mdev/Kconfig" +source "drivers/vfio/fsl-mc/Kconfig" source "virt/lib/Kconfig" diff --git a/drivers/vfio/Makefile b/drivers/vfio/Makefile index de67c4725cce..fee73f3d9480 100644 --- a/drivers/vfio/Makefile +++ b/drivers/vfio/Makefile @@ -9,3 +9,4 @@ obj-$(CONFIG_VFIO_SPAPR_EEH) += vfio_spapr_eeh.o obj-$(CONFIG_VFIO_PCI) += pci/ obj-$(CONFIG_VFIO_PLATFORM) += platform/ obj-$(CONFIG_VFIO_MDEV) += mdev/ +obj-$(CONFIG_VFIO_FSL_MC) += fsl-mc/ diff --git a/drivers/vfio/fsl-mc/Kconfig b/drivers/vfio/fsl-mc/Kconfig new file mode 100644 index 000000000000..b1a527d6b6f2 --- /dev/null +++ b/drivers/vfio/fsl-mc/Kconfig @@ -0,0 +1,9 @@ +config VFIO_FSL_MC + tristate "VFIO support for QorIQ DPAA2 fsl-mc bus devices" + depends on VFIO && FSL_MC_BUS && EVENTFD + help + Driver to enable support for the VFIO QorIQ DPAA2 fsl-mc + (Management Complex) devices. This is required to passthrough + fsl-mc bus devices using the VFIO framework. + + If you don't know what to do here, say N. diff --git a/drivers/vfio/fsl-mc/Makefile b/drivers/vfio/fsl-mc/Makefile new file mode 100644 index 000000000000..6f2b80645d5b --- /dev/null +++ b/drivers/vfio/fsl-mc/Makefile @@ -0,0 +1,2 @@ +vfio-fsl_mc-y := vfio_fsl_mc.o +obj-$(CONFIG_VFIO_FSL_MC) += vfio_fsl_mc.o diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c new file mode 100644 index 000000000000..320fb09b5691 --- /dev/null +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -0,0 +1,161 @@ +// SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause) +/* + * Copyright 2013-2016 Freescale Semiconductor Inc. + * Copyright 2016-2017,2019-2020 NXP + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "vfio_fsl_mc_private.h" + +static int vfio_fsl_mc_open(void *device_data) +{ + if (!try_module_get(THIS_MODULE)) + return -ENODEV; + + return 0; +} + +static void vfio_fsl_mc_release(void *device_data) +{ + module_put(THIS_MODULE); +} + +static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd, + unsigned long arg) +{ + switch (cmd) { + case VFIO_DEVICE_GET_INFO: + { + return -EINVAL; + } + case VFIO_DEVICE_GET_REGION_INFO: + { + return -EINVAL; + } + case VFIO_DEVICE_GET_IRQ_INFO: + { + return -EINVAL; + } + case VFIO_DEVICE_SET_IRQS: + { + return -EINVAL; + } + case VFIO_DEVICE_RESET: + { + return -EINVAL; + } + default: + return -EINVAL; + } +} + +static ssize_t vfio_fsl_mc_read(void *device_data, char __user *buf, + size_t count, loff_t *ppos) +{ + return -EINVAL; +} + +static ssize_t vfio_fsl_mc_write(void *device_data, const char __user *buf, + size_t count, loff_t *ppos) +{ + return -EINVAL; +} + +static int vfio_fsl_mc_mmap(void *device_data, struct vm_area_struct *vma) +{ + return -EINVAL; +} + +static const struct vfio_device_ops vfio_fsl_mc_ops = { + .name = "vfio-fsl-mc", + .open = vfio_fsl_mc_open, + .release = vfio_fsl_mc_release, + .ioctl = vfio_fsl_mc_ioctl, + .read = vfio_fsl_mc_read, + .write = vfio_fsl_mc_write, + .mmap = vfio_fsl_mc_mmap, +}; + +static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev) +{ + struct iommu_group *group; + struct vfio_fsl_mc_device *vdev; + struct device *dev = &mc_dev->dev; + int ret; + + group = vfio_iommu_group_get(dev); + if (!group) { + dev_err(dev, "%s: VFIO: No IOMMU group\n", __func__); + return -EINVAL; + } + + vdev = devm_kzalloc(dev, sizeof(*vdev), GFP_KERNEL); + if (!vdev) { + vfio_iommu_group_put(group, dev); + return -ENOMEM; + } + + vdev->mc_dev = mc_dev; + + ret = vfio_add_group_dev(dev, &vfio_fsl_mc_ops, vdev); + if (ret) { + dev_err(dev, "%s: Failed to add to vfio group\n", __func__); + vfio_iommu_group_put(group, dev); + return ret; + } + + return ret; +} + +static int vfio_fsl_mc_remove(struct fsl_mc_device *mc_dev) +{ + struct vfio_fsl_mc_device *vdev; + struct device *dev = &mc_dev->dev; + + vdev = vfio_del_group_dev(dev); + if (!vdev) + return -EINVAL; + + vfio_iommu_group_put(mc_dev->dev.iommu_group, dev); + devm_kfree(dev, vdev); + + return 0; +} + +/* + * vfio-fsl_mc is a meta-driver, so use driver_override interface to + * bind a fsl_mc container with this driver and match_id_table is NULL. + */ +static struct fsl_mc_driver vfio_fsl_mc_driver = { + .probe = vfio_fsl_mc_probe, + .remove = vfio_fsl_mc_remove, + .match_id_table = NULL, + .driver = { + .name = "vfio-fsl-mc", + .owner = THIS_MODULE, + }, +}; + +static int __init vfio_fsl_mc_driver_init(void) +{ + return fsl_mc_driver_register(&vfio_fsl_mc_driver); +} + +static void __exit vfio_fsl_mc_driver_exit(void) +{ + fsl_mc_driver_unregister(&vfio_fsl_mc_driver); +} + +module_init(vfio_fsl_mc_driver_init); +module_exit(vfio_fsl_mc_driver_exit); + +MODULE_LICENSE("GPL v2"); +MODULE_DESCRIPTION("VFIO for FSL-MC devices - User Level meta-driver"); diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h new file mode 100644 index 000000000000..b92858a003c0 --- /dev/null +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause) */ +/* + * Copyright 2013-2016 Freescale Semiconductor Inc. + * Copyright 2016,2019-2020 NXP + */ + +#ifndef VFIO_FSL_MC_PRIVATE_H +#define VFIO_FSL_MC_PRIVATE_H + +struct vfio_fsl_mc_device { + struct fsl_mc_device *mc_dev; +}; + +#endif /* VFIO_PCI_PRIVATE_H */ diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 9e843a147ead..6d0a7a071ef4 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -201,6 +201,7 @@ struct vfio_device_info { #define VFIO_DEVICE_FLAGS_AMBA (1 << 3) /* vfio-amba device */ #define VFIO_DEVICE_FLAGS_CCW (1 << 4) /* vfio-ccw device */ #define VFIO_DEVICE_FLAGS_AP (1 << 5) /* vfio-ap device */ +#define VFIO_DEVICE_FLAGS_FSL_MC (1 << 6) /* vfio-fsl-mc device */ __u32 num_regions; /* Max region index + 1 */ __u32 num_irqs; /* Max IRQ index + 1 */ }; From patchwork Mon Mar 23 17:19:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diana Madalina Craciun X-Patchwork-Id: 11453571 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 892CB1668 for ; Mon, 23 Mar 2020 17:19:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6751B2072E for ; Mon, 23 Mar 2020 17:19:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="g68sTma2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6751B2072E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oss.nxp.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=qVEp4rBB/9U3S+rIy7TTv07ONRpv0k2ND4lRtAeOpKM=; b=g68sTma2Sm74RcGAv5EhVneHCR q7m4r5f5G6tmJ3r02nkgSRIPHxaSw8w2OY0EJHupAMlarnMIu+PqXDPSZWnB5P47oP6M42kU2c50W ooNOlPmdUaYFUPdCWfK1PG37BrLD7c+5E1o1iT6DcyR2Ij3mRfbCUjDVVda6lcbNSezOJcGMZ6pB4 evAjvgelOPIHgz+5s07KpC87jIvyaFZ2IPoP8MN2BI2AsQpLxqOP0uMUN4ere6IBqEJmG4hlpz3Ox jzXz/41KGXsvOzqolygkq+gynYuSWpkbrDzalwlxxTv1o1NNCAyovE2XQ4MS7QPx1sUVXEG4olG1e vTnIPitA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQje-0003pr-8Z; Mon, 23 Mar 2020 17:19:34 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQja-0003n5-SZ for linux-arm-kernel@lists.infradead.org; Mon, 23 Mar 2020 17:19:32 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id E38D91A14D4; Mon, 23 Mar 2020 18:19:27 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id D73871A14A9; Mon, 23 Mar 2020 18:19:27 +0100 (CET) Received: from fsr-ub1864-111.ea.freescale.net (fsr-ub1864-111.ea.freescale.net [10.171.82.141]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id 599792035C; Mon, 23 Mar 2020 18:19:27 +0100 (CET) From: Diana Craciun To: kvm@vger.kernel.org, alex.williamson@redhat.com, laurentiu.tudor@nxp.com, linux-arm-kernel@lists.infradead.org, bharatb.yadav@gmail.com Subject: [PATCH 2/9] vfio/fsl-mc: Scan DPRC objects on vfio-fsl-mc driver bind Date: Mon, 23 Mar 2020 19:19:04 +0200 Message-Id: <20200323171911.27178-3-diana.craciun@oss.nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200323171911.27178-1-diana.craciun@oss.nxp.com> References: <20200323171911.27178-1-diana.craciun@oss.nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200323_101931_203329_539AE6B7 X-CRM114-Status: GOOD ( 11.19 ) X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [92.121.34.13 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bharat Bhushan , linux-kernel@vger.kernel.org, Diana Craciun MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org The DPRC (Data Path Resource Container) device is a bus device and has child devices attached to it. When the vfio-fsl-mc driver is probed the DPRC is scanned and the child devices discovered and initialized. Signed-off-by: Bharat Bhushan Signed-off-by: Diana Craciun Reported-by: kbuild test robot --- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 66 +++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index 320fb09b5691..5cc533808bc1 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -74,6 +74,34 @@ static int vfio_fsl_mc_mmap(void *device_data, struct vm_area_struct *vma) return -EINVAL; } +static int vfio_fsl_mc_init_device(struct vfio_fsl_mc_device *vdev) +{ + struct fsl_mc_device *mc_dev = vdev->mc_dev; + int ret = 0; + + /* Non-dprc devices share mc_io from parent */ + if (!is_fsl_mc_bus_dprc(mc_dev)) { + struct fsl_mc_device *mc_cont = to_fsl_mc_device(mc_dev->dev.parent); + + mc_dev->mc_io = mc_cont->mc_io; + return 0; + } + + /* open DPRC, allocate a MC portal */ + ret = dprc_setup(mc_dev); + if (ret < 0) { + dev_err(&mc_dev->dev, "Failed to setup DPRC (error = %d)\n", ret); + return ret; + } + + ret = dprc_scan_container(mc_dev, mc_dev->driver_override, false); + if (ret < 0) { + dev_err(&mc_dev->dev, "Container scanning failed: %d\n", ret); + dprc_cleanup(mc_dev); + } + + return 0; +} static const struct vfio_device_ops vfio_fsl_mc_ops = { .name = "vfio-fsl-mc", .open = vfio_fsl_mc_open, @@ -112,9 +140,42 @@ static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev) return ret; } + ret = vfio_fsl_mc_init_device(vdev); + if (ret) { + vfio_iommu_group_put(group, dev); + return ret; + } + return ret; } +static int vfio_fsl_mc_device_remove(struct device *dev, void *data) +{ + struct fsl_mc_device *mc_dev; + + WARN_ON(!dev); + mc_dev = to_fsl_mc_device(dev); + if (WARN_ON(!mc_dev)) + return -ENODEV; + + kfree(mc_dev->driver_override); + mc_dev->driver_override = NULL; + + /* + * The device-specific remove callback will get invoked by device_del() + */ + device_del(&mc_dev->dev); + put_device(&mc_dev->dev); + + return 0; +} + +static void vfio_fsl_mc_cleanup_dprc(struct fsl_mc_device *mc_dev) +{ + device_for_each_child(&mc_dev->dev, NULL, vfio_fsl_mc_device_remove); + dprc_cleanup(mc_dev); +} + static int vfio_fsl_mc_remove(struct fsl_mc_device *mc_dev) { struct vfio_fsl_mc_device *vdev; @@ -124,6 +185,11 @@ static int vfio_fsl_mc_remove(struct fsl_mc_device *mc_dev) if (!vdev) return -EINVAL; + if (is_fsl_mc_bus_dprc(mc_dev)) + vfio_fsl_mc_cleanup_dprc(vdev->mc_dev); + + mc_dev->mc_io = NULL; + vfio_iommu_group_put(mc_dev->dev.iommu_group, dev); devm_kfree(dev, vdev); From patchwork Mon Mar 23 17:19:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diana Madalina Craciun X-Patchwork-Id: 11453575 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 674331668 for ; Mon, 23 Mar 2020 17:19:45 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C64DD20722 for ; Mon, 23 Mar 2020 17:19:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Mj6hwo5+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C64DD20722 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oss.nxp.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=d3UPDAZaHHjEbdpcy0G9F5G75cqtR96eq/Ll5faCKCo=; b=Mj6hwo5+14Ire2zvDy53sXqlty Mo+Hn0FVDf6N4dlgw+0GRG4F1bgL1919akNZ+Vxdq6GgHOwM1MxyCRfuPh4fSPu8NhZLomEODJXzp RcHOVMGgzH/+pIuxNW0spFpS5dedesPxUTmXw6klN7pOnQAIwqeO7FIm4TPxhGrxXkxUfP0d5ih8/ h4d6yhWtff86khih4fgexfmXpQArxbAIVIFT+Mxg9A2Jo7k/WJnH6m/8ibANJ0yCH2Cx48Wg0nM6h hBEhUkmahXBCjmci+Wjyl9sgVreGU9KiUxGi8wM0f0LSkOqsRdnLxdhyp8vLudouXc2uc5k9NxISp pC5LTMTA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQjl-0003xV-0d; Mon, 23 Mar 2020 17:19:41 +0000 Received: from inva021.nxp.com ([92.121.34.21]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQjb-0003n8-9C for linux-arm-kernel@lists.infradead.org; Mon, 23 Mar 2020 17:19:32 +0000 Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 9B313201848; Mon, 23 Mar 2020 18:19:28 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 8E2A52012E7; Mon, 23 Mar 2020 18:19:28 +0100 (CET) Received: from fsr-ub1864-111.ea.freescale.net (fsr-ub1864-111.ea.freescale.net [10.171.82.141]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id F12E62035C; Mon, 23 Mar 2020 18:19:27 +0100 (CET) From: Diana Craciun To: kvm@vger.kernel.org, alex.williamson@redhat.com, laurentiu.tudor@nxp.com, linux-arm-kernel@lists.infradead.org, bharatb.yadav@gmail.com Subject: [PATCH 3/9] vfio/fsl-mc: Implement VFIO_DEVICE_GET_INFO ioctl Date: Mon, 23 Mar 2020 19:19:05 +0200 Message-Id: <20200323171911.27178-4-diana.craciun@oss.nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200323171911.27178-1-diana.craciun@oss.nxp.com> References: <20200323171911.27178-1-diana.craciun@oss.nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200323_101931_457021_368D4ACC X-CRM114-Status: UNSURE ( 7.86 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [92.121.34.21 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bharat Bhushan , linux-kernel@vger.kernel.org, Diana Craciun MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Allow userspace to get fsl-mc device info (number of regions and irqs). Signed-off-by: Bharat Bhushan Signed-off-by: Diana Craciun --- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index 5cc533808bc1..ab1cde375fc6 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -31,10 +31,30 @@ static void vfio_fsl_mc_release(void *device_data) static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd, unsigned long arg) { + unsigned long minsz; + struct vfio_fsl_mc_device *vdev = device_data; + struct fsl_mc_device *mc_dev = vdev->mc_dev; + switch (cmd) { case VFIO_DEVICE_GET_INFO: { - return -EINVAL; + struct vfio_device_info info; + + minsz = offsetofend(struct vfio_device_info, num_irqs); + + if (copy_from_user(&info, (void __user *)arg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + info.flags = VFIO_DEVICE_FLAGS_FSL_MC; + info.num_regions = mc_dev->obj_desc.region_count; + info.num_irqs = mc_dev->obj_desc.irq_count; + + return copy_to_user((void __user *)arg, &info, minsz) ? + -EFAULT : 0; + } case VFIO_DEVICE_GET_REGION_INFO: { From patchwork Mon Mar 23 17:19:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diana Madalina Craciun X-Patchwork-Id: 11453599 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F29DD139A for ; Mon, 23 Mar 2020 17:20:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CFDE02072D for ; Mon, 23 Mar 2020 17:20:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="DKaa9Ybp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CFDE02072D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oss.nxp.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=fbyDB25yuCMAVudjjWQ91vyPvnAVgOnZP7/u8eBh61w=; b=DKaa9Ybp8CzplusyiLMPCVT+o4 2+aHmCvxd2y4PAL0bhzih5sOB6oqVyTNOVrUdBInGGOBtJvfALaVYKff4WtL9oKIqPB9/KRw0xeS2 i6Y1hYYFOiO7KfjbuGYr7Old1cS6A6FbYWDuAaJqotEivNtyUk8lp0WBZabmop4sinUSVKrBAV5Se 98smSXh9TkdsCrd3KkWXtj3WqvzWeEOUN89VHDa7m2F5Nf/3Qzt4aBC7phVBTtyYgBeQpHQKUaR8g AfuD2x1jx4dGVF8bI3MTTo3qO+rQv8MZ+IRdlmD7TP38CBM5HkfIAAe1ULRqePtBb1LCQd1oYH7a5 c4OM5xuw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQki-0005y1-Ph; Mon, 23 Mar 2020 17:20:40 +0000 Received: from inva021.nxp.com ([92.121.34.21]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQjb-0003n9-9A for linux-arm-kernel@lists.infradead.org; Mon, 23 Mar 2020 17:19:34 +0000 Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 4603820183C; Mon, 23 Mar 2020 18:19:29 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 390812012E7; Mon, 23 Mar 2020 18:19:29 +0100 (CET) Received: from fsr-ub1864-111.ea.freescale.net (fsr-ub1864-111.ea.freescale.net [10.171.82.141]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id A83612035C; Mon, 23 Mar 2020 18:19:28 +0100 (CET) From: Diana Craciun To: kvm@vger.kernel.org, alex.williamson@redhat.com, laurentiu.tudor@nxp.com, linux-arm-kernel@lists.infradead.org, bharatb.yadav@gmail.com Subject: [PATCH 4/9] vfio/fsl-mc: Implement VFIO_DEVICE_GET_REGION_INFO ioctl call Date: Mon, 23 Mar 2020 19:19:06 +0200 Message-Id: <20200323171911.27178-5-diana.craciun@oss.nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200323171911.27178-1-diana.craciun@oss.nxp.com> References: <20200323171911.27178-1-diana.craciun@oss.nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200323_101931_613428_5D35E141 X-CRM114-Status: GOOD ( 12.20 ) X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [92.121.34.21 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bharat Bhushan , linux-kernel@vger.kernel.org, Diana Craciun MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Expose to userspace information about the memory regions. Signed-off-by: Bharat Bhushan Signed-off-by: Diana Craciun --- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 77 ++++++++++++++++++++++- drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 19 ++++++ 2 files changed, 95 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index ab1cde375fc6..094df3aa3710 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -15,16 +15,71 @@ #include "vfio_fsl_mc_private.h" +static int vfio_fsl_mc_regions_init(struct vfio_fsl_mc_device *vdev) +{ + struct fsl_mc_device *mc_dev = vdev->mc_dev; + int count = mc_dev->obj_desc.region_count; + int i; + + vdev->regions = kcalloc(count, sizeof(struct vfio_fsl_mc_region), + GFP_KERNEL); + if (!vdev->regions) + return -ENOMEM; + + for (i = 0; i < count; i++) { + struct resource *res = &mc_dev->regions[i]; + + vdev->regions[i].addr = res->start; + vdev->regions[i].size = PAGE_ALIGN((resource_size(res))); + vdev->regions[i].flags = 0; + } + + vdev->num_regions = mc_dev->obj_desc.region_count; + return 0; +} + +static void vfio_fsl_mc_regions_cleanup(struct vfio_fsl_mc_device *vdev) +{ + vdev->num_regions = 0; + kfree(vdev->regions); +} + static int vfio_fsl_mc_open(void *device_data) { + struct vfio_fsl_mc_device *vdev = device_data; + int ret; + if (!try_module_get(THIS_MODULE)) return -ENODEV; + mutex_lock(&vdev->driver_lock); + if (!vdev->refcnt) { + ret = vfio_fsl_mc_regions_init(vdev); + if (ret) + goto err_reg_init; + } + vdev->refcnt++; + + mutex_unlock(&vdev->driver_lock); return 0; + +err_reg_init: + mutex_unlock(&vdev->driver_lock); + module_put(THIS_MODULE); + return ret; } static void vfio_fsl_mc_release(void *device_data) { + struct vfio_fsl_mc_device *vdev = device_data; + + mutex_lock(&vdev->driver_lock); + + if (!(--vdev->refcnt)) + vfio_fsl_mc_regions_cleanup(vdev); + + mutex_unlock(&vdev->driver_lock); + module_put(THIS_MODULE); } @@ -58,7 +113,26 @@ static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd, } case VFIO_DEVICE_GET_REGION_INFO: { - return -EINVAL; + struct vfio_region_info info; + + minsz = offsetofend(struct vfio_region_info, offset); + + if (copy_from_user(&info, (void __user *)arg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + if (info.index >= vdev->num_regions) + return -EINVAL; + + /* map offset to the physical address */ + info.offset = VFIO_FSL_MC_INDEX_TO_OFFSET(info.index); + info.size = vdev->regions[info.index].size; + info.flags = vdev->regions[info.index].flags; + + return copy_to_user((void __user *)arg, &info, minsz); + } case VFIO_DEVICE_GET_IRQ_INFO: { @@ -165,6 +239,7 @@ static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev) vfio_iommu_group_put(group, dev); return ret; } + mutex_init(&vdev->driver_lock); return ret; } diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h index b92858a003c0..764e2cf2c70d 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h @@ -7,8 +7,27 @@ #ifndef VFIO_FSL_MC_PRIVATE_H #define VFIO_FSL_MC_PRIVATE_H +#define VFIO_FSL_MC_OFFSET_SHIFT 40 +#define VFIO_FSL_MC_OFFSET_MASK (((u64)(1) << VFIO_FSL_MC_OFFSET_SHIFT) - 1) + +#define VFIO_FSL_MC_OFFSET_TO_INDEX(off) ((off) >> VFIO_FSL_MC_OFFSET_SHIFT) + +#define VFIO_FSL_MC_INDEX_TO_OFFSET(index) \ + ((u64)(index) << VFIO_FSL_MC_OFFSET_SHIFT) + +struct vfio_fsl_mc_region { + u32 flags; + u32 type; + u64 addr; + resource_size_t size; +}; + struct vfio_fsl_mc_device { struct fsl_mc_device *mc_dev; + int refcnt; + u32 num_regions; + struct vfio_fsl_mc_region *regions; + struct mutex driver_lock; }; #endif /* VFIO_PCI_PRIVATE_H */ From patchwork Mon Mar 23 17:19:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diana Madalina Craciun X-Patchwork-Id: 11453591 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E89AF139A for ; Mon, 23 Mar 2020 17:20:09 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B57F72074D for ; Mon, 23 Mar 2020 17:20:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="sCj7cWkW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B57F72074D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oss.nxp.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=HcWgq9O/z+9yOOXBdtqLmFvRsVF8A8PQEnz2tJ1XMEA=; b=sCj7cWkW6CLSeco7seJLC3gCJE CB5CFMteNsrdA07tJxhdaBvWdl4KaMaOcxQfxbBEaRZMvu+LXwY/TyQMNGxJtrYCyIR+X0nfl00UL nbSqr6vjxSzMKcIayJ6pIusXGBa0eGt0UXvj3Y9hKbStlZMGbe5ve2NAC7/PwI94c1PbcmZZU0MT6 UaxpAvTm651kwuwSvRAN1gkB4TDRVNZ/BVutv3tWMCMdVFaBV8TI5gkrrJyi/lhHM7ywB4mkT/q24 q/lHACCOb6JH5k6qPoQ4Byx11pJOuQezR6gXQsrPQv2OyXF9q80KELR0TVHfUGKQbAmduLIruj0du RBlZto8A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQk6-0004Gk-QN; Mon, 23 Mar 2020 17:20:02 +0000 Received: from inva021.nxp.com ([92.121.34.21]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQjb-0003nA-9D for linux-arm-kernel@lists.infradead.org; Mon, 23 Mar 2020 17:19:34 +0000 Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id DB1F220184A; Mon, 23 Mar 2020 18:19:29 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id CE7292012E7; Mon, 23 Mar 2020 18:19:29 +0100 (CET) Received: from fsr-ub1864-111.ea.freescale.net (fsr-ub1864-111.ea.freescale.net [10.171.82.141]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id 528452035C; Mon, 23 Mar 2020 18:19:29 +0100 (CET) From: Diana Craciun To: kvm@vger.kernel.org, alex.williamson@redhat.com, laurentiu.tudor@nxp.com, linux-arm-kernel@lists.infradead.org, bharatb.yadav@gmail.com Subject: [PATCH 5/9] vfio/fsl-mc: Allow userspace to MMAP fsl-mc device MMIO regions Date: Mon, 23 Mar 2020 19:19:07 +0200 Message-Id: <20200323171911.27178-6-diana.craciun@oss.nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200323171911.27178-1-diana.craciun@oss.nxp.com> References: <20200323171911.27178-1-diana.craciun@oss.nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200323_101931_624228_7AF1DA6B X-CRM114-Status: GOOD ( 10.85 ) X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [92.121.34.21 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bharat Bhushan , linux-kernel@vger.kernel.org, Diana Craciun MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Allow userspace to mmap device regions for direct access of fsl-mc devices. Signed-off-by: Bharat Bhushan Signed-off-by: Diana Craciun --- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 64 ++++++++++++++++++++++- drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 2 + 2 files changed, 64 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index 094df3aa3710..6625b7cb0a3e 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -31,7 +31,11 @@ static int vfio_fsl_mc_regions_init(struct vfio_fsl_mc_device *vdev) vdev->regions[i].addr = res->start; vdev->regions[i].size = PAGE_ALIGN((resource_size(res))); - vdev->regions[i].flags = 0; + vdev->regions[i].flags = VFIO_REGION_INFO_FLAG_MMAP; + vdev->regions[i].flags |= VFIO_REGION_INFO_FLAG_READ; + if (!(mc_dev->regions[i].flags & IORESOURCE_READONLY)) + vdev->regions[i].flags |= VFIO_REGION_INFO_FLAG_WRITE; + vdev->regions[i].type = mc_dev->regions[i].flags & IORESOURCE_BITS; } vdev->num_regions = mc_dev->obj_desc.region_count; @@ -163,9 +167,65 @@ static ssize_t vfio_fsl_mc_write(void *device_data, const char __user *buf, return -EINVAL; } +static int vfio_fsl_mc_mmap_mmio(struct vfio_fsl_mc_region region, + struct vm_area_struct *vma) +{ + u64 size = vma->vm_end - vma->vm_start; + u64 pgoff, base; + + pgoff = vma->vm_pgoff & + ((1U << (VFIO_FSL_MC_OFFSET_SHIFT - PAGE_SHIFT)) - 1); + base = pgoff << PAGE_SHIFT; + + if (region.size < PAGE_SIZE || base + size > region.size) + return -EINVAL; + + if (!(region.type & VFIO_DPRC_REGION_CACHEABLE)) + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + + vma->vm_pgoff = (region.addr >> PAGE_SHIFT) + pgoff; + + return remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, + size, vma->vm_page_prot); +} + static int vfio_fsl_mc_mmap(void *device_data, struct vm_area_struct *vma) { - return -EINVAL; + struct vfio_fsl_mc_device *vdev = device_data; + struct fsl_mc_device *mc_dev = vdev->mc_dev; + unsigned long size, addr; + int index; + + index = vma->vm_pgoff >> (VFIO_FSL_MC_OFFSET_SHIFT - PAGE_SHIFT); + + if (vma->vm_end < vma->vm_start) + return -EINVAL; + if (vma->vm_start & ~PAGE_MASK) + return -EINVAL; + if (vma->vm_end & ~PAGE_MASK) + return -EINVAL; + if (!(vma->vm_flags & VM_SHARED)) + return -EINVAL; + if (index >= vdev->num_regions) + return -EINVAL; + + if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_MMAP)) + return -EINVAL; + + if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_READ) + && (vma->vm_flags & VM_READ)) + return -EINVAL; + + if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_WRITE) + && (vma->vm_flags & VM_WRITE)) + return -EINVAL; + + addr = vdev->regions[index].addr; + size = vdev->regions[index].size; + + vma->vm_private_data = mc_dev; + + return vfio_fsl_mc_mmap_mmio(vdev->regions[index], vma); } static int vfio_fsl_mc_init_device(struct vfio_fsl_mc_device *vdev) diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h index 764e2cf2c70d..e2a0ccdd8242 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h @@ -15,6 +15,8 @@ #define VFIO_FSL_MC_INDEX_TO_OFFSET(index) \ ((u64)(index) << VFIO_FSL_MC_OFFSET_SHIFT) +#define VFIO_DPRC_REGION_CACHEABLE 0x00000001 + struct vfio_fsl_mc_region { u32 flags; u32 type; From patchwork Mon Mar 23 17:19:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diana Madalina Craciun X-Patchwork-Id: 11453601 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1FB9F1668 for ; Mon, 23 Mar 2020 17:20:53 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F21502072D for ; Mon, 23 Mar 2020 17:20:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ZjBnn0oq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F21502072D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oss.nxp.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ylZrd/VXULj3GivtAWOj73SBvODZ6k0Ek7pPPrRe3RE=; b=ZjBnn0oqH9XF0msyJq9Dw3Qo9X n/Z1EQ4Ts15NDCvaI2sAm4N6S0uF60nGg2gZbvigw/b6W4MtT5LRqmSj6mVkrI0wyNzcfHh2qejur 6zG91mAypDJBE8C0bRZbRAGm6DwhcHQQAIjrMWJA2MR+LLTyoIlKWZXWEt3g+4vcRDmDw6q5EJn3s R+b6wXml4yIoxwOL42t4tUpWEUyeJR0QeW2uc++R/h4xGkKUWL8Vfr7mQpGFQ8JOUI9ZumZ5Cs/xe nvREV5tkxE/o4Tjr7Uncupe0qKoNNj7ZiOgUwc4yj7CvQQ6946W/2/DjnWxoqVsk3yWMuiFCo3WlW UDs4Ef0A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQks-0006Ar-CV; Mon, 23 Mar 2020 17:20:50 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQjb-0003nE-J3 for linux-arm-kernel@lists.infradead.org; Mon, 23 Mar 2020 17:19:34 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 66C341A14C6; Mon, 23 Mar 2020 18:19:30 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 5A7FA1A14A9; Mon, 23 Mar 2020 18:19:30 +0100 (CET) Received: from fsr-ub1864-111.ea.freescale.net (fsr-ub1864-111.ea.freescale.net [10.171.82.141]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id E64212035C; Mon, 23 Mar 2020 18:19:29 +0100 (CET) From: Diana Craciun To: kvm@vger.kernel.org, alex.williamson@redhat.com, laurentiu.tudor@nxp.com, linux-arm-kernel@lists.infradead.org, bharatb.yadav@gmail.com Subject: [PATCH 6/9] vfio/fsl-mc: Added lock support in preparation for interrupt handling Date: Mon, 23 Mar 2020 19:19:08 +0200 Message-Id: <20200323171911.27178-7-diana.craciun@oss.nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200323171911.27178-1-diana.craciun@oss.nxp.com> References: <20200323171911.27178-1-diana.craciun@oss.nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200323_101931_905067_21C22E56 X-CRM114-Status: GOOD ( 13.14 ) X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [92.121.34.13 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Diana Craciun , linux-kernel@vger.kernel.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org From: Diana Craciun Only the DPRC object allocates interrupts from the MSI interrupt domain. The interrupts are managed by the DPRC in a pool of interrupts. The access to this pool of interrupts has to be protected with a lock. This patch extends the current lock implementation to have a lock per DPRC. Signed-off-by: Diana Craciun --- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 90 +++++++++++++++++++++-- drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 7 +- 2 files changed, 90 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index 6625b7cb0a3e..ea5e81e7791c 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -15,6 +15,76 @@ #include "vfio_fsl_mc_private.h" +static DEFINE_MUTEX(reflck_lock); + +static void vfio_fsl_mc_reflck_get(struct vfio_fsl_mc_reflck *reflck) +{ + kref_get(&reflck->kref); +} + +static void vfio_fsl_mc_reflck_release(struct kref *kref) +{ + struct vfio_fsl_mc_reflck *reflck = container_of(kref, + struct vfio_fsl_mc_reflck, + kref); + + kfree(reflck); + mutex_unlock(&reflck_lock); +} + +static void vfio_fsl_mc_reflck_put(struct vfio_fsl_mc_reflck *reflck) +{ + kref_put_mutex(&reflck->kref, vfio_fsl_mc_reflck_release, &reflck_lock); +} + +static struct vfio_fsl_mc_reflck *vfio_fsl_mc_reflck_alloc(void) +{ + struct vfio_fsl_mc_reflck *reflck; + + reflck = kzalloc(sizeof(*reflck), GFP_KERNEL); + if (!reflck) + return ERR_PTR(-ENOMEM); + + kref_init(&reflck->kref); + mutex_init(&reflck->lock); + + return reflck; +} + +static int vfio_fsl_mc_reflck_attach(struct vfio_fsl_mc_device *vdev) +{ + int ret = 0; + + mutex_lock(&reflck_lock); + if (is_fsl_mc_bus_dprc(vdev->mc_dev)) { + vdev->reflck = vfio_fsl_mc_reflck_alloc(); + } else { + struct device *mc_cont_dev = vdev->mc_dev->dev.parent; + struct vfio_device *device; + struct vfio_fsl_mc_device *cont_vdev; + + device = vfio_device_get_from_dev(mc_cont_dev); + if (!device) { + ret = -ENODEV; + goto unlock; + } + + cont_vdev = vfio_device_data(device); + if (!cont_vdev->reflck) { + vfio_device_put(device); + ret = -ENODEV; + goto unlock; + } + vfio_fsl_mc_reflck_get(cont_vdev->reflck); + vdev->reflck = cont_vdev->reflck; + vfio_device_put(device); + } + +unlock: + mutex_unlock(&reflck_lock); + return ret; +} + static int vfio_fsl_mc_regions_init(struct vfio_fsl_mc_device *vdev) { struct fsl_mc_device *mc_dev = vdev->mc_dev; @@ -56,7 +126,7 @@ static int vfio_fsl_mc_open(void *device_data) if (!try_module_get(THIS_MODULE)) return -ENODEV; - mutex_lock(&vdev->driver_lock); + mutex_lock(&vdev->reflck->lock); if (!vdev->refcnt) { ret = vfio_fsl_mc_regions_init(vdev); if (ret) @@ -64,11 +134,11 @@ static int vfio_fsl_mc_open(void *device_data) } vdev->refcnt++; - mutex_unlock(&vdev->driver_lock); + mutex_unlock(&vdev->reflck->lock); return 0; err_reg_init: - mutex_unlock(&vdev->driver_lock); + mutex_unlock(&vdev->reflck->lock); module_put(THIS_MODULE); return ret; } @@ -77,12 +147,12 @@ static void vfio_fsl_mc_release(void *device_data) { struct vfio_fsl_mc_device *vdev = device_data; - mutex_lock(&vdev->driver_lock); + mutex_lock(&vdev->reflck->lock); if (!(--vdev->refcnt)) vfio_fsl_mc_regions_cleanup(vdev); - mutex_unlock(&vdev->driver_lock); + mutex_unlock(&vdev->reflck->lock); module_put(THIS_MODULE); } @@ -294,12 +364,18 @@ static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev) return ret; } + ret = vfio_fsl_mc_reflck_attach(vdev); + if (ret) { + vfio_iommu_group_put(group, dev); + return ret; + } + ret = vfio_fsl_mc_init_device(vdev); if (ret) { + vfio_fsl_mc_reflck_put(vdev->reflck); vfio_iommu_group_put(group, dev); return ret; } - mutex_init(&vdev->driver_lock); return ret; } @@ -340,6 +416,8 @@ static int vfio_fsl_mc_remove(struct fsl_mc_device *mc_dev) if (!vdev) return -EINVAL; + vfio_fsl_mc_reflck_put(vdev->reflck); + if (is_fsl_mc_bus_dprc(mc_dev)) vfio_fsl_mc_cleanup_dprc(vdev->mc_dev); diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h index e2a0ccdd8242..d072cccd93e0 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h @@ -17,6 +17,11 @@ #define VFIO_DPRC_REGION_CACHEABLE 0x00000001 +struct vfio_fsl_mc_reflck { + struct kref kref; + struct mutex lock; +}; + struct vfio_fsl_mc_region { u32 flags; u32 type; @@ -29,7 +34,7 @@ struct vfio_fsl_mc_device { int refcnt; u32 num_regions; struct vfio_fsl_mc_region *regions; - struct mutex driver_lock; + struct vfio_fsl_mc_reflck *reflck; }; #endif /* VFIO_PCI_PRIVATE_H */ From patchwork Mon Mar 23 17:19:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diana Madalina Craciun X-Patchwork-Id: 11453603 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 568AF1668 for ; Mon, 23 Mar 2020 17:21:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 353462072E for ; Mon, 23 Mar 2020 17:21:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="WSRMmt/w" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 353462072E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oss.nxp.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=jSReOwtYb4b+v6DWMSvJgse01/e01OLE2LbEL2+IEFE=; b=WSRMmt/wb2LxAhAdaIOqyJfrZw 5nziA/ge1J7ZcooWG5uA+kwGmaJAwpAuRfEi0zqEv93lrXppdGB6JKaUiuQUvdNvtQLQ/HoLpXEPg gdrjTup8FnilA9vy7ICBBm6VTGJ++sBYu081fPXi8270QdfQejZZTiLD35a7+31uyKzXAciF1TNy0 SM2r9I999bXMHgh+cRfeF7B649clC0zY2GtDGUBgxIEne1UyOgHkhW2/TzLhfx/9Lw/S3ldsbYM1i snZmBzJqr+Xmpbs8X2gpQdDbioabFy7O5pE1Om2+k7WOEFYLbKT0ziBIOKR1eFgD4aoc6TD6wkEEp ymd2BZ7A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQlB-0006Ra-00; Mon, 23 Mar 2020 17:21:09 +0000 Received: from inva021.nxp.com ([92.121.34.21]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQjc-0003nQ-4p for linux-arm-kernel@lists.infradead.org; Mon, 23 Mar 2020 17:19:34 +0000 Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id E955B20184D; Mon, 23 Mar 2020 18:19:30 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id DB0EB2012E7; Mon, 23 Mar 2020 18:19:30 +0100 (CET) Received: from fsr-ub1864-111.ea.freescale.net (fsr-ub1864-111.ea.freescale.net [10.171.82.141]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id 6E1212035C; Mon, 23 Mar 2020 18:19:30 +0100 (CET) From: Diana Craciun To: kvm@vger.kernel.org, alex.williamson@redhat.com, laurentiu.tudor@nxp.com, linux-arm-kernel@lists.infradead.org, bharatb.yadav@gmail.com Subject: [PATCH 7/9] vfio/fsl-mc: Add irq infrastructure for fsl-mc devices Date: Mon, 23 Mar 2020 19:19:09 +0200 Message-Id: <20200323171911.27178-8-diana.craciun@oss.nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200323171911.27178-1-diana.craciun@oss.nxp.com> References: <20200323171911.27178-1-diana.craciun@oss.nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200323_101932_466561_A0634989 X-CRM114-Status: GOOD ( 15.15 ) X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [92.121.34.21 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bharat Bhushan , linux-kernel@vger.kernel.org, Diana Craciun MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org This patch adds the skeleton for interrupt support for fsl-mc devices. The interrupts are not yet functional, the functionality will be added by subsequent patches. Signed-off-by: Bharat Bhushan Signed-off-by: Diana Craciun --- drivers/vfio/fsl-mc/Makefile | 6 +- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 71 ++++++++++++++++++++++- drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c | 63 ++++++++++++++++++++ drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 5 ++ 4 files changed, 141 insertions(+), 4 deletions(-) create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c diff --git a/drivers/vfio/fsl-mc/Makefile b/drivers/vfio/fsl-mc/Makefile index 6f2b80645d5b..cad6dbf0b735 100644 --- a/drivers/vfio/fsl-mc/Makefile +++ b/drivers/vfio/fsl-mc/Makefile @@ -1,2 +1,4 @@ -vfio-fsl_mc-y := vfio_fsl_mc.o -obj-$(CONFIG_VFIO_FSL_MC) += vfio_fsl_mc.o +# SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause) + +vfio-fsl-mc-y := vfio_fsl_mc.o vfio_fsl_mc_intr.o +obj-$(CONFIG_VFIO_FSL_MC) += vfio-fsl-mc.o diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index ea5e81e7791c..4d7baee2e474 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -210,11 +210,75 @@ static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd, } case VFIO_DEVICE_GET_IRQ_INFO: { - return -EINVAL; + struct vfio_irq_info info; + + minsz = offsetofend(struct vfio_irq_info, count); + if (copy_from_user(&info, (void __user *)arg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + if (info.index >= mc_dev->obj_desc.irq_count) + return -EINVAL; + + info.flags = VFIO_IRQ_INFO_EVENTFD; + info.count = 1; + + return copy_to_user((void __user *)arg, &info, minsz); } case VFIO_DEVICE_SET_IRQS: { - return -EINVAL; + struct vfio_irq_set hdr; + u8 *data = NULL; + int ret = 0; + + minsz = offsetofend(struct vfio_irq_set, count); + + if (copy_from_user(&hdr, (void __user *)arg, minsz)) + return -EFAULT; + + if (hdr.argsz < minsz) + return -EINVAL; + + if (hdr.index >= mc_dev->obj_desc.irq_count) + return -EINVAL; + + if (hdr.start != 0 || hdr.count > 1) + return -EINVAL; + + if (hdr.count == 0 && + (!(hdr.flags & VFIO_IRQ_SET_DATA_NONE) || + !(hdr.flags & VFIO_IRQ_SET_ACTION_TRIGGER))) + return -EINVAL; + + if (hdr.flags & ~(VFIO_IRQ_SET_DATA_TYPE_MASK | + VFIO_IRQ_SET_ACTION_TYPE_MASK)) + return -EINVAL; + + if (!(hdr.flags & VFIO_IRQ_SET_DATA_NONE)) { + size_t size; + + if (hdr.flags & VFIO_IRQ_SET_DATA_BOOL) + size = sizeof(uint8_t); + else if (hdr.flags & VFIO_IRQ_SET_DATA_EVENTFD) + size = sizeof(int32_t); + else + return -EINVAL; + + if (hdr.argsz - minsz < hdr.count * size) + return -EINVAL; + + data = memdup_user((void __user *)(arg + minsz), + hdr.count * size); + if (IS_ERR(data)) + return PTR_ERR(data); + } + + ret = vfio_fsl_mc_set_irqs_ioctl(vdev, hdr.flags, + hdr.index, hdr.start, + hdr.count, data); + return ret; } case VFIO_DEVICE_RESET: { @@ -303,6 +367,9 @@ static int vfio_fsl_mc_init_device(struct vfio_fsl_mc_device *vdev) struct fsl_mc_device *mc_dev = vdev->mc_dev; int ret = 0; + /* innherit the msi domain from parent */ + dev_set_msi_domain(&mc_dev->dev, dev_get_msi_domain(mc_dev->dev.parent)); + /* Non-dprc devices share mc_io from parent */ if (!is_fsl_mc_bus_dprc(mc_dev)) { struct fsl_mc_device *mc_cont = to_fsl_mc_device(mc_dev->dev.parent); diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c b/drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c new file mode 100644 index 000000000000..058aa97aa54a --- /dev/null +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause) +/* + * Copyright 2013-2016 Freescale Semiconductor Inc. + * Copyright 2019 NXP + */ + +#include +#include +#include +#include +#include + +#include "linux/fsl/mc.h" +#include "vfio_fsl_mc_private.h" + +static int vfio_fsl_mc_irq_mask(struct vfio_fsl_mc_device *vdev, + unsigned int index, unsigned int start, + unsigned int count, u32 flags, + void *data) +{ + return -EINVAL; +} + +static int vfio_fsl_mc_irq_unmask(struct vfio_fsl_mc_device *vdev, + unsigned int index, unsigned int start, + unsigned int count, u32 flags, + void *data) +{ + return -EINVAL; +} + +static int vfio_fsl_mc_set_irq_trigger(struct vfio_fsl_mc_device *vdev, + unsigned int index, unsigned int start, + unsigned int count, u32 flags, + void *data) +{ + return -EINVAL; +} + +int vfio_fsl_mc_set_irqs_ioctl(struct vfio_fsl_mc_device *vdev, + u32 flags, unsigned int index, + unsigned int start, unsigned int count, + void *data) +{ + int ret = -ENOTTY; + + switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { + case VFIO_IRQ_SET_ACTION_MASK: + ret = vfio_fsl_mc_irq_mask(vdev, index, start, count, + flags, data); + break; + case VFIO_IRQ_SET_ACTION_UNMASK: + ret = vfio_fsl_mc_irq_unmask(vdev, index, start, count, + flags, data); + break; + case VFIO_IRQ_SET_ACTION_TRIGGER: + ret = vfio_fsl_mc_set_irq_trigger(vdev, index, start, + count, flags, data); + break; + } + + return ret; +} diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h index d072cccd93e0..0c7506e43880 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h @@ -37,4 +37,9 @@ struct vfio_fsl_mc_device { struct vfio_fsl_mc_reflck *reflck; }; +extern int vfio_fsl_mc_set_irqs_ioctl(struct vfio_fsl_mc_device *vdev, + u32 flags, unsigned int index, + unsigned int start, unsigned int count, + void *data); + #endif /* VFIO_PCI_PRIVATE_H */ From patchwork Mon Mar 23 17:19:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diana Madalina Craciun X-Patchwork-Id: 11453605 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4BCF139A for ; Mon, 23 Mar 2020 17:21:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C34382072D for ; Mon, 23 Mar 2020 17:21:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="oiDezxVJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C34382072D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oss.nxp.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ynf0qgMDxFgLrqFUIOBvpunL8hTSinnElOiKgHpXae8=; b=oiDezxVJ9dvy4NDgYo03Uiscm6 J9lUJHRE05n0QjdtLhUnsPxoTL9SW3yBeHK9qUe1QHpt8sNgxXMuPYOhfPXpU+7c8T3YAx5RC69rR ZoSViOFu3I9xB3KfDYpL1N0Gmg4dgedHhiHOUYMUpKlhCcAyA7VKkfO6z/5O0To2QxsmzGojCZyWB sgVJYpDRmdQhyjdkrRT9AxwyKcFVU/bYGVgTP5HlfeV/+DqbaKGWyENef+pIsRFdo+0+C+QhOv1Rg AVzfIwHJyoYH0cj4ZNapsqvFu8VT0ktY4GLKfvHAWzp94em52Y2pBEhRT0WfOhtRLhp14S8Yzid/8 kvQtUFUQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQlK-0006d8-Kw; Mon, 23 Mar 2020 17:21:18 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQjc-0003oK-Km for linux-arm-kernel@lists.infradead.org; Mon, 23 Mar 2020 17:19:37 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 759CA1A14D6; Mon, 23 Mar 2020 18:19:31 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 68E081A14C9; Mon, 23 Mar 2020 18:19:31 +0100 (CET) Received: from fsr-ub1864-111.ea.freescale.net (fsr-ub1864-111.ea.freescale.net [10.171.82.141]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id EE0862035C; Mon, 23 Mar 2020 18:19:30 +0100 (CET) From: Diana Craciun To: kvm@vger.kernel.org, alex.williamson@redhat.com, laurentiu.tudor@nxp.com, linux-arm-kernel@lists.infradead.org, bharatb.yadav@gmail.com Subject: [PATCH 8/9] vfio/fsl-mc: trigger an interrupt via eventfd Date: Mon, 23 Mar 2020 19:19:10 +0200 Message-Id: <20200323171911.27178-9-diana.craciun@oss.nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200323171911.27178-1-diana.craciun@oss.nxp.com> References: <20200323171911.27178-1-diana.craciun@oss.nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200323_101932_979821_A3B9C778 X-CRM114-Status: GOOD ( 17.86 ) X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [92.121.34.13 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bharat Bhushan , linux-kernel@vger.kernel.org, Diana Craciun MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org This patch allows to set an eventfd for fsl-mc device interrupts and also to trigger the interrupt eventfd from userspace for testing. All fsl-mc device interrupts are MSIs. The MSIs are allocated from the MSI domain only once per DPRC and used by all the DPAA2 objects. The interrupts are managed by the DPRC in a pool of interrupts. Each device requests interrupts from this pool. The pool is allocated when the first virtual device is setting the interrupts. The pool of interrupts is protected by a lock. The DPRC has an interrupt of its own which indicates if the DPRC contents have changed. However, currently, the contents of a DPRC assigned to the guest cannot be changed at runtime, so this interrupt is not configured. Signed-off-by: Bharat Bhushan Signed-off-by: Diana Craciun --- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 17 ++- drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c | 160 +++++++++++++++++++++- drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 10 ++ 3 files changed, 185 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index 4d7baee2e474..ceb9d6b06624 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -146,12 +146,27 @@ static int vfio_fsl_mc_open(void *device_data) static void vfio_fsl_mc_release(void *device_data) { struct vfio_fsl_mc_device *vdev = device_data; + int ret; mutex_lock(&vdev->reflck->lock); - if (!(--vdev->refcnt)) + if (!(--vdev->refcnt)) { + struct fsl_mc_device *mc_dev = vdev->mc_dev; + struct device *cont_dev = fsl_mc_cont_dev(&mc_dev->dev); + struct fsl_mc_device *mc_cont = to_fsl_mc_device(cont_dev); + vfio_fsl_mc_regions_cleanup(vdev); + /* reset the device before cleaning up the interrupts */ + ret = dprc_reset_container(mc_dev->mc_io, 0, + mc_dev->mc_handle, + mc_dev->obj_desc.id); + + vfio_fsl_mc_irqs_cleanup(vdev); + + fsl_mc_cleanup_irq_pool(mc_cont); + } + mutex_unlock(&vdev->reflck->lock); module_put(THIS_MODULE); diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c b/drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c index 058aa97aa54a..409f3507fcf3 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c @@ -29,12 +29,149 @@ static int vfio_fsl_mc_irq_unmask(struct vfio_fsl_mc_device *vdev, return -EINVAL; } +int vfio_fsl_mc_irqs_allocate(struct vfio_fsl_mc_device *vdev) +{ + struct fsl_mc_device *mc_dev = vdev->mc_dev; + struct vfio_fsl_mc_irq *mc_irq; + int irq_count; + int ret, i; + + /* Device does not support any interrupt */ + if (mc_dev->obj_desc.irq_count == 0) + return 0; + + /* interrupts were already allocated for this device */ + if (vdev->mc_irqs) + return 0; + + irq_count = mc_dev->obj_desc.irq_count; + + mc_irq = kcalloc(irq_count, sizeof(*mc_irq), GFP_KERNEL); + if (!mc_irq) + return -ENOMEM; + + /* Allocate IRQs */ + ret = fsl_mc_allocate_irqs(mc_dev); + if (ret) { + kfree(mc_irq); + return ret; + } + + for (i = 0; i < irq_count; i++) { + mc_irq[i].count = 1; + mc_irq[i].flags = VFIO_IRQ_INFO_EVENTFD; + } + + vdev->mc_irqs = mc_irq; + + return 0; +} + +static irqreturn_t vfio_fsl_mc_irq_handler(int irq_num, void *arg) +{ + struct vfio_fsl_mc_irq *mc_irq = (struct vfio_fsl_mc_irq *)arg; + + eventfd_signal(mc_irq->trigger, 1); + return IRQ_HANDLED; +} + +static int vfio_set_trigger(struct vfio_fsl_mc_device *vdev, + int index, int fd) +{ + struct vfio_fsl_mc_irq *irq = &vdev->mc_irqs[index]; + struct eventfd_ctx *trigger; + int hwirq; + int ret; + + hwirq = vdev->mc_dev->irqs[index]->msi_desc->irq; + if (irq->trigger) { + free_irq(hwirq, irq); + kfree(irq->name); + eventfd_ctx_put(irq->trigger); + irq->trigger = NULL; + } + + if (fd < 0) /* Disable only */ + return 0; + + irq->name = kasprintf(GFP_KERNEL, "vfio-irq[%d](%s)", + hwirq, dev_name(&vdev->mc_dev->dev)); + if (!irq->name) + return -ENOMEM; + + trigger = eventfd_ctx_fdget(fd); + if (IS_ERR(trigger)) { + kfree(irq->name); + return PTR_ERR(trigger); + } + + irq->trigger = trigger; + + ret = request_irq(hwirq, vfio_fsl_mc_irq_handler, 0, + irq->name, irq); + if (ret) { + kfree(irq->name); + eventfd_ctx_put(trigger); + irq->trigger = NULL; + return ret; + } + + return 0; +} + static int vfio_fsl_mc_set_irq_trigger(struct vfio_fsl_mc_device *vdev, unsigned int index, unsigned int start, unsigned int count, u32 flags, void *data) { - return -EINVAL; + struct fsl_mc_device *mc_dev = vdev->mc_dev; + int ret, hwirq; + struct vfio_fsl_mc_irq *irq; + struct device *cont_dev = fsl_mc_cont_dev(&mc_dev->dev); + struct fsl_mc_device *mc_cont = to_fsl_mc_device(cont_dev); + + if (start != 0 || count != 1) + return -EINVAL; + + mutex_lock(&vdev->reflck->lock); + ret = fsl_mc_populate_irq_pool(mc_cont, + FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS); + if (ret) + goto unlock; + + ret = vfio_fsl_mc_irqs_allocate(vdev); + if (ret) + goto unlock; + mutex_unlock(&vdev->reflck->lock); + + if (!count && (flags & VFIO_IRQ_SET_DATA_NONE)) + return vfio_set_trigger(vdev, index, -1); + + if (flags & VFIO_IRQ_SET_DATA_EVENTFD) { + s32 fd = *(s32 *)data; + + return vfio_set_trigger(vdev, index, fd); + } + + hwirq = vdev->mc_dev->irqs[index]->msi_desc->irq; + + irq = &vdev->mc_irqs[index]; + + if (flags & VFIO_IRQ_SET_DATA_NONE) { + vfio_fsl_mc_irq_handler(hwirq, irq); + + } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { + u8 trigger = *(u8 *)data; + + if (trigger) + vfio_fsl_mc_irq_handler(hwirq, irq); + } + + return 0; + +unlock: + mutex_unlock(&vdev->reflck->lock); + return ret; } int vfio_fsl_mc_set_irqs_ioctl(struct vfio_fsl_mc_device *vdev, @@ -61,3 +198,24 @@ int vfio_fsl_mc_set_irqs_ioctl(struct vfio_fsl_mc_device *vdev, return ret; } + +/* Free All IRQs for the given MC object */ +void vfio_fsl_mc_irqs_cleanup(struct vfio_fsl_mc_device *vdev) +{ + struct fsl_mc_device *mc_dev = vdev->mc_dev; + int irq_count = mc_dev->obj_desc.irq_count; + int i; + + /* Device does not support any interrupt or the interrupts + * were not configured + */ + if (mc_dev->obj_desc.irq_count == 0 || !vdev->mc_irqs) + return; + + for (i = 0; i < irq_count; i++) + vfio_set_trigger(vdev, i, -1); + + fsl_mc_free_irqs(mc_dev); + kfree(vdev->mc_irqs); + vdev->mc_irqs = NULL; +} diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h index 0c7506e43880..cac0b205c3d4 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h @@ -17,6 +17,13 @@ #define VFIO_DPRC_REGION_CACHEABLE 0x00000001 +struct vfio_fsl_mc_irq { + u32 flags; + u32 count; + struct eventfd_ctx *trigger; + char *name; +}; + struct vfio_fsl_mc_reflck { struct kref kref; struct mutex lock; @@ -35,6 +42,7 @@ struct vfio_fsl_mc_device { u32 num_regions; struct vfio_fsl_mc_region *regions; struct vfio_fsl_mc_reflck *reflck; + struct vfio_fsl_mc_irq *mc_irqs; }; extern int vfio_fsl_mc_set_irqs_ioctl(struct vfio_fsl_mc_device *vdev, @@ -42,4 +50,6 @@ extern int vfio_fsl_mc_set_irqs_ioctl(struct vfio_fsl_mc_device *vdev, unsigned int start, unsigned int count, void *data); +void vfio_fsl_mc_irqs_cleanup(struct vfio_fsl_mc_device *vdev); + #endif /* VFIO_PCI_PRIVATE_H */ From patchwork Mon Mar 23 17:19:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diana Madalina Craciun X-Patchwork-Id: 11453607 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 140CE139A for ; Mon, 23 Mar 2020 17:21:40 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CCEA320722 for ; Mon, 23 Mar 2020 17:21:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="tNLiiCIm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CCEA320722 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oss.nxp.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=cTZmTmMbylRy+aAsjDDldGcDvMEUBs4N1h6Hy6qPGCU=; b=tNLiiCImg8z7eBwErLcIA4x0zo snglgBzBfb6CS3+mIONW4Z4Swmla918vkYgednqqX1dMBPsNry5V0byz5CsUp3z6mGIGk8waV2dGn P6lLtvJcDHPQvOOwUk2SDQp9khIgICg0B/ngADXDKNnq+CzPEdhDv0bwFtI7Bni9dt5Pq1nt+6A22 fTtsrNL2+eoOFtVHa+Zj3udt6m+5yxjtpPBtcKrZm8BdX3/vd4vgzX46fRWC/OngaX/dXBGZgd9ts eh7I7FtnoPl0JMXfHSJLPT0/m0YKhaorfMuFwdmkm7+CbmINhoUZPmNgACiwCxU7F9RXObxN2xkdd Vu/LWTAw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQla-0006uh-5J; Mon, 23 Mar 2020 17:21:34 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jGQjd-0003pO-OM for linux-arm-kernel@lists.infradead.org; Mon, 23 Mar 2020 17:19:37 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 0A0CE1A18A0; Mon, 23 Mar 2020 18:19:32 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id F189B1A14C9; Mon, 23 Mar 2020 18:19:31 +0100 (CET) Received: from fsr-ub1864-111.ea.freescale.net (fsr-ub1864-111.ea.freescale.net [10.171.82.141]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id 7E2742035C; Mon, 23 Mar 2020 18:19:31 +0100 (CET) From: Diana Craciun To: kvm@vger.kernel.org, alex.williamson@redhat.com, laurentiu.tudor@nxp.com, linux-arm-kernel@lists.infradead.org, bharatb.yadav@gmail.com Subject: [PATCH 9/9] vfio/fsl-mc: Add read/write support for fsl-mc devices Date: Mon, 23 Mar 2020 19:19:11 +0200 Message-Id: <20200323171911.27178-10-diana.craciun@oss.nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200323171911.27178-1-diana.craciun@oss.nxp.com> References: <20200323171911.27178-1-diana.craciun@oss.nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200323_101934_069775_FE4D2182 X-CRM114-Status: GOOD ( 16.46 ) X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [92.121.34.13 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Diana Craciun , Bharat Bhushan , linux-kernel@vger.kernel.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org From: Diana Craciun The software uses a memory-mapped I/O command interface (MC portals) to communicate with the MC hardware. This command interface is used to discover, enumerate, configure and remove DPAA2 objects. The DPAA2 objects use MSIs, so the command interface needs to be emulated such that the correct MSI is configured in the hardware (the guest has the virtual MSIs). This patch is adding read/write support for fsl-mc devices. The mc commands are emulated by the userspace. The host is just passing the correct command to the hardware. Also the current patch limits userspace to write complete 64byte command once and read 64byte response by one ioctl. Signed-off-by: Bharat Bhushan Signed-off-by: Diana Craciun --- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 122 +++++++++++++++++++++- drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 1 + 2 files changed, 121 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index ceb9d6b06624..107b4ab7b2d8 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -12,6 +12,7 @@ #include #include #include +#include #include "vfio_fsl_mc_private.h" @@ -114,6 +115,11 @@ static int vfio_fsl_mc_regions_init(struct vfio_fsl_mc_device *vdev) static void vfio_fsl_mc_regions_cleanup(struct vfio_fsl_mc_device *vdev) { + int i; + + for (i = 0; i < vdev->num_regions; i++) + iounmap(vdev->regions[i].ioaddr); + vdev->num_regions = 0; kfree(vdev->regions); } @@ -307,13 +313,125 @@ static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd, static ssize_t vfio_fsl_mc_read(void *device_data, char __user *buf, size_t count, loff_t *ppos) { - return -EINVAL; + struct vfio_fsl_mc_device *vdev = device_data; + unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos); + loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK; + struct vfio_fsl_mc_region *region; + u64 data[8]; + int i; + + /* Read ioctl supported only for DPRC and DPMCP device */ + if (strcmp(vdev->mc_dev->obj_desc.type, "dprc") && + strcmp(vdev->mc_dev->obj_desc.type, "dpmcp")) + return -EINVAL; + + if (index >= vdev->num_regions) + return -EINVAL; + + region = &vdev->regions[index]; + + if (!(region->flags & VFIO_REGION_INFO_FLAG_READ)) + return -EINVAL; + + if (!region->ioaddr) { + region->ioaddr = ioremap(region->addr, region->size); + if (!region->ioaddr) + return -ENOMEM; + } + + if (count != 64 || off != 0) + return -EINVAL; + + for (i = 7; i >= 0; i--) + data[i] = readq(region->ioaddr + i * sizeof(uint64_t)); + + if (copy_to_user(buf, data, 64)) + return -EFAULT; + + return count; +} + +#define MC_CMD_COMPLETION_TIMEOUT_MS 5000 +#define MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS 500 + +static int vfio_fsl_mc_send_command(void __iomem *ioaddr, uint64_t *cmd_data) +{ + int i; + enum mc_cmd_status status; + unsigned long timeout_usecs = MC_CMD_COMPLETION_TIMEOUT_MS * 1000; + + /* Write at command parameter into portal */ + for (i = 7; i >= 1; i--) + writeq_relaxed(cmd_data[i], ioaddr + i * sizeof(uint64_t)); + + /* Write command header in the end */ + writeq(cmd_data[0], ioaddr); + + /* Wait for response before returning to user-space + * This can be optimized in future to even prepare response + * before returning to user-space and avoid read ioctl. + */ + for (;;) { + u64 header; + struct mc_cmd_header *resp_hdr; + + header = cpu_to_le64(readq_relaxed(ioaddr)); + + resp_hdr = (struct mc_cmd_header *)&header; + status = (enum mc_cmd_status)resp_hdr->status; + if (status != MC_CMD_STATUS_READY) + break; + + udelay(MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS); + timeout_usecs -= MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS; + if (timeout_usecs == 0) + return -ETIMEDOUT; + } + + return 0; } static ssize_t vfio_fsl_mc_write(void *device_data, const char __user *buf, size_t count, loff_t *ppos) { - return -EINVAL; + struct vfio_fsl_mc_device *vdev = device_data; + unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos); + loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK; + struct vfio_fsl_mc_region *region; + u64 data[8]; + int ret; + + /* Write ioctl supported only for DPRC and DPMCP device */ + if (strcmp(vdev->mc_dev->obj_desc.type, "dprc") && + strcmp(vdev->mc_dev->obj_desc.type, "dpmcp")) + return -EINVAL; + + if (index >= vdev->num_regions) + return -EINVAL; + + region = &vdev->regions[index]; + + if (!(region->flags & VFIO_REGION_INFO_FLAG_WRITE)) + return -EINVAL; + + if (!region->ioaddr) { + region->ioaddr = ioremap(region->addr, region->size); + if (!region->ioaddr) + return -ENOMEM; + } + + if (count != 64 || off != 0) + return -EINVAL; + + if (copy_from_user(&data, buf, 64)) + return -EFAULT; + + ret = vfio_fsl_mc_send_command(region->ioaddr, data); + if (ret) + return ret; + + return count; + } static int vfio_fsl_mc_mmap_mmio(struct vfio_fsl_mc_region region, diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h index cac0b205c3d4..70824c4b10d6 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h @@ -34,6 +34,7 @@ struct vfio_fsl_mc_region { u32 type; u64 addr; resource_size_t size; + void __iomem *ioaddr; }; struct vfio_fsl_mc_device {