From patchwork Tue Sep 8 07:13:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yilun X-Patchwork-Id: 11762659 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6C08359D for ; Tue, 8 Sep 2020 07:18:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5773D218AC for ; Tue, 8 Sep 2020 07:18:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729278AbgIHHSE (ORCPT ); Tue, 8 Sep 2020 03:18:04 -0400 Received: from mga09.intel.com ([134.134.136.24]:30573 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729246AbgIHHSC (ORCPT ); Tue, 8 Sep 2020 03:18:02 -0400 IronPort-SDR: RV9utV7ksloTlCOhj9ft8L0rDC/J2HXyziceEBZM6Jkxsu3HNdTJRoTXIrOi4SemC2O/cYYhzm xaQgmZxtwasA== X-IronPort-AV: E=McAfee;i="6000,8403,9737"; a="159058737" X-IronPort-AV: E=Sophos;i="5.76,404,1592895600"; d="scan'208";a="159058737" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Sep 2020 00:18:01 -0700 IronPort-SDR: 16kxSz8FcibQjPAMjNL4fMfj/Uu0qMWU1yqlgh+aBu3zVs17c5oJ3mqCj+w2OBdwPmPu9UnxmK aM6CJrdBkFIw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,404,1592895600"; d="scan'208";a="448677709" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.141]) by orsmga004.jf.intel.com with ESMTP; 08 Sep 2020 00:17:59 -0700 From: Xu Yilun To: mdf@kernel.org, alex.williamson@redhat.com, kwankhede@nvidia.com, linux-fpga@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: trix@redhat.com, lgoncalv@redhat.com, yilun.xu@intel.com Subject: [PATCH 1/3] fpga: dfl: add driver_override support Date: Tue, 8 Sep 2020 15:13:30 +0800 Message-Id: <1599549212-24253-2-git-send-email-yilun.xu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1599549212-24253-1-git-send-email-yilun.xu@intel.com> References: <1599549212-24253-1-git-send-email-yilun.xu@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add support for overriding the default matching of a dfl device to a dfl driver. It follows the same way that can be used for PCI and platform devices. This patch adds the 'driver_override' sysfs file. It can be used by VFIO to bind dfl devices for user accessing. Signed-off-by: Xu Yilun --- Documentation/ABI/testing/sysfs-bus-dfl | 20 ++++++++++++ drivers/fpga/dfl.c | 54 ++++++++++++++++++++++++++++++++- include/linux/fpga/dfl-bus.h | 2 ++ 3 files changed, 75 insertions(+), 1 deletion(-) diff --git a/Documentation/ABI/testing/sysfs-bus-dfl b/Documentation/ABI/testing/sysfs-bus-dfl index 23543be..cf1357a 100644 --- a/Documentation/ABI/testing/sysfs-bus-dfl +++ b/Documentation/ABI/testing/sysfs-bus-dfl @@ -13,3 +13,23 @@ Contact: Xu Yilun Description: Read-only. It returns feature identifier local to its DFL FIU type. Format: 0x%x + +What: /sys/bus/dfl/devices/.../driver_override +Date: Sep 2020 +KernelVersion: 5.10 +Contact: Xu Yilun +Description: This file allows the driver for a device to be specified. When + specified, only a driver with a name matching the value written + to driver_override will have an opportunity to bind to the + device. The override is specified by writing a string to the + driver_override file (echo vfio-mdev-dfl > driver_override) and + may be cleared with an empty string (echo > driver_override). + This returns the device to standard matching rules binding. + Writing to driver_override does not automatically unbind the + device from its current driver or make any attempt to + automatically load the specified driver. If no driver with a + matching name is currently loaded in the kernel, the device + will not bind to any driver. This also allows devices to + opt-out of driver binding using a driver_override name such as + "none". Only a single driver may be specified in the override, + there is no support for parsing delimiters. diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c index 02a6780..98f88ee 100644 --- a/drivers/fpga/dfl.c +++ b/drivers/fpga/dfl.c @@ -262,6 +262,10 @@ static int dfl_bus_match(struct device *dev, struct device_driver *drv) struct dfl_driver *ddrv = to_dfl_drv(drv); const struct dfl_device_id *id_entry; + /* When driver_override is set, only bind to the matching driver */ + if (ddev->driver_override) + return !strcmp(ddev->driver_override, drv->name); + id_entry = ddrv->id_table; if (id_entry) { while (id_entry->feature_id) { @@ -304,6 +308,53 @@ static int dfl_bus_uevent(struct device *dev, struct kobj_uevent_env *env) ddev->type, ddev->feature_id); } +static ssize_t driver_override_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct dfl_device *ddev = to_dfl_dev(dev); + ssize_t len; + + device_lock(dev); + len = sprintf(buf, "%s\n", ddev->driver_override); + device_unlock(dev); + return len; +} + +static ssize_t driver_override_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct dfl_device *ddev = to_dfl_dev(dev); + char *driver_override, *old, *cp; + + /* We need to keep extra room for a newline */ + if (count >= (PAGE_SIZE - 1)) + return -EINVAL; + + driver_override = kstrndup(buf, count, GFP_KERNEL); + if (!driver_override) + return -ENOMEM; + + cp = strchr(driver_override, '\n'); + if (cp) + *cp = '\0'; + + device_lock(dev); + old = ddev->driver_override; + if (strlen(driver_override)) { + ddev->driver_override = driver_override; + } else { + kfree(driver_override); + ddev->driver_override = NULL; + } + device_unlock(dev); + + kfree(old); + + return count; +} +static DEVICE_ATTR_RW(driver_override); + static ssize_t type_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -325,6 +376,7 @@ static DEVICE_ATTR_RO(feature_id); static struct attribute *dfl_dev_attrs[] = { &dev_attr_type.attr, &dev_attr_feature_id.attr, + &dev_attr_driver_override.attr, NULL, }; ATTRIBUTE_GROUPS(dfl_dev); @@ -470,7 +522,7 @@ static int dfl_devs_add(struct dfl_feature_platform_data *pdata) int __dfl_driver_register(struct dfl_driver *dfl_drv, struct module *owner) { - if (!dfl_drv || !dfl_drv->probe || !dfl_drv->id_table) + if (!dfl_drv || !dfl_drv->probe) return -EINVAL; dfl_drv->drv.owner = owner; diff --git a/include/linux/fpga/dfl-bus.h b/include/linux/fpga/dfl-bus.h index 2a2b283..09a9ee1 100644 --- a/include/linux/fpga/dfl-bus.h +++ b/include/linux/fpga/dfl-bus.h @@ -43,6 +43,8 @@ struct dfl_device { unsigned int num_irqs; struct dfl_fpga_cdev *cdev; const struct dfl_device_id *id_entry; + + char *driver_override; }; /** From patchwork Tue Sep 8 07:13:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yilun X-Patchwork-Id: 11762663 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E82D59D for ; Tue, 8 Sep 2020 07:18:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2B8C021D20 for ; Tue, 8 Sep 2020 07:18:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729305AbgIHHSV (ORCPT ); Tue, 8 Sep 2020 03:18:21 -0400 Received: from mga09.intel.com ([134.134.136.24]:30573 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729295AbgIHHSK (ORCPT ); Tue, 8 Sep 2020 03:18:10 -0400 IronPort-SDR: 7s9ALvXixq2J3i33Jd1hKZHmU66gfdgvDtiWlXjpGhQNoFbvsrUd73C9hAIqOoOFtEnPI6Mdub yflSmVWD0kgg== X-IronPort-AV: E=McAfee;i="6000,8403,9737"; a="159058773" X-IronPort-AV: E=Sophos;i="5.76,404,1592895600"; d="scan'208";a="159058773" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Sep 2020 00:18:06 -0700 IronPort-SDR: ZndmxuCdVVQ2yvIAJKZS5QIjK5fY1BBWim7eczhXA6Mtpcen8h/fWe8Z1NlX+SECXzONNhH+sq tQuiMdlfrr8Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,404,1592895600"; d="scan'208";a="448677740" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.141]) by orsmga004.jf.intel.com with ESMTP; 08 Sep 2020 00:18:03 -0700 From: Xu Yilun To: mdf@kernel.org, alex.williamson@redhat.com, kwankhede@nvidia.com, linux-fpga@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: trix@redhat.com, lgoncalv@redhat.com, yilun.xu@intel.com Subject: [PATCH 2/3] fpga: dfl: VFIO mdev support for DFL devices Date: Tue, 8 Sep 2020 15:13:31 +0800 Message-Id: <1599549212-24253-3-git-send-email-yilun.xu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1599549212-24253-1-git-send-email-yilun.xu@intel.com> References: <1599549212-24253-1-git-send-email-yilun.xu@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch is to provide VFIO support to DFL devices. When bound this driver to a dfl device, it registers a set of callbacks to the mediated device framework to enable mediated device creation. When a mediated device is created by user via mdev interfaces, it will be exposed to user as a VFIO platform device. Signed-off-by: Xu Yilun --- drivers/fpga/Kconfig | 9 + drivers/fpga/Makefile | 1 + drivers/fpga/vfio-mdev-dfl.c | 391 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 401 insertions(+) create mode 100644 drivers/fpga/vfio-mdev-dfl.c diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig index 38c7130..88f64fb 100644 --- a/drivers/fpga/Kconfig +++ b/drivers/fpga/Kconfig @@ -202,6 +202,15 @@ config FPGA_DFL_NIOS_INTEL_PAC_N3000 the card. It also instantiates the SPI master (spi-altera) for the card's BMC (Board Management Controller) control. +config FPGA_VFIO_MDEV_DFL + tristate "VFIO Mdev support for DFL devices" + depends on FPGA_DFL && VFIO_MDEV + help + Support for DFL devices with VFIO Mdev. This is required to make use + of DFL devices present on the system using the VFIO Mdev framework. + + If you don't know what to do here, say N. + config FPGA_DFL_PCI tristate "FPGA DFL PCIe Device Driver" depends on PCI && FPGA_DFL diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile index 18dc9885..c69bfc9 100644 --- a/drivers/fpga/Makefile +++ b/drivers/fpga/Makefile @@ -45,6 +45,7 @@ dfl-afu-objs := dfl-afu-main.o dfl-afu-region.o dfl-afu-dma-region.o dfl-afu-objs += dfl-afu-error.o obj-$(CONFIG_FPGA_DFL_NIOS_INTEL_PAC_N3000) += dfl-n3000-nios.o +obj-$(CONFIG_FPGA_VFIO_MDEV_DFL) += vfio-mdev-dfl.o # Drivers for FPGAs which implement DFL obj-$(CONFIG_FPGA_DFL_PCI) += dfl-pci.o diff --git a/drivers/fpga/vfio-mdev-dfl.c b/drivers/fpga/vfio-mdev-dfl.c new file mode 100644 index 0000000..34fb19f --- /dev/null +++ b/drivers/fpga/vfio-mdev-dfl.c @@ -0,0 +1,391 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * VFIO Mediated device driver for DFL devices + * + * Copyright (C) 2019-2020 Intel Corporation, Inc. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct vfio_mdev_dfl_dev { + struct device *dev; + void __iomem *ioaddr; + resource_size_t phys; + resource_size_t memsize; + int num_irqs; + u32 region_flags; + atomic_t avail; +}; + +static ssize_t +name_show(struct kobject *kobj, struct device *dev, char *buf) +{ + return sprintf(buf, "%s-type1\n", dev_name(dev)); +} +static MDEV_TYPE_ATTR_RO(name); + +static ssize_t +available_instances_show(struct kobject *kobj, struct device *dev, char *buf) +{ + struct vfio_mdev_dfl_dev *vmdd = dev_get_drvdata(dev); + + return sprintf(buf, "%d\n", atomic_read(&vmdd->avail)); +} +static MDEV_TYPE_ATTR_RO(available_instances); + +static ssize_t device_api_show(struct kobject *kobj, struct device *dev, + char *buf) +{ + return sprintf(buf, "%s\n", VFIO_DEVICE_API_PLATFORM_STRING); +} +static MDEV_TYPE_ATTR_RO(device_api); + +static struct attribute *mdev_types_attrs[] = { + &mdev_type_attr_name.attr, + &mdev_type_attr_device_api.attr, + &mdev_type_attr_available_instances.attr, + NULL, +}; + +static struct attribute_group dfl_mdev_type_group1 = { + .name = "1", + .attrs = mdev_types_attrs, +}; + +static struct attribute_group *dfl_mdev_type_groups[] = { + &dfl_mdev_type_group1, + NULL, +}; + +static int dfl_mdev_create(struct kobject *kobj, struct mdev_device *mdev) +{ + struct vfio_mdev_dfl_dev *vmdd = + dev_get_drvdata(mdev_parent_dev(mdev)); + + if (atomic_dec_if_positive(&vmdd->avail) < 0) + return -EPERM; + + return 0; +} + +static int dfl_mdev_remove(struct mdev_device *mdev) +{ + struct vfio_mdev_dfl_dev *vmdd = + dev_get_drvdata(mdev_parent_dev(mdev)); + + atomic_inc(&vmdd->avail); + + return 0; +} + +static ssize_t dfl_mdev_read(struct mdev_device *mdev, char __user *buf, + size_t count, loff_t *ppos) +{ + struct vfio_mdev_dfl_dev *vmdd = + dev_get_drvdata(mdev_parent_dev(mdev)); + unsigned int done = 0; + loff_t off = *ppos; + + if (off + count > vmdd->memsize) + return -EFAULT; + + while (count) { + size_t filled; + + if (count >= 8 && !(off % 8)) { + u64 val; + + val = ioread64(vmdd->ioaddr + off); + if (copy_to_user(buf, &val, 8)) + goto err; + + filled = 8; + } else if (count >= 4 && !(off % 4)) { + u32 val; + + val = ioread32(vmdd->ioaddr + off); + if (copy_to_user(buf, &val, 4)) + goto err; + + filled = 4; + } else if (count >= 2 && !(off % 2)) { + u16 val; + + val = ioread16(vmdd->ioaddr + off); + if (copy_to_user(buf, &val, 2)) + goto err; + + filled = 2; + } else { + u8 val; + + val = ioread8(vmdd->ioaddr + off); + if (copy_to_user(buf, &val, 1)) + goto err; + + filled = 1; + } + + count -= filled; + done += filled; + off += filled; + buf += filled; + } + + return done; +err: + return -EFAULT; +} + +static ssize_t dfl_mdev_write(struct mdev_device *mdev, const char __user *buf, + size_t count, loff_t *ppos) +{ + struct vfio_mdev_dfl_dev *vmdd = + dev_get_drvdata(mdev_parent_dev(mdev)); + unsigned int done = 0; + loff_t off = *ppos; + + if (off + count > vmdd->memsize) + return -EFAULT; + + while (count) { + size_t filled; + + if (count >= 8 && !(off % 8)) { + u64 val; + + if (copy_from_user(&val, buf, 8)) + goto err; + iowrite64(val, vmdd->ioaddr + off); + + filled = 8; + } else if (count >= 4 && !(off % 4)) { + u32 val; + + if (copy_from_user(&val, buf, 4)) + goto err; + iowrite32(val, vmdd->ioaddr + off); + + filled = 4; + } else if (count >= 2 && !(off % 2)) { + u16 val; + + if (copy_from_user(&val, buf, 2)) + goto err; + iowrite16(val, vmdd->ioaddr + off); + + filled = 2; + } else { + u8 val; + + if (copy_from_user(&val, buf, 1)) + goto err; + iowrite8(val, vmdd->ioaddr + off); + + filled = 1; + } + + count -= filled; + done += filled; + off += filled; + buf += filled; + } + + return done; +err: + return -EFAULT; +} + +static long dfl_mdev_ioctl(struct mdev_device *mdev, unsigned int cmd, + unsigned long arg) +{ + struct vfio_mdev_dfl_dev *vmdd = + dev_get_drvdata(mdev_parent_dev(mdev)); + unsigned long minsz; + + switch (cmd) { + case VFIO_DEVICE_GET_INFO: + { + struct vfio_device_info info; + + minsz = offsetofend(struct vfio_device_info, num_irqs); + + if (copy_from_user(&info, (void __user *)arg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + info.flags = VFIO_DEVICE_FLAGS_PLATFORM; + info.num_regions = 1; + info.num_irqs = vmdd->num_irqs; + + return copy_to_user((void __user *)arg, &info, minsz) ? + -EFAULT : 0; + } + case VFIO_DEVICE_GET_REGION_INFO: + { + struct vfio_region_info info; + + minsz = offsetofend(struct vfio_region_info, offset); + + if (copy_from_user(&info, (void __user *)arg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + if (info.index >= 1) + return -EINVAL; + + info.offset = 0; + info.size = vmdd->memsize; + info.flags = vmdd->region_flags; + + return copy_to_user((void __user *)arg, &info, minsz) ? + -EFAULT : 0; + } + } + + return -ENOTTY; +} + +static int dfl_mdev_mmap_mmio(struct vfio_mdev_dfl_dev *vmdd, + struct vm_area_struct *vma) +{ + u64 req_len, req_start; + + req_len = vma->vm_end - vma->vm_start; + req_start = vma->vm_pgoff << PAGE_SHIFT; + + if (vmdd->memsize < PAGE_SIZE || req_start + req_len > vmdd->memsize) + return -EINVAL; + + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + + return remap_pfn_range(vma, vma->vm_start, + (vmdd->phys + req_start) >> PAGE_SHIFT, + req_len, vma->vm_page_prot); +} + +static int dfl_mdev_mmap(struct mdev_device *mdev, struct vm_area_struct *vma) +{ + struct vfio_mdev_dfl_dev *vmdd = + dev_get_drvdata(mdev_parent_dev(mdev)); + + if (vma->vm_end < vma->vm_start) + return -EINVAL; + if (!(vma->vm_flags & VM_SHARED)) + return -EINVAL; + if (vma->vm_start & ~PAGE_MASK) + return -EINVAL; + if (vma->vm_end & ~PAGE_MASK) + return -EINVAL; + + if (!(vmdd->region_flags & VFIO_REGION_INFO_FLAG_MMAP)) + return -EINVAL; + + if (!(vmdd->region_flags & VFIO_REGION_INFO_FLAG_READ) && + (vma->vm_flags & VM_READ)) + return -EINVAL; + + if (!(vmdd->region_flags & VFIO_REGION_INFO_FLAG_WRITE) && + (vma->vm_flags & VM_WRITE)) + return -EINVAL; + + vma->vm_private_data = vmdd; + + return dfl_mdev_mmap_mmio(vmdd, vma); +} + +static int dfl_mdev_open(struct mdev_device *mdev) +{ + if (!try_module_get(THIS_MODULE)) + return -ENODEV; + + return 0; +} + +static void dfl_mdev_close(struct mdev_device *mdev) +{ + module_put(THIS_MODULE); +} + +static const struct mdev_parent_ops dfl_mdev_ops = { + .owner = THIS_MODULE, + .supported_type_groups = dfl_mdev_type_groups, + .create = dfl_mdev_create, + .remove = dfl_mdev_remove, + .open = dfl_mdev_open, + .release = dfl_mdev_close, + .read = dfl_mdev_read, + .write = dfl_mdev_write, + .ioctl = dfl_mdev_ioctl, + .mmap = dfl_mdev_mmap, +}; + +static int vfio_mdev_dfl_probe(struct dfl_device *dfl_dev) +{ + struct device *dev = &dfl_dev->dev; + struct vfio_mdev_dfl_dev *vmdd; + + vmdd = devm_kzalloc(dev, sizeof(*vmdd), GFP_KERNEL); + if (!vmdd) + return -ENOMEM; + + dev_set_drvdata(&dfl_dev->dev, vmdd); + + atomic_set(&vmdd->avail, 1); + vmdd->dev = &dfl_dev->dev; + vmdd->phys = dfl_dev->mmio_res.start; + vmdd->memsize = resource_size(&dfl_dev->mmio_res); + vmdd->region_flags = VFIO_REGION_INFO_FLAG_READ | + VFIO_REGION_INFO_FLAG_WRITE; + /* + * Only regions addressed with PAGE granularity may be MMAPed + * securely. + */ + if (!(vmdd->phys & ~PAGE_MASK) && !(vmdd->memsize & ~PAGE_MASK)) + vmdd->region_flags |= VFIO_REGION_INFO_FLAG_MMAP; + + vmdd->ioaddr = devm_ioremap_resource(&dfl_dev->dev, &dfl_dev->mmio_res); + if (IS_ERR(vmdd->ioaddr)) { + dev_err(dev, "get mem resource fail!\n"); + return PTR_ERR(vmdd->ioaddr); + } + + /* irq not supported yet */ + vmdd->num_irqs = 0; + + return mdev_register_device(&dfl_dev->dev, &dfl_mdev_ops); +} + +static void vfio_mdev_dfl_remove(struct dfl_device *dfl_dev) +{ + mdev_unregister_device(&dfl_dev->dev); +} + +static struct dfl_driver vfio_mdev_dfl_driver = { + .drv = { + .name = "vfio-mdev-dfl", + }, + .probe = vfio_mdev_dfl_probe, + .remove = vfio_mdev_dfl_remove, +}; + +module_dfl_driver(vfio_mdev_dfl_driver); + +MODULE_DESCRIPTION("VFIO MDEV DFL driver"); +MODULE_AUTHOR("Intel Corporation"); +MODULE_LICENSE("GPL v2"); From patchwork Tue Sep 8 07:13:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yilun X-Patchwork-Id: 11762667 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D309D746 for ; Tue, 8 Sep 2020 07:19:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BC67721D43 for ; Tue, 8 Sep 2020 07:19:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729356AbgIHHSY (ORCPT ); Tue, 8 Sep 2020 03:18:24 -0400 Received: from mga09.intel.com ([134.134.136.24]:30606 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729319AbgIHHSP (ORCPT ); Tue, 8 Sep 2020 03:18:15 -0400 IronPort-SDR: +5hr4C9uP+aTZfLMkLPgtox2Zb0RtjaKTJYl6v5jMfcCuspBCCqZm/uI7RYVthehR00zyeMa4q xGX/98AprNCQ== X-IronPort-AV: E=McAfee;i="6000,8403,9737"; a="159058785" X-IronPort-AV: E=Sophos;i="5.76,404,1592895600"; d="scan'208";a="159058785" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Sep 2020 00:18:09 -0700 IronPort-SDR: 8JSwMJ6Tjug/3Tsn0DNYD/WMK5ydlgPjxoYidbQBmcwW4Uf7eMF9cw/PgMI4uR/ao3qqcX18Id CskrK1evAVwQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,404,1592895600"; d="scan'208";a="448677754" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.141]) by orsmga004.jf.intel.com with ESMTP; 08 Sep 2020 00:18:07 -0700 From: Xu Yilun To: mdf@kernel.org, alex.williamson@redhat.com, kwankhede@nvidia.com, linux-fpga@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: trix@redhat.com, lgoncalv@redhat.com, yilun.xu@intel.com, Matthew Gerlach Subject: [PATCH 3/3] Documentation: fpga: dfl: Add description for VFIO Mdev support Date: Tue, 8 Sep 2020 15:13:32 +0800 Message-Id: <1599549212-24253-4-git-send-email-yilun.xu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1599549212-24253-1-git-send-email-yilun.xu@intel.com> References: <1599549212-24253-1-git-send-email-yilun.xu@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch adds description for VFIO Mdev support for dfl devices on dfl bus. Signed-off-by: Xu Yilun Signed-off-by: Matthew Gerlach --- Documentation/fpga/dfl.rst | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/Documentation/fpga/dfl.rst b/Documentation/fpga/dfl.rst index 0404fe6..f077754 100644 --- a/Documentation/fpga/dfl.rst +++ b/Documentation/fpga/dfl.rst @@ -502,6 +502,26 @@ FME Partial Reconfiguration Sub Feature driver (see drivers/fpga/dfl-fme-pr.c) could be a reference. +VFIO Mdev support for DFL devices +================================= +As we introduced a dfl bus for private features, they could be added to dfl bus +as independent dfl devices. There is a requirement to handle these devices +either by kernel drivers or by direct access from userspace. Usually we bind +the kernel drivers to devices which provide board management functions, and +gives user direct access to devices which cooperate closely with user +controlled Accelerated Function Unit (AFU). We realize this with a VFIO Mdev +implementation. When we bind the vfio-mdev-dfl driver to a dfl device, it +realizes a group of callbacks and registers to the Mdev framework as a +parent (physical) device. It could then create one (available_instances == 1) +mdev device. +Since dfl devices are sub devices of FPGA DFL physical devices (e.g. PCIE +device), which provide no DMA isolation for each sub device, this may leads to +DMA isolation problem if a private feature is designed to be capable of DMA. +The AFU user could potentially access the whole device addressing space and +impact the private feature. So now the general HW design rule is, no DMA +capability for private features. It eliminates the DMA isolation problem. + + Open discussion =============== FME driver exports one ioctl (DFL_FPGA_FME_PORT_PR) for partial reconfiguration