From patchwork Mon Feb 24 08:46:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 11399537 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EA4E138D for ; Mon, 24 Feb 2020 08:56:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 38A1224650 for ; Mon, 24 Feb 2020 08:56:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727459AbgBXI4J (ORCPT ); Mon, 24 Feb 2020 03:56:09 -0500 Received: from mga12.intel.com ([192.55.52.136]:40096 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726925AbgBXI4I (ORCPT ); Mon, 24 Feb 2020 03:56:08 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Feb 2020 00:56:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,479,1574150400"; d="scan'208";a="231068681" Received: from joy-optiplex-7040.sh.intel.com ([10.239.13.16]) by fmsmga008.fm.intel.com with ESMTP; 24 Feb 2020 00:56:06 -0800 From: Yan Zhao To: alex.williamson@redhat.com Cc: zhenyuw@linux.intel.com, intel-gvt-dev@lists.freedesktop.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, kevin.tian@intel.com, peterx@redhat.com, Yan Zhao Subject: [PATCH v3 1/7] vfio: allow external user to get vfio group from device Date: Mon, 24 Feb 2020 03:46:41 -0500 Message-Id: <20200224084641.31696-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200224084350.31574-1-yan.y.zhao@intel.com> References: <20200224084350.31574-1-yan.y.zhao@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org external user is able to 1. add a device into an vfio group 2. call vfio_group_get_external_user_from_dev() with the device pointer to get vfio_group associated with this device and increments the container user counter to prevent the VFIO group from disposal before KVM exits. 3. When the external KVM finishes, it calls vfio_group_put_external_user() to release the VFIO group. Suggested-by: Alex Williamson Signed-off-by: Yan Zhao --- drivers/vfio/vfio.c | 37 +++++++++++++++++++++++++++++++++++++ include/linux/vfio.h | 2 ++ 2 files changed, 39 insertions(+) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index c8482624ca34..914bdf4b9d73 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1720,6 +1720,43 @@ struct vfio_group *vfio_group_get_external_user(struct file *filep) } EXPORT_SYMBOL_GPL(vfio_group_get_external_user); +/** + * External user API, exported by symbols to be linked dynamically. + * + * The protocol includes: + * 1. External user add a device into a vfio group + * + * 2. The external user calls vfio_group_get_external_user_from_dev() + * with the device pointer + * to verify that: + * - there's a vfio group associated with it and is initialized; + * - IOMMU is set for the vfio group. + * If both checks passed, vfio_group_get_external_user_from_dev() + * increments the container user counter to prevent + * the VFIO group from disposal before KVM exits. + * + * 3. When the external KVM finishes, it calls + * vfio_group_put_external_user() to release the VFIO group. + * This call decrements the container user counter. + */ + +struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev) +{ + struct vfio_group *group; + int ret; + + group = vfio_group_get_from_dev(dev); + if (!group) + return ERR_PTR(-ENODEV); + + ret = vfio_group_add_container_user(group); + if (ret) + return ERR_PTR(ret); + + return group; +} +EXPORT_SYMBOL_GPL(vfio_group_get_external_user_from_dev); + void vfio_group_put_external_user(struct vfio_group *group) { vfio_group_try_dissolve_container(group); diff --git a/include/linux/vfio.h b/include/linux/vfio.h index e42a711a2800..2e1fa0c7396f 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -94,6 +94,8 @@ extern void vfio_unregister_iommu_driver( */ extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); +extern +struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev); extern bool vfio_external_group_match_file(struct vfio_group *group, struct file *filep); extern int vfio_external_user_iommu_id(struct vfio_group *group); From patchwork Mon Feb 24 08:47:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 11399545 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6628817D5 for ; Mon, 24 Feb 2020 08:56:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4ED412192A for ; Mon, 24 Feb 2020 08:56:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727290AbgBXI4l (ORCPT ); Mon, 24 Feb 2020 03:56:41 -0500 Received: from mga03.intel.com ([134.134.136.65]:55914 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727235AbgBXI4l (ORCPT ); Mon, 24 Feb 2020 03:56:41 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Feb 2020 00:56:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,479,1574150400"; d="scan'208";a="231068759" Received: from joy-optiplex-7040.sh.intel.com ([10.239.13.16]) by fmsmga008.fm.intel.com with ESMTP; 24 Feb 2020 00:56:38 -0800 From: Yan Zhao To: alex.williamson@redhat.com Cc: zhenyuw@linux.intel.com, intel-gvt-dev@lists.freedesktop.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, kevin.tian@intel.com, peterx@redhat.com, Yan Zhao Subject: [PATCH v3 2/7] vfio: introduce vfio_dma_rw to read/write a range of IOVAs Date: Mon, 24 Feb 2020 03:47:15 -0500 Message-Id: <20200224084715.31753-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200224084350.31574-1-yan.y.zhao@intel.com> References: <20200224084350.31574-1-yan.y.zhao@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org vfio_dma_rw will read/write a range of user space memory pointed to by IOVA into/from a kernel buffer without enforcing pinning the user space memory. TODO: mark the IOVAs to user space memory dirty if they are written in vfio_dma_rw(). Cc: Kevin Tian Signed-off-by: Yan Zhao --- drivers/vfio/vfio.c | 49 +++++++++++++++++++++ drivers/vfio/vfio_iommu_type1.c | 77 +++++++++++++++++++++++++++++++++ include/linux/vfio.h | 5 +++ 3 files changed, 131 insertions(+) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 914bdf4b9d73..902867627cbf 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1998,6 +1998,55 @@ int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, int npage) } EXPORT_SYMBOL(vfio_unpin_pages); + +/* + * This interface allows the CPUs to perform some sort of virtual DMA on + * behalf of the device. + * + * CPUs read/write a range of IOVAs pointing to user space memory into/from + * a kernel buffer. + * + * As the read/write of user space memory is conducted via the CPUs and is + * not a real device DMA, it is not necessary to pin the user space memory. + * + * The caller needs to call vfio_group_get_external_user() or + * vfio_group_get_external_user_from_dev() prior to calling this interface, + * so as to prevent the VFIO group from disposal in the middle of the call. + * But it can keep the reference to the VFIO group for several calls into + * this interface. + * After finishing using of the VFIO group, the caller needs to release the + * VFIO group by calling vfio_group_put_external_user(). + * + * @group [in]: vfio group of a device + * @iova [in] : base IOVA of a user space buffer + * @data [in] : pointer to kernel buffer + * @len [in] : kernel buffer length + * @write : indicate read or write + * Return error code on failure or 0 on success. + */ +int vfio_dma_rw(struct vfio_group *group, dma_addr_t iova, + void *data, size_t len, bool write) +{ + struct vfio_container *container; + struct vfio_iommu_driver *driver; + int ret = 0; + + if (!group || !data || len <= 0) + return -EINVAL; + + container = group->container; + driver = container->iommu_driver; + + if (likely(driver && driver->ops->dma_rw)) + ret = driver->ops->dma_rw(container->iommu_data, + iova, data, len, write); + else + ret = -ENOTTY; + + return ret; +} +EXPORT_SYMBOL(vfio_dma_rw); + static int vfio_register_iommu_notifier(struct vfio_group *group, unsigned long *events, struct notifier_block *nb) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 2ada8e6cdb88..74e1c425943c 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -2326,6 +2327,81 @@ static int vfio_iommu_type1_unregister_notifier(void *iommu_data, return blocking_notifier_chain_unregister(&iommu->notifier, nb); } +static size_t vfio_iommu_type1_dma_rw_chunk(struct vfio_iommu *iommu, + dma_addr_t iova, void *data, + size_t count, bool write) +{ + struct mm_struct *mm; + unsigned long vaddr; + struct vfio_dma *dma; + bool kthread = current->mm == NULL; + size_t done = 0; + size_t offset; + + dma = vfio_find_dma(iommu, iova, 1); + if (!dma) + return 0; + + if ((write && !(dma->prot & IOMMU_WRITE)) || + !(dma->prot & IOMMU_READ)) + return 0; + + mm = get_task_mm(dma->task); + + if (!mm) + return 0; + + if (kthread) + use_mm(mm); + else if (current->mm != mm) + goto out; + + offset = iova - dma->iova; + + if (count > dma->size - offset) + count = dma->size - offset; + + vaddr = dma->vaddr + offset; + + if (write) + done = __copy_to_user((void __user *)vaddr, + data, count) ? 0 : count; + else + done = __copy_from_user(data, (void __user *)vaddr, + count) ? 0 : count; + + if (kthread) + unuse_mm(mm); +out: + mmput(mm); + return done; +} + +static int vfio_iommu_type1_dma_rw(void *iommu_data, dma_addr_t iova, + void *data, size_t count, bool write) +{ + struct vfio_iommu *iommu = iommu_data; + int ret = 0; + size_t done = 0; + + mutex_lock(&iommu->lock); + while (count > 0) { + done = vfio_iommu_type1_dma_rw_chunk(iommu, iova, data, + count, write); + if (!done) { + ret = -EFAULT; + break; + } + + count -= done; + data += done; + iova += done; + } + + mutex_unlock(&iommu->lock); + return ret; +} + static const struct vfio_iommu_driver_ops vfio_iommu_driver_ops_type1 = { .name = "vfio-iommu-type1", .owner = THIS_MODULE, @@ -2338,6 +2414,7 @@ static const struct vfio_iommu_driver_ops vfio_iommu_driver_ops_type1 = { .unpin_pages = vfio_iommu_type1_unpin_pages, .register_notifier = vfio_iommu_type1_register_notifier, .unregister_notifier = vfio_iommu_type1_unregister_notifier, + .dma_rw = vfio_iommu_type1_dma_rw, }; static int __init vfio_iommu_type1_init(void) diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 2e1fa0c7396f..fea0cb1e61d2 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -82,6 +82,8 @@ struct vfio_iommu_driver_ops { struct notifier_block *nb); int (*unregister_notifier)(void *iommu_data, struct notifier_block *nb); + int (*dma_rw)(void *iommu_data, dma_addr_t iova, + void *data, size_t count, bool write); }; extern int vfio_register_iommu_driver(const struct vfio_iommu_driver_ops *ops); @@ -109,6 +111,9 @@ extern int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, extern int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, int npage); +extern int vfio_dma_rw(struct vfio_group *group, dma_addr_t iova, void *data, + size_t len, bool write); + /* each type has independent events */ enum vfio_notify_type { VFIO_IOMMU_NOTIFY = 0, From patchwork Mon Feb 24 08:47:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 11399553 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43C55138D for ; Mon, 24 Feb 2020 08:56:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2425C22525 for ; Mon, 24 Feb 2020 08:56:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727450AbgBXI4y (ORCPT ); Mon, 24 Feb 2020 03:56:54 -0500 Received: from mga07.intel.com ([134.134.136.100]:25344 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726452AbgBXI4x (ORCPT ); Mon, 24 Feb 2020 03:56:53 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Feb 2020 00:56:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,479,1574150400"; d="scan'208";a="231068782" Received: from joy-optiplex-7040.sh.intel.com ([10.239.13.16]) by fmsmga008.fm.intel.com with ESMTP; 24 Feb 2020 00:56:50 -0800 From: Yan Zhao To: alex.williamson@redhat.com Cc: zhenyuw@linux.intel.com, intel-gvt-dev@lists.freedesktop.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, kevin.tian@intel.com, peterx@redhat.com, Yan Zhao Subject: [PATCH v3 3/7] vfio: avoid inefficient lookup of VFIO group in vfio_pin/unpin_pages Date: Mon, 24 Feb 2020 03:47:26 -0500 Message-Id: <20200224084726.31800-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200224084350.31574-1-yan.y.zhao@intel.com> References: <20200224084350.31574-1-yan.y.zhao@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org in vfio_pin_pages() and vfio_unpin_pages(), before calling into iommu driver, vfio group for the device is first searched and gets reference count increased; and after calling into iommu driver, the vfio group is dereference. This searching/ref/deref operations can be combined for several calls by invoking holding reference of the VFIO group prior to use it and not releasing the VFIO group until finishing using it. vfio_pin_pages_from_group() and vfio_unpin_pages_from_group() are introduced to avoid the above inefficient lookup of VFIO group. Suggested-by: Alex Williamson Signed-off-by: Yan Zhao --- drivers/vfio/vfio.c | 89 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/vfio.h | 6 +++ 2 files changed, 95 insertions(+) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 902867627cbf..08d9e8fb33e5 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1998,6 +1998,95 @@ int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, int npage) } EXPORT_SYMBOL(vfio_unpin_pages); +/* + * Pin a set of guest PFNs and return their associated host PFNs for local + * domain only. + * + * The caller needs to call vfio_group_get_external_user() or + * vfio_group_get_external_user_from_dev() prior to calling this interface, + * so as to prevent the VFIO group from disposal in the middle of the call. + * But it can keep the reference to the VFIO group for several calls into + * this interface. + * After finishing using of the VFIO group, the caller needs to release the + * VFIO group by calling vfio_group_put_external_user(). + * + * @group [in] : VFIO group of a device + * @user_pfn [in]: array of user/guest PFNs to be pinned. + * @npage [in] : count of elements in user_pfn array. This count should not + * be greater VFIO_PIN_PAGES_MAX_ENTRIES. + * @prot [in] : protection flags + * @phys_pfn[out]: array of host PFNs + * Return error or number of pages pinned. + */ +int vfio_pin_pages_from_group(struct vfio_group *group, + unsigned long *user_pfn, int npage, + int prot, unsigned long *phys_pfn) +{ + struct vfio_container *container; + struct vfio_iommu_driver *driver; + int ret; + + if (!group || !user_pfn || !phys_pfn || !npage) + return -EINVAL; + + if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) + return -E2BIG; + + container = group->container; + driver = container->iommu_driver; + if (likely(driver && driver->ops->pin_pages)) + ret = driver->ops->pin_pages(container->iommu_data, user_pfn, + npage, prot, phys_pfn); + else + ret = -ENOTTY; + + return ret; +} +EXPORT_SYMBOL(vfio_pin_pages_from_group); + +/* + * Unpin set of host PFNs for local domain only. + * + * The caller needs to call vfio_group_get_external_user() or + * vfio_group_get_external_user_from_dev() prior to calling this interface, + * so as to prevent the VFIO group from disposal in the middle of the call. + * But it can keep the reference to the VFIO group for several calls into + * this interface. + * After finishing using of the VFIO group, the caller needs to release the + * VFIO group by calling vfio_group_put_external_user(). + * + * @group [in] : vfio group of a device + * @user_pfn [in]: array of user/guest PFNs to be unpinned. Number of user/guest + * PFNs should not be greater than VFIO_PIN_PAGES_MAX_ENTRIES. + * @npage [in] : count of elements in user_pfn array. This count should not + * be greater than VFIO_PIN_PAGES_MAX_ENTRIES. + * Return error or number of pages unpinned. + */ +int vfio_unpin_pages_from_group(struct vfio_group *group, + unsigned long *user_pfn, int npage) +{ + struct vfio_container *container; + struct vfio_iommu_driver *driver; + int ret; + + if (!group || !user_pfn || !npage) + return -EINVAL; + + if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) + return -E2BIG; + + container = group->container; + driver = container->iommu_driver; + if (likely(driver && driver->ops->unpin_pages)) + ret = driver->ops->unpin_pages(container->iommu_data, user_pfn, + npage); + else + ret = -ENOTTY; + + return ret; +} +EXPORT_SYMBOL(vfio_unpin_pages_from_group); + /* * This interface allows the CPUs to perform some sort of virtual DMA on diff --git a/include/linux/vfio.h b/include/linux/vfio.h index fea0cb1e61d2..aacf6611c084 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -111,6 +111,12 @@ extern int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, extern int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, int npage); +extern int vfio_pin_pages_from_group(struct vfio_group *group, + unsigned long *user_pfn, int npage, + int prot, unsigned long *phys_pfn); +extern int vfio_unpin_pages_from_group(struct vfio_group *group, + unsigned long *user_pfn, int npage); + extern int vfio_dma_rw(struct vfio_group *group, dma_addr_t iova, void *data, size_t len, bool write); From patchwork Mon Feb 24 08:47:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 11399561 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A5B4F138D for ; Mon, 24 Feb 2020 08:57:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8E20D2465D for ; Mon, 24 Feb 2020 08:57:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727242AbgBXI5W (ORCPT ); Mon, 24 Feb 2020 03:57:22 -0500 Received: from mga17.intel.com ([192.55.52.151]:28743 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727115AbgBXI5W (ORCPT ); Mon, 24 Feb 2020 03:57:22 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Feb 2020 00:57:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,479,1574150400"; d="scan'208";a="231068861" Received: from joy-optiplex-7040.sh.intel.com ([10.239.13.16]) by fmsmga008.fm.intel.com with ESMTP; 24 Feb 2020 00:57:19 -0800 From: Yan Zhao To: zhenyuw@linux.intel.com Cc: alex.williamson@redhat.com, intel-gvt-dev@lists.freedesktop.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, kevin.tian@intel.com, peterx@redhat.com, Yan Zhao Subject: [PATCH v3 4/7] drm/i915/gvt: hold reference of VFIO group during opening of vgpu Date: Mon, 24 Feb 2020 03:47:56 -0500 Message-Id: <20200224084756.31851-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200224084350.31574-1-yan.y.zhao@intel.com> References: <20200224084350.31574-1-yan.y.zhao@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org hold reference count of the VFIO group for each vgpu at vgpu opening and release the reference at vgpu releasing. Signed-off-by: Yan Zhao --- drivers/gpu/drm/i915/gvt/gvt.h | 1 + drivers/gpu/drm/i915/gvt/kvmgt.c | 13 +++++++++++++ 2 files changed, 14 insertions(+) diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h index 0081b051d3e0..5230ac80b84c 100644 --- a/drivers/gpu/drm/i915/gvt/gvt.h +++ b/drivers/gpu/drm/i915/gvt/gvt.h @@ -219,6 +219,7 @@ struct intel_vgpu { struct work_struct release_work; atomic_t released; struct vfio_device *vfio_device; + struct vfio_group *vfio_group; } vdev; #endif diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c index bd79a9718cc7..ed4c79cc3e09 100644 --- a/drivers/gpu/drm/i915/gvt/kvmgt.c +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c @@ -747,6 +747,7 @@ static int intel_vgpu_open(struct mdev_device *mdev) struct intel_vgpu *vgpu = mdev_get_drvdata(mdev); unsigned long events; int ret; + struct vfio_group *vfio_group; vgpu->vdev.iommu_notifier.notifier_call = intel_vgpu_iommu_notifier; vgpu->vdev.group_notifier.notifier_call = intel_vgpu_group_notifier; @@ -769,6 +770,14 @@ static int intel_vgpu_open(struct mdev_device *mdev) goto undo_iommu; } + vfio_group = vfio_group_get_external_user_from_dev(mdev_dev(mdev)); + if (IS_ERR_OR_NULL(vfio_group)) { + ret = !vfio_group ? -EFAULT : PTR_ERR(vfio_group); + gvt_vgpu_err("vfio_group_get_external_user_from_dev failed\n"); + goto undo_register_group; + } + vgpu->vdev.vfio_group = vfio_group; + /* Take a module reference as mdev core doesn't take * a reference for vendor driver. */ @@ -785,6 +794,9 @@ static int intel_vgpu_open(struct mdev_device *mdev) return ret; undo_group: + vfio_group_put_external_user(vgpu->vdev.vfio_group); + +undo_register_group: vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, &vgpu->vdev.group_notifier); @@ -834,6 +846,7 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu) kvmgt_guest_exit(info); intel_vgpu_release_msi_eventfd_ctx(vgpu); + vfio_group_put_external_user(vgpu->vdev.vfio_group); vgpu->vdev.kvm = NULL; vgpu->handle = 0; From patchwork Mon Feb 24 08:48:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 11399565 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B8980159A for ; Mon, 24 Feb 2020 08:57:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9FDE2227BF for ; Mon, 24 Feb 2020 08:57:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727444AbgBXI5c (ORCPT ); Mon, 24 Feb 2020 03:57:32 -0500 Received: from mga06.intel.com ([134.134.136.31]:37373 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727183AbgBXI5c (ORCPT ); Mon, 24 Feb 2020 03:57:32 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Feb 2020 00:57:30 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,479,1574150400"; d="scan'208";a="231068883" Received: from joy-optiplex-7040.sh.intel.com ([10.239.13.16]) by fmsmga008.fm.intel.com with ESMTP; 24 Feb 2020 00:57:28 -0800 From: Yan Zhao To: zhenyuw@linux.intel.com Cc: alex.williamson@redhat.com, intel-gvt-dev@lists.freedesktop.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, kevin.tian@intel.com, peterx@redhat.com, Yan Zhao Subject: [PATCH v3 5/7] drm/i915/gvt: subsitute kvm_read/write_guest with vfio_dma_rw Date: Mon, 24 Feb 2020 03:48:05 -0500 Message-Id: <20200224084805.31898-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200224084350.31574-1-yan.y.zhao@intel.com> References: <20200224084350.31574-1-yan.y.zhao@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org As a device model, it is better to read/write guest memory using vfio interface, so that vfio is able to maintain dirty info of device IOVAs. Cc: Kevin Tian Signed-off-by: Yan Zhao --- drivers/gpu/drm/i915/gvt/kvmgt.c | 22 +--------------------- 1 file changed, 1 insertion(+), 21 deletions(-) diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c index ed4c79cc3e09..3d6362fd94e7 100644 --- a/drivers/gpu/drm/i915/gvt/kvmgt.c +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c @@ -1979,33 +1979,13 @@ static int kvmgt_rw_gpa(unsigned long handle, unsigned long gpa, void *buf, unsigned long len, bool write) { struct kvmgt_guest_info *info; - struct kvm *kvm; - int idx, ret; - bool kthread = current->mm == NULL; if (!handle_valid(handle)) return -ESRCH; info = (struct kvmgt_guest_info *)handle; - kvm = info->kvm; - - if (kthread) { - if (!mmget_not_zero(kvm->mm)) - return -EFAULT; - use_mm(kvm->mm); - } - idx = srcu_read_lock(&kvm->srcu); - ret = write ? kvm_write_guest(kvm, gpa, buf, len) : - kvm_read_guest(kvm, gpa, buf, len); - srcu_read_unlock(&kvm->srcu, idx); - - if (kthread) { - unuse_mm(kvm->mm); - mmput(kvm->mm); - } - - return ret; + return vfio_dma_rw(info->vgpu->vdev.vfio_group, gpa, buf, len, write); } static int kvmgt_read_gpa(unsigned long handle, unsigned long gpa, From patchwork Mon Feb 24 08:48:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 11399569 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF39E138D for ; Mon, 24 Feb 2020 08:57:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C871D2192A for ; Mon, 24 Feb 2020 08:57:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727552AbgBXI5l (ORCPT ); Mon, 24 Feb 2020 03:57:41 -0500 Received: from mga14.intel.com ([192.55.52.115]:57474 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727168AbgBXI5k (ORCPT ); Mon, 24 Feb 2020 03:57:40 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Feb 2020 00:57:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,479,1574150400"; d="scan'208";a="231068899" Received: from joy-optiplex-7040.sh.intel.com ([10.239.13.16]) by fmsmga008.fm.intel.com with ESMTP; 24 Feb 2020 00:57:38 -0800 From: Yan Zhao To: zhenyuw@linux.intel.com Cc: alex.williamson@redhat.com, intel-gvt-dev@lists.freedesktop.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, kevin.tian@intel.com, peterx@redhat.com, Yan Zhao Subject: [PATCH v3 6/7] drm/i915/gvt: avoid unnecessary lookup in each vfio pin & unpin pages Date: Mon, 24 Feb 2020 03:48:14 -0500 Message-Id: <20200224084814.31945-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200224084350.31574-1-yan.y.zhao@intel.com> References: <20200224084350.31574-1-yan.y.zhao@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org substitute vfio_pin_pages() and vfio_unpin_pages() with vfio_pin_pages_from_group() and vfio_unpin_pages_from_group(), so that it will not go through looking up, referencing, dereferencing of VFIO group in each call. Signed-off-by: Yan Zhao --- drivers/gpu/drm/i915/gvt/kvmgt.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c index 3d6362fd94e7..aa7c6f2f1fb8 100644 --- a/drivers/gpu/drm/i915/gvt/kvmgt.c +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c @@ -129,7 +129,8 @@ static void gvt_unpin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn, for (npage = 0; npage < total_pages; npage++) { unsigned long cur_gfn = gfn + npage; - ret = vfio_unpin_pages(mdev_dev(vgpu->vdev.mdev), &cur_gfn, 1); + ret = vfio_unpin_pages_from_group(vgpu->vdev.vfio_group, + &cur_gfn, 1); WARN_ON(ret != 1); } } @@ -152,8 +153,9 @@ static int gvt_pin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn, unsigned long cur_gfn = gfn + npage; unsigned long pfn; - ret = vfio_pin_pages(mdev_dev(vgpu->vdev.mdev), &cur_gfn, 1, - IOMMU_READ | IOMMU_WRITE, &pfn); + ret = vfio_pin_pages_from_group(vgpu->vdev.vfio_group, + &cur_gfn, 1, + IOMMU_READ | IOMMU_WRITE, &pfn); if (ret != 1) { gvt_vgpu_err("vfio_pin_pages failed for gfn 0x%lx, ret %d\n", cur_gfn, ret); From patchwork Mon Feb 24 08:48:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 11399575 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D741314E3 for ; Mon, 24 Feb 2020 08:57:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B66272192A for ; Mon, 24 Feb 2020 08:57:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727295AbgBXI5w (ORCPT ); Mon, 24 Feb 2020 03:57:52 -0500 Received: from mga09.intel.com ([134.134.136.24]:28301 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727170AbgBXI5w (ORCPT ); Mon, 24 Feb 2020 03:57:52 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Feb 2020 00:57:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,479,1574150400"; d="scan'208";a="231068941" Received: from joy-optiplex-7040.sh.intel.com ([10.239.13.16]) by fmsmga008.fm.intel.com with ESMTP; 24 Feb 2020 00:57:47 -0800 From: Yan Zhao To: zhenyuw@linux.intel.com Cc: alex.williamson@redhat.com, intel-gvt-dev@lists.freedesktop.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, kevin.tian@intel.com, peterx@redhat.com, Yan Zhao Subject: [PATCH v3 7/7] drm/i915/gvt: rw more pages a time for shadow context Date: Mon, 24 Feb 2020 03:48:24 -0500 Message-Id: <20200224084824.31992-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200224084350.31574-1-yan.y.zhao@intel.com> References: <20200224084350.31574-1-yan.y.zhao@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org 1. as shadow context is pinned in intel_vgpu_setup_submission() and unpinned in intel_vgpu_clean_submission(), its base virtual address of is safely obtained from lrc_reg_state. no need to call kmap()/kunmap() repeatedly. 2. IOVA(GPA)s of context pages are checked in this patch and if they are continuous, read/write them together in one intel_gvt_hypervisor_read_gpa() /intel_gvt_hypervisor_write_gpa(). after the two changes in this patch, average cycles for populate_shadow_context() and update_guest_context() are reduced by ~10000-20000 cycles, depending on the average continuous pages in each read/write. (1) comparison of cycles of populate_shadow_context() + update_guest_context() when executing different benchmarks ------------------------------------------------------------- | cycles | glmark2 | lightsmark | openarena | |-------------------------------------------------------------| | before this patch | 65968 | 97852 | 61373 | | after this patch | 56017 (85%) | 73862 (75%) | 47463 (77%) | ------------------------------------------------------------- (2) average count of pages read/written a time in populate_shadow_context() and update_guest_context() for each benchmark ----------------------------------------------------------- | page cnt | glmark2 | lightsmark | openarena | |-----------------------------------------------------------| | before this patch | 1 | 1 | 1 | | after this patch | 5.25 | 19.99 | 20 | ------------------------------------------------------------ (3) comparison of benchmarks scores --------------------------------------------------------------------- | score | glmark2 | lightsmark | openarena | |---------------------------------------------------------------------| | before this patch | 1244 | 222.18 | 114.4 | | after this patch | 1248 (100.3%) | 225.8 (101.6%) | 115.0 (100.9%) | --------------------------------------------------------------------- Signed-off-by: Yan Zhao --- drivers/gpu/drm/i915/gvt/scheduler.c | 101 +++++++++++++++++++-------- 1 file changed, 73 insertions(+), 28 deletions(-) diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c b/drivers/gpu/drm/i915/gvt/scheduler.c index 5b2a7d072ec9..c2b521b099bc 100644 --- a/drivers/gpu/drm/i915/gvt/scheduler.c +++ b/drivers/gpu/drm/i915/gvt/scheduler.c @@ -129,16 +129,22 @@ static int populate_shadow_context(struct intel_vgpu_workload *workload) struct intel_vgpu *vgpu = workload->vgpu; struct intel_gvt *gvt = vgpu->gvt; int ring_id = workload->ring_id; - struct drm_i915_gem_object *ctx_obj = - workload->req->hw_context->state->obj; + struct intel_context *ctx_ce = workload->req->hw_context; struct execlist_ring_context *shadow_ring_context; - struct page *page; void *dst; + void *context_base; unsigned long context_gpa, context_page_num; + unsigned long seq_gpa_base; /* first gpa of successive GPAs */ + int seq_page_cnt; /* cnt of GPAs that are successive */ int i; - page = i915_gem_object_get_page(ctx_obj, LRC_STATE_PN); - shadow_ring_context = kmap(page); + GEM_BUG_ON(!intel_context_is_pinned(ctx_ce)); + + context_base = (void *) ctx_ce->lrc_reg_state - + (LRC_STATE_PN << I915_GTT_PAGE_SHIFT); + + shadow_ring_context = context_base + + (LRC_STATE_PN << I915_GTT_PAGE_SHIFT); sr_oa_regs(workload, (u32 *)shadow_ring_context, true); #define COPY_REG(name) \ @@ -170,7 +176,6 @@ static int populate_shadow_context(struct intel_vgpu_workload *workload) I915_GTT_PAGE_SIZE - sizeof(*shadow_ring_context)); sr_oa_regs(workload, (u32 *)shadow_ring_context, false); - kunmap(page); if (IS_RESTORE_INHIBIT(shadow_ring_context->ctx_ctrl.val)) return 0; @@ -185,8 +190,12 @@ static int populate_shadow_context(struct intel_vgpu_workload *workload) if (IS_BROADWELL(gvt->dev_priv) && ring_id == RCS0) context_page_num = 19; - i = 2; - while (i < context_page_num) { + + /* find continuous GPAs from gma until the first discontinuous GPA + * read continuous GPAs into dst virtual address + */ + seq_page_cnt = 0; + for (i = 2; i < context_page_num; i++) { context_gpa = intel_vgpu_gma_to_gpa(vgpu->gtt.ggtt_mm, (u32)((workload->ctx_desc.lrca + i) << I915_GTT_PAGE_SHIFT)); @@ -195,12 +204,26 @@ static int populate_shadow_context(struct intel_vgpu_workload *workload) return -EFAULT; } - page = i915_gem_object_get_page(ctx_obj, i); - dst = kmap(page); - intel_gvt_hypervisor_read_gpa(vgpu, context_gpa, dst, - I915_GTT_PAGE_SIZE); - kunmap(page); - i++; + if (seq_page_cnt == 0) { + seq_gpa_base = context_gpa; + dst = context_base + (i << I915_GTT_PAGE_SHIFT); + } else if (context_gpa != seq_gpa_base + + (seq_page_cnt << I915_GTT_PAGE_SHIFT)) + goto read; + + seq_page_cnt++; + + if (i == context_page_num - 1) + goto read; + + continue; + +read: + intel_gvt_hypervisor_read_gpa(vgpu, seq_gpa_base, dst, + seq_page_cnt << I915_GTT_PAGE_SHIFT); + seq_page_cnt = 1; + seq_gpa_base = context_gpa; + dst = context_base + (i << I915_GTT_PAGE_SHIFT); } return 0; } @@ -787,20 +810,24 @@ static void update_guest_context(struct intel_vgpu_workload *workload) struct i915_request *rq = workload->req; struct intel_vgpu *vgpu = workload->vgpu; struct intel_gvt *gvt = vgpu->gvt; - struct drm_i915_gem_object *ctx_obj = rq->hw_context->state->obj; + struct intel_context *ctx_ce = workload->req->hw_context; struct execlist_ring_context *shadow_ring_context; - struct page *page; - void *src; unsigned long context_gpa, context_page_num; + unsigned long seq_gpa_base; /* first gpa of successive GPAs */ + int seq_page_cnt; /* cnt of GPAs that are successive */ int i; struct drm_i915_private *dev_priv = gvt->dev_priv; u32 ring_base; u32 head, tail; u16 wrap_count; + void *src; + void *context_base; gvt_dbg_sched("ring id %d workload lrca %x\n", rq->engine->id, workload->ctx_desc.lrca); + GEM_BUG_ON(!intel_context_is_pinned(ctx_ce)); + head = workload->rb_head; tail = workload->rb_tail; wrap_count = workload->guest_rb_head >> RB_HEAD_WRAP_CNT_OFF; @@ -824,9 +851,14 @@ static void update_guest_context(struct intel_vgpu_workload *workload) if (IS_BROADWELL(gvt->dev_priv) && rq->engine->id == RCS0) context_page_num = 19; - i = 2; + context_base = (void *) ctx_ce->lrc_reg_state - + LRC_STATE_PN * PAGE_SIZE; - while (i < context_page_num) { + /* find continuous GPAs from gma until the first discontinuous GPA + * write continuous GPAs from src virtual address + */ + seq_page_cnt = 0; + for (i = 2; i < context_page_num; i++) { context_gpa = intel_vgpu_gma_to_gpa(vgpu->gtt.ggtt_mm, (u32)((workload->ctx_desc.lrca + i) << I915_GTT_PAGE_SHIFT)); @@ -835,19 +867,33 @@ static void update_guest_context(struct intel_vgpu_workload *workload) return; } - page = i915_gem_object_get_page(ctx_obj, i); - src = kmap(page); - intel_gvt_hypervisor_write_gpa(vgpu, context_gpa, src, - I915_GTT_PAGE_SIZE); - kunmap(page); - i++; + if (seq_page_cnt == 0) { + seq_gpa_base = context_gpa; + src = context_base + (i << I915_GTT_PAGE_SHIFT); + } else if (context_gpa != seq_gpa_base + + (seq_page_cnt << I915_GTT_PAGE_SHIFT)) + goto write; + + seq_page_cnt++; + + if (i == context_page_num - 1) + goto write; + + continue; + +write: + intel_gvt_hypervisor_write_gpa(vgpu, seq_gpa_base, src, + seq_page_cnt << I915_GTT_PAGE_SHIFT); + seq_page_cnt = 1; + seq_gpa_base = context_gpa; + src = context_base + (i << I915_GTT_PAGE_SHIFT); } intel_gvt_hypervisor_write_gpa(vgpu, workload->ring_context_gpa + RING_CTX_OFF(ring_header.val), &workload->rb_tail, 4); - page = i915_gem_object_get_page(ctx_obj, LRC_STATE_PN); - shadow_ring_context = kmap(page); + shadow_ring_context = context_base + + (LRC_STATE_PN << I915_GTT_PAGE_SHIFT); #define COPY_REG(name) \ intel_gvt_hypervisor_write_gpa(vgpu, workload->ring_context_gpa + \ @@ -865,7 +911,6 @@ static void update_guest_context(struct intel_vgpu_workload *workload) sizeof(*shadow_ring_context), I915_GTT_PAGE_SIZE - sizeof(*shadow_ring_context)); - kunmap(page); } void intel_vgpu_clean_workloads(struct intel_vgpu *vgpu,