From patchwork Tue Jan 19 04:59:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 12029535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2085C433DB for ; Tue, 19 Jan 2021 08:39:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9C41B23131 for ; Tue, 19 Jan 2021 08:39:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388564AbhASFWo (ORCPT ); Tue, 19 Jan 2021 00:22:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389412AbhASFG7 (ORCPT ); Tue, 19 Jan 2021 00:06:59 -0500 Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C954C06179B for ; Mon, 18 Jan 2021 21:04:53 -0800 (PST) Received: by mail-pg1-x52f.google.com with SMTP id v19so12256751pgj.12 for ; Mon, 18 Jan 2021 21:04:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=U0120iNFu1UQ+XQP2804t9yWUT9nyzP1s7JWVirZ0CQ=; b=tT/tmNAi6NU57xmHjfGmqHgX95hCN5stlPBS4x34MDgSRzGJgUPlWjosZg+6PyfxjO IlCL7iHuRliYzhRIUJwqVTWRZl33hJvmOwmriMEbSe8wcZxoyWzpiSsyMJjN59tnB6yK PPMsjJIyNWtdFwH4fXQAEjDjWjDvv/QoSFx+BpJjhKLG5bk2eHnhh3/IMNIoj7J3zH/U kQegq2euiTFuF9uqisspaevugD1zMGgphJ3VAW+xVbApOGXlsUBgHcU+0uBphjIjnbvF 99auMuVMqsaT7A8eptrh9JDdjA7uJnYlGEhuScmBxfPlLzwx4zdrr4WesSpAulQibsru 5Lcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U0120iNFu1UQ+XQP2804t9yWUT9nyzP1s7JWVirZ0CQ=; b=g2D+564fCGUmPV+0G6tsOossM6vWtOv7Y86Bnqy/4My4eKY2P4ISxwxGm9X6LPa7FL Gnky7RxmY+hAi7LwVj4NuqgbWmRyjQO5DKbrSaANR1LRaslpvHVHLmoNws6lTtt5PkkR cdd8aYes2PGvbCGZ3v2Z2CUXm5lF2IAuXI4Gm1nQZHzuUglCGMXDJZYKVnDb34B0+b1f bT36hJ0wzmjATe7jYaAy9j/ZnrYGkmmRIdGV4Gv9ASj5xmoz/1neKcThr6dUdKIFL4DB V4KgvCM9c1UVuCuI+fiYjaeRpKWEfXXGYSgn0cGEg0y7ZLBNUfDbE091uR1kM1g9RHZF DjXQ== X-Gm-Message-State: AOAM530o3MoM/XOw2bomKtVqCuKivY2+hr4mAqy9NVTZPjqWY1ir2kaE usx8oJiYnFqDMD69rTZ3HoxM X-Google-Smtp-Source: ABdhPJz29+5qkk0tFpBdHE1YKxTcsaEJOC9RJCsgQpfbCkBSRwz57KBR46or6uBk7ojXObygUicQrg== X-Received: by 2002:a63:d601:: with SMTP id q1mr2833194pgg.417.1611032692863; Mon, 18 Jan 2021 21:04:52 -0800 (PST) Received: from localhost ([139.177.225.243]) by smtp.gmail.com with ESMTPSA id ob6sm1047360pjb.30.2021.01.18.21.04.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jan 2021 21:04:52 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, stefanha@redhat.com, sgarzare@redhat.com, parav@nvidia.com, bob.liu@oracle.com, hch@infradead.org, rdunlap@infradead.org, willy@infradead.org, viro@zeniv.linux.org.uk, axboe@kernel.dk, bcrl@kvack.org, corbet@lwn.net Cc: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, kvm@vger.kernel.org, linux-aio@kvack.org, linux-fsdevel@vger.kernel.org Subject: [RFC v3 05/11] vdpa: shared virtual addressing support Date: Tue, 19 Jan 2021 12:59:14 +0800 Message-Id: <20210119045920.447-6-xieyongji@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210119045920.447-1-xieyongji@bytedance.com> References: <20210119045920.447-1-xieyongji@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This patches introduces SVA (Shared Virtual Addressing) support for vDPA device. During vDPA device allocation, vDPA device driver needs to indicate whether SVA is supported by the device. Then vhost-vdpa bus driver will not pin user page and transfer userspace virtual address instead of physical address during DMA mapping. Suggested-by: Jason Wang Signed-off-by: Xie Yongji --- drivers/vdpa/ifcvf/ifcvf_main.c | 2 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +- drivers/vdpa/vdpa.c | 5 ++++- drivers/vdpa/vdpa_sim/vdpa_sim.c | 3 ++- drivers/vhost/vdpa.c | 35 +++++++++++++++++++++++------------ include/linux/vdpa.h | 10 +++++++--- 6 files changed, 38 insertions(+), 19 deletions(-) diff --git a/drivers/vdpa/ifcvf/ifcvf_main.c b/drivers/vdpa/ifcvf/ifcvf_main.c index 23474af7da40..95c4601f82f5 100644 --- a/drivers/vdpa/ifcvf/ifcvf_main.c +++ b/drivers/vdpa/ifcvf/ifcvf_main.c @@ -439,7 +439,7 @@ static int ifcvf_probe(struct pci_dev *pdev, const struct pci_device_id *id) adapter = vdpa_alloc_device(struct ifcvf_adapter, vdpa, dev, &ifc_vdpa_ops, - IFCVF_MAX_QUEUE_PAIRS * 2, NULL); + IFCVF_MAX_QUEUE_PAIRS * 2, NULL, false); if (adapter == NULL) { IFCVF_ERR(pdev, "Failed to allocate vDPA structure"); return -ENOMEM; diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index 77595c81488d..05988d6907f2 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -1959,7 +1959,7 @@ static int mlx5v_probe(struct auxiliary_device *adev, max_vqs = min_t(u32, max_vqs, MLX5_MAX_SUPPORTED_VQS); ndev = vdpa_alloc_device(struct mlx5_vdpa_net, mvdev.vdev, mdev->device, &mlx5_vdpa_ops, - 2 * mlx5_vdpa_max_qps(max_vqs), NULL); + 2 * mlx5_vdpa_max_qps(max_vqs), NULL, false); if (IS_ERR(ndev)) return PTR_ERR(ndev); diff --git a/drivers/vdpa/vdpa.c b/drivers/vdpa/vdpa.c index 32bd48baffab..50cab930b2e5 100644 --- a/drivers/vdpa/vdpa.c +++ b/drivers/vdpa/vdpa.c @@ -72,6 +72,7 @@ static void vdpa_release_dev(struct device *d) * @nvqs: number of virtqueues supported by this device * @size: size of the parent structure that contains private data * @name: name of the vdpa device; optional. + * @sva: indicate whether SVA (Shared Virtual Addressing) is supported * * Driver should use vdpa_alloc_device() wrapper macro instead of * using this directly. @@ -81,7 +82,8 @@ static void vdpa_release_dev(struct device *d) */ struct vdpa_device *__vdpa_alloc_device(struct device *parent, const struct vdpa_config_ops *config, - int nvqs, size_t size, const char *name) + int nvqs, size_t size, const char *name, + bool sva) { struct vdpa_device *vdev; int err = -EINVAL; @@ -108,6 +110,7 @@ struct vdpa_device *__vdpa_alloc_device(struct device *parent, vdev->config = config; vdev->features_valid = false; vdev->nvqs = nvqs; + vdev->sva = sva; if (name) err = dev_set_name(&vdev->dev, "%s", name); diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index 85776e4e6749..03c796873a6b 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -367,7 +367,8 @@ static struct vdpasim *vdpasim_create(const char *name) else ops = &vdpasim_net_config_ops; - vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, VDPASIM_VQ_NUM, name); + vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, + VDPASIM_VQ_NUM, name, false); if (!vdpasim) goto err_alloc; diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 4a241d380c40..36b6950ba37f 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -486,21 +486,25 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, static void vhost_vdpa_iotlb_unmap(struct vhost_vdpa *v, u64 start, u64 last) { struct vhost_dev *dev = &v->vdev; + struct vdpa_device *vdpa = v->vdpa; struct vhost_iotlb *iotlb = dev->iotlb; struct vhost_iotlb_map *map; struct page *page; unsigned long pfn, pinned; while ((map = vhost_iotlb_itree_first(iotlb, start, last)) != NULL) { - pinned = map->size >> PAGE_SHIFT; - for (pfn = map->addr >> PAGE_SHIFT; - pinned > 0; pfn++, pinned--) { - page = pfn_to_page(pfn); - if (map->perm & VHOST_ACCESS_WO) - set_page_dirty_lock(page); - unpin_user_page(page); + if (!vdpa->sva) { + pinned = map->size >> PAGE_SHIFT; + for (pfn = map->addr >> PAGE_SHIFT; + pinned > 0; pfn++, pinned--) { + page = pfn_to_page(pfn); + if (map->perm & VHOST_ACCESS_WO) + set_page_dirty_lock(page); + unpin_user_page(page); + } + atomic64_sub(map->size >> PAGE_SHIFT, + &dev->mm->pinned_vm); } - atomic64_sub(map->size >> PAGE_SHIFT, &dev->mm->pinned_vm); vhost_iotlb_map_free(iotlb, map); } } @@ -558,13 +562,15 @@ static int vhost_vdpa_map(struct vhost_vdpa *v, r = iommu_map(v->domain, iova, pa, size, perm_to_iommu_flags(perm)); } - - if (r) + if (r) { vhost_iotlb_del_range(dev->iotlb, iova, iova + size - 1); - else + return r; + } + + if (!vdpa->sva) atomic64_add(size >> PAGE_SHIFT, &dev->mm->pinned_vm); - return r; + return 0; } static void vhost_vdpa_unmap(struct vhost_vdpa *v, u64 iova, u64 size) @@ -589,6 +595,7 @@ static int vhost_vdpa_process_iotlb_update(struct vhost_vdpa *v, struct vhost_iotlb_msg *msg) { struct vhost_dev *dev = &v->vdev; + struct vdpa_device *vdpa = v->vdpa; struct vhost_iotlb *iotlb = dev->iotlb; struct page **page_list; unsigned long list_size = PAGE_SIZE / sizeof(struct page *); @@ -607,6 +614,10 @@ static int vhost_vdpa_process_iotlb_update(struct vhost_vdpa *v, msg->iova + msg->size - 1)) return -EEXIST; + if (vdpa->sva) + return vhost_vdpa_map(v, msg->iova, msg->size, + msg->uaddr, msg->perm); + /* Limit the use of memory for bookkeeping */ page_list = (struct page **) __get_free_page(GFP_KERNEL); if (!page_list) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index cb5a3d847af3..f86869651614 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -44,6 +44,7 @@ struct vdpa_parent_dev; * @config: the configuration ops for this device. * @index: device index * @features_valid: were features initialized? for legacy guests + * @sva: indicate whether SVA (Shared Virtual Addressing) is supported * @nvqs: maximum number of supported virtqueues * @pdev: parent device pointer; caller must setup when registering device as part * of dev_add() parentdev ops callback before invoking _vdpa_register_device(). @@ -54,6 +55,7 @@ struct vdpa_device { const struct vdpa_config_ops *config; unsigned int index; bool features_valid; + bool sva; int nvqs; struct vdpa_parent_dev *pdev; }; @@ -250,14 +252,16 @@ struct vdpa_config_ops { struct vdpa_device *__vdpa_alloc_device(struct device *parent, const struct vdpa_config_ops *config, - int nvqs, size_t size, const char *name); + int nvqs, size_t size, + const char *name, bool sva); -#define vdpa_alloc_device(dev_struct, member, parent, config, nvqs, name) \ +#define vdpa_alloc_device(dev_struct, member, parent, config, \ + nvqs, name, sva) \ container_of(__vdpa_alloc_device( \ parent, config, nvqs, \ sizeof(dev_struct) + \ BUILD_BUG_ON_ZERO(offsetof( \ - dev_struct, member)), name), \ + dev_struct, member)), name, sva), \ dev_struct, member) int vdpa_register_device(struct vdpa_device *vdev);