From patchwork Wed Jun 10 11:35:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597569 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A830739 for ; Wed, 10 Jun 2020 11:38:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E06F62078D for ; Wed, 10 Jun 2020 11:38:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="H0+B0rDg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728571AbgFJLgG (ORCPT ); Wed, 10 Jun 2020 07:36:06 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:57948 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728338AbgFJLgG (ORCPT ); Wed, 10 Jun 2020 07:36:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788962; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pyZgoc1IoaMWB9Jefr7e/GoAFJJxo7imj3dgpBB/etA=; b=H0+B0rDgheh2eKx84RIOPRHMr06LQqn1WZ+iKKW+7ZFRwChEtEZJxlWO1J+7wVbUVyi6DX rL0IEH1ZGF6PLTvsJ624xGlwsWiAbzpRKtBcGL9WBmx7eaRissQt8gPmQiB7u8Xvul2LhC sYoEmgBxQ402QHzCtNLmVmEKncghN10= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-425-ppScX3NmMWaOHqpNRJv79Q-1; Wed, 10 Jun 2020 07:36:01 -0400 X-MC-Unique: ppScX3NmMWaOHqpNRJv79Q-1 Received: by mail-wr1-f72.google.com with SMTP id c14so957154wrw.11 for ; Wed, 10 Jun 2020 04:36:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=pyZgoc1IoaMWB9Jefr7e/GoAFJJxo7imj3dgpBB/etA=; b=pfDFjHX1vqma+wIirGMYnfFuXOM4DI62PuUXY+et8TapbpDg38yMp2+H12nOx4JiTh dTLjzEvGoCngxVWZKqYL1RaXkzNelynqh2xHWTnQLuIpY411QwC5S5/ftFknBJuUFsKv Bw6Qq7Ykjix/GALStEcOHP5Atugrf8XrMz9cahKxjqKp0guOwendQTf6+5dipKlKvxri gEYyo3A6h+3bOu1UAJ44TbgSglljxO/MXYuoYt1QfiRtVlRCsGGoDUXGTPNxothBb64T jciFi8H3o3+9KqZ6VADcToNVfT3wF3a392+Lap6RGuUZhZI7069NR8uyxJge0JAuek7r 2UTw== X-Gm-Message-State: AOAM530p8wp8DlFAehfuIGVNR/7xEz39qnPrUA1twqOgGi6hVh/ERJlp RdC/uEfNBu83VfRSjFF/Z499tVS1/HDqLL6PztuqG1p0AkVUMmqkQCH6VL5HYtPFnkg09R5eP60 zN+++XhlyCr6A X-Received: by 2002:adf:bbc5:: with SMTP id z5mr3253346wrg.269.1591788959997; Wed, 10 Jun 2020 04:35:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxRnisEyiSe77VgOXdspneG3+R5Whgn51TWUKnQmdwoU0/5KTvFQ8dV5mziKKbguFJ0mnBDtQ== X-Received: by 2002:adf:bbc5:: with SMTP id z5mr3253304wrg.269.1591788959418; Wed, 10 Jun 2020 04:35:59 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id j5sm7713166wrq.39.2020.06.10.04.35.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:35:49 -0700 (PDT) Date: Wed, 10 Jun 2020 07:35:47 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 01/14] vhost: option to fetch descriptors through an independent struct Message-ID: <20200610113515.1497099-2-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The idea is to support multiple ring formats by converting to a format-independent array of descriptors. This costs extra cycles, but we gain in ability to fetch a batch of descriptors in one go, which is good for code cache locality. When used, this causes a minor performance degradation, it's been kept as simple as possible for ease of review. A follow-up patch gets us back the performance by adding batching. To simplify benchmarking, I kept the old code around so one can switch back and forth between old and new code. This will go away in the final submission. Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez Link: https://lore.kernel.org/r/20200401183118.8334-2-eperezma@redhat.com Signed-off-by: Michael S. Tsirkin --- drivers/vhost/vhost.c | 305 +++++++++++++++++++++++++++++++++++++++++- drivers/vhost/vhost.h | 16 +++ 2 files changed, 320 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 172da092107e..180b7b58c76b 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -303,6 +303,7 @@ static void vhost_vq_reset(struct vhost_dev *dev, struct vhost_virtqueue *vq) { vq->num = 1; + vq->ndescs = 0; vq->desc = NULL; vq->avail = NULL; vq->used = NULL; @@ -373,6 +374,9 @@ static int vhost_worker(void *data) static void vhost_vq_free_iovecs(struct vhost_virtqueue *vq) { + kfree(vq->descs); + vq->descs = NULL; + vq->max_descs = 0; kfree(vq->indirect); vq->indirect = NULL; kfree(vq->log); @@ -389,6 +393,10 @@ static long vhost_dev_alloc_iovecs(struct vhost_dev *dev) for (i = 0; i < dev->nvqs; ++i) { vq = dev->vqs[i]; + vq->max_descs = dev->iov_limit; + vq->descs = kmalloc_array(vq->max_descs, + sizeof(*vq->descs), + GFP_KERNEL); vq->indirect = kmalloc_array(UIO_MAXIOV, sizeof(*vq->indirect), GFP_KERNEL); @@ -396,7 +404,7 @@ static long vhost_dev_alloc_iovecs(struct vhost_dev *dev) GFP_KERNEL); vq->heads = kmalloc_array(dev->iov_limit, sizeof(*vq->heads), GFP_KERNEL); - if (!vq->indirect || !vq->log || !vq->heads) + if (!vq->indirect || !vq->log || !vq->heads || !vq->descs) goto err_nomem; } return 0; @@ -488,6 +496,8 @@ void vhost_dev_init(struct vhost_dev *dev, for (i = 0; i < dev->nvqs; ++i) { vq = dev->vqs[i]; + vq->descs = NULL; + vq->max_descs = 0; vq->log = NULL; vq->indirect = NULL; vq->heads = NULL; @@ -2315,6 +2325,299 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, } EXPORT_SYMBOL_GPL(vhost_get_vq_desc); +static struct vhost_desc *peek_split_desc(struct vhost_virtqueue *vq) +{ + BUG_ON(!vq->ndescs); + return &vq->descs[vq->ndescs - 1]; +} + +static void pop_split_desc(struct vhost_virtqueue *vq) +{ + BUG_ON(!vq->ndescs); + --vq->ndescs; +} + +#define VHOST_DESC_FLAGS (VRING_DESC_F_INDIRECT | VRING_DESC_F_WRITE | \ + VRING_DESC_F_NEXT) +static int push_split_desc(struct vhost_virtqueue *vq, struct vring_desc *desc, u16 id) +{ + struct vhost_desc *h; + + if (unlikely(vq->ndescs >= vq->max_descs)) + return -EINVAL; + h = &vq->descs[vq->ndescs++]; + h->addr = vhost64_to_cpu(vq, desc->addr); + h->len = vhost32_to_cpu(vq, desc->len); + h->flags = vhost16_to_cpu(vq, desc->flags) & VHOST_DESC_FLAGS; + h->id = id; + + return 0; +} + +static int fetch_indirect_descs(struct vhost_virtqueue *vq, + struct vhost_desc *indirect, + u16 head) +{ + struct vring_desc desc; + unsigned int i = 0, count, found = 0; + u32 len = indirect->len; + struct iov_iter from; + int ret; + + /* Sanity check */ + if (unlikely(len % sizeof desc)) { + vq_err(vq, "Invalid length in indirect descriptor: " + "len 0x%llx not multiple of 0x%zx\n", + (unsigned long long)len, + sizeof desc); + return -EINVAL; + } + + ret = translate_desc(vq, indirect->addr, len, vq->indirect, + UIO_MAXIOV, VHOST_ACCESS_RO); + if (unlikely(ret < 0)) { + if (ret != -EAGAIN) + vq_err(vq, "Translation failure %d in indirect.\n", ret); + return ret; + } + iov_iter_init(&from, READ, vq->indirect, ret, len); + + /* We will use the result as an address to read from, so most + * architectures only need a compiler barrier here. */ + read_barrier_depends(); + + count = len / sizeof desc; + /* Buffers are chained via a 16 bit next field, so + * we can have at most 2^16 of these. */ + if (unlikely(count > USHRT_MAX + 1)) { + vq_err(vq, "Indirect buffer length too big: %d\n", + indirect->len); + return -E2BIG; + } + if (unlikely(vq->ndescs + count > vq->max_descs)) { + vq_err(vq, "Too many indirect + direct descs: %d + %d\n", + vq->ndescs, indirect->len); + return -E2BIG; + } + + do { + if (unlikely(++found > count)) { + vq_err(vq, "Loop detected: last one at %u " + "indirect size %u\n", + i, count); + return -EINVAL; + } + if (unlikely(!copy_from_iter_full(&desc, sizeof(desc), &from))) { + vq_err(vq, "Failed indirect descriptor: idx %d, %zx\n", + i, (size_t)indirect->addr + i * sizeof desc); + return -EINVAL; + } + if (unlikely(desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_INDIRECT))) { + vq_err(vq, "Nested indirect descriptor: idx %d, %zx\n", + i, (size_t)indirect->addr + i * sizeof desc); + return -EINVAL; + } + + /* Note: push_split_desc can't fail here: + * we never fetch unless there's space. */ + ret = push_split_desc(vq, &desc, head); + WARN_ON(ret); + } while ((i = next_desc(vq, &desc)) != -1); + return 0; +} + +/* This function returns a value > 0 if a descriptor was found, or 0 if none were found. + * A negative code is returned on error. */ +static int fetch_descs(struct vhost_virtqueue *vq) +{ + unsigned int i, head, found = 0; + struct vhost_desc *last; + struct vring_desc desc; + __virtio16 avail_idx; + __virtio16 ring_head; + u16 last_avail_idx; + int ret; + + /* Check it isn't doing very strange things with descriptor numbers. */ + last_avail_idx = vq->last_avail_idx; + + if (vq->avail_idx == vq->last_avail_idx) { + if (unlikely(vhost_get_avail_idx(vq, &avail_idx))) { + vq_err(vq, "Failed to access avail idx at %p\n", + &vq->avail->idx); + return -EFAULT; + } + vq->avail_idx = vhost16_to_cpu(vq, avail_idx); + + if (unlikely((u16)(vq->avail_idx - last_avail_idx) > vq->num)) { + vq_err(vq, "Guest moved used index from %u to %u", + last_avail_idx, vq->avail_idx); + return -EFAULT; + } + + /* If there's nothing new since last we looked, return + * invalid. + */ + if (vq->avail_idx == last_avail_idx) + return 0; + + /* Only get avail ring entries after they have been + * exposed by guest. + */ + smp_rmb(); + } + + /* Grab the next descriptor number they're advertising */ + if (unlikely(vhost_get_avail_head(vq, &ring_head, last_avail_idx))) { + vq_err(vq, "Failed to read head: idx %d address %p\n", + last_avail_idx, + &vq->avail->ring[last_avail_idx % vq->num]); + return -EFAULT; + } + + head = vhost16_to_cpu(vq, ring_head); + + /* If their number is silly, that's an error. */ + if (unlikely(head >= vq->num)) { + vq_err(vq, "Guest says index %u > %u is available", + head, vq->num); + return -EINVAL; + } + + i = head; + do { + if (unlikely(i >= vq->num)) { + vq_err(vq, "Desc index is %u > %u, head = %u", + i, vq->num, head); + return -EINVAL; + } + if (unlikely(++found > vq->num)) { + vq_err(vq, "Loop detected: last one at %u " + "vq size %u head %u\n", + i, vq->num, head); + return -EINVAL; + } + ret = vhost_get_desc(vq, &desc, i); + if (unlikely(ret)) { + vq_err(vq, "Failed to get descriptor: idx %d addr %p\n", + i, vq->desc + i); + return -EFAULT; + } + ret = push_split_desc(vq, &desc, head); + if (unlikely(ret)) { + vq_err(vq, "Failed to save descriptor: idx %d\n", i); + return -EINVAL; + } + } while ((i = next_desc(vq, &desc)) != -1); + + last = peek_split_desc(vq); + if (unlikely(last->flags & VRING_DESC_F_INDIRECT)) { + pop_split_desc(vq); + ret = fetch_indirect_descs(vq, last, head); + if (unlikely(ret < 0)) { + if (ret != -EAGAIN) + vq_err(vq, "Failure detected " + "in indirect descriptor at idx %d\n", head); + return ret; + } + } + + /* Assume notifications from guest are disabled at this point, + * if they aren't we would need to update avail_event index. */ + BUG_ON(!(vq->used_flags & VRING_USED_F_NO_NOTIFY)); + + /* On success, increment avail index. */ + vq->last_avail_idx++; + + return 1; +} + +/* This looks in the virtqueue and for the first available buffer, and converts + * it to an iovec for convenient access. Since descriptors consist of some + * number of output then some number of input descriptors, it's actually two + * iovecs, but we pack them into one and note how many of each there were. + * + * This function returns the descriptor number found, or vq->num (which is + * never a valid descriptor number) if none was found. A negative code is + * returned on error. */ +int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, + struct iovec iov[], unsigned int iov_size, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num) +{ + int ret = fetch_descs(vq); + int i; + + if (ret <= 0) + goto err_fetch; + + /* Now convert to IOV */ + /* When we start there are none of either input nor output. */ + *out_num = *in_num = 0; + if (unlikely(log)) + *log_num = 0; + + for (i = 0; i < vq->ndescs; ++i) { + unsigned iov_count = *in_num + *out_num; + struct vhost_desc *desc = &vq->descs[i]; + int access; + + if (desc->flags & ~VHOST_DESC_FLAGS) { + vq_err(vq, "Unexpected flags: 0x%x at descriptor id 0x%x\n", + desc->flags, desc->id); + ret = -EINVAL; + goto err; + } + if (desc->flags & VRING_DESC_F_WRITE) + access = VHOST_ACCESS_WO; + else + access = VHOST_ACCESS_RO; + ret = translate_desc(vq, desc->addr, + desc->len, iov + iov_count, + iov_size - iov_count, access); + if (unlikely(ret < 0)) { + if (ret != -EAGAIN) + vq_err(vq, "Translation failure %d descriptor idx %d\n", + ret, i); + goto err; + } + if (access == VHOST_ACCESS_WO) { + /* If this is an input descriptor, + * increment that count. */ + *in_num += ret; + if (unlikely(log && ret)) { + log[*log_num].addr = desc->addr; + log[*log_num].len = desc->len; + ++*log_num; + } + } else { + /* If it's an output descriptor, they're all supposed + * to come before any input descriptors. */ + if (unlikely(*in_num)) { + vq_err(vq, "Descriptor has out after in: " + "idx %d\n", i); + ret = -EINVAL; + goto err; + } + *out_num += ret; + } + + ret = desc->id; + } + + vq->ndescs = 0; + + return ret; + +err: + vhost_discard_vq_desc(vq, 1); +err_fetch: + vq->ndescs = 0; + + return ret; +} +EXPORT_SYMBOL_GPL(vhost_get_vq_desc_batch); + /* Reverse the effect of vhost_get_vq_desc. Useful for error handling. */ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) { diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index c8e96a095d3b..87089d51490d 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -60,6 +60,13 @@ enum vhost_uaddr_type { VHOST_NUM_ADDRS = 3, }; +struct vhost_desc { + u64 addr; + u32 len; + u16 flags; /* VRING_DESC_F_WRITE, VRING_DESC_F_NEXT */ + u16 id; +}; + /* The virtqueue structure describes a queue attached to a device. */ struct vhost_virtqueue { struct vhost_dev *dev; @@ -71,6 +78,11 @@ struct vhost_virtqueue { vring_avail_t __user *avail; vring_used_t __user *used; const struct vhost_iotlb_map *meta_iotlb[VHOST_NUM_ADDRS]; + + struct vhost_desc *descs; + int ndescs; + int max_descs; + struct file *kick; struct eventfd_ctx *call_ctx; struct eventfd_ctx *error_ctx; @@ -177,6 +189,10 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg bool vhost_vq_access_ok(struct vhost_virtqueue *vq); bool vhost_log_access_ok(struct vhost_dev *); +int vhost_get_vq_desc_batch(struct vhost_virtqueue *, + struct iovec iov[], unsigned int iov_count, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num); int vhost_get_vq_desc(struct vhost_virtqueue *, struct iovec iov[], unsigned int iov_count, unsigned int *out_num, unsigned int *in_num, From patchwork Wed Jun 10 11:35:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597543 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3C8414E3 for ; Wed, 10 Jun 2020 11:36:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CA33D2074B for ; Wed, 10 Jun 2020 11:36:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AEMqal8D" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728581AbgFJLgH (ORCPT ); Wed, 10 Jun 2020 07:36:07 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:25936 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728540AbgFJLgG (ORCPT ); Wed, 10 Jun 2020 07:36:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788964; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QQIo3L2MWWvjb6mzSKSfjdSEv1rN96HPXR5N45kiTKk=; b=AEMqal8DFZiU3yL6IzPoiea7QFSiL7ueOYkC2NZJsIl3gTEGIynOhlVek9XoJOgCFvR70K aNGk6lad0egoqocnGCQcznOLgVZr2FZenJzJdhgECrFLt1wpKwMt+9IYRJKdq59Fqv+nsX jLgy8WmfOPzZUvh3i8lsPXXfLotUzcY= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-223-bOGI1rFVNu24qMykFIhxYQ-1; Wed, 10 Jun 2020 07:36:03 -0400 X-MC-Unique: bOGI1rFVNu24qMykFIhxYQ-1 Received: by mail-wm1-f72.google.com with SMTP id k185so332977wme.8 for ; Wed, 10 Jun 2020 04:36:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=QQIo3L2MWWvjb6mzSKSfjdSEv1rN96HPXR5N45kiTKk=; b=EsGi3IDtQslLd66EP4MN+tJ4fFChyzLtrzXds3O7bOCGQkfhtwpxyax16/aQxBZRyA Ey6TftC9ns8QwzvgfKO8bz4MU0qYSbG0+aUXGa+MYa7x3B4p7cgpmarK46cFYbKUozPt nyIHc/YBpP5ZhKl3y5PXJKtsj8/C7MWnJ1mk9L9uGMIEfNaaAi1P3R/P6D/aUN4xGq3K JZnCRYb3WDPUjyiOuh3HkB5KOE+lYTDcV2kA+RuYcidrgb23DYg03R02uiA//9X8V1Sz 9C0agzp05Ou4tRgCRdPzTEaKqDzklTre6Th2q+PCaFI2SzDNK+lH7kpPXf6LXOoby458 tgFw== X-Gm-Message-State: AOAM531YBY+t9rqRV9N58CvKI+RzXwHNvLMFfqb9Jn4d71dOvq2O0qpY 4b6dDMnGGwy7GNnLomK2l2lHRn091A8mkifnSwvsonXoCQgkndALZ80N8qtYaeHtSEe5gAgGFBe P//zgDLCyNK9B X-Received: by 2002:adf:a157:: with SMTP id r23mr3378907wrr.92.1591788962282; Wed, 10 Jun 2020 04:36:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxAHBbn1Hm38GkVQUdC9QO7S3prwhr5Px+sKP6OuFbtKtCWmiw5G13O+k9LGPpnGaeJbpRtfQ== X-Received: by 2002:adf:a157:: with SMTP id r23mr3378893wrr.92.1591788962077; Wed, 10 Jun 2020 04:36:02 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id c206sm7360362wmf.36.2020.06.10.04.36.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:01 -0700 (PDT) Date: Wed, 10 Jun 2020 07:35:59 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 02/14] fixup! vhost: option to fetch descriptors through an independent struct Message-ID: <20200610113515.1497099-3-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org --- drivers/vhost/vhost.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 180b7b58c76b..11433d709651 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2614,7 +2614,7 @@ int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, err_fetch: vq->ndescs = 0; - return ret; + return ret ? ret : vq->num; } EXPORT_SYMBOL_GPL(vhost_get_vq_desc_batch); From patchwork Wed Jun 10 11:36:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597567 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01F4A739 for ; Wed, 10 Jun 2020 11:38:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CFBAA20760 for ; Wed, 10 Jun 2020 11:38:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LjTVZP8v" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728613AbgFJLiR (ORCPT ); Wed, 10 Jun 2020 07:38:17 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:52561 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728510AbgFJLgM (ORCPT ); Wed, 10 Jun 2020 07:36:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788969; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+h6WNjt1EUJ7djnafdeXZVt1l60+nlqYD5MbYj/O2cA=; b=LjTVZP8vpju+Jf+8Eq8ko5pAbU2OWLksRY0w2UPlnaZGrw3B5HlPQa+rCrvBr9D+yKSpyN SXYAO7D7x69ln3Er3Y3HnrraUDc5X0ObDM4+CjcjNKjkxCX7ugznwKmYkOm4K9/CsoV12K 8rnUgCfsyoAU2XPjz7pCcmd6pJ9p20Q= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-290-DvvN5w78NVS80-P6t-_QUw-1; Wed, 10 Jun 2020 07:36:07 -0400 X-MC-Unique: DvvN5w78NVS80-P6t-_QUw-1 Received: by mail-wm1-f72.google.com with SMTP id k185so333032wme.8 for ; Wed, 10 Jun 2020 04:36:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=+h6WNjt1EUJ7djnafdeXZVt1l60+nlqYD5MbYj/O2cA=; b=WBPe2S6J/zDnWZTvOXjIixLzMP9LRWNZYDs+NY1jO+NcUEHPZ5Ln9/BUzOLPykwnEB XMsssHL2YXH5c7E5FRBOl8rTymjRD8H/uRypMSjJ9WoAfEU91wwDaQPgSneSe/mtEnoZ ULhEYYZgd2tV08wTLHP4gF1IFy6LkJbgge/hGY8DKRiylQgU4MWF+DmLgeXzA8ydMtLY AzX8zzKTgbKBxy6cfqMAgCsVv9hEnmEGibObctWBwSKPQ/RNHiTjBA/aFEiSO2Rxh+JB uQvjIOAAr+lV9A7LjLWqIbCGiBUyc0W6O8O7sqVNxDD/gFdg+8p89DEeVyf+66eo8TDd Hk7w== X-Gm-Message-State: AOAM531720exYERpGf3Z6+nicyqtSfjlKc6kB+jcunMcl02n/BHcXAeW ZzYrI5HFc8j4R6bzj4RHPGrC+zPE+PlgJ7ZIxYmLKe4oeAjkQ2Um+RXfmqG1mfZQiWnE3XUANSE W9ivgO7QMQRpU X-Received: by 2002:a5d:490f:: with SMTP id x15mr3144236wrq.259.1591788965610; Wed, 10 Jun 2020 04:36:05 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwues+Vtl6aY5DUeDEIDuDb9cULECCZ7k6aJQia4Eyl+dDBLUV4uwB3BChedOa92Aoj6O82sg== X-Received: by 2002:a5d:490f:: with SMTP id x15mr3144193wrq.259.1591788964996; Wed, 10 Jun 2020 04:36:04 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id t14sm8096781wrb.94.2020.06.10.04.36.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:04 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:02 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 03/14] vhost: use batched get_vq_desc version Message-ID: <20200610113515.1497099-4-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org As testing shows no performance change, switch to that now. Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez Link: https://lore.kernel.org/r/20200401183118.8334-3-eperezma@redhat.com Signed-off-by: Michael S. Tsirkin --- drivers/vhost/test.c | 2 +- drivers/vhost/vhost.c | 318 ++++++++---------------------------------- drivers/vhost/vhost.h | 7 +- 3 files changed, 65 insertions(+), 262 deletions(-) diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c index 0466921f4772..7d69778aaa26 100644 --- a/drivers/vhost/test.c +++ b/drivers/vhost/test.c @@ -119,7 +119,7 @@ static int vhost_test_open(struct inode *inode, struct file *f) dev = &n->dev; vqs[VHOST_TEST_VQ] = &n->vqs[VHOST_TEST_VQ]; n->vqs[VHOST_TEST_VQ].handle_kick = handle_vq_kick; - vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV, + vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV + 64, VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL); f->private_data = n; diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 11433d709651..28f324fd77df 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -304,6 +304,7 @@ static void vhost_vq_reset(struct vhost_dev *dev, { vq->num = 1; vq->ndescs = 0; + vq->first_desc = 0; vq->desc = NULL; vq->avail = NULL; vq->used = NULL; @@ -372,6 +373,11 @@ static int vhost_worker(void *data) return 0; } +static int vhost_vq_num_batch_descs(struct vhost_virtqueue *vq) +{ + return vq->max_descs - UIO_MAXIOV; +} + static void vhost_vq_free_iovecs(struct vhost_virtqueue *vq) { kfree(vq->descs); @@ -394,6 +400,9 @@ static long vhost_dev_alloc_iovecs(struct vhost_dev *dev) for (i = 0; i < dev->nvqs; ++i) { vq = dev->vqs[i]; vq->max_descs = dev->iov_limit; + if (vhost_vq_num_batch_descs(vq) < 0) { + return -EINVAL; + } vq->descs = kmalloc_array(vq->max_descs, sizeof(*vq->descs), GFP_KERNEL); @@ -1610,6 +1619,7 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg vq->last_avail_idx = s.num; /* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx; + vq->ndescs = vq->first_desc = 0; break; case VHOST_GET_VRING_BASE: s.index = idx; @@ -2078,253 +2088,6 @@ static unsigned next_desc(struct vhost_virtqueue *vq, struct vring_desc *desc) return next; } -static int get_indirect(struct vhost_virtqueue *vq, - struct iovec iov[], unsigned int iov_size, - unsigned int *out_num, unsigned int *in_num, - struct vhost_log *log, unsigned int *log_num, - struct vring_desc *indirect) -{ - struct vring_desc desc; - unsigned int i = 0, count, found = 0; - u32 len = vhost32_to_cpu(vq, indirect->len); - struct iov_iter from; - int ret, access; - - /* Sanity check */ - if (unlikely(len % sizeof desc)) { - vq_err(vq, "Invalid length in indirect descriptor: " - "len 0x%llx not multiple of 0x%zx\n", - (unsigned long long)len, - sizeof desc); - return -EINVAL; - } - - ret = translate_desc(vq, vhost64_to_cpu(vq, indirect->addr), len, vq->indirect, - UIO_MAXIOV, VHOST_ACCESS_RO); - if (unlikely(ret < 0)) { - if (ret != -EAGAIN) - vq_err(vq, "Translation failure %d in indirect.\n", ret); - return ret; - } - iov_iter_init(&from, READ, vq->indirect, ret, len); - - /* We will use the result as an address to read from, so most - * architectures only need a compiler barrier here. */ - read_barrier_depends(); - - count = len / sizeof desc; - /* Buffers are chained via a 16 bit next field, so - * we can have at most 2^16 of these. */ - if (unlikely(count > USHRT_MAX + 1)) { - vq_err(vq, "Indirect buffer length too big: %d\n", - indirect->len); - return -E2BIG; - } - - do { - unsigned iov_count = *in_num + *out_num; - if (unlikely(++found > count)) { - vq_err(vq, "Loop detected: last one at %u " - "indirect size %u\n", - i, count); - return -EINVAL; - } - if (unlikely(!copy_from_iter_full(&desc, sizeof(desc), &from))) { - vq_err(vq, "Failed indirect descriptor: idx %d, %zx\n", - i, (size_t)vhost64_to_cpu(vq, indirect->addr) + i * sizeof desc); - return -EINVAL; - } - if (unlikely(desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_INDIRECT))) { - vq_err(vq, "Nested indirect descriptor: idx %d, %zx\n", - i, (size_t)vhost64_to_cpu(vq, indirect->addr) + i * sizeof desc); - return -EINVAL; - } - - if (desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_WRITE)) - access = VHOST_ACCESS_WO; - else - access = VHOST_ACCESS_RO; - - ret = translate_desc(vq, vhost64_to_cpu(vq, desc.addr), - vhost32_to_cpu(vq, desc.len), iov + iov_count, - iov_size - iov_count, access); - if (unlikely(ret < 0)) { - if (ret != -EAGAIN) - vq_err(vq, "Translation failure %d indirect idx %d\n", - ret, i); - return ret; - } - /* If this is an input descriptor, increment that count. */ - if (access == VHOST_ACCESS_WO) { - *in_num += ret; - if (unlikely(log && ret)) { - log[*log_num].addr = vhost64_to_cpu(vq, desc.addr); - log[*log_num].len = vhost32_to_cpu(vq, desc.len); - ++*log_num; - } - } else { - /* If it's an output descriptor, they're all supposed - * to come before any input descriptors. */ - if (unlikely(*in_num)) { - vq_err(vq, "Indirect descriptor " - "has out after in: idx %d\n", i); - return -EINVAL; - } - *out_num += ret; - } - } while ((i = next_desc(vq, &desc)) != -1); - return 0; -} - -/* This looks in the virtqueue and for the first available buffer, and converts - * it to an iovec for convenient access. Since descriptors consist of some - * number of output then some number of input descriptors, it's actually two - * iovecs, but we pack them into one and note how many of each there were. - * - * This function returns the descriptor number found, or vq->num (which is - * never a valid descriptor number) if none was found. A negative code is - * returned on error. */ -int vhost_get_vq_desc(struct vhost_virtqueue *vq, - struct iovec iov[], unsigned int iov_size, - unsigned int *out_num, unsigned int *in_num, - struct vhost_log *log, unsigned int *log_num) -{ - struct vring_desc desc; - unsigned int i, head, found = 0; - u16 last_avail_idx; - __virtio16 avail_idx; - __virtio16 ring_head; - int ret, access; - - /* Check it isn't doing very strange things with descriptor numbers. */ - last_avail_idx = vq->last_avail_idx; - - if (vq->avail_idx == vq->last_avail_idx) { - if (unlikely(vhost_get_avail_idx(vq, &avail_idx))) { - vq_err(vq, "Failed to access avail idx at %p\n", - &vq->avail->idx); - return -EFAULT; - } - vq->avail_idx = vhost16_to_cpu(vq, avail_idx); - - if (unlikely((u16)(vq->avail_idx - last_avail_idx) > vq->num)) { - vq_err(vq, "Guest moved used index from %u to %u", - last_avail_idx, vq->avail_idx); - return -EFAULT; - } - - /* If there's nothing new since last we looked, return - * invalid. - */ - if (vq->avail_idx == last_avail_idx) - return vq->num; - - /* Only get avail ring entries after they have been - * exposed by guest. - */ - smp_rmb(); - } - - /* Grab the next descriptor number they're advertising, and increment - * the index we've seen. */ - if (unlikely(vhost_get_avail_head(vq, &ring_head, last_avail_idx))) { - vq_err(vq, "Failed to read head: idx %d address %p\n", - last_avail_idx, - &vq->avail->ring[last_avail_idx % vq->num]); - return -EFAULT; - } - - head = vhost16_to_cpu(vq, ring_head); - - /* If their number is silly, that's an error. */ - if (unlikely(head >= vq->num)) { - vq_err(vq, "Guest says index %u > %u is available", - head, vq->num); - return -EINVAL; - } - - /* When we start there are none of either input nor output. */ - *out_num = *in_num = 0; - if (unlikely(log)) - *log_num = 0; - - i = head; - do { - unsigned iov_count = *in_num + *out_num; - if (unlikely(i >= vq->num)) { - vq_err(vq, "Desc index is %u > %u, head = %u", - i, vq->num, head); - return -EINVAL; - } - if (unlikely(++found > vq->num)) { - vq_err(vq, "Loop detected: last one at %u " - "vq size %u head %u\n", - i, vq->num, head); - return -EINVAL; - } - ret = vhost_get_desc(vq, &desc, i); - if (unlikely(ret)) { - vq_err(vq, "Failed to get descriptor: idx %d addr %p\n", - i, vq->desc + i); - return -EFAULT; - } - if (desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_INDIRECT)) { - ret = get_indirect(vq, iov, iov_size, - out_num, in_num, - log, log_num, &desc); - if (unlikely(ret < 0)) { - if (ret != -EAGAIN) - vq_err(vq, "Failure detected " - "in indirect descriptor at idx %d\n", i); - return ret; - } - continue; - } - - if (desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_WRITE)) - access = VHOST_ACCESS_WO; - else - access = VHOST_ACCESS_RO; - ret = translate_desc(vq, vhost64_to_cpu(vq, desc.addr), - vhost32_to_cpu(vq, desc.len), iov + iov_count, - iov_size - iov_count, access); - if (unlikely(ret < 0)) { - if (ret != -EAGAIN) - vq_err(vq, "Translation failure %d descriptor idx %d\n", - ret, i); - return ret; - } - if (access == VHOST_ACCESS_WO) { - /* If this is an input descriptor, - * increment that count. */ - *in_num += ret; - if (unlikely(log && ret)) { - log[*log_num].addr = vhost64_to_cpu(vq, desc.addr); - log[*log_num].len = vhost32_to_cpu(vq, desc.len); - ++*log_num; - } - } else { - /* If it's an output descriptor, they're all supposed - * to come before any input descriptors. */ - if (unlikely(*in_num)) { - vq_err(vq, "Descriptor has out after in: " - "idx %d\n", i); - return -EINVAL; - } - *out_num += ret; - } - } while ((i = next_desc(vq, &desc)) != -1); - - /* On success, increment avail index. */ - vq->last_avail_idx++; - - /* Assume notifications from guest are disabled at this point, - * if they aren't we would need to update avail_event index. */ - BUG_ON(!(vq->used_flags & VRING_USED_F_NO_NOTIFY)); - return head; -} -EXPORT_SYMBOL_GPL(vhost_get_vq_desc); - static struct vhost_desc *peek_split_desc(struct vhost_virtqueue *vq) { BUG_ON(!vq->ndescs); @@ -2428,7 +2191,7 @@ static int fetch_indirect_descs(struct vhost_virtqueue *vq, /* This function returns a value > 0 if a descriptor was found, or 0 if none were found. * A negative code is returned on error. */ -static int fetch_descs(struct vhost_virtqueue *vq) +static int fetch_buf(struct vhost_virtqueue *vq) { unsigned int i, head, found = 0; struct vhost_desc *last; @@ -2441,7 +2204,11 @@ static int fetch_descs(struct vhost_virtqueue *vq) /* Check it isn't doing very strange things with descriptor numbers. */ last_avail_idx = vq->last_avail_idx; - if (vq->avail_idx == vq->last_avail_idx) { + if (unlikely(vq->avail_idx == vq->last_avail_idx)) { + /* If we already have work to do, don't bother re-checking. */ + if (likely(vq->ndescs)) + return 1; + if (unlikely(vhost_get_avail_idx(vq, &avail_idx))) { vq_err(vq, "Failed to access avail idx at %p\n", &vq->avail->idx); @@ -2532,6 +2299,41 @@ static int fetch_descs(struct vhost_virtqueue *vq) return 1; } +/* This function returns a value > 0 if a descriptor was found, or 0 if none were found. + * A negative code is returned on error. */ +static int fetch_descs(struct vhost_virtqueue *vq) +{ + int ret; + + if (unlikely(vq->first_desc >= vq->ndescs)) { + vq->first_desc = 0; + vq->ndescs = 0; + } + + if (vq->ndescs) + return 1; + + for (ret = 1; + ret > 0 && vq->ndescs <= vhost_vq_num_batch_descs(vq); + ret = fetch_buf(vq)) + ; + + /* On success we expect some descs */ + BUG_ON(ret > 0 && !vq->ndescs); + return ret; +} + +/* Reverse the effects of fetch_descs */ +static void unfetch_descs(struct vhost_virtqueue *vq) +{ + int i; + + for (i = vq->first_desc; i < vq->ndescs; ++i) + if (!(vq->descs[i].flags & VRING_DESC_F_NEXT)) + vq->last_avail_idx -= 1; + vq->ndescs = 0; +} + /* This looks in the virtqueue and for the first available buffer, and converts * it to an iovec for convenient access. Since descriptors consist of some * number of output then some number of input descriptors, it's actually two @@ -2540,7 +2342,7 @@ static int fetch_descs(struct vhost_virtqueue *vq) * This function returns the descriptor number found, or vq->num (which is * never a valid descriptor number) if none was found. A negative code is * returned on error. */ -int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, +int vhost_get_vq_desc(struct vhost_virtqueue *vq, struct iovec iov[], unsigned int iov_size, unsigned int *out_num, unsigned int *in_num, struct vhost_log *log, unsigned int *log_num) @@ -2549,7 +2351,7 @@ int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, int i; if (ret <= 0) - goto err_fetch; + goto err; /* Now convert to IOV */ /* When we start there are none of either input nor output. */ @@ -2557,7 +2359,7 @@ int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, if (unlikely(log)) *log_num = 0; - for (i = 0; i < vq->ndescs; ++i) { + for (i = vq->first_desc; i < vq->ndescs; ++i) { unsigned iov_count = *in_num + *out_num; struct vhost_desc *desc = &vq->descs[i]; int access; @@ -2603,24 +2405,26 @@ int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, } ret = desc->id; + + if (!(desc->flags & VRING_DESC_F_NEXT)) + break; } - vq->ndescs = 0; + vq->first_desc = i + 1; return ret; err: - vhost_discard_vq_desc(vq, 1); -err_fetch: - vq->ndescs = 0; + unfetch_descs(vq); return ret ? ret : vq->num; } -EXPORT_SYMBOL_GPL(vhost_get_vq_desc_batch); +EXPORT_SYMBOL_GPL(vhost_get_vq_desc); /* Reverse the effect of vhost_get_vq_desc. Useful for error handling. */ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) { + unfetch_descs(vq); vq->last_avail_idx -= n; } EXPORT_SYMBOL_GPL(vhost_discard_vq_desc); diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index 87089d51490d..fed36af5c444 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -81,6 +81,7 @@ struct vhost_virtqueue { struct vhost_desc *descs; int ndescs; + int first_desc; int max_descs; struct file *kick; @@ -189,10 +190,6 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg bool vhost_vq_access_ok(struct vhost_virtqueue *vq); bool vhost_log_access_ok(struct vhost_dev *); -int vhost_get_vq_desc_batch(struct vhost_virtqueue *, - struct iovec iov[], unsigned int iov_count, - unsigned int *out_num, unsigned int *in_num, - struct vhost_log *log, unsigned int *log_num); int vhost_get_vq_desc(struct vhost_virtqueue *, struct iovec iov[], unsigned int iov_count, unsigned int *out_num, unsigned int *in_num, @@ -261,6 +258,8 @@ static inline void vhost_vq_set_backend(struct vhost_virtqueue *vq, void *private_data) { vq->private_data = private_data; + vq->ndescs = 0; + vq->first_desc = 0; } /** From patchwork Wed Jun 10 11:36:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597565 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E0D9214B7 for ; Wed, 10 Jun 2020 11:38:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C51F120760 for ; Wed, 10 Jun 2020 11:38:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DdnfpOGf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728593AbgFJLiJ (ORCPT ); Wed, 10 Jun 2020 07:38:09 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:33950 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728615AbgFJLgO (ORCPT ); Wed, 10 Jun 2020 07:36:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788972; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VWXM+ipVrzQXYONFpt3zkOQmQtQZ231iT3DOSd7IMMo=; b=DdnfpOGfMI9Mk7zj+hxaSg133NvbcnQoOYGJnfbONsN52dVukJIBclB96XsF1d77qErqn+ Swt99TWmrqujYNJxv6GWhjpy10rkkBSlwJ5Ngnc8IXQwlI23P+1XlGzNsqzbo1fc/JJstT h3pDsERbQdaNr+VAdgiNu6DAR96xZPQ= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-37-lEA32kgNPwSpn_Wq6X8ozA-1; Wed, 10 Jun 2020 07:36:09 -0400 X-MC-Unique: lEA32kgNPwSpn_Wq6X8ozA-1 Received: by mail-wm1-f72.google.com with SMTP id p24so332766wma.4 for ; Wed, 10 Jun 2020 04:36:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=VWXM+ipVrzQXYONFpt3zkOQmQtQZ231iT3DOSd7IMMo=; b=PMdjsPlC5ggyxdCq6z1sZaf96idMhTR4rdOrvhZ61BlUrlfWCpUMs2gDjr48n3vF7F VzoUQwe19clQGf7wfmIm/Fxa+aMslcqShKVWuOhX6AQQBxdXixUBo16iiDs8sWo43ITO 5WkAZh52weg9gHpgLCXVTQAynGgEFn1z14ABWQA+JhCOWViheeyJd9wzEhTj3f1hto4b aTAQyfhMi+Eodmjmj4HsNCF7FUWpBqUvuObHJ231UvUEjVgwxsTDDbTk/sCSTp3/P06+ IkFr1TqxOOkQKp4lzR9Qh8Yqyb+nZY/8I53PyNHRfm+arIdNKMasRVu705mw+kqjeww4 yJZw== X-Gm-Message-State: AOAM531VDDm+uErz3clFB7L3w1BtzJ+TIUIJn5nR8U5upxPyF8fgZR0k k5o7wjkt7wMUbNMesEprkaaF5ZSnHFlZ1OBBGS9Y4o2XD92i1RTzWYY/m9F//RZJhTYs+z27Fxk ADSNgbrwJ3uH0 X-Received: by 2002:adf:ea11:: with SMTP id q17mr3146826wrm.75.1591788968132; Wed, 10 Jun 2020 04:36:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw+0YhjkTKfGeypFjYmQw4xelfruV1Ntm2M/rNlskwCxXmVD04EuumSwnMG+QPQp8710QWBNA== X-Received: by 2002:adf:ea11:: with SMTP id q17mr3146806wrm.75.1591788967929; Wed, 10 Jun 2020 04:36:07 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id a16sm7674114wrx.8.2020.06.10.04.36.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:07 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:05 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 04/14] vhost/net: pass net specific struct pointer Message-ID: <20200610113515.1497099-5-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org In preparation for further cleanup, pass net specific pointer to ubuf callbacks so we can move net specific fields out to net structures. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/net.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index bf5e1d81ae25..ff594eec8ae3 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -94,7 +94,7 @@ struct vhost_net_ubuf_ref { */ atomic_t refcount; wait_queue_head_t wait; - struct vhost_virtqueue *vq; + struct vhost_net_virtqueue *nvq; }; #define VHOST_NET_BATCH 64 @@ -231,7 +231,7 @@ static void vhost_net_enable_zcopy(int vq) } static struct vhost_net_ubuf_ref * -vhost_net_ubuf_alloc(struct vhost_virtqueue *vq, bool zcopy) +vhost_net_ubuf_alloc(struct vhost_net_virtqueue *nvq, bool zcopy) { struct vhost_net_ubuf_ref *ubufs; /* No zero copy backend? Nothing to count. */ @@ -242,7 +242,7 @@ vhost_net_ubuf_alloc(struct vhost_virtqueue *vq, bool zcopy) return ERR_PTR(-ENOMEM); atomic_set(&ubufs->refcount, 1); init_waitqueue_head(&ubufs->wait); - ubufs->vq = vq; + ubufs->nvq = nvq; return ubufs; } @@ -384,13 +384,13 @@ static void vhost_zerocopy_signal_used(struct vhost_net *net, static void vhost_zerocopy_callback(struct ubuf_info *ubuf, bool success) { struct vhost_net_ubuf_ref *ubufs = ubuf->ctx; - struct vhost_virtqueue *vq = ubufs->vq; + struct vhost_net_virtqueue *nvq = ubufs->nvq; int cnt; rcu_read_lock_bh(); /* set len to mark this desc buffers done DMA */ - vq->heads[ubuf->desc].len = success ? + nvq->vq.heads[ubuf->desc].in_len = success ? VHOST_DMA_DONE_LEN : VHOST_DMA_FAILED_LEN; cnt = vhost_net_ubuf_put(ubufs); @@ -402,7 +402,7 @@ static void vhost_zerocopy_callback(struct ubuf_info *ubuf, bool success) * less than 10% of times). */ if (cnt <= 1 || !(cnt % 16)) - vhost_poll_queue(&vq->poll); + vhost_poll_queue(&nvq->vq.poll); rcu_read_unlock_bh(); } @@ -1525,7 +1525,7 @@ static long vhost_net_set_backend(struct vhost_net *n, unsigned index, int fd) /* start polling new socket */ oldsock = vhost_vq_get_backend(vq); if (sock != oldsock) { - ubufs = vhost_net_ubuf_alloc(vq, + ubufs = vhost_net_ubuf_alloc(nvq, sock && vhost_sock_zcopy(sock)); if (IS_ERR(ubufs)) { r = PTR_ERR(ubufs); From patchwork Wed Jun 10 11:36:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597563 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 00D7D739 for ; Wed, 10 Jun 2020 11:38:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D88C420760 for ; Wed, 10 Jun 2020 11:38:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XR1X0qwm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728544AbgFJLiF (ORCPT ); Wed, 10 Jun 2020 07:38:05 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:55191 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728486AbgFJLgQ (ORCPT ); Wed, 10 Jun 2020 07:36:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=+SD3Vrt7U6HArLpzqOyIxuZrc1fS29Ei8HX5xaDKKOw=; b=XR1X0qwmqNbKqPwY25Jrq/T/gCXJBJtu8LeAEFSbEcKQdDbKDXoMejKUTIdLNJxYYwckwA 5wrkiDrvlxpcgqO22fmEkRYCll1SwRXYTy8qjk3mhacVkc9FrxltBz3UZ6ZAC5c0bCiupY dVvHc7wuv0Uq20VoALRGrKIuX0iVr3U= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-230-BdL4UjK7PpS--xahWFXlYg-1; Wed, 10 Jun 2020 07:36:12 -0400 X-MC-Unique: BdL4UjK7PpS--xahWFXlYg-1 Received: by mail-wr1-f69.google.com with SMTP id l18so979878wrm.0 for ; Wed, 10 Jun 2020 04:36:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=+SD3Vrt7U6HArLpzqOyIxuZrc1fS29Ei8HX5xaDKKOw=; b=eiCfBIkoWlmNQr9J3erLSB8/u6NarRw7+V/82KSeRxuueHPZ5GUlO30wjD+t/RrPRW xjN3qpqp2pMMQcXV8uZ3WA9x7CXQUpIKsp0lKfeRVs8s6h1QClvwXUWJ0zzSRhfScN8Y avad/nN7yo+vwJhiz9rjCIINkLw/i0m1bl04HjjSSWub5RrOXpkNJK+6cvrMyG6SNrm9 6EK6nNxYbx73MFZiYdTVt8dob3LGpctX6s02ByajUuKMsbN0Zh+fY3VIoBnnPzGNxOpM /ReL3hIa2HYF2U5H4ScAqrQTqAjC00voSDY8to/eDPaYcY+l/wqykqKGyyEWTqoz1ZM6 2wVg== X-Gm-Message-State: AOAM531D6Ojab2ecWPGq6sjGIHFKzF+lQPdFES8O1nxETINWZfSYFbei M2w7Qd6ZYVypAxuYai0iZu4tMw5eWdiF15B5OeMuXC3U903iZuGnVMwvagrukmkz1Yp5vA7WX+r xI+6pcA4ABhlV X-Received: by 2002:a1c:22d7:: with SMTP id i206mr2755087wmi.186.1591788970805; Wed, 10 Jun 2020 04:36:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxemEaCHMgCrHg7/oKSQ4yFnNXg1fLjZ5OHmMHvyeRIUoWRpQOB/0A8RO0HmZnjlr0fN5RKtA== X-Received: by 2002:a1c:22d7:: with SMTP id i206mr2755070wmi.186.1591788970615; Wed, 10 Jun 2020 04:36:10 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id a1sm6866225wmd.28.2020.06.10.04.36.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:10 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:08 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 05/14] vhost: reorder functions Message-ID: <20200610113515.1497099-6-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Reorder functions in the file to not rely on forward declarations, in preparation to making them static down the road. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/vhost.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 28f324fd77df..506208b63126 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2429,19 +2429,6 @@ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) } EXPORT_SYMBOL_GPL(vhost_discard_vq_desc); -/* After we've used one of their buffers, we tell them about it. We'll then - * want to notify the guest, using eventfd. */ -int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) -{ - struct vring_used_elem heads = { - cpu_to_vhost32(vq, head), - cpu_to_vhost32(vq, len) - }; - - return vhost_add_used_n(vq, &heads, 1); -} -EXPORT_SYMBOL_GPL(vhost_add_used); - static int __vhost_add_used_n(struct vhost_virtqueue *vq, struct vring_used_elem *heads, unsigned count) @@ -2511,6 +2498,19 @@ int vhost_add_used_n(struct vhost_virtqueue *vq, struct vring_used_elem *heads, } EXPORT_SYMBOL_GPL(vhost_add_used_n); +/* After we've used one of their buffers, we tell them about it. We'll then + * want to notify the guest, using eventfd. */ +int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) +{ + struct vring_used_elem heads = { + cpu_to_vhost32(vq, head), + cpu_to_vhost32(vq, len) + }; + + return vhost_add_used_n(vq, &heads, 1); +} +EXPORT_SYMBOL_GPL(vhost_add_used); + static bool vhost_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq) { __u16 old, new; From patchwork Wed Jun 10 11:36:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597559 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8896739 for ; Wed, 10 Jun 2020 11:37:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BB7112078C for ; Wed, 10 Jun 2020 11:37:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bVFmEACu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728675AbgFJLg0 (ORCPT ); Wed, 10 Jun 2020 07:36:26 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:23353 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728628AbgFJLgT (ORCPT ); Wed, 10 Jun 2020 07:36:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788976; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=eiJhq6XkiotudOh+ZUxsjtY7w6RhDlW9rQ8WDNYh8GQ=; b=bVFmEACuVOQxdotsJMN90zuG3AsNEZ3qb7hhAYac7EB2GWc2447aE9YVxQ5kg6Y/ZF9OON dfrZn26upCXzu2/0O9TGvRyomeJ0+4mBW5SrFPtEtjyrZrN9uCHala0mx6F2k2Ir6Rsx3U nJ8S5jNsIaVhIDv+YAHfg8TYD0cxShc= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-118-IOvz8mdxM3W7Kp_8o-ATyQ-1; Wed, 10 Jun 2020 07:36:15 -0400 X-MC-Unique: IOvz8mdxM3W7Kp_8o-ATyQ-1 Received: by mail-wm1-f71.google.com with SMTP id x6so399736wmj.9 for ; Wed, 10 Jun 2020 04:36:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=eiJhq6XkiotudOh+ZUxsjtY7w6RhDlW9rQ8WDNYh8GQ=; b=ipGG6WAOD/CPghX6SBUOhU2PE9TW+L4uvH4PxG+u6IH4dq5lTOOR3wgUFl+yO9moG3 RZPTCV26tYqk1iRjNxbjcC+6YY+fW1G0xfZS1c4zB0v9Uk6engAtdzqXV1RUfr5K4Jov m9CMWqUaXbhlWc97XtMQ0vVzW4pOUgT2iIkSKxA49VRkFNlJvnDHf8h1csFXJPeN1xG7 sEiTG4qVqGuJGOiOovPeuXRVHofvJfjdY7V3sdU0PZoVt6ohZIxXhxWBqTUhTWGTj8gf RDBOVRpKgC8Epz6Y+808DLTigQY41R0Xo2DbALzl23Ydrlyr93uapP/dqrFD50bqeVxK 096Q== X-Gm-Message-State: AOAM531SC9JBDhRvRR9BGXE4ZbJ4FQNWFGUOVbA3eXdL5yoCpWHayrj7 Ri/pifAAxt4kgkumkv1KuxG2tCNhH8mJF3O/0OLxthwN4SacEs78FD4HE+DiPjsIPPWhKIhGtTi sssrQeByUVjq2 X-Received: by 2002:a05:6000:90:: with SMTP id m16mr3301318wrx.191.1591788973797; Wed, 10 Jun 2020 04:36:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw3FpICO0VZ3PPjidPQ1UtjzBsY39xeOmErAI9iBPh+xj1HDRPcA/UbjrzbUg24Tj112L9uRg== X-Received: by 2002:a05:6000:90:: with SMTP id m16mr3301287wrx.191.1591788973510; Wed, 10 Jun 2020 04:36:13 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id b132sm6626772wmh.3.2020.06.10.04.36.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:13 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:10 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 06/14] vhost: format-independent API for used buffers Message-ID: <20200610113515.1497099-7-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a new API that doesn't assume used ring, heads, etc. For now, we keep the old APIs around to make it easier to convert drivers. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/vhost.c | 52 ++++++++++++++++++++++++++++++++++--------- drivers/vhost/vhost.h | 17 +++++++++++++- 2 files changed, 58 insertions(+), 11 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 506208b63126..e5763d81bf0f 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2339,13 +2339,12 @@ static void unfetch_descs(struct vhost_virtqueue *vq) * number of output then some number of input descriptors, it's actually two * iovecs, but we pack them into one and note how many of each there were. * - * This function returns the descriptor number found, or vq->num (which is - * never a valid descriptor number) if none was found. A negative code is - * returned on error. */ -int vhost_get_vq_desc(struct vhost_virtqueue *vq, - struct iovec iov[], unsigned int iov_size, - unsigned int *out_num, unsigned int *in_num, - struct vhost_log *log, unsigned int *log_num) + * This function returns a value > 0 if a descriptor was found, or 0 if none were found. + * A negative code is returned on error. */ +int vhost_get_avail_buf(struct vhost_virtqueue *vq, struct vhost_buf *buf, + struct iovec iov[], unsigned int iov_size, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num) { int ret = fetch_descs(vq); int i; @@ -2358,6 +2357,8 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, *out_num = *in_num = 0; if (unlikely(log)) *log_num = 0; + buf->in_len = buf->out_len = 0; + buf->descs = 0; for (i = vq->first_desc; i < vq->ndescs; ++i) { unsigned iov_count = *in_num + *out_num; @@ -2387,6 +2388,7 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, /* If this is an input descriptor, * increment that count. */ *in_num += ret; + buf->in_len += desc->len; if (unlikely(log && ret)) { log[*log_num].addr = desc->addr; log[*log_num].len = desc->len; @@ -2402,9 +2404,11 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, goto err; } *out_num += ret; + buf->out_len += desc->len; } - ret = desc->id; + buf->id = desc->id; + ++buf->descs; if (!(desc->flags & VRING_DESC_F_NEXT)) break; @@ -2412,14 +2416,22 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, vq->first_desc = i + 1; - return ret; + return 1; err: unfetch_descs(vq); return ret ? ret : vq->num; } -EXPORT_SYMBOL_GPL(vhost_get_vq_desc); +EXPORT_SYMBOL_GPL(vhost_get_avail_buf); + +/* Reverse the effect of vhost_get_avail_buf. Useful for error handling. */ +void vhost_discard_avail_bufs(struct vhost_virtqueue *vq, + struct vhost_buf *buf, unsigned count) +{ + vhost_discard_vq_desc(vq, count); +} +EXPORT_SYMBOL_GPL(vhost_discard_avail_bufs); /* Reverse the effect of vhost_get_vq_desc. Useful for error handling. */ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) @@ -2511,6 +2523,26 @@ int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) } EXPORT_SYMBOL_GPL(vhost_add_used); +int vhost_put_used_buf(struct vhost_virtqueue *vq, struct vhost_buf *buf) +{ + return vhost_add_used(vq, buf->id, buf->in_len); +} +EXPORT_SYMBOL_GPL(vhost_put_used_buf); + +int vhost_put_used_n_bufs(struct vhost_virtqueue *vq, + struct vhost_buf *bufs, unsigned count) +{ + unsigned i; + + for (i = 0; i < count; ++i) { + vq->heads[i].id = cpu_to_vhost32(vq, bufs[i].id); + vq->heads[i].len = cpu_to_vhost32(vq, bufs[i].in_len); + } + + return vhost_add_used_n(vq, vq->heads, count); +} +EXPORT_SYMBOL_GPL(vhost_put_used_n_bufs); + static bool vhost_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq) { __u16 old, new; diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index fed36af5c444..28eea0155efb 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -67,6 +67,13 @@ struct vhost_desc { u16 id; }; +struct vhost_buf { + u32 out_len; + u32 in_len; + u16 descs; + u16 id; +}; + /* The virtqueue structure describes a queue attached to a device. */ struct vhost_virtqueue { struct vhost_dev *dev; @@ -195,7 +202,12 @@ int vhost_get_vq_desc(struct vhost_virtqueue *, unsigned int *out_num, unsigned int *in_num, struct vhost_log *log, unsigned int *log_num); void vhost_discard_vq_desc(struct vhost_virtqueue *, int n); - +int vhost_get_avail_buf(struct vhost_virtqueue *, struct vhost_buf *buf, + struct iovec iov[], unsigned int iov_count, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num); +void vhost_discard_avail_bufs(struct vhost_virtqueue *, + struct vhost_buf *, unsigned count); int vhost_vq_init_access(struct vhost_virtqueue *); int vhost_add_used(struct vhost_virtqueue *, unsigned int head, int len); int vhost_add_used_n(struct vhost_virtqueue *, struct vring_used_elem *heads, @@ -204,6 +216,9 @@ void vhost_add_used_and_signal(struct vhost_dev *, struct vhost_virtqueue *, unsigned int id, int len); void vhost_add_used_and_signal_n(struct vhost_dev *, struct vhost_virtqueue *, struct vring_used_elem *heads, unsigned count); +int vhost_put_used_buf(struct vhost_virtqueue *, struct vhost_buf *buf); +int vhost_put_used_n_bufs(struct vhost_virtqueue *, + struct vhost_buf *bufs, unsigned count); void vhost_signal(struct vhost_dev *, struct vhost_virtqueue *); void vhost_disable_notify(struct vhost_dev *, struct vhost_virtqueue *); bool vhost_vq_avail_empty(struct vhost_dev *, struct vhost_virtqueue *); From patchwork Wed Jun 10 11:36:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597545 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7049613B1 for ; Wed, 10 Jun 2020 11:36:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5828E2074B for ; Wed, 10 Jun 2020 11:36:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="e4exZrjI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728691AbgFJLg1 (ORCPT ); Wed, 10 Jun 2020 07:36:27 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:21952 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728642AbgFJLgU (ORCPT ); Wed, 10 Jun 2020 07:36:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788979; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=g1mhrQ6NM5wYShJMoCPQWA/YwUtZgT5jddxZW0yxgs8=; b=e4exZrjI9HtnU7rNPcCffQSp6dcxNVzPTIOk9WOlEhevQmVFckVdD6wwrPz4beGBi294ot GjFTWLDX72WBmnpB0hJPN9LiHTo0iVrvnEclvOeUEX+xGkA7PpqXqYrhzlVXgqwx2LDFxi aDoHlxPegSPHT/43Gw5cVqlu6RSSu3w= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-370-5HhCwuLwPtykJnzcYNG_Wg-1; Wed, 10 Jun 2020 07:36:17 -0400 X-MC-Unique: 5HhCwuLwPtykJnzcYNG_Wg-1 Received: by mail-wr1-f71.google.com with SMTP id i6so952028wrr.23 for ; Wed, 10 Jun 2020 04:36:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=g1mhrQ6NM5wYShJMoCPQWA/YwUtZgT5jddxZW0yxgs8=; b=WHU9ZEGGq6vd/OQ+FcaulB0GmQb2yOtUqrsfQlgHecMcVyvwX/uQJS2rqJNqagwZTn d8CTwai3IUfCgiwatJ0n6HzkzSBF4dQ826OFg8P/VuR/tJKAU7q2UJwH5raIPt1oP/UD cnvpGL4QHjDGgofT+iRjdIktq1ZqhI2Omv4I7AP5b9hwPoLbKWNOki7lGODUvqC2qmwe VptHSVY/+0ckRHv7SKoZK33CfDWJorlj2jpnKnBILvAOiy4NwlQ0YyhfatVuoWff6Muc C2Kkg79Iuvm1WTKAgwq4vS627HUgB7dyfEp67Xfga1126WG2MWr48C6su6YxYSQqkIuE ajvA== X-Gm-Message-State: AOAM531eezsouDJ9rdtlVVeiPhI78GmMWUCXnUHf+J8p9Uz1f3J6aSaD C5p23u2I2SAU34H2uniGad2fdXh6E15Lhm2ZKmQhi2YGCiMs6iITFcj5Xf77Uul69XvJjxJ3ZHD lmDje7OFQl2YR X-Received: by 2002:a1c:254:: with SMTP id 81mr2746858wmc.93.1591788976421; Wed, 10 Jun 2020 04:36:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxFkt3Ubc8PKFjtb6gY/lDXcBZnr6E3I6m79bNAQFz95Fybj8NNv1w+aulFwppsq1zCYoyKZw== X-Received: by 2002:a1c:254:: with SMTP id 81mr2746830wmc.93.1591788976169; Wed, 10 Jun 2020 04:36:16 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id z22sm6583776wmf.9.2020.06.10.04.36.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:15 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:13 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 07/14] fixup! vhost: format-independent API for used buffers Message-ID: <20200610113515.1497099-8-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org --- drivers/vhost/vhost.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index e5763d81bf0f..7a587b13095c 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2421,7 +2421,7 @@ int vhost_get_avail_buf(struct vhost_virtqueue *vq, struct vhost_buf *buf, err: unfetch_descs(vq); - return ret ? ret : vq->num; + return ret; } EXPORT_SYMBOL_GPL(vhost_get_avail_buf); @@ -2433,6 +2433,27 @@ void vhost_discard_avail_bufs(struct vhost_virtqueue *vq, } EXPORT_SYMBOL_GPL(vhost_discard_avail_bufs); +/* This function returns the descriptor number found, or vq->num (which is + * never a valid descriptor number) if none was found. A negative code is + * returned on error. */ +int vhost_get_vq_desc(struct vhost_virtqueue *vq, + struct iovec iov[], unsigned int iov_size, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num) +{ + struct vhost_buf buf; + int ret = vhost_get_avail_buf(vq, &buf, + iov, iov_size, out_num, in_num, + log, log_num); + + if (likely(ret > 0)) + return buf->id; + if (likely(!ret)) + return vq->num; + return ret; +} +EXPORT_SYMBOL_GPL(vhost_get_vq_desc); + /* Reverse the effect of vhost_get_vq_desc. Useful for error handling. */ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) { From patchwork Wed Jun 10 11:36:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597561 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B275614B7 for ; Wed, 10 Jun 2020 11:37:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9A20E20812 for ; Wed, 10 Jun 2020 11:37:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="PD3DXviU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728873AbgFJLho (ORCPT ); Wed, 10 Jun 2020 07:37:44 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:32159 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728480AbgFJLgW (ORCPT ); Wed, 10 Jun 2020 07:36:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788982; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nqpRKPPRu97i04+lEwiVZnVP4RAISD6OyeyY71/1iaw=; b=PD3DXviUIXZSvLoLAGuF692PzUijlWhppV55ZYxp5ZvJhdv/oOyKxnqycdlsIKlAl05Ro9 vJXmZKpt9/tjiRcITzJF7kZ+lqnPFaDR6sRdKl9FT8WvPsPF/3XlmhxsbBVFUDAcD1u2C4 Q8BCO1YTt5uaGYZRw8wu8KRy9OYKJWc= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-358-bN5wehGAMTKchmSsljszBQ-1; Wed, 10 Jun 2020 07:36:20 -0400 X-MC-Unique: bN5wehGAMTKchmSsljszBQ-1 Received: by mail-wr1-f72.google.com with SMTP id s17so963015wrt.7 for ; Wed, 10 Jun 2020 04:36:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=nqpRKPPRu97i04+lEwiVZnVP4RAISD6OyeyY71/1iaw=; b=ZwWNeGlHqxHgdpFM7UlULtV5OIB1MWNMXPvKHS0vekGKDSxtutRDAqNkMlDrnGcrux cKpbHmZ7nijSTuyjuoqz2cYu8uIygmdjvxN9DKRDZOUlMBc7NkIboG6iascfi8P2lEw2 QK4tlpmimYMM3XJrQ2kwHhpRNWM7W7LY372SWoJE41kLS9NX80+56z8MYousII28sDIm kKCTpEzVaNnrdruyRHLMrpcbtBSPr8AyW9ycasNQGl1TcFT2Jmk0KpgaP2XnCMdPQlCw zA2dHev6MSY2jD1wrPFK3y3oRQTjKh/Yc/G/G0jpg7c7mowatv6D3xRhaKjx/ojhzEoy e7RA== X-Gm-Message-State: AOAM531AVxtvYISjDNuIF2GIwBydC5yyMGyyB/0G4w8B8EjW2+vZtOgP GTQSOkj8JxldpIeqkn60vQWC6yUTvmARulfSYLwUc+hDbP7dam5xhZ3aJiXawcNoxCJly7eKYG4 TjfF30dVAGuaO X-Received: by 2002:a7b:cc94:: with SMTP id p20mr2749315wma.148.1591788979279; Wed, 10 Jun 2020 04:36:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxk/lBSoKqQEzUm0ygb+JThKfl+fyQkS7QcmVzzuz7sF0aMX0Q8Jh3FrRflgePDOX+1hNCqVA== X-Received: by 2002:a7b:cc94:: with SMTP id p20mr2749304wma.148.1591788979097; Wed, 10 Jun 2020 04:36:19 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id d63sm6775894wmc.22.2020.06.10.04.36.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:18 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:16 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 08/14] fixup! vhost: use batched get_vq_desc version Message-ID: <20200610113515.1497099-9-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org --- drivers/vhost/vhost.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 7a587b13095c..03e6bca02288 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2205,10 +2205,6 @@ static int fetch_buf(struct vhost_virtqueue *vq) last_avail_idx = vq->last_avail_idx; if (unlikely(vq->avail_idx == vq->last_avail_idx)) { - /* If we already have work to do, don't bother re-checking. */ - if (likely(vq->ndescs)) - return 1; - if (unlikely(vhost_get_avail_idx(vq, &avail_idx))) { vq_err(vq, "Failed to access avail idx at %p\n", &vq->avail->idx); From patchwork Wed Jun 10 11:36:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597555 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56358739 for ; Wed, 10 Jun 2020 11:37:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2D49B2074B for ; Wed, 10 Jun 2020 11:37:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WGqLrzDR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728817AbgFJLhP (ORCPT ); Wed, 10 Jun 2020 07:37:15 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:52614 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728671AbgFJLg2 (ORCPT ); Wed, 10 Jun 2020 07:36:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788985; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=UNoiNEJvdgOZ9xjiajXEUBmq7bdVAGmYmXBWji52Kn0=; b=WGqLrzDRLYJ3gfzYY7qoYGEGoZ8W61bqEelkMBeUg1Gwb9+sqyLqoohBcd9/Ndj/0gReY3 dX4F8xJGfMXZSEvPIDTLCRbniIf3cWMWCza01e6HhQWSD/WIx1ejozwRsBJUM43QLsxcHc DBtu43GUnB6IZYcvFIBC5o/ZdcrVHyY= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-454-m3WcJAVCNTGhJ4IkbFlpgA-1; Wed, 10 Jun 2020 07:36:24 -0400 X-MC-Unique: m3WcJAVCNTGhJ4IkbFlpgA-1 Received: by mail-wr1-f72.google.com with SMTP id l18so980088wrm.0 for ; Wed, 10 Jun 2020 04:36:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=UNoiNEJvdgOZ9xjiajXEUBmq7bdVAGmYmXBWji52Kn0=; b=PA0ZP4B+jjqJb+zn9pyQAMES1ZoQtEsaR+9czRIeXTkjGSwFN3eBvxB3wrhWEh81Ys UJIPa/Cj7lTHhdkphEzw2tED0gZRix3CZzXRjjyNTYAfDU/b9w2cvnkKoEWBl3BUKu4q sg+dAGgLAt4TLo+Tk6U+MIEnVzQm0tdvwHo+Eb/G72R2Nb+rRAO8caQz+5pxldA5bJ1j PDcG+s8YEtbtIh9O3rk8o2uyeMg5CdGYZqXf5qrQCXyxIWuMheGIxmKNcqRv+iR8jYOb 05toRTZIExmtT0AkEg/IRBRm8rvzNLW8rnr+u7ycHQyozAHmYr4bF4xsYb7aznFB00d1 HcYw== X-Gm-Message-State: AOAM530Uj7CLJM34HzomLYd9bOraXYjKGUaCO43wxm6S8F55dOMFt8Pp rd03NMH6Wu2KKvdVOsTS+H62UYU32MsUxKZmGuE2MRX0MpQkHrjm85eKLvC74kE+fB8np4kTiQp ycOP3hbTr8Fnc X-Received: by 2002:a5d:440c:: with SMTP id z12mr3268424wrq.241.1591788982303; Wed, 10 Jun 2020 04:36:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwXGUgH38dPw7JvUUtamSsjhEq0TgT1zn0wn1ZH4IjU6BlsD9AgrHYHsB3Wss1dcFdfVl3Zig== X-Received: by 2002:a5d:440c:: with SMTP id z12mr3268375wrq.241.1591788981792; Wed, 10 Jun 2020 04:36:21 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id e5sm7970763wrw.19.2020.06.10.04.36.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:21 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:19 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 09/14] vhost/net: convert to new API: heads->bufs Message-ID: <20200610113515.1497099-10-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Convert vhost net to use the new format-agnostic API. In particular, don't poke at vq internals such as the heads array. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/net.c | 154 +++++++++++++++++++++++--------------------- 1 file changed, 82 insertions(+), 72 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index ff594eec8ae3..830fe84912a5 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -59,13 +59,13 @@ MODULE_PARM_DESC(experimental_zcopytx, "Enable Zero Copy TX;" * status internally; used for zerocopy tx only. */ /* Lower device DMA failed */ -#define VHOST_DMA_FAILED_LEN ((__force __virtio32)3) +#define VHOST_DMA_FAILED_LEN (3) /* Lower device DMA done */ -#define VHOST_DMA_DONE_LEN ((__force __virtio32)2) +#define VHOST_DMA_DONE_LEN (2) /* Lower device DMA in progress */ -#define VHOST_DMA_IN_PROGRESS ((__force __virtio32)1) +#define VHOST_DMA_IN_PROGRESS (1) /* Buffer unused */ -#define VHOST_DMA_CLEAR_LEN ((__force __virtio32)0) +#define VHOST_DMA_CLEAR_LEN (0) #define VHOST_DMA_IS_DONE(len) ((__force u32)(len) >= (__force u32)VHOST_DMA_DONE_LEN) @@ -112,9 +112,12 @@ struct vhost_net_virtqueue { /* last used idx for outstanding DMA zerocopy buffers */ int upend_idx; /* For TX, first used idx for DMA done zerocopy buffers - * For RX, number of batched heads + * For RX, number of batched bufs */ int done_idx; + /* Outstanding user bufs. UIO_MAXIOV in length. */ + /* TODO: we can make this smaller for sure. */ + struct vhost_buf *bufs; /* Number of XDP frames batched */ int batched_xdp; /* an array of userspace buffers info */ @@ -271,6 +274,8 @@ static void vhost_net_clear_ubuf_info(struct vhost_net *n) int i; for (i = 0; i < VHOST_NET_VQ_MAX; ++i) { + kfree(n->vqs[i].bufs); + n->vqs[i].bufs = NULL; kfree(n->vqs[i].ubuf_info); n->vqs[i].ubuf_info = NULL; } @@ -282,6 +287,12 @@ static int vhost_net_set_ubuf_info(struct vhost_net *n) int i; for (i = 0; i < VHOST_NET_VQ_MAX; ++i) { + n->vqs[i].bufs = kmalloc_array(UIO_MAXIOV, + sizeof(*n->vqs[i].bufs), + GFP_KERNEL); + if (!n->vqs[i].bufs) + goto err; + zcopy = vhost_net_zcopy_mask & (0x1 << i); if (!zcopy) continue; @@ -364,18 +375,18 @@ static void vhost_zerocopy_signal_used(struct vhost_net *net, int j = 0; for (i = nvq->done_idx; i != nvq->upend_idx; i = (i + 1) % UIO_MAXIOV) { - if (vq->heads[i].len == VHOST_DMA_FAILED_LEN) + if (nvq->bufs[i].in_len == VHOST_DMA_FAILED_LEN) vhost_net_tx_err(net); - if (VHOST_DMA_IS_DONE(vq->heads[i].len)) { - vq->heads[i].len = VHOST_DMA_CLEAR_LEN; + if (VHOST_DMA_IS_DONE(nvq->bufs[i].in_len)) { + nvq->bufs[i].in_len = VHOST_DMA_CLEAR_LEN; ++j; } else break; } while (j) { add = min(UIO_MAXIOV - nvq->done_idx, j); - vhost_add_used_and_signal_n(vq->dev, vq, - &vq->heads[nvq->done_idx], add); + vhost_put_used_n_bufs(vq, &nvq->bufs[nvq->done_idx], add); + vhost_signal(vq->dev, vq); nvq->done_idx = (nvq->done_idx + add) % UIO_MAXIOV; j -= add; } @@ -390,7 +401,7 @@ static void vhost_zerocopy_callback(struct ubuf_info *ubuf, bool success) rcu_read_lock_bh(); /* set len to mark this desc buffers done DMA */ - nvq->vq.heads[ubuf->desc].in_len = success ? + nvq->bufs[ubuf->desc].in_len = success ? VHOST_DMA_DONE_LEN : VHOST_DMA_FAILED_LEN; cnt = vhost_net_ubuf_put(ubufs); @@ -452,7 +463,8 @@ static void vhost_net_signal_used(struct vhost_net_virtqueue *nvq) if (!nvq->done_idx) return; - vhost_add_used_and_signal_n(dev, vq, vq->heads, nvq->done_idx); + vhost_put_used_n_bufs(vq, nvq->bufs, nvq->done_idx); + vhost_signal(dev, vq); nvq->done_idx = 0; } @@ -558,6 +570,7 @@ static void vhost_net_busy_poll(struct vhost_net *net, static int vhost_net_tx_get_vq_desc(struct vhost_net *net, struct vhost_net_virtqueue *tnvq, + struct vhost_buf *buf, unsigned int *out_num, unsigned int *in_num, struct msghdr *msghdr, bool *busyloop_intr) { @@ -565,10 +578,10 @@ static int vhost_net_tx_get_vq_desc(struct vhost_net *net, struct vhost_virtqueue *rvq = &rnvq->vq; struct vhost_virtqueue *tvq = &tnvq->vq; - int r = vhost_get_vq_desc(tvq, tvq->iov, ARRAY_SIZE(tvq->iov), - out_num, in_num, NULL, NULL); + int r = vhost_get_avail_buf(tvq, buf, tvq->iov, ARRAY_SIZE(tvq->iov), + out_num, in_num, NULL, NULL); - if (r == tvq->num && tvq->busyloop_timeout) { + if (!r && tvq->busyloop_timeout) { /* Flush batched packets first */ if (!vhost_sock_zcopy(vhost_vq_get_backend(tvq))) vhost_tx_batch(net, tnvq, @@ -577,8 +590,8 @@ static int vhost_net_tx_get_vq_desc(struct vhost_net *net, vhost_net_busy_poll(net, rvq, tvq, busyloop_intr, false); - r = vhost_get_vq_desc(tvq, tvq->iov, ARRAY_SIZE(tvq->iov), - out_num, in_num, NULL, NULL); + r = vhost_get_avail_buf(tvq, buf, tvq->iov, ARRAY_SIZE(tvq->iov), + out_num, in_num, NULL, NULL); } return r; @@ -607,6 +620,7 @@ static size_t init_iov_iter(struct vhost_virtqueue *vq, struct iov_iter *iter, static int get_tx_bufs(struct vhost_net *net, struct vhost_net_virtqueue *nvq, + struct vhost_buf *buf, struct msghdr *msg, unsigned int *out, unsigned int *in, size_t *len, bool *busyloop_intr) @@ -614,9 +628,9 @@ static int get_tx_bufs(struct vhost_net *net, struct vhost_virtqueue *vq = &nvq->vq; int ret; - ret = vhost_net_tx_get_vq_desc(net, nvq, out, in, msg, busyloop_intr); + ret = vhost_net_tx_get_vq_desc(net, nvq, buf, out, in, msg, busyloop_intr); - if (ret < 0 || ret == vq->num) + if (ret <= 0) return ret; if (*in) { @@ -761,7 +775,7 @@ static void handle_tx_copy(struct vhost_net *net, struct socket *sock) struct vhost_net_virtqueue *nvq = &net->vqs[VHOST_NET_VQ_TX]; struct vhost_virtqueue *vq = &nvq->vq; unsigned out, in; - int head; + int ret; struct msghdr msg = { .msg_name = NULL, .msg_namelen = 0, @@ -773,6 +787,7 @@ static void handle_tx_copy(struct vhost_net *net, struct socket *sock) int err; int sent_pkts = 0; bool sock_can_batch = (sock->sk->sk_sndbuf == INT_MAX); + struct vhost_buf buf; do { bool busyloop_intr = false; @@ -780,13 +795,13 @@ static void handle_tx_copy(struct vhost_net *net, struct socket *sock) if (nvq->done_idx == VHOST_NET_BATCH) vhost_tx_batch(net, nvq, sock, &msg); - head = get_tx_bufs(net, nvq, &msg, &out, &in, &len, - &busyloop_intr); + ret = get_tx_bufs(net, nvq, &buf, &msg, &out, &in, &len, + &busyloop_intr); /* On error, stop handling until the next kick. */ - if (unlikely(head < 0)) + if (unlikely(ret < 0)) break; /* Nothing new? Wait for eventfd to tell us they refilled. */ - if (head == vq->num) { + if (!ret) { if (unlikely(busyloop_intr)) { vhost_poll_queue(&vq->poll); } else if (unlikely(vhost_enable_notify(&net->dev, @@ -808,7 +823,7 @@ static void handle_tx_copy(struct vhost_net *net, struct socket *sock) goto done; } else if (unlikely(err != -ENOSPC)) { vhost_tx_batch(net, nvq, sock, &msg); - vhost_discard_vq_desc(vq, 1); + vhost_discard_avail_bufs(vq, &buf, 1); vhost_net_enable_vq(net, vq); break; } @@ -829,7 +844,7 @@ static void handle_tx_copy(struct vhost_net *net, struct socket *sock) /* TODO: Check specific error and bomb out unless ENOBUFS? */ err = sock->ops->sendmsg(sock, &msg, len); if (unlikely(err < 0)) { - vhost_discard_vq_desc(vq, 1); + vhost_discard_avail_bufs(vq, &buf, 1); vhost_net_enable_vq(net, vq); break; } @@ -837,8 +852,7 @@ static void handle_tx_copy(struct vhost_net *net, struct socket *sock) pr_debug("Truncated TX packet: len %d != %zd\n", err, len); done: - vq->heads[nvq->done_idx].id = cpu_to_vhost32(vq, head); - vq->heads[nvq->done_idx].len = 0; + nvq->bufs[nvq->done_idx] = buf; ++nvq->done_idx; } while (likely(!vhost_exceeds_weight(vq, ++sent_pkts, total_len))); @@ -850,7 +864,7 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock) struct vhost_net_virtqueue *nvq = &net->vqs[VHOST_NET_VQ_TX]; struct vhost_virtqueue *vq = &nvq->vq; unsigned out, in; - int head; + int ret; struct msghdr msg = { .msg_name = NULL, .msg_namelen = 0, @@ -864,6 +878,7 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock) struct vhost_net_ubuf_ref *uninitialized_var(ubufs); bool zcopy_used; int sent_pkts = 0; + struct vhost_buf buf; do { bool busyloop_intr; @@ -872,13 +887,13 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock) vhost_zerocopy_signal_used(net, vq); busyloop_intr = false; - head = get_tx_bufs(net, nvq, &msg, &out, &in, &len, - &busyloop_intr); + ret = get_tx_bufs(net, nvq, &buf, &msg, &out, &in, &len, + &busyloop_intr); /* On error, stop handling until the next kick. */ - if (unlikely(head < 0)) + if (unlikely(ret < 0)) break; /* Nothing new? Wait for eventfd to tell us they refilled. */ - if (head == vq->num) { + if (!ret) { if (unlikely(busyloop_intr)) { vhost_poll_queue(&vq->poll); } else if (unlikely(vhost_enable_notify(&net->dev, vq))) { @@ -897,8 +912,8 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock) struct ubuf_info *ubuf; ubuf = nvq->ubuf_info + nvq->upend_idx; - vq->heads[nvq->upend_idx].id = cpu_to_vhost32(vq, head); - vq->heads[nvq->upend_idx].len = VHOST_DMA_IN_PROGRESS; + nvq->bufs[nvq->upend_idx] = buf; + nvq->bufs[nvq->upend_idx].in_len = VHOST_DMA_IN_PROGRESS; ubuf->callback = vhost_zerocopy_callback; ubuf->ctx = nvq->ubufs; ubuf->desc = nvq->upend_idx; @@ -930,17 +945,19 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock) nvq->upend_idx = ((unsigned)nvq->upend_idx - 1) % UIO_MAXIOV; } - vhost_discard_vq_desc(vq, 1); + vhost_discard_avail_bufs(vq, &buf, 1); vhost_net_enable_vq(net, vq); break; } if (err != len) pr_debug("Truncated TX packet: " " len %d != %zd\n", err, len); - if (!zcopy_used) - vhost_add_used_and_signal(&net->dev, vq, head, 0); - else + if (!zcopy_used) { + vhost_put_used_buf(vq, &buf); + vhost_signal(&net->dev, vq); + } else { vhost_zerocopy_signal_used(net, vq); + } vhost_net_tx_packet(net); } while (likely(!vhost_exceeds_weight(vq, ++sent_pkts, total_len))); } @@ -1004,7 +1021,7 @@ static int vhost_net_rx_peek_head_len(struct vhost_net *net, struct sock *sk, int len = peek_head_len(rnvq, sk); if (!len && rvq->busyloop_timeout) { - /* Flush batched heads first */ + /* Flush batched bufs first */ vhost_net_signal_used(rnvq); /* Both tx vq and rx socket were polled here */ vhost_net_busy_poll(net, rvq, tvq, busyloop_intr, true); @@ -1022,11 +1039,11 @@ static int vhost_net_rx_peek_head_len(struct vhost_net *net, struct sock *sk, * @iovcount - returned count of io vectors we fill * @log - vhost log * @log_num - log offset - * @quota - headcount quota, 1 for big buffer - * returns number of buffer heads allocated, negative on error + * @quota - bufcount quota, 1 for big buffer + * returns number of buffers allocated, negative on error */ static int get_rx_bufs(struct vhost_virtqueue *vq, - struct vring_used_elem *heads, + struct vhost_buf *bufs, int datalen, unsigned *iovcount, struct vhost_log *log, @@ -1035,30 +1052,24 @@ static int get_rx_bufs(struct vhost_virtqueue *vq, { unsigned int out, in; int seg = 0; - int headcount = 0; - unsigned d; + int bufcount = 0; int r, nlogs = 0; /* len is always initialized before use since we are always called with * datalen > 0. */ u32 uninitialized_var(len); - while (datalen > 0 && headcount < quota) { + while (datalen > 0 && bufcount < quota) { if (unlikely(seg >= UIO_MAXIOV)) { r = -ENOBUFS; goto err; } - r = vhost_get_vq_desc(vq, vq->iov + seg, - ARRAY_SIZE(vq->iov) - seg, &out, - &in, log, log_num); - if (unlikely(r < 0)) + r = vhost_get_avail_buf(vq, bufs + bufcount, vq->iov + seg, + ARRAY_SIZE(vq->iov) - seg, &out, + &in, log, log_num); + if (unlikely(r <= 0)) goto err; - d = r; - if (d == vq->num) { - r = 0; - goto err; - } if (unlikely(out || in <= 0)) { vq_err(vq, "unexpected descriptor format for RX: " "out %d, in %d\n", out, in); @@ -1069,14 +1080,12 @@ static int get_rx_bufs(struct vhost_virtqueue *vq, nlogs += *log_num; log += *log_num; } - heads[headcount].id = cpu_to_vhost32(vq, d); len = iov_length(vq->iov + seg, in); - heads[headcount].len = cpu_to_vhost32(vq, len); datalen -= len; - ++headcount; + ++bufcount; seg += in; } - heads[headcount - 1].len = cpu_to_vhost32(vq, len + datalen); + bufs[bufcount - 1].in_len = len + datalen; *iovcount = seg; if (unlikely(log)) *log_num = nlogs; @@ -1086,9 +1095,9 @@ static int get_rx_bufs(struct vhost_virtqueue *vq, r = UIO_MAXIOV + 1; goto err; } - return headcount; + return bufcount; err: - vhost_discard_vq_desc(vq, headcount); + vhost_discard_avail_bufs(vq, bufs, bufcount); return r; } @@ -1113,7 +1122,7 @@ static void handle_rx(struct vhost_net *net) }; size_t total_len = 0; int err, mergeable; - s16 headcount; + int bufcount; size_t vhost_hlen, sock_hlen; size_t vhost_len, sock_len; bool busyloop_intr = false; @@ -1147,14 +1156,14 @@ static void handle_rx(struct vhost_net *net) break; sock_len += sock_hlen; vhost_len = sock_len + vhost_hlen; - headcount = get_rx_bufs(vq, vq->heads + nvq->done_idx, - vhost_len, &in, vq_log, &log, - likely(mergeable) ? UIO_MAXIOV : 1); + bufcount = get_rx_bufs(vq, nvq->bufs + nvq->done_idx, + vhost_len, &in, vq_log, &log, + likely(mergeable) ? UIO_MAXIOV : 1); /* On error, stop handling until the next kick. */ - if (unlikely(headcount < 0)) + if (unlikely(bufcount < 0)) goto out; /* OK, now we need to know about added descriptors. */ - if (!headcount) { + if (!bufcount) { if (unlikely(busyloop_intr)) { vhost_poll_queue(&vq->poll); } else if (unlikely(vhost_enable_notify(&net->dev, vq))) { @@ -1171,7 +1180,7 @@ static void handle_rx(struct vhost_net *net) if (nvq->rx_ring) msg.msg_control = vhost_net_buf_consume(&nvq->rxq); /* On overrun, truncate and discard */ - if (unlikely(headcount > UIO_MAXIOV)) { + if (unlikely(bufcount > UIO_MAXIOV)) { iov_iter_init(&msg.msg_iter, READ, vq->iov, 1, 1); err = sock->ops->recvmsg(sock, &msg, 1, MSG_DONTWAIT | MSG_TRUNC); @@ -1195,7 +1204,7 @@ static void handle_rx(struct vhost_net *net) if (unlikely(err != sock_len)) { pr_debug("Discarded rx packet: " " len %d, expected %zd\n", err, sock_len); - vhost_discard_vq_desc(vq, headcount); + vhost_discard_avail_bufs(vq, nvq->bufs + nvq->done_idx, bufcount); continue; } /* Supply virtio_net_hdr if VHOST_NET_F_VIRTIO_NET_HDR */ @@ -1214,15 +1223,15 @@ static void handle_rx(struct vhost_net *net) } /* TODO: Should check and handle checksum. */ - num_buffers = cpu_to_vhost16(vq, headcount); + num_buffers = cpu_to_vhost16(vq, bufcount); if (likely(mergeable) && copy_to_iter(&num_buffers, sizeof num_buffers, &fixup) != sizeof num_buffers) { vq_err(vq, "Failed num_buffers write"); - vhost_discard_vq_desc(vq, headcount); + vhost_discard_avail_bufs(vq, nvq->bufs + nvq->done_idx, bufcount); goto out; } - nvq->done_idx += headcount; + nvq->done_idx += bufcount; if (nvq->done_idx > VHOST_NET_BATCH) vhost_net_signal_used(nvq); if (unlikely(vq_log)) @@ -1314,6 +1323,7 @@ static int vhost_net_open(struct inode *inode, struct file *f) n->vqs[VHOST_NET_VQ_TX].vq.handle_kick = handle_tx_kick; n->vqs[VHOST_NET_VQ_RX].vq.handle_kick = handle_rx_kick; for (i = 0; i < VHOST_NET_VQ_MAX; i++) { + n->vqs[i].bufs = NULL; n->vqs[i].ubufs = NULL; n->vqs[i].ubuf_info = NULL; n->vqs[i].upend_idx = 0; From patchwork Wed Jun 10 11:36:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597557 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D17214B7 for ; Wed, 10 Jun 2020 11:37:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6509C20760 for ; Wed, 10 Jun 2020 11:37:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IBUFsf1e" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728871AbgFJLha (ORCPT ); Wed, 10 Jun 2020 07:37:30 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:25520 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728841AbgFJLh3 (ORCPT ); Wed, 10 Jun 2020 07:37:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591789047; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8yeTF/4xoJH8JkJEZ+aqTMVrfSsZomUNfl5tfcaaPRM=; b=IBUFsf1e6dMESgbXsywOBeXAykVt2rSXtJ05QpCJc28Jshk0RxVRSqzBDumXDR79fOMP0t UWge1GQ3w7HX2NyySsxAHtw1XuQSwwwxGLv2rODNfZpRHEEex8YMzYUa/JcyNZ2Ukkp18R Dn/JKa1GffjhczBueLLYbXWFZsIljsY= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-219-YWkeI5QFO8uzhDXY-YaOmg-1; Wed, 10 Jun 2020 07:37:25 -0400 X-MC-Unique: YWkeI5QFO8uzhDXY-YaOmg-1 Received: by mail-wr1-f69.google.com with SMTP id m14so961384wrj.12 for ; Wed, 10 Jun 2020 04:37:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=8yeTF/4xoJH8JkJEZ+aqTMVrfSsZomUNfl5tfcaaPRM=; b=TarAjs4iwwP7hnzy3EgDeEY1dSGyqFy6H8mX7h3ZUm6tru+Ew2sfwBOh7RUTJPQRYs UC6/Xpb2lzo5i/qwL/0R4FENxQIGyLlmwuWFLzqc6/lmAcHoLzK6VBJZpUhSocTvAM1z iPHnNkmrE2iJ3WRT2dx5Fux7KGOR/Pq9jSOyCU1BQmwKtiQDysBsOvXs1wJHvq112Xkn CK7lwwtETSMSV4Pu/xB0mx1CdO8V/4ePm3OO0KoEZLosWETIM5OoLbQGThEnx34n7jqs bZIAOQTg5UAXVoYTwxmFETNDgBA+WgZqQNAXBucgLYFAS9iejUzohhYKYokYjnN9Z3cc 3NwA== X-Gm-Message-State: AOAM5314Uu3NEBt+rPScQP1UmyfhMRRsNyoG8pCfRks86U/cJby9iyvJ yhcdoq+8NOsE1Lqu8jpnLODpBsjqK7W66Uv3ykl0wmS2tui3DU4UFop31Y0sJrnX93JLotjekjc MVNa/LVNglhr/ X-Received: by 2002:adf:f611:: with SMTP id t17mr3115450wrp.69.1591788984598; Wed, 10 Jun 2020 04:36:24 -0700 (PDT) X-Google-Smtp-Source: ABdhPJySUb6SmdRetYLS6/lDFnA63vhXBdDD3cCFK65tVJXhEwb5+hLdb3cRehsL9GzZU3pDEV7NVg== X-Received: by 2002:adf:f611:: with SMTP id t17mr3115431wrp.69.1591788984381; Wed, 10 Jun 2020 04:36:24 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id x8sm7694296wrs.43.2020.06.10.04.36.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:23 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:22 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 10/14] vhost/net: avoid iov length math Message-ID: <20200610113515.1497099-11-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Now that API exposes buffer length, we no longer need to scan IOVs to figure it out. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/net.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index 830fe84912a5..0b509be8d7b1 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -607,11 +607,9 @@ static bool vhost_exceeds_maxpend(struct vhost_net *net) } static size_t init_iov_iter(struct vhost_virtqueue *vq, struct iov_iter *iter, - size_t hdr_size, int out) + size_t len, size_t hdr_size, int out) { /* Skip header. TODO: support TSO. */ - size_t len = iov_length(vq->iov, out); - iov_iter_init(iter, WRITE, vq->iov, out, len); iov_iter_advance(iter, hdr_size); @@ -640,7 +638,7 @@ static int get_tx_bufs(struct vhost_net *net, } /* Sanity check */ - *len = init_iov_iter(vq, &msg->msg_iter, nvq->vhost_hlen, *out); + *len = init_iov_iter(vq, &msg->msg_iter, buf->out_len, nvq->vhost_hlen, *out); if (*len == 0) { vq_err(vq, "Unexpected header len for TX: %zd expected %zd\n", *len, nvq->vhost_hlen); @@ -1080,7 +1078,7 @@ static int get_rx_bufs(struct vhost_virtqueue *vq, nlogs += *log_num; log += *log_num; } - len = iov_length(vq->iov + seg, in); + len = bufs[bufcount].in_len; datalen -= len; ++bufcount; seg += in; From patchwork Wed Jun 10 11:36:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597547 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9EF1314E3 for ; Wed, 10 Jun 2020 11:36:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 80054204EC for ; Wed, 10 Jun 2020 11:36:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LLISva7r" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728743AbgFJLgj (ORCPT ); Wed, 10 Jun 2020 07:36:39 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:40160 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728705AbgFJLge (ORCPT ); Wed, 10 Jun 2020 07:36:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788993; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nNh1TL7xZUtXOusVSg8tIG35kTb/6beDbMDeGMbkL8c=; b=LLISva7riDzlkHbtZ3J/AB6Z/EPZCawO7XpVz/8M7yy7YATyMWbQrFjREhHyYzyVHwkLQs GXmA0+Fd5bQpqNlix7kDVssp2lwYNZY6Mxsv5QC4nO4F1wlM2qN2ReRvIiLcV3PYMLcOpe P7AfWTiHOsnuVGo+ZV2JT+NMDn2fxg8= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-328-mw6nZ3CTNfCBI3XCloJjrw-1; Wed, 10 Jun 2020 07:36:31 -0400 X-MC-Unique: mw6nZ3CTNfCBI3XCloJjrw-1 Received: by mail-wr1-f69.google.com with SMTP id i6so952291wrr.23 for ; Wed, 10 Jun 2020 04:36:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=nNh1TL7xZUtXOusVSg8tIG35kTb/6beDbMDeGMbkL8c=; b=Ki+ppk+OnOhCraujkdXq4qtC/XXw157jBp4D00r9p1eJjtC7zRVdRHFiUPhAmubKD/ a+BhLNEvXb+2Gl+FpipVwkI1HjZ5QrpVVa4UmGSV+iUsy/04gv0DD4dLGjZEfZW/RFqr mQLnZI0Jq2JIqrxHZ4W6jl3N8J0Qgy30JZKKmTXNDG9LKxPMu1C98+Yhdhj2e2+ewzQQ h6wYhEv/MT9rCKHYBjp0ooa2aFtLQcpQyvVAI9CrNv+i7UYQWL84vU4x4p2ugGPHqfmU TAE8gaSKgwpYNBwyUSDi5fToqyqxTlQOel67UB2IPsVGnOFr6QydXTeCsMiz4qZdk/w4 JW4Q== X-Gm-Message-State: AOAM530k/PnXHw7PeYoYNd3w3HJRqYNRP7D/Fy7OdEea7CRFrr3DhKZU I2elSKv+tBlbuAZJLitcvqc7ntgwAnco94oJf8txp3c8D5YZyU8Q4CiHR4vsYbPYnStziLgM3Hx 535VbWnm47l8Q X-Received: by 2002:adf:e9cb:: with SMTP id l11mr3131431wrn.86.1591788987304; Wed, 10 Jun 2020 04:36:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz+fpU0gApAo9WdU2/MAJH8DR4JAaeeYujgzZhmzjJ0R2CYxZQQqTFJTYVCM7So04QnKQlLdw== X-Received: by 2002:adf:e9cb:: with SMTP id l11mr3131407wrn.86.1591788987053; Wed, 10 Jun 2020 04:36:27 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id a16sm7675327wrx.8.2020.06.10.04.36.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:26 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:24 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 11/14] vhost/test: convert to the buf API Message-ID: <20200610113515.1497099-12-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Michael S. Tsirkin --- drivers/vhost/test.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c index 7d69778aaa26..12304eb8da15 100644 --- a/drivers/vhost/test.c +++ b/drivers/vhost/test.c @@ -44,9 +44,10 @@ static void handle_vq(struct vhost_test *n) { struct vhost_virtqueue *vq = &n->vqs[VHOST_TEST_VQ]; unsigned out, in; - int head; + int ret; size_t len, total_len = 0; void *private; + struct vhost_buf buf; mutex_lock(&vq->mutex); private = vhost_vq_get_backend(vq); @@ -58,15 +59,15 @@ static void handle_vq(struct vhost_test *n) vhost_disable_notify(&n->dev, vq); for (;;) { - head = vhost_get_vq_desc(vq, vq->iov, - ARRAY_SIZE(vq->iov), - &out, &in, - NULL, NULL); + ret = vhost_get_avail_buf(vq, &buf, vq->iov, + ARRAY_SIZE(vq->iov), + &out, &in, + NULL, NULL); /* On error, stop handling until the next kick. */ - if (unlikely(head < 0)) + if (unlikely(ret < 0)) break; /* Nothing new? Wait for eventfd to tell us they refilled. */ - if (head == vq->num) { + if (!ret) { if (unlikely(vhost_enable_notify(&n->dev, vq))) { vhost_disable_notify(&n->dev, vq); continue; @@ -78,13 +79,14 @@ static void handle_vq(struct vhost_test *n) "out %d, int %d\n", out, in); break; } - len = iov_length(vq->iov, out); + len = buf.out_len; /* Sanity check */ if (!len) { vq_err(vq, "Unexpected 0 len for TX\n"); break; } - vhost_add_used_and_signal(&n->dev, vq, head, 0); + vhost_put_used_buf(vq, &buf); + vhost_signal(&n->dev, vq); total_len += len; if (unlikely(vhost_exceeds_weight(vq, 0, total_len))) break; From patchwork Wed Jun 10 11:36:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597549 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0CACE739 for ; Wed, 10 Jun 2020 11:36:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E2B832074B for ; Wed, 10 Jun 2020 11:36:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="G6IIp7u2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728694AbgFJLgo (ORCPT ); Wed, 10 Jun 2020 07:36:44 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:41608 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728731AbgFJLgi (ORCPT ); Wed, 10 Jun 2020 07:36:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788996; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=dyvL3TJdoghmFE7taHXWfXC0TleKg1Dbjs9VDE83TZ8=; b=G6IIp7u2+2QlwWi4fvagdHeY9mac1rd69CV0Kpn23u21gcjBiFRnrrCeCwSU19bNgeAfjS okSm4t9vxwnbhYGawQjoEKuIfu1I7+uhXaNhTQ4T6PDWM0+mQ+MsG7VNE36L8LcpANrGjy dH5h3kEjiMO+8ShsJ9gC+1Kc6VP1CpU= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-256-xAQGB8HrMDihPU2Ws7ExPA-1; Wed, 10 Jun 2020 07:36:34 -0400 X-MC-Unique: xAQGB8HrMDihPU2Ws7ExPA-1 Received: by mail-wr1-f71.google.com with SMTP id d6so972005wrn.1 for ; Wed, 10 Jun 2020 04:36:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=dyvL3TJdoghmFE7taHXWfXC0TleKg1Dbjs9VDE83TZ8=; b=eLmrdUVjTnNCdq7czu7joB6DQtU+p719oH2HNuH8YIAC/YzZuO3gnR3rWTIOigHNFD 6V1+EYVW+U8kS696dDoAhA7zUakogAiUWVxkOAv8aNzwGkHNee3R5fByYhQMjb2Ca4SJ AguySCPzP8VW/qAqEUkFiYPiuxi+vQnPdO8H1eNMWHcdYEAUCpzpPazJPK0aKCaGI6Cm v1cVZa8ZL/R8yUACFrzopJHr0ckO/8Fejj9591cUs1PBZPwSdrXYHL1CA36oUeJL4XdE 2lYZOhbIhVG9qTgbC63+Z/QFsTQr+4hDUKThfBoUCjLBhzbM1FRAU5LBrIh+Y+/Rkqz7 6ClQ== X-Gm-Message-State: AOAM532XFIIeHlUVVyiXVAs9FPOizMFFb2b1VhNkU96X9zF0FjApflhO AuI4NncFWqSVu2ZbM9Lh27cwqHuiKS30/o9xkkrJ3QLeYvK+d1t2suUistGpKuj5fdAijV7KzXs 1k8SjKOvrL7i1 X-Received: by 2002:a1c:7f44:: with SMTP id a65mr2922741wmd.53.1591788990338; Wed, 10 Jun 2020 04:36:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJznUYRu63BAJ9mdaT9zK9V46K+GwKX25MjG2rCtScjt4V5cEmG2YBYMAluQ10cEhLhQH5k3sg== X-Received: by 2002:a1c:7f44:: with SMTP id a65mr2922715wmd.53.1591788990005; Wed, 10 Jun 2020 04:36:30 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id o18sm6917555wme.19.2020.06.10.04.36.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:29 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:27 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com, Paolo Bonzini , Stefan Hajnoczi Subject: [PATCH RFC v7 12/14] vhost/scsi: switch to buf APIs Message-ID: <20200610113515.1497099-13-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Switch to buf APIs. Doing this exposes a spec violation in vhost scsi: all used bufs are marked with length 0. Fix that is left for another day. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/scsi.c | 73 ++++++++++++++++++++++++++------------------ 1 file changed, 44 insertions(+), 29 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 0cbaa0b3893d..a5cdd4c01a3a 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -71,8 +71,8 @@ struct vhost_scsi_inflight { }; struct vhost_scsi_cmd { - /* Descriptor from vhost_get_vq_desc() for virt_queue segment */ - int tvc_vq_desc; + /* Descriptor from vhost_get_avail_buf() for virt_queue segment */ + struct vhost_buf tvc_vq_desc; /* virtio-scsi initiator task attribute */ int tvc_task_attr; /* virtio-scsi response incoming iovecs */ @@ -213,7 +213,7 @@ struct vhost_scsi { * Context for processing request and control queue operations. */ struct vhost_scsi_ctx { - int head; + struct vhost_buf buf; unsigned int out, in; size_t req_size, rsp_size; size_t out_size, in_size; @@ -443,6 +443,20 @@ static int vhost_scsi_check_stop_free(struct se_cmd *se_cmd) return target_put_sess_cmd(se_cmd); } +/* Signal to guest that request finished with no input buffer. */ +/* TODO calling this when writing into buffer and most likely a bug */ +static void vhost_scsi_signal_noinput(struct vhost_dev *vdev, + struct vhost_virtqueue *vq, + struct vhost_buf *bufp) +{ + struct vhost_buf buf = *bufp; + + buf.in_len = 0; + vhost_put_used_buf(vq, &buf); + vhost_signal(vdev, vq); +} + + static void vhost_scsi_do_evt_work(struct vhost_scsi *vs, struct vhost_scsi_evt *evt) { @@ -450,7 +464,8 @@ vhost_scsi_do_evt_work(struct vhost_scsi *vs, struct vhost_scsi_evt *evt) struct virtio_scsi_event *event = &evt->event; struct virtio_scsi_event __user *eventp; unsigned out, in; - int head, ret; + struct vhost_buf buf; + int ret; if (!vhost_vq_get_backend(vq)) { vs->vs_events_missed = true; @@ -459,14 +474,14 @@ vhost_scsi_do_evt_work(struct vhost_scsi *vs, struct vhost_scsi_evt *evt) again: vhost_disable_notify(&vs->dev, vq); - head = vhost_get_vq_desc(vq, vq->iov, - ARRAY_SIZE(vq->iov), &out, &in, - NULL, NULL); - if (head < 0) { + ret = vhost_get_avail_buf(vq, &buf, + vq->iov, ARRAY_SIZE(vq->iov), &out, &in, + NULL, NULL); + if (ret < 0) { vs->vs_events_missed = true; return; } - if (head == vq->num) { + if (!ret) { if (vhost_enable_notify(&vs->dev, vq)) goto again; vs->vs_events_missed = true; @@ -488,7 +503,7 @@ vhost_scsi_do_evt_work(struct vhost_scsi *vs, struct vhost_scsi_evt *evt) eventp = vq->iov[out].iov_base; ret = __copy_to_user(eventp, event, sizeof(*event)); if (!ret) - vhost_add_used_and_signal(&vs->dev, vq, head, 0); + vhost_scsi_signal_noinput(&vs->dev, vq, &buf); else vq_err(vq, "Faulted on vhost_scsi_send_event\n"); } @@ -549,7 +564,7 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work) ret = copy_to_iter(&v_rsp, sizeof(v_rsp), &iov_iter); if (likely(ret == sizeof(v_rsp))) { struct vhost_scsi_virtqueue *q; - vhost_add_used(cmd->tvc_vq, cmd->tvc_vq_desc, 0); + vhost_put_used_buf(cmd->tvc_vq, &cmd->tvc_vq_desc); q = container_of(cmd->tvc_vq, struct vhost_scsi_virtqueue, vq); vq = q - vs->vqs; __set_bit(vq, signal); @@ -793,7 +808,7 @@ static void vhost_scsi_submission_work(struct work_struct *work) static void vhost_scsi_send_bad_target(struct vhost_scsi *vs, struct vhost_virtqueue *vq, - int head, unsigned out) + struct vhost_buf *buf, unsigned out) { struct virtio_scsi_cmd_resp __user *resp; struct virtio_scsi_cmd_resp rsp; @@ -804,7 +819,7 @@ vhost_scsi_send_bad_target(struct vhost_scsi *vs, resp = vq->iov[out].iov_base; ret = __copy_to_user(resp, &rsp, sizeof(rsp)); if (!ret) - vhost_add_used_and_signal(&vs->dev, vq, head, 0); + vhost_scsi_signal_noinput(&vs->dev, vq, buf); else pr_err("Faulted on virtio_scsi_cmd_resp\n"); } @@ -813,21 +828,21 @@ static int vhost_scsi_get_desc(struct vhost_scsi *vs, struct vhost_virtqueue *vq, struct vhost_scsi_ctx *vc) { - int ret = -ENXIO; + int r, ret = -ENXIO; - vc->head = vhost_get_vq_desc(vq, vq->iov, - ARRAY_SIZE(vq->iov), &vc->out, &vc->in, - NULL, NULL); + r = vhost_get_avail_buf(vq, &vc->buf, + vq->iov, ARRAY_SIZE(vq->iov), &vc->out, &vc->in, + NULL, NULL); - pr_debug("vhost_get_vq_desc: head: %d, out: %u in: %u\n", - vc->head, vc->out, vc->in); + pr_debug("vhost_get_avail_buf: buf: %d, out: %u in: %u\n", + vc->buf.id, vc->out, vc->in); /* On error, stop handling until the next kick. */ - if (unlikely(vc->head < 0)) + if (unlikely(r < 0)) goto done; /* Nothing new? Wait for eventfd to tell us they refilled. */ - if (vc->head == vq->num) { + if (!r) { if (unlikely(vhost_enable_notify(&vs->dev, vq))) { vhost_disable_notify(&vs->dev, vq); ret = -EAGAIN; @@ -1093,11 +1108,11 @@ vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq) } } /* - * Save the descriptor from vhost_get_vq_desc() to be used to + * Save the descriptor from vhost_get_avail_buf() to be used to * complete the virtio-scsi request in TCM callback context via * vhost_scsi_queue_data_in() and vhost_scsi_queue_status() */ - cmd->tvc_vq_desc = vc.head; + cmd->tvc_vq_desc = vc.buf; /* * Dispatch cmd descriptor for cmwq execution in process * context provided by vhost_scsi_workqueue. This also ensures @@ -1117,7 +1132,7 @@ vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq) if (ret == -ENXIO) break; else if (ret == -EIO) - vhost_scsi_send_bad_target(vs, vq, vc.head, vc.out); + vhost_scsi_send_bad_target(vs, vq, &vc.buf, vc.out); } while (likely(!vhost_exceeds_weight(vq, ++c, 0))); out: mutex_unlock(&vq->mutex); @@ -1139,9 +1154,9 @@ vhost_scsi_send_tmf_reject(struct vhost_scsi *vs, iov_iter_init(&iov_iter, READ, &vq->iov[vc->out], vc->in, sizeof(rsp)); ret = copy_to_iter(&rsp, sizeof(rsp), &iov_iter); - if (likely(ret == sizeof(rsp))) - vhost_add_used_and_signal(&vs->dev, vq, vc->head, 0); - else + if (likely(ret == sizeof(rsp))) { + vhost_scsi_signal_noinput(&vs->dev, vq, &vc->buf); + } else pr_err("Faulted on virtio_scsi_ctrl_tmf_resp\n"); } @@ -1162,7 +1177,7 @@ vhost_scsi_send_an_resp(struct vhost_scsi *vs, ret = copy_to_iter(&rsp, sizeof(rsp), &iov_iter); if (likely(ret == sizeof(rsp))) - vhost_add_used_and_signal(&vs->dev, vq, vc->head, 0); + vhost_scsi_signal_noinput(&vs->dev, vq, &vc->buf); else pr_err("Faulted on virtio_scsi_ctrl_an_resp\n"); } @@ -1269,7 +1284,7 @@ vhost_scsi_ctl_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq) if (ret == -ENXIO) break; else if (ret == -EIO) - vhost_scsi_send_bad_target(vs, vq, vc.head, vc.out); + vhost_scsi_send_bad_target(vs, vq, &vc.buf, vc.out); } while (likely(!vhost_exceeds_weight(vq, ++c, 0))); out: mutex_unlock(&vq->mutex); From patchwork Wed Jun 10 11:36:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597553 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 563C314B7 for ; Wed, 10 Jun 2020 11:36:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3B46A2074B for ; Wed, 10 Jun 2020 11:36:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FQftU9BE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728754AbgFJLgn (ORCPT ); Wed, 10 Jun 2020 07:36:43 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:31770 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728729AbgFJLgh (ORCPT ); Wed, 10 Jun 2020 07:36:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591788995; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=TEosg1cbEhe9hhKHYKqZg2N/D89Ri/A+CQJLw8Cq4Bg=; b=FQftU9BEltRqD5BE3/7iZiakIslSTabk2rQ8luCT1A5Aqb5hqYHGO8B6OoEr1eimx9kzqk aG7p/cZBD9+dYNMZ0Asd+PRKBt47GJC3y8x5gASZknAw9WEO7OAQxKzbuCtJBzBlhwmetC GLI4kDkRn+Wgslau3Uj3b+ymCDCBNHM= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-310-sZQMeJgeNlqMnVwmu2uvAg-1; Wed, 10 Jun 2020 07:36:34 -0400 X-MC-Unique: sZQMeJgeNlqMnVwmu2uvAg-1 Received: by mail-wm1-f69.google.com with SMTP id g84so400733wmf.4 for ; Wed, 10 Jun 2020 04:36:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=TEosg1cbEhe9hhKHYKqZg2N/D89Ri/A+CQJLw8Cq4Bg=; b=QjSUN1WPqXxbH7jf6MkiWHTMy6tYGS5zLw8v+Clmu9M7002gEPIFrZFrFjViSecAff rKLWQCWxxLNUmAoJNh23pYSEkpOjbd2sSFbOWlJ7xtWI9W5dKiMnwzR8Ts9YWW//4xO+ bu9/OOcX+hm40DrL1PGEM4Yh9FiXNvUv7tJosbHFMn/SjZcrlcmVunvmcloVL8tfaeh9 fx/STyNUABHBOLuMkEYLIrI/3QDYs1kwVnWrqqrg/JFnSk5iAhLmRSp+WCU4UurR6jrk iFDYPaxWaL7mV73bz1FFE6TGnyeAwiiaNKxPpMGpY++b7MFnqYGgvXz03UZ2PneUxxh+ Ej7Q== X-Gm-Message-State: AOAM531W52j6Rf98HPkQkELToFcEGBEN/FNw5MMCRAHCsLKf4vDIvsvE q0ZeNueQI1jusDVJuvS3BfM44aaTLRPaNvaa6WXfFVT/3JrFGA8F6BHVM/VmKR0T6aJCUtYcSM1 QEpdQGoXgnlCB X-Received: by 2002:a05:600c:645:: with SMTP id p5mr2706635wmm.156.1591788993080; Wed, 10 Jun 2020 04:36:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwILwbhe48IqgvIMdBk87XXEqvw7WkIjeF+xwxu1AVsMzWmysnuJrztuEZ4mW7XzCxYzVz5BQ== X-Received: by 2002:a05:600c:645:: with SMTP id p5mr2706609wmm.156.1591788992842; Wed, 10 Jun 2020 04:36:32 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id e15sm6867215wme.9.2020.06.10.04.36.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:32 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:30 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com, Stefan Hajnoczi , Stefano Garzarella Subject: [PATCH RFC v7 13/14] vhost/vsock: switch to the buf API Message-ID: <20200610113515.1497099-14-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org A straight-forward conversion. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/vsock.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index a483cec31d5c..61c6d3dd2ae3 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -103,7 +103,8 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, unsigned out, in; size_t nbytes; size_t iov_len, payload_len; - int head; + struct vhost_buf buf; + int ret; spin_lock_bh(&vsock->send_pkt_list_lock); if (list_empty(&vsock->send_pkt_list)) { @@ -117,16 +118,17 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, list_del_init(&pkt->list); spin_unlock_bh(&vsock->send_pkt_list_lock); - head = vhost_get_vq_desc(vq, vq->iov, ARRAY_SIZE(vq->iov), - &out, &in, NULL, NULL); - if (head < 0) { + ret = vhost_get_avail_buf(vq, &buf, + vq->iov, ARRAY_SIZE(vq->iov), + &out, &in, NULL, NULL); + if (ret < 0) { spin_lock_bh(&vsock->send_pkt_list_lock); list_add(&pkt->list, &vsock->send_pkt_list); spin_unlock_bh(&vsock->send_pkt_list_lock); break; } - if (head == vq->num) { + if (!ret) { spin_lock_bh(&vsock->send_pkt_list_lock); list_add(&pkt->list, &vsock->send_pkt_list); spin_unlock_bh(&vsock->send_pkt_list_lock); @@ -186,7 +188,8 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, */ virtio_transport_deliver_tap_pkt(pkt); - vhost_add_used(vq, head, sizeof(pkt->hdr) + payload_len); + buf.in_len = sizeof(pkt->hdr) + payload_len; + vhost_put_used_buf(vq, &buf); added = true; pkt->off += payload_len; @@ -440,7 +443,8 @@ static void vhost_vsock_handle_tx_kick(struct vhost_work *work) struct vhost_vsock *vsock = container_of(vq->dev, struct vhost_vsock, dev); struct virtio_vsock_pkt *pkt; - int head, pkts = 0, total_len = 0; + int ret, pkts = 0, total_len = 0; + struct vhost_buf buf; unsigned int out, in; bool added = false; @@ -461,12 +465,13 @@ static void vhost_vsock_handle_tx_kick(struct vhost_work *work) goto no_more_replies; } - head = vhost_get_vq_desc(vq, vq->iov, ARRAY_SIZE(vq->iov), - &out, &in, NULL, NULL); - if (head < 0) + ret = vhost_get_avail_buf(vq, &buf, + vq->iov, ARRAY_SIZE(vq->iov), + &out, &in, NULL, NULL); + if (ret < 0) break; - if (head == vq->num) { + if (!ret) { if (unlikely(vhost_enable_notify(&vsock->dev, vq))) { vhost_disable_notify(&vsock->dev, vq); continue; @@ -494,7 +499,8 @@ static void vhost_vsock_handle_tx_kick(struct vhost_work *work) virtio_transport_free_pkt(pkt); len += sizeof(pkt->hdr); - vhost_add_used(vq, head, len); + buf.in_len = len; + vhost_put_used_buf(vq, &buf); total_len += len; added = true; } while(likely(!vhost_exceeds_weight(vq, ++pkts, total_len))); From patchwork Wed Jun 10 11:36:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 11597551 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF9E0739 for ; Wed, 10 Jun 2020 11:36:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D1FD7204EC for ; Wed, 10 Jun 2020 11:36:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iW+sFOGP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728773AbgFJLgr (ORCPT ); Wed, 10 Jun 2020 07:36:47 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:55551 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728749AbgFJLgn (ORCPT ); Wed, 10 Jun 2020 07:36:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591789000; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=cEu9UKQDih3Vjd1MZ21gGzO+ccbzDvhEg895z/Nzqys=; b=iW+sFOGPegxGRf59LsvFXnAxCdkvDmoGxUbyJIID4Bnm6GAqQzHINmV5wbiAsWukNsGSKD NNDAatilFZ9N6Xux7txYY28vTs/liZ4yLYUe41IY1FuexdR/vsGy5lDfq7dKC/uwaeAnY6 fVi9XFypWbAls3wgx8wOQnJIpMtkC2I= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-392-sZ9N6PNeMlGaNV8aG7kzOQ-1; Wed, 10 Jun 2020 07:36:37 -0400 X-MC-Unique: sZ9N6PNeMlGaNV8aG7kzOQ-1 Received: by mail-wr1-f71.google.com with SMTP id e1so973495wrm.3 for ; Wed, 10 Jun 2020 04:36:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=cEu9UKQDih3Vjd1MZ21gGzO+ccbzDvhEg895z/Nzqys=; b=tVfV5W4j56EzAERa0AuLwCtzaT0Y0vSKrknkOJu6GoIIqqIPDakKpHVGccvo2XI1rZ Isf7n3c54KubGUtDuptTeS+HjEJXYdrErE9XfIRi8l0UAY2Hi+4GSlMb7xH/H1DgN+hA 70pL92t0foHnWtEOjkQPrCyra6DF/B8lbUMMdMwY4CNem5mydKJsDBXev9Ofr34hSPrA lPpo4nN10HNIp4rPOdglPTESCAKz3vFCTC8Mo7KUr1ERpmJuw3c5mwxRZ5bCjxec1rBM kIM/SHEf/dihBjmtNAhwhmOE9qAUbQQwAZE32nr1mPkeK6TZjKjxmw2d83wRWYFUx802 sJIg== X-Gm-Message-State: AOAM532jTY7/9iWALhfuyGX3z9DQG61qI8i/Zp7lDmEurvqRs9EttETm aOW7vXK+0wqQUJpRNLrEjitgboJzOGC7yepIJVjcsYemBKFi9zZZ8qI1zoKiq+toH8GIo5LmUOY 08IwqwaX9QHWA X-Received: by 2002:adf:dcd2:: with SMTP id x18mr3204092wrm.258.1591788995971; Wed, 10 Jun 2020 04:36:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz+lrg/RmUfW2lVPzuXlXWaz8rOcMJq8w1XKgyPuKyHyoR97XZsCDeD1ieVxspClyZH+J9RRQ== X-Received: by 2002:adf:dcd2:: with SMTP id x18mr3204063wrm.258.1591788995672; Wed, 10 Jun 2020 04:36:35 -0700 (PDT) Received: from redhat.com ([212.92.121.57]) by smtp.gmail.com with ESMTPSA id d9sm7727647wre.28.2020.06.10.04.36.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jun 2020 04:36:35 -0700 (PDT) Date: Wed, 10 Jun 2020 07:36:33 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v7 14/14] vhost: drop head based APIs Message-ID: <20200610113515.1497099-15-mst@redhat.com> References: <20200610113515.1497099-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200610113515.1497099-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Everyone's using buf APIs, no need for head based ones anymore. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/vhost.c | 64 ++++++------------------------------------- drivers/vhost/vhost.h | 12 -------- 2 files changed, 8 insertions(+), 68 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 03e6bca02288..9096bd291c91 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2425,39 +2425,11 @@ EXPORT_SYMBOL_GPL(vhost_get_avail_buf); void vhost_discard_avail_bufs(struct vhost_virtqueue *vq, struct vhost_buf *buf, unsigned count) { - vhost_discard_vq_desc(vq, count); + unfetch_descs(vq); + vq->last_avail_idx -= count; } EXPORT_SYMBOL_GPL(vhost_discard_avail_bufs); -/* This function returns the descriptor number found, or vq->num (which is - * never a valid descriptor number) if none was found. A negative code is - * returned on error. */ -int vhost_get_vq_desc(struct vhost_virtqueue *vq, - struct iovec iov[], unsigned int iov_size, - unsigned int *out_num, unsigned int *in_num, - struct vhost_log *log, unsigned int *log_num) -{ - struct vhost_buf buf; - int ret = vhost_get_avail_buf(vq, &buf, - iov, iov_size, out_num, in_num, - log, log_num); - - if (likely(ret > 0)) - return buf->id; - if (likely(!ret)) - return vq->num; - return ret; -} -EXPORT_SYMBOL_GPL(vhost_get_vq_desc); - -/* Reverse the effect of vhost_get_vq_desc. Useful for error handling. */ -void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) -{ - unfetch_descs(vq); - vq->last_avail_idx -= n; -} -EXPORT_SYMBOL_GPL(vhost_discard_vq_desc); - static int __vhost_add_used_n(struct vhost_virtqueue *vq, struct vring_used_elem *heads, unsigned count) @@ -2490,8 +2462,7 @@ static int __vhost_add_used_n(struct vhost_virtqueue *vq, return 0; } -/* After we've used one of their buffers, we tell them about it. We'll then - * want to notify the guest, using eventfd. */ +static int vhost_add_used_n(struct vhost_virtqueue *vq, struct vring_used_elem *heads, unsigned count) { @@ -2525,10 +2496,8 @@ int vhost_add_used_n(struct vhost_virtqueue *vq, struct vring_used_elem *heads, } return r; } -EXPORT_SYMBOL_GPL(vhost_add_used_n); -/* After we've used one of their buffers, we tell them about it. We'll then - * want to notify the guest, using eventfd. */ +static int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) { struct vring_used_elem heads = { @@ -2538,14 +2507,17 @@ int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) return vhost_add_used_n(vq, &heads, 1); } -EXPORT_SYMBOL_GPL(vhost_add_used); +/* After we've used one of their buffers, we tell them about it. We'll then + * want to notify the guest, using vhost_signal. */ int vhost_put_used_buf(struct vhost_virtqueue *vq, struct vhost_buf *buf) { return vhost_add_used(vq, buf->id, buf->in_len); } EXPORT_SYMBOL_GPL(vhost_put_used_buf); +/* After we've used one of their buffers, we tell them about it. We'll then + * want to notify the guest, using vhost_signal. */ int vhost_put_used_n_bufs(struct vhost_virtqueue *vq, struct vhost_buf *bufs, unsigned count) { @@ -2606,26 +2578,6 @@ void vhost_signal(struct vhost_dev *dev, struct vhost_virtqueue *vq) } EXPORT_SYMBOL_GPL(vhost_signal); -/* And here's the combo meal deal. Supersize me! */ -void vhost_add_used_and_signal(struct vhost_dev *dev, - struct vhost_virtqueue *vq, - unsigned int head, int len) -{ - vhost_add_used(vq, head, len); - vhost_signal(dev, vq); -} -EXPORT_SYMBOL_GPL(vhost_add_used_and_signal); - -/* multi-buffer version of vhost_add_used_and_signal */ -void vhost_add_used_and_signal_n(struct vhost_dev *dev, - struct vhost_virtqueue *vq, - struct vring_used_elem *heads, unsigned count) -{ - vhost_add_used_n(vq, heads, count); - vhost_signal(dev, vq); -} -EXPORT_SYMBOL_GPL(vhost_add_used_and_signal_n); - /* return true if we're sure that avaiable ring is empty */ bool vhost_vq_avail_empty(struct vhost_dev *dev, struct vhost_virtqueue *vq) { diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index 28eea0155efb..264a2a2fae97 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -197,11 +197,6 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg bool vhost_vq_access_ok(struct vhost_virtqueue *vq); bool vhost_log_access_ok(struct vhost_dev *); -int vhost_get_vq_desc(struct vhost_virtqueue *, - struct iovec iov[], unsigned int iov_count, - unsigned int *out_num, unsigned int *in_num, - struct vhost_log *log, unsigned int *log_num); -void vhost_discard_vq_desc(struct vhost_virtqueue *, int n); int vhost_get_avail_buf(struct vhost_virtqueue *, struct vhost_buf *buf, struct iovec iov[], unsigned int iov_count, unsigned int *out_num, unsigned int *in_num, @@ -209,13 +204,6 @@ int vhost_get_avail_buf(struct vhost_virtqueue *, struct vhost_buf *buf, void vhost_discard_avail_bufs(struct vhost_virtqueue *, struct vhost_buf *, unsigned count); int vhost_vq_init_access(struct vhost_virtqueue *); -int vhost_add_used(struct vhost_virtqueue *, unsigned int head, int len); -int vhost_add_used_n(struct vhost_virtqueue *, struct vring_used_elem *heads, - unsigned count); -void vhost_add_used_and_signal(struct vhost_dev *, struct vhost_virtqueue *, - unsigned int id, int len); -void vhost_add_used_and_signal_n(struct vhost_dev *, struct vhost_virtqueue *, - struct vring_used_elem *heads, unsigned count); int vhost_put_used_buf(struct vhost_virtqueue *, struct vhost_buf *buf); int vhost_put_used_n_bufs(struct vhost_virtqueue *, struct vhost_buf *bufs, unsigned count);