From patchwork Tue Jan 30 11:42:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537413 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 274646D1A3; Tue, 30 Jan 2024 11:42:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614961; cv=none; b=rHK2cDyGVv7iVWbDEJIxamzycivsA4/uHeghW2vCg1tXURKN/u3Rll6d9PN6C3+52aWc+i0bu64MLaSrwFmttvKJd8KWxXqHS7b9SS8bMe2+XSzqyuXGlnPQIjIvDEFbJDFXfaKYgeHXWLPEUCaXB7a/1SLD7exBVoOeodyXbLQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614961; c=relaxed/simple; bh=w7D3FAxacz1cRePcp5nH/LN7vWiwihTU58oyIbIbYOM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NODGu3vTR0/hFC4JaKcxrkcUuncXPoReftMCRw5FKDMDzIH+pBtOZqsIzjXu9Y6jlduZZmqeFtxVc43uGcOpDAAGxxP3u+vs1vKTZEsp2qBIyFdvsiyjVM8vcqYoPhIo0jirgvfwjh39UTy+ySOo0Uk7w3tZal2Y2v/1Kuw2gvA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=l/ZB2OuE; arc=none smtp.client-ip=115.124.30.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="l/ZB2OuE" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614949; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=HjElYrqlM2BjB7Xez3T79umxevfUAwxmJHZN/Z/xJIo=; b=l/ZB2OuEGYgssJk5UWUBF7Ogqdf5cVjbhQwlPdM1+UGWfEiJkjWZsYRK4YMJWmmL39kJsw9lSGl87n9hOeAfw6NI3Hk3K6Y3c3BZHd+e670/HOsDXIbuAlRMi/RKya4UkULU6sPTs56sBtNeroi3ET9nhNr+CFWYtXtacxT3a+w= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R311e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g4wLD_1706614946; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g4wLD_1706614946) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:27 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 01/17] virtio_ring: introduce vring_need_unmap_buffer Date: Tue, 30 Jan 2024 19:42:08 +0800 Message-Id: <20240130114224.86536-2-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 To make the code readable, introduce vring_need_unmap_buffer() to replace do_unmap. use_dma_api premapped -> vring_need_unmap_buffer() 1. false false false 2. true false true 3. true true false Signed-off-by: Xuan Zhuo Acked-by: Jason Wang --- drivers/virtio/virtio_ring.c | 33 +++++++++++++++------------------ 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 49299b1f9ec7..4677831e6c26 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -175,11 +175,6 @@ struct vring_virtqueue { /* Do DMA mapping by driver */ bool premapped; - /* Do unmap or not for desc. Just when premapped is False and - * use_dma_api is true, this is true. - */ - bool do_unmap; - /* Head of free buffer list. */ unsigned int free_head; /* Number we've added since last sync. */ @@ -297,6 +292,11 @@ static bool vring_use_dma_api(const struct virtio_device *vdev) return false; } +static bool vring_need_unmap_buffer(const struct vring_virtqueue *vring) +{ + return vring->use_dma_api && !vring->premapped; +} + size_t virtio_max_dma_size(const struct virtio_device *vdev) { size_t max_segment_size = SIZE_MAX; @@ -445,7 +445,7 @@ static void vring_unmap_one_split_indirect(const struct vring_virtqueue *vq, { u16 flags; - if (!vq->do_unmap) + if (!vring_need_unmap_buffer(vq)) return; flags = virtio16_to_cpu(vq->vq.vdev, desc->flags); @@ -475,7 +475,7 @@ static unsigned int vring_unmap_one_split(const struct vring_virtqueue *vq, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } else { - if (!vq->do_unmap) + if (!vring_need_unmap_buffer(vq)) goto out; dma_unmap_page(vring_dma_dev(vq), @@ -643,7 +643,7 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, } /* Last one doesn't continue. */ desc[prev].flags &= cpu_to_virtio16(_vq->vdev, ~VRING_DESC_F_NEXT); - if (!indirect && vq->do_unmap) + if (!indirect && vring_need_unmap_buffer(vq)) vq->split.desc_extra[prev & (vq->split.vring.num - 1)].flags &= ~VRING_DESC_F_NEXT; @@ -802,7 +802,7 @@ static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, VRING_DESC_F_INDIRECT)); BUG_ON(len == 0 || len % sizeof(struct vring_desc)); - if (vq->do_unmap) { + if (vring_need_unmap_buffer(vq)) { for (j = 0; j < len / sizeof(struct vring_desc); j++) vring_unmap_one_split_indirect(vq, &indir_desc[j]); } @@ -1236,7 +1236,7 @@ static void vring_unmap_extra_packed(const struct vring_virtqueue *vq, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } else { - if (!vq->do_unmap) + if (!vring_need_unmap_buffer(vq)) return; dma_unmap_page(vring_dma_dev(vq), @@ -1251,7 +1251,7 @@ static void vring_unmap_desc_packed(const struct vring_virtqueue *vq, { u16 flags; - if (!vq->do_unmap) + if (!vring_need_unmap_buffer(vq)) return; flags = le16_to_cpu(desc->flags); @@ -1340,7 +1340,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, sizeof(struct vring_packed_desc)); vq->packed.vring.desc[head].id = cpu_to_le16(id); - if (vq->do_unmap) { + if (vring_need_unmap_buffer(vq)) { vq->packed.desc_extra[id].addr = addr; vq->packed.desc_extra[id].len = total_sg * sizeof(struct vring_packed_desc); @@ -1481,7 +1481,7 @@ static inline int virtqueue_add_packed(struct virtqueue *_vq, desc[i].len = cpu_to_le32(sg->length); desc[i].id = cpu_to_le16(id); - if (unlikely(vq->do_unmap)) { + if (unlikely(vring_need_unmap_buffer(vq))) { vq->packed.desc_extra[curr].addr = addr; vq->packed.desc_extra[curr].len = sg->length; vq->packed.desc_extra[curr].flags = @@ -1615,7 +1615,7 @@ static void detach_buf_packed(struct vring_virtqueue *vq, vq->free_head = id; vq->vq.num_free += state->num; - if (unlikely(vq->do_unmap)) { + if (unlikely(vring_need_unmap_buffer(vq))) { curr = id; for (i = 0; i < state->num; i++) { vring_unmap_extra_packed(vq, @@ -1632,7 +1632,7 @@ static void detach_buf_packed(struct vring_virtqueue *vq, if (!desc) return; - if (vq->do_unmap) { + if (vring_need_unmap_buffer(vq)) { len = vq->packed.desc_extra[id].len; for (i = 0; i < len / sizeof(struct vring_packed_desc); i++) @@ -2091,7 +2091,6 @@ static struct virtqueue *vring_create_virtqueue_packed( vq->dma_dev = dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); vq->premapped = false; - vq->do_unmap = vq->use_dma_api; vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !context; @@ -2636,7 +2635,6 @@ static struct virtqueue *__vring_new_virtqueue(unsigned int index, vq->dma_dev = dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); vq->premapped = false; - vq->do_unmap = vq->use_dma_api; vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !context; @@ -2804,7 +2802,6 @@ int virtqueue_set_dma_premapped(struct virtqueue *_vq) } vq->premapped = true; - vq->do_unmap = false; END_USE(vq); From patchwork Tue Jan 30 11:42:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537409 Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31EFD6A010; Tue, 30 Jan 2024 11:42:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.110 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614956; cv=none; b=rnVpl4ovbHGPFWv4uFv/bII/ii9qTj3k4/X4PSLwtJPcKErkOJI3wWVZgj2jAuqkhT5un45AzmteWvaATReb2yVo61jp6G5iUHmdnvKI3lJT8JX48I+azQYGI33w9kdAMBQR7gQj8mPHBTslodOU2xMC0S1xWdKkIKzoZN12yqs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614956; c=relaxed/simple; bh=OLUJgu2GtDakzRNx8JjX/f1thHwVn9dH+TV16TIJ45Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=I8uNj/4CM/di0OBD5UOsqt2kU9Icp2OBlilezYOwUXoIYL6QeZKsD0gAUtFLky7/2VG7uBhAOCpUl/PuR6xVICF8WvxBxUdQe+dhTGgzWcBtVNuHOMon9R70F9a2xH3haRBy2jn1/FZ77kY1pAzE4KOrSiJcEeTSNfgYguKuL1c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=yl9md2ph; arc=none smtp.client-ip=115.124.30.110 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="yl9md2ph" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614951; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=M/gA0U5hbdGfXbE+ate8yVaxvtF+5l5LRTg2RSnUFsU=; b=yl9md2phHgGmOSnSXnO6WiVhX26je6fHHVVJ8VQGhfVm8qkn4jNOGE9OhM5BQw5GC7MtzWEOd7r42yvXgdjjjoeMnmoT4NgYOjN1MX7C0mS3MqdoR074ukUND6zzkls7yy2Q0ByJf0FhpUHnB3AJo1XK/ZH01bZyLz7cx887y0Q= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R751e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g5g8s_1706614948; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g5g8s_1706614948) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:29 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 02/17] virtio_ring: packed: remove double check of the unmap ops Date: Tue, 30 Jan 2024 19:42:09 +0800 Message-Id: <20240130114224.86536-3-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 In the functions vring_unmap_extra_packed and vring_unmap_desc_packed, multiple checks are made whether unmap is performed and whether it is INDIRECT. These two functions are usually called in a loop, and we should put the check outside the loop. And we unmap the descs with VRING_DESC_F_INDIRECT on the same path with other descs, that make the thing more complex. If we distinguish the descs with VRING_DESC_F_INDIRECT before unmap, thing will be clearer. 1. only one desc of the desc table is used, we do not need the loop 2. the called unmap api is difference from the other desc 3. the vq->premapped is not needed to check 4. the vq->indirect is not needed to check 5. the state->indir_desc must not be null Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 76 ++++++++++++++++++------------------ 1 file changed, 39 insertions(+), 37 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 4677831e6c26..7280a1706cca 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1220,6 +1220,7 @@ static u16 packed_last_used(u16 last_used_idx) return last_used_idx & ~(-(1 << VRING_PACKED_EVENT_F_WRAP_CTR)); } +/* caller must check vring_need_unmap_buffer() */ static void vring_unmap_extra_packed(const struct vring_virtqueue *vq, const struct vring_desc_extra *extra) { @@ -1227,33 +1228,18 @@ static void vring_unmap_extra_packed(const struct vring_virtqueue *vq, flags = extra->flags; - if (flags & VRING_DESC_F_INDIRECT) { - if (!vq->use_dma_api) - return; - - dma_unmap_single(vring_dma_dev(vq), - extra->addr, extra->len, - (flags & VRING_DESC_F_WRITE) ? - DMA_FROM_DEVICE : DMA_TO_DEVICE); - } else { - if (!vring_need_unmap_buffer(vq)) - return; - - dma_unmap_page(vring_dma_dev(vq), - extra->addr, extra->len, - (flags & VRING_DESC_F_WRITE) ? - DMA_FROM_DEVICE : DMA_TO_DEVICE); - } + dma_unmap_page(vring_dma_dev(vq), + extra->addr, extra->len, + (flags & VRING_DESC_F_WRITE) ? + DMA_FROM_DEVICE : DMA_TO_DEVICE); } +/* caller must check vring_need_unmap_buffer() */ static void vring_unmap_desc_packed(const struct vring_virtqueue *vq, const struct vring_packed_desc *desc) { u16 flags; - if (!vring_need_unmap_buffer(vq)) - return; - flags = le16_to_cpu(desc->flags); dma_unmap_page(vring_dma_dev(vq), @@ -1329,7 +1315,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, total_sg * sizeof(struct vring_packed_desc), DMA_TO_DEVICE); if (vring_mapping_error(vq, addr)) { - if (vq->premapped) + if (!vring_need_unmap_buffer(vq)) goto free_desc; goto unmap_release; @@ -1344,10 +1330,11 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, vq->packed.desc_extra[id].addr = addr; vq->packed.desc_extra[id].len = total_sg * sizeof(struct vring_packed_desc); - vq->packed.desc_extra[id].flags = VRING_DESC_F_INDIRECT | - vq->packed.avail_used_flags; } + vq->packed.desc_extra[id].flags = VRING_DESC_F_INDIRECT | + vq->packed.avail_used_flags; + /* * A driver MUST NOT make the first descriptor in the list * available before all subsequent descriptors comprising @@ -1388,6 +1375,8 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, unmap_release: err_idx = i; + WARN_ON(!vring_need_unmap_buffer(vq)); + for (i = 0; i < err_idx; i++) vring_unmap_desc_packed(vq, &desc[i]); @@ -1484,9 +1473,10 @@ static inline int virtqueue_add_packed(struct virtqueue *_vq, if (unlikely(vring_need_unmap_buffer(vq))) { vq->packed.desc_extra[curr].addr = addr; vq->packed.desc_extra[curr].len = sg->length; - vq->packed.desc_extra[curr].flags = - le16_to_cpu(flags); } + + vq->packed.desc_extra[curr].flags = le16_to_cpu(flags); + prev = curr; curr = vq->packed.desc_extra[curr].next; @@ -1536,6 +1526,8 @@ static inline int virtqueue_add_packed(struct virtqueue *_vq, vq->packed.avail_used_flags = avail_used_flags; + WARN_ON(!vring_need_unmap_buffer(vq)); + for (n = 0; n < total_sg; n++) { if (i == err_idx) break; @@ -1605,7 +1597,9 @@ static void detach_buf_packed(struct vring_virtqueue *vq, struct vring_desc_state_packed *state = NULL; struct vring_packed_desc *desc; unsigned int i, curr; + u16 flags; + flags = vq->packed.desc_extra[id].flags; state = &vq->packed.desc_state[id]; /* Clear data ptr. */ @@ -1615,22 +1609,32 @@ static void detach_buf_packed(struct vring_virtqueue *vq, vq->free_head = id; vq->vq.num_free += state->num; - if (unlikely(vring_need_unmap_buffer(vq))) { - curr = id; - for (i = 0; i < state->num; i++) { - vring_unmap_extra_packed(vq, - &vq->packed.desc_extra[curr]); - curr = vq->packed.desc_extra[curr].next; + if (!(flags & VRING_DESC_F_INDIRECT)) { + if (vring_need_unmap_buffer(vq)) { + curr = id; + for (i = 0; i < state->num; i++) { + vring_unmap_extra_packed(vq, + &vq->packed.desc_extra[curr]); + curr = vq->packed.desc_extra[curr].next; + } } - } - if (vq->indirect) { + if (ctx) + *ctx = state->indir_desc; + } else { + const struct vring_desc_extra *extra; u32 len; + if (vq->use_dma_api) { + extra = &vq->packed.desc_extra[id]; + dma_unmap_single(vring_dma_dev(vq), + extra->addr, extra->len, + (flags & VRING_DESC_F_WRITE) ? + DMA_FROM_DEVICE : DMA_TO_DEVICE); + } + /* Free the indirect table, if any, now that it's unmapped. */ desc = state->indir_desc; - if (!desc) - return; if (vring_need_unmap_buffer(vq)) { len = vq->packed.desc_extra[id].len; @@ -1640,8 +1644,6 @@ static void detach_buf_packed(struct vring_virtqueue *vq, } kfree(desc); state->indir_desc = NULL; - } else if (ctx) { - *ctx = state->indir_desc; } } From patchwork Tue Jan 30 11:42:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537410 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E8F6679EE; Tue, 30 Jan 2024 11:42:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.112 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614957; cv=none; b=f6iDjrKPPhil4rIf13CQCNaFdcE1ZhThucNMzqVx4ldzuXgfXbe1x4PMVO05eBUo/Gh83Sa1I1UoyD9OUrzmlp3cUzbNi7s0/VNMvFBv+w9n+ZYTQCNebRN73dLNxtgEvJ1IegOLX8f3HjMN29vU7bbSZU+DNSN6cpPgXGlthjg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614957; c=relaxed/simple; bh=wtKZ3JnkbyRol+1QUCI3g+aTLGcAE86kgna4ZI/a4ik=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uSjquzYcEZVv3W6qAtqlHYZkpXgk6wvgwjF3mMGwrtSWXCLw4WcHVnh5AF5BJwY/5E32SgZ956HUv23ckyrRPvS4COGC+3zOS9Qj5knpRnpQzC4KMhsmOYgpFzX5P0hjvL5IyhNykPKLSm30KksS0cXaKdbp7t64KRqfwjcOKJU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=aK0KOkyc; arc=none smtp.client-ip=115.124.30.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="aK0KOkyc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614952; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=PHE2stYM55HtJOlk3lTfdUsq+qQbKVYlz5rHB+a8ViQ=; b=aK0KOkycAoTJo38hKqjOSPez0YTyD8lQFs5a18FcOzYhGOaeBjGdZ3dWUlVGcwVmsJc5/IukHEESRuO0qQLaAtrBnVdzPHa+04K4MjYEL4pQQFYp5yiDRv7vzVYupgzrAjengyiy2ZW37EUFNw9sVuBNsp5rSfxTVwf5VXB/eIc= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R201e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g61Rq_1706614949; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g61Rq_1706614949) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:30 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 03/17] virtio_ring: packed: structure the indirect desc table Date: Tue, 30 Jan 2024 19:42:10 +0800 Message-Id: <20240130114224.86536-4-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 This commit structure the indirect desc table. Then we can get the desc num directly when doing unmap. And save the dma info to the struct, then the indirect will not use the dma fields of the desc_extra. The subsequent commits will make the dma fields are optional. But for the indirect case, we must record the dma info. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 63 ++++++++++++++++++++---------------- 1 file changed, 35 insertions(+), 28 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 7280a1706cca..dd03bc5a81fe 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -72,9 +72,16 @@ struct vring_desc_state_split { struct vring_desc *indir_desc; /* Indirect descriptor, if any. */ }; +struct vring_packed_desc_indir { + dma_addr_t addr; /* Descriptor Array DMA addr. */ + u32 len; /* Descriptor Array length. */ + u32 num; + struct vring_packed_desc desc[]; +}; + struct vring_desc_state_packed { void *data; /* Data for callback. */ - struct vring_packed_desc *indir_desc; /* Indirect descriptor, if any. */ + struct vring_packed_desc_indir *indir_desc; /* Indirect descriptor, if any. */ u16 num; /* Descriptor list length. */ u16 last; /* The last desc state in a list. */ }; @@ -1249,10 +1256,13 @@ static void vring_unmap_desc_packed(const struct vring_virtqueue *vq, DMA_FROM_DEVICE : DMA_TO_DEVICE); } -static struct vring_packed_desc *alloc_indirect_packed(unsigned int total_sg, +static struct vring_packed_desc_indir *alloc_indirect_packed(unsigned int total_sg, gfp_t gfp) { - struct vring_packed_desc *desc; + struct vring_packed_desc_indir *in_desc; + u32 size; + + size = struct_size(in_desc, desc, total_sg); /* * We require lowmem mappings for the descriptors because @@ -1261,9 +1271,10 @@ static struct vring_packed_desc *alloc_indirect_packed(unsigned int total_sg, */ gfp &= ~__GFP_HIGHMEM; - desc = kmalloc_array(total_sg, sizeof(struct vring_packed_desc), gfp); - return desc; + in_desc = kmalloc(size, gfp); + + return in_desc; } static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, @@ -1274,6 +1285,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, void *data, gfp_t gfp) { + struct vring_packed_desc_indir *in_desc; struct vring_packed_desc *desc; struct scatterlist *sg; unsigned int i, n, err_idx; @@ -1281,10 +1293,12 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, dma_addr_t addr; head = vq->packed.next_avail_idx; - desc = alloc_indirect_packed(total_sg, gfp); - if (!desc) + in_desc = alloc_indirect_packed(total_sg, gfp); + if (!in_desc) return -ENOMEM; + desc = in_desc->desc; + if (unlikely(vq->vq.num_free < 1)) { pr_debug("Can't add buf len 1 - avail = 0\n"); kfree(desc); @@ -1321,17 +1335,15 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, goto unmap_release; } + in_desc->num = i; + in_desc->addr = addr; + in_desc->len = total_sg * sizeof(struct vring_packed_desc); + vq->packed.vring.desc[head].addr = cpu_to_le64(addr); vq->packed.vring.desc[head].len = cpu_to_le32(total_sg * sizeof(struct vring_packed_desc)); vq->packed.vring.desc[head].id = cpu_to_le16(id); - if (vring_need_unmap_buffer(vq)) { - vq->packed.desc_extra[id].addr = addr; - vq->packed.desc_extra[id].len = total_sg * - sizeof(struct vring_packed_desc); - } - vq->packed.desc_extra[id].flags = VRING_DESC_F_INDIRECT | vq->packed.avail_used_flags; @@ -1362,7 +1374,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, /* Store token and indirect buffer state. */ vq->packed.desc_state[id].num = 1; vq->packed.desc_state[id].data = data; - vq->packed.desc_state[id].indir_desc = desc; + vq->packed.desc_state[id].indir_desc = in_desc; vq->packed.desc_state[id].last = id; vq->num_added += 1; @@ -1381,7 +1393,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, vring_unmap_desc_packed(vq, &desc[i]); free_desc: - kfree(desc); + kfree(in_desc); END_USE(vq); return -ENOMEM; @@ -1595,7 +1607,6 @@ static void detach_buf_packed(struct vring_virtqueue *vq, unsigned int id, void **ctx) { struct vring_desc_state_packed *state = NULL; - struct vring_packed_desc *desc; unsigned int i, curr; u16 flags; @@ -1621,28 +1632,24 @@ static void detach_buf_packed(struct vring_virtqueue *vq, if (ctx) *ctx = state->indir_desc; + } else { - const struct vring_desc_extra *extra; - u32 len; + struct vring_packed_desc_indir *in_desc; + + in_desc = state->indir_desc; if (vq->use_dma_api) { - extra = &vq->packed.desc_extra[id]; dma_unmap_single(vring_dma_dev(vq), - extra->addr, extra->len, + in_desc->addr, in_desc->len, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } - /* Free the indirect table, if any, now that it's unmapped. */ - desc = state->indir_desc; - if (vring_need_unmap_buffer(vq)) { - len = vq->packed.desc_extra[id].len; - for (i = 0; i < len / sizeof(struct vring_packed_desc); - i++) - vring_unmap_desc_packed(vq, &desc[i]); + for (i = 0; i < in_desc->num; i++) + vring_unmap_desc_packed(vq, &in_desc->desc[i]); } - kfree(desc); + kfree(in_desc); state->indir_desc = NULL; } } From patchwork Tue Jan 30 11:42:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537411 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E38826D1C6; Tue, 30 Jan 2024 11:42:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.97 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614960; cv=none; b=QltWxVm0cRIiWsQcpHkNup7MbrnJhkAgv7UhheqkH1bQUnu4UQApicrKey04xxOmmiy6D60tCcZC2hJoCxrRIdPXHAICp/aJyx8qme1/Dic5z7Fn3v7roviAARKUFfP1f+dWvzMGtVx8JzgKoBXu7MuEAHs25G0xscS1mc2W/+U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614960; c=relaxed/simple; bh=csVqBjWHLd0wx8pt5oq5iqtKVwsT+VuwzwY4U+gqvHU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rbyYJLSnsuY86Yg2PKXcbEt3wBh2ok5QxCyKjtICUlWWSzv+xelYzTTYJC8Pi46AZtbfDMZCpvDHjFp3SLPH3SxL79dY/X96vlXZJMT3Glm7e9TBJKhWG/P3UQwe4SkKWLoLlFOLtMtqh+kH40JpuaUPyprFS7+lgIViBo47Jcs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=PnInmAKc; arc=none smtp.client-ip=115.124.30.97 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="PnInmAKc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614956; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=HrZuvU9yrR6T2DolteXZPfHuA2cDqF8W4GegYsiFG0w=; b=PnInmAKcMxnyTSS9YZoTwqHkhQHzhZGX126kTii3kldj0G9oo+IaoaFFoy0NT/7tgMEI/Sn+cu8RGC7aNbqW/T2zt+YxDjGwUJzhxwP/mb710/bAUtgCMyc3tuARng06AXumUrRGMirp7I727xum0wv793Y8Fjc7+IgdjfNI7cw= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g4wMQ_1706614951; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g4wMQ_1706614951) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:32 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 04/17] virtio_ring: split: remove double check of the unmap ops Date: Tue, 30 Jan 2024 19:42:11 +0800 Message-Id: <20240130114224.86536-5-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 In the functions vring_unmap_one_split and vring_unmap_one_split_indirect, multiple checks are made whether unmap is performed and whether it is INDIRECT. These two functions are usually called in a loop, and we should put the check outside the loop. And we unmap the descs with VRING_DESC_F_INDIRECT on the same path with other descs, that make the thing more complex. If we distinguish the descs with VRING_DESC_F_INDIRECT before unmap, thing will be clearer. 1. only one desc of the desc table is used, we do not need the loop 2. the called unmap api is difference from the other desc 3. the vq->premapped is not needed to check 4. the vq->indirect is not needed to check 5. the state->indir_desc must not be null Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 80 ++++++++++++++++++------------------ 1 file changed, 39 insertions(+), 41 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index dd03bc5a81fe..2b41fdbce975 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -452,9 +452,6 @@ static void vring_unmap_one_split_indirect(const struct vring_virtqueue *vq, { u16 flags; - if (!vring_need_unmap_buffer(vq)) - return; - flags = virtio16_to_cpu(vq->vq.vdev, desc->flags); dma_unmap_page(vring_dma_dev(vq), @@ -472,27 +469,12 @@ static unsigned int vring_unmap_one_split(const struct vring_virtqueue *vq, flags = extra[i].flags; - if (flags & VRING_DESC_F_INDIRECT) { - if (!vq->use_dma_api) - goto out; - - dma_unmap_single(vring_dma_dev(vq), - extra[i].addr, - extra[i].len, - (flags & VRING_DESC_F_WRITE) ? - DMA_FROM_DEVICE : DMA_TO_DEVICE); - } else { - if (!vring_need_unmap_buffer(vq)) - goto out; - - dma_unmap_page(vring_dma_dev(vq), - extra[i].addr, - extra[i].len, - (flags & VRING_DESC_F_WRITE) ? - DMA_FROM_DEVICE : DMA_TO_DEVICE); - } + dma_unmap_page(vring_dma_dev(vq), + extra[i].addr, + extra[i].len, + (flags & VRING_DESC_F_WRITE) ? + DMA_FROM_DEVICE : DMA_TO_DEVICE); -out: return extra[i].next; } @@ -660,7 +642,7 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, vq, desc, total_sg * sizeof(struct vring_desc), DMA_TO_DEVICE); if (vring_mapping_error(vq, addr)) { - if (vq->premapped) + if (!vring_need_unmap_buffer(vq)) goto free_indirect; goto unmap_release; @@ -713,6 +695,9 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, return 0; unmap_release: + + WARN_ON(!vring_need_unmap_buffer(vq)); + err_idx = i; if (indirect) @@ -774,34 +759,42 @@ static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, { unsigned int i, j; __virtio16 nextflag = cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_NEXT); + u16 flags; /* Clear data ptr. */ vq->split.desc_state[head].data = NULL; + flags = vq->split.desc_extra[head].flags; /* Put back on free list: unmap first-level descriptors and find end */ i = head; - while (vq->split.vring.desc[i].flags & nextflag) { - vring_unmap_one_split(vq, i); - i = vq->split.desc_extra[i].next; - vq->vq.num_free++; - } - - vring_unmap_one_split(vq, i); - vq->split.desc_extra[i].next = vq->free_head; - vq->free_head = head; + if (!(flags & VRING_DESC_F_INDIRECT)) { + while (vq->split.vring.desc[i].flags & nextflag) { + if (vring_need_unmap_buffer(vq)) + vring_unmap_one_split(vq, i); + i = vq->split.desc_extra[i].next; + vq->vq.num_free++; + } - /* Plus final descriptor */ - vq->vq.num_free++; + if (vring_need_unmap_buffer(vq)) + vring_unmap_one_split(vq, i); - if (vq->indirect) { + if (ctx) + *ctx = vq->split.desc_state[head].indir_desc; + } else { struct vring_desc *indir_desc = vq->split.desc_state[head].indir_desc; u32 len; - /* Free the indirect table, if any, now that it's unmapped. */ - if (!indir_desc) - return; + if (vq->use_dma_api) { + struct vring_desc_extra *extra = vq->split.desc_extra; + + dma_unmap_single(vring_dma_dev(vq), + extra[i].addr, + extra[i].len, + (flags & VRING_DESC_F_WRITE) ? + DMA_FROM_DEVICE : DMA_TO_DEVICE); + } len = vq->split.desc_extra[head].len; @@ -816,9 +809,14 @@ static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, kfree(indir_desc); vq->split.desc_state[head].indir_desc = NULL; - } else if (ctx) { - *ctx = vq->split.desc_state[head].indir_desc; + } + + vq->split.desc_extra[i].next = vq->free_head; + vq->free_head = head; + + /* Plus final descriptor */ + vq->vq.num_free++; } static bool more_used_split(const struct vring_virtqueue *vq) From patchwork Tue Jan 30 11:42:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537412 Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 191996D1C9; Tue, 30 Jan 2024 11:42:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.118 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614960; cv=none; b=ksXbGQy6lpzhOPydGHG1pU3cr8PJVkt05+wH1s3FRRtTUY25kJWehYbxkbjen8VNlYMXqgYaryg0zfyq21Y+kQtLGmdzTz5h37hcBAv3v68xRpK+4s81VsPKhLGhmAzlSEw1T6JOepZ0cxTMCGpKF5139G1d6bIx7qf0y9g8XSQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614960; c=relaxed/simple; bh=6G571sJ4EdFAIcFL28AlPWeFbDahxvOWW2JSK/6QgjY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hJQNNSEqMhFO5Mjj5w2KcBOhMS71hpokQQx3NYA8ZhRjy428EfNjBq/rdsDOvWeqD5A0gyb5umTvGRsZLjbD3z3cslacJxQsUZLhIhfL63BkX5tUWhAv96Ed4PCMRUcbJOjZG0Nhsv20rzTybH55b0z1djC1Y2HSA/aK+dN/Fh4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=Ww6TKdEC; arc=none smtp.client-ip=115.124.30.118 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="Ww6TKdEC" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614956; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=z3UTRsbRsFcB2sqFN3XBDD7wrtKUw8K4tiUA1HwO3kw=; b=Ww6TKdECJxUGFW4PAme4hH3xcNh9c3/KCadii+J1oKFZDD8HS4X3bqptDlGETUv/UBFplRfjY8nyfT7KbPJqhHpwW8W7wNC20g3H3cpG+HgeptfCuUBpDUhWAfe6w/G1An51qOsw5Obykp+8OFfBkCZY+p24Ob2AbOcfIuHgtEU= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g77xE_1706614953; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g77xE_1706614953) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:34 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 05/17] virtio_ring: split: structure the indirect desc table Date: Tue, 30 Jan 2024 19:42:12 +0800 Message-Id: <20240130114224.86536-6-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 This commit structure the indirect desc table. Then we can get the desc num directly when doing unmap. And save the dma info to the struct, then the indirect will not use the dma fields of the desc_extra. The subsequent commits will make the dma fields are optional. But for the indirect case, we must record the dma info. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 86 ++++++++++++++++++++++-------------- 1 file changed, 52 insertions(+), 34 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 2b41fdbce975..831667a57429 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -67,9 +67,16 @@ #define LAST_ADD_TIME_INVALID(vq) #endif +struct vring_split_desc_indir { + dma_addr_t addr; /* Descriptor Array DMA addr. */ + u32 len; /* Descriptor Array length. */ + u32 num; + struct vring_desc desc[]; +}; + struct vring_desc_state_split { void *data; /* Data for callback. */ - struct vring_desc *indir_desc; /* Indirect descriptor, if any. */ + struct vring_split_desc_indir *indir_desc; /* Indirect descriptor, if any. */ }; struct vring_packed_desc_indir { @@ -478,12 +485,16 @@ static unsigned int vring_unmap_one_split(const struct vring_virtqueue *vq, return extra[i].next; } -static struct vring_desc *alloc_indirect_split(struct virtqueue *_vq, - unsigned int total_sg, - gfp_t gfp) +static struct vring_split_desc_indir *alloc_indirect_split(struct virtqueue *_vq, + unsigned int total_sg, + gfp_t gfp) { + struct vring_split_desc_indir *in_desc; struct vring_desc *desc; unsigned int i; + u32 size; + + size = struct_size(in_desc, desc, total_sg); /* * We require lowmem mappings for the descriptors because @@ -492,13 +503,16 @@ static struct vring_desc *alloc_indirect_split(struct virtqueue *_vq, */ gfp &= ~__GFP_HIGHMEM; - desc = kmalloc_array(total_sg, sizeof(struct vring_desc), gfp); - if (!desc) + in_desc = kmalloc(size, gfp); + if (!in_desc) return NULL; + desc = in_desc->desc; + for (i = 0; i < total_sg; i++) desc[i].next = cpu_to_virtio16(_vq->vdev, i + 1); - return desc; + + return in_desc; } static inline unsigned int virtqueue_add_desc_split(struct virtqueue *vq, @@ -540,6 +554,7 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, gfp_t gfp) { struct vring_virtqueue *vq = to_vvq(_vq); + struct vring_split_desc_indir *in_desc; struct scatterlist *sg; struct vring_desc *desc; unsigned int i, n, avail, descs_used, prev, err_idx; @@ -562,9 +577,13 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, head = vq->free_head; - if (virtqueue_use_indirect(vq, total_sg)) - desc = alloc_indirect_split(_vq, total_sg, gfp); - else { + if (virtqueue_use_indirect(vq, total_sg)) { + in_desc = alloc_indirect_split(_vq, total_sg, gfp); + if (!in_desc) + desc = NULL; + else + desc = in_desc->desc; + } else { desc = NULL; WARN_ON_ONCE(total_sg > vq->split.vring.num && !vq->indirect); } @@ -637,10 +656,10 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, ~VRING_DESC_F_NEXT; if (indirect) { + u32 size = total_sg * sizeof(struct vring_desc); + /* Now that the indirect table is filled in, map it. */ - dma_addr_t addr = vring_map_single( - vq, desc, total_sg * sizeof(struct vring_desc), - DMA_TO_DEVICE); + dma_addr_t addr = vring_map_single(vq, desc, size, DMA_TO_DEVICE); if (vring_mapping_error(vq, addr)) { if (!vring_need_unmap_buffer(vq)) goto free_indirect; @@ -648,11 +667,17 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, goto unmap_release; } - virtqueue_add_desc_split(_vq, vq->split.vring.desc, - head, addr, - total_sg * sizeof(struct vring_desc), - VRING_DESC_F_INDIRECT, - false); + desc = &vq->split.vring.desc[head]; + + desc->flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_INDIRECT); + desc->addr = cpu_to_virtio64(_vq->vdev, addr); + desc->len = cpu_to_virtio32(_vq->vdev, size); + + vq->split.desc_extra[head].flags = VRING_DESC_F_INDIRECT; + + in_desc->addr = addr; + in_desc->len = size; + in_desc->num = total_sg; } /* We're using some buffers from the free list. */ @@ -667,7 +692,7 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, /* Store token and indirect buffer state. */ vq->split.desc_state[head].data = data; if (indirect) - vq->split.desc_state[head].indir_desc = desc; + vq->split.desc_state[head].indir_desc = in_desc; else vq->split.desc_state[head].indir_desc = ctx; @@ -717,7 +742,7 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, free_indirect: if (indirect) - kfree(desc); + kfree(in_desc); END_USE(vq); return -ENOMEM; @@ -782,34 +807,27 @@ static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, if (ctx) *ctx = vq->split.desc_state[head].indir_desc; } else { - struct vring_desc *indir_desc = - vq->split.desc_state[head].indir_desc; - u32 len; + struct vring_split_desc_indir *in_desc; - if (vq->use_dma_api) { - struct vring_desc_extra *extra = vq->split.desc_extra; + in_desc = vq->split.desc_state[head].indir_desc; + if (vq->use_dma_api) { dma_unmap_single(vring_dma_dev(vq), - extra[i].addr, - extra[i].len, + in_desc->addr, in_desc->len, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } - len = vq->split.desc_extra[head].len; - BUG_ON(!(vq->split.desc_extra[head].flags & VRING_DESC_F_INDIRECT)); - BUG_ON(len == 0 || len % sizeof(struct vring_desc)); if (vring_need_unmap_buffer(vq)) { - for (j = 0; j < len / sizeof(struct vring_desc); j++) - vring_unmap_one_split_indirect(vq, &indir_desc[j]); + for (j = 0; j < in_desc->num; j++) + vring_unmap_one_split_indirect(vq, &in_desc->desc[j]); } - kfree(indir_desc); + kfree(in_desc); vq->split.desc_state[head].indir_desc = NULL; - } vq->split.desc_extra[i].next = vq->free_head; From patchwork Tue Jan 30 11:42:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537414 Received: from out30-100.freemail.mail.aliyun.com (out30-100.freemail.mail.aliyun.com [115.124.30.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E76267E9A; Tue, 30 Jan 2024 11:42:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.100 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614962; cv=none; b=dYN3DV/nHG0VZUTc9NiYuRElMh0CY5BIT8Go74SJ1THDoepxbqf8HFfKCsnvUhx5f4+s6MgGNz720Tbk1wAw/s0hDWSJNbSKa97lU8KvXR149k/7kvqt5k+ioa5+qUSv2U81faT5j8f2qr6MUKVMUmf5bMtiWcHkFlC04xG6Axg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614962; c=relaxed/simple; bh=sioBitvVODKWFEi3It7y9hTCuYhRhlwoPb8vZ+NKaXo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=C00PeZ2JGFGZatcl4uCQ9khhg9n4ESmOCY2fPmXVUqidq346o4rHsKFgMaAiagqAKxCBzVfYCtzJt4qMavcoEaPUPDnBtHgSwlQ5ENy1ieK6wlu2yjCn7j4rMRcC1U3UEWf9453zD1p6QUSGSZdVVloq24kE5yAlk1P11Ko/aUU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=CVbHDMgH; arc=none smtp.client-ip=115.124.30.100 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="CVbHDMgH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614957; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=aJW45Loje2yp/4dQUt3QyiyDsaI3FGnc4OA+IXxlF9E=; b=CVbHDMgHaW4A62oq6v1Ah4DOoynnKqtaE2dCjMGrrBqAQ1QVsewT3wDyTnVx7nwwqdrH1OdlXaMwlMjm7au6ppvwKFMr7PiOYEMJ1rW067J/zBlSdot9k+IN4baTRWh967e5Gu8uXAOc2jawrfUP7wVj+w3305jJOWyzbVizLLw= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R751e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g4wNH_1706614954; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g4wNH_1706614954) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:35 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 06/17] virtio_ring: no store dma info when unmap is not needed Date: Tue, 30 Jan 2024 19:42:13 +0800 Message-Id: <20240130114224.86536-7-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 As discussed: http://lore.kernel.org/all/CACGkMEug-=C+VQhkMYSgUKMC==04m7-uem_yC21bgGkKZh845w@mail.gmail.com When the vq is premapped mode, the driver manages the dma info is a good way. So this commit make the virtio core not to store the dma info and release the memory which is used to store the dma info. If the use_dma_api is false, the memory is also not allocated. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 89 ++++++++++++++++++++++++++++-------- 1 file changed, 70 insertions(+), 19 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 831667a57429..5bea25167259 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -94,12 +94,15 @@ struct vring_desc_state_packed { }; struct vring_desc_extra { - dma_addr_t addr; /* Descriptor DMA addr. */ - u32 len; /* Descriptor length. */ u16 flags; /* Descriptor flags. */ u16 next; /* The next desc state in a list. */ }; +struct vring_desc_dma { + dma_addr_t addr; /* Descriptor DMA addr. */ + u32 len; /* Descriptor length. */ +}; + struct vring_virtqueue_split { /* Actual memory layout for this queue. */ struct vring vring; @@ -116,6 +119,7 @@ struct vring_virtqueue_split { /* Per-descriptor state. */ struct vring_desc_state_split *desc_state; struct vring_desc_extra *desc_extra; + struct vring_desc_dma *desc_dma; /* DMA address and size information */ dma_addr_t queue_dma_addr; @@ -156,6 +160,7 @@ struct vring_virtqueue_packed { /* Per-descriptor state. */ struct vring_desc_state_packed *desc_state; struct vring_desc_extra *desc_extra; + struct vring_desc_dma *desc_dma; /* DMA address and size information */ dma_addr_t ring_dma_addr; @@ -472,13 +477,14 @@ static unsigned int vring_unmap_one_split(const struct vring_virtqueue *vq, unsigned int i) { struct vring_desc_extra *extra = vq->split.desc_extra; + struct vring_desc_dma *dma = vq->split.desc_dma; u16 flags; flags = extra[i].flags; dma_unmap_page(vring_dma_dev(vq), - extra[i].addr, - extra[i].len, + dma[i].addr, + dma[i].len, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); @@ -535,8 +541,11 @@ static inline unsigned int virtqueue_add_desc_split(struct virtqueue *vq, next = extra[i].next; desc[i].next = cpu_to_virtio16(vq->vdev, next); - extra[i].addr = addr; - extra[i].len = len; + if (vring->split.desc_dma) { + vring->split.desc_dma[i].addr = addr; + vring->split.desc_dma[i].len = len; + } + extra[i].flags = flags; } else next = virtio16_to_cpu(vq->vdev, desc[i].next); @@ -1072,16 +1081,26 @@ static void virtqueue_vring_attach_split(struct vring_virtqueue *vq, vq->free_head = 0; } -static int vring_alloc_state_extra_split(struct vring_virtqueue_split *vring_split) +static int vring_alloc_state_extra_split(struct vring_virtqueue_split *vring_split, + bool need_unmap) { struct vring_desc_state_split *state; struct vring_desc_extra *extra; + struct vring_desc_dma *dma; u32 num = vring_split->vring.num; state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL); if (!state) goto err_state; + if (need_unmap) { + dma = kmalloc_array(num, sizeof(struct vring_desc_dma), GFP_KERNEL); + if (!dma) + goto err_dma; + } else { + dma = NULL; + } + extra = vring_alloc_desc_extra(num); if (!extra) goto err_extra; @@ -1090,9 +1109,12 @@ static int vring_alloc_state_extra_split(struct vring_virtqueue_split *vring_spl vring_split->desc_state = state; vring_split->desc_extra = extra; + vring_split->desc_dma = dma; return 0; err_extra: + kfree(dma); +err_dma: kfree(state); err_state: return -ENOMEM; @@ -1108,6 +1130,7 @@ static void vring_free_split(struct vring_virtqueue_split *vring_split, kfree(vring_split->desc_state); kfree(vring_split->desc_extra); + kfree(vring_split->desc_dma); } static int vring_alloc_queue_split(struct vring_virtqueue_split *vring_split, @@ -1209,7 +1232,8 @@ static int virtqueue_resize_split(struct virtqueue *_vq, u32 num) if (err) goto err; - err = vring_alloc_state_extra_split(&vring_split); + err = vring_alloc_state_extra_split(&vring_split, + vring_need_unmap_buffer(vq)); if (err) goto err_state_extra; @@ -1245,14 +1269,16 @@ static u16 packed_last_used(u16 last_used_idx) /* caller must check vring_need_unmap_buffer() */ static void vring_unmap_extra_packed(const struct vring_virtqueue *vq, - const struct vring_desc_extra *extra) + unsigned int i) { + const struct vring_desc_extra *extra = &vq->packed.desc_extra[i]; + const struct vring_desc_dma *dma = &vq->packed.desc_dma[i]; u16 flags; flags = extra->flags; dma_unmap_page(vring_dma_dev(vq), - extra->addr, extra->len, + dma->addr, dma->len, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } @@ -1499,8 +1525,8 @@ static inline int virtqueue_add_packed(struct virtqueue *_vq, desc[i].id = cpu_to_le16(id); if (unlikely(vring_need_unmap_buffer(vq))) { - vq->packed.desc_extra[curr].addr = addr; - vq->packed.desc_extra[curr].len = sg->length; + vq->packed.desc_dma[curr].addr = addr; + vq->packed.desc_dma[curr].len = sg->length; } vq->packed.desc_extra[curr].flags = le16_to_cpu(flags); @@ -1559,7 +1585,7 @@ static inline int virtqueue_add_packed(struct virtqueue *_vq, for (n = 0; n < total_sg; n++) { if (i == err_idx) break; - vring_unmap_extra_packed(vq, &vq->packed.desc_extra[curr]); + vring_unmap_extra_packed(vq, curr); curr = vq->packed.desc_extra[curr].next; i++; if (i >= vq->packed.vring.num) @@ -1640,8 +1666,7 @@ static void detach_buf_packed(struct vring_virtqueue *vq, if (vring_need_unmap_buffer(vq)) { curr = id; for (i = 0; i < state->num; i++) { - vring_unmap_extra_packed(vq, - &vq->packed.desc_extra[curr]); + vring_unmap_extra_packed(vq, curr); curr = vq->packed.desc_extra[curr].next; } } @@ -1955,6 +1980,7 @@ static void vring_free_packed(struct vring_virtqueue_packed *vring_packed, kfree(vring_packed->desc_state); kfree(vring_packed->desc_extra); + kfree(vring_packed->desc_dma); } static int vring_alloc_queue_packed(struct vring_virtqueue_packed *vring_packed, @@ -2011,10 +2037,12 @@ static int vring_alloc_queue_packed(struct vring_virtqueue_packed *vring_packed, return -ENOMEM; } -static int vring_alloc_state_extra_packed(struct vring_virtqueue_packed *vring_packed) +static int vring_alloc_state_extra_packed(struct vring_virtqueue_packed *vring_packed, + bool need_unmap) { struct vring_desc_state_packed *state; struct vring_desc_extra *extra; + struct vring_desc_dma *dma; u32 num = vring_packed->vring.num; state = kmalloc_array(num, sizeof(struct vring_desc_state_packed), GFP_KERNEL); @@ -2023,6 +2051,14 @@ static int vring_alloc_state_extra_packed(struct vring_virtqueue_packed *vring_p memset(state, 0, num * sizeof(struct vring_desc_state_packed)); + if (need_unmap) { + dma = kmalloc_array(num, sizeof(struct vring_desc_dma), GFP_KERNEL); + if (!dma) + goto err_desc_dma; + } else { + dma = NULL; + } + extra = vring_alloc_desc_extra(num); if (!extra) goto err_desc_extra; @@ -2033,6 +2069,8 @@ static int vring_alloc_state_extra_packed(struct vring_virtqueue_packed *vring_p return 0; err_desc_extra: + kfree(dma); +err_desc_dma: kfree(state); err_desc_state: return -ENOMEM; @@ -2124,7 +2162,8 @@ static struct virtqueue *vring_create_virtqueue_packed( if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM)) vq->weak_barriers = false; - err = vring_alloc_state_extra_packed(&vring_packed); + err = vring_alloc_state_extra_packed(&vring_packed, + vring_need_unmap_buffer(vq)); if (err) goto err_state_extra; @@ -2156,7 +2195,8 @@ static int virtqueue_resize_packed(struct virtqueue *_vq, u32 num) if (vring_alloc_queue_packed(&vring_packed, vdev, num, vring_dma_dev(vq))) goto err_ring; - err = vring_alloc_state_extra_packed(&vring_packed); + err = vring_alloc_state_extra_packed(&vring_packed, + vring_need_unmap_buffer(vq)); if (err) goto err_state_extra; @@ -2668,7 +2708,8 @@ static struct virtqueue *__vring_new_virtqueue(unsigned int index, if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM)) vq->weak_barriers = false; - err = vring_alloc_state_extra_split(vring_split); + err = vring_alloc_state_extra_split(vring_split, + vring_need_unmap_buffer(vq)); if (err) { kfree(vq); return NULL; @@ -2828,6 +2869,14 @@ int virtqueue_set_dma_premapped(struct virtqueue *_vq) vq->premapped = true; + if (vq->packed_ring) { + kfree(vq->packed.desc_dma); + vq->packed.desc_dma = NULL; + } else { + kfree(vq->split.desc_dma); + vq->split.desc_dma = NULL; + } + END_USE(vq); return 0; @@ -2917,6 +2966,7 @@ static void vring_free(struct virtqueue *_vq) kfree(vq->packed.desc_state); kfree(vq->packed.desc_extra); + kfree(vq->packed.desc_dma); } else { vring_free_queue(vq->vq.vdev, vq->split.queue_size_in_bytes, @@ -2928,6 +2978,7 @@ static void vring_free(struct virtqueue *_vq) if (!vq->packed_ring) { kfree(vq->split.desc_state); kfree(vq->split.desc_extra); + kfree(vq->split.desc_dma); } } From patchwork Tue Jan 30 11:42:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537415 Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8E616D1DC; Tue, 30 Jan 2024 11:42:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.118 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614964; cv=none; b=MCS86MSD1ZLZGVI+1Bm0gmrAZtwQyBtmCqqaSc88OALmdX5f7BErO4vdXDxkZCpe8FH0ZxZlK7UVqnPrwdxRkz6QcThtk44CtBsCjbUV4qQSKW9zlsF+H5ZDAjX2+XDClpfiNbFVW64hmB66ir+TR8Gn/9Ekvds4GwdKkn63hgA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614964; c=relaxed/simple; bh=ZmXtSEbmDA3MB/F+7Bebw+2IiyJXTgIjv4fqG8b+aNo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=L09Wb8BxQY+B6fJPkdEOLSgjGp6VDQy1GNKsCMV+3cKO9LYJBfPQH6Iw7R7XdKY8Cuny9PeovP1shYu6JovrRbESAnJij7qLfjvFlIuVFhB1RlLjCdo0pFVuRLyKJ74Y3U5A/AHTPoWkE54+/yYYb5pZFDq0Uo8aC/fwwTeot7o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=wGMYuqwA; arc=none smtp.client-ip=115.124.30.118 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="wGMYuqwA" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614959; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=QNcadSU4TpNcce2SQ2UUPJ56wmbf2PVr6nbDtPNoeOc=; b=wGMYuqwA6Vbn2WqwT50NazzildzuO0vWpWhg9d/e313IRb9MTt8YDtyiXYvstznCX8PGRfTm5PFvDO6t/PUg9Ck+Lh9FmAQWznrJfa5t70UeIsfWsjkBdamlYJctb6cbDeXRBRY+AcYG9rGtTDsV+cCMZ/Me/MmykeOP+T3uqCU= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g61Tu_1706614956; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g61Tu_1706614956) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:37 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 07/17] virtio: find_vqs: pass struct instead of multi parameters Date: Tue, 30 Jan 2024 19:42:14 +0800 Message-Id: <20240130114224.86536-8-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 Now, we pass multi parameters to find_vqs. These parameters may work for transport or work for vring. And find_vqs has multi implements in many places: But every time, arch/um/drivers/virtio_uml.c drivers/platform/mellanox/mlxbf-tmfifo.c drivers/remoteproc/remoteproc_virtio.c drivers/s390/virtio/virtio_ccw.c drivers/virtio/virtio_mmio.c drivers/virtio/virtio_pci_legacy.c drivers/virtio/virtio_pci_modern.c drivers/virtio/virtio_vdpa.c Every time, we try to add a new parameter, that is difficult. We must change every find_vqs implement. One the other side, if we want to pass a parameter to vring, we must change the call path from transport to vring. Too many functions need to be changed. So it is time to refactor the find_vqs. We pass a structure cfg to find_vqs(), that will be passed to vring by transport. And squish the parameters from transport to a structure. Signed-off-by: Xuan Zhuo --- arch/um/drivers/virtio_uml.c | 29 +++++++----- drivers/platform/mellanox/mlxbf-tmfifo.c | 14 +++--- drivers/remoteproc/remoteproc_virtio.c | 28 ++++++----- drivers/s390/virtio/virtio_ccw.c | 33 ++++++------- drivers/virtio/virtio_mmio.c | 30 ++++++------ drivers/virtio/virtio_pci_common.c | 59 +++++++++++------------- drivers/virtio/virtio_pci_common.h | 9 +--- drivers/virtio/virtio_pci_legacy.c | 16 ++++--- drivers/virtio/virtio_pci_modern.c | 24 +++++----- drivers/virtio/virtio_ring.c | 59 ++++++++++-------------- drivers/virtio/virtio_vdpa.c | 33 +++++++------ include/linux/virtio_config.h | 51 ++++++++++++++++---- include/linux/virtio_ring.h | 40 ++++++---------- 13 files changed, 217 insertions(+), 208 deletions(-) diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c index 8adca2000e51..161bac67e454 100644 --- a/arch/um/drivers/virtio_uml.c +++ b/arch/um/drivers/virtio_uml.c @@ -937,11 +937,12 @@ static int vu_setup_vq_call_fd(struct virtio_uml_device *vu_dev, } static struct virtqueue *vu_setup_vq(struct virtio_device *vdev, - unsigned index, vq_callback_t *callback, - const char *name, bool ctx) + unsigned index, + struct virtio_vq_config *cfg) { struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev); struct platform_device *pdev = vu_dev->pdev; + struct transport_vq_config tp_cfg = {}; struct virtio_uml_vq_info *info; struct virtqueue *vq; int num = MAX_SUPPORTED_QUEUE_SIZE; @@ -953,10 +954,15 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev, goto error_kzalloc; } snprintf(info->name, sizeof(info->name), "%s.%d-%s", pdev->name, - pdev->id, name); + pdev->id, cfg->names[cfg->cfg_idx]); - vq = vring_create_virtqueue(index, num, PAGE_SIZE, vdev, true, true, - ctx, vu_notify, callback, info->name); + tp_cfg.num = num; + tp_cfg.vring_align = PAGE_SIZE; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = true; + tp_cfg.notify = vu_notify; + + vq = vring_create_virtqueue(vdev, index, &tp_cfg, cfg); if (!vq) { rc = -ENOMEM; goto error_create; @@ -1013,12 +1019,11 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev, return ERR_PTR(rc); } -static int vu_find_vqs(struct virtio_device *vdev, unsigned nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc) +static int vu_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev); + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; int i, queue_idx = 0, rc; struct virtqueue *vq; @@ -1031,13 +1036,13 @@ static int vu_find_vqs(struct virtio_device *vdev, unsigned nvqs, return rc; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = vu_setup_vq(vdev, queue_idx++, callbacks[i], names[i], - ctx ? ctx[i] : false); + cfg->cfg_idx = i; + vqs[i] = vu_setup_vq(vdev, queue_idx++, cfg); if (IS_ERR(vqs[i])) { rc = PTR_ERR(vqs[i]); goto error_setup; diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c index 5c683b4eaf10..1b5593965068 100644 --- a/drivers/platform/mellanox/mlxbf-tmfifo.c +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c @@ -989,15 +989,12 @@ static void mlxbf_tmfifo_virtio_del_vqs(struct virtio_device *vdev) /* Create and initialize the virtual queues. */ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev, - unsigned int nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], - const bool *ctx, - struct irq_affinity *desc) + struct virtio_vq_config *cfg) { struct mlxbf_tmfifo_vdev *tm_vdev = mlxbf_vdev_to_tmfifo(vdev); + struct virtqueue **vqs = cfg->vqs; struct mlxbf_tmfifo_vring *vring; + unsigned int nvqs = cfg->nvqs; struct virtqueue *vq; int i, ret, size; @@ -1005,7 +1002,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev, return -EINVAL; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { ret = -EINVAL; goto error; } @@ -1014,10 +1011,11 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev, /* zero vring */ size = vring_size(vring->num, vring->align); memset(vring->va, 0, size); + vq = vring_new_virtqueue(i, vring->num, vring->align, vdev, false, false, vring->va, mlxbf_tmfifo_virtio_notify, - callbacks[i], names[i]); + cfg->callbacks[i], cfg->names[i]); if (!vq) { dev_err(&vdev->dev, "vring_new_virtqueue failed\n"); ret = -ENOMEM; diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c index 83d76915a6ad..57d51c9c7b63 100644 --- a/drivers/remoteproc/remoteproc_virtio.c +++ b/drivers/remoteproc/remoteproc_virtio.c @@ -102,8 +102,7 @@ EXPORT_SYMBOL(rproc_vq_interrupt); static struct virtqueue *rp_find_vq(struct virtio_device *vdev, unsigned int id, - void (*callback)(struct virtqueue *vq), - const char *name, bool ctx) + struct virtio_vq_config *cfg) { struct rproc_vdev *rvdev = vdev_to_rvdev(vdev); struct rproc *rproc = vdev_to_rproc(vdev); @@ -119,7 +118,7 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev, if (id >= ARRAY_SIZE(rvdev->vring)) return ERR_PTR(-EINVAL); - if (!name) + if (!cfg->names[cfg->cfg_idx]) return NULL; /* Search allocated memory region by name */ @@ -143,10 +142,12 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev, * Create the new vq, and tell virtio we're not interested in * the 'weak' smp barriers, since we're talking with a real device. */ - vq = vring_new_virtqueue(id, num, rvring->align, vdev, false, ctx, - addr, rproc_virtio_notify, callback, name); + vq = vring_new_virtqueue(id, num, rvring->align, vdev, false, + cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, + addr, rproc_virtio_notify, cfg->callbacks[cfg->cfg_idx], + cfg->names[cfg->cfg_idx]); if (!vq) { - dev_err(dev, "vring_new_virtqueue %s failed\n", name); + dev_err(dev, "vring_new_virtqueue %s failed\n", cfg->names[cfg->cfg_idx]); rproc_free_vring(rvring); return ERR_PTR(-ENOMEM); } @@ -180,23 +181,20 @@ static void rproc_virtio_del_vqs(struct virtio_device *vdev) __rproc_virtio_del_vqs(vdev); } -static int rproc_virtio_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], - const bool * ctx, - struct irq_affinity *desc) +static int rproc_virtio_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; int i, ret, queue_idx = 0; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = rp_find_vq(vdev, queue_idx++, callbacks[i], names[i], - ctx ? ctx[i] : false); + cfg->cfg_idx = i; + vqs[i] = rp_find_vq(vdev, queue_idx++, cfg); if (IS_ERR(vqs[i])) { ret = PTR_ERR(vqs[i]); goto error; diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c index ac67576301bf..c7734cd54187 100644 --- a/drivers/s390/virtio/virtio_ccw.c +++ b/drivers/s390/virtio/virtio_ccw.c @@ -499,11 +499,11 @@ static void virtio_ccw_del_vqs(struct virtio_device *vdev) } static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev, - int i, vq_callback_t *callback, - const char *name, bool ctx, - struct ccw1 *ccw) + int i, struct ccw1 *ccw, + struct virtio_vq_config *cfg) { struct virtio_ccw_device *vcdev = to_vc_device(vdev); + struct transport_vq_config tp_cfg = {}; bool (*notify)(struct virtqueue *vq); int err; struct virtqueue *vq = NULL; @@ -537,10 +537,14 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev, goto out_err; } may_reduce = vcdev->revision > 0; - vq = vring_create_virtqueue(i, info->num, KVM_VIRTIO_CCW_RING_ALIGN, - vdev, true, may_reduce, ctx, - notify, callback, name); + tp_cfg.num = info->num; + tp_cfg.vring_align = KVM_VIRTIO_CCW_RING_ALIGN; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = may_reduce; + tp_cfg.notify = notify; + + vq = vring_create_virtqueue(vdev, i, &tp_cfg, cfg); if (!vq) { /* For now, we fail if we can't get the requested size. */ dev_warn(&vcdev->cdev->dev, "no vq\n"); @@ -650,15 +654,13 @@ static int virtio_ccw_register_adapter_ind(struct virtio_ccw_device *vcdev, return ret; } -static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], - const bool *ctx, - struct irq_affinity *desc) +static int virtio_ccw_find_vqs(struct virtio_device *vdev, + struct virtio_vq_config *cfg) { struct virtio_ccw_device *vcdev = to_vc_device(vdev); + struct virtqueue **vqs = cfg->vqs; unsigned long *indicatorp = NULL; + unsigned int nvqs = cfg->nvqs; int ret, i, queue_idx = 0; struct ccw1 *ccw; @@ -667,14 +669,13 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs, return -ENOMEM; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = virtio_ccw_setup_vq(vdev, queue_idx++, callbacks[i], - names[i], ctx ? ctx[i] : false, - ccw); + cfg->cfg_idx = i; + vqs[i] = virtio_ccw_setup_vq(vdev, queue_idx++, ccw, cfg); if (IS_ERR(vqs[i])) { ret = PTR_ERR(vqs[i]); vqs[i] = NULL; diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c index 59892a31cf76..ceb7c312a616 100644 --- a/drivers/virtio/virtio_mmio.c +++ b/drivers/virtio/virtio_mmio.c @@ -370,10 +370,10 @@ static void vm_synchronize_cbs(struct virtio_device *vdev) } static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int index, - void (*callback)(struct virtqueue *vq), - const char *name, bool ctx) + struct virtio_vq_config *cfg) { struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev); + struct transport_vq_config tp_cfg = {}; bool (*notify)(struct virtqueue *vq); struct virtio_mmio_vq_info *info; struct virtqueue *vq; @@ -386,7 +386,7 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int in else notify = vm_notify; - if (!name) + if (!cfg->names[index]) return NULL; /* Select the queue we're interested in */ @@ -412,9 +412,14 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int in goto error_new_virtqueue; } + tp_cfg.num = num; + tp_cfg.vring_align = VIRTIO_MMIO_VRING_ALIGN; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = true; + tp_cfg.notify = notify; + /* Create the vring */ - vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev, - true, true, ctx, notify, callback, name); + vq = vring_create_virtqueue(vdev, index, &tp_cfg, cfg); if (!vq) { err = -ENOMEM; goto error_new_virtqueue; @@ -487,15 +492,12 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int in return ERR_PTR(err); } -static int vm_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], - const bool *ctx, - struct irq_affinity *desc) +static int vm_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev); int irq = platform_get_irq(vm_dev->pdev, 0); + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; int i, err, queue_idx = 0; if (irq < 0) @@ -510,13 +512,13 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned int nvqs, enable_irq_wake(irq); for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = vm_setup_vq(vdev, queue_idx++, callbacks[i], names[i], - ctx ? ctx[i] : false); + cfg->cfg_idx = i; + vqs[i] = vm_setup_vq(vdev, queue_idx++, cfg); if (IS_ERR(vqs[i])) { vm_del_vqs(vdev); return PTR_ERR(vqs[i]); diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c index 1d21d1a1b3f5..0ebee2b53eed 100644 --- a/drivers/virtio/virtio_pci_common.c +++ b/drivers/virtio/virtio_pci_common.c @@ -172,9 +172,7 @@ static int vp_request_msix_vectors(struct virtio_device *vdev, int nvectors, } static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned int index, - void (*callback)(struct virtqueue *vq), - const char *name, - bool ctx, + struct virtio_vq_config *cfg, u16 msix_vec) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); @@ -186,13 +184,13 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned int in if (!info) return ERR_PTR(-ENOMEM); - vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, ctx, + vq = vp_dev->setup_vq(vp_dev, info, index, cfg, msix_vec); if (IS_ERR(vq)) goto out_info; info->vq = vq; - if (callback) { + if (cfg->callbacks[cfg->cfg_idx]) { spin_lock_irqsave(&vp_dev->lock, flags); list_add(&info->node, &vp_dev->virtqueues); spin_unlock_irqrestore(&vp_dev->lock, flags); @@ -281,15 +279,15 @@ void vp_del_vqs(struct virtio_device *vdev) vp_dev->vqs = NULL; } -static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], bool per_vq_vectors, - const bool *ctx, - struct irq_affinity *desc) +static int vp_find_vqs_msix(struct virtio_device *vdev, + struct virtio_vq_config *cfg, + bool per_vq_vectors) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); u16 msix_vec; int i, err, nvectors, allocated_vectors, queue_idx = 0; + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; vp_dev->vqs = kcalloc(nvqs, sizeof(*vp_dev->vqs), GFP_KERNEL); if (!vp_dev->vqs) @@ -299,7 +297,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned int nvqs, /* Best option: one for change interrupt, one per vq. */ nvectors = 1; for (i = 0; i < nvqs; ++i) - if (names[i] && callbacks[i]) + if (cfg->names[i] && cfg->callbacks[i]) ++nvectors; } else { /* Second best: one for change, shared for all vqs. */ @@ -307,27 +305,27 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned int nvqs, } err = vp_request_msix_vectors(vdev, nvectors, per_vq_vectors, - per_vq_vectors ? desc : NULL); + per_vq_vectors ? cfg->desc : NULL); if (err) goto error_find; vp_dev->per_vq_vectors = per_vq_vectors; allocated_vectors = vp_dev->msix_used_vectors; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - if (!callbacks[i]) + if (!cfg->callbacks[i]) msix_vec = VIRTIO_MSI_NO_VECTOR; else if (vp_dev->per_vq_vectors) msix_vec = allocated_vectors++; else msix_vec = VP_MSIX_VQ_VECTOR; - vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i], - ctx ? ctx[i] : false, - msix_vec); + + cfg->cfg_idx = i; + vqs[i] = vp_setup_vq(vdev, queue_idx++, cfg, msix_vec); if (IS_ERR(vqs[i])) { err = PTR_ERR(vqs[i]); goto error_find; @@ -340,7 +338,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned int nvqs, snprintf(vp_dev->msix_names[msix_vec], sizeof *vp_dev->msix_names, "%s-%s", - dev_name(&vp_dev->vdev.dev), names[i]); + dev_name(&vp_dev->vdev.dev), cfg->names[i]); err = request_irq(pci_irq_vector(vp_dev->pci_dev, msix_vec), vring_interrupt, 0, vp_dev->msix_names[msix_vec], @@ -355,11 +353,11 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned int nvqs, return err; } -static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx) +static int vp_find_vqs_intx(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; int i, err, queue_idx = 0; vp_dev->vqs = kcalloc(nvqs, sizeof(*vp_dev->vqs), GFP_KERNEL); @@ -374,13 +372,13 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned int nvqs, vp_dev->intx_enabled = 1; vp_dev->per_vq_vectors = false; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i], - ctx ? ctx[i] : false, - VIRTIO_MSI_NO_VECTOR); + + cfg->cfg_idx = i; + vqs[i] = vp_setup_vq(vdev, queue_idx++, cfg, VIRTIO_MSI_NO_VECTOR); if (IS_ERR(vqs[i])) { err = PTR_ERR(vqs[i]); goto out_del_vqs; @@ -394,26 +392,23 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned int nvqs, } /* the config->find_vqs() implementation */ -int vp_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc) +int vp_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { int err; /* Try MSI-X with one vector per queue. */ - err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, true, ctx, desc); + err = vp_find_vqs_msix(vdev, cfg, true); if (!err) return 0; /* Fallback: MSI-X with one vector for config, one shared for queues. */ - err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, false, ctx, desc); + err = vp_find_vqs_msix(vdev, cfg, false); if (!err) return 0; /* Is there an interrupt? If not give up. */ if (!(to_vp_device(vdev)->pci_dev->irq)) return err; /* Finally fall back to regular interrupts. */ - return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, ctx); + return vp_find_vqs_intx(vdev, cfg); } const char *vp_bus_name(struct virtio_device *vdev) diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h index 4b773bd7c58c..12b171364e54 100644 --- a/drivers/virtio/virtio_pci_common.h +++ b/drivers/virtio/virtio_pci_common.h @@ -79,9 +79,7 @@ struct virtio_pci_device { struct virtqueue *(*setup_vq)(struct virtio_pci_device *vp_dev, struct virtio_pci_vq_info *info, unsigned int idx, - void (*callback)(struct virtqueue *vq), - const char *name, - bool ctx, + struct virtio_vq_config *vq_cfg, u16 msix_vec); void (*del_vq)(struct virtio_pci_vq_info *info); @@ -109,10 +107,7 @@ bool vp_notify(struct virtqueue *vq); /* the config->del_vqs() implementation */ void vp_del_vqs(struct virtio_device *vdev); /* the config->find_vqs() implementation */ -int vp_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc); +int vp_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg); const char *vp_bus_name(struct virtio_device *vdev); /* Setup the affinity for a virtqueue: diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c index d9cbb02b35a1..508a31a81499 100644 --- a/drivers/virtio/virtio_pci_legacy.c +++ b/drivers/virtio/virtio_pci_legacy.c @@ -110,11 +110,10 @@ static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector) static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, struct virtio_pci_vq_info *info, unsigned int index, - void (*callback)(struct virtqueue *vq), - const char *name, - bool ctx, + struct virtio_vq_config *cfg, u16 msix_vec) { + struct transport_vq_config tp_cfg = {}; struct virtqueue *vq; u16 num; int err; @@ -127,11 +126,14 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, info->msix_vector = msix_vec; + tp_cfg.num = num; + tp_cfg.vring_align = VIRTIO_PCI_VRING_ALIGN; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = false; + tp_cfg.notify = vp_notify; + /* create the vring */ - vq = vring_create_virtqueue(index, num, - VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev, - true, false, ctx, - vp_notify, callback, name); + vq = vring_create_virtqueue(&vp_dev->vdev, index, &tp_cfg, cfg); if (!vq) return ERR_PTR(-ENOMEM); diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c index ee6a386d250b..9caca1eeb726 100644 --- a/drivers/virtio/virtio_pci_modern.c +++ b/drivers/virtio/virtio_pci_modern.c @@ -336,13 +336,12 @@ static bool vp_notify_with_data(struct virtqueue *vq) static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, struct virtio_pci_vq_info *info, unsigned int index, - void (*callback)(struct virtqueue *vq), - const char *name, - bool ctx, + struct virtio_vq_config *cfg, u16 msix_vec) { struct virtio_pci_modern_device *mdev = &vp_dev->mdev; + struct transport_vq_config tp_cfg = {}; bool (*notify)(struct virtqueue *vq); struct virtqueue *vq; u16 num; @@ -363,11 +362,14 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, info->msix_vector = msix_vec; + tp_cfg.num = num; + tp_cfg.vring_align = SMP_CACHE_BYTES; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = true; + tp_cfg.notify = notify; + /* create the vring */ - vq = vring_create_virtqueue(index, num, - SMP_CACHE_BYTES, &vp_dev->vdev, - true, true, ctx, - notify, callback, name); + vq = vring_create_virtqueue(&vp_dev->vdev, index, &tp_cfg, cfg); if (!vq) return ERR_PTR(-ENOMEM); @@ -390,15 +392,11 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, return ERR_PTR(err); } -static int vp_modern_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc) +static int vp_modern_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); struct virtqueue *vq; - int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, desc); + int rc = vp_find_vqs(vdev, cfg); if (rc) return rc; diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 5bea25167259..5652ff91c6f9 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2726,43 +2726,34 @@ static struct virtqueue *__vring_new_virtqueue(unsigned int index, return &vq->vq; } -struct virtqueue *vring_create_virtqueue( - unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name) +struct virtqueue *vring_create_virtqueue(struct virtio_device *vdev, + unsigned int index, + struct transport_vq_config *tp_cfg, + struct virtio_vq_config *cfg) { + struct device *dma_dev; + unsigned int num; + unsigned int vring_align; + bool weak_barriers; + bool may_reduce_num; + bool context; + bool (*notify)(struct virtqueue *_); + void (*callback)(struct virtqueue *_); + const char *name; - if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) - return vring_create_virtqueue_packed(index, num, vring_align, - vdev, weak_barriers, may_reduce_num, - context, notify, callback, name, vdev->dev.parent); + dma_dev = tp_cfg->dma_dev; + if (!dma_dev) + dma_dev = vdev->dev.parent; - return vring_create_virtqueue_split(index, num, vring_align, - vdev, weak_barriers, may_reduce_num, - context, notify, callback, name, vdev->dev.parent); -} -EXPORT_SYMBOL_GPL(vring_create_virtqueue); + num = tp_cfg->num; + vring_align = tp_cfg->vring_align; + weak_barriers = tp_cfg->weak_barriers; + may_reduce_num = tp_cfg->may_reduce_num; + notify = tp_cfg->notify; -struct virtqueue *vring_create_virtqueue_dma( - unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name, - struct device *dma_dev) -{ + name = cfg->names[cfg->cfg_idx]; + callback = cfg->callbacks[cfg->cfg_idx]; + context = cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false; if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) return vring_create_virtqueue_packed(index, num, vring_align, @@ -2773,7 +2764,7 @@ struct virtqueue *vring_create_virtqueue_dma( vdev, weak_barriers, may_reduce_num, context, notify, callback, name, dma_dev); } -EXPORT_SYMBOL_GPL(vring_create_virtqueue_dma); +EXPORT_SYMBOL_GPL(vring_create_virtqueue); /** * virtqueue_resize - resize the vring of vq diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c index 8d63e5923d24..dd58d23f711e 100644 --- a/drivers/virtio/virtio_vdpa.c +++ b/drivers/virtio/virtio_vdpa.c @@ -147,7 +147,7 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index, { struct virtio_vdpa_device *vd_dev = to_virtio_vdpa_device(vdev); struct vdpa_device *vdpa = vd_get_vdpa(vdev); - struct device *dma_dev; + struct transport_vq_config tp_cfg = {}; const struct vdpa_config_ops *ops = vdpa->config; struct virtio_vdpa_vq_info *info; bool (*notify)(struct virtqueue *vq) = virtio_vdpa_notify; @@ -199,12 +199,17 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index, align = ops->get_vq_align(vdpa); if (ops->get_vq_dma_dev) - dma_dev = ops->get_vq_dma_dev(vdpa, index); + tp_cfg.dma_dev = ops->get_vq_dma_dev(vdpa, index); else - dma_dev = vdpa_get_dma_dev(vdpa); - vq = vring_create_virtqueue_dma(index, max_num, align, vdev, - true, may_reduce_num, ctx, - notify, callback, name, dma_dev); + tp_cfg.dma_dev = vdpa_get_dma_dev(vdpa); + + tp_cfg.num = max_num; + tp_cfg.vring_align = align; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = may_reduce_num; + tp_cfg.notify = notify; + + vq = vring_create_virtqueue(vdev, index, &tp_cfg, cfg); if (!vq) { err = -ENOMEM; goto error_new_virtqueue; @@ -353,11 +358,8 @@ create_affinity_masks(unsigned int nvecs, struct irq_affinity *affd) return masks; } -static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], - const bool *ctx, +static int virtio_vdpa_find_vqs(struct virtio_device *vdev, + struct virtio_vq_config *cfg, struct irq_affinity *desc) { struct virtio_vdpa_device *vd_dev = to_virtio_vdpa_device(vdev); @@ -367,6 +369,8 @@ static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned int nvqs, struct cpumask *masks; struct vdpa_callback cb; bool has_affinity = desc && ops->set_vq_affinity; + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; int i, err, queue_idx = 0; if (has_affinity) { @@ -376,14 +380,13 @@ static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned int nvqs, } for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = virtio_vdpa_setup_vq(vdev, queue_idx++, - callbacks[i], names[i], ctx ? - ctx[i] : false); + cfg->cfg_idx = i; + vqs[i] = virtio_vdpa_setup_vq(vdev, queue_idx++, cfg); if (IS_ERR(vqs[i])) { err = PTR_ERR(vqs[i]); goto err_setup_vq; diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h index 2b3438de2c4d..e2c72e125dae 100644 --- a/include/linux/virtio_config.h +++ b/include/linux/virtio_config.h @@ -94,6 +94,20 @@ typedef void vq_callback_t(struct virtqueue *); * If disable_vq_and_reset is set, then enable_vq_after_reset must also be * set. */ + +struct virtio_vq_config { + unsigned int nvqs; + + /* the vq index may not eq to the cfg index of the other array items */ + unsigned int cfg_idx; + + struct virtqueue **vqs; + vq_callback_t **callbacks; + const char *const *names; + const bool *ctx; + struct irq_affinity *desc; +}; + struct virtio_config_ops { void (*get)(struct virtio_device *vdev, unsigned offset, void *buf, unsigned len); @@ -103,10 +117,7 @@ struct virtio_config_ops { u8 (*get_status)(struct virtio_device *vdev); void (*set_status)(struct virtio_device *vdev, u8 status); void (*reset)(struct virtio_device *vdev); - int (*find_vqs)(struct virtio_device *, unsigned nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc); + int (*find_vqs)(struct virtio_device *vdev, struct virtio_vq_config *cfg); void (*del_vqs)(struct virtio_device *); void (*synchronize_cbs)(struct virtio_device *); u64 (*get_features)(struct virtio_device *vdev); @@ -213,8 +224,14 @@ struct virtqueue *virtio_find_single_vq(struct virtio_device *vdev, vq_callback_t *callbacks[] = { c }; const char *names[] = { n }; struct virtqueue *vq; - int err = vdev->config->find_vqs(vdev, 1, &vq, callbacks, names, NULL, - NULL); + struct virtio_vq_config cfg = {}; + + cfg.nvqs = 1; + cfg.vqs = &vq; + cfg.callbacks = callbacks; + cfg.names = names; + + int err = vdev->config->find_vqs(vdev, &cfg); if (err < 0) return ERR_PTR(err); return vq; @@ -226,7 +243,15 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs, const char * const names[], struct irq_affinity *desc) { - return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL, desc); + struct virtio_vq_config cfg = {}; + + cfg.nvqs = nvqs; + cfg.vqs = vqs; + cfg.callbacks = callbacks; + cfg.names = names; + cfg.desc = desc; + + return vdev->config->find_vqs(vdev, &cfg); } static inline @@ -235,8 +260,16 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs, const char * const names[], const bool *ctx, struct irq_affinity *desc) { - return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, - desc); + struct virtio_vq_config cfg = {}; + + cfg.nvqs = nvqs; + cfg.vqs = vqs; + cfg.callbacks = callbacks; + cfg.names = names; + cfg.ctx = ctx; + cfg.desc = desc; + + return vdev->config->find_vqs(vdev, &cfg); } /** diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h index 9b33df741b63..0de46ed17cc0 100644 --- a/include/linux/virtio_ring.h +++ b/include/linux/virtio_ring.h @@ -5,6 +5,7 @@ #include #include #include +#include /* * Barriers in virtio are tricky. Non-SMP virtio guests can't assume @@ -60,38 +61,25 @@ struct virtio_device; struct virtqueue; struct device; +struct transport_vq_config { + unsigned int num; + unsigned int vring_align; + bool weak_barriers; + bool may_reduce_num; + bool (*notify)(struct virtqueue *vq); + struct device *dma_dev; +}; + /* * Creates a virtqueue and allocates the descriptor ring. If * may_reduce_num is set, then this may allocate a smaller ring than * expected. The caller should query virtqueue_get_vring_size to learn * the actual size of the ring. */ -struct virtqueue *vring_create_virtqueue(unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool ctx, - bool (*notify)(struct virtqueue *vq), - void (*callback)(struct virtqueue *vq), - const char *name); - -/* - * Creates a virtqueue and allocates the descriptor ring with per - * virtqueue DMA device. - */ -struct virtqueue *vring_create_virtqueue_dma(unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool ctx, - bool (*notify)(struct virtqueue *vq), - void (*callback)(struct virtqueue *vq), - const char *name, - struct device *dma_dev); +struct virtqueue *vring_create_virtqueue(struct virtio_device *vdev, + unsigned int index, + struct transport_vq_config *tp_cfg, + struct virtio_vq_config *cfg); /* * Creates a virtqueue with a standard layout but a caller-allocated From patchwork Tue Jan 30 11:42:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537417 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4A516D1DC; Tue, 30 Jan 2024 11:42:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.132 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614971; cv=none; b=WZeVtKBp3WLpwoheQXWnNNJXPtO/1Ua0/VnagBm9xE7i42rSIoIwguEH+Er1HZ0N1LEnrAV5F0HGEYfLVgJaOdhnZ+ZKC36PHX2np+ICHtCMO2LeLetxetVTd4QplRNIx1DYlhiuUBdpXFSvQTj4xsu8tFb+hTWmIUYxfyud2T0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614971; c=relaxed/simple; bh=5IgO3w2jHi8Q2MuL+rtVJApgugyMR6DBtap1jTRrado=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JS+aVG1UUbvIM55bMfM6DJIrPbwmdLg3bbJjlZ9AQz6aypsdfQY3BmD3g8CdF3GBOsj3mE1IaOHiZ/Bqsm/5IJrUTwzX8qSg+kB9Mc9MZiP1xUL+zrendtZKzoCbw0tPERDmArq3EaH9OZB7exwSFUXpJjpcnY9CClwoC5EJB+s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=yfRxuvRe; arc=none smtp.client-ip=115.124.30.132 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="yfRxuvRe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614961; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=VKC8H6yGQqai4Wir7Iq0SqQqScYgwESPvSh8zQB9EUA=; b=yfRxuvResV8YIV4Od6r8JuIwl9txQUo53YyzMaaVi4HvVrdqxwKTnuNaDFYDDddXMrwMLzB8DnbzU7zbI4nxLECv9tfOIW1rd1jgscH9yRPrrWKL3mkBZppmBK6YfoK7656e99smjtSzVIRQ72YTxPgegIPfE5drqnnHbzI9Hkc= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R861e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046050;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g61UH_1706614958; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g61UH_1706614958) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:38 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 08/17] virtio: vring_new_virtqueue(): pass struct instead of multi parameters Date: Tue, 30 Jan 2024 19:42:15 +0800 Message-Id: <20240130114224.86536-9-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 Just like find_vqs(), it is time to refactor the vring_new_virtqueue(). We pass the similar struct to vring_new_virtqueue. Signed-off-by: Xuan Zhuo --- drivers/platform/mellanox/mlxbf-tmfifo.c | 12 ++++--- drivers/remoteproc/remoteproc_virtio.c | 11 ++++--- drivers/virtio/virtio_ring.c | 29 +++++++++++----- include/linux/virtio_ring.h | 42 +++++++++++++++++++----- tools/virtio/virtio_test.c | 4 +-- tools/virtio/vringh_test.c | 32 +++++++++--------- 6 files changed, 86 insertions(+), 44 deletions(-) diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c index 1b5593965068..765f54a16bf7 100644 --- a/drivers/platform/mellanox/mlxbf-tmfifo.c +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c @@ -992,6 +992,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct mlxbf_tmfifo_vdev *tm_vdev = mlxbf_vdev_to_tmfifo(vdev); + struct transport_vq_config tp_cfg = {}; struct virtqueue **vqs = cfg->vqs; struct mlxbf_tmfifo_vring *vring; unsigned int nvqs = cfg->nvqs; @@ -1012,10 +1013,13 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev, size = vring_size(vring->num, vring->align); memset(vring->va, 0, size); - vq = vring_new_virtqueue(i, vring->num, vring->align, vdev, - false, false, vring->va, - mlxbf_tmfifo_virtio_notify, - cfg->callbacks[i], cfg->names[i]); + tp_cfg.num = vring->num; + tp_cfg.vring_align = vring->align; + tp_cfg.weak_barriers = false; + tp_cfg.notify = mlxbf_tmfifo_virtio_notify; + + cfg->cfg_idx = i; + vq = vring_new_virtqueue(vdev, i, vring->va, &tp_cfg, cfg); if (!vq) { dev_err(&vdev->dev, "vring_new_virtqueue failed\n"); ret = -ENOMEM; diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c index 57d51c9c7b63..d1212361da30 100644 --- a/drivers/remoteproc/remoteproc_virtio.c +++ b/drivers/remoteproc/remoteproc_virtio.c @@ -106,6 +106,7 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev, { struct rproc_vdev *rvdev = vdev_to_rvdev(vdev); struct rproc *rproc = vdev_to_rproc(vdev); + struct transport_vq_config tp_cfg; struct device *dev = &rproc->dev; struct rproc_mem_entry *mem; struct rproc_vring *rvring; @@ -138,14 +139,16 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev, dev_dbg(dev, "vring%d: va %pK qsz %d notifyid %d\n", id, addr, num, rvring->notifyid); + tp_cfg.num = num; + tp_cfg.vring_align = rvring->align; + tp_cfg.weak_barriers = false; + tp_cfg.notify = rproc_virtio_notify; + /* * Create the new vq, and tell virtio we're not interested in * the 'weak' smp barriers, since we're talking with a real device. */ - vq = vring_new_virtqueue(id, num, rvring->align, vdev, false, - cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, - addr, rproc_virtio_notify, cfg->callbacks[cfg->cfg_idx], - cfg->names[cfg->cfg_idx]); + vq = vring_new_virtqueue(vdev, id, addr, &tp_cfg, cfg); if (!vq) { dev_err(dev, "vring_new_virtqueue %s failed\n", cfg->names[cfg->cfg_idx]); rproc_free_vring(rvring); diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 5652ff91c6f9..d87b80a5b9df 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2908,18 +2908,29 @@ int virtqueue_reset(struct virtqueue *_vq, EXPORT_SYMBOL_GPL(virtqueue_reset); /* Only available for split ring */ -struct virtqueue *vring_new_virtqueue(unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool context, +struct virtqueue *vring_new_virtqueue(struct virtio_device *vdev, + unsigned int index, void *pages, - bool (*notify)(struct virtqueue *vq), - void (*callback)(struct virtqueue *vq), - const char *name) + struct transport_vq_config *tp_cfg, + struct virtio_vq_config *cfg) { struct vring_virtqueue_split vring_split = {}; + unsigned int num; + unsigned int vring_align; + bool weak_barriers; + bool context; + bool (*notify)(struct virtqueue *_); + void (*callback)(struct virtqueue *_); + const char *name; + + num = tp_cfg->num; + vring_align = tp_cfg->vring_align; + weak_barriers = tp_cfg->weak_barriers; + notify = tp_cfg->notify; + + name = cfg->names[cfg->cfg_idx]; + callback = cfg->callbacks[cfg->cfg_idx]; + context = cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false; if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) return NULL; diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h index 0de46ed17cc0..316dadcde47a 100644 --- a/include/linux/virtio_ring.h +++ b/include/linux/virtio_ring.h @@ -85,16 +85,40 @@ struct virtqueue *vring_create_virtqueue(struct virtio_device *vdev, * Creates a virtqueue with a standard layout but a caller-allocated * ring. */ -struct virtqueue *vring_new_virtqueue(unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool ctx, +struct virtqueue *vring_new_virtqueue(struct virtio_device *vdev, + unsigned int index, void *pages, - bool (*notify)(struct virtqueue *vq), - void (*callback)(struct virtqueue *vq), - const char *name); + struct transport_vq_config *tp_cfg, + struct virtio_vq_config *cfg); + +static inline struct virtqueue *vring_new_virtqueue_one(unsigned int index, + unsigned int num, + unsigned int vring_align, + struct virtio_device *vdev, + bool weak_barriers, + bool context, + void *pages, + bool (*notify)(struct virtqueue *vq), + void (*callback)(struct virtqueue *vq), + const char *name) +{ + struct transport_vq_config tp_cfg = {}; + struct virtio_vq_config cfg = {}; + vq_callback_t *callbacks[] = { callback }; + const char *names[] = { name }; + + tp_cfg.num = num; + tp_cfg.vring_align = vring_align; + tp_cfg.weak_barriers = weak_barriers; + tp_cfg.notify = notify; + + cfg.nvqs = 1; + cfg.callbacks = callbacks; + cfg.names = names; + cfg.ctx = &context; + + return vring_new_virtqueue(vdev, index, pages, &tp_cfg, &cfg); +} /* * Destroys a virtqueue. If created with vring_create_virtqueue, this diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c index 028f54e6854a..e41300d71d5e 100644 --- a/tools/virtio/virtio_test.c +++ b/tools/virtio/virtio_test.c @@ -102,8 +102,8 @@ static void vq_reset(struct vq_info *info, int num, struct virtio_device *vdev) memset(info->ring, 0, vring_size(num, 4096)); vring_init(&info->vring, num, info->ring, 4096); - info->vq = vring_new_virtqueue(info->idx, num, 4096, vdev, true, false, - info->ring, vq_notify, vq_callback, "test"); + info->vq = vring_new_virtqueue_one(info->idx, num, 4096, vdev, true, false, + info->ring, vq_notify, vq_callback, "test"); assert(info->vq); info->vq->priv = info; } diff --git a/tools/virtio/vringh_test.c b/tools/virtio/vringh_test.c index 98ff808d6f0c..37f8c5d34285 100644 --- a/tools/virtio/vringh_test.c +++ b/tools/virtio/vringh_test.c @@ -316,11 +316,11 @@ static int parallel_test(u64 features, if (sched_setaffinity(getpid(), sizeof(cpu_set), &cpu_set)) err(1, "Could not set affinity to cpu %u", first_cpu); - vq = vring_new_virtqueue(0, RINGSIZE, ALIGN, &gvdev.vdev, true, - false, guest_map, - fast_vringh ? no_notify_host - : parallel_notify_host, - never_callback_guest, "guest vq"); + vq = vring_new_virtqueue_one(0, RINGSIZE, ALIGN, &gvdev.vdev, true, + false, guest_map, + fast_vringh ? no_notify_host + : parallel_notify_host, + never_callback_guest, "guest vq"); /* Don't kfree indirects. */ __kfree_ignore_start = indirects; @@ -391,7 +391,7 @@ static int parallel_test(u64 features, /* Swallow all notifies at once. */ if (read(to_guest[0], buf, sizeof(buf)) < 1) break; - + receives++; virtqueue_disable_cb(vq); continue; @@ -424,7 +424,7 @@ static int parallel_test(u64 features, continue; if (read(to_guest[0], buf, sizeof(buf)) < 1) break; - + receives++; virtqueue_disable_cb(vq); } @@ -485,10 +485,10 @@ int main(int argc, char *argv[]) memset(__user_addr_min, 0, vring_size(RINGSIZE, ALIGN)); /* Set up guest side. */ - vq = vring_new_virtqueue(0, RINGSIZE, ALIGN, &vdev, true, false, - __user_addr_min, - never_notify_host, never_callback_guest, - "guest vq"); + vq = vring_new_virtqueue_one(0, RINGSIZE, ALIGN, &vdev, true, false, + __user_addr_min, + never_notify_host, never_callback_guest, + "guest vq"); /* Set up host side. */ vring_init(&vrh.vring, RINGSIZE, __user_addr_min, ALIGN); @@ -668,11 +668,11 @@ int main(int argc, char *argv[]) /* Force creation of direct, which we modify. */ __virtio_clear_bit(&vdev, VIRTIO_RING_F_INDIRECT_DESC); - vq = vring_new_virtqueue(0, RINGSIZE, ALIGN, &vdev, true, - false, __user_addr_min, - never_notify_host, - never_callback_guest, - "guest vq"); + vq = vring_new_virtqueue_one(0, RINGSIZE, ALIGN, &vdev, true, + false, __user_addr_min, + never_notify_host, + never_callback_guest, + "guest vq"); sg_init_table(guest_sg, 4); sg_set_buf(&guest_sg[0], d, sizeof(*d)*2); From patchwork Tue Jan 30 11:42:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537420 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D967F7C08F; Tue, 30 Jan 2024 11:42:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.132 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614974; cv=none; b=Jfek4Ia/ZLaxTsYLHSjcLddI5QejqH6lKvHw4WpsQNMnUZZY1qYl7lVrjDdLIzE75u5ORO/TYbBGsLrwc6R16g27DCaVxokGVHLclQ3khDHOLs3Odoe/w8YZlmoHDJ20BttyrSjfZ113pBC2xIv54Ugxi98Nak0pH/LsumLzXgM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614974; c=relaxed/simple; bh=S85QgqcqXlX88ExzF4q47DLua5UgAiU2ZQs/BOj4wAA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AAxWnfZftFki8aWw9MHoA3DEjcu6cxXj4WsWMUtj6blT0d27w14MUEOdMt48ivOmxmBLN/uxrs5YQmca2qg8uhC8/rBQrl9kDuQ95OQekTUk1nPh4WqaQvF3cI0njpzCh/W/wYVT/E/zM0ecsjkDIqEaAmdDcmJHlVw1oW2W9DQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=Cf8lajC8; arc=none smtp.client-ip=115.124.30.132 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="Cf8lajC8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614963; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=5y28Ihr+AJzyTWf01VNyD6gpI2+tIOJhKvCZ6gW/9DA=; b=Cf8lajC8ACJwvmXGaofg5bvKXb0H9bbJhrknl+ez1C+3ksaCeX1C0Gx+WSAhvKwSrSvKsyUTH7NK5tBDx4sjum0kduljtkBlu/YWTOV4zr8wf3OKimev39JYgNUUaeV7XgPiMZoE//XSBKNBEXUj8ZdBJX2gFKxcEkaIzunQDmo= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R541e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g.laf_1706614959; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g.laf_1706614959) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:40 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 09/17] virtio_ring: reuse the parameter struct of find_vqs() Date: Tue, 30 Jan 2024 19:42:16 +0800 Message-Id: <20240130114224.86536-10-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 As the refactor of find_vqs()/vring_new_virtqueue(), the cfg/tp_cfg are passed to vring. This patch refactors the vring by this structure. This can simplify the code. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 162 ++++++++++++----------------------- 1 file changed, 53 insertions(+), 109 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index d87b80a5b9df..769ec6360d7f 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -237,15 +237,11 @@ struct vring_virtqueue { #endif }; -static struct virtqueue *__vring_new_virtqueue(unsigned int index, +static struct virtqueue *__vring_new_virtqueue(struct virtio_device *vdev, + unsigned int index, struct vring_virtqueue_split *vring_split, - struct virtio_device *vdev, - bool weak_barriers, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name, - struct device *dma_dev); + struct transport_vq_config *tp_cfg, + struct virtio_vq_config *cfg); static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num); static void vring_free(struct virtqueue *_vq); @@ -254,6 +250,8 @@ static void vring_free(struct virtqueue *_vq); */ #define to_vvq(_vq) container_of_const(_vq, struct vring_virtqueue, vq) +#define cfg_vq_val(cfg, key) (cfg->key[cfg->cfg_idx]) +#define cfg_vq_get(cfg, key) (cfg->key ? cfg_vq_val(cfg, key) : false) static bool virtqueue_use_indirect(const struct vring_virtqueue *vq, unsigned int total_sg) @@ -1184,32 +1182,29 @@ static int vring_alloc_queue_split(struct vring_virtqueue_split *vring_split, return 0; } -static struct virtqueue *vring_create_virtqueue_split( - unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name, - struct device *dma_dev) +static struct virtqueue *vring_create_virtqueue_split(struct virtio_device *vdev, + unsigned int index, + struct transport_vq_config *tp_cfg, + struct virtio_vq_config *cfg) { struct vring_virtqueue_split vring_split = {}; struct virtqueue *vq; int err; - err = vring_alloc_queue_split(&vring_split, vdev, num, vring_align, - may_reduce_num, dma_dev); + if (!tp_cfg->dma_dev) + tp_cfg->dma_dev = vdev->dev.parent; + + err = vring_alloc_queue_split(&vring_split, vdev, + tp_cfg->num, + tp_cfg->vring_align, + tp_cfg->may_reduce_num, + tp_cfg->dma_dev); if (err) return NULL; - vq = __vring_new_virtqueue(index, &vring_split, vdev, weak_barriers, - context, notify, callback, name, dma_dev); + vq = __vring_new_virtqueue(vdev, index, &vring_split, tp_cfg, cfg); if (!vq) { - vring_free_split(&vring_split, vdev, dma_dev); + vring_free_split(&vring_split, vdev, tp_cfg->dma_dev); return NULL; } @@ -2113,38 +2108,35 @@ static void virtqueue_reinit_packed(struct vring_virtqueue *vq) virtqueue_vring_init_packed(&vq->packed, !!vq->vq.callback); } -static struct virtqueue *vring_create_virtqueue_packed( - unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name, - struct device *dma_dev) +static struct virtqueue *vring_create_virtqueue_packed(struct virtio_device *vdev, + unsigned int index, + struct transport_vq_config *tp_cfg, + struct virtio_vq_config *cfg) { struct vring_virtqueue_packed vring_packed = {}; struct vring_virtqueue *vq; + struct device *dma_dev; int err; - if (vring_alloc_queue_packed(&vring_packed, vdev, num, dma_dev)) + dma_dev = tp_cfg->dma_dev; + if (!dma_dev) + dma_dev = vdev->dev.parent; + + if (vring_alloc_queue_packed(&vring_packed, vdev, tp_cfg->num, dma_dev)) goto err_ring; vq = kmalloc(sizeof(*vq), GFP_KERNEL); if (!vq) goto err_vq; - vq->vq.callback = callback; + vq->vq.callback = cfg_vq_val(cfg, callbacks); vq->vq.vdev = vdev; - vq->vq.name = name; + vq->vq.name = cfg_vq_val(cfg, names); vq->vq.index = index; vq->vq.reset = false; vq->we_own_ring = true; - vq->notify = notify; - vq->weak_barriers = weak_barriers; + vq->notify = tp_cfg->notify; + vq->weak_barriers = tp_cfg->weak_barriers; #ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION vq->broken = true; #else @@ -2156,7 +2148,7 @@ static struct virtqueue *vring_create_virtqueue_packed( vq->premapped = false; vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && - !context; + !cfg_vq_get(cfg, ctx); vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX); if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM)) @@ -2167,9 +2159,9 @@ static struct virtqueue *vring_create_virtqueue_packed( if (err) goto err_state_extra; - virtqueue_vring_init_packed(&vring_packed, !!callback); + virtqueue_vring_init_packed(&vring_packed, !!cfg_vq_val(cfg, callbacks)); - virtqueue_init(vq, num); + virtqueue_init(vq, tp_cfg->num); virtqueue_vring_attach_packed(vq, &vring_packed); spin_lock(&vdev->vqs_list_lock); @@ -2663,15 +2655,11 @@ irqreturn_t vring_interrupt(int irq, void *_vq) EXPORT_SYMBOL_GPL(vring_interrupt); /* Only available for split ring */ -static struct virtqueue *__vring_new_virtqueue(unsigned int index, +static struct virtqueue *__vring_new_virtqueue(struct virtio_device *vdev, + unsigned int index, struct vring_virtqueue_split *vring_split, - struct virtio_device *vdev, - bool weak_barriers, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name, - struct device *dma_dev) + struct transport_vq_config *tp_cfg, + struct virtio_vq_config *cfg) { struct vring_virtqueue *vq; int err; @@ -2684,25 +2672,25 @@ static struct virtqueue *__vring_new_virtqueue(unsigned int index, return NULL; vq->packed_ring = false; - vq->vq.callback = callback; + vq->vq.callback = cfg_vq_val(cfg, callbacks); vq->vq.vdev = vdev; - vq->vq.name = name; + vq->vq.name = cfg_vq_val(cfg, names); vq->vq.index = index; vq->vq.reset = false; vq->we_own_ring = false; - vq->notify = notify; - vq->weak_barriers = weak_barriers; + vq->notify = tp_cfg->notify; + vq->weak_barriers = tp_cfg->weak_barriers; #ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION vq->broken = true; #else vq->broken = false; #endif - vq->dma_dev = dma_dev; + vq->dma_dev = tp_cfg->dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); vq->premapped = false; vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && - !context; + !cfg_vq_get(cfg, ctx); vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX); if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM)) @@ -2731,38 +2719,10 @@ struct virtqueue *vring_create_virtqueue(struct virtio_device *vdev, struct transport_vq_config *tp_cfg, struct virtio_vq_config *cfg) { - struct device *dma_dev; - unsigned int num; - unsigned int vring_align; - bool weak_barriers; - bool may_reduce_num; - bool context; - bool (*notify)(struct virtqueue *_); - void (*callback)(struct virtqueue *_); - const char *name; - - dma_dev = tp_cfg->dma_dev; - if (!dma_dev) - dma_dev = vdev->dev.parent; - - num = tp_cfg->num; - vring_align = tp_cfg->vring_align; - weak_barriers = tp_cfg->weak_barriers; - may_reduce_num = tp_cfg->may_reduce_num; - notify = tp_cfg->notify; - - name = cfg->names[cfg->cfg_idx]; - callback = cfg->callbacks[cfg->cfg_idx]; - context = cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false; - if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) - return vring_create_virtqueue_packed(index, num, vring_align, - vdev, weak_barriers, may_reduce_num, - context, notify, callback, name, dma_dev); + return vring_create_virtqueue_packed(vdev, index, tp_cfg, cfg); - return vring_create_virtqueue_split(index, num, vring_align, - vdev, weak_barriers, may_reduce_num, - context, notify, callback, name, dma_dev); + return vring_create_virtqueue_split(vdev, index, tp_cfg, cfg); } EXPORT_SYMBOL_GPL(vring_create_virtqueue); @@ -2915,30 +2875,14 @@ struct virtqueue *vring_new_virtqueue(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct vring_virtqueue_split vring_split = {}; - unsigned int num; - unsigned int vring_align; - bool weak_barriers; - bool context; - bool (*notify)(struct virtqueue *_); - void (*callback)(struct virtqueue *_); - const char *name; - - num = tp_cfg->num; - vring_align = tp_cfg->vring_align; - weak_barriers = tp_cfg->weak_barriers; - notify = tp_cfg->notify; - - name = cfg->names[cfg->cfg_idx]; - callback = cfg->callbacks[cfg->cfg_idx]; - context = cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false; if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) return NULL; - vring_init(&vring_split.vring, num, pages, vring_align); - return __vring_new_virtqueue(index, &vring_split, vdev, weak_barriers, - context, notify, callback, name, - vdev->dev.parent); + tp_cfg->dma_dev = vdev->dev.parent; + + vring_init(&vring_split.vring, tp_cfg->num, pages, tp_cfg->vring_align); + return __vring_new_virtqueue(vdev, index, &vring_split, tp_cfg, cfg); } EXPORT_SYMBOL_GPL(vring_new_virtqueue); From patchwork Tue Jan 30 11:42:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537421 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0EDBC67E8D; Tue, 30 Jan 2024 11:42:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614974; cv=none; b=NilF4segIKstwsH+Ed7rROyx+sYtGoCwfLSuZMdN4OaFtbEeyldpxqwdlp2R71b1R9uz9DhcjSD4c03RiSRuIUjl/BZT2Vbslzl1lkcpCNMqC6lPHVKWeE/fDmuxv4tH5b87ht3ZpB7w0AFIj9A2RGQoev/uyic3TKNnVlCFCQ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614974; c=relaxed/simple; bh=oC0A2iqTgzltxNX7HZCNhxU/K71nLhzjfobA+3jDzcI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mgLfMI27fkv7/LpXxEwNc1km7kAHBZDzTlhlR70CFT3ywnWPbbHzudrPNThypOa0who6ygOVd/TbPlky2QWZAAqq1W3PyK4Ronj8ip03umkdn6I5/DzdMa0ylQ+0p+NIbw1Xja/ascuNGt+fj1JOonQHusxcdtwYmUfb732BS8s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=c/NwOmRY; arc=none smtp.client-ip=115.124.30.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="c/NwOmRY" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614964; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=Dgn6m2tLOO6WFRL4YWGeqj72OqIhjZpHBxeWCefu0YE=; b=c/NwOmRYcGH7zZNPG26b2iPGT+oo9sDt7Wn2ZGnjK6cvIMUI9CF4kZu4LIP2bNVrCYzUv9s2DGv3QAVYTpyUa8ppiiT5Ihm2BGIxrjfwvkP5Zu/yUAHHoqzZIkN2YHq/RyrIKUrHbLgTUio1ro/dxNTcQss2jS2IQC2k+cFzzo0= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g78-7_1706614961; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g78-7_1706614961) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:42 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 10/17] virtio: find_vqs: add new parameter premapped Date: Tue, 30 Jan 2024 19:42:17 +0800 Message-Id: <20240130114224.86536-11-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 If the premapped mode is enabled, the dma array(struct vring_desc_dma) of virtio core will not be allocated. That is judged when find_vqs() is called. To avoid allocating dma array in find_vqs() and releasing it immediately by virtqueue_set_dma_premapped(). This patch introduces a new parameter to find_vqs(). Then we can judge should we allocate the dma array(struct vring_desc_dma) or not inside find_vqs(). The driver must check the premapped mode of every vq after find_vqs(). Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 4 ++-- include/linux/virtio_config.h | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 769ec6360d7f..e5e9672b4ab2 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2145,7 +2145,7 @@ static struct virtqueue *vring_create_virtqueue_packed(struct virtio_device *vde vq->packed_ring = true; vq->dma_dev = dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); - vq->premapped = false; + vq->premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !cfg_vq_get(cfg, ctx); @@ -2687,7 +2687,7 @@ static struct virtqueue *__vring_new_virtqueue(struct virtio_device *vdev, #endif vq->dma_dev = tp_cfg->dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); - vq->premapped = false; + vq->premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !cfg_vq_get(cfg, ctx); diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h index e2c72e125dae..f462c3f1691c 100644 --- a/include/linux/virtio_config.h +++ b/include/linux/virtio_config.h @@ -105,6 +105,7 @@ struct virtio_vq_config { vq_callback_t **callbacks; const char *const *names; const bool *ctx; + const bool *premapped; struct irq_affinity *desc; }; From patchwork Tue Jan 30 11:42:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537416 Received: from out30-113.freemail.mail.aliyun.com (out30-113.freemail.mail.aliyun.com [115.124.30.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99EC67762C; Tue, 30 Jan 2024 11:42:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.113 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614970; cv=none; b=AiZatHD6s0xQ3PGW8B/U9nD2g63fwvJBKWtLMurJJuSsvyHRF5vX3Acy3mvuurtjLNcKNImCDaOykJjFV+vmS1x0xO5iadixx3rKAAOteI+VdWTp4Y5S9L4hOuH7V2a7UYRBpUeGftwHOqdA6IxcCC29tbvmsvWuHqJSn3CIsMY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614970; c=relaxed/simple; bh=SxgS51tymsPGy/2QSld4zjMIJ3zmmbLvOnx6TVmlyAE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=njMfmllRHf8nD1I4a6yQCFnEGDKg+p03lmLE5HsO9Q/RU9BsWtqrKt5pOBBcz8U1YEBXtcNfufaTUMZyp2Tpg0H/J9G7dTzBkSY5w4/AzbfAUzFKpW3IYVY4EBHOonx/f2zAidq7hq/81b6d2qH2+Y8lst8rruSjiBt4lOwd45Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=DrButARU; arc=none smtp.client-ip=115.124.30.113 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="DrButARU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614966; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=IhSe/hckx6ypBcdE4k41LnQFuwciEnm8GoFA8OoZo2s=; b=DrButARUz/+I3T9d6zNlwa+hUKx1a1JltfDt48+/6UG4gpT1CYK4+4VION876n+nMapFMpy9eDgrqBN8LOHHHr3aKOgpyUW4l/ADAtuWYNXZjntWzmchm+InJZfyC0+dJQrYn+xAz8DF1eJTnSB2l3JgLzW9sumMhBF9ZpMKfPQ= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g61W-_1706614962; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g61W-_1706614962) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:43 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 11/17] virtio_ring: export premapped to driver by struct virtqueue Date: Tue, 30 Jan 2024 19:42:18 +0800 Message-Id: <20240130114224.86536-12-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 Export the premapped to drivers, then drivers can check the vq premapped mode after the find_vqs(). Because the find_vqs() just try to enable the vq premapped mode, the driver must check that after find_vqs(). Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 13 +++++-------- include/linux/virtio.h | 1 + 2 files changed, 6 insertions(+), 8 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index e5e9672b4ab2..e7badc81642d 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -191,9 +191,6 @@ struct vring_virtqueue { /* Host publishes avail event idx */ bool event; - /* Do DMA mapping by driver */ - bool premapped; - /* Head of free buffer list. */ unsigned int free_head; /* Number we've added since last sync. */ @@ -311,7 +308,7 @@ static bool vring_use_dma_api(const struct virtio_device *vdev) static bool vring_need_unmap_buffer(const struct vring_virtqueue *vring) { - return vring->use_dma_api && !vring->premapped; + return vring->use_dma_api && !vring->vq.premapped; } size_t virtio_max_dma_size(const struct virtio_device *vdev) @@ -383,7 +380,7 @@ static struct device *vring_dma_dev(const struct vring_virtqueue *vq) static int vring_map_one_sg(const struct vring_virtqueue *vq, struct scatterlist *sg, enum dma_data_direction direction, dma_addr_t *addr) { - if (vq->premapped) { + if (vq->vq.premapped) { *addr = sg_dma_address(sg); return 0; } @@ -2145,7 +2142,7 @@ static struct virtqueue *vring_create_virtqueue_packed(struct virtio_device *vde vq->packed_ring = true; vq->dma_dev = dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); - vq->premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); + vq->vq.premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !cfg_vq_get(cfg, ctx); @@ -2687,7 +2684,7 @@ static struct virtqueue *__vring_new_virtqueue(struct virtio_device *vdev, #endif vq->dma_dev = tp_cfg->dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); - vq->premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); + vq->vq.premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !cfg_vq_get(cfg, ctx); @@ -2818,7 +2815,7 @@ int virtqueue_set_dma_premapped(struct virtqueue *_vq) return -EINVAL; } - vq->premapped = true; + vq->vq.premapped = true; if (vq->packed_ring) { kfree(vq->packed.desc_dma); diff --git a/include/linux/virtio.h b/include/linux/virtio.h index 4cc614a38376..37ba44dc34de 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -36,6 +36,7 @@ struct virtqueue { unsigned int num_free; unsigned int num_max; bool reset; + bool premapped; void *priv; }; From patchwork Tue Jan 30 11:42:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537419 X-Patchwork-Delegate: kuba@kernel.org Received: from out30-113.freemail.mail.aliyun.com (out30-113.freemail.mail.aliyun.com [115.124.30.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A782C7A73C; Tue, 30 Jan 2024 11:42:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.113 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614973; cv=none; b=e9yCSSljgUaxHLNn8mKFILtGqfQu6b2Ac0KEFmIv6j443n2QYRVXH5Ufmhktf8yArWVP6n6Q+gmOn3AWfZSDsIVpVUna3qK0HzrVrqOnfw5F1Sxh+Qk0X4Gg9OjDY59jNhukRzB/K4oRgDEFb1oAfStnLdj/cqLc7suswe4vgHU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614973; c=relaxed/simple; bh=uLHUzpXsbL7X0Nr5V/0eLUfjAV6FgU7EbUcqlCj3jGo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=E/FSZrSTHYZLKUFuyDBvBt5ayXOvAx3Se7o9rTCz/07obEXRRHn89FcNU5l4/+AhvNbuWIstjrVwURkoQ36ygfJB5O0a7op4e0zTCNerY/EXCPTQG7Wweb4nx3p/+AvdTByecQmoZbqNbxbiVwtBxc9io1C/0V5QehTHZAc6sak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=t+sK5GNG; arc=none smtp.client-ip=115.124.30.113 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="t+sK5GNG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614969; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=zz6HlhZ2m5QpUdXygmyUdjzYx5sKxdD3g1gFCx/3l2I=; b=t+sK5GNGJqWiEhitRVEOhFJA1eSNJnjy5oa3T7mcJrZkkRJyspqB1hCTymi5QcWB5fHjJvEdAcB1g265gQVE1lsrfarVsl0Folzvj1lhoDAlcxkZ2ASoy6q1sGlpcCm4bjgI3vDrX3sRyGxvX/34nlVnOjwc+MXZ+noJCcBedGE= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g5gDu_1706614964; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g5gDu_1706614964) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:45 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 12/17] virtio_net: set premapped mode by find_vqs() Date: Tue, 30 Jan 2024 19:42:19 +0800 Message-Id: <20240130114224.86536-13-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 X-Patchwork-Delegate: kuba@kernel.org Now, the virtio core can set the premapped mode by find_vqs(). If the premapped can be enabled, the dma array will not be allocated. So virtio-net use the api of find_vqs to enable the premapped. Judge the premapped mode by the vq->premapped instead of saving local variable. Signed-off-by: Xuan Zhuo --- drivers/net/virtio_net.c | 57 +++++++++++++++++------------------ include/linux/virtio_config.h | 16 ++-------- 2 files changed, 29 insertions(+), 44 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 51b1868d2f22..12961fc0879e 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -196,9 +196,6 @@ struct receive_queue { /* Record the last dma info to free after new pages is allocated. */ struct virtnet_rq_dma *last_dma; - - /* Do dma by self */ - bool do_dma; }; /* This structure can contain rss message with maximum settings for indirection table and keysize @@ -638,7 +635,7 @@ static void *virtnet_rq_get_buf(struct receive_queue *rq, u32 *len, void **ctx) void *buf; buf = virtqueue_get_buf_ctx(rq->vq, len, ctx); - if (buf && rq->do_dma) + if (buf && rq->vq->premapped) virtnet_rq_unmap(rq, buf, *len); return buf; @@ -651,7 +648,7 @@ static void virtnet_rq_init_one_sg(struct receive_queue *rq, void *buf, u32 len) u32 offset; void *head; - if (!rq->do_dma) { + if (!rq->vq->premapped) { sg_init_one(rq->sg, buf, len); return; } @@ -681,7 +678,7 @@ static void *virtnet_rq_alloc(struct receive_queue *rq, u32 size, gfp_t gfp) head = page_address(alloc_frag->page); - if (rq->do_dma) { + if (rq->vq->premapped) { dma = head; /* new pages */ @@ -727,22 +724,6 @@ static void *virtnet_rq_alloc(struct receive_queue *rq, u32 size, gfp_t gfp) return buf; } -static void virtnet_rq_set_premapped(struct virtnet_info *vi) -{ - int i; - - /* disable for big mode */ - if (!vi->mergeable_rx_bufs && vi->big_packets) - return; - - for (i = 0; i < vi->max_queue_pairs; i++) { - if (virtqueue_set_dma_premapped(vi->rq[i].vq)) - continue; - - vi->rq[i].do_dma = true; - } -} - static void virtnet_rq_unmap_free_buf(struct virtqueue *vq, void *buf) { struct virtnet_info *vi = vq->vdev->priv; @@ -751,7 +732,7 @@ static void virtnet_rq_unmap_free_buf(struct virtqueue *vq, void *buf) rq = &vi->rq[i]; - if (rq->do_dma) + if (rq->vq->premapped) virtnet_rq_unmap(rq, buf, 0); virtnet_rq_free_buf(vi, rq, buf); @@ -1846,7 +1827,7 @@ static int add_recvbuf_small(struct virtnet_info *vi, struct receive_queue *rq, err = virtqueue_add_inbuf_ctx(rq->vq, rq->sg, 1, buf, ctx, gfp); if (err < 0) { - if (rq->do_dma) + if (rq->vq->premapped) virtnet_rq_unmap(rq, buf, 0); put_page(virt_to_head_page(buf)); } @@ -1961,7 +1942,7 @@ static int add_recvbuf_mergeable(struct virtnet_info *vi, ctx = mergeable_len_to_ctx(len + room, headroom); err = virtqueue_add_inbuf_ctx(rq->vq, rq->sg, 1, buf, ctx, gfp); if (err < 0) { - if (rq->do_dma) + if (rq->vq->premapped) virtnet_rq_unmap(rq, buf, 0); put_page(virt_to_head_page(buf)); } @@ -4030,7 +4011,7 @@ static void free_receive_page_frags(struct virtnet_info *vi) int i; for (i = 0; i < vi->max_queue_pairs; i++) if (vi->rq[i].alloc_frag.page) { - if (vi->rq[i].do_dma && vi->rq[i].last_dma) + if (vi->rq[i].vq->premapped && vi->rq[i].last_dma) virtnet_rq_unmap(&vi->rq[i], vi->rq[i].last_dma, 0); put_page(vi->rq[i].alloc_frag.page); } @@ -4094,11 +4075,13 @@ static unsigned int mergeable_min_buf_len(struct virtnet_info *vi, struct virtqu static int virtnet_find_vqs(struct virtnet_info *vi) { + struct virtio_vq_config cfg = {}; vq_callback_t **callbacks; struct virtqueue **vqs; int ret = -ENOMEM; int i, total_vqs; const char **names; + bool *premapped; bool *ctx; /* We expect 1 RX virtqueue followed by 1 TX virtqueue, followed by @@ -4122,8 +4105,13 @@ static int virtnet_find_vqs(struct virtnet_info *vi) ctx = kcalloc(total_vqs, sizeof(*ctx), GFP_KERNEL); if (!ctx) goto err_ctx; + + premapped = kcalloc(total_vqs, sizeof(*ctx), GFP_KERNEL); + if (!ctx) + goto err_premapped; } else { ctx = NULL; + premapped = NULL; } /* Parameters for control virtqueue, if any */ @@ -4142,10 +4130,19 @@ static int virtnet_find_vqs(struct virtnet_info *vi) names[txq2vq(i)] = vi->sq[i].name; if (ctx) ctx[rxq2vq(i)] = true; + + if (premapped) + premapped[rxq2vq(i)] = true; } - ret = virtio_find_vqs_ctx(vi->vdev, total_vqs, vqs, callbacks, - names, ctx, NULL); + cfg.nvqs = total_vqs; + cfg.vqs = vqs; + cfg.callbacks = callbacks; + cfg.names = names; + cfg.ctx = ctx; + cfg.premapped = premapped; + + ret = virtio_find_vqs_cfg(vi->vdev, &cfg); if (ret) goto err_find; @@ -4165,6 +4162,8 @@ static int virtnet_find_vqs(struct virtnet_info *vi) err_find: + kfree(premapped); +err_premapped: kfree(ctx); err_ctx: kfree(names); @@ -4234,8 +4233,6 @@ static int init_vqs(struct virtnet_info *vi) if (ret) goto err_free; - virtnet_rq_set_premapped(vi); - cpus_read_lock(); virtnet_set_affinity(vi); cpus_read_unlock(); diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h index f462c3f1691c..e8c4d2d7c827 100644 --- a/include/linux/virtio_config.h +++ b/include/linux/virtio_config.h @@ -256,21 +256,9 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs, } static inline -int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc) +int virtio_find_vqs_cfg(struct virtio_device *vdev, struct virtio_vq_config *cfg) { - struct virtio_vq_config cfg = {}; - - cfg.nvqs = nvqs; - cfg.vqs = vqs; - cfg.callbacks = callbacks; - cfg.names = names; - cfg.ctx = ctx; - cfg.desc = desc; - - return vdev->config->find_vqs(vdev, &cfg); + return vdev->config->find_vqs(vdev, cfg); } /** From patchwork Tue Jan 30 11:42:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537418 Received: from out30-113.freemail.mail.aliyun.com (out30-113.freemail.mail.aliyun.com [115.124.30.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C301F67E98; Tue, 30 Jan 2024 11:42:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.113 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614972; cv=none; b=aYVhhgh98RmVYPHyJwYnbUZLGwvleJmRq15rxE9YVp9q7+ED2bbYAzOO+Z1d2GscRpt7EWIdO4QzzU+I6gZrOQg8UUEXd0D2aKO0a2hF51RPTzZnQ/S4Ms8vomITtcfkFexOdVmz55hHooP4EhOpGDqXsUmSEf4CxvKIfhgZRnU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614972; c=relaxed/simple; bh=kzt88VdnpTofgP+iR2skJ+NHH7VpB8COj0SURHqNfgU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Hftp5XHC5+lfWII3XiZcHkVn7TYdBD6/1P1u9DlLobDejwHG+CcY9a0y7zLcPLwpIr40PyE7YhDMOCztZrSG5eKjd4S5J6F+LjOBbvdzVe09TPUFdiHJHO1N5sQG8tNjKHOtw0xVcUVh6vLWDyCkhPmQ3CcQer6bUHgnJPvkvJ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=CySrsmS8; arc=none smtp.client-ip=115.124.30.113 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="CySrsmS8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614968; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=r6iCdzsV8xEvLLqA0Ba/UisWAzFxmo/I9b/Z3w5ndek=; b=CySrsmS8ySuizfrzaMKq39iCPDyWaz9SP7DEhv817FXW6LmrCKzfSU7R1ZuYQ4b2Fzj9vjReNPzUaxh6xU3qOCmEwBl4bLwRTajbIyVFiacBWUpLyEAOqqX5B7PV12rtFO063emdHtH39F9X70wtoaIdS8VvV7Xc/jyTDjmFFDI= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g78.V_1706614966; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g78.V_1706614966) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:46 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 13/17] virtio_ring: remove api of setting vq premapped Date: Tue, 30 Jan 2024 19:42:20 +0800 Message-Id: <20240130114224.86536-14-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 The virtio-net sets the premapped mode by find_vqs(). So the below API is not used by anyone, we remove it. int virtqueue_set_dma_premapped(struct virtqueue *_vq); Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 56 ------------------------------------ include/linux/virtio.h | 2 -- 2 files changed, 58 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index e7badc81642d..ccdd6dd3062f 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2775,62 +2775,6 @@ int virtqueue_resize(struct virtqueue *_vq, u32 num, } EXPORT_SYMBOL_GPL(virtqueue_resize); -/** - * virtqueue_set_dma_premapped - set the vring premapped mode - * @_vq: the struct virtqueue we're talking about. - * - * Enable the premapped mode of the vq. - * - * The vring in premapped mode does not do dma internally, so the driver must - * do dma mapping in advance. The driver must pass the dma_address through - * dma_address of scatterlist. When the driver got a used buffer from - * the vring, it has to unmap the dma address. - * - * This function must be called immediately after creating the vq, or after vq - * reset, and before adding any buffers to it. - * - * Caller must ensure we don't call this with other virtqueue operations - * at the same time (except where noted). - * - * Returns zero or a negative error. - * 0: success. - * -EINVAL: vring does not use the dma api, so we can not enable premapped mode. - */ -int virtqueue_set_dma_premapped(struct virtqueue *_vq) -{ - struct vring_virtqueue *vq = to_vvq(_vq); - u32 num; - - START_USE(vq); - - num = vq->packed_ring ? vq->packed.vring.num : vq->split.vring.num; - - if (num != vq->vq.num_free) { - END_USE(vq); - return -EINVAL; - } - - if (!vq->use_dma_api) { - END_USE(vq); - return -EINVAL; - } - - vq->vq.premapped = true; - - if (vq->packed_ring) { - kfree(vq->packed.desc_dma); - vq->packed.desc_dma = NULL; - } else { - kfree(vq->split.desc_dma); - vq->split.desc_dma = NULL; - } - - END_USE(vq); - - return 0; -} -EXPORT_SYMBOL_GPL(virtqueue_set_dma_premapped); - /** * virtqueue_reset - detach and recycle all unused buffers * @_vq: the struct virtqueue we're talking about. diff --git a/include/linux/virtio.h b/include/linux/virtio.h index 37ba44dc34de..7f16bacc6c49 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -82,8 +82,6 @@ bool virtqueue_enable_cb(struct virtqueue *vq); unsigned virtqueue_enable_cb_prepare(struct virtqueue *vq); -int virtqueue_set_dma_premapped(struct virtqueue *_vq); - bool virtqueue_poll(struct virtqueue *vq, unsigned); bool virtqueue_enable_cb_delayed(struct virtqueue *vq); From patchwork Tue Jan 30 11:42:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537422 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D0E9E8002C; Tue, 30 Jan 2024 11:42:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.97 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614975; cv=none; b=RXb+GF+u1cl673KnIq1HZbVj84jKgHFHgWszyGRuat4MuZRQziLR5p194RDalqhhDjlia4EcKKyVL2gs2CmGtZoVw0DGU32edjJpPHvFhr1vc7JTslYn+rXJ+LfD4EZ/xaJKgFg19f92I7ZO92lgOImZ507pEtSan8UaRjHYUMw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614975; c=relaxed/simple; bh=xVm9n4On0W5ou3XqaPsnj91t2Cna1sE2h7qK6b1t8/g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=K7Z6Gez7irdkBJXpbRym1MmRcfQZO+nB7h3c7n+Y/MFGD8LDpOQ/kds8T8H/ae/pYtnWyc9BhKwrsQrtVZRimfYsuNLbme43H/19vabHfbkdLr9UDIDvwmKoXeYAT69XBrUkVKTnssYCWzMdowscl+sdVXIq0QyrT5L1HCFg9Mg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=C2d1vBUU; arc=none smtp.client-ip=115.124.30.97 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="C2d1vBUU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614971; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=P3G6i/Ax0Z+AAMFYFKvCu0HZ8Ra+Ilb8/1gq6QO23qs=; b=C2d1vBUUIkB/NJfwn4X4WAj5kHldFdI55budwY4ZCZEfV5CnHSA/seIX4F92TZquckt6SwLCRZEMfFnmha55WL4OkCKpBVlMs4E7ByE5vrNbK6wVapJlysY7KUVZFaxlo5aw5sYjQPFhrfaNx7fV55991MpRMKhvMtboI4aP4Kw= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046056;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g5gEc_1706614967; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g5gEc_1706614967) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:48 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 14/17] virtio_ring: introduce dma map api for page Date: Tue, 30 Jan 2024 19:42:21 +0800 Message-Id: <20240130114224.86536-15-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 The virtio-net sq will use these APIs to map the scatterlist. For scatterlist, the page dma APIs are more appropriate. dma_addr_t virtqueue_dma_map_page_attrs(struct virtqueue *_vq, struct page *page, size_t offset, size_t size, enum dma_data_direction dir, unsigned long attrs); void virtqueue_dma_unmap_page_attrs(struct virtqueue *_vq, dma_addr_t addr, size_t size, enum dma_data_direction dir, unsigned long attrs); Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 52 ++++++++++++++++++++++++++++++++++++ include/linux/virtio.h | 7 +++++ 2 files changed, 59 insertions(+) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index ccdd6dd3062f..47aa31ce69e7 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -3117,6 +3117,58 @@ void virtqueue_dma_unmap_single_attrs(struct virtqueue *_vq, dma_addr_t addr, } EXPORT_SYMBOL_GPL(virtqueue_dma_unmap_single_attrs); +/** + * virtqueue_dma_map_page_attrs - map DMA for _vq + * @_vq: the struct virtqueue we're talking about. + * @page: the page to do dma + * @offset: the offset inside the page + * @size: the size of the page to do dma + * @dir: DMA direction + * @attrs: DMA Attrs + * + * The caller calls this to do dma mapping in advance. The DMA address can be + * passed to this _vq when it is in pre-mapped mode. + * + * return DMA address. Caller should check that by virtqueue_dma_mapping_error(). + */ +dma_addr_t virtqueue_dma_map_page_attrs(struct virtqueue *_vq, struct page *page, + size_t offset, size_t size, + enum dma_data_direction dir, + unsigned long attrs) +{ + struct vring_virtqueue *vq = to_vvq(_vq); + + if (!vq->use_dma_api) + return page_to_phys(page) + offset; + + return dma_map_page_attrs(vring_dma_dev(vq), page, offset, size, dir, attrs); +} +EXPORT_SYMBOL_GPL(virtqueue_dma_map_page_attrs); + +/** + * virtqueue_dma_unmap_page_attrs - unmap DMA for _vq + * @_vq: the struct virtqueue we're talking about. + * @addr: the dma address to unmap + * @size: the size of the buffer + * @dir: DMA direction + * @attrs: DMA Attrs + * + * Unmap the address that is mapped by the virtqueue_dma_map_* APIs. + * + */ +void virtqueue_dma_unmap_page_attrs(struct virtqueue *_vq, dma_addr_t addr, + size_t size, enum dma_data_direction dir, + unsigned long attrs) +{ + struct vring_virtqueue *vq = to_vvq(_vq); + + if (!vq->use_dma_api) + return; + + dma_unmap_page_attrs(vring_dma_dev(vq), addr, size, dir, attrs); +} +EXPORT_SYMBOL_GPL(virtqueue_dma_unmap_page_attrs); + /** * virtqueue_dma_mapping_error - check dma address * @_vq: the struct virtqueue we're talking about. diff --git a/include/linux/virtio.h b/include/linux/virtio.h index 7f16bacc6c49..8a8d282982ad 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -216,6 +216,13 @@ dma_addr_t virtqueue_dma_map_single_attrs(struct virtqueue *_vq, void *ptr, size void virtqueue_dma_unmap_single_attrs(struct virtqueue *_vq, dma_addr_t addr, size_t size, enum dma_data_direction dir, unsigned long attrs); +dma_addr_t virtqueue_dma_map_page_attrs(struct virtqueue *_vq, struct page *page, + size_t offset, size_t size, + enum dma_data_direction dir, + unsigned long attrs); +void virtqueue_dma_unmap_page_attrs(struct virtqueue *_vq, dma_addr_t addr, + size_t size, enum dma_data_direction dir, + unsigned long attrs); int virtqueue_dma_mapping_error(struct virtqueue *_vq, dma_addr_t addr); bool virtqueue_dma_need_sync(struct virtqueue *_vq, dma_addr_t addr); From patchwork Tue Jan 30 11:42:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537423 X-Patchwork-Delegate: kuba@kernel.org Received: from out30-98.freemail.mail.aliyun.com (out30-98.freemail.mail.aliyun.com [115.124.30.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF9C6823CF; Tue, 30 Jan 2024 11:42:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.98 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614976; cv=none; b=S1OC2cTz9H6jiHLv/8CNi71PkN4NcfLSw3iaQ3kfn1FVJFf1+AUo71OISbwRei83Ei2PiBke9tdRPFLdwMZ4tGyqNuDkt3X9ac9Je/TeMvYb3+PZgMsUrO3VlqU64rBiGI3OwZ6SyyZCDIAgmWzsoyCK+Q4FUCBJYDbzgD+PeUg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614976; c=relaxed/simple; bh=P1EuAKIYKR8oUbIq4ooX+jiHttt0zxADPIYwqeGcDq8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eXY9sGCNnWASHzAdbdwUz1Mh6NXYMTgrLOFIKKRDNpRk6KV+SLklmu1q4D3+IKsYZDPCf6bzl8dwCpPN55zesxQcGOLz2l6DtTJRb3tAizLnLMXQA7OcgJy/cLFN8MMcMl/XMeXUXXyMxcdzHnojU/u0POUY/te7C94kIVix50w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=UxV7wlQg; arc=none smtp.client-ip=115.124.30.98 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="UxV7wlQg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614972; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=MJPPX0Mbe/Fi3ODIrg/IdS8p5bwqdjZiSVg8RtlcgFY=; b=UxV7wlQgNIh+vm/tQQlwScYA1m9mJII++x6WmO33NicPtK//uxXBkiwjdytqUkx7wr6ltewe/+fQFtdFdiWdOcZTp4PwTN4W69FO4lgznm8tOIB54Ld7zM+foOKN4d/YAviGyAdvwEzbf88Yn4LuW4ttoJgcLa6zM28WoRDVNSg= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g5I9y_1706614969; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g5I9y_1706614969) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:50 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 15/17] virtio_net: unify the code for recycling the xmit ptr Date: Tue, 30 Jan 2024 19:42:22 +0800 Message-Id: <20240130114224.86536-16-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 X-Patchwork-Delegate: kuba@kernel.org There are two completely similar and independent implementations. This is inconvenient for the subsequent addition of new types. So extract a function from this piece of code and call this function uniformly to recover old xmit ptr. Signed-off-by: Xuan Zhuo Acked-by: Jason Wang --- drivers/net/virtio_net.c | 66 +++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 38 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 12961fc0879e..c7409344dda1 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -348,6 +348,30 @@ static struct xdp_frame *ptr_to_xdp(void *ptr) return (struct xdp_frame *)((unsigned long)ptr & ~VIRTIO_XDP_FLAG); } +static void __free_old_xmit(struct send_queue *sq, bool in_napi, + u64 *bytes, u64 *packets) +{ + unsigned int len; + void *ptr; + + while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { + if (!is_xdp_frame(ptr)) { + struct sk_buff *skb = ptr; + + pr_debug("Sent skb %p\n", skb); + + *bytes += skb->len; + napi_consume_skb(skb, in_napi); + } else { + struct xdp_frame *frame = ptr_to_xdp(ptr); + + *bytes += xdp_get_frame_len(frame); + xdp_return_frame(frame); + } + (*packets)++; + } +} + /* Converting between virtqueue no. and kernel tx/rx queue no. * 0:rx0 1:tx0 2:rx1 3:tx1 ... 2N:rxN 2N+1:txN 2N+2:cvq */ @@ -740,27 +764,9 @@ static void virtnet_rq_unmap_free_buf(struct virtqueue *vq, void *buf) static void free_old_xmit_skbs(struct send_queue *sq, bool in_napi) { - unsigned int len; - unsigned int packets = 0; - unsigned int bytes = 0; - void *ptr; - - while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { - if (likely(!is_xdp_frame(ptr))) { - struct sk_buff *skb = ptr; - - pr_debug("Sent skb %p\n", skb); + u64 bytes = 0, packets = 0; - bytes += skb->len; - napi_consume_skb(skb, in_napi); - } else { - struct xdp_frame *frame = ptr_to_xdp(ptr); - - bytes += xdp_get_frame_len(frame); - xdp_return_frame(frame); - } - packets++; - } + __free_old_xmit(sq, in_napi, &bytes, &packets); /* Avoid overhead when no packets have been processed * happens when called speculatively from start_xmit. @@ -910,14 +916,11 @@ static int virtnet_xdp_xmit(struct net_device *dev, { struct virtnet_info *vi = netdev_priv(dev); struct receive_queue *rq = vi->rq; + u64 bytes = 0, packets = 0; struct bpf_prog *xdp_prog; struct send_queue *sq; - unsigned int len; - int packets = 0; - int bytes = 0; int nxmit = 0; int kicks = 0; - void *ptr; int ret; int i; @@ -936,20 +939,7 @@ static int virtnet_xdp_xmit(struct net_device *dev, } /* Free up any pending old buffers before queueing new ones. */ - while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { - if (likely(is_xdp_frame(ptr))) { - struct xdp_frame *frame = ptr_to_xdp(ptr); - - bytes += xdp_get_frame_len(frame); - xdp_return_frame(frame); - } else { - struct sk_buff *skb = ptr; - - bytes += skb->len; - napi_consume_skb(skb, false); - } - packets++; - } + __free_old_xmit(sq, false, &bytes, &packets); for (i = 0; i < n; i++) { struct xdp_frame *xdpf = frames[i]; From patchwork Tue Jan 30 11:42:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537424 X-Patchwork-Delegate: kuba@kernel.org Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com [115.124.30.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FD336A02B; Tue, 30 Jan 2024 11:42:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614978; cv=none; b=YIe5/JUjoL+A4GT6ahfEj7MBmBjqdr5fvop9NBk8y1NkGiGfY4opUvZo2yh29jx2Y2W+uPdRDitC1iQ69hpVDmIzWLtputHQmNA1wAanSi5kNsf+y8QBkXvYt5oodExuIto/atgievNdInQVe3sszICDYfeHNNshJ3WNwbFtIv0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614978; c=relaxed/simple; bh=opE8fb/c6GatOjc68YoX3PoKPWv8ZmAFDRId6/naxUo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tKMvrkeSi+xwU3QGKOXoGc7ijaLrwMnlEjqCdsDmAXvpvp8MWzriuHQZD/lpJ2PZRKy4d2YTYXDQLlk9ZpEKrUmk/txLguyQF2Npk2XCEQM3A0R8PbWEVlFV0d1N/LBA50+Pw4UYDHGYPiZRNT1Wo7/h5epSA/KA2mC2FFCCMX0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=ecgfgoA6; arc=none smtp.client-ip=115.124.30.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="ecgfgoA6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614973; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=KKWDp4LnQBc4ECc6orQM1FFYA9jb53yriKSRR0kD+HE=; b=ecgfgoA6vq/aAlWyzYhJ04LWkVJ7Aowfrw9IoJm1ikb9W+tO10Llxx/CjeG8dALCUjDdv/T2SRKTVmRmMVmNq/90GU6IHxej3m6NmteTsjgMysPBMmM+RGkCF1oAuRDu2NsnWJfjAA19TTIbVEY0+vEO45QrjaXkkXhmm8nwY+s= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g61YO_1706614970; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g61YO_1706614970) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:51 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 16/17] virtio_net: rename free_old_xmit_skbs to free_old_xmit Date: Tue, 30 Jan 2024 19:42:23 +0800 Message-Id: <20240130114224.86536-17-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 X-Patchwork-Delegate: kuba@kernel.org Since free_old_xmit_skbs not only deals with skb, but also xdp frame and subsequent added xsk, so change the name of this function to free_old_xmit. Signed-off-by: Xuan Zhuo Acked-by: Jason Wang --- drivers/net/virtio_net.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index c7409344dda1..226ab830870e 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -762,7 +762,7 @@ static void virtnet_rq_unmap_free_buf(struct virtqueue *vq, void *buf) virtnet_rq_free_buf(vi, rq, buf); } -static void free_old_xmit_skbs(struct send_queue *sq, bool in_napi) +static void free_old_xmit(struct send_queue *sq, bool in_napi) { u64 bytes = 0, packets = 0; @@ -816,7 +816,7 @@ static void check_sq_full_and_disable(struct virtnet_info *vi, virtqueue_napi_schedule(&sq->napi, sq->vq); } else if (unlikely(!virtqueue_enable_cb_delayed(sq->vq))) { /* More just got used, free them then recheck. */ - free_old_xmit_skbs(sq, false); + free_old_xmit(sq, false); if (sq->vq->num_free >= 2+MAX_SKB_FRAGS) { netif_start_subqueue(dev, qnum); virtqueue_disable_cb(sq->vq); @@ -2111,7 +2111,7 @@ static void virtnet_poll_cleantx(struct receive_queue *rq) do { virtqueue_disable_cb(sq->vq); - free_old_xmit_skbs(sq, true); + free_old_xmit(sq, true); } while (unlikely(!virtqueue_enable_cb_delayed(sq->vq))); if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) @@ -2233,7 +2233,7 @@ static int virtnet_poll_tx(struct napi_struct *napi, int budget) txq = netdev_get_tx_queue(vi->dev, index); __netif_tx_lock(txq, raw_smp_processor_id()); virtqueue_disable_cb(sq->vq); - free_old_xmit_skbs(sq, true); + free_old_xmit(sq, true); if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) netif_tx_wake_queue(txq); @@ -2323,7 +2323,7 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev) if (use_napi) virtqueue_disable_cb(sq->vq); - free_old_xmit_skbs(sq, false); + free_old_xmit(sq, false); } while (use_napi && kick && unlikely(!virtqueue_enable_cb_delayed(sq->vq))); From patchwork Tue Jan 30 11:42:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13537425 X-Patchwork-Delegate: kuba@kernel.org Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8E0E1272B4; Tue, 30 Jan 2024 11:42:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614979; cv=none; b=HTUxZWweUqkd5KoCniZBcGOwOz5ZBmY5fIujc8oV1KAGFLN71w3jr/0Jm3tkuU2FmavyWvVYidMsVhpM5fXWcxbvM9hGxWWbIXla8PMzaAbHEjuUym5G2mse7cEJbKzdgTNte49o3Pb78M5gkx6w/w6GJYSUutvAzpVfS/VTads= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706614979; c=relaxed/simple; bh=wzHVgK9RuDds3X3HVc5b7y3mnb3EC3On3hWf1G0sWm4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VZ6Hr9gLxBr1NQM3zmspUB3HfpyB/sDoM+ok13lTomykKxHLqDuEMjzz558crzv+cKGhjutdXH4hpQWTlObJpe2kIJgZM8vpzJApyrCWAC7gHTqyVhf0nQgq+fnC11R1e0cQYJA3RRU95Ak09THV8b6o7GYLJ7dhp9Pf5YHvYRA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=yDrMV3st; arc=none smtp.client-ip=115.124.30.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="yDrMV3st" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1706614975; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=rzoNTb2RQ+EX1nOkOX8zMleimrxnQrRGUEJK5nSFc0M=; b=yDrMV3stj5bipxbIywKPk/7tu8dOCuiipI2ZfeRGyyBiIT9NT1l1TeqnZ10Qxw7tTXKmoXg9g1v15/Pv1Vvwv550bO49mORNkstHTPIpoqU+B79qTYia5OernKT6weW0KfcTZwJ2A+gXw/oaSkzjDpqjYC0BvB6vClQOSyXWLo8= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=37;SR=0;TI=SMTPD_---0W.g5gGI_1706614972; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W.g5gGI_1706614972) by smtp.aliyun-inc.com; Tue, 30 Jan 2024 19:42:53 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Benjamin Berg , Yang Li , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost 17/17] virtio_net: sq support premapped mode Date: Tue, 30 Jan 2024 19:42:24 +0800 Message-Id: <20240130114224.86536-18-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> References: <20240130114224.86536-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 239d1d475be4 X-Patchwork-Delegate: kuba@kernel.org If the xsk is enabling, the xsk tx will share the send queue. But the xsk requires that the send queue use the premapped mode. So the send queue must support premapped mode. Signed-off-by: Xuan Zhuo --- drivers/net/virtio_net.c | 167 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 163 insertions(+), 4 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 226ab830870e..cf0c67380b07 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -46,6 +46,7 @@ module_param(napi_tx, bool, 0644); #define VIRTIO_XDP_REDIR BIT(1) #define VIRTIO_XDP_FLAG BIT(0) +#define VIRTIO_DMA_FLAG BIT(1) /* RX packet size EWMA. The average packet size is used to determine the packet * buffer size when refilling RX rings. As the entire RX ring may be refilled @@ -140,6 +141,21 @@ struct virtnet_rq_dma { u16 need_sync; }; +struct virtnet_sq_dma { + union { + struct virtnet_sq_dma *next; + void *data; + }; + dma_addr_t addr; + u32 len; + bool is_tail; +}; + +struct virtnet_sq_dma_head { + struct virtnet_sq_dma *free; + struct virtnet_sq_dma *head; +}; + /* Internal representation of a send virtqueue */ struct send_queue { /* Virtqueue associated with this send _queue */ @@ -159,6 +175,8 @@ struct send_queue { /* Record whether sq is in reset state. */ bool reset; + + struct virtnet_sq_dma_head dmainfo; }; /* Internal representation of a receive virtqueue */ @@ -348,6 +366,131 @@ static struct xdp_frame *ptr_to_xdp(void *ptr) return (struct xdp_frame *)((unsigned long)ptr & ~VIRTIO_XDP_FLAG); } +static inline void *virtnet_sq_unmap(struct send_queue *sq, void *data) +{ + struct virtnet_sq_dma *head, *tail; + + if (!((unsigned long)data & VIRTIO_DMA_FLAG)) + return data; + + head = (void *)((unsigned long)data & ~VIRTIO_DMA_FLAG); + + tail = head; + + while (true) { + virtqueue_dma_unmap_page_attrs(sq->vq, tail->addr, tail->len, + DMA_TO_DEVICE, 0); + + if (tail->is_tail) + break; + + tail = tail->next; + } + + data = tail->data; + tail->is_tail = false; + + tail->next = sq->dmainfo.free; + sq->dmainfo.free = head; + + return data; +} + +static void *virtnet_sq_dma_splice(struct send_queue *sq, + struct virtnet_sq_dma *head, + struct virtnet_sq_dma *tail, + void *data) +{ + sq->dmainfo.free = tail->next; + + tail->is_tail = true; + tail->data = data; + + head = (void *)((unsigned long)head | VIRTIO_DMA_FLAG); + + return head; +} + +static struct virtnet_sq_dma *virtnet_sq_map_sg(struct send_queue *sq, int nents, void *data) +{ + struct virtnet_sq_dma *head, *tail, *p; + struct scatterlist *sg; + dma_addr_t addr; + int i; + + head = sq->dmainfo.free; + p = head; + + tail = NULL; + + for_each_sg(sq->sg, sg, nents, i) { + addr = virtqueue_dma_map_page_attrs(sq->vq, sg_page(sg), + sg->offset, sg->length, + DMA_TO_DEVICE, 0); + if (virtqueue_dma_mapping_error(sq->vq, addr)) + goto err; + + sg->dma_address = addr; + + tail = p; + tail->addr = sg->dma_address; + tail->len = sg->length; + + p = p->next; + } + + return virtnet_sq_dma_splice(sq, head, tail, data); + +err: + if (tail) + virtnet_sq_unmap(sq, virtnet_sq_dma_splice(sq, head, tail, data)); + + return NULL; +} + +static int virtnet_add_outbuf(struct send_queue *sq, u32 num, void *data) +{ + int ret; + + if (sq->vq->premapped) { + data = virtnet_sq_map_sg(sq, num, data); + if (!data) + return -ENOMEM; + } + + ret = virtqueue_add_outbuf(sq->vq, sq->sg, num, data, GFP_ATOMIC); + if (ret && sq->vq->premapped) + virtnet_sq_unmap(sq, data); + + return ret; +} + +static int virtnet_sq_init_dma_mate(struct send_queue *sq) +{ + struct virtnet_sq_dma *d; + int size, i; + + size = virtqueue_get_vring_size(sq->vq); + + size += MAX_SKB_FRAGS + 2; + + sq->dmainfo.head = kcalloc(size, sizeof(*sq->dmainfo.head), GFP_KERNEL); + if (!sq->dmainfo.head) + return -ENOMEM; + + sq->dmainfo.free = sq->dmainfo.head; + + for (i = 0; i < size; ++i) { + d = &sq->dmainfo.head[i]; + d->is_tail = false; + d->next = d + 1; + } + + d->next = NULL; + + return 0; +} + static void __free_old_xmit(struct send_queue *sq, bool in_napi, u64 *bytes, u64 *packets) { @@ -355,6 +498,8 @@ static void __free_old_xmit(struct send_queue *sq, bool in_napi, void *ptr; while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { + ptr = virtnet_sq_unmap(sq, ptr); + if (!is_xdp_frame(ptr)) { struct sk_buff *skb = ptr; @@ -865,8 +1010,7 @@ static int __virtnet_xdp_xmit_one(struct virtnet_info *vi, skb_frag_size(frag), skb_frag_off(frag)); } - err = virtqueue_add_outbuf(sq->vq, sq->sg, nr_frags + 1, - xdp_to_ptr(xdpf), GFP_ATOMIC); + err = virtnet_add_outbuf(sq, nr_frags + 1, xdp_to_ptr(xdpf)); if (unlikely(err)) return -ENOSPC; /* Caller handle free/refcnt */ @@ -2305,7 +2449,7 @@ static int xmit_skb(struct send_queue *sq, struct sk_buff *skb) return num_sg; num_sg++; } - return virtqueue_add_outbuf(sq->vq, sq->sg, num_sg, skb, GFP_ATOMIC); + return virtnet_add_outbuf(sq, num_sg, skb); } static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev) @@ -3961,6 +4105,8 @@ static void virtnet_free_queues(struct virtnet_info *vi) for (i = 0; i < vi->max_queue_pairs; i++) { __netif_napi_del(&vi->rq[i].napi); __netif_napi_del(&vi->sq[i].napi); + + kfree(vi->sq[i].dmainfo.head); } /* We called __netif_napi_del(), @@ -4009,6 +4155,14 @@ static void free_receive_page_frags(struct virtnet_info *vi) static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf) { + struct virtnet_info *vi = vq->vdev->priv; + struct send_queue *sq; + int i = vq2rxq(vq); + + sq = &vi->sq[i]; + + buf = virtnet_sq_unmap(sq, buf); + if (!is_xdp_frame(buf)) dev_kfree_skb(buf); else @@ -4121,8 +4275,10 @@ static int virtnet_find_vqs(struct virtnet_info *vi) if (ctx) ctx[rxq2vq(i)] = true; - if (premapped) + if (premapped) { premapped[rxq2vq(i)] = true; + premapped[txq2vq(i)] = true; + } } cfg.nvqs = total_vqs; @@ -4146,6 +4302,9 @@ static int virtnet_find_vqs(struct virtnet_info *vi) vi->rq[i].vq = vqs[rxq2vq(i)]; vi->rq[i].min_buf_len = mergeable_min_buf_len(vi, vi->rq[i].vq); vi->sq[i].vq = vqs[txq2vq(i)]; + + if (vi->sq[i].vq->premapped) + virtnet_sq_init_dma_mate(&vi->sq[i]); } /* run here: ret == 0. */