| Message ID | 20190717113030.163499-6-sgarzare@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | vsock/virtio: optimizations to increase the throughput | expand |
On Wed, Jul 17, 2019 at 01:30:30PM +0200, Stefano Garzarella wrote: > Since now we are able to split packets, we can avoid limiting > their sizes to VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE. > Instead, we can use VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max > packet size. > > Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> OK so this is kind of like GSO where we are passing 64K packets to the vsock and then split at the low level. > --- > net/vmw_vsock/virtio_transport_common.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c > index 56fab3f03d0e..94cc0fa3e848 100644 > --- a/net/vmw_vsock/virtio_transport_common.c > +++ b/net/vmw_vsock/virtio_transport_common.c > @@ -181,8 +181,8 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, > vvs = vsk->trans; > > /* we can send less than pkt_len bytes */ > - if (pkt_len > VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE) > - pkt_len = VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE; > + if (pkt_len > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE) > + pkt_len = VIRTIO_VSOCK_MAX_PKT_BUF_SIZE; > > /* virtio_transport_get_credit might return less than pkt_len credit */ > pkt_len = virtio_transport_get_credit(vvs, pkt_len); > -- > 2.20.1
On Wed, Jul 17, 2019 at 5:00 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Wed, Jul 17, 2019 at 01:30:30PM +0200, Stefano Garzarella wrote: > > Since now we are able to split packets, we can avoid limiting > > their sizes to VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE. > > Instead, we can use VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max > > packet size. > > > > Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> > > > OK so this is kind of like GSO where we are passing > 64K packets to the vsock and then split at the > low level. Exactly, something like that in the Host->Guest path, instead in the Guest->Host we use the entire 64K packet. Thanks, Stefano
On Thu, Jul 18, 2019 at 09:52:41AM +0200, Stefano Garzarella wrote: > On Wed, Jul 17, 2019 at 5:00 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Wed, Jul 17, 2019 at 01:30:30PM +0200, Stefano Garzarella wrote: > > > Since now we are able to split packets, we can avoid limiting > > > their sizes to VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE. > > > Instead, we can use VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max > > > packet size. > > > > > > Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> > > > > > > OK so this is kind of like GSO where we are passing > > 64K packets to the vsock and then split at the > > low level. > > Exactly, something like that in the Host->Guest path, instead in the > Guest->Host we use the entire 64K packet. > > Thanks, > Stefano btw two allocations for each packet isn't great. How about allocating the struct linearly with the data? And all buffers are same length for you - so you can actually do alloc_pages. Allocating/freeing pages in a batch should also be considered.
On Thu, Jul 18, 2019 at 08:33:40AM -0400, Michael S. Tsirkin wrote: > On Thu, Jul 18, 2019 at 09:52:41AM +0200, Stefano Garzarella wrote: > > On Wed, Jul 17, 2019 at 5:00 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Wed, Jul 17, 2019 at 01:30:30PM +0200, Stefano Garzarella wrote: > > > > Since now we are able to split packets, we can avoid limiting > > > > their sizes to VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE. > > > > Instead, we can use VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max > > > > packet size. > > > > > > > > Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> > > > > > > > > > OK so this is kind of like GSO where we are passing > > > 64K packets to the vsock and then split at the > > > low level. > > > > Exactly, something like that in the Host->Guest path, instead in the > > Guest->Host we use the entire 64K packet. > > > > Thanks, > > Stefano > > btw two allocations for each packet isn't great. How about > allocating the struct linearly with the data? Are you referring to the kzalloc() to allocate the 'struct virtio_vsock_pkt', followed by the kmalloc() to allocate the buffer? Actually they don't look great, I will try to do a single allocation. > And all buffers are same length for you - so you can actually > do alloc_pages. Yes, also Jason suggested it and we decided to postpone since we will try to reuse the virtio-net where it comes for free. > Allocating/freeing pages in a batch should also be considered. For the allocation of guest rx buffers we do some kind of batching (we refill the queue when it reaches the half), but only it this case :( I'll try to do more alloc/free batching. Thanks, Stefano
On Wed, Jul 17, 2019 at 01:30:30PM +0200, Stefano Garzarella wrote: > Since now we are able to split packets, we can avoid limiting > their sizes to VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE. > Instead, we can use VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max > packet size. > > Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> > --- > net/vmw_vsock/virtio_transport_common.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index 56fab3f03d0e..94cc0fa3e848 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -181,8 +181,8 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, vvs = vsk->trans; /* we can send less than pkt_len bytes */ - if (pkt_len > VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE) - pkt_len = VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE; + if (pkt_len > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE) + pkt_len = VIRTIO_VSOCK_MAX_PKT_BUF_SIZE; /* virtio_transport_get_credit might return less than pkt_len credit */ pkt_len = virtio_transport_get_credit(vvs, pkt_len);
Since now we are able to split packets, we can avoid limiting their sizes to VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE. Instead, we can use VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max packet size. Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> --- net/vmw_vsock/virtio_transport_common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)