| Message ID | 20240506150428.1203387-4-jonah.palmer@oracle.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | virtio,vhost: Add VIRTIO_F_IN_ORDER support | expand |
On Mon, May 6, 2024 at 5:05 PM Jonah Palmer <jonah.palmer@oracle.com> wrote: > > Add VIRTIO_F_IN_ORDER feature support for virtqueue_fill operations. > > The goal of the virtqueue_fill operation when the VIRTIO_F_IN_ORDER > feature has been negotiated is to search for this now-used element, > set its length, and mark the element as filled in the VirtQueue's > used_elems array. > > By marking the element as filled, it will indicate that this element is > ready to be flushed, so long as the element is in-order. > > Tested-by: Lei Yang <leiyang@redhat.com> > Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com> > --- > hw/virtio/virtio.c | 26 +++++++++++++++++++++++++- > 1 file changed, 25 insertions(+), 1 deletion(-) > > diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c > index e6eb1bb453..064046b5e2 100644 > --- a/hw/virtio/virtio.c > +++ b/hw/virtio/virtio.c > @@ -873,6 +873,28 @@ static void virtqueue_packed_fill(VirtQueue *vq, const VirtQueueElement *elem, > vq->used_elems[idx].ndescs = elem->ndescs; > } > > +static void virtqueue_ordered_fill(VirtQueue *vq, const VirtQueueElement *elem, > + unsigned int len) > +{ > + unsigned int i = vq->used_idx; > + > + /* Search for element in vq->used_elems */ > + while (i != vq->last_avail_idx) { > + /* Found element, set length and mark as filled */ > + if (vq->used_elems[i].index == elem->index) { > + vq->used_elems[i].len = len; > + vq->used_elems[i].filled = true; > + break; > + } > + > + i += vq->used_elems[i].ndescs; > + > + if (i >= vq->vring.num) { > + i -= vq->vring.num; > + } > + } This has a subtle problem: ndescs and elems->id are controlled by the guest, so it could make QEMU to loop forever looking for the right descriptor. For each iteration, the code must control that the variable "i" will be different for the next iteration, and that there will be no more than vq->last_avail_idx - vq->used_idx iterations. Apart of that, I think it makes more sense to split the logical sections of the function this way: /* declarations */ i = vq->used_idx /* Search for element in vq->used_elems */ while (vq->used_elems[i].index != elem->index && vq->used_elems[i].index i != vq->last_avail_idx && ...) { ... } /* Set length and mark as filled */ vq->used_elems[i].len = len; vq->used_elems[i].filled = true; --- But I'm ok either way. > +} > + > static void virtqueue_packed_fill_desc(VirtQueue *vq, > const VirtQueueElement *elem, > unsigned int idx, > @@ -923,7 +945,9 @@ void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem, > return; > } > > - if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { > + if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_IN_ORDER)) { > + virtqueue_ordered_fill(vq, elem, len); > + } else if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { > virtqueue_packed_fill(vq, elem, len, idx); > } else { > virtqueue_split_fill(vq, elem, len, idx); > -- > 2.39.3 >
On 5/9/24 10:08 AM, Eugenio Perez Martin wrote: > On Mon, May 6, 2024 at 5:05 PM Jonah Palmer <jonah.palmer@oracle.com> wrote: >> >> Add VIRTIO_F_IN_ORDER feature support for virtqueue_fill operations. >> >> The goal of the virtqueue_fill operation when the VIRTIO_F_IN_ORDER >> feature has been negotiated is to search for this now-used element, >> set its length, and mark the element as filled in the VirtQueue's >> used_elems array. >> >> By marking the element as filled, it will indicate that this element is >> ready to be flushed, so long as the element is in-order. >> >> Tested-by: Lei Yang <leiyang@redhat.com> >> Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com> >> --- >> hw/virtio/virtio.c | 26 +++++++++++++++++++++++++- >> 1 file changed, 25 insertions(+), 1 deletion(-) >> >> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c >> index e6eb1bb453..064046b5e2 100644 >> --- a/hw/virtio/virtio.c >> +++ b/hw/virtio/virtio.c >> @@ -873,6 +873,28 @@ static void virtqueue_packed_fill(VirtQueue *vq, const VirtQueueElement *elem, >> vq->used_elems[idx].ndescs = elem->ndescs; >> } >> >> +static void virtqueue_ordered_fill(VirtQueue *vq, const VirtQueueElement *elem, >> + unsigned int len) >> +{ >> + unsigned int i = vq->used_idx; >> + >> + /* Search for element in vq->used_elems */ >> + while (i != vq->last_avail_idx) { >> + /* Found element, set length and mark as filled */ >> + if (vq->used_elems[i].index == elem->index) { >> + vq->used_elems[i].len = len; >> + vq->used_elems[i].filled = true; >> + break; >> + } >> + >> + i += vq->used_elems[i].ndescs; >> + >> + if (i >= vq->vring.num) { >> + i -= vq->vring.num; >> + } >> + } > > This has a subtle problem: ndescs and elems->id are controlled by the > guest, so it could make QEMU to loop forever looking for the right > descriptor. For each iteration, the code must control that the > variable "i" will be different for the next iteration, and that there > will be no more than vq->last_avail_idx - vq->used_idx iterations. > Very true and something I was worried about, e.g. what if, for some strange reason, we could never get i == vq->last_avail_idx. Perhaps as a surefire way to make sure we terminate appropriately, as you mentioned, 'i' should not increase by more than the distance between used_idx and last_avail_idx. If it does, we exit the while loop: unsigned int steps = 0; unsigned int max_steps = (vq->last_avail_idx + vq->vring.num - vq->used_idx) % vq->vring.num; while (steps <= max_steps) { ... steps += vq->used_elems[i].ndescs; ... } Though if we do find that steps <= max_steps, should we treat this as an error or give some kind of warning? Since I believe that, under normal behavior, we shouldn't find ourselves in a situation where we weren't able to find the matching VirtQueueElement in the used_elems array. And not setting 'vq->used_elems[i].filled = true' may cause issues later. > Apart of that, I think it makes more sense to split the logical > sections of the function this way: > /* declarations */ > i = vq->used_idx > > /* Search for element in vq->used_elems */ > while (vq->used_elems[i].index != elem->index && > vq->used_elems[i].index i != vq->last_avail_idx && ...) { > ... > } > > /* Set length and mark as filled */ > vq->used_elems[i].len = len; > vq->used_elems[i].filled = true; > --- > > But I'm ok either way. > Let me know what you think of the proposed solution above. It doesn't explicitly separate the search and find operation like you're proposing here but it does clearly show the bounds of our search. But doing: while (vq->used_elems[i].index != elem->index && vq->used_elems[i].index != vq->last_avail_idx && steps <= max_steps) { ... } Works too. >> +} >> + >> static void virtqueue_packed_fill_desc(VirtQueue *vq, >> const VirtQueueElement *elem, >> unsigned int idx, >> @@ -923,7 +945,9 @@ void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem, >> return; >> } >> >> - if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { >> + if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_IN_ORDER)) { >> + virtqueue_ordered_fill(vq, elem, len); >> + } else if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { >> virtqueue_packed_fill(vq, elem, len, idx); >> } else { >> virtqueue_split_fill(vq, elem, len, idx); >> -- >> 2.39.3 >> >
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index e6eb1bb453..064046b5e2 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -873,6 +873,28 @@ static void virtqueue_packed_fill(VirtQueue *vq, const VirtQueueElement *elem, vq->used_elems[idx].ndescs = elem->ndescs; } +static void virtqueue_ordered_fill(VirtQueue *vq, const VirtQueueElement *elem, + unsigned int len) +{ + unsigned int i = vq->used_idx; + + /* Search for element in vq->used_elems */ + while (i != vq->last_avail_idx) { + /* Found element, set length and mark as filled */ + if (vq->used_elems[i].index == elem->index) { + vq->used_elems[i].len = len; + vq->used_elems[i].filled = true; + break; + } + + i += vq->used_elems[i].ndescs; + + if (i >= vq->vring.num) { + i -= vq->vring.num; + } + } +} + static void virtqueue_packed_fill_desc(VirtQueue *vq, const VirtQueueElement *elem, unsigned int idx, @@ -923,7 +945,9 @@ void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem, return; } - if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { + if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_IN_ORDER)) { + virtqueue_ordered_fill(vq, elem, len); + } else if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { virtqueue_packed_fill(vq, elem, len, idx); } else { virtqueue_split_fill(vq, elem, len, idx);