| Message ID | 20190624091304.666-1-stefanha@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | docs: clarify multiqueue vs multiple virtqueues | expand |
Hi On Mon, Jun 24, 2019 at 11:13 AM Stefan Hajnoczi <stefanha@redhat.com> wrote: > > The vhost-user specification does not explain when > VHOST_USER_PROTOCOL_F_MQ must be implemented. This may lead > implementors of vhost-user masters to believe that this protocol feature > is required for any device that has multiple virtqueues. That would be > a mistake since existing vhost-user slaves offer multiple virtqueues but > do not advertise VHOST_USER_PROTOCOL_F_MQ. > > For example, a vhost-net device with one rx/tx queue pair is not > multiqueue. The slave does not need to advertise > VHOST_USER_PROTOCOL_F_MQ. Therefore the master must assume it has these > virtqueues and cannot rely on askingt the slave how many virtqueues > exist. > > Extend the specification to explain the different between true > multiqueue and regular devices with a fixed virtqueue layout. > > Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> > --- > Based-on: <20190621094005.4134-1-stefanha@redhat.com> > --- > docs/interop/vhost-user.rst | 17 +++++++++++++++++ > 1 file changed, 17 insertions(+) > > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst > index 5750668aba..7827b710aa 100644 > --- a/docs/interop/vhost-user.rst > +++ b/docs/interop/vhost-user.rst > @@ -324,6 +324,15 @@ must support changing some configuration aspects on the fly. > Multiple queue support > ---------------------- > > +Many devices have a fixed number of virtqueues. In this case the master > +already knows the number of available virtqueues without communicating with the > +slave. > + > +Some devices do not have a fixed number of virtqueues. Instead the maximum > +number of virtqueues is chosen by the slave. The number can depend on host > +resource availability or slave implementation details. Such devices are called > +multiple queue devices. > + > Multiple queue support allows the slave to advertise the maximum number of > queues. This is treated as a protocol extension, hence the slave has to > implement protocol features first. The multiple queues feature is supported > @@ -339,6 +348,14 @@ queue in the sent message to identify a specified queue. > The master enables queues by sending message ``VHOST_USER_SET_VRING_ENABLE``. > vhost-user-net has historically automatically enabled the first queue pair. > > +Slaves should always implement the ``VHOST_USER_PROTOCOL_F_MQ`` protocol > +feature, even for devices with a fixed number of virtqueues, since it is simple > +to implement and offers a degree of introspection. > + > +Masters must not rely on the ``VHOST_USER_PROTOCOL_F_MQ`` protocol feature for > +devices with a fixed number of virtqueues. Only true multiqueue devices > +require this protocol feature. > + > Migration > --------- > > -- > 2.21.0 > >
On Mon, Jun 24, 2019 at 10:13:04AM +0100, Stefan Hajnoczi wrote: > The vhost-user specification does not explain when > VHOST_USER_PROTOCOL_F_MQ must be implemented. This may lead > implementors of vhost-user masters to believe that this protocol feature > is required for any device that has multiple virtqueues. That would be > a mistake since existing vhost-user slaves offer multiple virtqueues but > do not advertise VHOST_USER_PROTOCOL_F_MQ. > > For example, a vhost-net device with one rx/tx queue pair is not > multiqueue. The slave does not need to advertise > VHOST_USER_PROTOCOL_F_MQ. Therefore the master must assume it has these > virtqueues and cannot rely on askingt the slave how many virtqueues > exist. > > Extend the specification to explain the different between true > multiqueue and regular devices with a fixed virtqueue layout. > > Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> > --- > Based-on: <20190621094005.4134-1-stefanha@redhat.com> > --- > docs/interop/vhost-user.rst | 17 +++++++++++++++++ > 1 file changed, 17 insertions(+) Ping? > > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst > index 5750668aba..7827b710aa 100644 > --- a/docs/interop/vhost-user.rst > +++ b/docs/interop/vhost-user.rst > @@ -324,6 +324,15 @@ must support changing some configuration aspects on the fly. > Multiple queue support > ---------------------- > > +Many devices have a fixed number of virtqueues. In this case the master > +already knows the number of available virtqueues without communicating with the > +slave. > + > +Some devices do not have a fixed number of virtqueues. Instead the maximum > +number of virtqueues is chosen by the slave. The number can depend on host > +resource availability or slave implementation details. Such devices are called > +multiple queue devices. > + > Multiple queue support allows the slave to advertise the maximum number of > queues. This is treated as a protocol extension, hence the slave has to > implement protocol features first. The multiple queues feature is supported > @@ -339,6 +348,14 @@ queue in the sent message to identify a specified queue. > The master enables queues by sending message ``VHOST_USER_SET_VRING_ENABLE``. > vhost-user-net has historically automatically enabled the first queue pair. > > +Slaves should always implement the ``VHOST_USER_PROTOCOL_F_MQ`` protocol > +feature, even for devices with a fixed number of virtqueues, since it is simple > +to implement and offers a degree of introspection. > + > +Masters must not rely on the ``VHOST_USER_PROTOCOL_F_MQ`` protocol feature for > +devices with a fixed number of virtqueues. Only true multiqueue devices > +require this protocol feature. > + > Migration > --------- > > -- > 2.21.0 >
On Wed, Jul 17, 2019 at 11:14:53AM +0100, Stefan Hajnoczi wrote: > On Mon, Jun 24, 2019 at 10:13:04AM +0100, Stefan Hajnoczi wrote: > > The vhost-user specification does not explain when > > VHOST_USER_PROTOCOL_F_MQ must be implemented. This may lead > > implementors of vhost-user masters to believe that this protocol feature > > is required for any device that has multiple virtqueues. That would be > > a mistake since existing vhost-user slaves offer multiple virtqueues but > > do not advertise VHOST_USER_PROTOCOL_F_MQ. > > > > For example, a vhost-net device with one rx/tx queue pair is not > > multiqueue. The slave does not need to advertise > > VHOST_USER_PROTOCOL_F_MQ. Therefore the master must assume it has these > > virtqueues and cannot rely on askingt the slave how many virtqueues > > exist. > > > > Extend the specification to explain the different between true > > multiqueue and regular devices with a fixed virtqueue layout. > > > > Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> > > --- > > Based-on: <20190621094005.4134-1-stefanha@redhat.com> > > --- > > docs/interop/vhost-user.rst | 17 +++++++++++++++++ > > 1 file changed, 17 insertions(+) > > Ping? queued, thanks! > > > > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst > > index 5750668aba..7827b710aa 100644 > > --- a/docs/interop/vhost-user.rst > > +++ b/docs/interop/vhost-user.rst > > @@ -324,6 +324,15 @@ must support changing some configuration aspects on the fly. > > Multiple queue support > > ---------------------- > > > > +Many devices have a fixed number of virtqueues. In this case the master > > +already knows the number of available virtqueues without communicating with the > > +slave. > > + > > +Some devices do not have a fixed number of virtqueues. Instead the maximum > > +number of virtqueues is chosen by the slave. The number can depend on host > > +resource availability or slave implementation details. Such devices are called > > +multiple queue devices. > > + > > Multiple queue support allows the slave to advertise the maximum number of > > queues. This is treated as a protocol extension, hence the slave has to > > implement protocol features first. The multiple queues feature is supported > > @@ -339,6 +348,14 @@ queue in the sent message to identify a specified queue. > > The master enables queues by sending message ``VHOST_USER_SET_VRING_ENABLE``. > > vhost-user-net has historically automatically enabled the first queue pair. > > > > +Slaves should always implement the ``VHOST_USER_PROTOCOL_F_MQ`` protocol > > +feature, even for devices with a fixed number of virtqueues, since it is simple > > +to implement and offers a degree of introspection. > > + > > +Masters must not rely on the ``VHOST_USER_PROTOCOL_F_MQ`` protocol feature for > > +devices with a fixed number of virtqueues. Only true multiqueue devices > > +require this protocol feature. > > + > > Migration > > --------- > > > > -- > > 2.21.0 > >
diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst index 5750668aba..7827b710aa 100644 --- a/docs/interop/vhost-user.rst +++ b/docs/interop/vhost-user.rst @@ -324,6 +324,15 @@ must support changing some configuration aspects on the fly. Multiple queue support ---------------------- +Many devices have a fixed number of virtqueues. In this case the master +already knows the number of available virtqueues without communicating with the +slave. + +Some devices do not have a fixed number of virtqueues. Instead the maximum +number of virtqueues is chosen by the slave. The number can depend on host +resource availability or slave implementation details. Such devices are called +multiple queue devices. + Multiple queue support allows the slave to advertise the maximum number of queues. This is treated as a protocol extension, hence the slave has to implement protocol features first. The multiple queues feature is supported @@ -339,6 +348,14 @@ queue in the sent message to identify a specified queue. The master enables queues by sending message ``VHOST_USER_SET_VRING_ENABLE``. vhost-user-net has historically automatically enabled the first queue pair. +Slaves should always implement the ``VHOST_USER_PROTOCOL_F_MQ`` protocol +feature, even for devices with a fixed number of virtqueues, since it is simple +to implement and offers a degree of introspection. + +Masters must not rely on the ``VHOST_USER_PROTOCOL_F_MQ`` protocol feature for +devices with a fixed number of virtqueues. Only true multiqueue devices +require this protocol feature. + Migration ---------
The vhost-user specification does not explain when VHOST_USER_PROTOCOL_F_MQ must be implemented. This may lead implementors of vhost-user masters to believe that this protocol feature is required for any device that has multiple virtqueues. That would be a mistake since existing vhost-user slaves offer multiple virtqueues but do not advertise VHOST_USER_PROTOCOL_F_MQ. For example, a vhost-net device with one rx/tx queue pair is not multiqueue. The slave does not need to advertise VHOST_USER_PROTOCOL_F_MQ. Therefore the master must assume it has these virtqueues and cannot rely on askingt the slave how many virtqueues exist. Extend the specification to explain the different between true multiqueue and regular devices with a fixed virtqueue layout. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> --- Based-on: <20190621094005.4134-1-stefanha@redhat.com> --- docs/interop/vhost-user.rst | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)