| Message ID | 1460029495-7146-1-git-send-email-pbonzini@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Apr 07, 2016 at 01:44:55PM +0200, Paolo Bonzini wrote: > Until commit 1c778ef7 ("nbd: convert to using I/O channels for actual > socket I/O", 2016-02-16), nbd_wr_sync returned -EAGAIN this scenario. > nbd_reply_ready required these semantics because it has two conflicting > requirements: > > 1) if a reply can be received on the socket, nbd_reply_ready needs > to read the header outside coroutine context to identify _which_ > coroutine to enter to process the rest of the reply > > 2) on the other hand, nbd_reply_ready can find a false positive if > another thread (e.g. a VCPU thread running aio_poll) sneaks in and > calls nbd_reply_ready too. In this case nbd_reply_ready does nothing > and expects nbd_wr_syncv to return -EAGAIN. > > Currently, the solution to the first requirement is to wait in the very > rare case of a read() that doesn't retrieve the reply header in its > entirety; this is what nbd_wr_syncv does by calling qio_channel_wait(). > However, the unconditional call to qio_channel_wait() breaks the second > requirement. To fix this, the patch makes nbd_wr_syncv return -EAGAIN > if done is zero, similar to the code before commit 1c778ef7. > > This is okay because NBD client-side negotiation is the only other case > that calls nbd_wr_syncv outside a coroutine, and it places the socket > in blocking mode. On the other hand, it is a bit unpleasant to put > this in nbd_wr_syncv(), because the function is used by both client > and server. > > The full fix would be to add a counter to NbdClientSession for how > many bytes have been filled in s->reply. Then a reply can be filled > by multiple separate invocations of nbd_reply_ready and the > qio_channel_wait() call can be removed completely. Something to > consider for 2.7... > > Reported-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com> > Cc: Daniel P. Berrange <berrange@redhat.com> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > nbd/common.c | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) Reviewed-by: Daniel P. berrange <berrange@redhat.com> Regards, Daniel
On 04/07/2016 07:44 PM, Paolo Bonzini wrote: > Until commit 1c778ef7 ("nbd: convert to using I/O channels for actual > socket I/O", 2016-02-16), nbd_wr_sync returned -EAGAIN this scenario. > nbd_reply_ready required these semantics because it has two conflicting > requirements: > > 1) if a reply can be received on the socket, nbd_reply_ready needs > to read the header outside coroutine context to identify _which_ > coroutine to enter to process the rest of the reply > > 2) on the other hand, nbd_reply_ready can find a false positive if > another thread (e.g. a VCPU thread running aio_poll) sneaks in and > calls nbd_reply_ready too. In this case nbd_reply_ready does nothing > and expects nbd_wr_syncv to return -EAGAIN. > > Currently, the solution to the first requirement is to wait in the very > rare case of a read() that doesn't retrieve the reply header in its > entirety; this is what nbd_wr_syncv does by calling qio_channel_wait(). > However, the unconditional call to qio_channel_wait() breaks the second > requirement. To fix this, the patch makes nbd_wr_syncv return -EAGAIN > if done is zero, similar to the code before commit 1c778ef7. > > This is okay because NBD client-side negotiation is the only other case > that calls nbd_wr_syncv outside a coroutine, and it places the socket > in blocking mode. On the other hand, it is a bit unpleasant to put > this in nbd_wr_syncv(), because the function is used by both client > and server. > > The full fix would be to add a counter to NbdClientSession for how > many bytes have been filled in s->reply. Then a reply can be filled > by multiple separate invocations of nbd_reply_ready and the > qio_channel_wait() call can be removed completely. Something to > consider for 2.7... > > Reported-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com> > Cc: Daniel P. Berrange <berrange@redhat.com> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > nbd/common.c | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/nbd/common.c b/nbd/common.c > index a44718c..8ddb2dd 100644 > --- a/nbd/common.c > +++ b/nbd/common.c > @@ -50,9 +50,12 @@ ssize_t nbd_wr_syncv(QIOChannel *ioc, > * qio_channel_yield() that works with AIO contexts > * and consider using that in this branch */ > qemu_coroutine_yield(); > - } else { > + } else if (done) { > + /* XXX this is needed by nbd_reply_ready. */ > qio_channel_wait(ioc, > do_read ? G_IO_IN : G_IO_OUT); > + } else { > + return -EAGAIN; > } With this patch, nbd works well now. Thanks -Xie > continue; > } >
diff --git a/nbd/common.c b/nbd/common.c index a44718c..8ddb2dd 100644 --- a/nbd/common.c +++ b/nbd/common.c @@ -50,9 +50,12 @@ ssize_t nbd_wr_syncv(QIOChannel *ioc, * qio_channel_yield() that works with AIO contexts * and consider using that in this branch */ qemu_coroutine_yield(); - } else { + } else if (done) { + /* XXX this is needed by nbd_reply_ready. */ qio_channel_wait(ioc, do_read ? G_IO_IN : G_IO_OUT); + } else { + return -EAGAIN; } continue; }
Until commit 1c778ef7 ("nbd: convert to using I/O channels for actual socket I/O", 2016-02-16), nbd_wr_sync returned -EAGAIN this scenario. nbd_reply_ready required these semantics because it has two conflicting requirements: 1) if a reply can be received on the socket, nbd_reply_ready needs to read the header outside coroutine context to identify _which_ coroutine to enter to process the rest of the reply 2) on the other hand, nbd_reply_ready can find a false positive if another thread (e.g. a VCPU thread running aio_poll) sneaks in and calls nbd_reply_ready too. In this case nbd_reply_ready does nothing and expects nbd_wr_syncv to return -EAGAIN. Currently, the solution to the first requirement is to wait in the very rare case of a read() that doesn't retrieve the reply header in its entirety; this is what nbd_wr_syncv does by calling qio_channel_wait(). However, the unconditional call to qio_channel_wait() breaks the second requirement. To fix this, the patch makes nbd_wr_syncv return -EAGAIN if done is zero, similar to the code before commit 1c778ef7. This is okay because NBD client-side negotiation is the only other case that calls nbd_wr_syncv outside a coroutine, and it places the socket in blocking mode. On the other hand, it is a bit unpleasant to put this in nbd_wr_syncv(), because the function is used by both client and server. The full fix would be to add a counter to NbdClientSession for how many bytes have been filled in s->reply. Then a reply can be filled by multiple separate invocations of nbd_reply_ready and the qio_channel_wait() call can be removed completely. Something to consider for 2.7... Reported-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com> Cc: Daniel P. Berrange <berrange@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- nbd/common.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)