[099/104] virtiofsd: use fuse_buf_writev to replace fuse_buf_write for better performance

Commit Message

Dr. David Alan Gilbert Dec. 12, 2019, 4:38 p.m. UTC

From: piaojun <[e-mail]>

fuse_buf_writev() only handles the normal write in which src is buffer
and dest is fd. Specially if src buffer represents guest physical
address that can't be mapped by the daemon process, IO must be bounced
back to the VMM to do it by fuse_buf_copy().

Signed-off-by: Jun Piao <[e-mail]>
Suggested-by: Dr. David Alan Gilbert <[e-mail]>
Suggested-by: Stefan Hajnoczi <[e-mail]>
---
 tools/virtiofsd/buffer.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

Comments

Daniel P. Berrangé Jan. 7, 2020, 12:23 p.m. UTC | #1

On Thu, Dec 12, 2019 at 04:38:59PM +0000, Dr. David Alan Gilbert (git) wrote:
> From: piaojun <[e-mail]>
> 
> fuse_buf_writev() only handles the normal write in which src is buffer
> and dest is fd. Specially if src buffer represents guest physical
> address that can't be mapped by the daemon process, IO must be bounced
> back to the VMM to do it by fuse_buf_copy().
> 
> Signed-off-by: Jun Piao <[e-mail]>
> Suggested-by: Dr. David Alan Gilbert <[e-mail]>
> Suggested-by: Stefan Hajnoczi <[e-mail]>
> ---
>  tools/virtiofsd/buffer.c | 23 +++++++++++++++++++----
>  1 file changed, 19 insertions(+), 4 deletions(-)

Reviewed-by: Daniel P. Berrangé <[e-mail]>

> 
> diff --git a/tools/virtiofsd/buffer.c b/tools/virtiofsd/buffer.c
> index ae420c70c4..4875473785 100644
> --- a/tools/virtiofsd/buffer.c
> +++ b/tools/virtiofsd/buffer.c
> @@ -33,9 +33,7 @@ size_t fuse_buf_size(const struct fuse_bufvec *bufv)
>      return size;
>  }
>  
> -__attribute__((unused))
> -static ssize_t fuse_buf_writev(fuse_req_t req,

Lets cull the fuse_req_t param in the previous patch

> -                               struct fuse_buf *out_buf,
> +static ssize_t fuse_buf_writev(struct fuse_buf *out_buf,
>                                 struct fuse_bufvec *in_buf)
>  {
>      ssize_t res, i, j;

Regards,
Daniel

Dr. David Alan Gilbert Jan. 10, 2020, 1:15 p.m. UTC | #2

* Daniel P. Berrangé ([e-mail]) wrote:
> On Thu, Dec 12, 2019 at 04:38:59PM +0000, Dr. David Alan Gilbert (git) wrote:
> > From: piaojun <[e-mail]>
> > 
> > fuse_buf_writev() only handles the normal write in which src is buffer
> > and dest is fd. Specially if src buffer represents guest physical
> > address that can't be mapped by the daemon process, IO must be bounced
> > back to the VMM to do it by fuse_buf_copy().
> > 
> > Signed-off-by: Jun Piao <[e-mail]>
> > Suggested-by: Dr. David Alan Gilbert <[e-mail]>
> > Suggested-by: Stefan Hajnoczi <[e-mail]>
> > ---
> >  tools/virtiofsd/buffer.c | 23 +++++++++++++++++++----
> >  1 file changed, 19 insertions(+), 4 deletions(-)
> 
> Reviewed-by: Daniel P. Berrangé <[e-mail]>
> 
> > 
> > diff --git a/tools/virtiofsd/buffer.c b/tools/virtiofsd/buffer.c
> > index ae420c70c4..4875473785 100644
> > --- a/tools/virtiofsd/buffer.c
> > +++ b/tools/virtiofsd/buffer.c
> > @@ -33,9 +33,7 @@ size_t fuse_buf_size(const struct fuse_bufvec *bufv)
> >      return size;
> >  }
> >  
> > -__attribute__((unused))
> > -static ssize_t fuse_buf_writev(fuse_req_t req,
> 
> Lets cull the fuse_req_t param in the previous patch

Done.

> > -                               struct fuse_buf *out_buf,
> > +static ssize_t fuse_buf_writev(struct fuse_buf *out_buf,
> >                                 struct fuse_bufvec *in_buf)
> >  {
> >      ssize_t res, i, j;
> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
--
Dr. David Alan Gilbert / [e-mail] / Manchester, UK

Patch

diff --git a/tools/virtiofsd/buffer.c b/tools/virtiofsd/buffer.c
index ae420c70c4..4875473785 100644
--- a/tools/virtiofsd/buffer.c
+++ b/tools/virtiofsd/buffer.c
@@ -33,9 +33,7 @@  size_t fuse_buf_size(const struct fuse_bufvec *bufv)
     return size;
 }
 
-__attribute__((unused))
-static ssize_t fuse_buf_writev(fuse_req_t req,
-                               struct fuse_buf *out_buf,
+static ssize_t fuse_buf_writev(struct fuse_buf *out_buf,
                                struct fuse_bufvec *in_buf)
 {
     ssize_t res, i, j;
@@ -334,12 +332,29 @@  static int fuse_bufvec_advance(struct fuse_bufvec *bufv, size_t len)
 ssize_t fuse_buf_copy(struct fuse_bufvec *dstv, struct fuse_bufvec *srcv,
                       enum fuse_buf_copy_flags flags)
 {
-    size_t copied = 0;
+    size_t copied = 0, i;
 
     if (dstv == srcv) {
         return fuse_buf_size(dstv);
     }
 
+    /*
+     * use writev to improve bandwidth when all the
+     * src buffers already mapped by the daemon
+     * process
+     */
+    for (i = 0; i < srcv->count; i++) {
+        if (srcv->buf[i].flags & FUSE_BUF_IS_FD) {
+            break;
+        }
+    }
+    if ((i == srcv->count) && (dstv->count == 1) &&
+        (dstv->idx == 0) &&
+        (dstv->buf[0].flags & FUSE_BUF_IS_FD)) {
+        dstv->buf[0].pos += dstv->off;
+        return fuse_buf_writev(&dstv->buf[0], srcv);
+    }
+
     for (;;) {
         const struct fuse_buf *src = fuse_bufvec_current(srcv);
         const struct fuse_buf *dst = fuse_bufvec_current(dstv);