mbox

[PULL,00/23] 9p queue 2022-10-24

Message ID cover.1666608862.git.qemu_oss@crudebyte.com (mailing list archive)
State New, archived
Headers show

Pull-request

https://github.com/cschoenebeck/qemu.git tags/pull-9p-20221024

Message

Christian Schoenebeck Oct. 24, 2022, 10:54 a.m. UTC
The following changes since commit 0529245488865038344d64fff7ee05864d3d17f6:

  Merge tag 'pull-target-arm-20221020' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2022-10-20 14:36:12 -0400)

are available in the Git repository at:

  https://github.com/cschoenebeck/qemu.git tags/pull-9p-20221024

for you to fetch changes up to 3ce77865bf813f313cf79c00fd951bfc95a50165:

  tests/9p: remove unnecessary g_strdup() calls (2022-10-24 12:24:32 +0200)

----------------------------------------------------------------
9pfs: performance, Windows host prep, tests restructure

* Highlight of this PR is Linus Heckemann's GHashTable patch which
  brings massive general performance improvements of 9p server
  somewhere between factor 6 .. 12.

* Bin Meng's g_mkdir patch is a preparatory patch for upcoming
  Windows host support of 9p server.

* The rest of the patches in this PR are 9p test code restructuring
  and refactoring changes to improve readability and to ease
  maintenance of 9p test code on the long-term.

----------------------------------------------------------------
Bin Meng (1):
      fsdev/virtfs-proxy-helper: Use g_mkdir()

Christian Schoenebeck (21):
      tests/9p: split virtio-9p-test.c into tests and 9p client part
      tests/9p: merge *walk*() functions
      tests/9p: simplify callers of twalk()
      tests/9p: merge v9fs_tversion() and do_version()
      tests/9p: merge v9fs_tattach(), do_attach(), do_attach_rqid()
      tests/9p: simplify callers of tattach()
      tests/9p: convert v9fs_tgetattr() to declarative arguments
      tests/9p: simplify callers of tgetattr()
      tests/9p: convert v9fs_treaddir() to declarative arguments
      tests/9p: simplify callers of treaddir()
      tests/9p: convert v9fs_tlopen() to declarative arguments
      tests/9p: simplify callers of tlopen()
      tests/9p: convert v9fs_twrite() to declarative arguments
      tests/9p: simplify callers of twrite()
      tests/9p: convert v9fs_tflush() to declarative arguments
      tests/9p: merge v9fs_tmkdir() and do_mkdir()
      tests/9p: merge v9fs_tlcreate() and do_lcreate()
      tests/9p: merge v9fs_tsymlink() and do_symlink()
      tests/9p: merge v9fs_tlink() and do_hardlink()
      tests/9p: merge v9fs_tunlinkat() and do_unlinkat()
      tests/9p: remove unnecessary g_strdup() calls

Linus Heckemann (1):
      9pfs: use GHashTable for fid table

 fsdev/virtfs-proxy-helper.c           |    3 +-
 hw/9pfs/9p.c                          |  196 ++---
 hw/9pfs/9p.h                          |    2 +-
 tests/qtest/libqos/meson.build        |    1 +
 tests/qtest/libqos/virtio-9p-client.c | 1049 ++++++++++++++++++++++++++
 tests/qtest/libqos/virtio-9p-client.h |  494 +++++++++++++
 tests/qtest/virtio-9p-test.c          | 1299 ++++++---------------------------
 7 files changed, 1867 insertions(+), 1177 deletions(-)
 create mode 100644 tests/qtest/libqos/virtio-9p-client.c
 create mode 100644 tests/qtest/libqos/virtio-9p-client.h

Comments

Stefan Hajnoczi Oct. 24, 2022, 9:07 p.m. UTC | #1
Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/7.2 for any user-visible changes.
Thomas Huth Oct. 27, 2022, 3:53 p.m. UTC | #2
On 24/10/2022 12.54, Christian Schoenebeck wrote:
> The following changes since commit 0529245488865038344d64fff7ee05864d3d17f6:
> 
>    Merge tag 'pull-target-arm-20221020' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2022-10-20 14:36:12 -0400)
> 
> are available in the Git repository at:
> 
>    https://github.com/cschoenebeck/qemu.git tags/pull-9p-20221024
> 
> for you to fetch changes up to 3ce77865bf813f313cf79c00fd951bfc95a50165:
> 
>    tests/9p: remove unnecessary g_strdup() calls (2022-10-24 12:24:32 +0200)
> 
> ----------------------------------------------------------------
> 9pfs: performance, Windows host prep, tests restructure
> 
> * Highlight of this PR is Linus Heckemann's GHashTable patch which
>    brings massive general performance improvements of 9p server
>    somewhere between factor 6 .. 12.
> 
> * Bin Meng's g_mkdir patch is a preparatory patch for upcoming
>    Windows host support of 9p server.
> 
> * The rest of the patches in this PR are 9p test code restructuring
>    and refactoring changes to improve readability and to ease
>    maintenance of 9p test code on the long-term.

  Hi Christian,

I think this PR broke the FreeBSD CI jobs:

  https://gitlab.com/qemu-project/qemu/-/jobs/3219611457#L3116

  https://gitlab.com/qemu-project/qemu/-/jobs/3219611460#L3372

Could you please have a look?

  Thanks!
   Thomas
Christian Schoenebeck Oct. 27, 2022, 4:37 p.m. UTC | #3
On Thursday, October 27, 2022 5:53:47 PM CEST Thomas Huth wrote:
> On 24/10/2022 12.54, Christian Schoenebeck wrote:
> > The following changes since commit 0529245488865038344d64fff7ee05864d3d17f6:
> > 
> >    Merge tag 'pull-target-arm-20221020' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2022-10-20 14:36:12 -0400)
> > 
> > are available in the Git repository at:
> > 
> >    https://github.com/cschoenebeck/qemu.git tags/pull-9p-20221024
> > 
> > for you to fetch changes up to 3ce77865bf813f313cf79c00fd951bfc95a50165:
> > 
> >    tests/9p: remove unnecessary g_strdup() calls (2022-10-24 12:24:32 +0200)
> > 
> > ----------------------------------------------------------------
> > 9pfs: performance, Windows host prep, tests restructure
> > 
> > * Highlight of this PR is Linus Heckemann's GHashTable patch which
> >    brings massive general performance improvements of 9p server
> >    somewhere between factor 6 .. 12.
> > 
> > * Bin Meng's g_mkdir patch is a preparatory patch for upcoming
> >    Windows host support of 9p server.
> > 
> > * The rest of the patches in this PR are 9p test code restructuring
> >    and refactoring changes to improve readability and to ease
> >    maintenance of 9p test code on the long-term.
> 
>   Hi Christian,
> 
> I think this PR broke the FreeBSD CI jobs:
> 
>   https://gitlab.com/qemu-project/qemu/-/jobs/3219611457#L3116
> 
>   https://gitlab.com/qemu-project/qemu/-/jobs/3219611460#L3372
> 
> Could you please have a look?
> 
>   Thanks!
>    Thomas

I try, but will certainly take some days, especially as I currently don't have
a BSD installation at hand to try the changes.

Right now I don't see something obvious. I would expect statfs being defined
in <sys/types.h> on BSD which is already included via "qemu/osdep.h" from
virtio-9p-client.c.

Best regards,
Christian Schoenebeck
Stefan Hajnoczi Oct. 27, 2022, 5:37 p.m. UTC | #4
On Thu, 27 Oct 2022 at 12:38, Christian Schoenebeck
<qemu_oss@crudebyte.com> wrote:
>
> On Thursday, October 27, 2022 5:53:47 PM CEST Thomas Huth wrote:
> > On 24/10/2022 12.54, Christian Schoenebeck wrote:
> > > The following changes since commit 0529245488865038344d64fff7ee05864d3d17f6:
> > >
> > >    Merge tag 'pull-target-arm-20221020' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2022-10-20 14:36:12 -0400)
> > >
> > > are available in the Git repository at:
> > >
> > >    https://github.com/cschoenebeck/qemu.git tags/pull-9p-20221024
> > >
> > > for you to fetch changes up to 3ce77865bf813f313cf79c00fd951bfc95a50165:
> > >
> > >    tests/9p: remove unnecessary g_strdup() calls (2022-10-24 12:24:32 +0200)
> > >
> > > ----------------------------------------------------------------
> > > 9pfs: performance, Windows host prep, tests restructure
> > >
> > > * Highlight of this PR is Linus Heckemann's GHashTable patch which
> > >    brings massive general performance improvements of 9p server
> > >    somewhere between factor 6 .. 12.
> > >
> > > * Bin Meng's g_mkdir patch is a preparatory patch for upcoming
> > >    Windows host support of 9p server.
> > >
> > > * The rest of the patches in this PR are 9p test code restructuring
> > >    and refactoring changes to improve readability and to ease
> > >    maintenance of 9p test code on the long-term.
> >
> >   Hi Christian,
> >
> > I think this PR broke the FreeBSD CI jobs:
> >
> >   https://gitlab.com/qemu-project/qemu/-/jobs/3219611457#L3116
> >
> >   https://gitlab.com/qemu-project/qemu/-/jobs/3219611460#L3372
> >
> > Could you please have a look?
> >
> >   Thanks!
> >    Thomas
>
> I try, but will certainly take some days, especially as I currently don't have
> a BSD installation at hand to try the changes.

QEMU has the automation to run FreeBSD builds locally (in a VM):
$ make vm-build-freebsd

Not sure if that FreeBSD environment matches the one in Cirrus CI
though. If they are different then maybe it won't reproduce locally.

Stefan
Christian Schoenebeck Oct. 27, 2022, 8:42 p.m. UTC | #5
On Thursday, October 27, 2022 7:37:07 PM CEST Stefan Hajnoczi wrote:
> On Thu, 27 Oct 2022 at 12:38, Christian Schoenebeck
> <qemu_oss@crudebyte.com> wrote:
> >
> > On Thursday, October 27, 2022 5:53:47 PM CEST Thomas Huth wrote:
> > > On 24/10/2022 12.54, Christian Schoenebeck wrote:
> > > > The following changes since commit 0529245488865038344d64fff7ee05864d3d17f6:
> > > >
> > > >    Merge tag 'pull-target-arm-20221020' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2022-10-20 14:36:12 -0400)
> > > >
> > > > are available in the Git repository at:
> > > >
> > > >    https://github.com/cschoenebeck/qemu.git tags/pull-9p-20221024
> > > >
> > > > for you to fetch changes up to 3ce77865bf813f313cf79c00fd951bfc95a50165:
> > > >
> > > >    tests/9p: remove unnecessary g_strdup() calls (2022-10-24 12:24:32 +0200)
> > > >
> > > > ----------------------------------------------------------------
> > > > 9pfs: performance, Windows host prep, tests restructure
> > > >
> > > > * Highlight of this PR is Linus Heckemann's GHashTable patch which
> > > >    brings massive general performance improvements of 9p server
> > > >    somewhere between factor 6 .. 12.
> > > >
> > > > * Bin Meng's g_mkdir patch is a preparatory patch for upcoming
> > > >    Windows host support of 9p server.
> > > >
> > > > * The rest of the patches in this PR are 9p test code restructuring
> > > >    and refactoring changes to improve readability and to ease
> > > >    maintenance of 9p test code on the long-term.
> > >
> > >   Hi Christian,
> > >
> > > I think this PR broke the FreeBSD CI jobs:
> > >
> > >   https://gitlab.com/qemu-project/qemu/-/jobs/3219611457#L3116
> > >
> > >   https://gitlab.com/qemu-project/qemu/-/jobs/3219611460#L3372
> > >
> > > Could you please have a look?
> > >
> > >   Thanks!
> > >    Thomas
> >
> > I try, but will certainly take some days, especially as I currently don't have
> > a BSD installation at hand to try the changes.
> 
> QEMU has the automation to run FreeBSD builds locally (in a VM):
> $ make vm-build-freebsd
> 
> Not sure if that FreeBSD environment matches the one in Cirrus CI
> though. If they are different then maybe it won't reproduce locally.

Something must be different, because e750a7ace492f0b450653d4ad368a77d6f660fb8
compiles fine locally with `make vm-build-freebsd` and all tests pass, too.

Ideas?

Best regards,
Christian Schoenebeck
Warner Losh Oct. 27, 2022, 8:53 p.m. UTC | #6
On Thu, Oct 27, 2022, 1:45 PM Christian Schoenebeck <qemu_oss@crudebyte.com>
wrote:

> On Thursday, October 27, 2022 7:37:07 PM CEST Stefan Hajnoczi wrote:
> > On Thu, 27 Oct 2022 at 12:38, Christian Schoenebeck
> > <qemu_oss@crudebyte.com> wrote:
> > >
> > > On Thursday, October 27, 2022 5:53:47 PM CEST Thomas Huth wrote:
> > > > On 24/10/2022 12.54, Christian Schoenebeck wrote:
> > > > > The following changes since commit
> 0529245488865038344d64fff7ee05864d3d17f6:
> > > > >
> > > > >    Merge tag 'pull-target-arm-20221020' of
> https://git.linaro.org/people/pmaydell/qemu-arm into staging (2022-10-20
> 14:36:12 -0400)
> > > > >
> > > > > are available in the Git repository at:
> > > > >
> > > > >    https://github.com/cschoenebeck/qemu.git tags/pull-9p-20221024
> > > > >
> > > > > for you to fetch changes up to
> 3ce77865bf813f313cf79c00fd951bfc95a50165:
> > > > >
> > > > >    tests/9p: remove unnecessary g_strdup() calls (2022-10-24
> 12:24:32 +0200)
> > > > >
> > > > > ----------------------------------------------------------------
> > > > > 9pfs: performance, Windows host prep, tests restructure
> > > > >
> > > > > * Highlight of this PR is Linus Heckemann's GHashTable patch which
> > > > >    brings massive general performance improvements of 9p server
> > > > >    somewhere between factor 6 .. 12.
> > > > >
> > > > > * Bin Meng's g_mkdir patch is a preparatory patch for upcoming
> > > > >    Windows host support of 9p server.
> > > > >
> > > > > * The rest of the patches in this PR are 9p test code restructuring
> > > > >    and refactoring changes to improve readability and to ease
> > > > >    maintenance of 9p test code on the long-term.
> > > >
> > > >   Hi Christian,
> > > >
> > > > I think this PR broke the FreeBSD CI jobs:
> > > >
> > > >   https://gitlab.com/qemu-project/qemu/-/jobs/3219611457#L3116
> > > >
> > > >   https://gitlab.com/qemu-project/qemu/-/jobs/3219611460#L3372
> > > >
> > > > Could you please have a look?
> > > >
> > > >   Thanks!
> > > >    Thomas
> > >
> > > I try, but will certainly take some days, especially as I currently
> don't have
> > > a BSD installation at hand to try the changes.
> >
> > QEMU has the automation to run FreeBSD builds locally (in a VM):
> > $ make vm-build-freebsd
> >
> > Not sure if that FreeBSD environment matches the one in Cirrus CI
> > though. If they are different then maybe it won't reproduce locally.
>
> Something must be different, because
> e750a7ace492f0b450653d4ad368a77d6f660fb8
> compiles fine locally with `make vm-build-freebsd` and all tests pass, too.
>

I have a pull request I'm sending tomorrow that fixes a build issue on
FreeBSD for about 6 or so months on FreeBSD current. If it's hitting
that... but I didn't think our normal CI hit that... I'd submit it today,
but I'm traveling and there is an issue getting to my machines at home, but
I'll be home tomorrow.

Warner


Ideas?
>
> Best regards,
> Christian Schoenebeck
>
>
>
>
Thomas Huth Oct. 28, 2022, 7:07 a.m. UTC | #7
On 27/10/2022 22.42, Christian Schoenebeck wrote:
> On Thursday, October 27, 2022 7:37:07 PM CEST Stefan Hajnoczi wrote:
>> On Thu, 27 Oct 2022 at 12:38, Christian Schoenebeck
>> <qemu_oss@crudebyte.com> wrote:
>>>
>>> On Thursday, October 27, 2022 5:53:47 PM CEST Thomas Huth wrote:
>>>> On 24/10/2022 12.54, Christian Schoenebeck wrote:
>>>>> The following changes since commit 0529245488865038344d64fff7ee05864d3d17f6:
>>>>>
>>>>>     Merge tag 'pull-target-arm-20221020' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2022-10-20 14:36:12 -0400)
>>>>>
>>>>> are available in the Git repository at:
>>>>>
>>>>>     https://github.com/cschoenebeck/qemu.git tags/pull-9p-20221024
>>>>>
>>>>> for you to fetch changes up to 3ce77865bf813f313cf79c00fd951bfc95a50165:
>>>>>
>>>>>     tests/9p: remove unnecessary g_strdup() calls (2022-10-24 12:24:32 +0200)
>>>>>
>>>>> ----------------------------------------------------------------
>>>>> 9pfs: performance, Windows host prep, tests restructure
>>>>>
>>>>> * Highlight of this PR is Linus Heckemann's GHashTable patch which
>>>>>     brings massive general performance improvements of 9p server
>>>>>     somewhere between factor 6 .. 12.
>>>>>
>>>>> * Bin Meng's g_mkdir patch is a preparatory patch for upcoming
>>>>>     Windows host support of 9p server.
>>>>>
>>>>> * The rest of the patches in this PR are 9p test code restructuring
>>>>>     and refactoring changes to improve readability and to ease
>>>>>     maintenance of 9p test code on the long-term.
>>>>
>>>>    Hi Christian,
>>>>
>>>> I think this PR broke the FreeBSD CI jobs:
>>>>
>>>>    https://gitlab.com/qemu-project/qemu/-/jobs/3219611457#L3116
>>>>
>>>>    https://gitlab.com/qemu-project/qemu/-/jobs/3219611460#L3372
>>>>
>>>> Could you please have a look?
>>>>
>>>>    Thanks!
>>>>     Thomas
>>>
>>> I try, but will certainly take some days, especially as I currently don't have
>>> a BSD installation at hand to try the changes.
>>
>> QEMU has the automation to run FreeBSD builds locally (in a VM):
>> $ make vm-build-freebsd
>>
>> Not sure if that FreeBSD environment matches the one in Cirrus CI
>> though. If they are different then maybe it won't reproduce locally.
> 
> Something must be different, because e750a7ace492f0b450653d4ad368a77d6f660fb8
> compiles fine locally with `make vm-build-freebsd` and all tests pass, too.
> 
> Ideas?

You've got to --enable-werror to make it fail, try something like this:

make vm-build-freebsd J=8 TARGET_LIST=x86_64-softmmu \
      BUILD_TARGET=check-build DEBUG=1 \
      EXTRA_CONFIGURE_OPTS="--enable-werror"

  HTH,
   Thomas
Christian Schoenebeck Oct. 28, 2022, 10:40 a.m. UTC | #8
On Friday, October 28, 2022 9:07:17 AM CEST Thomas Huth wrote:
> On 27/10/2022 22.42, Christian Schoenebeck wrote:
> > On Thursday, October 27, 2022 7:37:07 PM CEST Stefan Hajnoczi wrote:
> >> On Thu, 27 Oct 2022 at 12:38, Christian Schoenebeck
> >> <qemu_oss@crudebyte.com> wrote:
> >>>
> >>> On Thursday, October 27, 2022 5:53:47 PM CEST Thomas Huth wrote:
> >>>> On 24/10/2022 12.54, Christian Schoenebeck wrote:
> >>>>> The following changes since commit 0529245488865038344d64fff7ee05864d3d17f6:
> >>>>>
> >>>>>     Merge tag 'pull-target-arm-20221020' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2022-10-20 14:36:12 -0400)
> >>>>>
> >>>>> are available in the Git repository at:
> >>>>>
> >>>>>     https://github.com/cschoenebeck/qemu.git tags/pull-9p-20221024
> >>>>>
> >>>>> for you to fetch changes up to 3ce77865bf813f313cf79c00fd951bfc95a50165:
> >>>>>
> >>>>>     tests/9p: remove unnecessary g_strdup() calls (2022-10-24 12:24:32 +0200)
> >>>>>
> >>>>> ----------------------------------------------------------------
> >>>>> 9pfs: performance, Windows host prep, tests restructure
> >>>>>
> >>>>> * Highlight of this PR is Linus Heckemann's GHashTable patch which
> >>>>>     brings massive general performance improvements of 9p server
> >>>>>     somewhere between factor 6 .. 12.
> >>>>>
> >>>>> * Bin Meng's g_mkdir patch is a preparatory patch for upcoming
> >>>>>     Windows host support of 9p server.
> >>>>>
> >>>>> * The rest of the patches in this PR are 9p test code restructuring
> >>>>>     and refactoring changes to improve readability and to ease
> >>>>>     maintenance of 9p test code on the long-term.
> >>>>
> >>>>    Hi Christian,
> >>>>
> >>>> I think this PR broke the FreeBSD CI jobs:
> >>>>
> >>>>    https://gitlab.com/qemu-project/qemu/-/jobs/3219611457#L3116
> >>>>
> >>>>    https://gitlab.com/qemu-project/qemu/-/jobs/3219611460#L3372
> >>>>
> >>>> Could you please have a look?
> >>>>
> >>>>    Thanks!
> >>>>     Thomas
> >>>
> >>> I try, but will certainly take some days, especially as I currently don't have
> >>> a BSD installation at hand to try the changes.
> >>
> >> QEMU has the automation to run FreeBSD builds locally (in a VM):
> >> $ make vm-build-freebsd
> >>
> >> Not sure if that FreeBSD environment matches the one in Cirrus CI
> >> though. If they are different then maybe it won't reproduce locally.
> > 
> > Something must be different, because e750a7ace492f0b450653d4ad368a77d6f660fb8
> > compiles fine locally with `make vm-build-freebsd` and all tests pass, too.
> > 
> > Ideas?
> 
> You've got to --enable-werror to make it fail, try something like this:
> 
> make vm-build-freebsd J=8 TARGET_LIST=x86_64-softmmu \
>       BUILD_TARGET=check-build DEBUG=1 \
>       EXTRA_CONFIGURE_OPTS="--enable-werror"

That was it, reproduced. Thanks Thomas!

statfs is defined in <sys/mount.h>. I prepare a patch.

Best regards,
Christian Schoenebeck