diff mbox series

[v5,6/8] configure: Enable test and libs for zstd

Message ID 20200129115655.10414-7-quintela@redhat.com (mailing list archive)
State New, archived
Headers show
Series Multifd Migration Compression | expand

Commit Message

Juan Quintela Jan. 29, 2020, 11:56 a.m. UTC 
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 configure | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

Comments

Daniel P. Berrangé Feb. 11, 2020, 8:11 p.m. UTC | #1 
On Wed, Jan 29, 2020 at 12:56:53PM +0100, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
>  configure | 30 ++++++++++++++++++++++++++++++
>  1 file changed, 30 insertions(+)

This is adding a new 3rd party library to QEMU that we've not previously
built and so isn't included in any of our CI platforms.

This commit should be updating at least some of our CI platforms to
request the libzstd library installation to get CI coverage for the
latest patches in this series.  Probably the docker files, the VM
installs for FreeBSD at least, travis and gitlab CI.

Regards,
Daniel
Juan Quintela Feb. 13, 2020, 9:08 p.m. UTC | #2 
Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Wed, Jan 29, 2020 at 12:56:53PM +0100, Juan Quintela wrote:
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
>> ---
>>  configure | 30 ++++++++++++++++++++++++++++++
>>  1 file changed, 30 insertions(+)
>
> This is adding a new 3rd party library to QEMU that we've not previously
> built and so isn't included in any of our CI platforms.

Ok.

Learning how one does that.

> This commit should be updating at least some of our CI platforms to
> request the libzstd library installation to get CI coverage for the
> latest patches in this series.

> Probably the docker files,

I added it in all debian/centos/fedora files that zlib-dev or xen-dev

> the VM installs for FreeBSD at least,

A fast google finds that library is called "zstd" and that it includes
the includes (put intended)

tests/vm/freebsd

Once there, include it in fedora

> travis and

I added it to .travis.yml

> gitlab CI.

gitlab-ci.yml (just when we compile x86_64-softmmu)

I have something like this, but net real clue how to test that I haven't
broken anything:

commit 59d8694dfcfc3d5ef36bc72a5c02bedaa3a6a6ec
Author: Juan Quintela <quintela@redhat.com>
Date:   Thu Feb 13 22:06:16 2020 +0100

    Use zstd packages
    
    Signed-off-by: Juan Quintela <quintela@redhat.com>

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index c15e394f09..72f8b8aa51 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -21,6 +21,7 @@ build-system2:
  script:
  - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
       libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
+      libzstd-dev
  - mkdir build
  - cd build
  - ../configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
diff --git a/.travis.yml b/.travis.yml
index 5887055951..dd17301f3b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -48,6 +48,7 @@ addons:
       - libusb-1.0-0-dev
       - libvdeplug-dev
       - libvte-2.91-dev
+      - libzstd-dev
       - sparse
       - uuid-dev
       - gcovr
diff --git a/tests/docker/dockerfiles/centos7.docker b/tests/docker/dockerfiles/centos7.docker
index 562d65be9e..cdd72de7eb 100644
--- a/tests/docker/dockerfiles/centos7.docker
+++ b/tests/docker/dockerfiles/centos7.docker
@@ -33,6 +33,7 @@ ENV PACKAGES \
     tar \
     vte-devel \
     xen-devel \
-    zlib-devel
+    zlib-devel \
+    libzstd-devel
 RUN yum install -y $PACKAGES
 RUN rpm -q $PACKAGES | sort > /packages.txt
diff --git a/tests/docker/dockerfiles/fedora-i386-cross.docker b/tests/docker/dockerfiles/fedora-i386-cross.docker
index 9106cf9ebe..cd16cd1bfa 100644
--- a/tests/docker/dockerfiles/fedora-i386-cross.docker
+++ b/tests/docker/dockerfiles/fedora-i386-cross.docker
@@ -7,7 +7,8 @@ ENV PACKAGES \
     gnutls-devel.i686 \
     nettle-devel.i686 \
     pixman-devel.i686 \
-    zlib-devel.i686
+    zlib-devel.i686 \
+    libzstd-devel.i686
 
 RUN dnf install -y $PACKAGES
 RUN rpm -q $PACKAGES | sort > /packages.txt
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 987a3c170a..a6522228c0 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -92,7 +92,8 @@ ENV PACKAGES \
     vte291-devel \
     which \
     xen-devel \
-    zlib-devel
+    zlib-devel \
+    libzstd-devel
 ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3
 
 RUN dnf install -y $PACKAGES
diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index 4177f33691..b6c7b41ddd 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -58,6 +58,7 @@ ENV PACKAGES flex bison \
     libvdeplug-dev \
     libvte-2.91-dev \
     libxen-dev \
+    libzstd-dev \
     make \
     python3-yaml \
     python3-sphinx \
diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
index 0766f94cf4..1efedeef99 100644
--- a/tests/docker/dockerfiles/ubuntu1804.docker
+++ b/tests/docker/dockerfiles/ubuntu1804.docker
@@ -44,6 +44,7 @@ ENV PACKAGES flex bison \
     libvdeplug-dev \
     libvte-2.91-dev \
     libxen-dev \
+    libzstd-dev \
     make \
     python3-yaml \
     python3-sphinx \
diff --git a/tests/vm/fedora b/tests/vm/fedora
index 4d7d6049f4..4843b4175e 100755
--- a/tests/vm/fedora
+++ b/tests/vm/fedora
@@ -53,7 +53,10 @@ class FedoraVM(basevm.BaseVM):
         # libs: audio
         '"pkgconfig(libpulse)"',
         '"pkgconfig(alsa)"',
-    ]
+
+        # libs: migration
+        '"pkgconfig(libzstd)"',
+]
 
     BUILD_SCRIPT = """
         set -e;
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index fb54334696..86770878b6 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -55,6 +55,9 @@ class FreeBSDVM(basevm.BaseVM):
         # libs: opengl
         "libepoxy",
         "mesa-libs",
+
+        # libs: migration
+        "zstd",
     ]
 
     BUILD_SCRIPT = """
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index c5069a45f4..55590f4601 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -49,6 +49,9 @@ class NetBSDVM(basevm.BaseVM):
         "SDL2",
         "gtk3+",
         "libxkbcommon",
+
+        # libs: migration
+        "zstd",
     ]
 
     BUILD_SCRIPT = """
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 22cd9513dd..ab6abbedab 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -51,6 +51,9 @@ class OpenBSDVM(basevm.BaseVM):
         "sdl2",
         "gtk+3",
         "libxkbcommon",
+
+        # libs: migration
+        "zstd",
     ]
 
     BUILD_SCRIPT = """
Daniel P. Berrangé Feb. 14, 2020, 10:26 a.m. UTC | #3 
On Thu, Feb 13, 2020 at 10:08:59PM +0100, Juan Quintela wrote:
> Daniel P. Berrangé <berrange@redhat.com> wrote:
> > On Wed, Jan 29, 2020 at 12:56:53PM +0100, Juan Quintela wrote:
> >> Signed-off-by: Juan Quintela <quintela@redhat.com>
> >> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> >> ---
> >>  configure | 30 ++++++++++++++++++++++++++++++
> >>  1 file changed, 30 insertions(+)
> >
> > This is adding a new 3rd party library to QEMU that we've not previously
> > built and so isn't included in any of our CI platforms.
> 
> Ok.
> 
> Learning how one does that.
> 
> > This commit should be updating at least some of our CI platforms to
> > request the libzstd library installation to get CI coverage for the
> > latest patches in this series.
> 
> > Probably the docker files,
> 
> I added it in all debian/centos/fedora files that zlib-dev or xen-dev
> 
> > the VM installs for FreeBSD at least,
> 
> A fast google finds that library is called "zstd" and that it includes
> the includes (put intended)
> 
> tests/vm/freebsd
> 
> Once there, include it in fedora
> 
> > travis and
> 
> I added it to .travis.yml
> 
> > gitlab CI.
> 
> gitlab-ci.yml (just when we compile x86_64-softmmu)
> 
> I have something like this, but net real clue how to test that I haven't
> broken anything:
> 
> commit 59d8694dfcfc3d5ef36bc72a5c02bedaa3a6a6ec
> Author: Juan Quintela <quintela@redhat.com>
> Date:   Thu Feb 13 22:06:16 2020 +0100
> 
>     Use zstd packages
>     
>     Signed-off-by: Juan Quintela <quintela@redhat.com>
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index c15e394f09..72f8b8aa51 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -21,6 +21,7 @@ build-system2:
>   script:
>   - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
>        libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
> +      libzstd-dev
>   - mkdir build
>   - cd build
>   - ../configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
> diff --git a/.travis.yml b/.travis.yml
> index 5887055951..dd17301f3b 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -48,6 +48,7 @@ addons:
>        - libusb-1.0-0-dev
>        - libvdeplug-dev
>        - libvte-2.91-dev
> +      - libzstd-dev
>        - sparse
>        - uuid-dev
>        - gcovr
> diff --git a/tests/docker/dockerfiles/centos7.docker b/tests/docker/dockerfiles/centos7.docker
> index 562d65be9e..cdd72de7eb 100644
> --- a/tests/docker/dockerfiles/centos7.docker
> +++ b/tests/docker/dockerfiles/centos7.docker
> @@ -33,6 +33,7 @@ ENV PACKAGES \
>      tar \
>      vte-devel \
>      xen-devel \
> -    zlib-devel
> +    zlib-devel \
> +    libzstd-devel
>  RUN yum install -y $PACKAGES
>  RUN rpm -q $PACKAGES | sort > /packages.txt
> diff --git a/tests/docker/dockerfiles/fedora-i386-cross.docker b/tests/docker/dockerfiles/fedora-i386-cross.docker
> index 9106cf9ebe..cd16cd1bfa 100644
> --- a/tests/docker/dockerfiles/fedora-i386-cross.docker
> +++ b/tests/docker/dockerfiles/fedora-i386-cross.docker
> @@ -7,7 +7,8 @@ ENV PACKAGES \
>      gnutls-devel.i686 \
>      nettle-devel.i686 \
>      pixman-devel.i686 \
> -    zlib-devel.i686
> +    zlib-devel.i686 \
> +    libzstd-devel.i686
>  
>  RUN dnf install -y $PACKAGES
>  RUN rpm -q $PACKAGES | sort > /packages.txt
> diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
> index 987a3c170a..a6522228c0 100644
> --- a/tests/docker/dockerfiles/fedora.docker
> +++ b/tests/docker/dockerfiles/fedora.docker
> @@ -92,7 +92,8 @@ ENV PACKAGES \
>      vte291-devel \
>      which \
>      xen-devel \
> -    zlib-devel
> +    zlib-devel \
> +    libzstd-devel
>  ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3
>  
>  RUN dnf install -y $PACKAGES
> diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
> index 4177f33691..b6c7b41ddd 100644
> --- a/tests/docker/dockerfiles/ubuntu.docker
> +++ b/tests/docker/dockerfiles/ubuntu.docker
> @@ -58,6 +58,7 @@ ENV PACKAGES flex bison \
>      libvdeplug-dev \
>      libvte-2.91-dev \
>      libxen-dev \
> +    libzstd-dev \
>      make \
>      python3-yaml \
>      python3-sphinx \
> diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
> index 0766f94cf4..1efedeef99 100644
> --- a/tests/docker/dockerfiles/ubuntu1804.docker
> +++ b/tests/docker/dockerfiles/ubuntu1804.docker
> @@ -44,6 +44,7 @@ ENV PACKAGES flex bison \
>      libvdeplug-dev \
>      libvte-2.91-dev \
>      libxen-dev \
> +    libzstd-dev \
>      make \
>      python3-yaml \
>      python3-sphinx \
> diff --git a/tests/vm/fedora b/tests/vm/fedora
> index 4d7d6049f4..4843b4175e 100755
> --- a/tests/vm/fedora
> +++ b/tests/vm/fedora
> @@ -53,7 +53,10 @@ class FedoraVM(basevm.BaseVM):
>          # libs: audio
>          '"pkgconfig(libpulse)"',
>          '"pkgconfig(alsa)"',
> -    ]
> +
> +        # libs: migration
> +        '"pkgconfig(libzstd)"',
> +]
>  
>      BUILD_SCRIPT = """
>          set -e;
> diff --git a/tests/vm/freebsd b/tests/vm/freebsd
> index fb54334696..86770878b6 100755
> --- a/tests/vm/freebsd
> +++ b/tests/vm/freebsd
> @@ -55,6 +55,9 @@ class FreeBSDVM(basevm.BaseVM):
>          # libs: opengl
>          "libepoxy",
>          "mesa-libs",
> +
> +        # libs: migration
> +        "zstd",
>      ]
>  
>      BUILD_SCRIPT = """
> diff --git a/tests/vm/netbsd b/tests/vm/netbsd
> index c5069a45f4..55590f4601 100755
> --- a/tests/vm/netbsd
> +++ b/tests/vm/netbsd
> @@ -49,6 +49,9 @@ class NetBSDVM(basevm.BaseVM):
>          "SDL2",
>          "gtk3+",
>          "libxkbcommon",
> +
> +        # libs: migration
> +        "zstd",
>      ]
>  
>      BUILD_SCRIPT = """
> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> index 22cd9513dd..ab6abbedab 100755
> --- a/tests/vm/openbsd
> +++ b/tests/vm/openbsd
> @@ -51,6 +51,9 @@ class OpenBSDVM(basevm.BaseVM):
>          "sdl2",
>          "gtk+3",
>          "libxkbcommon",
> +
> +        # libs: migration
> +        "zstd",
>      ]
>  
>      BUILD_SCRIPT = """

This looks good to me, thanks.

Regards,
Daniel
diff mbox series

Patch

diff --git a/configure b/configure
index a72a5def57..7c1dca326f 100755
--- a/configure
+++ b/configure
@@ -448,6 +448,7 @@  lzo=""
 snappy=""
 bzip2=""
 lzfse=""
+zstd=""
 guest_agent=""
 guest_agent_with_vss="no"
 guest_agent_ntddscsi="no"
@@ -1343,6 +1344,10 @@  for opt do
   ;;
   --disable-lzfse) lzfse="no"
   ;;
+  --disable-zstd) zstd="no"
+  ;;
+  --enable-zstd) zstd="yes"
+  ;;
   --enable-guest-agent) guest_agent="yes"
   ;;
   --disable-guest-agent) guest_agent="no"
@@ -1795,6 +1800,8 @@  disabled with --disable-FEATURE, default is enabled if available:
                   (for reading bzip2-compressed dmg images)
   lzfse           support of lzfse compression library
                   (for reading lzfse-compressed dmg images)
+  zstd            support for zstd compression library
+                  (for migration compression)
   seccomp         seccomp support
   coroutine-pool  coroutine freelist (better performance)
   glusterfs       GlusterFS backend
@@ -2409,6 +2416,24 @@  EOF
     fi
 fi
 
+##########################################
+# zstd check
+
+if test "$zstd" != "no" ; then
+    if $pkg_config --exist libzstd ; then
+        zstd_cflags="$($pkg_config --cflags libzstd)"
+        zstd_libs="$($pkg_config --libs libzstd)"
+        LIBS="$zstd_libs $LIBS"
+        QEMU_CFLAGS="$QEMU_CFLAGS $zstd_cflags"
+        zstd="yes"
+    else
+        if test "$zstd" = "yes" ; then
+            feature_not_found "libzstd" "Install libzstd devel"
+        fi
+        zstd="no"
+    fi
+fi
+
 ##########################################
 # libseccomp check
 
@@ -6578,6 +6603,7 @@  echo "lzo support       $lzo"
 echo "snappy support    $snappy"
 echo "bzip2 support     $bzip2"
 echo "lzfse support     $lzfse"
+echo "zstd support      $zstd"
 echo "NUMA host support $numa"
 echo "libxml2           $libxml2"
 echo "tcmalloc support  $tcmalloc"
@@ -7143,6 +7169,10 @@  if test "$lzfse" = "yes" ; then
   echo "LZFSE_LIBS=-llzfse" >> $config_host_mak
 fi
 
+if test "$zstd" = "yes" ; then
+  echo "CONFIG_ZSTD=y" >> $config_host_mak
+fi
+
 if test "$libiscsi" = "yes" ; then
   echo "CONFIG_LIBISCSI=m" >> $config_host_mak
   echo "LIBISCSI_CFLAGS=$libiscsi_cflags" >> $config_host_mak