| Message ID | 20240711115546.40859-8-thuth@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Convert avocado tests to normal Python unittests | expand |
On Thu, Jul 11, 2024 at 01:55:45PM +0200, Thomas Huth wrote: > Some Avocado-based tests use the "archive" module from avocado.utils > to extract files from an archive. To be able to use these tests > without Avocado, we have to provide our own function for extracting > files. Fortunately, there is already the tarfile module that will > provide us with this functionality, so let's just add a nice wrapper > function around that. > > Signed-off-by: Thomas Huth <thuth@redhat.com> > --- > tests/pytest/qemu_pytest/utils.py | 21 +++++++++++++++++++++ > 1 file changed, 21 insertions(+) > create mode 100644 tests/pytest/qemu_pytest/utils.py > > diff --git a/tests/pytest/qemu_pytest/utils.py b/tests/pytest/qemu_pytest/utils.py > new file mode 100644 > index 0000000000..4eb5e5d5e5 > --- /dev/null > +++ b/tests/pytest/qemu_pytest/utils.py > @@ -0,0 +1,21 @@ > +# Utilities for python-based QEMU tests > +# > +# Copyright 2024 Red Hat, Inc. > +# > +# Authors: > +# Thomas Huth <thuth@redhat.com> > +# > +# This work is licensed under the terms of the GNU GPL, version 2 or > +# later. See the COPYING file in the top-level directory. > + > +import tarfile > + > +def archive_extract(archive, dest_dir, member=None): > + with tarfile.open(archive) as tf: > + if hasattr(tarfile, 'data_filter'): Not convinced this is still needed. The python docs don't say anything about 'data_filter' being introduced after 3.0, so can likely assume it always exists. > + tf.extraction_filter = getattr(tarfile, 'data_filter', > + (lambda member, path: member)) > + if member: > + tf.extract(member=member, path=dest_dir) > + else: > + tf.extractall(path=dest_dir) > -- > 2.45.2 > With regards, Daniel
On 12/07/2024 11.14, Daniel P. Berrangé wrote: > On Thu, Jul 11, 2024 at 01:55:45PM +0200, Thomas Huth wrote: >> Some Avocado-based tests use the "archive" module from avocado.utils >> to extract files from an archive. To be able to use these tests >> without Avocado, we have to provide our own function for extracting >> files. Fortunately, there is already the tarfile module that will >> provide us with this functionality, so let's just add a nice wrapper >> function around that. >> >> Signed-off-by: Thomas Huth <thuth@redhat.com> >> --- >> tests/pytest/qemu_pytest/utils.py | 21 +++++++++++++++++++++ >> 1 file changed, 21 insertions(+) >> create mode 100644 tests/pytest/qemu_pytest/utils.py >> >> diff --git a/tests/pytest/qemu_pytest/utils.py b/tests/pytest/qemu_pytest/utils.py >> new file mode 100644 >> index 0000000000..4eb5e5d5e5 >> --- /dev/null >> +++ b/tests/pytest/qemu_pytest/utils.py >> @@ -0,0 +1,21 @@ >> +# Utilities for python-based QEMU tests >> +# >> +# Copyright 2024 Red Hat, Inc. >> +# >> +# Authors: >> +# Thomas Huth <thuth@redhat.com> >> +# >> +# This work is licensed under the terms of the GNU GPL, version 2 or >> +# later. See the COPYING file in the top-level directory. >> + >> +import tarfile >> + >> +def archive_extract(archive, dest_dir, member=None): >> + with tarfile.open(archive) as tf: >> + if hasattr(tarfile, 'data_filter'): > > Not convinced this is still needed. The python docs don't say anything > about 'data_filter' being introduced after 3.0, so can likely > assume it always exists. According to https://docs.python.org/3/library/tarfile.html : "Extraction filters were added to Python 3.12, but may be backported to older versions as security updates. To check whether the feature is available, use e.g. hasattr(tarfile, 'data_filter') rather than checking the Python version." And it seems to be missing in Python 3.7, indeed: https://docs.python.org/3.7/library/tarfile.html So as long as we still support this old version, I think I've got to keep this check. Thomas
On Fri, Jul 12, 2024 at 01:52:03PM +0200, Thomas Huth wrote: > On 12/07/2024 11.14, Daniel P. Berrangé wrote: > > On Thu, Jul 11, 2024 at 01:55:45PM +0200, Thomas Huth wrote: > > > Some Avocado-based tests use the "archive" module from avocado.utils > > > to extract files from an archive. To be able to use these tests > > > without Avocado, we have to provide our own function for extracting > > > files. Fortunately, there is already the tarfile module that will > > > provide us with this functionality, so let's just add a nice wrapper > > > function around that. > > > > > > Signed-off-by: Thomas Huth <thuth@redhat.com> > > > --- > > > tests/pytest/qemu_pytest/utils.py | 21 +++++++++++++++++++++ > > > 1 file changed, 21 insertions(+) > > > create mode 100644 tests/pytest/qemu_pytest/utils.py > > > > > > diff --git a/tests/pytest/qemu_pytest/utils.py b/tests/pytest/qemu_pytest/utils.py > > > new file mode 100644 > > > index 0000000000..4eb5e5d5e5 > > > --- /dev/null > > > +++ b/tests/pytest/qemu_pytest/utils.py > > > @@ -0,0 +1,21 @@ > > > +# Utilities for python-based QEMU tests > > > +# > > > +# Copyright 2024 Red Hat, Inc. > > > +# > > > +# Authors: > > > +# Thomas Huth <thuth@redhat.com> > > > +# > > > +# This work is licensed under the terms of the GNU GPL, version 2 or > > > +# later. See the COPYING file in the top-level directory. > > > + > > > +import tarfile > > > + > > > +def archive_extract(archive, dest_dir, member=None): > > > + with tarfile.open(archive) as tf: > > > + if hasattr(tarfile, 'data_filter'): > > > > Not convinced this is still needed. The python docs don't say anything > > about 'data_filter' being introduced after 3.0, so can likely > > assume it always exists. > > According to https://docs.python.org/3/library/tarfile.html : > > "Extraction filters were added to Python 3.12, but may be backported to > older versions as security updates. To check whether the feature is > available, use e.g. hasattr(tarfile, 'data_filter') rather than checking the > Python version." > > And it seems to be missing in Python 3.7, indeed: > > https://docs.python.org/3.7/library/tarfile.html > > So as long as we still support this old version, I think I've got to keep > this check. Opps, yes, I missed the docs. The version info is against the top heading, not repeated against each method. With regards, Daniel
diff --git a/tests/pytest/qemu_pytest/utils.py b/tests/pytest/qemu_pytest/utils.py new file mode 100644 index 0000000000..4eb5e5d5e5 --- /dev/null +++ b/tests/pytest/qemu_pytest/utils.py @@ -0,0 +1,21 @@ +# Utilities for python-based QEMU tests +# +# Copyright 2024 Red Hat, Inc. +# +# Authors: +# Thomas Huth <thuth@redhat.com> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import tarfile + +def archive_extract(archive, dest_dir, member=None): + with tarfile.open(archive) as tf: + if hasattr(tarfile, 'data_filter'): + tf.extraction_filter = getattr(tarfile, 'data_filter', + (lambda member, path: member)) + if member: + tf.extract(member=member, path=dest_dir) + else: + tf.extractall(path=dest_dir)
Some Avocado-based tests use the "archive" module from avocado.utils to extract files from an archive. To be able to use these tests without Avocado, we have to provide our own function for extracting files. Fortunately, there is already the tarfile module that will provide us with this functionality, so let's just add a nice wrapper function around that. Signed-off-by: Thomas Huth <thuth@redhat.com> --- tests/pytest/qemu_pytest/utils.py | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 tests/pytest/qemu_pytest/utils.py