| Message ID | 20230328211119.2748442-1-jsnow@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | configure: create a python venv and install meson | expand |
Thanks for posting this! On 3/28/23 23:11, John Snow wrote: > This series is in RFC state; some notable things: > - So far, only meson is included in this setup. > - There's a ton of debugging prints everywhere. It's extremely chatty right now. > - Sphinx and qemu.qmp are not yet included in this venv. > (qemu.qmp isn't able to be removed from the tree yet.) > - Testing isn't yet*fully* switched over. > - There is no online functionality yet, this series is *100% offline* - > it's the harder option, so I tackled it first. > > Some known bugs as of now: > - venv-in-venv setups are not yet handled. > - python3.7 setups without setuptools/pip in the host environment may be > unable to generate script file shims; a workaround is in development > but wasn't ready today. I decided to exclude it. I don't think that's needed at all---requiring setuptools on those older Python versions is good enough and can be mentioned in either an error message or the release notes. But out of curiosity what was the workaround? I think the steps for the first working version should be: * fix venv-in-venv using the .pth trick * undo the meson parts from PATCH 3; make patch 3 create the venv + subsume the MKVENV parts of the Makefiles + always set explicit_python=yes (so that at this point the in-tree meson is always used). * add a patch that starts rejecting --meson=/path/to/meson and drops explicit_python (instead using pyvenv/bin/meson to check whether a system meson is usable) * make Meson use a sphinx-build binary from the virtual environment (i.e. pass -Dsphinx_build=$PWD/pyvenv/bin/sphinx-build) These are the bare minimum needed to drop Python 3.6 support in QEMU 8.1. Paolo
On Wed, Mar 29, 2023, 9:02 AM Paolo Bonzini <pbonzini@redhat.com> wrote: > Thanks for posting this! > > On 3/28/23 23:11, John Snow wrote: > > This series is in RFC state; some notable things: > > - So far, only meson is included in this setup. > > - There's a ton of debugging prints everywhere. It's extremely chatty > right now. > > - Sphinx and qemu.qmp are not yet included in this venv. > > (qemu.qmp isn't able to be removed from the tree yet.) > > - Testing isn't yet*fully* switched over. > > - There is no online functionality yet, this series is *100% offline* - > > it's the harder option, so I tackled it first. > > > > Some known bugs as of now: > > - venv-in-venv setups are not yet handled. > > - python3.7 setups without setuptools/pip in the host environment may be > > unable to generate script file shims; a workaround is in development > > but wasn't ready today. I decided to exclude it. > > I don't think that's needed at all---requiring setuptools on those older > Python versions is good enough and can be mentioned in either an error > message or the release notes. But out of curiosity what was the > workaround? > Run the shim generation inside the venv, alongside checkpip. That way we know we have setuptools. (It broke for some other reason I didn't diagnose in time to send.) > I think the steps for the first working version should be: > > * fix venv-in-venv using the .pth trick > Yep. > * undo the meson parts from PATCH 3; make patch 3 create the venv + > subsume the MKVENV parts of the Makefiles + always set > explicit_python=yes (so that at this point the in-tree meson is always > used). > > * add a patch that starts rejecting --meson=/path/to/meson and drops > explicit_python (instead using pyvenv/bin/meson to check whether a > system meson is usable) > > * make Meson use a sphinx-build binary from the virtual environment > (i.e. pass -Dsphinx_build=$PWD/pyvenv/bin/sphinx-build) > Yep, let's talk about this part in particular. > These are the bare minimum needed to drop Python 3.6 support in QEMU 8.1. > > Paolo > >
On 3/30/23 16:11, John Snow wrote: > * undo the meson parts from PATCH 3; make patch 3 create the venv + > subsume the MKVENV parts of the Makefiles + always set > explicit_python=yes (so that at this point the in-tree meson is always > used). > > * add a patch that starts rejecting --meson=/path/to/meson and drops > explicit_python (instead using pyvenv/bin/meson to check whether a > system meson is usable) > > * make Meson use a sphinx-build binary from the virtual environment > (i.e. pass -Dsphinx_build=$PWD/pyvenv/bin/sphinx-build) > > > Yep, let's talk about this part in particular. Oh, wait, for this one I already have a patch from my experiment that used importlib.metadata to look up the entry point dynamically[1] (and that's where the shim idea developed from). All I need to do is change the path passed to find_program() and rewrite the commit message. Paolo [1] https://lore.kernel.org/qemu-devel/2c63f79d-b46d-841b-bed3-0dca33eab2c0@redhat.com/ --------------------------- 8< -------------------------------- From: Paolo Bonzini <pbonzini@redhat.com> Subject: [PATCH] meson: pick sphinx-build from virtual environment configure is now creating a virtual environment and populating it with shim binaries that always refer to the correct Python runtime. docs/meson.build can rely on this, and stop using a sphinx_build option that may or may not refer to the same version of Python that is used for the rest of the build. In the long term, it may actually make sense for Meson's Python module to include the logic to build such shims, so that other programs can do the same without needing a full-blown virtual environment. However, in the context of QEMU there is no need to wait for that; QEMU's meson.build already relies on config-host.mak and on the target list that configure prepares, i.e. it is not standalone. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> diff --git a/docs/conf.py b/docs/conf.py index 7e215aa9a5c6..c687ff266301 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -32,15 +32,6 @@ from distutils.version import LooseVersion from sphinx.errors import ConfigError -# Make Sphinx fail cleanly if using an old Python, rather than obscurely -# failing because some code in one of our extensions doesn't work there. -# In newer versions of Sphinx this will display nicely; in older versions -# Sphinx will also produce a Python backtrace but at least the information -# gets printed... -if sys.version_info < (3,7): - raise ConfigError( - "QEMU requires a Sphinx that uses Python 3.7 or better\n") - # The per-manual conf.py will set qemu_docdir for a single-manual build; # otherwise set it here if this is an entire-manual-set build. # This is always the absolute path of the docs/ directory in the source tree. diff --git a/docs/meson.build b/docs/meson.build index f220800e3e59..1c5fd66bfa7f 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -1,5 +1,6 @@ -sphinx_build = find_program(get_option('sphinx_build'), # This assumes that Python is inside the venv that configure prepares +sphinx_build = find_program(fs.parent(python.full_path()) / 'sphinx-build', required: get_option('docs')) # Check if tools are available to build documentation. diff --git a/meson_options.txt b/meson_options.txt index b541ab2851dd..8dedec0cf91a 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -12,8 +12,6 @@ option('pkgversion', type : 'string', value : '', description: 'use specified string as sub-version of the package') option('smbd', type : 'string', value : '', description: 'Path to smbd for slirp networking') -option('sphinx_build', type : 'string', value : 'sphinx-build', - description: 'Use specified sphinx-build for building document') option('iasl', type : 'string', value : '', description: 'Path to ACPI disassembler') option('tls_priority', type : 'string', value : 'NORMAL', diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index bf852f4b957e..6a71c3bad296 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -58,8 +58,6 @@ meson_options_help() { printf "%s\n" ' --localedir=VALUE Locale data directory [share/locale]' printf "%s\n" ' --localstatedir=VALUE Localstate data directory [/var/local]' printf "%s\n" ' --mandir=VALUE Manual page directory [share/man]' - printf "%s\n" ' --sphinx-build=VALUE Use specified sphinx-build for building document' - printf "%s\n" ' [sphinx-build]' printf "%s\n" ' --sysconfdir=VALUE Sysconf data directory [etc]' printf "%s\n" ' --tls-priority=VALUE Default TLS protocol/cipher priority string' printf "%s\n" ' [NORMAL]' @@ -429,7 +427,6 @@ _meson_option_parse() { --disable-sndio) printf "%s" -Dsndio=disabled ;; --enable-sparse) printf "%s" -Dsparse=enabled ;; --disable-sparse) printf "%s" -Dsparse=disabled ;; - --sphinx-build=*) quote_sh "-Dsphinx_build=$2" ;; --enable-spice) printf "%s" -Dspice=enabled ;; --disable-spice) printf "%s" -Dspice=disabled ;; --enable-spice-protocol) printf "%s" -Dspice_protocol=enabled ;;