diff mbox series

python 3.5 compatibility

Message ID 20201127183619.8958-1-info@metux.net (mailing list archive)
State New, archived
Headers show
Series python 3.5 compatibility | expand

Commit Message

Enrico Weigelt, metux IT consult Nov. 27, 2020, 6:36 p.m. UTC
Several stable distros are still on python-3.5, so qemu cannot be
built there, w/o backporting the whole python stack. In order to
make qemu build there, make it running w/ python-3.5. The changes
are mostly removing the new type annotations (which, IMHO, just
serve for documentation purpose) and replacing f-strings by classic
string formatting.

Signed-off-by: Enrico Weigelt, metux IT consult <info@metux.net>
---
 configure                          |  4 +--
 scripts/block-coroutine-wrapper.py | 56 ++++++++++++++++++--------------------
 scripts/qapi/commands.py           |  2 +-
 scripts/qapi/common.py             |  2 +-
 scripts/qapi/events.py             |  2 +-
 scripts/qapi/gen.py                | 10 +++----
 scripts/qapi/main.py               |  6 ++--
 scripts/qapi/schema.py             |  2 +-
 scripts/qapi/source.py             | 10 +++----
 9 files changed, 46 insertions(+), 48 deletions(-)

Comments

Peter Maydell Nov. 27, 2020, 7:15 p.m. UTC | #1
On Fri, 27 Nov 2020 at 19:12, Enrico Weigelt, metux IT consult
<info@metux.net> wrote:
> Several stable distros are still on python-3.5, so qemu cannot be
> built there, w/o backporting the whole python stack. In order to
> make qemu build there, make it running w/ python-3.5. The changes
> are mostly removing the new type annotations (which, IMHO, just
> serve for documentation purpose) and replacing f-strings by classic
> string formatting.

We dropped Python 3.5 support deliberately in commit
1b11f28d05121fec43 on the basis that all our supported build
platforms are on at least 3.6 and there were features we wanted there.

Could you say which "stable distros" you have in mind, and whether
they are covered by our "supported build platforms" policy
https://www.qemu.org/docs/master/system/build-platforms.html  ?

thanks
-- PMM
Kevin Wolf Nov. 30, 2020, 9:44 a.m. UTC | #2
Am 27.11.2020 um 19:36 hat Enrico Weigelt, metux IT consult geschrieben:
> Several stable distros are still on python-3.5, so qemu cannot be
> built there, w/o backporting the whole python stack. In order to
> make qemu build there, make it running w/ python-3.5. The changes
> are mostly removing the new type annotations (which, IMHO, just
> serve for documentation purpose)

While type hints are valuable documentation, they are more than just
that. They help to find and prevent avoidable bugs in the code. We are
actively in the process of adding them to everything in the QAPI
generator to improve maintainability rather than removing them.

Kevin

> and replacing f-strings by classic
> string formatting.
> 
> Signed-off-by: Enrico Weigelt, metux IT consult <info@metux.net>
> ---
>  configure                          |  4 +--
>  scripts/block-coroutine-wrapper.py | 56 ++++++++++++++++++--------------------
>  scripts/qapi/commands.py           |  2 +-
>  scripts/qapi/common.py             |  2 +-
>  scripts/qapi/events.py             |  2 +-
>  scripts/qapi/gen.py                | 10 +++----
>  scripts/qapi/main.py               |  6 ++--
>  scripts/qapi/schema.py             |  2 +-
>  scripts/qapi/source.py             | 10 +++----
>  9 files changed, 46 insertions(+), 48 deletions(-)
Enrico Weigelt, metux IT consult Nov. 30, 2020, 6:27 p.m. UTC | #3
On 27.11.20 20:15, Peter Maydell wrote:

Hi,

> Could you say which "stable distros" you have in mind, and whether
> they are covered by our "supported build platforms" policy
> https://www.qemu.org/docs/master/system/build-platforms.html  ?

I'm running on Devuan Ascii.

And packaging python-3.6 just for a few pieces of one application (qemu)
is far too much work.

--mtx
Enrico Weigelt, metux IT consult Nov. 30, 2020, 6:30 p.m. UTC | #4
On 30.11.20 10:44, Kevin Wolf wrote:

Hi,

> While type hints are valuable documentation, they are more than just
> that. They help to find and prevent avoidable bugs in the code. We are
> actively in the process of adding them to everything in the QAPI
> generator to improve maintainability rather than removing them.

IOW: I'll have to do lots of backporting work, just to keep it running :(


--mtx
Markus Armbruster Dec. 1, 2020, 6:33 a.m. UTC | #5
"Enrico Weigelt, metux IT consult" <lkml@metux.net> writes:

> On 27.11.20 20:15, Peter Maydell wrote:
>
> Hi,
>
>> Could you say which "stable distros" you have in mind, and whether
>> they are covered by our "supported build platforms" policy
>> https://www.qemu.org/docs/master/system/build-platforms.html  ?
>
> I'm running on Devuan Ascii.

Which has oldstable status.  Good for running the old and stable
software packaged by it (such as QEMU 2.8), and old (and hopefully
stable) software of similar vintage.

Devian ASCII appears to be a derivative of Debian 9 Stretch, which we no
longer support, because it's EOL.  docs/system/build-platforms.rst:

    For distributions with long-lifetime releases, the project will aim to
    support the most recent major version at all times. Support for the
    previous major version will be dropped 2 years after the new major
    version is released, or when it reaches "end of life". For the purposes
    of identifying supported software versions, the project will look at
    RHEL, Debian, Ubuntu LTS, and SLES distros. Other long-lifetime distros
    will be assumed to ship similar software versions.

This policy balances cost and benefit of keeping bleeding-edge QEMU work
on a wide variety of hosts.  Years of balancing.

> And packaging python-3.6 just for a few pieces of one application (qemu)
> is far too much work.

Have you considered upgrading to stable?
Marc-André Lureau Dec. 1, 2020, 7:32 a.m. UTC | #6
Hi

On Mon, Nov 30, 2020 at 10:29 PM Enrico Weigelt, metux IT consult <
lkml@metux.net> wrote:

> On 27.11.20 20:15, Peter Maydell wrote:
>
> Hi,
>
> > Could you say which "stable distros" you have in mind, and whether
> > they are covered by our "supported build platforms" policy
> > https://www.qemu.org/docs/master/system/build-platforms.html  ?
>
> I'm running on Devuan Ascii.
>
> And packaging python-3.6 just for a few pieces of one application (qemu)
> is far too much work.
>

It might be necessary to keep that distro alive, as python 3.5 is EOL:

https://www.python.org/downloads/release/python-3510/

(fwiw, upcoming meson version will also depend on >= 3.6)


> --mtx
>
> --
> ---
> Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
> werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
> GPG/PGP-Schlüssel zu.
> ---
> Enrico Weigelt, metux IT consult
> Free software and Linux embedded engineering
> info@metux.net -- +49-151-27565287
>
>
Enrico Weigelt, metux IT consult Dec. 1, 2020, 3:59 p.m. UTC | #7
On 01.12.20 07:33, Markus Armbruster wrote:

> Which has oldstable status.  Good for running the old and stable
> software packaged by it (such as QEMU 2.8), and old (and hopefully
> stable) software of similar vintage.

It's still heavily used out in the field, and officially supported.
But that's just one example.

Perhaps I should add some more details on the situation: I'm using
(specially built) qemu with ptxdist/dkit - not the distro package.
The idea w/ ptxdist is that you just pull the trigger and it builds
everything needed for some project. Qemu is built specifically for
the configured target. Perhaps you've noticed I'm also doing development
of new qemu features - something that one doesn't want to do on old
versions.

> Have you considered upgrading to stable?

This would solve the problem just for me alone, not for others out
there, who're working w/ the BSP. And asking everybody (especially in
enterprise environments) to do a full release upgrade just for one
single tool (qemu) isn't someting that works easily.

If you insist in having python3.6 a hard requirement for qemu, you're
putting me into the situation of having to do lots of backport work
for quite long time (until everybody really did the upgrade). :(

This individual patch wasn't very complicated, but expect even more
(and more tricky cases) on the horizon.


--mtx
Daniel P. Berrangé Dec. 1, 2020, 4:39 p.m. UTC | #8
On Tue, Dec 01, 2020 at 04:59:23PM +0100, Enrico Weigelt, metux IT consult wrote:
> On 01.12.20 07:33, Markus Armbruster wrote:
> 
> > Which has oldstable status.  Good for running the old and stable
> > software packaged by it (such as QEMU 2.8), and old (and hopefully
> > stable) software of similar vintage.
> 
> It's still heavily used out in the field, and officially supported.
> But that's just one example.
> 
> Perhaps I should add some more details on the situation: I'm using
> (specially built) qemu with ptxdist/dkit - not the distro package.
> The idea w/ ptxdist is that you just pull the trigger and it builds
> everything needed for some project. Qemu is built specifically for
> the configured target. Perhaps you've noticed I'm also doing development
> of new qemu features - something that one doesn't want to do on old
> versions.
> 
> > Have you considered upgrading to stable?
> 
> This would solve the problem just for me alone, not for others out
> there, who're working w/ the BSP. And asking everybody (especially in
> enterprise environments) to do a full release upgrade just for one
> single tool (qemu) isn't someting that works easily.
> 
> If you insist in having python3.6 a hard requirement for qemu, you're
> putting me into the situation of having to do lots of backport work
> for quite long time (until everybody really did the upgrade). :(

python3.6 is just the one problem you've currently hit. In order to keep
the burden of maintaining support for old software under control we have
a well defined set of platforms that we target. When a particular version
of a distro drops off our list, we will bump the minimum versions of any
software we depend on. In this case you've hit python3.6 so far, but we
are liable to bump other minimum versions too which will also impact
this old distro. So I'm afraid it will be a loosing battle to stay on
this old distro, while building cutting edge QEMU.

I appreciate this isn't the answer you want to hear, but we've defined
our support platform matrix to try to balance multiple competing factors
and unfortunately this means distros are going to periodically get dropped
by new QEMU as they get older.

One possible way you can mitigate this is to make use of containers for
your development and deployment. eg even tough you're using an old
Devuan, you can use docker/podman to build and deploy QEMU inside a
stable Devuan container.

Regards,
Daniel
diff mbox series

Patch

diff --git a/configure b/configure
index 18c26e0389..e1119929fa 100755
--- a/configure
+++ b/configure
@@ -1866,8 +1866,8 @@  fi
 
 # Note that if the Python conditional here evaluates True we will exit
 # with status 1 which is a shell 'false' value.
-if ! $python -c 'import sys; sys.exit(sys.version_info < (3,6))'; then
-  error_exit "Cannot use '$python', Python >= 3.6 is required." \
+if ! $python -c 'import sys; sys.exit(sys.version_info < (3,5))'; then
+  error_exit "Cannot use '$python', Python >= 3.5 is required." \
       "Use --python=/path/to/python to specify a supported Python."
 fi
 
diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py
index 0461fd1c45..a36f7af06f 100644
--- a/scripts/block-coroutine-wrapper.py
+++ b/scripts/block-coroutine-wrapper.py
@@ -32,11 +32,11 @@  def gen_header():
     copyright = re.sub('^.*Copyright', 'Copyright', __doc__, flags=re.DOTALL)
     copyright = re.sub('^(?=.)', ' * ', copyright.strip(), flags=re.MULTILINE)
     copyright = re.sub('^$', ' *', copyright, flags=re.MULTILINE)
-    return f"""\
+    return """\
 /*
  * File is generated by scripts/block-coroutine-wrapper.py
  *
-{copyright}
+"""+copyright+"""
  */
 
 #include "qemu/osdep.h"
@@ -55,7 +55,7 @@  class ParamDecl:
     def __init__(self, param_decl: str) -> None:
         m = self.param_re.match(param_decl.strip())
         if m is None:
-            raise ValueError(f'Wrong parameter declaration: "{param_decl}"')
+            raise ValueError('Wrong parameter declaration: '+param_decl)
         self.decl = m.group('decl')
         self.type = m.group('type')
         self.name = m.group('name')
@@ -107,44 +107,42 @@  def gen_wrapper(func: FuncDecl) -> str:
     bs = 'bs' if func.args[0].type == 'BlockDriverState *' else 'child->bs'
     struct_name = snake_to_camel(name)
 
-    return f"""\
+    return """\
 /*
- * Wrappers for {name}
+ * Wrappers for """+name+"""
  */
 
-typedef struct {struct_name} {{
+typedef struct """+struct_name+""" {
     BdrvPollCo poll_state;
-{ func.gen_block('    {decl};') }
-}} {struct_name};
+"""+func.gen_block('    {decl};')+"""
+} """+struct_name+""";
 
-static void coroutine_fn {name}_entry(void *opaque)
-{{
-    {struct_name} *s = opaque;
+static void coroutine_fn """+name+"""_entry(void *opaque)
+{
+    """+struct_name+""" *s = opaque;
 
-    s->poll_state.ret = {name}({ func.gen_list('s->{name}') });
+    s->poll_state.ret = """+name+"""("""+func.gen_list('s->{name}')+""");
     s->poll_state.in_progress = false;
 
     aio_wait_kick();
-}}
-
-int {func.name}({ func.gen_list('{decl}') })
-{{
-    if (qemu_in_coroutine()) {{
-        return {name}({ func.gen_list('{name}') });
-    }} else {{
-        {struct_name} s = {{
-            .poll_state.bs = {bs},
+}
+
+int """+func.name+"""("""+func.gen_list('{decl}')+""")
+{
+    if (qemu_in_coroutine()) {
+        return """+name+"""("""+func.gen_list('{name}')+""");
+    } else {
+        """+struct_name+""" s = {
+            .poll_state.bs = """+bs+""",
             .poll_state.in_progress = true,
+"""+func.gen_block('            .{name} = {name},')+"""
+        };
 
-{ func.gen_block('            .{name} = {name},') }
-        }};
-
-        s.poll_state.co = qemu_coroutine_create({name}_entry, &s);
+        s.poll_state.co = qemu_coroutine_create("""+name+"""_entry, &s);
 
         return bdrv_poll_co(&s.poll_state);
-    }}
-}}"""
-
+    }
+}"""
 
 def gen_wrappers(input_code: str) -> str:
     res = ''
@@ -157,7 +155,7 @@  def gen_wrappers(input_code: str) -> str:
 
 if __name__ == '__main__':
     if len(sys.argv) < 3:
-        exit(f'Usage: {sys.argv[0]} OUT_FILE.c IN_FILE.[ch]...')
+        exit('Usage: '+sys.argv[0]+' OUT_FILE.c IN_FILE.[ch]...')
 
     with open(sys.argv[1], 'w', encoding='utf-8') as f_out:
         f_out.write(gen_header())
diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
index 50978090b4..3b22a16759 100644
--- a/scripts/qapi/commands.py
+++ b/scripts/qapi/commands.py
@@ -256,7 +256,7 @@  class QAPISchemaGenCommandVisitor(QAPISchemaModularCVisitor):
             prefix, 'qapi-commands',
             ' * Schema-defined QAPI/QMP commands', None, __doc__)
         self._regy = QAPIGenCCode(None)
-        self._visited_ret_types: Dict[QAPIGenC, Set[QAPISchemaType]] = {}
+        self._visited_ret_types = {}
 
     def _begin_user_module(self, name: str) -> None:
         self._visited_ret_types[self._genc] = set()
diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 11b86beeab..d107d09d8e 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -147,7 +147,7 @@  class Indentation:
         """Decrease the indentation level by ``amount``, default 4."""
         if self._level < amount:
             raise ArithmeticError(
-                f"Can't remove {amount:d} spaces from {self!r}")
+                "Can't remove %d spaces from %s" % (amount, self.r))
         self._level -= amount
 
 
diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
index 599f3d1f56..b4e6438b9d 100644
--- a/scripts/qapi/events.py
+++ b/scripts/qapi/events.py
@@ -162,7 +162,7 @@  class QAPISchemaGenEventVisitor(QAPISchemaModularCVisitor):
             prefix, 'qapi-events',
             ' * Schema-defined QAPI/QMP events', None, __doc__)
         self._event_enum_name = c_name(prefix + 'QAPIEvent', protect=False)
-        self._event_enum_members: List[QAPISchemaEnumMember] = []
+        self._event_enum_members = []
         self._event_emit_name = c_name(prefix + 'qapi_event_emit')
 
     def _begin_user_module(self, name: str) -> None:
diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index b40f18eee3..666f50478c 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -123,7 +123,7 @@  def build_params(arg_type: Optional[QAPISchemaObjectType],
 class QAPIGenCCode(QAPIGen):
     def __init__(self, fname: Optional[str]):
         super().__init__(fname)
-        self._start_if: Optional[Tuple[List[str], str, str]] = None
+        self._start_if = None
 
     def start_if(self, ifcond: List[str]) -> None:
         assert self._start_if is None
@@ -243,10 +243,10 @@  class QAPISchemaModularCVisitor(QAPISchemaVisitor):
         self._user_blurb = user_blurb
         self._builtin_blurb = builtin_blurb
         self._pydoc = pydoc
-        self._genc: Optional[QAPIGenC] = None
-        self._genh: Optional[QAPIGenH] = None
-        self._module: Dict[Optional[str], Tuple[QAPIGenC, QAPIGenH]] = {}
-        self._main_module: Optional[str] = None
+        self._genc = None
+        self._genh = None
+        self._module = {}
+        self._main_module = None
 
     @staticmethod
     def _is_user_module(name: Optional[str]) -> bool:
diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py
index 42517210b8..2636227915 100644
--- a/scripts/qapi/main.py
+++ b/scripts/qapi/main.py
@@ -79,8 +79,8 @@  def main() -> int:
 
     funny_char = invalid_prefix_char(args.prefix)
     if funny_char:
-        msg = f"funny character '{funny_char}' in argument of --prefix"
-        print(f"{sys.argv[0]}: {msg}", file=sys.stderr)
+        msg = ("funny character '%s' in argument of --prefix" % funny_char)
+        print("%s: %s" % (sys.argv[0], msg), file=sys.stderr)
         return 1
 
     try:
@@ -90,6 +90,6 @@  def main() -> int:
                  unmask=args.unmask,
                  builtins=args.builtins)
     except QAPIError as err:
-        print(f"{sys.argv[0]}: {str(err)}", file=sys.stderr)
+        print("%s: %s" % (sys.argv[0], str(err)), file=sys.stderr)
         return 1
     return 0
diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index 720449feee..027a517e47 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -26,7 +26,7 @@  from .parser import QAPISchemaParser
 
 
 class QAPISchemaEntity:
-    meta: Optional[str] = None
+    meta = None
 
     def __init__(self, name, info, doc, ifcond=None, features=None):
         assert name is None or isinstance(name, str)
diff --git a/scripts/qapi/source.py b/scripts/qapi/source.py
index d7a79a9b8a..b7ff61b230 100644
--- a/scripts/qapi/source.py
+++ b/scripts/qapi/source.py
@@ -22,9 +22,9 @@  class QAPISchemaPragma:
         # Are documentation comments required?
         self.doc_required = False
         # Whitelist of commands allowed to return a non-dictionary
-        self.returns_whitelist: List[str] = []
+        self.returns_whitelist = []
         # Whitelist of entities allowed to violate case conventions
-        self.name_case_whitelist: List[str] = []
+        self.name_case_whitelist = []
 
 
 class QAPISourceInfo:
@@ -35,11 +35,11 @@  class QAPISourceInfo:
         self.fname = fname
         self.line = line
         self.parent = parent
-        self.pragma: QAPISchemaPragma = (
+        self.pragma = (
             parent.pragma if parent else QAPISchemaPragma()
         )
-        self.defn_meta: Optional[str] = None
-        self.defn_name: Optional[str] = None
+        self.defn_meta = None
+        self.defn_name = None
 
     def set_defn(self, meta: str, name: str) -> None:
         self.defn_meta = meta