| Message ID | 20210924090427.9218-12-kwolf@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | qdev: Add JSON -device and fix QMP device_add | expand |
On Fri, Sep 24, 2021 at 11:04:27AM +0200, Kevin Wolf wrote: > We want to switch both from QemuOpts to the keyval parser in the future, > which results in some incompatibilities, mainly around list handling. > Mark the non-JSON version of both as unstable syntax so that management > tools switch to JSON and we can later make the change without breaking > things. > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > --- > docs/about/deprecated.rst | 11 +++++++++++ > 1 file changed, 11 insertions(+) Reviewed-by: Eric Blake <eblake@redhat.com> > > diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst > index 3c2be84d80..42f6a478fb 100644 > --- a/docs/about/deprecated.rst > +++ b/docs/about/deprecated.rst > @@ -160,6 +160,17 @@ Use ``-display sdl`` instead. > > Use ``-display curses`` instead. > > +Stable non-JSON ``-device`` and ``-object`` syntax (since 6.2) > +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' > + > +If you rely on a stable interface for ``-device`` and ``-object`` that doesn't > +change incompatibly between QEMU versions (e.g. because you are using the QEMU > +command line as a machine interface in scripts rather than interactively), use > +JSON syntax for these options instead. > + > +There is no intention to remove support for non-JSON syntax entirely, but > +future versions may change the way to spell some options. > + > > Plugin argument passing through ``arg=<string>`` (since 6.1) > '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' > -- > 2.31.1 >
On 24/09/21 11:04, Kevin Wolf wrote: > We want to switch both from QemuOpts to the keyval parser in the future, > which results in some incompatibilities, mainly around list handling. > Mark the non-JSON version of both as unstable syntax so that management > tools switch to JSON and we can later make the change without breaking > things. Maybe we need a different section for unstable syntaxes, rather than overloading deprecated.rst? Paolo
On Mon, Sep 27, 2021 at 10:15:43AM +0200, Paolo Bonzini wrote: > On 24/09/21 11:04, Kevin Wolf wrote: > > We want to switch both from QemuOpts to the keyval parser in the future, > > which results in some incompatibilities, mainly around list handling. > > Mark the non-JSON version of both as unstable syntax so that management > > tools switch to JSON and we can later make the change without breaking > > things. > > Maybe we need a different section for unstable syntaxes, rather than > overloading deprecated.rst? This case feels like it hits two scenarios - we want to declare it unstable, which is something we should document in qemu-options.hx. We want to also to warn of specific breakage when the impl changes which is something suitable for deprecations. Regards, Daniel
On Fri, 24 Sept 2021 at 10:14, Kevin Wolf <kwolf@redhat.com> wrote: > > We want to switch both from QemuOpts to the keyval parser in the future, > which results in some incompatibilities, mainly around list handling. > Mark the non-JSON version of both as unstable syntax so that management > tools switch to JSON and we can later make the change without breaking > things. > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > +Stable non-JSON ``-device`` and ``-object`` syntax (since 6.2) > +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' > + > +If you rely on a stable interface for ``-device`` and ``-object`` that doesn't > +change incompatibly between QEMU versions (e.g. because you are using the QEMU > +command line as a machine interface in scripts rather than interactively), use > +JSON syntax for these options instead. > + > +There is no intention to remove support for non-JSON syntax entirely, but > +future versions may change the way to spell some options. As it stands, this is basically saying "pretty much anybody using the command line, your stuff may break in future, instead use some other interface you've never heard of, which doesn't appear to be documented in the manual and which none of the documentation's examples use". Is there some more limited deprecation we can do rather than "the entire commandline for almost all users" ? thanks -- PMM
Am 27.09.2021 um 10:21 hat Daniel P. Berrangé geschrieben: > On Mon, Sep 27, 2021 at 10:15:43AM +0200, Paolo Bonzini wrote: > > On 24/09/21 11:04, Kevin Wolf wrote: > > > We want to switch both from QemuOpts to the keyval parser in the future, > > > which results in some incompatibilities, mainly around list handling. > > > Mark the non-JSON version of both as unstable syntax so that management > > > tools switch to JSON and we can later make the change without breaking > > > things. > > > > Maybe we need a different section for unstable syntaxes, rather than > > overloading deprecated.rst? > > This case feels like it hits two scenarios - we want to declare it > unstable, which is something we should document in qemu-options.hx. Actually, I think a section for unstable syntaxes or generally compatibility promises wouldn't hurt. When I checked, I couldn't find any documentation about the support status of HMP either. Basically, I imagine HMP and non-JSON -device/-object would be on the same level: We don't change things without a reason, but if we do want to change things, compatibility isn't an argument against making the change. > We want to also to warn of specific breakage when the impl changes > which is something suitable for deprecations. We don't do this for HMP either for individual changes. Basically this deprecation notice was meant to make people aware that we're lowering the support status from a long-term stable interface to HMP-like. Kevin
On Mon, Sep 27, 2021 at 12:17:03PM +0200, Kevin Wolf wrote: > Am 27.09.2021 um 10:21 hat Daniel P. Berrangé geschrieben: > > On Mon, Sep 27, 2021 at 10:15:43AM +0200, Paolo Bonzini wrote: > > > On 24/09/21 11:04, Kevin Wolf wrote: > > > > We want to switch both from QemuOpts to the keyval parser in the future, > > > > which results in some incompatibilities, mainly around list handling. > > > > Mark the non-JSON version of both as unstable syntax so that management > > > > tools switch to JSON and we can later make the change without breaking > > > > things. > > > > > > Maybe we need a different section for unstable syntaxes, rather than > > > overloading deprecated.rst? > > > > This case feels like it hits two scenarios - we want to declare it > > unstable, which is something we should document in qemu-options.hx. > > Actually, I think a section for unstable syntaxes or generally > compatibility promises wouldn't hurt. When I checked, I couldn't find > any documentation about the support status of HMP either. > > Basically, I imagine HMP and non-JSON -device/-object would be on the > same level: We don't change things without a reason, but if we do want > to change things, compatibility isn't an argument against making the > change. > > > We want to also to warn of specific breakage when the impl changes > > which is something suitable for deprecations. > > We don't do this for HMP either for individual changes. Well HMP as a whole is considered non-stable, so we don't need to call out individual things. We've got a simple story that QMP == stable, HMP == unstable. The comparison here would be if we declared the entire QEMU CLI to be unstable, except for JSON syntax args. > Basically this deprecation notice was meant to make people aware that > we're lowering the support status from a long-term stable interface to > HMP-like. Bearing in mind our previous discussions it feels like our goal is that we're tending towards a world where we are only wanting to consider JSON based configuration to be stable, and everything else non-stable. I think that's a good long term plan, but if we're really doing that then I think we need to big picture explain it in our docs rather than mention it in passing against individual args. BTW I'm also not a fan of deprecating stuff when our documentation is still using the deprecated syntax and nothing shows the new preferred syntax. We've got alot of results for $ git grep -- ' -object' Regards, Daniel
Am 27.09.2021 um 11:00 hat Peter Maydell geschrieben: > On Fri, 24 Sept 2021 at 10:14, Kevin Wolf <kwolf@redhat.com> wrote: > > > > We want to switch both from QemuOpts to the keyval parser in the future, > > which results in some incompatibilities, mainly around list handling. > > Mark the non-JSON version of both as unstable syntax so that management > > tools switch to JSON and we can later make the change without breaking > > things. > > > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > > > +Stable non-JSON ``-device`` and ``-object`` syntax (since 6.2) > > +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' > > + > > +If you rely on a stable interface for ``-device`` and ``-object`` that doesn't > > +change incompatibly between QEMU versions (e.g. because you are using the QEMU > > +command line as a machine interface in scripts rather than interactively), use > > +JSON syntax for these options instead. > > + > > +There is no intention to remove support for non-JSON syntax entirely, but > > +future versions may change the way to spell some options. > > As it stands, this is basically saying "pretty much anybody > using the command line, your stuff may break in future, instead > use some other interface you've never heard of, which doesn't > appear to be documented in the manual and which none of the > documentation's examples use". The documentation is a valid criticism. We need to document the JSON interfaces properly (which will really mostly be a pointer to the existing QMP documentation at least for -object, but it's important to tell people where to look for the details). > Is there some more limited deprecation we can do rather than "the > entire commandline for almost all users" ? I don't think "almost all" users is true. I see three groups of users: 1. Using a management tool that is probably using libvirt. This is likely the vast majority of users. They won't notice a difference because libvirt abstracts it away. libvirt developers are actively asking us for JSON (and QAPI) based interfaces because using the same representation to describe configurations in QMP and on the CLI makes their life easier. 2. People starting QEMU on the command line manually. This is essentially the same situation as HMP: If something changes, you get an error message, you look up the new syntax, done. Small inconvenience, but that's it. This includes simple scripts that just start QEMU and are only used to store a long command line somewhere. 3. People writing more complex scripts or applications that invoke QEMU manually instead of using libvirt. They may actually be hurt by such changes. They should probably be using a proper machine interface, i.e. JSON mode, so the deprecation notice is for them to change their code. This is probably a small minority and not "almost all users". Yes, we could in theory do a more limited deprecation. The planned change from my side is just going from QemuOpts to the keyval parser, which doesn't change anything in the vast majority of cases. But we have the separation in the monitor between QMP and HMP for a good reason: Requirements for a good machine interface are different from a good human interface. The same is true for the command line. So it seems to make a lot of sense to me to have both a machine interface (JSON based) and a human interface (non-JSON) on the command line, too, and take the same liberties for evolving the human interface as we already do in HMP - which means that it's technically an unstable interface where compatibility doesn't prevent improvements, but not that it looks completely different in every QEMU version. Kevin
On Mon, 27 Sept 2021 at 12:27, Kevin Wolf <kwolf@redhat.com> wrote: > > Am 27.09.2021 um 11:00 hat Peter Maydell geschrieben: > > On Fri, 24 Sept 2021 at 10:14, Kevin Wolf <kwolf@redhat.com> wrote: > > > > > > We want to switch both from QemuOpts to the keyval parser in the future, > > > which results in some incompatibilities, mainly around list handling. > > > Mark the non-JSON version of both as unstable syntax so that management > > > tools switch to JSON and we can later make the change without breaking > > > things. > > > > > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > > > > > +Stable non-JSON ``-device`` and ``-object`` syntax (since 6.2) > > > +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' > > > + > > > +If you rely on a stable interface for ``-device`` and ``-object`` that doesn't > > > +change incompatibly between QEMU versions (e.g. because you are using the QEMU > > > +command line as a machine interface in scripts rather than interactively), use > > > +JSON syntax for these options instead. > > > + > > > +There is no intention to remove support for non-JSON syntax entirely, but > > > +future versions may change the way to spell some options. > > > > As it stands, this is basically saying "pretty much anybody > > using the command line, your stuff may break in future, instead > > use some other interface you've never heard of, which doesn't > > appear to be documented in the manual and which none of the > > documentation's examples use". > > The documentation is a valid criticism. We need to document the JSON > interfaces properly (which will really mostly be a pointer to the > existing QMP documentation at least for -object, but it's important to > tell people where to look for the details). > > > Is there some more limited deprecation we can do rather than "the > > entire commandline for almost all users" ? > > I don't think "almost all" users is true. > I see three groups of users ...all of whom "rely on a stable interface for -device and -object", and only two of whom it's reasonable to say "use the JSON version" to. > 1. Using a management tool that is probably using libvirt. This is > likely the vast majority of users. They won't notice a difference > because libvirt abstracts it away. libvirt developers are actively > asking us for JSON (and QAPI) based interfaces because using the same > representation to describe configurations in QMP and on the CLI makes > their life easier. Yes, absolutely we should be recommending that libvirt and other management interfaces use the JSON. > 2. People starting QEMU on the command line manually. This is > essentially the same situation as HMP: If something changes, you get > an error message, you look up the new syntax, done. Small > inconvenience, but that's it. This includes simple scripts that just > start QEMU and are only used to store a long command line somewhere. It's a small inconvenience that we seem to be imposing on our users on a pretty frequent basis. Moreover, each one of these really needs to be its own deprecation, so that users actually can have some advance notice if they need it and look up the new syntax. We shouldn't hide them all under this umbrella "we might break anything at any time" entry, which I think will pretty much encourage breaking compatibility more often because you don't have to think about "oh, we should deprecate this and maybe print warnings about use of deprecated syntax and then drop it later", you can just break things and point to this "we said -device wasn't going to be stable" entry. As a concrete example, the commit message for this patch vaguely mentions some issues "around list handling", which gives me no idea at all about what syntax is going to break in future or whether it is likely to affect scripts I've written. > 3. People writing more complex scripts or applications that invoke QEMU > manually instead of using libvirt. They may actually be hurt by such > changes. They should probably be using a proper machine interface, > i.e. JSON mode, so the deprecation notice is for them to change > their code. This is probably a small minority and not "almost all > users". Yeah, this group is kind of similar to group 1 (well, at one end it shades into group 1 and at the other into group 2). More generally, I think I'd rather see the deprecation of the old approach appear after some period when the JSON command line version has been available to users and adopted by major consumers like libvirt, not as a patch in the same series which is introducing the JSON syntax in the first plaec. -- PMM
Am 27.09.2021 um 14:52 hat Peter Maydell geschrieben: > On Mon, 27 Sept 2021 at 12:27, Kevin Wolf <kwolf@redhat.com> wrote: > > > > Am 27.09.2021 um 11:00 hat Peter Maydell geschrieben: > > > On Fri, 24 Sept 2021 at 10:14, Kevin Wolf <kwolf@redhat.com> wrote: > > > > > > > > We want to switch both from QemuOpts to the keyval parser in the future, > > > > which results in some incompatibilities, mainly around list handling. > > > > Mark the non-JSON version of both as unstable syntax so that management > > > > tools switch to JSON and we can later make the change without breaking > > > > things. > > > > > > > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > > > > > > > +Stable non-JSON ``-device`` and ``-object`` syntax (since 6.2) > > > > +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' > > > > + > > > > +If you rely on a stable interface for ``-device`` and ``-object`` that doesn't > > > > +change incompatibly between QEMU versions (e.g. because you are using the QEMU > > > > +command line as a machine interface in scripts rather than interactively), use > > > > +JSON syntax for these options instead. > > > > + > > > > +There is no intention to remove support for non-JSON syntax entirely, but > > > > +future versions may change the way to spell some options. > > > > > > As it stands, this is basically saying "pretty much anybody > > > using the command line, your stuff may break in future, instead > > > use some other interface you've never heard of, which doesn't > > > appear to be documented in the manual and which none of the > > > documentation's examples use". > > > > The documentation is a valid criticism. We need to document the JSON > > interfaces properly (which will really mostly be a pointer to the > > existing QMP documentation at least for -object, but it's important to > > tell people where to look for the details). > > > > > Is there some more limited deprecation we can do rather than "the > > > entire commandline for almost all users" ? > > > > I don't think "almost all" users is true. > > > I see three groups of users > > ...all of whom "rely on a stable interface for -device and -object", > and only two of whom it's reasonable to say "use the JSON version" to. I'm not sure that the human interactive case requires unchanged syntax as strictly as the other cases do. After each distro upgrade (or even a browser upgrade within the same distro), some UI changes somewhere and I have to adapt. I don't think anyone ever makes promises like "this button is going to stay in the exact same place forever". And our policy is already that we're not making such promises for HMP either. > > 1. Using a management tool that is probably using libvirt. This is > > likely the vast majority of users. They won't notice a difference > > because libvirt abstracts it away. libvirt developers are actively > > asking us for JSON (and QAPI) based interfaces because using the same > > representation to describe configurations in QMP and on the CLI makes > > their life easier. > > Yes, absolutely we should be recommending that libvirt and > other management interfaces use the JSON. > > > 2. People starting QEMU on the command line manually. This is > > essentially the same situation as HMP: If something changes, you get > > an error message, you look up the new syntax, done. Small > > inconvenience, but that's it. This includes simple scripts that just > > start QEMU and are only used to store a long command line somewhere. > > It's a small inconvenience that we seem to be imposing on our > users on a pretty frequent basis. Moreover, each one of these > really needs to be its own deprecation, so that users actually > can have some advance notice if they need it and look up the > new syntax. We shouldn't hide them all under this umbrella > "we might break anything at any time" entry, which I think > will pretty much encourage breaking compatibility more often > because you don't have to think about "oh, we should deprecate > this and maybe print warnings about use of deprecated syntax > and then drop it later", you can just break things and point > to this "we said -device wasn't going to be stable" entry. Are you suggesting bringing back stricter compatibility rules for HMP then? The problem with the deprecation period is that you need to have a time where both options work. But when you have two different parsers, you can't just magically have the union of their accepted syntaxes. Unless you invest a lot of time and effort, you have a flag day where the syntax changes. And I think this is perfectly reasonable for interactive use. Deprecations are the wrong tool for human interfaces. You don't need to know months in advance that something will change in the UI (you'll forget it again anyway until the change happens and the information becomes relevant), but this is a case for the changelog. If you do need to know months in advance, JSON is probably better suited for your use case. > As a concrete example, the commit message for this patch vaguely > mentions some issues "around list handling", which gives me no > idea at all about what syntax is going to break in future or > whether it is likely to affect scripts I've written. The one known incompatible case for -object is the 'host-nodes' property of memory-backend, which is a list. QemuOpts doesn't support lists at all, but the string visitor allows using a comma separated list of integer ranges. (By the way, the existing syntax of it seems to be undocumented, so not sure how many users even know about how to use this list support.) For -device, who knows. I'm positive that the string visitor list support isn't used there and a patch in this series removes the string visitor from -device therefore. What other incompatibilities and hacks there are in some devices, we'll find out when we actually try to describe devices in QAPI. I'm not even entirely sure yet that there will be incompatibilities. But if we do find a case, I don't want to have to stop and delay work for another year to have a deprecation period for interactive human interfaces. Another similar case is -drive: We only still have it in its current form because even figuring out what the exact cases are that would be incompatible is hard enough to stop us from even trying to deprecate them individually. But actually it would seem reasonable enough to me to just change -drive because that makes sense as a convenient option for interactive use, while -blockdev is more verbose and covers the stable API part. > > 3. People writing more complex scripts or applications that invoke QEMU > > manually instead of using libvirt. They may actually be hurt by such > > changes. They should probably be using a proper machine interface, > > i.e. JSON mode, so the deprecation notice is for them to change > > their code. This is probably a small minority and not "almost all > > users". > > Yeah, this group is kind of similar to group 1 (well, at one > end it shades into group 1 and at the other into group 2). > > More generally, I think I'd rather see the deprecation of > the old approach appear after some period when the JSON > command line version has been available to users and adopted > by major consumers like libvirt, not as a patch in the same > series which is introducing the JSON syntax in the first plaec. Okay, that can be done. So I can post this final patch separately and only for -object for now, and we'll do -device separately once libvirt uses JSON for it. Kevin
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 3c2be84d80..42f6a478fb 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -160,6 +160,17 @@ Use ``-display sdl`` instead. Use ``-display curses`` instead. +Stable non-JSON ``-device`` and ``-object`` syntax (since 6.2) +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +If you rely on a stable interface for ``-device`` and ``-object`` that doesn't +change incompatibly between QEMU versions (e.g. because you are using the QEMU +command line as a machine interface in scripts rather than interactively), use +JSON syntax for these options instead. + +There is no intention to remove support for non-JSON syntax entirely, but +future versions may change the way to spell some options. + Plugin argument passing through ``arg=<string>`` (since 6.1) ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
We want to switch both from QemuOpts to the keyval parser in the future, which results in some incompatibilities, mainly around list handling. Mark the non-JSON version of both as unstable syntax so that management tools switch to JSON and we can later make the change without breaking things. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- docs/about/deprecated.rst | 11 +++++++++++ 1 file changed, 11 insertions(+)