| Message ID | 20250110104946.74960-1-victortoso@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | qapi-go: add generator for Golang interfaces | expand |
Victor Toso <victortoso@redhat.com> writes: > This patch series intent is to introduce a generator that produces a Go > module for Go applications to interact over QMP with QEMU. > > The initial Goal is to have a Go module that works as intended and can > be improved upon. I'd consider initial releases to be alpha while we > work with utilities tools and libraries on top of this. > > The generated code should reside in a separated Git repository, similar > to python-qemu-qmp. > > Applications should be able to consume this under qemu.org > namespace (e.g: import "qemu.org/go/qemu"), see Daniel's suggestion: > https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg07024.html > > This is the third iteration: > v2: https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg04785.html > > I've pushed this series in my gitlab fork: > https://gitlab.com/victortoso/qapi-go/ > > The fork contains some tests, including tests that were generated from > QAPI's own examples from another generator created for testing, if you > are interested in it: > https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg04946.html > > I've generated the qapi-go module over each commit of this series, see: > https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v3-by-patch > > I've also generated the qapi-go module over QEMU tags: v9.1.0, v9.2.0: > https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v3-by-tags > > -- > > Sorry that its been awhile between v2 and v3, I had to prioritize other > things. I hope to get this back on track in 2025. > > Cheers, > Victor > > * Changes: > > On generated go: > - the output should be formatted as gofmt/goimports tools (Daniel) > > - Included QAPI's documentation too (Daniel), see: > https://lists.gnu.org/archive/html/qemu-devel/2024-11/msg01621.html > > - Commands and Events should Marshal directly (Andrea) > > On python script: > - rebased: now uses QAPISchemaBranches, QAPISchemaAlternatives > > - use textwrap as much as possible (Andrea) > > - lots of changes to make the output like gofmt does > > Victor Toso (8): > qapi: golang: Generate enum type > qapi: golang: Generate alternate types > qapi: golang: Generate struct types > qapi: golang: structs: Address nullable members > qapi: golang: Generate union type > qapi: golang: Generate event type > qapi: golang: Generate command type > docs: add notes on Golang code generator > > docs/devel/index-build.rst | 1 + > docs/devel/qapi-golang-code-gen.rst | 548 +++++++++ > scripts/qapi/golang.py | 1645 +++++++++++++++++++++++++++ > scripts/qapi/main.py | 3 + > 4 files changed, 2197 insertions(+) > create mode 100644 docs/devel/qapi-golang-code-gen.rst > create mode 100644 scripts/qapi/golang.py This is series adds a backend that slots in cleanly, i.e. without any changes to the core. That makes it as low-risk to merge as it gets. I'd like an Acked-by for the generated Go from someone familiar the kind of software that could use it. The Go backend is a single golang.py, which generates: * Types: alternates.go enums.go structs.go unions.go * Commands: commands.go * Events: events.go It doesn't generate visitors or introspection code. Correct? The existing C backend generates code for * Types (types.py): qapi-builtin-types.[ch] qapi-types.[ch] qapi-types-*.[ch] * Visitors (visit.py): qapi-builtin-visit.h qapi-visit.[ch] qapi-visit-*.[ch] * Commands (commands.py): qapi-init-commands.h qapi-commands.[ch] qapi-commands-*.[ch] * Events (events.py): qapi-emit-events.h qapi-events.[ch] qapi-events-*.[ch] * Introspection (introspect.py): qapi-introspect.h The -* files are all one pair of files per module (the things pulled in with include directives), if any. We do this to avoid "touch the QAPI schema, recompile the world." The generated Go is monolithic. No "recompile the world" problem with Go? golang.py is somewhat big. Whether splitting it up along the lines of the C backend would improve things I can't say. No need to worry about that now.
Hi, On Mon, Jan 13, 2025 at 01:52:25PM +0100, Markus Armbruster wrote: > Victor Toso <victortoso@redhat.com> writes: > > > This patch series intent is to introduce a generator that produces a Go > > module for Go applications to interact over QMP with QEMU. > > > > The initial Goal is to have a Go module that works as intended and can > > be improved upon. I'd consider initial releases to be alpha while we > > work with utilities tools and libraries on top of this. > > > > The generated code should reside in a separated Git repository, similar > > to python-qemu-qmp. > > > > Applications should be able to consume this under qemu.org > > namespace (e.g: import "qemu.org/go/qemu"), see Daniel's suggestion: > > https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg07024.html > > > > This is the third iteration: > > v2: https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg04785.html > > > > I've pushed this series in my gitlab fork: > > https://gitlab.com/victortoso/qapi-go/ > > > > The fork contains some tests, including tests that were generated from > > QAPI's own examples from another generator created for testing, if you > > are interested in it: > > https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg04946.html > > > > I've generated the qapi-go module over each commit of this series, see: > > https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v3-by-patch > > > > I've also generated the qapi-go module over QEMU tags: v9.1.0, v9.2.0: > > https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v3-by-tags > > > > -- > > > > Sorry that its been awhile between v2 and v3, I had to prioritize other > > things. I hope to get this back on track in 2025. > > > > Cheers, > > Victor > > > > * Changes: > > > > On generated go: > > - the output should be formatted as gofmt/goimports tools (Daniel) > > > > - Included QAPI's documentation too (Daniel), see: > > https://lists.gnu.org/archive/html/qemu-devel/2024-11/msg01621.html > > > > - Commands and Events should Marshal directly (Andrea) > > > > On python script: > > - rebased: now uses QAPISchemaBranches, QAPISchemaAlternatives > > > > - use textwrap as much as possible (Andrea) > > > > - lots of changes to make the output like gofmt does > > > > Victor Toso (8): > > qapi: golang: Generate enum type > > qapi: golang: Generate alternate types > > qapi: golang: Generate struct types > > qapi: golang: structs: Address nullable members > > qapi: golang: Generate union type > > qapi: golang: Generate event type > > qapi: golang: Generate command type > > docs: add notes on Golang code generator > > > > docs/devel/index-build.rst | 1 + > > docs/devel/qapi-golang-code-gen.rst | 548 +++++++++ > > scripts/qapi/golang.py | 1645 +++++++++++++++++++++++++++ > > scripts/qapi/main.py | 3 + > > 4 files changed, 2197 insertions(+) > > create mode 100644 docs/devel/qapi-golang-code-gen.rst > > create mode 100644 scripts/qapi/golang.py > > This is series adds a backend that slots in cleanly, i.e. without any > changes to the core. That makes it as low-risk to merge as it gets. > > I'd like an Acked-by for the generated Go from someone familiar the kind > of software that could use it. > > The Go backend is a single golang.py, which generates: > > * Types: > alternates.go enums.go structs.go unions.go > > * Commands: > commands.go > > * Events: > events.go > > It doesn't generate visitors or introspection code. > > Correct? Correct. I've actually thought about following what we did with libvirt-go-module which appends _generated to the files that are generated, meaning that we would also have files that are without _generated in the name, with custoam/manual code related to that file. Either way, that does not change the Go module (does not break API/ABI) so we can customize it if needed at a later time. > The existing C backend generates code for > > * Types (types.py): > qapi-builtin-types.[ch] > qapi-types.[ch] qapi-types-*.[ch] > > * Visitors (visit.py): > > qapi-builtin-visit.h > qapi-visit.[ch] qapi-visit-*.[ch] > > * Commands (commands.py): > > qapi-init-commands.h > qapi-commands.[ch] qapi-commands-*.[ch] > > * Events (events.py): > > qapi-emit-events.h > qapi-events.[ch] qapi-events-*.[ch] > > * Introspection (introspect.py): > > qapi-introspect.h > > The -* files are all one pair of files per module (the things pulled in > with include directives), if any. We do this to avoid "touch the QAPI > schema, recompile the world." > > The generated Go is monolithic. No "recompile the world" problem with > Go? From my experience, Go compiler only builds when the .go file changes. If the QAPI changes and the generator outputs different .go files, it'll recompile at the project using it. > golang.py is somewhat big. Whether splitting it up along the lines of > the C backend would improve things I can't say. No need to worry about > that now. Yes, I'm not super experienced with python. I'm all ears to any suggestions on how to organize the source better. I'm happy to do it as a follow-up. Cheers, Victor
This patch series intent is to introduce a generator that produces a Go module for Go applications to interact over QMP with QEMU. The initial Goal is to have a Go module that works as intended and can be improved upon. I'd consider initial releases to be alpha while we work with utilities tools and libraries on top of this. The generated code should reside in a separated Git repository, similar to python-qemu-qmp. Applications should be able to consume this under qemu.org namespace (e.g: import "qemu.org/go/qemu"), see Daniel's suggestion: https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg07024.html This is the third iteration: v2: https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg04785.html I've pushed this series in my gitlab fork: https://gitlab.com/victortoso/qapi-go/ The fork contains some tests, including tests that were generated from QAPI's own examples from another generator created for testing, if you are interested in it: https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg04946.html I've generated the qapi-go module over each commit of this series, see: https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v3-by-patch I've also generated the qapi-go module over QEMU tags: v9.1.0, v9.2.0: https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v3-by-tags -- Sorry that its been awhile between v2 and v3, I had to prioritize other things. I hope to get this back on track in 2025. Cheers, Victor * Changes: On generated go: - the output should be formatted as gofmt/goimports tools (Daniel) - Included QAPI's documentation too (Daniel), see: https://lists.gnu.org/archive/html/qemu-devel/2024-11/msg01621.html - Commands and Events should Marshal directly (Andrea) On python script: - rebased: now uses QAPISchemaBranches, QAPISchemaAlternatives - use textwrap as much as possible (Andrea) - lots of changes to make the output like gofmt does Victor Toso (8): qapi: golang: Generate enum type qapi: golang: Generate alternate types qapi: golang: Generate struct types qapi: golang: structs: Address nullable members qapi: golang: Generate union type qapi: golang: Generate event type qapi: golang: Generate command type docs: add notes on Golang code generator docs/devel/index-build.rst | 1 + docs/devel/qapi-golang-code-gen.rst | 548 +++++++++ scripts/qapi/golang.py | 1645 +++++++++++++++++++++++++++ scripts/qapi/main.py | 3 + 4 files changed, 2197 insertions(+) create mode 100644 docs/devel/qapi-golang-code-gen.rst create mode 100644 scripts/qapi/golang.py