| Message ID | 20220721195620.123837-3-het.gala@nutanix.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | multifd: Multiple interface support on top of Multifd | expand |
On Thu, Jul 21, 2022 at 07:56:15PM +0000, Het Gala wrote: > i) Modified the format of the qemu monitor command : 'migrate' by adding a list, > each element in the list consisting of multifd connection parameters: source > uri, destination uri and of the number of multifd channels between each pair. > > ii) Information of all multifd connection parameters' list and length of the > list is stored in 'OutgoingMigrateParams' struct. > > Suggested-by: Manish Mishra <manish.mishra@nutanix.com> > Signed-off-by: Het Gala <het.gala@nutanix.com> > --- > migration/migration.c | 52 +++++++++++++++++++++++++++++-------- > migration/socket.c | 60 ++++++++++++++++++++++++++++++++++++++++--- > migration/socket.h | 19 +++++++++++++- > monitor/hmp-cmds.c | 1 + > qapi/migration.json | 47 +++++++++++++++++++++++++++++---- > 5 files changed, 160 insertions(+), 19 deletions(-) > > diff --git a/qapi/migration.json b/qapi/migration.json > index 81185d4311..456247af8f 100644 > --- a/qapi/migration.json > +++ b/qapi/migration.json > @@ -1449,12 +1449,37 @@ > ## > { 'command': 'migrate-continue', 'data': {'state': 'MigrationStatus'} } > > +## > +# @MigrateUriParameter: > +# > +# Information regarding which source interface is connected to which > +# destination interface and number of multifd channels over each interface. > +# > +# @source-uri: uri of the source VM. Default port number is 0. > +# > +# @destination-uri: uri of the destination VM > +# > +# @multifd-channels: number of parallel multifd channels used to migrate data > +# for specific source-uri and destination-uri. Default value > +# in this case is 2 (Since 7.1) > +# > +## > +{ 'struct' : 'MigrateUriParameter', > + 'data' : { 'source-uri' : 'str', > + 'destination-uri' : 'str', > + '*multifd-channels' : 'uint8'} } > + > ## > # @migrate: > # > # Migrates the current running guest to another Virtual Machine. > # > # @uri: the Uniform Resource Identifier of the destination VM > +# for migration thread > +# > +# @multi-fd-uri-list: list of pair of source and destination VM Uniform > +# Resource Identifiers with number of multifd-channels > +# for each pair > # > # @blk: do block migration (full disk copy) > # > @@ -1474,20 +1499,32 @@ > # 1. The 'query-migrate' command should be used to check migration's progress > # and final result (this information is provided by the 'status' member) > # > -# 2. All boolean arguments default to false > +# 2. The uri argument should have the Uniform Resource Identifier of default > +# destination VM. This connection will be bound to default network > # > -# 3. The user Monitor's "detach" argument is invalid in QMP and should not > +# 3. All boolean arguments default to false > +# > +# 4. The user Monitor's "detach" argument is invalid in QMP and should not > # be used > # > # Example: > # > -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } > +# -> { "execute": "migrate", > +# "arguments": { > +# "uri": "tcp:0:4446", > +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", > +# "destination-uri": "tcp:0:4480", > +# "multifd-channels": 4}, > +# { "source-uri": "tcp:10.0.0.0: ", > +# "destination-uri": "tcp:11.0.0.0:7789", > +# "multifd-channels": 5} ] } } > # <- { "return": {} } > # > ## > { 'command': 'migrate', > - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', > - '*detach': 'bool', '*resume': 'bool' } } > + 'data': {'uri': 'str', '*multi-fd-uri-list': ['MigrateUriParameter'], > + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool', > + '*resume': 'bool' } } Considering the existing migrate API from a QAPI design POV, I think there are several significant flaws with it The use of URIs is the big red flag. It is basically a data encoding scheme within a data encoding scheme. QEMU code should be able to directly work with the results from QAPI, without having todo a second level of parsing. URIs made sense in the context of HMP or the QemuOpts CLI, but do not make sense in QMP. We made a mistake in this respect when we first introduced QMP and implemented 'migrate'. If we going to extend the migrate API I think we should stop using URIs for the new fields, and instead define a QAPI discriminated union for the different data transport backends we offer. { 'enum': 'MigrateTransport', 'data': ['socket', 'exec'] } { 'union': 'MigrateAddress', 'base': { 'transport': 'MigrateTransport'}, 'discriminator': 'transport', 'data': { 'socket': 'SocketAddress', 'exec': ['str'], } NB, 'socket' should be able to cover all of 'tcp', 'unix', 'vsock' and 'fd' already. I'm fuzzy on best way to represent RDMA. IIUC, the desire of migration maintainers is that we can ultimately have multifd as the preferred, or even only, mechanism. Aside from the main outbound migration control channel, and the multifd data channels, IIUC we have a potential desire to have more channels for post-copy async requests. This all suggests to me a more general representation along the lines of: { 'enum': 'MigrateChannelType', 'data': ['control', 'data', 'async'] } { 'struct': 'MigrateChannel', 'data': { 'type': 'MigrateChannelType', 'src-addr': 'MigrateAddress', 'dst-addr': 'MigrateAddress', 'count': 'int', } } { 'comand': 'migrate', 'data': { '*uri': 'str' '*channels': ['MigrateChannel'] } } With 'uri' and 'channels' being mutually exclusive here. This whole scheme brings in redundancy wrt to the 'migrate-set-parameters' API wrt multifd - essentally the same data is now being set in two different places. IMHO, we should declare the 'multifd' capability and the 'multifd-chanels' parameter deprecated, since the information they provide is totally redundant, if you're giving an explicit list of channels to 'migrate'. With regards, Daniel
On 26/07/22 4:43 pm, Daniel P. Berrangé wrote: > On Thu, Jul 21, 2022 at 07:56:15PM +0000, Het Gala wrote: >> i) Modified the format of the qemu monitor command : 'migrate' by adding a list, >> each element in the list consisting of multifd connection parameters: source >> uri, destination uri and of the number of multifd channels between each pair. >> >> ii) Information of all multifd connection parameters' list and length of the >> list is stored in 'OutgoingMigrateParams' struct. >> >> Suggested-by: Manish Mishra <manish.mishra@nutanix.com> >> Signed-off-by: Het Gala <het.gala@nutanix.com> >> --- >> migration/migration.c | 52 +++++++++++++++++++++++++++++-------- >> migration/socket.c | 60 ++++++++++++++++++++++++++++++++++++++++--- >> migration/socket.h | 19 +++++++++++++- >> monitor/hmp-cmds.c | 1 + >> qapi/migration.json | 47 +++++++++++++++++++++++++++++---- >> 5 files changed, 160 insertions(+), 19 deletions(-) >> >> diff --git a/qapi/migration.json b/qapi/migration.json >> index 81185d4311..456247af8f 100644 >> --- a/qapi/migration.json >> +++ b/qapi/migration.json >> @@ -1449,12 +1449,37 @@ >> ## >> { 'command': 'migrate-continue', 'data': {'state': 'MigrationStatus'} } >> >> +## >> +# @MigrateUriParameter: >> +# >> +# Information regarding which source interface is connected to which >> +# destination interface and number of multifd channels over each interface. >> +# >> +# @source-uri: uri of the source VM. Default port number is 0. >> +# >> +# @destination-uri: uri of the destination VM >> +# >> +# @multifd-channels: number of parallel multifd channels used to migrate data >> +# for specific source-uri and destination-uri. Default value >> +# in this case is 2 (Since 7.1) >> +# >> +## >> +{ 'struct' : 'MigrateUriParameter', >> + 'data' : { 'source-uri' : 'str', >> + 'destination-uri' : 'str', >> + '*multifd-channels' : 'uint8'} } >> + >> ## >> # @migrate: >> # >> # Migrates the current running guest to another Virtual Machine. >> # >> # @uri: the Uniform Resource Identifier of the destination VM >> +# for migration thread >> +# >> +# @multi-fd-uri-list: list of pair of source and destination VM Uniform >> +# Resource Identifiers with number of multifd-channels >> +# for each pair >> # >> # @blk: do block migration (full disk copy) >> # >> @@ -1474,20 +1499,32 @@ >> # 1. The 'query-migrate' command should be used to check migration's progress >> # and final result (this information is provided by the 'status' member) >> # >> -# 2. All boolean arguments default to false >> +# 2. The uri argument should have the Uniform Resource Identifier of default >> +# destination VM. This connection will be bound to default network >> # >> -# 3. The user Monitor's "detach" argument is invalid in QMP and should not >> +# 3. All boolean arguments default to false >> +# >> +# 4. The user Monitor's "detach" argument is invalid in QMP and should not >> # be used >> # >> # Example: >> # >> -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } >> +# -> { "execute": "migrate", >> +# "arguments": { >> +# "uri": "tcp:0:4446", >> +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", >> +# "destination-uri": "tcp:0:4480", >> +# "multifd-channels": 4}, >> +# { "source-uri": "tcp:10.0.0.0: ", >> +# "destination-uri": "tcp:11.0.0.0:7789", >> +# "multifd-channels": 5} ] } } >> # <- { "return": {} } >> # >> ## >> { 'command': 'migrate', >> - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', >> - '*detach': 'bool', '*resume': 'bool' } } >> + 'data': {'uri': 'str', '*multi-fd-uri-list': ['MigrateUriParameter'], >> + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool', >> + '*resume': 'bool' } } > Considering the existing migrate API from a QAPI design POV, I > think there are several significant flaws with it > > The use of URIs is the big red flag. It is basically a data encoding > scheme within a data encoding scheme. QEMU code should be able to > directly work with the results from QAPI, without having todo a > second level of parsing. > > URIs made sense in the context of HMP or the QemuOpts CLI, but do not > make sense in QMP. We made a mistake in this respect when we first > introduced QMP and implemented 'migrate'. > > If we going to extend the migrate API I think we should stop using URIs > for the new fields, and instead define a QAPI discriminated union for > the different data transport backends we offer. > > { 'enum': 'MigrateTransport', > 'data': ['socket', 'exec'] } > > { 'union': 'MigrateAddress', > 'base': { 'transport': 'MigrateTransport'}, > 'discriminator': 'transport', > 'data': { > 'socket': 'SocketAddress', > 'exec': ['str'], > } > > NB, 'socket' should be able to cover all of 'tcp', 'unix', 'vsock' > and 'fd' already. I'm fuzzy on best way to represent RDMA. > > > IIUC, the desire of migration maintainers is that we can ultimately > have multifd as the preferred, or even only, mechanism. Aside from > the main outbound migration control channel, and the multifd > data channels, IIUC we have a potential desire to have more channels > for post-copy async requests. > > This all suggests to me a more general representation along the > lines of: > > { 'enum': 'MigrateChannelType', > 'data': ['control', 'data', 'async'] } > > { 'struct': 'MigrateChannel', > 'data': { > 'type': 'MigrateChannelType', > 'src-addr': 'MigrateAddress', > 'dst-addr': 'MigrateAddress', > 'count': 'int', > } } > > { 'comand': 'migrate', > 'data': { > '*uri': 'str' > '*channels': ['MigrateChannel'] > } > } > > With 'uri' and 'channels' being mutually exclusive here. > > This whole scheme brings in redundancy wrt to the 'migrate-set-parameters' > API wrt multifd - essentally the same data is now being set in two > different places. IMHO, we should declare the 'multifd' capability > and the 'multifd-chanels' parameter deprecated, since the information > they provide is totally redundant, if you're giving an explicit list > of channels to 'migrate'. > Hi Daniel. Initially while brainstorming this idea for the first time, we also came up with the same thought of depricating the migrate API. But how will we achieve this now and how is it going to work. Is it like we will be making migate V2 APIs initially, integrate it and then depricate the old one? would be happy to get some pointers from your end. > With regards, > Daniel With Regards, Het Gala
Het Gala <het.gala@nutanix.com> writes: > On 26/07/22 4:43 pm, Daniel P. Berrangé wrote: >> On Thu, Jul 21, 2022 at 07:56:15PM +0000, Het Gala wrote: >>> i) Modified the format of the qemu monitor command : 'migrate' by adding a list, >>> each element in the list consisting of multifd connection parameters: source >>> uri, destination uri and of the number of multifd channels between each pair. >>> >>> ii) Information of all multifd connection parameters' list and length of the >>> list is stored in 'OutgoingMigrateParams' struct. >>> >>> Suggested-by: Manish Mishra <manish.mishra@nutanix.com> >>> Signed-off-by: Het Gala <het.gala@nutanix.com> >>> --- >>> migration/migration.c | 52 +++++++++++++++++++++++++++++-------- >>> migration/socket.c | 60 ++++++++++++++++++++++++++++++++++++++++--- >>> migration/socket.h | 19 +++++++++++++- >>> monitor/hmp-cmds.c | 1 + >>> qapi/migration.json | 47 +++++++++++++++++++++++++++++---- >>> 5 files changed, 160 insertions(+), 19 deletions(-) >>> >>> diff --git a/qapi/migration.json b/qapi/migration.json >>> index 81185d4311..456247af8f 100644 >>> --- a/qapi/migration.json >>> +++ b/qapi/migration.json >>> @@ -1449,12 +1449,37 @@ >>> ## >>> { 'command': 'migrate-continue', 'data': {'state': 'MigrationStatus'} } >>> +## >>> +# @MigrateUriParameter: >>> +# >>> +# Information regarding which source interface is connected to which >>> +# destination interface and number of multifd channels over each interface. >>> +# >>> +# @source-uri: uri of the source VM. Default port number is 0. >>> +# >>> +# @destination-uri: uri of the destination VM >>> +# >>> +# @multifd-channels: number of parallel multifd channels used to migrate data >>> +# for specific source-uri and destination-uri. Default value >>> +# in this case is 2 (Since 7.1) >>> +# >>> +## >>> +{ 'struct' : 'MigrateUriParameter', >>> + 'data' : { 'source-uri' : 'str', >>> + 'destination-uri' : 'str', >>> + '*multifd-channels' : 'uint8'} } >>> + >>> ## >>> # @migrate: >>> # >>> # Migrates the current running guest to another Virtual Machine. >>> # >>> # @uri: the Uniform Resource Identifier of the destination VM >>> +# for migration thread >>> +# >>> +# @multi-fd-uri-list: list of pair of source and destination VM Uniform >>> +# Resource Identifiers with number of multifd-channels >>> +# for each pair >>> # >>> # @blk: do block migration (full disk copy) >>> # >>> @@ -1474,20 +1499,32 @@ >>> # 1. The 'query-migrate' command should be used to check migration's progress >>> # and final result (this information is provided by the 'status' member) >>> # >>> -# 2. All boolean arguments default to false >>> +# 2. The uri argument should have the Uniform Resource Identifier of default >>> +# destination VM. This connection will be bound to default network >>> # >>> -# 3. The user Monitor's "detach" argument is invalid in QMP and should not >>> +# 3. All boolean arguments default to false >>> +# >>> +# 4. The user Monitor's "detach" argument is invalid in QMP and should not >>> # be used >>> # >>> # Example: >>> # >>> -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } >>> +# -> { "execute": "migrate", >>> +# "arguments": { >>> +# "uri": "tcp:0:4446", >>> +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", >>> +# "destination-uri": "tcp:0:4480", >>> +# "multifd-channels": 4}, >>> +# { "source-uri": "tcp:10.0.0.0: ", >>> +# "destination-uri": "tcp:11.0.0.0:7789", >>> +# "multifd-channels": 5} ] } } >>> # <- { "return": {} } >>> # >>> ## >>> { 'command': 'migrate', >>> - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', >>> - '*detach': 'bool', '*resume': 'bool' } } >>> + 'data': {'uri': 'str', '*multi-fd-uri-list': ['MigrateUriParameter'], >>> + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool', >>> + '*resume': 'bool' } } >> >> Considering the existing migrate API from a QAPI design POV, I >> think there are several significant flaws with it >> >> The use of URIs is the big red flag. It is basically a data encoding >> scheme within a data encoding scheme. QEMU code should be able to >> directly work with the results from QAPI, without having todo a >> second level of parsing. Concur. >> URIs made sense in the context of HMP or the QemuOpts CLI, but do not >> make sense in QMP. We made a mistake in this respect when we first >> introduced QMP and implemented 'migrate'. >> >> If we going to extend the migrate API I think we should stop using URIs >> for the new fields, and instead define a QAPI discriminated union for >> the different data transport backends we offer. >> >> { 'enum': 'MigrateTransport', >> 'data': ['socket', 'exec'] } >> >> { 'union': 'MigrateAddress', >> 'base': { 'transport': 'MigrateTransport'}, >> 'discriminator': 'transport', >> 'data': { >> 'socket': 'SocketAddress', >> 'exec': ['str'], >> } >> >> NB, 'socket' should be able to cover all of 'tcp', 'unix', 'vsock' >> and 'fd' already. I'm fuzzy on best way to represent RDMA. >> >> >> IIUC, the desire of migration maintainers is that we can ultimately >> have multifd as the preferred, or even only, mechanism. Aside from >> the main outbound migration control channel, and the multifd >> data channels, IIUC we have a potential desire to have more channels >> for post-copy async requests. >> >> This all suggests to me a more general representation along the >> lines of: >> >> { 'enum': 'MigrateChannelType', >> 'data': ['control', 'data', 'async'] } >> >> { 'struct': 'MigrateChannel', >> 'data': { >> 'type': 'MigrateChannelType', >> 'src-addr': 'MigrateAddress', >> 'dst-addr': 'MigrateAddress', >> 'count': 'int', >> } } >> >> { 'comand': 'migrate', >> 'data': { >> '*uri': 'str' >> '*channels': ['MigrateChannel'] >> } >> } >> >> With 'uri' and 'channels' being mutually exclusive here. >> >> This whole scheme brings in redundancy wrt to the 'migrate-set-parameters' >> API wrt multifd - essentally the same data is now being set in two >> different places. IMHO, we should declare the 'multifd' capability >> and the 'multifd-chanels' parameter deprecated, since the information >> they provide is totally redundant, if you're giving an explicit list >> of channels to 'migrate'. > > Hi Daniel. Initially while brainstorming this idea for the first time, we also came up with the same thought of depricating the migrate > API. But how will we achieve this now and how is it going to work. Is it like we will be making migate V2 APIs initially, integrate it and then > depricate the old one? would be happy to get some pointers from your end. Migration maintainers, please advise.
On 02/08/22 1:23 pm, Markus Armbruster wrote: > Het Gala<het.gala@nutanix.com> writes: > >> On 26/07/22 4:43 pm, Daniel P. Berrangé wrote: >>> On Thu, Jul 21, 2022 at 07:56:15PM +0000, Het Gala wrote: >>>> i) Modified the format of the qemu monitor command : 'migrate' by adding a list, >>>> each element in the list consisting of multifd connection parameters: source >>>> uri, destination uri and of the number of multifd channels between each pair. >>>> >>>> ii) Information of all multifd connection parameters' list and length of the >>>> list is stored in 'OutgoingMigrateParams' struct. >>>> >>>> Suggested-by: Manish Mishra<manish.mishra@nutanix.com> >>>> Signed-off-by: Het Gala<het.gala@nutanix.com> >>>> --- >>>> migration/migration.c | 52 +++++++++++++++++++++++++++++-------- >>>> migration/socket.c | 60 ++++++++++++++++++++++++++++++++++++++++--- >>>> migration/socket.h | 19 +++++++++++++- >>>> monitor/hmp-cmds.c | 1 + >>>> qapi/migration.json | 47 +++++++++++++++++++++++++++++---- >>>> 5 files changed, 160 insertions(+), 19 deletions(-) >>>> >>>> diff --git a/qapi/migration.json b/qapi/migration.json >>>> index 81185d4311..456247af8f 100644 >>>> --- a/qapi/migration.json >>>> +++ b/qapi/migration.json >>>> @@ -1449,12 +1449,37 @@ >>>> ## >>>> { 'command': 'migrate-continue', 'data': {'state': 'MigrationStatus'} } >>>> +## >>>> +# @MigrateUriParameter: >>>> +# >>>> +# Information regarding which source interface is connected to which >>>> +# destination interface and number of multifd channels over each interface. >>>> +# >>>> +# @source-uri: uri of the source VM. Default port number is 0. >>>> +# >>>> +# @destination-uri: uri of the destination VM >>>> +# >>>> +# @multifd-channels: number of parallel multifd channels used to migrate data >>>> +# for specific source-uri and destination-uri. Default value >>>> +# in this case is 2 (Since 7.1) >>>> +# >>>> +## >>>> +{ 'struct' : 'MigrateUriParameter', >>>> + 'data' : { 'source-uri' : 'str', >>>> + 'destination-uri' : 'str', >>>> + '*multifd-channels' : 'uint8'} } >>>> + >>>> ## >>>> # @migrate: >>>> # >>>> # Migrates the current running guest to another Virtual Machine. >>>> # >>>> # @uri: the Uniform Resource Identifier of the destination VM >>>> +# for migration thread >>>> +# >>>> +# @multi-fd-uri-list: list of pair of source and destination VM Uniform >>>> +# Resource Identifiers with number of multifd-channels >>>> +# for each pair >>>> # >>>> # @blk: do block migration (full disk copy) >>>> # >>>> @@ -1474,20 +1499,32 @@ >>>> # 1. The 'query-migrate' command should be used to check migration's progress >>>> # and final result (this information is provided by the 'status' member) >>>> # >>>> -# 2. All boolean arguments default to false >>>> +# 2. The uri argument should have the Uniform Resource Identifier of default >>>> +# destination VM. This connection will be bound to default network >>>> # >>>> -# 3. The user Monitor's "detach" argument is invalid in QMP and should not >>>> +# 3. All boolean arguments default to false >>>> +# >>>> +# 4. The user Monitor's "detach" argument is invalid in QMP and should not >>>> # be used >>>> # >>>> # Example: >>>> # >>>> -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } >>>> +# -> { "execute": "migrate", >>>> +# "arguments": { >>>> +# "uri": "tcp:0:4446", >>>> +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", >>>> +# "destination-uri": "tcp:0:4480", >>>> +# "multifd-channels": 4}, >>>> +# { "source-uri": "tcp:10.0.0.0: ", >>>> +# "destination-uri": "tcp:11.0.0.0:7789", >>>> +# "multifd-channels": 5} ] } } >>>> # <- { "return": {} } >>>> # >>>> ## >>>> { 'command': 'migrate', >>>> - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', >>>> - '*detach': 'bool', '*resume': 'bool' } } >>>> + 'data': {'uri': 'str', '*multi-fd-uri-list': ['MigrateUriParameter'], >>>> + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool', >>>> + '*resume': 'bool' } } >>> Considering the existing migrate API from a QAPI design POV, I >>> think there are several significant flaws with it >>> >>> The use of URIs is the big red flag. It is basically a data encoding >>> scheme within a data encoding scheme. QEMU code should be able to >>> directly work with the results from QAPI, without having todo a >>> second level of parsing. > Concur. > >>> URIs made sense in the context of HMP or the QemuOpts CLI, but do not >>> make sense in QMP. We made a mistake in this respect when we first >>> introduced QMP and implemented 'migrate'. >>> >>> If we going to extend the migrate API I think we should stop using URIs >>> for the new fields, and instead define a QAPI discriminated union for >>> the different data transport backends we offer. >>> >>> { 'enum': 'MigrateTransport', >>> 'data': ['socket', 'exec'] } >>> >>> { 'union': 'MigrateAddress', >>> 'base': { 'transport': 'MigrateTransport'}, >>> 'discriminator': 'transport', >>> 'data': { >>> 'socket': 'SocketAddress', >>> 'exec': ['str'], >>> } >>> >>> NB, 'socket' should be able to cover all of 'tcp', 'unix', 'vsock' >>> and 'fd' already. I'm fuzzy on best way to represent RDMA. >>> >>> >>> IIUC, the desire of migration maintainers is that we can ultimately >>> have multifd as the preferred, or even only, mechanism. Aside from >>> the main outbound migration control channel, and the multifd >>> data channels, IIUC we have a potential desire to have more channels >>> for post-copy async requests. >>> >>> This all suggests to me a more general representation along the >>> lines of: >>> >>> { 'enum': 'MigrateChannelType', >>> 'data': ['control', 'data', 'async'] } >>> >>> { 'struct': 'MigrateChannel', >>> 'data': { >>> 'type': 'MigrateChannelType', >>> 'src-addr': 'MigrateAddress', >>> 'dst-addr': 'MigrateAddress', >>> 'count': 'int', >>> } } >>> >>> { 'comand': 'migrate', >>> 'data': { >>> '*uri': 'str' >>> '*channels': ['MigrateChannel'] >>> } >>> } >>> >>> With 'uri' and 'channels' being mutually exclusive here. >>> >>> This whole scheme brings in redundancy wrt to the 'migrate-set-parameters' >>> API wrt multifd - essentally the same data is now being set in two >>> different places. IMHO, we should declare the 'multifd' capability >>> and the 'multifd-chanels' parameter deprecated, since the information >>> they provide is totally redundant, if you're giving an explicit list >>> of channels to 'migrate'. >> Hi Daniel. Initially while brainstorming this idea for the first time, we also came up with the same thought of depricating the migrate >> API. But how will we achieve this now and how is it going to work. Is it like we will be making migate V2 APIs initially, integrate it and then >> depricate the old one? would be happy to get some pointers from your end. > Migration maintainers, please advise. Hi Daniel and other migraton maintainers: Dr. David and Juan, what is your opinion on this. And how can we go forward implementing this. Some pointers and ideas from your end would be helpful too. Regards, Het Gala
On Thu, Jul 28, 2022 at 08:32:39PM +0530, Het Gala wrote: > > On 26/07/22 4:43 pm, Daniel P. Berrangé wrote: > > On Thu, Jul 21, 2022 at 07:56:15PM +0000, Het Gala wrote: > > > i) Modified the format of the qemu monitor command : 'migrate' by adding a list, > > > each element in the list consisting of multifd connection parameters: source > > > uri, destination uri and of the number of multifd channels between each pair. > > > > > > ii) Information of all multifd connection parameters' list and length of the > > > list is stored in 'OutgoingMigrateParams' struct. > > > > > > Suggested-by: Manish Mishra <manish.mishra@nutanix.com> > > > Signed-off-by: Het Gala <het.gala@nutanix.com> > > > --- > > > migration/migration.c | 52 +++++++++++++++++++++++++++++-------- > > > migration/socket.c | 60 ++++++++++++++++++++++++++++++++++++++++--- > > > migration/socket.h | 19 +++++++++++++- > > > monitor/hmp-cmds.c | 1 + > > > qapi/migration.json | 47 +++++++++++++++++++++++++++++---- > > > 5 files changed, 160 insertions(+), 19 deletions(-) > > > > > > diff --git a/qapi/migration.json b/qapi/migration.json > > > index 81185d4311..456247af8f 100644 > > > --- a/qapi/migration.json > > > +++ b/qapi/migration.json > > > @@ -1449,12 +1449,37 @@ > > > ## > > > { 'command': 'migrate-continue', 'data': {'state': 'MigrationStatus'} } > > > +## > > > +# @MigrateUriParameter: > > > +# > > > +# Information regarding which source interface is connected to which > > > +# destination interface and number of multifd channels over each interface. > > > +# > > > +# @source-uri: uri of the source VM. Default port number is 0. > > > +# > > > +# @destination-uri: uri of the destination VM > > > +# > > > +# @multifd-channels: number of parallel multifd channels used to migrate data > > > +# for specific source-uri and destination-uri. Default value > > > +# in this case is 2 (Since 7.1) > > > +# > > > +## > > > +{ 'struct' : 'MigrateUriParameter', > > > + 'data' : { 'source-uri' : 'str', > > > + 'destination-uri' : 'str', > > > + '*multifd-channels' : 'uint8'} } > > > + > > > ## > > > # @migrate: > > > # > > > # Migrates the current running guest to another Virtual Machine. > > > # > > > # @uri: the Uniform Resource Identifier of the destination VM > > > +# for migration thread > > > +# > > > +# @multi-fd-uri-list: list of pair of source and destination VM Uniform > > > +# Resource Identifiers with number of multifd-channels > > > +# for each pair > > > # > > > # @blk: do block migration (full disk copy) > > > # > > > @@ -1474,20 +1499,32 @@ > > > # 1. The 'query-migrate' command should be used to check migration's progress > > > # and final result (this information is provided by the 'status' member) > > > # > > > -# 2. All boolean arguments default to false > > > +# 2. The uri argument should have the Uniform Resource Identifier of default > > > +# destination VM. This connection will be bound to default network > > > # > > > -# 3. The user Monitor's "detach" argument is invalid in QMP and should not > > > +# 3. All boolean arguments default to false > > > +# > > > +# 4. The user Monitor's "detach" argument is invalid in QMP and should not > > > # be used > > > # > > > # Example: > > > # > > > -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } > > > +# -> { "execute": "migrate", > > > +# "arguments": { > > > +# "uri": "tcp:0:4446", > > > +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", > > > +# "destination-uri": "tcp:0:4480", > > > +# "multifd-channels": 4}, > > > +# { "source-uri": "tcp:10.0.0.0: ", > > > +# "destination-uri": "tcp:11.0.0.0:7789", > > > +# "multifd-channels": 5} ] } } > > > # <- { "return": {} } > > > # > > > ## > > > { 'command': 'migrate', > > > - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', > > > - '*detach': 'bool', '*resume': 'bool' } } > > > + 'data': {'uri': 'str', '*multi-fd-uri-list': ['MigrateUriParameter'], > > > + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool', > > > + '*resume': 'bool' } } > > Considering the existing migrate API from a QAPI design POV, I > > think there are several significant flaws with it > > > > The use of URIs is the big red flag. It is basically a data encoding > > scheme within a data encoding scheme. QEMU code should be able to > > directly work with the results from QAPI, without having todo a > > second level of parsing. > > > > URIs made sense in the context of HMP or the QemuOpts CLI, but do not > > make sense in QMP. We made a mistake in this respect when we first > > introduced QMP and implemented 'migrate'. > > > > If we going to extend the migrate API I think we should stop using URIs > > for the new fields, and instead define a QAPI discriminated union for > > the different data transport backends we offer. > > > > { 'enum': 'MigrateTransport', > > 'data': ['socket', 'exec'] } > > > > { 'union': 'MigrateAddress', > > 'base': { 'transport': 'MigrateTransport'}, > > 'discriminator': 'transport', > > 'data': { > > 'socket': 'SocketAddress', > > 'exec': ['str'], > > } > > > > NB, 'socket' should be able to cover all of 'tcp', 'unix', 'vsock' > > and 'fd' already. I'm fuzzy on best way to represent RDMA. > > > > > > IIUC, the desire of migration maintainers is that we can ultimately > > have multifd as the preferred, or even only, mechanism. Aside from > > the main outbound migration control channel, and the multifd > > data channels, IIUC we have a potential desire to have more channels > > for post-copy async requests. > > > > This all suggests to me a more general representation along the > > lines of: > > > > { 'enum': 'MigrateChannelType', > > 'data': ['control', 'data', 'async'] } > > > > { 'struct': 'MigrateChannel', > > 'data': { > > 'type': 'MigrateChannelType', > > 'src-addr': 'MigrateAddress', > > 'dst-addr': 'MigrateAddress', > > 'count': 'int', > > } } > > > > { 'comand': 'migrate', > > 'data': { > > '*uri': 'str' > > '*channels': ['MigrateChannel'] > > } > > } > > > > With 'uri' and 'channels' being mutually exclusive here. > > > > This whole scheme brings in redundancy wrt to the 'migrate-set-parameters' > > API wrt multifd - essentally the same data is now being set in two > > different places. IMHO, we should declare the 'multifd' capability > > and the 'multifd-chanels' parameter deprecated, since the information > > they provide is totally redundant, if you're giving an explicit list > > of channels to 'migrate'. > > Hi Daniel. Initially while brainstorming this idea for the first time, we > also came up with the same thought of depricating the migrate API. But how > will we achieve this now and how is it going to work. Is it like we will be > making migate V2 APIs initially, integrate it and then depricate the old > one? would be happy to get some pointers from your end. I don't think we need to replace 'migrate' with 'migrate2', because our QAPI deprecation & feature addition policy lets us evolve the existing commands over time. We currently have { 'comand': 'migrate', 'data': { 'uri': 'str' } } Our ABI policy lets us make 'uri' optional, and as well as adding new optional fields: { 'comand': 'migrate', 'data': { '*uri': 'str' '*channels': ['MigrateChannel'] } } At some point we can choose to deprecate 'uri' entirely and just ask people to use a single element 'channels' array. After having 'uri' deprecated for a while we can delete it and make 'channels' mandatory { 'comand': 'migrate', 'data': { 'channels': ['MigrateChannel'] } } This gives us the same end point as if we had added { 'comand': 'migrate2', 'data': { 'channels': ['MigrateChannel'] } } and deprecated &deleted 'migrate' entirely, but without leaving us with the ugly naming. FWIW, if I was going to majorly refactor migration QAPI design, I would also seek to get rid of 'migrate-set-capability' entirely and just make it a parameter to 'migrate', and likewise for the same for any of 'migrate-set-parameter' that are only used as upfront tunables. We should only need 'migrate-set-parameter' for runtime tunables that need setting /after/ migration has already started. I didn't mention this before though, as I didn't want to impose a big chunk of extra work on this feature of yours, as it can be done separately at any time. With regards, Daniel
On Mon, Aug 08, 2022 at 11:41:21AM +0530, Het Gala wrote: > > On 02/08/22 1:23 pm, Markus Armbruster wrote: > > Het Gala<het.gala@nutanix.com> writes: > > > > > On 26/07/22 4:43 pm, Daniel P. Berrangé wrote: > > > > On Thu, Jul 21, 2022 at 07:56:15PM +0000, Het Gala wrote: > > > > > i) Modified the format of the qemu monitor command : 'migrate' by adding a list, > > > > > each element in the list consisting of multifd connection parameters: source > > > > > uri, destination uri and of the number of multifd channels between each pair. > > > > > > > > > > ii) Information of all multifd connection parameters' list and length of the > > > > > list is stored in 'OutgoingMigrateParams' struct. > > > > > > > > > > Suggested-by: Manish Mishra<manish.mishra@nutanix.com> > > > > > Signed-off-by: Het Gala<het.gala@nutanix.com> > > > > > --- > > > > > migration/migration.c | 52 +++++++++++++++++++++++++++++-------- > > > > > migration/socket.c | 60 ++++++++++++++++++++++++++++++++++++++++--- > > > > > migration/socket.h | 19 +++++++++++++- > > > > > monitor/hmp-cmds.c | 1 + > > > > > qapi/migration.json | 47 +++++++++++++++++++++++++++++---- > > > > > 5 files changed, 160 insertions(+), 19 deletions(-) > > > > > > > > > > diff --git a/qapi/migration.json b/qapi/migration.json > > > > > index 81185d4311..456247af8f 100644 > > > > > --- a/qapi/migration.json > > > > > +++ b/qapi/migration.json > > > > > @@ -1449,12 +1449,37 @@ > > > > > ## > > > > > { 'command': 'migrate-continue', 'data': {'state': 'MigrationStatus'} } > > > > > +## > > > > > +# @MigrateUriParameter: > > > > > +# > > > > > +# Information regarding which source interface is connected to which > > > > > +# destination interface and number of multifd channels over each interface. > > > > > +# > > > > > +# @source-uri: uri of the source VM. Default port number is 0. > > > > > +# > > > > > +# @destination-uri: uri of the destination VM > > > > > +# > > > > > +# @multifd-channels: number of parallel multifd channels used to migrate data > > > > > +# for specific source-uri and destination-uri. Default value > > > > > +# in this case is 2 (Since 7.1) > > > > > +# > > > > > +## > > > > > +{ 'struct' : 'MigrateUriParameter', > > > > > + 'data' : { 'source-uri' : 'str', > > > > > + 'destination-uri' : 'str', > > > > > + '*multifd-channels' : 'uint8'} } > > > > > + > > > > > ## > > > > > # @migrate: > > > > > # > > > > > # Migrates the current running guest to another Virtual Machine. > > > > > # > > > > > # @uri: the Uniform Resource Identifier of the destination VM > > > > > +# for migration thread > > > > > +# > > > > > +# @multi-fd-uri-list: list of pair of source and destination VM Uniform > > > > > +# Resource Identifiers with number of multifd-channels > > > > > +# for each pair > > > > > # > > > > > # @blk: do block migration (full disk copy) > > > > > # > > > > > @@ -1474,20 +1499,32 @@ > > > > > # 1. The 'query-migrate' command should be used to check migration's progress > > > > > # and final result (this information is provided by the 'status' member) > > > > > # > > > > > -# 2. All boolean arguments default to false > > > > > +# 2. The uri argument should have the Uniform Resource Identifier of default > > > > > +# destination VM. This connection will be bound to default network > > > > > # > > > > > -# 3. The user Monitor's "detach" argument is invalid in QMP and should not > > > > > +# 3. All boolean arguments default to false > > > > > +# > > > > > +# 4. The user Monitor's "detach" argument is invalid in QMP and should not > > > > > # be used > > > > > # > > > > > # Example: > > > > > # > > > > > -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } > > > > > +# -> { "execute": "migrate", > > > > > +# "arguments": { > > > > > +# "uri": "tcp:0:4446", > > > > > +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", > > > > > +# "destination-uri": "tcp:0:4480", > > > > > +# "multifd-channels": 4}, > > > > > +# { "source-uri": "tcp:10.0.0.0: ", > > > > > +# "destination-uri": "tcp:11.0.0.0:7789", > > > > > +# "multifd-channels": 5} ] } } > > > > > # <- { "return": {} } > > > > > # > > > > > ## > > > > > { 'command': 'migrate', > > > > > - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', > > > > > - '*detach': 'bool', '*resume': 'bool' } } > > > > > + 'data': {'uri': 'str', '*multi-fd-uri-list': ['MigrateUriParameter'], > > > > > + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool', > > > > > + '*resume': 'bool' } } > > > > Considering the existing migrate API from a QAPI design POV, I > > > > think there are several significant flaws with it > > > > > > > > The use of URIs is the big red flag. It is basically a data encoding > > > > scheme within a data encoding scheme. QEMU code should be able to > > > > directly work with the results from QAPI, without having todo a > > > > second level of parsing. > > Concur. > > > > > > URIs made sense in the context of HMP or the QemuOpts CLI, but do not > > > > make sense in QMP. We made a mistake in this respect when we first > > > > introduced QMP and implemented 'migrate'. > > > > > > > > If we going to extend the migrate API I think we should stop using URIs > > > > for the new fields, and instead define a QAPI discriminated union for > > > > the different data transport backends we offer. > > > > > > > > { 'enum': 'MigrateTransport', > > > > 'data': ['socket', 'exec'] } > > > > > > > > { 'union': 'MigrateAddress', > > > > 'base': { 'transport': 'MigrateTransport'}, > > > > 'discriminator': 'transport', > > > > 'data': { > > > > 'socket': 'SocketAddress', > > > > 'exec': ['str'], > > > > } > > > > > > > > NB, 'socket' should be able to cover all of 'tcp', 'unix', 'vsock' > > > > and 'fd' already. I'm fuzzy on best way to represent RDMA. > > > > > > > > > > > > IIUC, the desire of migration maintainers is that we can ultimately > > > > have multifd as the preferred, or even only, mechanism. Aside from > > > > the main outbound migration control channel, and the multifd > > > > data channels, IIUC we have a potential desire to have more channels > > > > for post-copy async requests. > > > > > > > > This all suggests to me a more general representation along the > > > > lines of: > > > > > > > > { 'enum': 'MigrateChannelType', > > > > 'data': ['control', 'data', 'async'] } > > > > > > > > { 'struct': 'MigrateChannel', > > > > 'data': { > > > > 'type': 'MigrateChannelType', > > > > 'src-addr': 'MigrateAddress', > > > > 'dst-addr': 'MigrateAddress', > > > > 'count': 'int', > > > > } } > > > > > > > > { 'comand': 'migrate', > > > > 'data': { > > > > '*uri': 'str' > > > > '*channels': ['MigrateChannel'] > > > > } > > > > } > > > > > > > > With 'uri' and 'channels' being mutually exclusive here. > > > > > > > > This whole scheme brings in redundancy wrt to the 'migrate-set-parameters' > > > > API wrt multifd - essentally the same data is now being set in two > > > > different places. IMHO, we should declare the 'multifd' capability > > > > and the 'multifd-chanels' parameter deprecated, since the information > > > > they provide is totally redundant, if you're giving an explicit list > > > > of channels to 'migrate'. > > > Hi Daniel. Initially while brainstorming this idea for the first time, we also came up with the same thought of depricating the migrate > > > API. But how will we achieve this now and how is it going to work. Is it like we will be making migate V2 APIs initially, integrate it and then > > > depricate the old one? would be happy to get some pointers from your end. > > Migration maintainers, please advise. > > Hi Daniel and other migraton maintainers: Dr. David and Juan, what > is your opinion on this. And how can we go forward implementing > this. Some pointers and ideas from your end would be helpful too. Note, i'm not a migration maintainer, just offering advice from the sidelines :-) With regards, Daniel
On 08/08/22 11:41 am, Het Gala wrote: > > > On 02/08/22 1:23 pm, Markus Armbruster wrote: >> Het Gala<het.gala@nutanix.com> writes: >> >>> On 26/07/22 4:43 pm, Daniel P. Berrangé wrote: >>>> On Thu, Jul 21, 2022 at 07:56:15PM +0000, Het Gala wrote: >>>>> i) Modified the format of the qemu monitor command : 'migrate' by adding a list, >>>>> each element in the list consisting of multifd connection parameters: source >>>>> uri, destination uri and of the number of multifd channels between each pair. >>>>> >>>>> ii) Information of all multifd connection parameters' list and length of the >>>>> list is stored in 'OutgoingMigrateParams' struct. >>>>> >>>>> Suggested-by: Manish Mishra<manish.mishra@nutanix.com> >>>>> Signed-off-by: Het Gala<het.gala@nutanix.com> >>>>> --- >>>>> migration/migration.c | 52 +++++++++++++++++++++++++++++-------- >>>>> migration/socket.c | 60 ++++++++++++++++++++++++++++++++++++++++--- >>>>> migration/socket.h | 19 +++++++++++++- >>>>> monitor/hmp-cmds.c | 1 + >>>>> qapi/migration.json | 47 +++++++++++++++++++++++++++++---- >>>>> 5 files changed, 160 insertions(+), 19 deletions(-) >>>>> >>>>> diff --git a/qapi/migration.json b/qapi/migration.json >>>>> index 81185d4311..456247af8f 100644 >>>>> --- a/qapi/migration.json >>>>> +++ b/qapi/migration.json >>>>> @@ -1449,12 +1449,37 @@ >>>>> ## >>>>> { 'command': 'migrate-continue', 'data': {'state': 'MigrationStatus'} } >>>>> +## >>>>> +# @MigrateUriParameter: >>>>> +# >>>>> +# Information regarding which source interface is connected to which >>>>> +# destination interface and number of multifd channels over each interface. >>>>> +# >>>>> +# @source-uri: uri of the source VM. Default port number is 0. >>>>> +# >>>>> +# @destination-uri: uri of the destination VM >>>>> +# >>>>> +# @multifd-channels: number of parallel multifd channels used to migrate data >>>>> +# for specific source-uri and destination-uri. Default value >>>>> +# in this case is 2 (Since 7.1) >>>>> +# >>>>> +## >>>>> +{ 'struct' : 'MigrateUriParameter', >>>>> + 'data' : { 'source-uri' : 'str', >>>>> + 'destination-uri' : 'str', >>>>> + '*multifd-channels' : 'uint8'} } >>>>> + >>>>> ## >>>>> # @migrate: >>>>> # >>>>> # Migrates the current running guest to another Virtual Machine. >>>>> # >>>>> # @uri: the Uniform Resource Identifier of the destination VM >>>>> +# for migration thread >>>>> +# >>>>> +# @multi-fd-uri-list: list of pair of source and destination VM Uniform >>>>> +# Resource Identifiers with number of multifd-channels >>>>> +# for each pair >>>>> # >>>>> # @blk: do block migration (full disk copy) >>>>> # >>>>> @@ -1474,20 +1499,32 @@ >>>>> # 1. The 'query-migrate' command should be used to check migration's progress >>>>> # and final result (this information is provided by the 'status' member) >>>>> # >>>>> -# 2. All boolean arguments default to false >>>>> +# 2. The uri argument should have the Uniform Resource Identifier of default >>>>> +# destination VM. This connection will be bound to default network >>>>> # >>>>> -# 3. The user Monitor's "detach" argument is invalid in QMP and should not >>>>> +# 3. All boolean arguments default to false >>>>> +# >>>>> +# 4. The user Monitor's "detach" argument is invalid in QMP and should not >>>>> # be used >>>>> # >>>>> # Example: >>>>> # >>>>> -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } >>>>> +# -> { "execute": "migrate", >>>>> +# "arguments": { >>>>> +# "uri": "tcp:0:4446", >>>>> +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", >>>>> +# "destination-uri": "tcp:0:4480", >>>>> +# "multifd-channels": 4}, >>>>> +# { "source-uri": "tcp:10.0.0.0: ", >>>>> +# "destination-uri": "tcp:11.0.0.0:7789", >>>>> +# "multifd-channels": 5} ] } } >>>>> # <- { "return": {} } >>>>> # >>>>> ## >>>>> { 'command': 'migrate', >>>>> - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', >>>>> - '*detach': 'bool', '*resume': 'bool' } } >>>>> + 'data': {'uri': 'str', '*multi-fd-uri-list': ['MigrateUriParameter'], >>>>> + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool', >>>>> + '*resume': 'bool' } } >>>> Considering the existing migrate API from a QAPI design POV, I >>>> think there are several significant flaws with it >>>> >>>> The use of URIs is the big red flag. It is basically a data encoding >>>> scheme within a data encoding scheme. QEMU code should be able to >>>> directly work with the results from QAPI, without having todo a >>>> second level of parsing. >> Concur. >> >>>> URIs made sense in the context of HMP or the QemuOpts CLI, but do not >>>> make sense in QMP. We made a mistake in this respect when we first >>>> introduced QMP and implemented 'migrate'. >>>> >>>> If we going to extend the migrate API I think we should stop using URIs >>>> for the new fields, and instead define a QAPI discriminated union for >>>> the different data transport backends we offer. >>>> >>>> { 'enum': 'MigrateTransport', >>>> 'data': ['socket', 'exec'] } >>>> >>>> { 'union': 'MigrateAddress', >>>> 'base': { 'transport': 'MigrateTransport'}, >>>> 'discriminator': 'transport', >>>> 'data': { >>>> 'socket': 'SocketAddress', >>>> 'exec': ['str'], >>>> } >>>> >>>> NB, 'socket' should be able to cover all of 'tcp', 'unix', 'vsock' >>>> and 'fd' already. I'm fuzzy on best way to represent RDMA. >>>> >>>> >>>> IIUC, the desire of migration maintainers is that we can ultimately >>>> have multifd as the preferred, or even only, mechanism. Aside from >>>> the main outbound migration control channel, and the multifd >>>> data channels, IIUC we have a potential desire to have more channels >>>> for post-copy async requests. >>>> >>>> This all suggests to me a more general representation along the >>>> lines of: >>>> >>>> { 'enum': 'MigrateChannelType', >>>> 'data': ['control', 'data', 'async'] } >>>> >>>> { 'struct': 'MigrateChannel', >>>> 'data': { >>>> 'type': 'MigrateChannelType', >>>> 'src-addr': 'MigrateAddress', >>>> 'dst-addr': 'MigrateAddress', >>>> 'count': 'int', >>>> } } >>>> >>>> { 'comand': 'migrate', >>>> 'data': { >>>> '*uri': 'str' >>>> '*channels': ['MigrateChannel'] >>>> } >>>> } >>>> >>>> With 'uri' and 'channels' being mutually exclusive here. >>>> >>>> This whole scheme brings in redundancy wrt to the 'migrate-set-parameters' >>>> API wrt multifd - essentally the same data is now being set in two >>>> different places. IMHO, we should declare the 'multifd' capability >>>> and the 'multifd-chanels' parameter deprecated, since the information >>>> they provide is totally redundant, if you're giving an explicit list >>>> of channels to 'migrate'. >>> Hi Daniel. Initially while brainstorming this idea for the first time, we also came up with the same thought of depricating the migrate >>> API. But how will we achieve this now and how is it going to work. Is it like we will be making migate V2 APIs initially, integrate it and then >>> depricate the old one? would be happy to get some pointers from your end. >> Migration maintainers, please advise. > Hi Daniel and other migraton maintainers: Dr. David and Juan, what is your opinion on this. And how can we go forward implementing this. Some pointers and ideas from your end would be helpful too. > Regards, > Het Gala Just a gentle reminder Dr. David and Juan. Daniel already expressed his opinion to refactor the QAPI design for multifd. Your inputs and advice will also be very valuable to us. Thankyou. Regards, Het Gala
On 29/08/22 10:04 am, Het Gala wrote: > > > On 08/08/22 11:41 am, Het Gala wrote: >> >> >> On 02/08/22 1:23 pm, Markus Armbruster wrote: >>> Het Gala<het.gala@nutanix.com> writes: >>> >>>> On 26/07/22 4:43 pm, Daniel P. Berrangé wrote: >>>>> On Thu, Jul 21, 2022 at 07:56:15PM +0000, Het Gala wrote: >>>>>> i) Modified the format of the qemu monitor command : 'migrate' by adding a list, >>>>>> each element in the list consisting of multifd connection parameters: source >>>>>> uri, destination uri and of the number of multifd channels between each pair. >>>>>> >>>>>> ii) Information of all multifd connection parameters' list and length of the >>>>>> list is stored in 'OutgoingMigrateParams' struct. >>>>>> >>>>>> Suggested-by: Manish Mishra<manish.mishra@nutanix.com> >>>>>> Signed-off-by: Het Gala<het.gala@nutanix.com> >>>>>> --- >>>>>> migration/migration.c | 52 +++++++++++++++++++++++++++++-------- >>>>>> migration/socket.c | 60 ++++++++++++++++++++++++++++++++++++++++--- >>>>>> migration/socket.h | 19 +++++++++++++- >>>>>> monitor/hmp-cmds.c | 1 + >>>>>> qapi/migration.json | 47 +++++++++++++++++++++++++++++---- >>>>>> 5 files changed, 160 insertions(+), 19 deletions(-) >>>>>> >>>>>> diff --git a/qapi/migration.json b/qapi/migration.json >>>>>> index 81185d4311..456247af8f 100644 >>>>>> --- a/qapi/migration.json >>>>>> +++ b/qapi/migration.json >>>>>> @@ -1449,12 +1449,37 @@ >>>>>> ## >>>>>> { 'command': 'migrate-continue', 'data': {'state': 'MigrationStatus'} } >>>>>> +## >>>>>> +# @MigrateUriParameter: >>>>>> +# >>>>>> +# Information regarding which source interface is connected to which >>>>>> +# destination interface and number of multifd channels over each interface. >>>>>> +# >>>>>> +# @source-uri: uri of the source VM. Default port number is 0. >>>>>> +# >>>>>> +# @destination-uri: uri of the destination VM >>>>>> +# >>>>>> +# @multifd-channels: number of parallel multifd channels used to migrate data >>>>>> +# for specific source-uri and destination-uri. Default value >>>>>> +# in this case is 2 (Since 7.1) >>>>>> +# >>>>>> +## >>>>>> +{ 'struct' : 'MigrateUriParameter', >>>>>> + 'data' : { 'source-uri' : 'str', >>>>>> + 'destination-uri' : 'str', >>>>>> + '*multifd-channels' : 'uint8'} } >>>>>> + >>>>>> ## >>>>>> # @migrate: >>>>>> # >>>>>> # Migrates the current running guest to another Virtual Machine. >>>>>> # >>>>>> # @uri: the Uniform Resource Identifier of the destination VM >>>>>> +# for migration thread >>>>>> +# >>>>>> +# @multi-fd-uri-list: list of pair of source and destination VM Uniform >>>>>> +# Resource Identifiers with number of multifd-channels >>>>>> +# for each pair >>>>>> # >>>>>> # @blk: do block migration (full disk copy) >>>>>> # >>>>>> @@ -1474,20 +1499,32 @@ >>>>>> # 1. The 'query-migrate' command should be used to check migration's progress >>>>>> # and final result (this information is provided by the 'status' member) >>>>>> # >>>>>> -# 2. All boolean arguments default to false >>>>>> +# 2. The uri argument should have the Uniform Resource Identifier of default >>>>>> +# destination VM. This connection will be bound to default network >>>>>> # >>>>>> -# 3. The user Monitor's "detach" argument is invalid in QMP and should not >>>>>> +# 3. All boolean arguments default to false >>>>>> +# >>>>>> +# 4. The user Monitor's "detach" argument is invalid in QMP and should not >>>>>> # be used >>>>>> # >>>>>> # Example: >>>>>> # >>>>>> -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } >>>>>> +# -> { "execute": "migrate", >>>>>> +# "arguments": { >>>>>> +# "uri": "tcp:0:4446", >>>>>> +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", >>>>>> +# "destination-uri": "tcp:0:4480", >>>>>> +# "multifd-channels": 4}, >>>>>> +# { "source-uri": "tcp:10.0.0.0: ", >>>>>> +# "destination-uri": "tcp:11.0.0.0:7789", >>>>>> +# "multifd-channels": 5} ] } } >>>>>> # <- { "return": {} } >>>>>> # >>>>>> ## >>>>>> { 'command': 'migrate', >>>>>> - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', >>>>>> - '*detach': 'bool', '*resume': 'bool' } } >>>>>> + 'data': {'uri': 'str', '*multi-fd-uri-list': ['MigrateUriParameter'], >>>>>> + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool', >>>>>> + '*resume': 'bool' } } >>>>> Considering the existing migrate API from a QAPI design POV, I >>>>> think there are several significant flaws with it >>>>> >>>>> The use of URIs is the big red flag. It is basically a data encoding >>>>> scheme within a data encoding scheme. QEMU code should be able to >>>>> directly work with the results from QAPI, without having todo a >>>>> second level of parsing. >>> Concur. >>> >>>>> URIs made sense in the context of HMP or the QemuOpts CLI, but do not >>>>> make sense in QMP. We made a mistake in this respect when we first >>>>> introduced QMP and implemented 'migrate'. >>>>> >>>>> If we going to extend the migrate API I think we should stop using URIs >>>>> for the new fields, and instead define a QAPI discriminated union for >>>>> the different data transport backends we offer. >>>>> >>>>> { 'enum': 'MigrateTransport', >>>>> 'data': ['socket', 'exec'] } >>>>> >>>>> { 'union': 'MigrateAddress', >>>>> 'base': { 'transport': 'MigrateTransport'}, >>>>> 'discriminator': 'transport', >>>>> 'data': { >>>>> 'socket': 'SocketAddress', >>>>> 'exec': ['str'], >>>>> } >>>>> >>>>> NB, 'socket' should be able to cover all of 'tcp', 'unix', 'vsock' >>>>> and 'fd' already. I'm fuzzy on best way to represent RDMA. >>>>> >>>>> >>>>> IIUC, the desire of migration maintainers is that we can ultimately >>>>> have multifd as the preferred, or even only, mechanism. Aside from >>>>> the main outbound migration control channel, and the multifd >>>>> data channels, IIUC we have a potential desire to have more channels >>>>> for post-copy async requests. >>>>> >>>>> This all suggests to me a more general representation along the >>>>> lines of: >>>>> >>>>> { 'enum': 'MigrateChannelType', >>>>> 'data': ['control', 'data', 'async'] } >>>>> >>>>> { 'struct': 'MigrateChannel', >>>>> 'data': { >>>>> 'type': 'MigrateChannelType', >>>>> 'src-addr': 'MigrateAddress', >>>>> 'dst-addr': 'MigrateAddress', >>>>> 'count': 'int', >>>>> } } >>>>> >>>>> { 'comand': 'migrate', >>>>> 'data': { >>>>> '*uri': 'str' >>>>> '*channels': ['MigrateChannel'] >>>>> } >>>>> } >>>>> >>>>> With 'uri' and 'channels' being mutually exclusive here. >>>>> >>>>> This whole scheme brings in redundancy wrt to the 'migrate-set-parameters' >>>>> API wrt multifd - essentally the same data is now being set in two >>>>> different places. IMHO, we should declare the 'multifd' capability >>>>> and the 'multifd-chanels' parameter deprecated, since the information >>>>> they provide is totally redundant, if you're giving an explicit list >>>>> of channels to 'migrate'. >>>> Hi Daniel. Initially while brainstorming this idea for the first time, we also came up with the same thought of depricating the migrate >>>> API. But how will we achieve this now and how is it going to work. Is it like we will be making migate V2 APIs initially, integrate it and then >>>> depricate the old one? would be happy to get some pointers from your end. >>> Migration maintainers, please advise. >> Hi Daniel and other migraton maintainers: Dr. David and Juan, what is your opinion on this. And how can we go forward implementing this. Some pointers and ideas from your end would be helpful too. >> Regards, >> Het Gala > > Just a gentle reminder Dr. David and Juan. Daniel already expressed > his opinion to refactor the QAPI design for multifd. Your inputs and > advice will also be very valuable to us. Thankyou. > > Regards, > Het Gala Hi all, I think the maintainers were occupied for quite sometime by KVM forum work. This is again a gentle reminder mail for the migration maintainers - Dr. David and Juan to share some ideas and inputs on refactorisation of QAPI design for multifd suggested by Daniel, and how this idea will pan out in the migration workflow. Looking forward for a positive discussion on this topic :) Regards, Het Gala
Markus Armbruster <armbru@redhat.com> wrote: > Het Gala <het.gala@nutanix.com> writes: Hi >>>> # Example: >>>> # >>>> -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } >>>> +# -> { "execute": "migrate", >>>> +# "arguments": { >>>> +# "uri": "tcp:0:4446", >>>> +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", >>>> +# "destination-uri": "tcp:0:4480", >>>> +# "multifd-channels": 4}, >>>> +# { "source-uri": "tcp:10.0.0.0: ", >>>> +# "destination-uri": "tcp:11.0.0.0:7789", >>>> +# "multifd-channels": 5} ] } } Why would one put the source uri and destination uri on the command? It looks more complicated to me, but I guess there is a good reason. >>> >>> This whole scheme brings in redundancy wrt to the 'migrate-set-parameters' >>> API wrt multifd - essentally the same data is now being set in two >>> different places. IMHO, we should declare the 'multifd' capability >>> and the 'multifd-chanels' parameter deprecated, since the information >>> they provide is totally redundant, if you're giving an explicit list >>> of channels to 'migrate'. >> >> Hi Daniel. Initially while brainstorming this idea for the first >> time, we also came up with the same thought of depricating the >> migrate >> API. But how will we achieve this now and how is it going to >> work. Is it like we will be making migate V2 APIs initially, >> integrate it and then >> depricate the old one? would be happy to get some pointers from your end. > > Migration maintainers, please advise. I would put the old one in top of the new one, and call it a day. I really hate the old one, but I haven't had the time to think about a better one (nor I have had the time to look into this one). The problem that I am seing here is that we are adding the number of multifd channels here, and we were trying to not add migration parameters into the migrate command. BTW, once that we are at it, I guess we can just drop the inc/blk parameters, we have had them deprecated ... forever? Later, Juan.
On Mon, Nov 21, 2022 at 01:26:55PM +0100, Juan Quintela wrote: > Markus Armbruster <armbru@redhat.com> wrote: > > Het Gala <het.gala@nutanix.com> writes: > > > > Hi > > >>>> # Example: > >>>> # > >>>> -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } > >>>> +# -> { "execute": "migrate", > >>>> +# "arguments": { > >>>> +# "uri": "tcp:0:4446", > >>>> +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", > >>>> +# "destination-uri": "tcp:0:4480", > >>>> +# "multifd-channels": 4}, > >>>> +# { "source-uri": "tcp:10.0.0.0: ", > >>>> +# "destination-uri": "tcp:11.0.0.0:7789", > >>>> +# "multifd-channels": 5} ] } } > > Why would one put the source uri and destination uri on the command? > It looks more complicated to me, but I guess there is a good reason. > > >>> > >>> This whole scheme brings in redundancy wrt to the 'migrate-set-parameters' > >>> API wrt multifd - essentally the same data is now being set in two > >>> different places. IMHO, we should declare the 'multifd' capability > >>> and the 'multifd-chanels' parameter deprecated, since the information > >>> they provide is totally redundant, if you're giving an explicit list > >>> of channels to 'migrate'. > >> > >> Hi Daniel. Initially while brainstorming this idea for the first > >> time, we also came up with the same thought of depricating the > >> migrate > >> API. But how will we achieve this now and how is it going to > >> work. Is it like we will be making migate V2 APIs initially, > >> integrate it and then > >> depricate the old one? would be happy to get some pointers from your end. > > > > Migration maintainers, please advise. > > I would put the old one in top of the new one, and call it a day. > I really hate the old one, but I haven't had the time to think about a > better one (nor I have had the time to look into this one). > > The problem that I am seing here is that we are adding the number of > multifd channels here, and we were trying to not add migration > parameters into the migrate command. The issue of migration parameters is a much bigger one - a lot of them should never have existed, if QEMU had a proper migration wire protocol that could do feature negotiation. We need to replace the wire protocol as the priority, at which point the QMP side becomes simpler as a result. Starting with the QMP side, without addressing the wire protocol first will never give us a good long term result. I've written more about that in my reply to Het's other patch. With regards, Daniel
diff --git a/migration/migration.c b/migration/migration.c index e03f698a3c..572b909423 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2380,13 +2380,14 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, return true; } -void qmp_migrate(const char *uri, bool has_blk, bool blk, +void qmp_migrate(const char *uri, bool has_multi_fd_uri_list, + MigrateUriParameterList *cap, bool has_blk, bool blk, bool has_inc, bool inc, bool has_detach, bool detach, bool has_resume, bool resume, Error **errp) { Error *local_err = NULL; MigrationState *s = migrate_get_current(); - const char *p = NULL; + const char *dst_ptr = NULL; if (!migrate_prepare(s, has_blk && blk, has_inc && inc, has_resume && resume, errp)) { @@ -2400,20 +2401,51 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, } } + /* + * In case of Multi-FD migration, source and destination uri + * supports only tcp network protocol. + */ + if (has_multi_fd_uri_list) { + int length = QAPI_LIST_LENGTH(cap); + init_multifd_array(length); + for (int i = 0; i < length; i++) { + const char *pd = NULL, *ps = NULL; + const char *multifd_dst_uri = cap->value->destination_uri; + const char *multifd_src_uri = cap->value->source_uri; + uint8_t multifd_channels = cap->value->multifd_channels; + if (!strstart(multifd_dst_uri, "tcp:", &pd) || + !strstart(multifd_src_uri, "tcp:", &ps)) { + error_setg(errp, "multi-fd destination and multi-fd source " + "uri, both should be present and follows tcp protocol only"); + return; + } else { + store_multifd_migration_params(pd ? pd : multifd_dst_uri, + ps ? ps : multifd_src_uri, + multifd_channels, i, &local_err); + } + cap = cap->next; + } + + if (outgoing_param_total_multifds() != migrate_multifd_channels()) { + error_setg(errp, "Total multifd channel number mismatch"); + return; + } + } + migrate_protocol_allow_multi_channels(false); - if (strstart(uri, "tcp:", &p) || + if (strstart(uri, "tcp:", &dst_ptr) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { migrate_protocol_allow_multi_channels(true); - socket_start_outgoing_migration(s, p ? p : uri, &local_err); + socket_start_outgoing_migration(s, dst_ptr ? dst_ptr : uri, &local_err); #ifdef CONFIG_RDMA - } else if (strstart(uri, "rdma:", &p)) { - rdma_start_outgoing_migration(s, p, &local_err); + } else if (strstart(uri, "rdma:", &dst_ptr)) { + rdma_start_outgoing_migration(s, dst_ptr, &local_err); #endif - } else if (strstart(uri, "exec:", &p)) { - exec_start_outgoing_migration(s, p, &local_err); - } else if (strstart(uri, "fd:", &p)) { - fd_start_outgoing_migration(s, p, &local_err); + } else if (strstart(uri, "exec:", &dst_ptr)) { + exec_start_outgoing_migration(s, dst_ptr, &local_err); + } else if (strstart(uri, "fd:", &dst_ptr)) { + fd_start_outgoing_migration(s, dst_ptr, &local_err); } else { if (!(has_resume && resume)) { yank_unregister_instance(MIGRATION_YANK_INSTANCE); diff --git a/migration/socket.c b/migration/socket.c index e6fdf3c5e1..f199430bc2 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -32,6 +32,28 @@ struct SocketOutgoingArgs { SocketAddress *saddr; } outgoing_args; +struct SocketArgs { + struct SrcDestAddr address; + uint8_t multifd_channels; +}; + +struct OutgoingMigrateParams { + struct SocketArgs *socket_args; + size_t socket_args_arr_len; +} outgoing_migrate_params; + +int outgoing_param_total_multifds(void) +{ + size_t len = outgoing_migrate_params.socket_args_arr_len; + uint64_t total_multifd_channels = 0; + + for (int i = 0; i < len; i++) { + total_multifd_channels += + outgoing_migrate_params.socket_args[i].multifd_channels; + } + return total_multifd_channels; +} + void socket_send_channel_create(QIOTaskFunc f, void *data) { QIOChannelSocket *sioc = qio_channel_socket_new(); @@ -65,6 +87,9 @@ int socket_send_channel_destroy(QIOChannel *send) qapi_free_SocketAddress(outgoing_args.saddr); outgoing_args.saddr = NULL; } + g_free(outgoing_migrate_params.socket_args); + outgoing_migrate_params.socket_args = NULL; + outgoing_migrate_params.socket_args_arr_len = 0; return 0; } @@ -135,17 +160,46 @@ socket_start_outgoing_migration_internal(MigrationState *s, } void socket_start_outgoing_migration(MigrationState *s, - const char *str, + const char *dst_str, Error **errp) { Error *err = NULL; - SocketAddress *saddr = socket_parse(str, &err); + SocketAddress *dst_saddr = socket_parse(dst_str, &err); if (!err) { - socket_start_outgoing_migration_internal(s, saddr, &err); + socket_start_outgoing_migration_internal(s, dst_saddr, &err); } error_propagate(errp, err); } +void init_multifd_array(int length) +{ + outgoing_migrate_params.socket_args = g_new0(struct SocketArgs, length); + outgoing_migrate_params.socket_args_arr_len = length; +} + +void store_multifd_migration_params(const char *dst_uri, + const char *src_uri, + uint8_t multifd_channels, + int idx, Error **errp) +{ + struct SocketArgs *sa = &outgoing_migrate_params.socket_args[idx]; + SocketAddress *src_addr, *dst_addr; + + src_addr = socket_parse(src_uri, errp); + if (!src_addr) { + return; + } + + dst_addr = socket_parse(dst_uri, errp); + if (!dst_addr) { + return; + } + + sa->address.dst_addr = dst_addr; + sa->address.src_addr = src_addr; + sa->multifd_channels = multifd_channels; +} + static void socket_accept_incoming_migration(QIONetListener *listener, QIOChannelSocket *cioc, gpointer opaque) diff --git a/migration/socket.h b/migration/socket.h index dc54df4e6c..0cbb7220ac 100644 --- a/migration/socket.h +++ b/migration/socket.h @@ -19,13 +19,30 @@ #include "io/channel.h" #include "io/task.h" +#include "migration.h" +/* info regarding destination and source uri */ +typedef struct SrcDestAddr { + SocketAddress *dst_addr; + SocketAddress *src_addr; +} SrcDestAddr; + + +int outgoing_param_total_multifds(void); void socket_send_channel_create(QIOTaskFunc f, void *data); QIOChannel *socket_send_channel_create_sync(Error **errp); int socket_send_channel_destroy(QIOChannel *send); void socket_start_incoming_migration(const char *str, Error **errp); -void socket_start_outgoing_migration(MigrationState *s, const char *str, +void socket_start_outgoing_migration(MigrationState *s, const char *dst_str, Error **errp); + +int multifd_list_length(MigrateUriParameterList *list); + +void init_multifd_array(int length); + +void store_multifd_migration_params(const char *dst_uri, const char *src_uri, + uint8_t multifd_channels, int idx, + Error **erp); #endif diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 6bb6424215..8d25fee4be 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -59,6 +59,7 @@ #include "migration/snapshot.h" #include "migration/misc.h" + #ifdef CONFIG_SPICE #include <spice/enums.h> #endif diff --git a/qapi/migration.json b/qapi/migration.json index 81185d4311..456247af8f 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1449,12 +1449,37 @@ ## { 'command': 'migrate-continue', 'data': {'state': 'MigrationStatus'} } +## +# @MigrateUriParameter: +# +# Information regarding which source interface is connected to which +# destination interface and number of multifd channels over each interface. +# +# @source-uri: uri of the source VM. Default port number is 0. +# +# @destination-uri: uri of the destination VM +# +# @multifd-channels: number of parallel multifd channels used to migrate data +# for specific source-uri and destination-uri. Default value +# in this case is 2 (Since 7.1) +# +## +{ 'struct' : 'MigrateUriParameter', + 'data' : { 'source-uri' : 'str', + 'destination-uri' : 'str', + '*multifd-channels' : 'uint8'} } + ## # @migrate: # # Migrates the current running guest to another Virtual Machine. # # @uri: the Uniform Resource Identifier of the destination VM +# for migration thread +# +# @multi-fd-uri-list: list of pair of source and destination VM Uniform +# Resource Identifiers with number of multifd-channels +# for each pair # # @blk: do block migration (full disk copy) # @@ -1474,20 +1499,32 @@ # 1. The 'query-migrate' command should be used to check migration's progress # and final result (this information is provided by the 'status' member) # -# 2. All boolean arguments default to false +# 2. The uri argument should have the Uniform Resource Identifier of default +# destination VM. This connection will be bound to default network # -# 3. The user Monitor's "detach" argument is invalid in QMP and should not +# 3. All boolean arguments default to false +# +# 4. The user Monitor's "detach" argument is invalid in QMP and should not # be used # # Example: # -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } +# -> { "execute": "migrate", +# "arguments": { +# "uri": "tcp:0:4446", +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", +# "destination-uri": "tcp:0:4480", +# "multifd-channels": 4}, +# { "source-uri": "tcp:10.0.0.0: ", +# "destination-uri": "tcp:11.0.0.0:7789", +# "multifd-channels": 5} ] } } # <- { "return": {} } # ## { 'command': 'migrate', - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', - '*detach': 'bool', '*resume': 'bool' } } + 'data': {'uri': 'str', '*multi-fd-uri-list': ['MigrateUriParameter'], + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool', + '*resume': 'bool' } } ## # @migrate-incoming:
i) Modified the format of the qemu monitor command : 'migrate' by adding a list, each element in the list consisting of multifd connection parameters: source uri, destination uri and of the number of multifd channels between each pair. ii) Information of all multifd connection parameters' list and length of the list is stored in 'OutgoingMigrateParams' struct. Suggested-by: Manish Mishra <manish.mishra@nutanix.com> Signed-off-by: Het Gala <het.gala@nutanix.com> --- migration/migration.c | 52 +++++++++++++++++++++++++++++-------- migration/socket.c | 60 ++++++++++++++++++++++++++++++++++++++++--- migration/socket.h | 19 +++++++++++++- monitor/hmp-cmds.c | 1 + qapi/migration.json | 47 +++++++++++++++++++++++++++++---- 5 files changed, 160 insertions(+), 19 deletions(-)