| Message ID | 20220106221341.8779-4-leobras@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | MSG_ZEROCOPY + multifd | expand |
On Thu, Jan 06, 2022 at 07:13:40PM -0300, Leonardo Bras wrote: > Add property that allows zero-copy migration of memory pages, > and also includes a helper function migrate_use_zero_copy() to check > if it's enabled. > > No code is introduced to actually do the migration, but it allow > future implementations to enable/disable this feature. > > On non-Linux builds this parameter is compiled-out. I feel sad every time seeing a new parameter needs to be mostly duplicated 3 times in the code. :( > diff --git a/migration/socket.c b/migration/socket.c > index 05705a32d8..f7a77aafd3 100644 > --- a/migration/socket.c > +++ b/migration/socket.c > @@ -77,6 +77,11 @@ static void socket_outgoing_migration(QIOTask *task, > } else { > trace_migration_socket_outgoing_connected(data->hostname); > } > + > + if (migrate_use_zero_copy()) { > + error_setg(&err, "Zero copy not available in migration"); > + } I got confused the 1st time looking at it.. I think this is not strongly needed, but that's okay: Reviewed-by: Peter Xu <peterx@redhat.com> Thanks,
On Thu, Jan 06, 2022 at 07:13:40PM -0300, Leonardo Bras wrote: > Add property that allows zero-copy migration of memory pages, > and also includes a helper function migrate_use_zero_copy() to check > if it's enabled. > > No code is introduced to actually do the migration, but it allow > future implementations to enable/disable this feature. > > On non-Linux builds this parameter is compiled-out. > > Signed-off-by: Leonardo Bras <leobras@redhat.com> > --- > qapi/migration.json | 24 ++++++++++++++++++++++++ > migration/migration.h | 5 +++++ > migration/migration.c | 32 ++++++++++++++++++++++++++++++++ > migration/socket.c | 5 +++++ > monitor/hmp-cmds.c | 6 ++++++ > 5 files changed, 72 insertions(+) Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> > > diff --git a/qapi/migration.json b/qapi/migration.json > index bbfd48cf0b..2e62ea6ebd 100644 > --- a/qapi/migration.json > +++ b/qapi/migration.json > @@ -730,6 +730,13 @@ > # will consume more CPU. > # Defaults to 1. (Since 5.0) > # > +# @zero-copy: Controls behavior on sending memory pages on migration. > +# When true, enables a zero-copy mechanism for sending memory > +# pages, if host supports it. > +# Requires that QEMU be permitted to use locked memory for guest > +# RAM pages. > +# Defaults to false. (Since 7.0) > +# > # @block-bitmap-mapping: Maps block nodes and bitmaps on them to > # aliases for the purpose of dirty bitmap migration. Such > # aliases may for example be the corresponding names on the > @@ -769,6 +776,7 @@ > 'xbzrle-cache-size', 'max-postcopy-bandwidth', > 'max-cpu-throttle', 'multifd-compression', > 'multifd-zlib-level' ,'multifd-zstd-level', > + { 'name': 'zero-copy', 'if' : 'CONFIG_LINUX'}, > 'block-bitmap-mapping' ] } > > ## > @@ -895,6 +903,13 @@ > # will consume more CPU. > # Defaults to 1. (Since 5.0) > # > +# @zero-copy: Controls behavior on sending memory pages on migration. > +# When true, enables a zero-copy mechanism for sending memory > +# pages, if host supports it. > +# Requires that QEMU be permitted to use locked memory for guest > +# RAM pages. > +# Defaults to false. (Since 7.0) > +# > # @block-bitmap-mapping: Maps block nodes and bitmaps on them to > # aliases for the purpose of dirty bitmap migration. Such > # aliases may for example be the corresponding names on the > @@ -949,6 +964,7 @@ > '*multifd-compression': 'MultiFDCompression', > '*multifd-zlib-level': 'uint8', > '*multifd-zstd-level': 'uint8', > + '*zero-copy': { 'type': 'bool', 'if': 'CONFIG_LINUX' }, > '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } The current zerocopy impl is for the send path. Do you expect we might get zerocopy in the receive path later ? If so then either call this 'send-zero-copy', or change it from a bool to an enum taking '["send", "recv", "both"]'. I'd probably take the former and just rename it. Regards, Daniel
Hello Peter, On Thu, Jan 13, 2022 at 4:00 AM Peter Xu <peterx@redhat.com> wrote: > > On Thu, Jan 06, 2022 at 07:13:40PM -0300, Leonardo Bras wrote: > > Add property that allows zero-copy migration of memory pages, > > and also includes a helper function migrate_use_zero_copy() to check > > if it's enabled. > > > > No code is introduced to actually do the migration, but it allow > > future implementations to enable/disable this feature. > > > > On non-Linux builds this parameter is compiled-out. > > I feel sad every time seeing a new parameter needs to be mostly duplicated 3 > times in the code. :( > > > diff --git a/migration/socket.c b/migration/socket.c > > index 05705a32d8..f7a77aafd3 100644 > > --- a/migration/socket.c > > +++ b/migration/socket.c > > @@ -77,6 +77,11 @@ static void socket_outgoing_migration(QIOTask *task, > > } else { > > trace_migration_socket_outgoing_connected(data->hostname); > > } > > + > > + if (migrate_use_zero_copy()) { > > + error_setg(&err, "Zero copy not available in migration"); > > + } > > I got confused the 1st time looking at it.. I think this is not strongly > needed, but that's okay: The idea is to avoid some future issues on testing migration while bisecting. > > Reviewed-by: Peter Xu <peterx@redhat.com> Thanks Peter! > > Thanks, > > -- > Peter Xu >
Hello Daniel, On Thu, Jan 13, 2022 at 10:10 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Thu, Jan 06, 2022 at 07:13:40PM -0300, Leonardo Bras wrote: > > Add property that allows zero-copy migration of memory pages, > > and also includes a helper function migrate_use_zero_copy() to check > > if it's enabled. > > > > No code is introduced to actually do the migration, but it allow > > future implementations to enable/disable this feature. > > > > On non-Linux builds this parameter is compiled-out. > > > > Signed-off-by: Leonardo Bras <leobras@redhat.com> > > --- > > qapi/migration.json | 24 ++++++++++++++++++++++++ > > migration/migration.h | 5 +++++ > > migration/migration.c | 32 ++++++++++++++++++++++++++++++++ > > migration/socket.c | 5 +++++ > > monitor/hmp-cmds.c | 6 ++++++ > > 5 files changed, 72 insertions(+) > > Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Thanks! > > > > > diff --git a/qapi/migration.json b/qapi/migration.json > > index bbfd48cf0b..2e62ea6ebd 100644 > > --- a/qapi/migration.json > > +++ b/qapi/migration.json > > @@ -730,6 +730,13 @@ > > # will consume more CPU. > > # Defaults to 1. (Since 5.0) > > # > > +# @zero-copy: Controls behavior on sending memory pages on migration. > > +# When true, enables a zero-copy mechanism for sending memory > > +# pages, if host supports it. > > +# Requires that QEMU be permitted to use locked memory for guest > > +# RAM pages. > > +# Defaults to false. (Since 7.0) > > +# > > # @block-bitmap-mapping: Maps block nodes and bitmaps on them to > > # aliases for the purpose of dirty bitmap migration. Such > > # aliases may for example be the corresponding names on the > > @@ -769,6 +776,7 @@ > > 'xbzrle-cache-size', 'max-postcopy-bandwidth', > > 'max-cpu-throttle', 'multifd-compression', > > 'multifd-zlib-level' ,'multifd-zstd-level', > > + { 'name': 'zero-copy', 'if' : 'CONFIG_LINUX'}, > > 'block-bitmap-mapping' ] } > > > > ## > > @@ -895,6 +903,13 @@ > > # will consume more CPU. > > # Defaults to 1. (Since 5.0) > > # > > +# @zero-copy: Controls behavior on sending memory pages on migration. > > +# When true, enables a zero-copy mechanism for sending memory > > +# pages, if host supports it. > > +# Requires that QEMU be permitted to use locked memory for guest > > +# RAM pages. > > +# Defaults to false. (Since 7.0) > > +# > > # @block-bitmap-mapping: Maps block nodes and bitmaps on them to > > # aliases for the purpose of dirty bitmap migration. Such > > # aliases may for example be the corresponding names on the > > @@ -949,6 +964,7 @@ > > '*multifd-compression': 'MultiFDCompression', > > '*multifd-zlib-level': 'uint8', > > '*multifd-zstd-level': 'uint8', > > + '*zero-copy': { 'type': 'bool', 'if': 'CONFIG_LINUX' }, > > '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } > > The current zerocopy impl is for the send path. > > Do you expect we might get zerocopy in the receive path > later ? It's possible, but I haven't started the implementation yet. > > If so then either call this 'send-zero-copy', or change it > from a bool to an enum taking '["send", "recv", "both"]'. > > I'd probably take the former and just rename it. > Well, my rationale: - I want to set zero copy sending: zero-copy is set in the sending host, start migration. - I want to set zero copy receiving: zero-copy is set in the receiving host, wait for migration. (Of course host support is checked when setting the parameter). The problem with the current approach is trying to enable zero-copy on receive before it's implemented, which will 'fail' silently . A possible solution would be to add a patch to check in the receiving path if zero-copy is enabled, and fail for now. What do you think? Best regards, Leo > > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| >
On Wed, Jan 19, 2022 at 03:03:29PM -0300, Leonardo Bras Soares Passos wrote: > Hello Daniel, > > On Thu, Jan 13, 2022 at 10:10 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > On Thu, Jan 06, 2022 at 07:13:40PM -0300, Leonardo Bras wrote: > > > Add property that allows zero-copy migration of memory pages, > > > and also includes a helper function migrate_use_zero_copy() to check > > > if it's enabled. > > > > > > No code is introduced to actually do the migration, but it allow > > > future implementations to enable/disable this feature. > > > > > > On non-Linux builds this parameter is compiled-out. > > > > > > Signed-off-by: Leonardo Bras <leobras@redhat.com> > > > --- > > > qapi/migration.json | 24 ++++++++++++++++++++++++ > > > migration/migration.h | 5 +++++ > > > migration/migration.c | 32 ++++++++++++++++++++++++++++++++ > > > migration/socket.c | 5 +++++ > > > monitor/hmp-cmds.c | 6 ++++++ > > > 5 files changed, 72 insertions(+) > > > > Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> > > Thanks! > > > > > > > > > diff --git a/qapi/migration.json b/qapi/migration.json > > > index bbfd48cf0b..2e62ea6ebd 100644 > > > --- a/qapi/migration.json > > > +++ b/qapi/migration.json > > > @@ -730,6 +730,13 @@ > > > # will consume more CPU. > > > # Defaults to 1. (Since 5.0) > > > # > > > +# @zero-copy: Controls behavior on sending memory pages on migration. > > > +# When true, enables a zero-copy mechanism for sending memory > > > +# pages, if host supports it. > > > +# Requires that QEMU be permitted to use locked memory for guest > > > +# RAM pages. > > > +# Defaults to false. (Since 7.0) > > > +# > > > # @block-bitmap-mapping: Maps block nodes and bitmaps on them to > > > # aliases for the purpose of dirty bitmap migration. Such > > > # aliases may for example be the corresponding names on the > > > @@ -769,6 +776,7 @@ > > > 'xbzrle-cache-size', 'max-postcopy-bandwidth', > > > 'max-cpu-throttle', 'multifd-compression', > > > 'multifd-zlib-level' ,'multifd-zstd-level', > > > + { 'name': 'zero-copy', 'if' : 'CONFIG_LINUX'}, > > > 'block-bitmap-mapping' ] } > > > > > > ## > > > @@ -895,6 +903,13 @@ > > > # will consume more CPU. > > > # Defaults to 1. (Since 5.0) > > > # > > > +# @zero-copy: Controls behavior on sending memory pages on migration. > > > +# When true, enables a zero-copy mechanism for sending memory > > > +# pages, if host supports it. > > > +# Requires that QEMU be permitted to use locked memory for guest > > > +# RAM pages. > > > +# Defaults to false. (Since 7.0) > > > +# > > > # @block-bitmap-mapping: Maps block nodes and bitmaps on them to > > > # aliases for the purpose of dirty bitmap migration. Such > > > # aliases may for example be the corresponding names on the > > > @@ -949,6 +964,7 @@ > > > '*multifd-compression': 'MultiFDCompression', > > > '*multifd-zlib-level': 'uint8', > > > '*multifd-zstd-level': 'uint8', > > > + '*zero-copy': { 'type': 'bool', 'if': 'CONFIG_LINUX' }, > > > '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } > > > > The current zerocopy impl is for the send path. > > > > Do you expect we might get zerocopy in the receive path > > later ? > > It's possible, but I haven't started the implementation yet. > > > > > If so then either call this 'send-zero-copy', or change it > > from a bool to an enum taking '["send", "recv", "both"]'. > > > > I'd probably take the former and just rename it. > > > > Well, my rationale: > - I want to set zero copy sending: > zero-copy is set in the sending host, start migration. > > - I want to set zero copy receiving: > zero-copy is set in the receiving host, wait for migration. > (Of course host support is checked when setting the parameter). > > The problem with the current approach is trying to enable zero-copy on > receive before it's implemented, which will 'fail' silently . > A possible solution would be to add a patch to check in the receiving > path if zero-copy is enabled, and fail for now. That's not good because mgmt apps cannot query the QAPI schema to find out if this feature is supported or not. If we wantt o support zero copy recv, then we need an explicit flag for it that is distinct from zero copy send, so that apps can introspect whether the feature is implemneted in QEMU or not. Distinct named bool flags feels better, and also makes it clear to anyone not familiar with the impl that the current code is strictly send only. Regards, Daniel
On Wed, Jan 19, 2022 at 3:16 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Wed, Jan 19, 2022 at 03:03:29PM -0300, Leonardo Bras Soares Passos wrote: > > Hello Daniel, > > > > On Thu, Jan 13, 2022 at 10:10 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > > > On Thu, Jan 06, 2022 at 07:13:40PM -0300, Leonardo Bras wrote: > > > > Add property that allows zero-copy migration of memory pages, > > > > and also includes a helper function migrate_use_zero_copy() to check > > > > if it's enabled. > > > > > > > > No code is introduced to actually do the migration, but it allow > > > > future implementations to enable/disable this feature. > > > > > > > > On non-Linux builds this parameter is compiled-out. > > > > > > > > Signed-off-by: Leonardo Bras <leobras@redhat.com> > > > > --- > > > > qapi/migration.json | 24 ++++++++++++++++++++++++ > > > > migration/migration.h | 5 +++++ > > > > migration/migration.c | 32 ++++++++++++++++++++++++++++++++ > > > > migration/socket.c | 5 +++++ > > > > monitor/hmp-cmds.c | 6 ++++++ > > > > 5 files changed, 72 insertions(+) > > > > > > Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> > > > > Thanks! > > > > > > > > > > > > > diff --git a/qapi/migration.json b/qapi/migration.json > > > > index bbfd48cf0b..2e62ea6ebd 100644 > > > > --- a/qapi/migration.json > > > > +++ b/qapi/migration.json > > > > @@ -730,6 +730,13 @@ > > > > # will consume more CPU. > > > > # Defaults to 1. (Since 5.0) > > > > # > > > > +# @zero-copy: Controls behavior on sending memory pages on migration. > > > > +# When true, enables a zero-copy mechanism for sending memory > > > > +# pages, if host supports it. > > > > +# Requires that QEMU be permitted to use locked memory for guest > > > > +# RAM pages. > > > > +# Defaults to false. (Since 7.0) > > > > +# > > > > # @block-bitmap-mapping: Maps block nodes and bitmaps on them to > > > > # aliases for the purpose of dirty bitmap migration. Such > > > > # aliases may for example be the corresponding names on the > > > > @@ -769,6 +776,7 @@ > > > > 'xbzrle-cache-size', 'max-postcopy-bandwidth', > > > > 'max-cpu-throttle', 'multifd-compression', > > > > 'multifd-zlib-level' ,'multifd-zstd-level', > > > > + { 'name': 'zero-copy', 'if' : 'CONFIG_LINUX'}, > > > > 'block-bitmap-mapping' ] } > > > > > > > > ## > > > > @@ -895,6 +903,13 @@ > > > > # will consume more CPU. > > > > # Defaults to 1. (Since 5.0) > > > > # > > > > +# @zero-copy: Controls behavior on sending memory pages on migration. > > > > +# When true, enables a zero-copy mechanism for sending memory > > > > +# pages, if host supports it. > > > > +# Requires that QEMU be permitted to use locked memory for guest > > > > +# RAM pages. > > > > +# Defaults to false. (Since 7.0) > > > > +# > > > > # @block-bitmap-mapping: Maps block nodes and bitmaps on them to > > > > # aliases for the purpose of dirty bitmap migration. Such > > > > # aliases may for example be the corresponding names on the > > > > @@ -949,6 +964,7 @@ > > > > '*multifd-compression': 'MultiFDCompression', > > > > '*multifd-zlib-level': 'uint8', > > > > '*multifd-zstd-level': 'uint8', > > > > + '*zero-copy': { 'type': 'bool', 'if': 'CONFIG_LINUX' }, > > > > '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } > > > > > > The current zerocopy impl is for the send path. > > > > > > Do you expect we might get zerocopy in the receive path > > > later ? > > > > It's possible, but I haven't started the implementation yet. > > > > > > > > If so then either call this 'send-zero-copy', or change it > > > from a bool to an enum taking '["send", "recv", "both"]'. > > > > > > I'd probably take the former and just rename it. > > > > > > > Well, my rationale: > > - I want to set zero copy sending: > > zero-copy is set in the sending host, start migration. > > > > - I want to set zero copy receiving: > > zero-copy is set in the receiving host, wait for migration. > > (Of course host support is checked when setting the parameter). > > > > The problem with the current approach is trying to enable zero-copy on > > receive before it's implemented, which will 'fail' silently . > > A possible solution would be to add a patch to check in the receiving > > path if zero-copy is enabled, and fail for now. > > That's not good because mgmt apps cannot query the QAPI schema > to find out if this feature is supported or not. > > If we wantt o support zero copy recv, then we need an explicit > flag for it that is distinct from zero copy send, so that apps > can introspect whether the feature is implemneted in QEMU or > not. > > Distinct named bool flags feels better, and also makes it clear > to anyone not familiar with the impl that the current code is > strictly send only. > Ok, I see the point. I will try to refactor the code changing zero-copy to zero-copy-send or something like that. Best regards, Leo
Leonardo Bras Soares Passos <leobras@redhat.com> wrote: > On Wed, Jan 19, 2022 at 3:16 PM Daniel P. Berrangé <berrange@redhat.com> wrote: >> >> On Wed, Jan 19, 2022 at 03:03:29PM -0300, Leonardo Bras Soares Passos wrote: >> > Hello Daniel, >> > >> > On Thu, Jan 13, 2022 at 10:10 AM Daniel P. Berrangé <berrange@redhat.com> wrote: >> > > >> > > On Thu, Jan 06, 2022 at 07:13:40PM -0300, Leonardo Bras wrote: >> > > > Add property that allows zero-copy migration of memory pages, >> > > > and also includes a helper function migrate_use_zero_copy() to check >> > > > if it's enabled. >> > > > >> > > > No code is introduced to actually do the migration, but it allow >> > > > future implementations to enable/disable this feature. >> > > > >> > > > On non-Linux builds this parameter is compiled-out. >> > > > >> > > > Signed-off-by: Leonardo Bras <leobras@redhat.com> >> > > > --- >> > > > qapi/migration.json | 24 ++++++++++++++++++++++++ >> > > > migration/migration.h | 5 +++++ >> > > > migration/migration.c | 32 ++++++++++++++++++++++++++++++++ >> > > > migration/socket.c | 5 +++++ >> > > > monitor/hmp-cmds.c | 6 ++++++ >> > > > 5 files changed, 72 insertions(+) >> > > >> > > Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> >> > >> > Thanks! >> > > Ok, I see the point. > I will try to refactor the code changing zero-copy to zero-copy-send > or something like that. Hi I am late to the party, but I agree with Dan that we need two flags. Thre reason is that you can be the target of one migration, and later be the source of a next one. If we only have one flag that means different things on the source and destination side, things become really complicated. Later, Juan.
Hello Juan, Thanks for thew feedback! On Fri, Feb 18, 2022 at 1:31 PM Juan Quintela <quintela@redhat.com> wrote: > > Leonardo Bras Soares Passos <leobras@redhat.com> wrote: > > On Wed, Jan 19, 2022 at 3:16 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > >> > >> On Wed, Jan 19, 2022 at 03:03:29PM -0300, Leonardo Bras Soares Passos wrote: > >> > Hello Daniel, > >> > > >> > On Thu, Jan 13, 2022 at 10:10 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > >> > > > >> > > On Thu, Jan 06, 2022 at 07:13:40PM -0300, Leonardo Bras wrote: > >> > > > Add property that allows zero-copy migration of memory pages, > >> > > > and also includes a helper function migrate_use_zero_copy() to check > >> > > > if it's enabled. > >> > > > > >> > > > No code is introduced to actually do the migration, but it allow > >> > > > future implementations to enable/disable this feature. > >> > > > > >> > > > On non-Linux builds this parameter is compiled-out. > >> > > > > >> > > > Signed-off-by: Leonardo Bras <leobras@redhat.com> > >> > > > --- > >> > > > qapi/migration.json | 24 ++++++++++++++++++++++++ > >> > > > migration/migration.h | 5 +++++ > >> > > > migration/migration.c | 32 ++++++++++++++++++++++++++++++++ > >> > > > migration/socket.c | 5 +++++ > >> > > > monitor/hmp-cmds.c | 6 ++++++ > >> > > > 5 files changed, 72 insertions(+) > >> > > > >> > > Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> > >> > > >> > Thanks! > >> > > > > Ok, I see the point. > > I will try to refactor the code changing zero-copy to zero-copy-send > > or something like that. > > Hi > > I am late to the party, but I agree with Dan that we need two flags. > > Thre reason is that you can be the target of one migration, and later be > the source of a next one. If we only have one flag that means different > things on the source and destination side, things become really > complicated. > > Later, Juan. > Yeah, that makes sense. :) Best regards, Leo
diff --git a/qapi/migration.json b/qapi/migration.json index bbfd48cf0b..2e62ea6ebd 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -730,6 +730,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @zero-copy: Controls behavior on sending memory pages on migration. +# When true, enables a zero-copy mechanism for sending memory +# pages, if host supports it. +# Requires that QEMU be permitted to use locked memory for guest +# RAM pages. +# Defaults to false. (Since 7.0) +# # @block-bitmap-mapping: Maps block nodes and bitmaps on them to # aliases for the purpose of dirty bitmap migration. Such # aliases may for example be the corresponding names on the @@ -769,6 +776,7 @@ 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', 'multifd-zlib-level' ,'multifd-zstd-level', + { 'name': 'zero-copy', 'if' : 'CONFIG_LINUX'}, 'block-bitmap-mapping' ] } ## @@ -895,6 +903,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @zero-copy: Controls behavior on sending memory pages on migration. +# When true, enables a zero-copy mechanism for sending memory +# pages, if host supports it. +# Requires that QEMU be permitted to use locked memory for guest +# RAM pages. +# Defaults to false. (Since 7.0) +# # @block-bitmap-mapping: Maps block nodes and bitmaps on them to # aliases for the purpose of dirty bitmap migration. Such # aliases may for example be the corresponding names on the @@ -949,6 +964,7 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', + '*zero-copy': { 'type': 'bool', 'if': 'CONFIG_LINUX' }, '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } ## @@ -1095,6 +1111,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @zero-copy: Controls behavior on sending memory pages on migration. +# When true, enables a zero-copy mechanism for sending memory +# pages, if host supports it. +# Requires that QEMU be permitted to use locked memory for guest +# RAM pages. +# Defaults to false. (Since 7.0) +# # @block-bitmap-mapping: Maps block nodes and bitmaps on them to # aliases for the purpose of dirty bitmap migration. Such # aliases may for example be the corresponding names on the @@ -1147,6 +1170,7 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', + '*zero-copy': { 'type': 'bool', 'if': 'CONFIG_LINUX' }, '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } ## diff --git a/migration/migration.h b/migration/migration.h index 8130b703eb..1489eeb165 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -339,6 +339,11 @@ MultiFDCompression migrate_multifd_compression(void); int migrate_multifd_zlib_level(void); int migrate_multifd_zstd_level(void); +#ifdef CONFIG_LINUX +bool migrate_use_zero_copy(void); +#else +#define migrate_use_zero_copy() (false) +#endif int migrate_use_xbzrle(void); uint64_t migrate_xbzrle_cache_size(void); bool migrate_colo_enabled(void); diff --git a/migration/migration.c b/migration/migration.c index 0652165610..aa8f1dc835 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -893,6 +893,10 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->multifd_zlib_level = s->parameters.multifd_zlib_level; params->has_multifd_zstd_level = true; params->multifd_zstd_level = s->parameters.multifd_zstd_level; +#ifdef CONFIG_LINUX + params->has_zero_copy = true; + params->zero_copy = s->parameters.zero_copy; +#endif params->has_xbzrle_cache_size = true; params->xbzrle_cache_size = s->parameters.xbzrle_cache_size; params->has_max_postcopy_bandwidth = true; @@ -1546,6 +1550,11 @@ static void migrate_params_test_apply(MigrateSetParameters *params, if (params->has_multifd_compression) { dest->multifd_compression = params->multifd_compression; } +#ifdef CONFIG_LINUX + if (params->has_zero_copy) { + dest->zero_copy = params->zero_copy; + } +#endif if (params->has_xbzrle_cache_size) { dest->xbzrle_cache_size = params->xbzrle_cache_size; } @@ -1658,6 +1667,11 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) if (params->has_multifd_compression) { s->parameters.multifd_compression = params->multifd_compression; } +#ifdef CONFIG_LINUX + if (params->has_zero_copy) { + s->parameters.zero_copy = params->zero_copy; + } +#endif if (params->has_xbzrle_cache_size) { s->parameters.xbzrle_cache_size = params->xbzrle_cache_size; xbzrle_cache_resize(params->xbzrle_cache_size, errp); @@ -2548,6 +2562,17 @@ int migrate_multifd_zstd_level(void) return s->parameters.multifd_zstd_level; } +#ifdef CONFIG_LINUX +bool migrate_use_zero_copy(void) +{ + MigrationState *s; + + s = migrate_get_current(); + + return s->parameters.zero_copy; +} +#endif + int migrate_use_xbzrle(void) { MigrationState *s; @@ -4200,6 +4225,10 @@ static Property migration_properties[] = { DEFINE_PROP_UINT8("multifd-zstd-level", MigrationState, parameters.multifd_zstd_level, DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL), +#ifdef CONFIG_LINUX + DEFINE_PROP_BOOL("zero_copy", MigrationState, + parameters.zero_copy, false), +#endif DEFINE_PROP_SIZE("xbzrle-cache-size", MigrationState, parameters.xbzrle_cache_size, DEFAULT_MIGRATE_XBZRLE_CACHE_SIZE), @@ -4297,6 +4326,9 @@ static void migration_instance_init(Object *obj) params->has_multifd_compression = true; params->has_multifd_zlib_level = true; params->has_multifd_zstd_level = true; +#ifdef CONFIG_LINUX + params->has_zero_copy = true; +#endif params->has_xbzrle_cache_size = true; params->has_max_postcopy_bandwidth = true; params->has_max_cpu_throttle = true; diff --git a/migration/socket.c b/migration/socket.c index 05705a32d8..f7a77aafd3 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -77,6 +77,11 @@ static void socket_outgoing_migration(QIOTask *task, } else { trace_migration_socket_outgoing_connected(data->hostname); } + + if (migrate_use_zero_copy()) { + error_setg(&err, "Zero copy not available in migration"); + } + migration_channel_connect(data->s, sioc, data->hostname, err); object_unref(OBJECT(sioc)); } diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 2669156b28..35d47e250d 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -1297,6 +1297,12 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_multifd_zstd_level = true; visit_type_uint8(v, param, &p->multifd_zstd_level, &err); break; +#ifdef CONFIG_LINUX + case MIGRATION_PARAMETER_ZERO_COPY: + p->has_zero_copy = true; + visit_type_bool(v, param, &p->zero_copy, &err); + break; +#endif case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE: p->has_xbzrle_cache_size = true; if (!visit_type_size(v, param, &cache_size, &err)) {
Add property that allows zero-copy migration of memory pages, and also includes a helper function migrate_use_zero_copy() to check if it's enabled. No code is introduced to actually do the migration, but it allow future implementations to enable/disable this feature. On non-Linux builds this parameter is compiled-out. Signed-off-by: Leonardo Bras <leobras@redhat.com> --- qapi/migration.json | 24 ++++++++++++++++++++++++ migration/migration.h | 5 +++++ migration/migration.c | 32 ++++++++++++++++++++++++++++++++ migration/socket.c | 5 +++++ monitor/hmp-cmds.c | 6 ++++++ 5 files changed, 72 insertions(+)