| Message ID | 20180126155940.26994-1-danielhb@linux.vnet.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> wrote: > MAX_VM_CMD_PACKAGED_SIZE is a constant used in qemu_savevm_send_packaged > and loadvm_handle_cmd_packaged to determine whether a package is too > big to be sent or received. qemu_savevm_send_packaged is called inside > postcopy_start (migration/migration.c) to send the MigrationState > in a single blob to the destination, using the MIG_CMD_PACKAGED subcommand, > which will read it up using loadvm_handle_cmd_packaged. If the blob is > larger than MAX_VM_CMD_PACKAGED_SIZE, an error is thrown and the postcopy > migration is aborted. Both MAX_VM_CMD_PACKAGED_SIZE and MIG_CMD_PACKAGED > were introduced by commit 11cf1d984b ("MIG_CMD_PACKAGED: Send a packaged > chunk ..."). The constant has its original value of 1ul << 24 (16MB). > > The current MAX_VM_CMD_PACKAGED_SIZE value is not enough to support postcopy > migration of bigger pseries guests. The blob size for a postcopy migration of > a pseries guest with the following setup: > > qemu-system-ppc64 --nographic -vga none -machine pseries,accel=kvm -m 64G \ > -smp 1,maxcpus=32 -device virtio-blk-pci,drive=rootdisk \ > -drive file=f27.qcow2,if=none,cache=none,format=qcow2,id=rootdisk \ > -netdev user,id=u1 -net nic,netdev=u1 > > Goes around 12MB. Bumping the RAM to 128G makes the blob sizes goes to 20MB. > With 256G the blob goes to 37MB - more than twice the current maximum size. > At this moment the pseries machine can handle guests with up to 1TB of RAM, > making this postcopy blob goes to 128MB of size approximately. > > Following the discussions made in [1], there is a need to understand what > devices are aggressively consuming the blob in that manner and see if that > can be mitigated. Until then, we can set MAX_VM_CMD_PACKAGED_SIZE to the > maximum value allowed. Since the size is a 32 bit int variable, we can set > it as 1ul << 32, giving a maximum blob size of 4G that is enough to support > postcopy migration of 32TB RAM guests given the above constraints. > > [1] https://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg06313.html > > Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> > Reported-by: Balamuruhan S <bala24@linux.vnet.ibm.com> Reviewed-by: Juan Quintela <quintela@redhat.com>
diff --git a/migration/savevm.c b/migration/savevm.c index b7908f62be..cff679fff8 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -81,7 +81,7 @@ enum qemu_vm_cmd { MIG_CMD_MAX }; -#define MAX_VM_CMD_PACKAGED_SIZE (1ul << 24) +#define MAX_VM_CMD_PACKAGED_SIZE (1ul << 32) static struct mig_cmd_args { ssize_t len; /* -1 = variable */ const char *name;
MAX_VM_CMD_PACKAGED_SIZE is a constant used in qemu_savevm_send_packaged and loadvm_handle_cmd_packaged to determine whether a package is too big to be sent or received. qemu_savevm_send_packaged is called inside postcopy_start (migration/migration.c) to send the MigrationState in a single blob to the destination, using the MIG_CMD_PACKAGED subcommand, which will read it up using loadvm_handle_cmd_packaged. If the blob is larger than MAX_VM_CMD_PACKAGED_SIZE, an error is thrown and the postcopy migration is aborted. Both MAX_VM_CMD_PACKAGED_SIZE and MIG_CMD_PACKAGED were introduced by commit 11cf1d984b ("MIG_CMD_PACKAGED: Send a packaged chunk ..."). The constant has its original value of 1ul << 24 (16MB). The current MAX_VM_CMD_PACKAGED_SIZE value is not enough to support postcopy migration of bigger pseries guests. The blob size for a postcopy migration of a pseries guest with the following setup: qemu-system-ppc64 --nographic -vga none -machine pseries,accel=kvm -m 64G \ -smp 1,maxcpus=32 -device virtio-blk-pci,drive=rootdisk \ -drive file=f27.qcow2,if=none,cache=none,format=qcow2,id=rootdisk \ -netdev user,id=u1 -net nic,netdev=u1 Goes around 12MB. Bumping the RAM to 128G makes the blob sizes goes to 20MB. With 256G the blob goes to 37MB - more than twice the current maximum size. At this moment the pseries machine can handle guests with up to 1TB of RAM, making this postcopy blob goes to 128MB of size approximately. Following the discussions made in [1], there is a need to understand what devices are aggressively consuming the blob in that manner and see if that can be mitigated. Until then, we can set MAX_VM_CMD_PACKAGED_SIZE to the maximum value allowed. Since the size is a 32 bit int variable, we can set it as 1ul << 32, giving a maximum blob size of 4G that is enough to support postcopy migration of 32TB RAM guests given the above constraints. [1] https://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg06313.html Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> Reported-by: Balamuruhan S <bala24@linux.vnet.ibm.com> --- migration/savevm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)