| Message ID | 20200520220118.1037094-5-eblake@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | qemu-img: Add convert --bitmaps | expand |
On Thu, May 21, 2020 at 1:01 AM Eric Blake <eblake@redhat.com> wrote: > > Make it easier to copy all the persistent bitmaps of (the top layer > of) a source image along with its guest-visible contents, by adding a > boolean flag for use with qemu-img convert. This is basically > shorthand, as the same effect could be accomplished with a series of > 'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source' > commands, or by QMP commands. > > Note that this command will fail in the same scenarios where 'qemu-img > measure --bitmaps' fails, when either the source or the destanation > lacks persistent bitmap support altogether. If we remove --bitmaps option from qemu-img measure, we need to remove this note. > > See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893 > > While touching this, clean up a couple coding issues spotted in the > same function: an extra blank line, and merging back-to-back 'if > (!skip_create)' blocks. > > Signed-off-by: Eric Blake <eblake@redhat.com> > Message-Id: <20200513011648.166876-9-eblake@redhat.com> > --- > docs/tools/qemu-img.rst | 6 +++- > qemu-img.c | 77 +++++++++++++++++++++++++++++++++++++++-- > qemu-img-cmds.hx | 4 +-- > 3 files changed, 81 insertions(+), 6 deletions(-) > > diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst > index 9a8112fc9f58..35050fc51070 100644 > --- a/docs/tools/qemu-img.rst > +++ b/docs/tools/qemu-img.rst > @@ -162,6 +162,10 @@ Parameters to convert subcommand: > > .. program:: qemu-img-convert > > +.. option:: --bitmaps > + > + Additionally copy all persistent bitmaps from the top layer of the source > + > .. option:: -n > > Skip the creation of the target volume > @@ -397,7 +401,7 @@ Command description: > 4 > Error on reading data > > -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME > +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME > > Convert the disk image *FILENAME* or a snapshot *SNAPSHOT_PARAM* > to disk image *OUTPUT_FILENAME* using format *OUTPUT_FMT*. It can > diff --git a/qemu-img.c b/qemu-img.c > index c1bafb57023a..1494d8f5c409 100644 > --- a/qemu-img.c > +++ b/qemu-img.c > @@ -192,6 +192,7 @@ static void QEMU_NORETURN help(void) > " hiding corruption that has already occurred.\n" > "\n" > "Parameters to convert subcommand:\n" > + " '--bitmaps' copies all top-level persistent bitmaps to destination\n" > " '-m' specifies how many coroutines work in parallel during the convert\n" > " process (defaults to 8)\n" > " '-W' allow to write to the target out of order rather than sequential\n" > @@ -2139,6 +2140,39 @@ static int convert_do_copy(ImgConvertState *s) > return s->ret; > } > > +static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst) > +{ > + BdrvDirtyBitmap *bm; > + Error *err = NULL; > + > + FOR_EACH_DIRTY_BITMAP(src, bm) { > + const char *name; > + > + if (!bdrv_dirty_bitmap_get_persistence(bm)) { > + continue; > + } > + name = bdrv_dirty_bitmap_name(bm); > + qmp_block_dirty_bitmap_add(dst->node_name, name, > + true, bdrv_dirty_bitmap_granularity(bm), > + true, true, > + true, !bdrv_dirty_bitmap_enabled(bm), > + &err); > + if (err) { > + error_reportf_err(err, "Failed to create bitmap %s: ", name); > + return -1; > + } > + > + do_dirty_bitmap_merge(dst->node_name, name, src->node_name, name, > + &err); > + if (err) { > + error_reportf_err(err, "Failed to populate bitmap %s: ", name); > + return -1; > + } > + } > + > + return 0; > +} > + > #define MAX_BUF_SECTORS 32768 > > static int img_convert(int argc, char **argv) > @@ -2160,6 +2194,8 @@ static int img_convert(int argc, char **argv) > int64_t ret = -EINVAL; > bool force_share = false; > bool explict_min_sparse = false; > + bool bitmaps = false; > + size_t nbitmaps = 0; > > ImgConvertState s = (ImgConvertState) { > /* Need at least 4k of zeros for sparse detection */ > @@ -2179,6 +2215,7 @@ static int img_convert(int argc, char **argv) > {"target-image-opts", no_argument, 0, OPTION_TARGET_IMAGE_OPTS}, > {"salvage", no_argument, 0, OPTION_SALVAGE}, > {"target-is-zero", no_argument, 0, OPTION_TARGET_IS_ZERO}, > + {"bitmaps", no_argument, 0, OPTION_BITMAPS}, > {0, 0, 0, 0} > }; > c = getopt_long(argc, argv, ":hf:O:B:Cco:l:S:pt:T:qnm:WU", > @@ -2304,6 +2341,9 @@ static int img_convert(int argc, char **argv) > */ > s.has_zero_init = true; > break; > + case OPTION_BITMAPS: > + bitmaps = true; > + break; > } > } > > @@ -2365,7 +2405,6 @@ static int img_convert(int argc, char **argv) > goto fail_getopt; > } > > - > /* ret is still -EINVAL until here */ > ret = bdrv_parse_cache_mode(src_cache, &src_flags, &src_writethrough); > if (ret < 0) { > @@ -2525,6 +2564,27 @@ static int img_convert(int argc, char **argv) > } > } > > + /* Determine how many bitmaps need copying */ > + if (bitmaps) { > + BdrvDirtyBitmap *bm; > + > + if (s.src_num > 1) { > + error_report("Copying bitmaps only possible with single source"); > + ret = -1; > + goto out; > + } > + if (!bdrv_supports_persistent_dirty_bitmap(blk_bs(s.src[0]))) { > + error_report("Source lacks bitmap support"); > + ret = -1; > + goto out; > + } > + FOR_EACH_DIRTY_BITMAP(blk_bs(s.src[0]), bm) { > + if (bdrv_dirty_bitmap_get_persistence(bm)) { > + nbitmaps++; > + } > + } > + } > + > /* > * The later open call will need any decryption secrets, and > * bdrv_create() will purge "opts", so extract them now before > @@ -2533,9 +2593,7 @@ static int img_convert(int argc, char **argv) > if (!skip_create) { > open_opts = qdict_new(); > qemu_opt_foreach(opts, img_add_key_secrets, open_opts, &error_abort); > - } > > - if (!skip_create) { > /* Create the new image */ > ret = bdrv_create(drv, out_filename, opts, &local_err); > if (ret < 0) { > @@ -2573,6 +2631,13 @@ static int img_convert(int argc, char **argv) > } > out_bs = blk_bs(s.target); > > + if (nbitmaps > 0 && !bdrv_supports_persistent_dirty_bitmap(out_bs)) { > + error_report("Format driver '%s' does not support bitmaps", > + out_fmt); > + ret = -1; > + goto out; > + } > + > if (s.compressed && !block_driver_can_compress(out_bs->drv)) { > error_report("Compression not supported for this file format"); > ret = -1; > @@ -2632,6 +2697,12 @@ static int img_convert(int argc, char **argv) > } > > ret = convert_do_copy(&s); > + > + /* Now copy the bitmaps */ > + if (nbitmaps > 0 && ret == 0) { > + ret = convert_copy_bitmaps(blk_bs(s.src[0]), out_bs); > + } > + > out: > if (!ret) { > qemu_progress_print(100, 0); > diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx > index 235cc5fffadc..e9beb15e614e 100644 > --- a/qemu-img-cmds.hx > +++ b/qemu-img-cmds.hx > @@ -46,9 +46,9 @@ SRST > ERST > > DEF("convert", img_convert, > - "convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename") > + "convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename") > SRST > -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME > +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME > ERST > > DEF("create", img_create, > -- > 2.26.2 >
On 5/21/20 10:11 AM, Nir Soffer wrote: > On Thu, May 21, 2020 at 1:01 AM Eric Blake <eblake@redhat.com> wrote: >> >> Make it easier to copy all the persistent bitmaps of (the top layer >> of) a source image along with its guest-visible contents, by adding a >> boolean flag for use with qemu-img convert. This is basically >> shorthand, as the same effect could be accomplished with a series of >> 'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source' >> commands, or by QMP commands. >> >> Note that this command will fail in the same scenarios where 'qemu-img >> measure --bitmaps' fails, when either the source or the destanation destination >> lacks persistent bitmap support altogether. > > If we remove --bitmaps option from qemu-img measure, we need to remove > this note. Or rather, adjust it to state that "this command will fail in the scenarios where 'qemu-img measure' omits a 'bitmaps size:' output line.
diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst index 9a8112fc9f58..35050fc51070 100644 --- a/docs/tools/qemu-img.rst +++ b/docs/tools/qemu-img.rst @@ -162,6 +162,10 @@ Parameters to convert subcommand: .. program:: qemu-img-convert +.. option:: --bitmaps + + Additionally copy all persistent bitmaps from the top layer of the source + .. option:: -n Skip the creation of the target volume @@ -397,7 +401,7 @@ Command description: 4 Error on reading data -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME Convert the disk image *FILENAME* or a snapshot *SNAPSHOT_PARAM* to disk image *OUTPUT_FILENAME* using format *OUTPUT_FMT*. It can diff --git a/qemu-img.c b/qemu-img.c index c1bafb57023a..1494d8f5c409 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -192,6 +192,7 @@ static void QEMU_NORETURN help(void) " hiding corruption that has already occurred.\n" "\n" "Parameters to convert subcommand:\n" + " '--bitmaps' copies all top-level persistent bitmaps to destination\n" " '-m' specifies how many coroutines work in parallel during the convert\n" " process (defaults to 8)\n" " '-W' allow to write to the target out of order rather than sequential\n" @@ -2139,6 +2140,39 @@ static int convert_do_copy(ImgConvertState *s) return s->ret; } +static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst) +{ + BdrvDirtyBitmap *bm; + Error *err = NULL; + + FOR_EACH_DIRTY_BITMAP(src, bm) { + const char *name; + + if (!bdrv_dirty_bitmap_get_persistence(bm)) { + continue; + } + name = bdrv_dirty_bitmap_name(bm); + qmp_block_dirty_bitmap_add(dst->node_name, name, + true, bdrv_dirty_bitmap_granularity(bm), + true, true, + true, !bdrv_dirty_bitmap_enabled(bm), + &err); + if (err) { + error_reportf_err(err, "Failed to create bitmap %s: ", name); + return -1; + } + + do_dirty_bitmap_merge(dst->node_name, name, src->node_name, name, + &err); + if (err) { + error_reportf_err(err, "Failed to populate bitmap %s: ", name); + return -1; + } + } + + return 0; +} + #define MAX_BUF_SECTORS 32768 static int img_convert(int argc, char **argv) @@ -2160,6 +2194,8 @@ static int img_convert(int argc, char **argv) int64_t ret = -EINVAL; bool force_share = false; bool explict_min_sparse = false; + bool bitmaps = false; + size_t nbitmaps = 0; ImgConvertState s = (ImgConvertState) { /* Need at least 4k of zeros for sparse detection */ @@ -2179,6 +2215,7 @@ static int img_convert(int argc, char **argv) {"target-image-opts", no_argument, 0, OPTION_TARGET_IMAGE_OPTS}, {"salvage", no_argument, 0, OPTION_SALVAGE}, {"target-is-zero", no_argument, 0, OPTION_TARGET_IS_ZERO}, + {"bitmaps", no_argument, 0, OPTION_BITMAPS}, {0, 0, 0, 0} }; c = getopt_long(argc, argv, ":hf:O:B:Cco:l:S:pt:T:qnm:WU", @@ -2304,6 +2341,9 @@ static int img_convert(int argc, char **argv) */ s.has_zero_init = true; break; + case OPTION_BITMAPS: + bitmaps = true; + break; } } @@ -2365,7 +2405,6 @@ static int img_convert(int argc, char **argv) goto fail_getopt; } - /* ret is still -EINVAL until here */ ret = bdrv_parse_cache_mode(src_cache, &src_flags, &src_writethrough); if (ret < 0) { @@ -2525,6 +2564,27 @@ static int img_convert(int argc, char **argv) } } + /* Determine how many bitmaps need copying */ + if (bitmaps) { + BdrvDirtyBitmap *bm; + + if (s.src_num > 1) { + error_report("Copying bitmaps only possible with single source"); + ret = -1; + goto out; + } + if (!bdrv_supports_persistent_dirty_bitmap(blk_bs(s.src[0]))) { + error_report("Source lacks bitmap support"); + ret = -1; + goto out; + } + FOR_EACH_DIRTY_BITMAP(blk_bs(s.src[0]), bm) { + if (bdrv_dirty_bitmap_get_persistence(bm)) { + nbitmaps++; + } + } + } + /* * The later open call will need any decryption secrets, and * bdrv_create() will purge "opts", so extract them now before @@ -2533,9 +2593,7 @@ static int img_convert(int argc, char **argv) if (!skip_create) { open_opts = qdict_new(); qemu_opt_foreach(opts, img_add_key_secrets, open_opts, &error_abort); - } - if (!skip_create) { /* Create the new image */ ret = bdrv_create(drv, out_filename, opts, &local_err); if (ret < 0) { @@ -2573,6 +2631,13 @@ static int img_convert(int argc, char **argv) } out_bs = blk_bs(s.target); + if (nbitmaps > 0 && !bdrv_supports_persistent_dirty_bitmap(out_bs)) { + error_report("Format driver '%s' does not support bitmaps", + out_fmt); + ret = -1; + goto out; + } + if (s.compressed && !block_driver_can_compress(out_bs->drv)) { error_report("Compression not supported for this file format"); ret = -1; @@ -2632,6 +2697,12 @@ static int img_convert(int argc, char **argv) } ret = convert_do_copy(&s); + + /* Now copy the bitmaps */ + if (nbitmaps > 0 && ret == 0) { + ret = convert_copy_bitmaps(blk_bs(s.src[0]), out_bs); + } + out: if (!ret) { qemu_progress_print(100, 0); diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx index 235cc5fffadc..e9beb15e614e 100644 --- a/qemu-img-cmds.hx +++ b/qemu-img-cmds.hx @@ -46,9 +46,9 @@ SRST ERST DEF("convert", img_convert, - "convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename") + "convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename") SRST -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME ERST DEF("create", img_create,
Make it easier to copy all the persistent bitmaps of (the top layer of) a source image along with its guest-visible contents, by adding a boolean flag for use with qemu-img convert. This is basically shorthand, as the same effect could be accomplished with a series of 'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source' commands, or by QMP commands. Note that this command will fail in the same scenarios where 'qemu-img measure --bitmaps' fails, when either the source or the destanation lacks persistent bitmap support altogether. See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893 While touching this, clean up a couple coding issues spotted in the same function: an extra blank line, and merging back-to-back 'if (!skip_create)' blocks. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200513011648.166876-9-eblake@redhat.com> --- docs/tools/qemu-img.rst | 6 +++- qemu-img.c | 77 +++++++++++++++++++++++++++++++++++++++-- qemu-img-cmds.hx | 4 +-- 3 files changed, 81 insertions(+), 6 deletions(-)