From patchwork Sat May 14 12:45:54 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 9095091 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 267C1BF440 for ; Sat, 14 May 2016 12:50:07 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1FF98200E1 for ; Sat, 14 May 2016 12:50:06 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CE52120220 for ; Sat, 14 May 2016 12:50:02 +0000 (UTC) Received: from localhost ([::1]:37649 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b1Z18-0003qw-7P for patchwork-qemu-devel@patchwork.kernel.org; Sat, 14 May 2016 08:50:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44219) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b1Yxm-0005O1-0j for qemu-devel@nongnu.org; Sat, 14 May 2016 08:46:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b1Yxg-0006vq-7D for qemu-devel@nongnu.org; Sat, 14 May 2016 08:46:32 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:28964 helo=relay.sw.ru) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b1Yxf-0006uv-IT for qemu-devel@nongnu.org; Sat, 14 May 2016 08:46:28 -0400 Received: from irbis.sw.ru ([10.24.38.119]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id u4ECjvib001269; Sat, 14 May 2016 15:46:13 +0300 (MSK) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Date: Sat, 14 May 2016 15:45:54 +0300 Message-Id: <1463229957-14253-7-git-send-email-den@openvz.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1463229957-14253-1-git-send-email-den@openvz.org> References: <1463229957-14253-1-git-send-email-den@openvz.org> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 06/10] drive-backup: added support for data compression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Pavel Butsykin , Jeff Cody , Markus Armbruster , Stefan Hajnoczi , den@openvz.org, John Snow Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Pavel Butsykin The idea is simple - backup is "written-once" data. It is written block by block and it is large enough. It would be nice to save storage space and compress it. The patch adds a flag to the qmp/hmp drive-backup command which enables block compression. Compression should be implemented in the format driver to enable this feature. There are some limitations of the format driver to allow compressed writes. We can write data only once. Though for backup this is perfectly fine. These limitations are maintained by the driver and the error will be reported if we are doing something wrong. Signed-off-by: Pavel Butsykin Signed-off-by: Denis V. Lunev CC: Jeff Cody CC: Markus Armbruster CC: Eric Blake CC: John Snow CC: Stefan Hajnoczi CC: Kevin Wolf Reviewed-by: Stefan Hajnoczi --- block/backup.c | 13 +++++++++++++ blockdev.c | 12 ++++++++++-- hmp-commands.hx | 8 +++++--- hmp.c | 3 ++- include/block/block_int.h | 1 + qapi/block-core.json | 2 +- qmp-commands.hx | 4 +++- 7 files changed, 35 insertions(+), 8 deletions(-) diff --git a/block/backup.c b/block/backup.c index 491fd14..0bcfb23 100644 --- a/block/backup.c +++ b/block/backup.c @@ -47,6 +47,7 @@ typedef struct BackupBlockJob { uint64_t sectors_read; unsigned long *done_bitmap; int64_t cluster_size; + bool compress; QLIST_HEAD(, CowRequest) inflight_reqs; } BackupBlockJob; @@ -157,6 +158,10 @@ static int coroutine_fn backup_do_cow(BlockDriverState *bs, ret = bdrv_co_write_zeroes(job->target, start * sectors_per_cluster, n, BDRV_REQ_MAY_UNMAP); + } else if (job->compress) { + ret = bdrv_co_write_compressed(job->target, + start * sectors_per_cluster, n, + &bounce_qiov); } else { ret = bdrv_co_writev(job->target, start * sectors_per_cluster, n, @@ -497,6 +502,7 @@ static void coroutine_fn backup_run(void *opaque) void backup_start(BlockDriverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, + bool compress, BlockdevOnError on_source_error, BlockdevOnError on_target_error, BlockCompletionFunc *cb, void *opaque, @@ -534,6 +540,12 @@ void backup_start(BlockDriverState *bs, BlockDriverState *target, return; } + if (compress && target->drv->bdrv_co_write_compressed == NULL) { + error_setg(errp, "Compression is not supported for this drive %s", + bdrv_get_device_name(target)); + return; + } + if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) { return; } @@ -580,6 +592,7 @@ void backup_start(BlockDriverState *bs, BlockDriverState *target, job->sync_mode = sync_mode; job->sync_bitmap = sync_mode == MIRROR_SYNC_MODE_INCREMENTAL ? sync_bitmap : NULL; + job->compress = compress; /* If there is no backing file on the target, we cannot rely on COW if our * backup cluster size is smaller than the target cluster size. Even for diff --git a/blockdev.c b/blockdev.c index 1892b8e..1ef9e31 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1856,6 +1856,7 @@ static void do_drive_backup(const char *device, const char *target, bool has_mode, enum NewImageMode mode, bool has_speed, int64_t speed, bool has_bitmap, const char *bitmap, + bool has_compress, bool compress, bool has_on_source_error, BlockdevOnError on_source_error, bool has_on_target_error, @@ -1896,6 +1897,7 @@ static void drive_backup_prepare(BlkActionState *common, Error **errp) backup->has_mode, backup->mode, backup->has_speed, backup->speed, backup->has_bitmap, backup->bitmap, + backup->has_compress, backup->compress, backup->has_on_source_error, backup->on_source_error, backup->has_on_target_error, backup->on_target_error, common->block_job_txn, &local_err); @@ -3170,6 +3172,7 @@ static void do_drive_backup(const char *device, const char *target, bool has_mode, enum NewImageMode mode, bool has_speed, int64_t speed, bool has_bitmap, const char *bitmap, + bool has_compress, bool compress, bool has_on_source_error, BlockdevOnError on_source_error, bool has_on_target_error, @@ -3200,6 +3203,9 @@ static void do_drive_backup(const char *device, const char *target, if (!has_mode) { mode = NEW_IMAGE_MODE_ABSOLUTE_PATHS; } + if (!has_compress) { + compress = false; + } blk = blk_by_name(device); if (!blk) { @@ -3288,7 +3294,7 @@ static void do_drive_backup(const char *device, const char *target, } } - backup_start(bs, target_bs, speed, sync, bmap, + backup_start(bs, target_bs, speed, sync, bmap, compress, on_source_error, on_target_error, block_job_cb, bs, txn, &local_err); if (local_err != NULL) { @@ -3307,6 +3313,7 @@ void qmp_drive_backup(const char *device, const char *target, bool has_mode, enum NewImageMode mode, bool has_speed, int64_t speed, bool has_bitmap, const char *bitmap, + bool has_compress, bool compress, bool has_on_source_error, BlockdevOnError on_source_error, bool has_on_target_error, BlockdevOnError on_target_error, Error **errp) @@ -3314,6 +3321,7 @@ void qmp_drive_backup(const char *device, const char *target, return do_drive_backup(device, target, has_format, format, sync, has_mode, mode, has_speed, speed, has_bitmap, bitmap, + has_compress, compress, has_on_source_error, on_source_error, has_on_target_error, on_target_error, NULL, errp); @@ -3378,7 +3386,7 @@ void do_blockdev_backup(const char *device, const char *target, bdrv_ref(target_bs); bdrv_set_aio_context(target_bs, aio_context); - backup_start(bs, target_bs, speed, sync, NULL, on_source_error, + backup_start(bs, target_bs, speed, sync, NULL, false, on_source_error, on_target_error, block_job_cb, bs, txn, &local_err); if (local_err != NULL) { bdrv_unref(target_bs); diff --git a/hmp-commands.hx b/hmp-commands.hx index 4f4f60a..4b1d15b 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -1181,8 +1181,8 @@ ETEXI { .name = "drive_backup", - .args_type = "reuse:-n,full:-f,device:B,target:s,format:s?", - .params = "[-n] [-f] device target [format]", + .args_type = "reuse:-n,full:-f,compress:-c,device:B,target:s,format:s?", + .params = "[-n] [-f] [-c] device target [format]", .help = "initiates a point-in-time\n\t\t\t" "copy for a device. The device's contents are\n\t\t\t" "copied to the new image file, excluding data that\n\t\t\t" @@ -1190,7 +1190,9 @@ ETEXI "The -n flag requests QEMU to reuse the image found\n\t\t\t" "in new-image-file, instead of recreating it from scratch.\n\t\t\t" "The -f flag requests QEMU to copy the whole disk,\n\t\t\t" - "so that the result does not need a backing file.\n\t\t\t", + "so that the result does not need a backing file.\n\t\t\t" + "The -c flag requests QEMU to compress backup data\n\t\t\t" + "(if the target format supports it).\n\t\t\t", .mhandler.cmd = hmp_drive_backup, }, STEXI diff --git a/hmp.c b/hmp.c index d510236..e34593d 100644 --- a/hmp.c +++ b/hmp.c @@ -1094,6 +1094,7 @@ void hmp_drive_backup(Monitor *mon, const QDict *qdict) const char *format = qdict_get_try_str(qdict, "format"); bool reuse = qdict_get_try_bool(qdict, "reuse", false); bool full = qdict_get_try_bool(qdict, "full", false); + bool compress = qdict_get_try_bool(qdict, "compress", false); enum NewImageMode mode; Error *err = NULL; @@ -1111,7 +1112,7 @@ void hmp_drive_backup(Monitor *mon, const QDict *qdict) qmp_drive_backup(device, filename, !!format, format, full ? MIRROR_SYNC_MODE_FULL : MIRROR_SYNC_MODE_TOP, - true, mode, false, 0, false, NULL, + true, mode, false, 0, false, NULL, compress, compress, false, 0, false, 0, &err); hmp_handle_error(mon, &err); } diff --git a/include/block/block_int.h b/include/block/block_int.h index cae838c..34c5ec7 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -712,6 +712,7 @@ void mirror_start(BlockDriverState *bs, BlockDriverState *target, void backup_start(BlockDriverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, + bool compress, BlockdevOnError on_source_error, BlockdevOnError on_target_error, BlockCompletionFunc *cb, void *opaque, diff --git a/qapi/block-core.json b/qapi/block-core.json index 98a20d2..85e5950 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -905,7 +905,7 @@ { 'struct': 'DriveBackup', 'data': { 'device': 'str', 'target': 'str', '*format': 'str', 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode', - '*speed': 'int', '*bitmap': 'str', + '*speed': 'int', '*bitmap': 'str', '*compress': 'bool', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError' } } diff --git a/qmp-commands.hx b/qmp-commands.hx index 94847e5..8915a0b 100644 --- a/qmp-commands.hx +++ b/qmp-commands.hx @@ -1186,7 +1186,8 @@ EQMP { .name = "drive-backup", .args_type = "sync:s,device:B,target:s,speed:i?,mode:s?,format:s?," - "bitmap:s?,on-source-error:s?,on-target-error:s?", + "bitmap:s?,compress:b?," + "on-source-error:s?,on-target-error:s?", .mhandler.cmd_new = qmp_marshal_drive_backup, }, @@ -1220,6 +1221,7 @@ Arguments: - "mode": whether and how QEMU should create a new image (NewImageMode, optional, default 'absolute-paths') - "speed": the maximum speed, in bytes per second (json-int, optional) +- "compress": compress data blocks (if the target format supports it). - "on-source-error": the action to take on an error on the source, default 'report'. 'stop' and 'enospc' can only be used if the block device supports io-status.