From patchwork Fri Jul 22 08:17:52 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 9243133 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9E338602F0 for ; Fri, 22 Jul 2016 08:31:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8BD3F27D76 for ; Fri, 22 Jul 2016 08:31:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 808A027F95; Fri, 22 Jul 2016 08:31:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C182E27D76 for ; Fri, 22 Jul 2016 08:31:42 +0000 (UTC) Received: from localhost ([::1]:45853 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQVrx-00017F-RU for patchwork-qemu-devel@patchwork.kernel.org; Fri, 22 Jul 2016 04:31:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56002) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQVer-0005vA-Le for qemu-devel@nongnu.org; Fri, 22 Jul 2016 04:18:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bQVel-0001rm-9x for qemu-devel@nongnu.org; Fri, 22 Jul 2016 04:18:08 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:45819 helo=relay.sw.ru) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQVek-0001qB-Oa for qemu-devel@nongnu.org; Fri, 22 Jul 2016 04:18:03 -0400 Received: from irbis.sw.ru ([10.30.2.139]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id u6L0IZxK024246; Thu, 21 Jul 2016 03:18:39 +0300 (MSK) From: "Denis V. Lunev" To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 22 Jul 2016 11:17:52 +0300 Message-Id: <1469175475-15420-14-git-send-email-den@openvz.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1469175475-15420-1-git-send-email-den@openvz.org> References: <1469175475-15420-1-git-send-email-den@openvz.org> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v7 13/16] drive-backup: added support for data compression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Pavel Butsykin , Jeff Cody , Markus Armbruster , Stefan Hajnoczi , den@openvz.org, John Snow Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Pavel Butsykin The idea is simple - backup is "written-once" data. It is written block by block and it is large enough. It would be nice to save storage space and compress it. The patch adds a flag to the qmp/hmp drive-backup command which enables block compression. Compression should be implemented in the format driver to enable this feature. There are some limitations of the format driver to allow compressed writes. We can write data only once. Though for backup this is perfectly fine. These limitations are maintained by the driver and the error will be reported if we are doing something wrong. Signed-off-by: Pavel Butsykin Reviewed-by: Stefan Hajnoczi Signed-off-by: Denis V. Lunev CC: Jeff Cody CC: Markus Armbruster CC: Eric Blake CC: John Snow CC: Stefan Hajnoczi CC: Kevin Wolf --- block/backup.c | 12 +++++++++++- blockdev.c | 9 ++++++--- hmp-commands.hx | 8 +++++--- hmp.c | 3 +++ include/block/block_int.h | 1 + qapi/block-core.json | 5 ++++- qmp-commands.hx | 5 ++++- 7 files changed, 34 insertions(+), 9 deletions(-) diff --git a/block/backup.c b/block/backup.c index 2c05323..bb3bb9a 100644 --- a/block/backup.c +++ b/block/backup.c @@ -47,6 +47,7 @@ typedef struct BackupBlockJob { uint64_t sectors_read; unsigned long *done_bitmap; int64_t cluster_size; + bool compress; NotifierWithReturn before_write; QLIST_HEAD(, CowRequest) inflight_reqs; } BackupBlockJob; @@ -154,7 +155,8 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, bounce_qiov.size, BDRV_REQ_MAY_UNMAP); } else { ret = blk_co_pwritev(job->target, start * job->cluster_size, - bounce_qiov.size, &bounce_qiov, 0); + bounce_qiov.size, &bounce_qiov, + job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0); } if (ret < 0) { trace_backup_do_cow_write_fail(job, start, ret); @@ -477,6 +479,7 @@ static void coroutine_fn backup_run(void *opaque) void backup_start(const char *job_id, BlockDriverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, + bool compress, BlockdevOnError on_source_error, BlockdevOnError on_target_error, BlockCompletionFunc *cb, void *opaque, @@ -507,6 +510,12 @@ void backup_start(const char *job_id, BlockDriverState *bs, return; } + if (compress && target->drv->bdrv_co_pwritev_compressed == NULL) { + error_setg(errp, "Compression is not supported for this drive %s", + bdrv_get_device_name(target)); + return; + } + if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) { return; } @@ -555,6 +564,7 @@ void backup_start(const char *job_id, BlockDriverState *bs, job->sync_mode = sync_mode; job->sync_bitmap = sync_mode == MIRROR_SYNC_MODE_INCREMENTAL ? sync_bitmap : NULL; + job->compress = compress; /* If there is no backing file on the target, we cannot rely on COW if our * backup cluster size is smaller than the target cluster size. Even for diff --git a/blockdev.c b/blockdev.c index 0c5ea25..587d76b 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3154,6 +3154,9 @@ static void do_drive_backup(DriveBackup *backup, BlockJobTxn *txn, Error **errp) if (!backup->has_job_id) { backup->job_id = NULL; } + if (!backup->has_compress) { + backup->compress = false; + } blk = blk_by_name(backup->device); if (!blk) { @@ -3242,8 +3245,8 @@ static void do_drive_backup(DriveBackup *backup, BlockJobTxn *txn, Error **errp) } backup_start(backup->job_id, bs, target_bs, backup->speed, backup->sync, - bmap, backup->on_source_error, backup->on_target_error, - block_job_cb, bs, txn, &local_err); + bmap, backup->compress, backup->on_source_error, + backup->on_target_error, block_job_cb, bs, txn, &local_err); bdrv_unref(target_bs); if (local_err != NULL) { error_propagate(errp, local_err); @@ -3317,7 +3320,7 @@ void do_blockdev_backup(BlockdevBackup *backup, BlockJobTxn *txn, Error **errp) } } backup_start(backup->job_id, bs, target_bs, backup->speed, backup->sync, - NULL, backup->on_source_error, backup->on_target_error, + NULL, false, backup->on_source_error, backup->on_target_error, block_job_cb, bs, txn, &local_err); if (local_err != NULL) { error_propagate(errp, local_err); diff --git a/hmp-commands.hx b/hmp-commands.hx index 848efee..74f32e5 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -1182,8 +1182,8 @@ ETEXI { .name = "drive_backup", - .args_type = "reuse:-n,full:-f,device:B,target:s,format:s?", - .params = "[-n] [-f] device target [format]", + .args_type = "reuse:-n,full:-f,compress:-c,device:B,target:s,format:s?", + .params = "[-n] [-f] [-c] device target [format]", .help = "initiates a point-in-time\n\t\t\t" "copy for a device. The device's contents are\n\t\t\t" "copied to the new image file, excluding data that\n\t\t\t" @@ -1191,7 +1191,9 @@ ETEXI "The -n flag requests QEMU to reuse the image found\n\t\t\t" "in new-image-file, instead of recreating it from scratch.\n\t\t\t" "The -f flag requests QEMU to copy the whole disk,\n\t\t\t" - "so that the result does not need a backing file.\n\t\t\t", + "so that the result does not need a backing file.\n\t\t\t" + "The -c flag requests QEMU to compress backup data\n\t\t\t" + "(if the target format supports it).\n\t\t\t", .mhandler.cmd = hmp_drive_backup, }, STEXI diff --git a/hmp.c b/hmp.c index 3c06200..a7dfe6f 100644 --- a/hmp.c +++ b/hmp.c @@ -1109,6 +1109,7 @@ void hmp_drive_backup(Monitor *mon, const QDict *qdict) const char *format = qdict_get_try_str(qdict, "format"); bool reuse = qdict_get_try_bool(qdict, "reuse", false); bool full = qdict_get_try_bool(qdict, "full", false); + bool compress = qdict_get_try_bool(qdict, "compress", false); Error *err = NULL; DriveBackup backup = { .device = (char *)device, @@ -1118,6 +1119,8 @@ void hmp_drive_backup(Monitor *mon, const QDict *qdict) .sync = full ? MIRROR_SYNC_MODE_FULL : MIRROR_SYNC_MODE_TOP, .has_mode = true, .mode = reuse ? NEW_IMAGE_MODE_EXISTING : NEW_IMAGE_MODE_ABSOLUTE_PATHS, + .has_compress = !!compress, + .compress = compress, }; if (!filename) { diff --git a/include/block/block_int.h b/include/block/block_int.h index 378c966..da94066 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -762,6 +762,7 @@ void mirror_start(const char *job_id, BlockDriverState *bs, void backup_start(const char *job_id, BlockDriverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, + bool compress, BlockdevOnError on_source_error, BlockdevOnError on_target_error, BlockCompletionFunc *cb, void *opaque, diff --git a/qapi/block-core.json b/qapi/block-core.json index 44cf9c6..6d98da7 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -894,6 +894,9 @@ # Must be present if sync is "incremental", must NOT be present # otherwise. (Since 2.4) # +# @compress: #optional true to compress data, if the target format supports it. +# (default: false) (since 2.7) +# # @on-source-error: #optional the action to take on an error on the source, # default 'report'. 'stop' and 'enospc' can only be used # if the block device supports io-status (see BlockInfo). @@ -911,7 +914,7 @@ { 'struct': 'DriveBackup', 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str', '*format': 'str', 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode', - '*speed': 'int', '*bitmap': 'str', + '*speed': 'int', '*bitmap': 'str', '*compress': 'bool', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError' } } diff --git a/qmp-commands.hx b/qmp-commands.hx index c8d360a..ff72194 100644 --- a/qmp-commands.hx +++ b/qmp-commands.hx @@ -1217,7 +1217,8 @@ EQMP { .name = "drive-backup", .args_type = "job-id:s?,sync:s,device:B,target:s,speed:i?,mode:s?," - "format:s?,bitmap:s?,on-source-error:s?,on-target-error:s?", + "format:s?,bitmap:s?,compress:b?," + "on-source-error:s?,on-target-error:s?", .mhandler.cmd_new = qmp_marshal_drive_backup, }, @@ -1253,6 +1254,8 @@ Arguments: - "mode": whether and how QEMU should create a new image (NewImageMode, optional, default 'absolute-paths') - "speed": the maximum speed, in bytes per second (json-int, optional) +- "compress": true to compress data, if the target format supports it. + (json-bool, optional, default false) - "on-source-error": the action to take on an error on the source, default 'report'. 'stop' and 'enospc' can only be used if the block device supports io-status.