From patchwork Tue Jul 30 17:24:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066323 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C83F013B1 for ; Tue, 30 Jul 2019 17:28:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A8D26285D4 for ; Tue, 30 Jul 2019 17:28:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9951A205A8; Tue, 30 Jul 2019 17:28:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 31D43205A8 for ; Tue, 30 Jul 2019 17:28:31 +0000 (UTC) Received: from localhost ([::1]:34920 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVvK-0004m9-Ip for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:28:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33459) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsI-0006eG-Hi for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsH-0002Wc-Ed for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:22 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57972) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsA-0002Pg-D0; Tue, 30 Jul 2019 13:25:14 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3B68E3083391; Tue, 30 Jul 2019 17:25:13 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A398F60856; Tue, 30 Jul 2019 17:25:12 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:24:56 +0200 Message-Id: <20190730172508.19911-2-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Tue, 30 Jul 2019 17:25:13 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 01/13] qcow2: Add Error ** to qcow2_read_snapshots() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.h | 2 +- block/qcow2-snapshot.c | 7 ++++++- block/qcow2.c | 3 +-- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index fc1b0d3c1e..175708cee0 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -704,7 +704,7 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs, Error **errp); void qcow2_free_snapshots(BlockDriverState *bs); -int qcow2_read_snapshots(BlockDriverState *bs); +int qcow2_read_snapshots(BlockDriverState *bs, Error **errp); /* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables, diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index d0e7fa9311..6295ba4c9c 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -43,7 +43,7 @@ void qcow2_free_snapshots(BlockDriverState *bs) s->nb_snapshots = 0; } -int qcow2_read_snapshots(BlockDriverState *bs) +int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) { BDRVQcow2State *s = bs->opaque; QCowSnapshotHeader h; @@ -68,6 +68,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) offset = ROUND_UP(offset, 8); ret = bdrv_pread(bs->file, offset, &h, sizeof(h)); if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read snapshot table"); goto fail; } @@ -88,6 +89,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) ret = bdrv_pread(bs->file, offset, &extra, MIN(sizeof(extra), extra_data_size)); if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read snapshot table"); goto fail; } offset += extra_data_size; @@ -106,6 +108,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) sn->id_str = g_malloc(id_str_size + 1); ret = bdrv_pread(bs->file, offset, sn->id_str, id_str_size); if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read snapshot table"); goto fail; } offset += id_str_size; @@ -115,6 +118,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) sn->name = g_malloc(name_size + 1); ret = bdrv_pread(bs->file, offset, sn->name, name_size); if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read snapshot table"); goto fail; } offset += name_size; @@ -122,6 +126,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) if (offset - s->snapshots_offset > QCOW_MAX_SNAPSHOTS_SIZE) { ret = -EFBIG; + error_setg(errp, "Snapshot table is too big"); goto fail; } } diff --git a/block/qcow2.c b/block/qcow2.c index 039bdc2f7e..0336deaf2c 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1577,9 +1577,8 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, s->snapshots_offset = header.snapshots_offset; s->nb_snapshots = header.nb_snapshots; - ret = qcow2_read_snapshots(bs); + ret = qcow2_read_snapshots(bs, errp); if (ret < 0) { - error_setg_errno(errp, -ret, "Could not read snapshots"); goto fail; } From patchwork Tue Jul 30 17:24:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066341 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6414614DB for ; Tue, 30 Jul 2019 17:32:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 488AC286B9 for ; Tue, 30 Jul 2019 17:32:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3C6C8286BE; Tue, 30 Jul 2019 17:32:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id AFA75286B9 for ; Tue, 30 Jul 2019 17:32:56 +0000 (UTC) Received: from localhost ([::1]:35028 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVzb-0005tl-T0 for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:32:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33503) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsK-0006iv-Ee for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsJ-0002Yk-2R for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55712) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsE-0002RQ-7f; Tue, 30 Jul 2019 13:25:19 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5EEBD3082E55; Tue, 30 Jul 2019 17:25:16 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AEDAD5C1A1; Tue, 30 Jul 2019 17:25:15 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:24:57 +0200 Message-Id: <20190730172508.19911-3-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Tue, 30 Jul 2019 17:25:16 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 02/13] qcow2: Keep unknown extra snapshot data X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP The qcow2 specification says to ignore unknown extra data fields in snapshot table entries. Currently, we discard it whenever we update the image, which is a bit different from "ignore". This patch makes the qcow2 driver keep all unknown extra data fields when updating an image's snapshot table. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.h | 5 ++++ block/qcow2-snapshot.c | 59 +++++++++++++++++++++++++++++++++++------- 2 files changed, 55 insertions(+), 9 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 175708cee0..290a48b77e 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -61,6 +61,9 @@ * space for snapshot names and IDs */ #define QCOW_MAX_SNAPSHOTS_SIZE (1024 * QCOW_MAX_SNAPSHOTS) +/* Maximum amount of extra data per snapshot table entry to accept */ +#define QCOW_MAX_SNAPSHOT_EXTRA_DATA 1024 + /* Bitmap header extension constraints */ #define QCOW2_MAX_BITMAPS 65535 #define QCOW2_MAX_BITMAP_DIRECTORY_SIZE (1024 * QCOW2_MAX_BITMAPS) @@ -178,6 +181,8 @@ typedef struct QCowSnapshot { uint32_t date_sec; uint32_t date_nsec; uint64_t vm_clock_nsec; + uint32_t extra_data_size; + void *unknown_extra_data; /* Extra data past QCowSnapshotExtraData */ } QCowSnapshot; struct Qcow2Cache; diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 6295ba4c9c..b037c6e5fd 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -37,6 +37,7 @@ void qcow2_free_snapshots(BlockDriverState *bs) for(i = 0; i < s->nb_snapshots; i++) { g_free(s->snapshots[i].name); g_free(s->snapshots[i].id_str); + g_free(s->snapshots[i].unknown_extra_data); } g_free(s->snapshots); s->snapshots = NULL; @@ -51,7 +52,6 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) QCowSnapshot *sn; int i, id_str_size, name_size; int64_t offset; - uint32_t extra_data_size; int ret; if (!s->nb_snapshots) { @@ -80,30 +80,52 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) sn->date_sec = be32_to_cpu(h.date_sec); sn->date_nsec = be32_to_cpu(h.date_nsec); sn->vm_clock_nsec = be64_to_cpu(h.vm_clock_nsec); - extra_data_size = be32_to_cpu(h.extra_data_size); + sn->extra_data_size = be32_to_cpu(h.extra_data_size); id_str_size = be16_to_cpu(h.id_str_size); name_size = be16_to_cpu(h.name_size); - /* Read extra data */ + if (sn->extra_data_size > QCOW_MAX_SNAPSHOT_EXTRA_DATA) { + ret = -EFBIG; + error_setg(errp, "Too much extra metadata in snapshot table " + "entry %i", i); + goto fail; + } + + /* Read known extra data */ ret = bdrv_pread(bs->file, offset, &extra, - MIN(sizeof(extra), extra_data_size)); + MIN(sizeof(extra), sn->extra_data_size)); if (ret < 0) { error_setg_errno(errp, -ret, "Failed to read snapshot table"); goto fail; } - offset += extra_data_size; + offset += MIN(sizeof(extra), sn->extra_data_size); - if (extra_data_size >= 8) { + if (sn->extra_data_size >= 8) { sn->vm_state_size = be64_to_cpu(extra.vm_state_size_large); } - if (extra_data_size >= 16) { + if (sn->extra_data_size >= 16) { sn->disk_size = be64_to_cpu(extra.disk_size); } else { sn->disk_size = bs->total_sectors * BDRV_SECTOR_SIZE; } + if (sn->extra_data_size > sizeof(extra)) { + /* Store unknown extra data */ + size_t unknown_extra_data_size = + sn->extra_data_size - sizeof(extra); + + sn->unknown_extra_data = g_malloc(unknown_extra_data_size); + ret = bdrv_pread(bs->file, offset, sn->unknown_extra_data, + unknown_extra_data_size); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read snapshot table"); + goto fail; + } + offset += unknown_extra_data_size; + } + /* Read snapshot ID */ sn->id_str = g_malloc(id_str_size + 1); ret = bdrv_pread(bs->file, offset, sn->id_str, id_str_size); @@ -161,7 +183,7 @@ static int qcow2_write_snapshots(BlockDriverState *bs) sn = s->snapshots + i; offset = ROUND_UP(offset, 8); offset += sizeof(h); - offset += sizeof(extra); + offset += MAX(sizeof(extra), sn->extra_data_size); offset += strlen(sn->id_str); offset += strlen(sn->name); @@ -208,7 +230,8 @@ static int qcow2_write_snapshots(BlockDriverState *bs) h.date_sec = cpu_to_be32(sn->date_sec); h.date_nsec = cpu_to_be32(sn->date_nsec); h.vm_clock_nsec = cpu_to_be64(sn->vm_clock_nsec); - h.extra_data_size = cpu_to_be32(sizeof(extra)); + h.extra_data_size = cpu_to_be32(MAX(sizeof(extra), + sn->extra_data_size)); memset(&extra, 0, sizeof(extra)); extra.vm_state_size_large = cpu_to_be64(sn->vm_state_size); @@ -233,6 +256,22 @@ static int qcow2_write_snapshots(BlockDriverState *bs) } offset += sizeof(extra); + if (sn->extra_data_size > sizeof(extra)) { + size_t unknown_extra_data_size = + sn->extra_data_size - sizeof(extra); + + /* qcow2_read_snapshots() ensures no unbounded allocation */ + assert(unknown_extra_data_size <= INT_MAX); + assert(sn->unknown_extra_data); + + ret = bdrv_pwrite(bs->file, offset, sn->unknown_extra_data, + unknown_extra_data_size); + if (ret < 0) { + goto fail; + } + offset += unknown_extra_data_size; + } + ret = bdrv_pwrite(bs->file, offset, sn->id_str, id_str_size); if (ret < 0) { goto fail; @@ -375,6 +414,7 @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info) sn->date_sec = sn_info->date_sec; sn->date_nsec = sn_info->date_nsec; sn->vm_clock_nsec = sn_info->vm_clock_nsec; + sn->extra_data_size = sizeof(QCowSnapshotExtraData); /* Allocate the L1 table of the snapshot and copy the current one there. */ l1_table_offset = qcow2_alloc_clusters(bs, s->l1_size * sizeof(uint64_t)); @@ -646,6 +686,7 @@ int qcow2_snapshot_delete(BlockDriverState *bs, * The snapshot is now unused, clean up. If we fail after this point, we * won't recover but just leak clusters. */ + g_free(sn.unknown_extra_data); g_free(sn.id_str); g_free(sn.name); From patchwork Tue Jul 30 17:24:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066347 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A81D41395 for ; Tue, 30 Jul 2019 17:33:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F90B286BE for ; Tue, 30 Jul 2019 17:33:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 836ED28710; Tue, 30 Jul 2019 17:33:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 366C4286BE for ; Tue, 30 Jul 2019 17:33:58 +0000 (UTC) Received: from localhost ([::1]:35046 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsW0b-0008I3-HF for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:33:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33474) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsJ-0006hJ-FO for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsI-0002YF-HI for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38410) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsG-0002Tq-83; Tue, 30 Jul 2019 13:25:20 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0D35B81DFF; Tue, 30 Jul 2019 17:25:19 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7F3B060856; Tue, 30 Jul 2019 17:25:18 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:24:58 +0200 Message-Id: <20190730172508.19911-4-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 30 Jul 2019 17:25:19 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 03/13] qcow2: Make qcow2_write_snapshots() public X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Updating the snapshot list will be useful when upgrading a v2 image to v3, so we will need to call this function in qcow2.c. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.h | 1 + block/qcow2-snapshot.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/block/qcow2.h b/block/qcow2.h index 290a48b77e..77586d81b9 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -710,6 +710,7 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs, void qcow2_free_snapshots(BlockDriverState *bs); int qcow2_read_snapshots(BlockDriverState *bs, Error **errp); +int qcow2_write_snapshots(BlockDriverState *bs); /* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables, diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index b037c6e5fd..3d097b92d7 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -163,7 +163,7 @@ fail: } /* add at the end of the file a new list of snapshots */ -static int qcow2_write_snapshots(BlockDriverState *bs) +int qcow2_write_snapshots(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; QCowSnapshot *sn; From patchwork Tue Jul 30 17:24:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066333 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1DF971395 for ; Tue, 30 Jul 2019 17:30:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 00C7D1FFBD for ; Tue, 30 Jul 2019 17:30:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E9173205A8; Tue, 30 Jul 2019 17:30:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 911E81FFBD for ; Tue, 30 Jul 2019 17:30:22 +0000 (UTC) Received: from localhost ([::1]:34952 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVx7-0008MQ-Va for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:30:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33523) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsM-0006qs-Hg for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsL-0002aS-FF for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39457) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsI-0002X8-LY; Tue, 30 Jul 2019 13:25:22 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EEF07796ED; Tue, 30 Jul 2019 17:25:21 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5568C5C1A1; Tue, 30 Jul 2019 17:25:21 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:24:59 +0200 Message-Id: <20190730172508.19911-5-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 30 Jul 2019 17:25:22 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 04/13] qcow2: Put qcow2_upgrade() into an own function X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This does not make sense right now, but it will make sense once we need to do more than to just update s->qcow_version. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.c | 43 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 38 insertions(+), 5 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 0336deaf2c..99e9dad798 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -4721,12 +4721,46 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version, return 0; } +/* + * Upgrades an image's version. While newer versions encompass all + * features of older versions, some things may have to be presented + * differently. + */ +static int qcow2_upgrade(BlockDriverState *bs, int target_version, + BlockDriverAmendStatusCB *status_cb, void *cb_opaque, + Error **errp) +{ + BDRVQcow2State *s = bs->opaque; + int current_version = s->qcow_version; + int ret; + + /* This is qcow2_upgrade(), not qcow2_downgrade() */ + assert(target_version > current_version); + + /* There are no other versions (yet) that you can upgrade to */ + assert(target_version == 3); + + status_cb(bs, 0, 1, cb_opaque); + + s->qcow_version = target_version; + ret = qcow2_update_header(bs); + if (ret < 0) { + s->qcow_version = current_version; + error_setg_errno(errp, -ret, "Failed to update the image header"); + return ret; + } + status_cb(bs, 1, 1, cb_opaque); + + return 0; +} + typedef enum Qcow2AmendOperation { /* This is the value Qcow2AmendHelperCBInfo::last_operation will be * statically initialized to so that the helper CB can discern the first * invocation from an operation change */ QCOW2_NO_OPERATION = 0, + QCOW2_UPGRADING, QCOW2_CHANGING_REFCOUNT_ORDER, QCOW2_DOWNGRADING, } Qcow2AmendOperation; @@ -4909,17 +4943,16 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, helper_cb_info = (Qcow2AmendHelperCBInfo){ .original_status_cb = status_cb, .original_cb_opaque = cb_opaque, - .total_operations = (new_version < old_version) + .total_operations = (new_version != old_version) + (s->refcount_bits != refcount_bits) }; /* Upgrade first (some features may require compat=1.1) */ if (new_version > old_version) { - s->qcow_version = new_version; - ret = qcow2_update_header(bs); + helper_cb_info.current_operation = QCOW2_UPGRADING; + ret = qcow2_upgrade(bs, new_version, &qcow2_amend_helper_cb, + &helper_cb_info, errp); if (ret < 0) { - s->qcow_version = old_version; - error_setg_errno(errp, -ret, "Failed to update the image header"); return ret; } } From patchwork Tue Jul 30 17:25:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066335 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7F7FC14DB for ; Tue, 30 Jul 2019 17:32:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 65D81286BE for ; Tue, 30 Jul 2019 17:32:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5A59428710; Tue, 30 Jul 2019 17:32:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 04C5C286BE for ; Tue, 30 Jul 2019 17:32:10 +0000 (UTC) Received: from localhost ([::1]:34992 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVyr-0003bQ-D2 for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:32:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33553) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsT-0006yV-31 for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsQ-0002fv-KX for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:32 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41582) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsL-0002aK-Ku; Tue, 30 Jul 2019 13:25:25 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E3303300CA4E; Tue, 30 Jul 2019 17:25:24 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5DBEA5D978; Tue, 30 Jul 2019 17:25:24 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:25:00 +0200 Message-Id: <20190730172508.19911-6-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Tue, 30 Jul 2019 17:25:24 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 05/13] qcow2: Write v3-compliant snapshot list on upgrade X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP qcow2 v3 requires every snapshot table entry to have two extra data fields: The 64-bit VM state size, and the virtual disk size. Both are optional for v2 images, so they may not be present. qcow2_upgrade() therefore should update the snapshot table to ensure all entries have these extra data fields. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1727347 Reported-by: Eric Blake Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 99e9dad798..75d41683a1 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -4731,7 +4731,9 @@ static int qcow2_upgrade(BlockDriverState *bs, int target_version, Error **errp) { BDRVQcow2State *s = bs->opaque; + bool need_snapshot_update; int current_version = s->qcow_version; + int i; int ret; /* This is qcow2_upgrade(), not qcow2_downgrade() */ @@ -4740,7 +4742,30 @@ static int qcow2_upgrade(BlockDriverState *bs, int target_version, /* There are no other versions (yet) that you can upgrade to */ assert(target_version == 3); - status_cb(bs, 0, 1, cb_opaque); + status_cb(bs, 0, 2, cb_opaque); + + /* + * In v2, snapshots do not need to have extra data. v3 requires + * the 64-bit VM state size and the virtual disk size to be + * present. + * qcow2_write_snapshots() will always write the list in the + * v3-compliant format. + */ + need_snapshot_update = false; + for (i = 0; i < s->nb_snapshots; i++) { + if (s->snapshots[i].extra_data_size < 16) { + need_snapshot_update = true; + break; + } + } + if (need_snapshot_update) { + ret = qcow2_write_snapshots(bs); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to update the snapshot table"); + return ret; + } + } + status_cb(bs, 1, 2, cb_opaque); s->qcow_version = target_version; ret = qcow2_update_header(bs); @@ -4749,7 +4774,7 @@ static int qcow2_upgrade(BlockDriverState *bs, int target_version, error_setg_errno(errp, -ret, "Failed to update the image header"); return ret; } - status_cb(bs, 1, 1, cb_opaque); + status_cb(bs, 2, 2, cb_opaque); return 0; } From patchwork Tue Jul 30 17:25:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066355 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E2A814DB for ; Tue, 30 Jul 2019 17:35:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 34FE8205AD for ; Tue, 30 Jul 2019 17:35:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 29491286F1; Tue, 30 Jul 2019 17:35:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9CAE2205AD for ; Tue, 30 Jul 2019 17:35:15 +0000 (UTC) Received: from localhost ([::1]:35072 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsW1q-0003Ao-VU for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:35:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33590) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsW-000705-Ae for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsU-0002is-EA for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:19689) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsO-0002bj-VL; Tue, 30 Jul 2019 13:25:29 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9FE3A3087958; Tue, 30 Jul 2019 17:25:27 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EE90F5D9C5; Tue, 30 Jul 2019 17:25:26 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:25:01 +0200 Message-Id: <20190730172508.19911-7-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Tue, 30 Jul 2019 17:25:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 06/13] qcow2: Separate qcow2_check_read_snapshot_table() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Reading the snapshot table can fail. That is a problem when we want to repair the image. Therefore, stop reading the snapshot table in qcow2_do_open() in check mode. Instead, add a new function qcow2_check_read_snapshot_table() that reads the snapshot table at a later point. In the future, we want to handle errors here and fix them. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.h | 4 +++ block/qcow2-snapshot.c | 58 ++++++++++++++++++++++++++++++++ block/qcow2.c | 76 ++++++++++++++++++++++++++++++++---------- 3 files changed, 120 insertions(+), 18 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 77586d81b9..50c7eefb5b 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -712,6 +712,10 @@ void qcow2_free_snapshots(BlockDriverState *bs); int qcow2_read_snapshots(BlockDriverState *bs, Error **errp); int qcow2_write_snapshots(BlockDriverState *bs); +int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, + BdrvCheckResult *result, + BdrvCheckMode fix); + /* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables, unsigned table_size); diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 3d097b92d7..cbc6d699b6 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -321,6 +321,64 @@ fail: return ret; } +int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, + BdrvCheckResult *result, + BdrvCheckMode fix) +{ + BDRVQcow2State *s = bs->opaque; + Error *local_err = NULL; + int ret; + struct { + uint32_t nb_snapshots; + uint64_t snapshots_offset; + } QEMU_PACKED snapshot_table_pointer; + + /* qcow2_do_open() discards this information in check mode */ + ret = bdrv_pread(bs->file, 60, &snapshot_table_pointer, + sizeof(snapshot_table_pointer)); + if (ret < 0) { + result->check_errors++; + fprintf(stderr, "ERROR failed to read the snapshot table pointer from " + "the image header: %s\n", strerror(-ret)); + return ret; + } + + s->snapshots_offset = be64_to_cpu(snapshot_table_pointer.snapshots_offset); + s->nb_snapshots = be32_to_cpu(snapshot_table_pointer.nb_snapshots); + + ret = qcow2_validate_table(bs, s->snapshots_offset, s->nb_snapshots, + sizeof(QCowSnapshotHeader), + sizeof(QCowSnapshotHeader) * QCOW_MAX_SNAPSHOTS, + "snapshot table", &local_err); + if (ret < 0) { + result->check_errors++; + error_reportf_err(local_err, "ERROR "); + + /* We did not read the snapshot table, so invalidate this information */ + s->snapshots_offset = 0; + s->nb_snapshots = 0; + + return ret; + } + + qemu_co_mutex_unlock(&s->lock); + ret = qcow2_read_snapshots(bs, &local_err); + qemu_co_mutex_lock(&s->lock); + if (ret < 0) { + result->check_errors++; + error_reportf_err(local_err, + "ERROR failed to read the snapshot table: "); + + /* We did not read the snapshot table, so invalidate this information */ + s->snapshots_offset = 0; + s->nb_snapshots = 0; + + return ret; + } + + return 0; +} + static void find_new_snapshot_id(BlockDriverState *bs, char *id_str, int id_str_size) { diff --git a/block/qcow2.c b/block/qcow2.c index 75d41683a1..2983a7795a 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -567,11 +567,40 @@ int qcow2_mark_consistent(BlockDriverState *bs) return 0; } +static void qcow2_add_check_result(BdrvCheckResult *out, + const BdrvCheckResult *src, + bool set_allocation_info) +{ + out->corruptions += src->corruptions; + out->leaks += src->leaks; + out->check_errors += src->check_errors; + out->corruptions_fixed += src->corruptions_fixed; + out->leaks_fixed += src->leaks_fixed; + + if (set_allocation_info) { + out->image_end_offset = src->image_end_offset; + out->bfi = src->bfi; + } +} + static int coroutine_fn qcow2_co_check_locked(BlockDriverState *bs, BdrvCheckResult *result, BdrvCheckMode fix) { - int ret = qcow2_check_refcounts(bs, result, fix); + BdrvCheckResult snapshot_res = {}; + BdrvCheckResult refcount_res = {}; + int ret; + + memset(result, 0, sizeof(*result)); + + ret = qcow2_check_read_snapshot_table(bs, &snapshot_res, fix); + qcow2_add_check_result(result, &snapshot_res, false); + if (ret < 0) { + return ret; + } + + ret = qcow2_check_refcounts(bs, &refcount_res, fix); + qcow2_add_check_result(result, &refcount_res, true); if (ret < 0) { return ret; } @@ -1403,17 +1432,22 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, goto fail; } - /* The total size in bytes of the snapshot table is checked in - * qcow2_read_snapshots() because the size of each snapshot is - * variable and we don't know it yet. - * Here we only check the offset and number of snapshots. */ - ret = qcow2_validate_table(bs, header.snapshots_offset, - header.nb_snapshots, - sizeof(QCowSnapshotHeader), - sizeof(QCowSnapshotHeader) * QCOW_MAX_SNAPSHOTS, - "Snapshot table", errp); - if (ret < 0) { - goto fail; + if (!(flags & BDRV_O_CHECK)) { + /* + * The total size in bytes of the snapshot table is checked in + * qcow2_read_snapshots() because the size of each snapshot is + * variable and we don't know it yet. + * Here we only check the offset and number of snapshots. + */ + ret = qcow2_validate_table(bs, header.snapshots_offset, + header.nb_snapshots, + sizeof(QCowSnapshotHeader), + sizeof(QCowSnapshotHeader) * + QCOW_MAX_SNAPSHOTS, + "Snapshot table", errp); + if (ret < 0) { + goto fail; + } } /* read the level 1 table */ @@ -1573,13 +1607,19 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, s->image_backing_file = g_strdup(bs->auto_backing_file); } - /* Internal snapshots */ - s->snapshots_offset = header.snapshots_offset; - s->nb_snapshots = header.nb_snapshots; + /* + * Internal snapshots; skip reading them in check mode, because + * we do not need them then, and we do not want to abort because + * of a broken table. + */ + if (!(flags & BDRV_O_CHECK)) { + s->snapshots_offset = header.snapshots_offset; + s->nb_snapshots = header.nb_snapshots; - ret = qcow2_read_snapshots(bs, errp); - if (ret < 0) { - goto fail; + ret = qcow2_read_snapshots(bs, errp); + if (ret < 0) { + goto fail; + } } /* Clear unknown autoclear feature bits */ From patchwork Tue Jul 30 17:25:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066337 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0C0B81395 for ; Tue, 30 Jul 2019 17:32:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5C46286BE for ; Tue, 30 Jul 2019 17:32:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DA55928710; Tue, 30 Jul 2019 17:32:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7DC59286BE for ; Tue, 30 Jul 2019 17:32:11 +0000 (UTC) Received: from localhost ([::1]:34996 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVys-0003fb-ST for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:32:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33637) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsb-000765-6O for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsY-0002lO-Lo for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42304) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsS-0002g3-Fu; Tue, 30 Jul 2019 13:25:34 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A56F3C0669D5; Tue, 30 Jul 2019 17:25:30 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1D4D05D9C5; Tue, 30 Jul 2019 17:25:29 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:25:02 +0200 Message-Id: <20190730172508.19911-8-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 30 Jul 2019 17:25:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 07/13] qcow2: Add qcow2_check_fix_snapshot_table() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP qcow2_check_read_snapshot_table() can perform consistency checks, but it cannot fix everything. Specifically, it cannot allocate new clusters, because that should wait until the refcount structures are known to be consistent (i.e., after qcow2_check_refcounts()). Thus, it cannot call qcow2_write_snapshots(). Do that in qcow2_check_fix_snapshot_table(), which is called after qcow2_check_refcounts(). Currently, there is nothing that would set result->corruptions, so this is a no-op. A follow-up patch will change that. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.h | 3 +++ block/qcow2-snapshot.c | 25 +++++++++++++++++++++++++ block/qcow2.c | 9 ++++++++- 3 files changed, 36 insertions(+), 1 deletion(-) diff --git a/block/qcow2.h b/block/qcow2.h index 50c7eefb5b..6823c37e82 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -715,6 +715,9 @@ int qcow2_write_snapshots(BlockDriverState *bs); int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, BdrvCheckResult *result, BdrvCheckMode fix); +int coroutine_fn qcow2_check_fix_snapshot_table(BlockDriverState *bs, + BdrvCheckResult *result, + BdrvCheckMode fix); /* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables, diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index cbc6d699b6..58609b6ea4 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -379,6 +379,31 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, return 0; } +int coroutine_fn qcow2_check_fix_snapshot_table(BlockDriverState *bs, + BdrvCheckResult *result, + BdrvCheckMode fix) +{ + BDRVQcow2State *s = bs->opaque; + int ret; + + if (result->corruptions && (fix & BDRV_FIX_ERRORS)) { + qemu_co_mutex_unlock(&s->lock); + ret = qcow2_write_snapshots(bs); + qemu_co_mutex_lock(&s->lock); + if (ret < 0) { + result->check_errors++; + fprintf(stderr, "ERROR failed to update snapshot table: %s\n", + strerror(-ret)); + return ret; + } + + result->corruptions_fixed += result->corruptions; + result->corruptions = 0; + } + + return 0; +} + static void find_new_snapshot_id(BlockDriverState *bs, char *id_str, int id_str_size) { diff --git a/block/qcow2.c b/block/qcow2.c index 2983a7795a..ac94cc7807 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -594,13 +594,20 @@ static int coroutine_fn qcow2_co_check_locked(BlockDriverState *bs, memset(result, 0, sizeof(*result)); ret = qcow2_check_read_snapshot_table(bs, &snapshot_res, fix); - qcow2_add_check_result(result, &snapshot_res, false); if (ret < 0) { + qcow2_add_check_result(result, &snapshot_res, false); return ret; } ret = qcow2_check_refcounts(bs, &refcount_res, fix); qcow2_add_check_result(result, &refcount_res, true); + if (ret < 0) { + qcow2_add_check_result(result, &snapshot_res, false); + return ret; + } + + ret = qcow2_check_fix_snapshot_table(bs, &snapshot_res, fix); + qcow2_add_check_result(result, &snapshot_res, false); if (ret < 0) { return ret; } From patchwork Tue Jul 30 17:25:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066345 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 524CA1395 for ; Tue, 30 Jul 2019 17:33:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3738F286B9 for ; Tue, 30 Jul 2019 17:33:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2B87B286BC; Tue, 30 Jul 2019 17:33:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B07B628714 for ; Tue, 30 Jul 2019 17:33:20 +0000 (UTC) Received: from localhost ([::1]:35038 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsW00-000756-11 for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:33:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33645) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsb-00077I-H4 for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsY-0002lM-Li for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48614) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsU-0002iM-B6; Tue, 30 Jul 2019 13:25:34 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9DCE287633; Tue, 30 Jul 2019 17:25:33 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 19AFC19C5B; Tue, 30 Jul 2019 17:25:32 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:25:03 +0200 Message-Id: <20190730172508.19911-9-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 30 Jul 2019 17:25:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 08/13] qcow2: Fix broken snapshot table entries X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP The only case where we currently reject snapshot table entries is when they have too much extra data. Fix them with qemu-img check -r all by counting it as a corruption, reducing their extra_data_size, and then letting qcow2_check_fix_snapshot_table() do the rest. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2-snapshot.c | 69 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 56 insertions(+), 13 deletions(-) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 58609b6ea4..9956c32964 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -44,7 +44,23 @@ void qcow2_free_snapshots(BlockDriverState *bs) s->nb_snapshots = 0; } -int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) +/* + * If @repair is true, try to repair a broken snapshot table instead + * of just returning an error: + * + * - If there were snapshots with too much extra metadata, increment + * *extra_data_dropped for each. + * This requires the caller to eventually rewrite the whole snapshot + * table, which requires cluster allocation. Therefore, this should + * be done only after qcow2_check_refcounts() made sure the refcount + * structures are valid. + * (In the meantime, the image is still valid because + * qcow2_check_refcounts() does not do anything with snapshots' + * extra data.) + */ +static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, + int *extra_data_dropped, + Error **errp) { BDRVQcow2State *s = bs->opaque; QCowSnapshotHeader h; @@ -64,6 +80,8 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) s->snapshots = g_new0(QCowSnapshot, s->nb_snapshots); for(i = 0; i < s->nb_snapshots; i++) { + bool discard_unknown_extra_data = false; + /* Read statically sized part of the snapshot header */ offset = ROUND_UP(offset, 8); ret = bdrv_pread(bs->file, offset, &h, sizeof(h)); @@ -86,10 +104,21 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) name_size = be16_to_cpu(h.name_size); if (sn->extra_data_size > QCOW_MAX_SNAPSHOT_EXTRA_DATA) { - ret = -EFBIG; - error_setg(errp, "Too much extra metadata in snapshot table " - "entry %i", i); - goto fail; + if (!repair) { + ret = -EFBIG; + error_setg(errp, "Too much extra metadata in snapshot table " + "entry %i", i); + error_append_hint(errp, "You can force-remove this extra " + "metadata with qemu-img check -r all\n"); + goto fail; + } + + fprintf(stderr, "Discarding too much extra metadata in snapshot " + "table entry %i (%" PRIu32 " > %u)\n", + i, sn->extra_data_size, QCOW_MAX_SNAPSHOT_EXTRA_DATA); + + (*extra_data_dropped)++; + discard_unknown_extra_data = true; } /* Read known extra data */ @@ -112,16 +141,22 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) } if (sn->extra_data_size > sizeof(extra)) { - /* Store unknown extra data */ size_t unknown_extra_data_size = sn->extra_data_size - sizeof(extra); - sn->unknown_extra_data = g_malloc(unknown_extra_data_size); - ret = bdrv_pread(bs->file, offset, sn->unknown_extra_data, - unknown_extra_data_size); - if (ret < 0) { - error_setg_errno(errp, -ret, "Failed to read snapshot table"); - goto fail; + if (discard_unknown_extra_data) { + /* Discard unknown extra data */ + sn->extra_data_size = sizeof(extra); + } else { + /* Store unknown extra data */ + sn->unknown_extra_data = g_malloc(unknown_extra_data_size); + ret = bdrv_pread(bs->file, offset, sn->unknown_extra_data, + unknown_extra_data_size); + if (ret < 0) { + error_setg_errno(errp, -ret, + "Failed to read snapshot table"); + goto fail; + } } offset += unknown_extra_data_size; } @@ -162,6 +197,11 @@ fail: return ret; } +int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) +{ + return qcow2_do_read_snapshots(bs, false, NULL, errp); +} + /* add at the end of the file a new list of snapshots */ int qcow2_write_snapshots(BlockDriverState *bs) { @@ -327,6 +367,7 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, { BDRVQcow2State *s = bs->opaque; Error *local_err = NULL; + int extra_data_dropped = 0; int ret; struct { uint32_t nb_snapshots; @@ -362,7 +403,8 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, } qemu_co_mutex_unlock(&s->lock); - ret = qcow2_read_snapshots(bs, &local_err); + ret = qcow2_do_read_snapshots(bs, fix & BDRV_FIX_ERRORS, + &extra_data_dropped, &local_err); qemu_co_mutex_lock(&s->lock); if (ret < 0) { result->check_errors++; @@ -375,6 +417,7 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, return ret; } + result->corruptions += extra_data_dropped; return 0; } From patchwork Tue Jul 30 17:25:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066359 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0FAE814DB for ; Tue, 30 Jul 2019 17:36:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA677266F3 for ; Tue, 30 Jul 2019 17:36:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DE7BD286F1; Tue, 30 Jul 2019 17:36:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6493F266F3 for ; Tue, 30 Jul 2019 17:36:25 +0000 (UTC) Received: from localhost ([::1]:35094 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsW2y-0005fd-OU for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:36:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33691) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVse-0007Ir-CZ for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsd-0002pO-2u for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57812) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsY-0002kP-3d; Tue, 30 Jul 2019 13:25:40 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B7CC5308FEDF; Tue, 30 Jul 2019 17:25:36 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 046B460BEC; Tue, 30 Jul 2019 17:25:35 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:25:04 +0200 Message-Id: <20190730172508.19911-10-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Tue, 30 Jul 2019 17:25:36 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 09/13] qcow2: Fix overly long snapshot tables X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP We currently refuse to open qcow2 images with overly long snapshot tables. This patch makes qemu-img check -r all drop all offending entries past what we deem acceptable. Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 89 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 79 insertions(+), 10 deletions(-) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 9956c32964..bd8e56a99e 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -29,15 +29,24 @@ #include "qemu/error-report.h" #include "qemu/cutils.h" +static void qcow2_free_single_snapshot(BlockDriverState *bs, int i) +{ + BDRVQcow2State *s = bs->opaque; + + assert(i >= 0 && i < s->nb_snapshots); + g_free(s->snapshots[i].name); + g_free(s->snapshots[i].id_str); + g_free(s->snapshots[i].unknown_extra_data); + memset(&s->snapshots[i], 0, sizeof(s->snapshots[i])); +} + void qcow2_free_snapshots(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; int i; for(i = 0; i < s->nb_snapshots; i++) { - g_free(s->snapshots[i].name); - g_free(s->snapshots[i].id_str); - g_free(s->snapshots[i].unknown_extra_data); + qcow2_free_single_snapshot(bs, i); } g_free(s->snapshots); s->snapshots = NULL; @@ -48,6 +57,14 @@ void qcow2_free_snapshots(BlockDriverState *bs) * If @repair is true, try to repair a broken snapshot table instead * of just returning an error: * + * - If the snapshot table was too long, set *nb_clusters_reduced to + * the number of snapshots removed off the end. + * The caller will update the on-disk nb_snapshots accordingly; + * this leaks clusters, but is safe. + * (The on-disk information must be updated before + * qcow2_check_refcounts(), because that function relies on + * s->nb_snapshots to reflect the on-disk value.) + * * - If there were snapshots with too much extra metadata, increment * *extra_data_dropped for each. * This requires the caller to eventually rewrite the whole snapshot @@ -59,6 +76,7 @@ void qcow2_free_snapshots(BlockDriverState *bs) * extra data.) */ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, + int *nb_clusters_reduced, int *extra_data_dropped, Error **errp) { @@ -67,7 +85,7 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, QCowSnapshotExtraData extra; QCowSnapshot *sn; int i, id_str_size, name_size; - int64_t offset; + int64_t offset, pre_sn_offset; int ret; if (!s->nb_snapshots) { @@ -82,6 +100,8 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, for(i = 0; i < s->nb_snapshots; i++) { bool discard_unknown_extra_data = false; + pre_sn_offset = offset; + /* Read statically sized part of the snapshot header */ offset = ROUND_UP(offset, 8); ret = bdrv_pread(bs->file, offset, &h, sizeof(h)); @@ -182,9 +202,31 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, sn->name[name_size] = '\0'; if (offset - s->snapshots_offset > QCOW_MAX_SNAPSHOTS_SIZE) { - ret = -EFBIG; - error_setg(errp, "Snapshot table is too big"); - goto fail; + if (!repair) { + ret = -EFBIG; + error_setg(errp, "Snapshot table is too big"); + error_append_hint(errp, "You can force-remove all %u " + "overhanging snapshots with qemu-img check " + "-r all\n", s->nb_snapshots - i); + goto fail; + } + + fprintf(stderr, "Discarding %u overhanging snapshots (snapshot " + "table is too big)\n", s->nb_snapshots - i); + + *nb_clusters_reduced += (s->nb_snapshots - i); + + /* Discard current snapshot also */ + qcow2_free_single_snapshot(bs, i); + + /* + * This leaks all the rest of the snapshot table and the + * snapshots' clusters, but we run in check -r all mode, + * so qcow2_check_refcounts() will take care of it. + */ + s->nb_snapshots = i; + offset = pre_sn_offset; + break; } } @@ -199,7 +241,7 @@ fail: int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) { - return qcow2_do_read_snapshots(bs, false, NULL, errp); + return qcow2_do_read_snapshots(bs, false, NULL, NULL, errp); } /* add at the end of the file a new list of snapshots */ @@ -367,6 +409,7 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, { BDRVQcow2State *s = bs->opaque; Error *local_err = NULL; + int nb_clusters_reduced = 0; int extra_data_dropped = 0; int ret; struct { @@ -404,7 +447,8 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, qemu_co_mutex_unlock(&s->lock); ret = qcow2_do_read_snapshots(bs, fix & BDRV_FIX_ERRORS, - &extra_data_dropped, &local_err); + &nb_clusters_reduced, &extra_data_dropped, + &local_err); qemu_co_mutex_lock(&s->lock); if (ret < 0) { result->check_errors++; @@ -417,7 +461,32 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, return ret; } - result->corruptions += extra_data_dropped; + result->corruptions += nb_clusters_reduced + extra_data_dropped; + + if (nb_clusters_reduced) { + /* + * Update image header now, because: + * (1) qcow2_check_refcounts() relies on s->nb_snapshots to be + * the same as what the image header says, + * (2) this leaks clusters, but qcow2_check_refcounts() will + * fix that. + */ + assert(fix & BDRV_FIX_ERRORS); + + snapshot_table_pointer.nb_snapshots = cpu_to_be32(s->nb_snapshots); + ret = bdrv_pwrite_sync(bs->file, 60, + &snapshot_table_pointer.nb_snapshots, + sizeof(snapshot_table_pointer.nb_snapshots)); + if (ret < 0) { + result->check_errors++; + fprintf(stderr, "ERROR failed to update the snapshot count in the " + "image header: %s\n", strerror(-ret)); + return ret; + } + + result->corruptions_fixed += nb_clusters_reduced; + result->corruptions -= nb_clusters_reduced; + } return 0; } From patchwork Tue Jul 30 17:25:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066349 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A5C3314DB for ; Tue, 30 Jul 2019 17:34:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D9D1286BC for ; Tue, 30 Jul 2019 17:34:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8142B286F1; Tue, 30 Jul 2019 17:34:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 36FB6286BC for ; Tue, 30 Jul 2019 17:34:46 +0000 (UTC) Received: from localhost ([::1]:35066 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsW1N-000262-Jm for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:34:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33678) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsd-0007Ew-Ij for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsc-0002pD-Ko for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:43 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60694) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsa-0002lu-BK; Tue, 30 Jul 2019 13:25:40 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A2589C049E12; Tue, 30 Jul 2019 17:25:39 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 225726062D; Tue, 30 Jul 2019 17:25:38 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:25:05 +0200 Message-Id: <20190730172508.19911-11-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 30 Jul 2019 17:25:39 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 10/13] qcow2: Repair snapshot table with too many entries X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index bd8e56a99e..9e8c7c1f7f 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -430,6 +430,14 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, s->snapshots_offset = be64_to_cpu(snapshot_table_pointer.snapshots_offset); s->nb_snapshots = be32_to_cpu(snapshot_table_pointer.nb_snapshots); + if (s->nb_snapshots > QCOW_MAX_SNAPSHOTS && (fix & BDRV_FIX_ERRORS)) { + fprintf(stderr, "Discarding %u overhanging snapshots\n", + s->nb_snapshots - QCOW_MAX_SNAPSHOTS); + + nb_clusters_reduced += s->nb_snapshots - QCOW_MAX_SNAPSHOTS; + s->nb_snapshots = QCOW_MAX_SNAPSHOTS; + } + ret = qcow2_validate_table(bs, s->snapshots_offset, s->nb_snapshots, sizeof(QCowSnapshotHeader), sizeof(QCowSnapshotHeader) * QCOW_MAX_SNAPSHOTS, @@ -438,6 +446,12 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, result->check_errors++; error_reportf_err(local_err, "ERROR "); + if (s->nb_snapshots > QCOW_MAX_SNAPSHOTS) { + fprintf(stderr, "You can force-remove all %u overhanging snapshots " + "with qemu-img check -r all\n", + s->nb_snapshots - QCOW_MAX_SNAPSHOTS); + } + /* We did not read the snapshot table, so invalidate this information */ s->snapshots_offset = 0; s->nb_snapshots = 0; From patchwork Tue Jul 30 17:25:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066363 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CEF2E14DB for ; Tue, 30 Jul 2019 17:37:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B62D5286BC for ; Tue, 30 Jul 2019 17:37:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A73E728710; Tue, 30 Jul 2019 17:37:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CFB90286BC for ; Tue, 30 Jul 2019 17:37:46 +0000 (UTC) Received: from localhost ([::1]:35136 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsW4I-0007eM-88 for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:37:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33718) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsg-0007R6-RG for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsf-0002r1-Su for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:46 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37486) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsd-0002pY-OB; Tue, 30 Jul 2019 13:25:43 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1A6603A20F; Tue, 30 Jul 2019 17:25:43 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 49E9C5D982; Tue, 30 Jul 2019 17:25:41 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:25:06 +0200 Message-Id: <20190730172508.19911-12-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 30 Jul 2019 17:25:43 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 11/13] qcow2: Fix v3 snapshot table entry compliancy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP qcow2 v3 images require every snapshot table entry to have at least 16 bytes of extra data. If they do not, let qemu-img check -r all fix it. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2-snapshot.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 9e8c7c1f7f..e22c964b2a 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -502,6 +502,21 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, result->corruptions -= nb_clusters_reduced; } + /* + * All of v3 images' snapshot table entries need to have at least + * 16 bytes of extra data. + */ + if (s->qcow_version >= 3) { + int i; + for (i = 0; i < s->nb_snapshots; i++) { + if (s->snapshots[i].extra_data_size < 16) { + result->corruptions++; + fprintf(stderr, "%s snapshot table entry %i is incomplete\n", + fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR", i); + } + } + } + return 0; } From patchwork Tue Jul 30 17:25:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066343 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6FAE414DB for ; Tue, 30 Jul 2019 17:33:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54FB2286B9 for ; Tue, 30 Jul 2019 17:33:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 48C93286BE; Tue, 30 Jul 2019 17:33:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 015B1286B9 for ; Tue, 30 Jul 2019 17:33:19 +0000 (UTC) Received: from localhost ([::1]:35036 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVzy-000716-AU for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:33:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33739) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsj-0007Yc-EH for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsi-0002sY-HN for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:25:49 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42456) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsg-0002r8-ES; Tue, 30 Jul 2019 13:25:46 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BEDF630B1AD1; Tue, 30 Jul 2019 17:25:45 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 182145D9C5; Tue, 30 Jul 2019 17:25:44 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:25:07 +0200 Message-Id: <20190730172508.19911-13-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Tue, 30 Jul 2019 17:25:45 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 12/13] iotests: Add peek_file* functions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/common.rc | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 5502c3da2f..78decfd5d5 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -53,6 +53,26 @@ poke_file() printf "$3" | dd "of=$1" bs=1 "seek=$2" conv=notrunc &>/dev/null } +# peek_file_le 'test.img' 512 2 => 65534 +peek_file_le() +{ + # Wrap in echo $() to strip spaces + echo $(od -j"$2" -N"$3" --endian=little -An -vtu"$3" "$1") +} + +# peek_file_be 'test.img' 512 2 => 65279 +peek_file_be() +{ + # Wrap in echo $() to strip spaces + echo $(od -j"$2" -N"$3" --endian=big -An -vtu"$3" "$1") +} + +# peek_file_raw 'test.img' 512 2 => '\xff\xfe' +peek_file_raw() +{ + dd if="$1" bs=1 skip="$2" count="$3" status=none +} + if ! . ./common.config then From patchwork Tue Jul 30 17:25:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11066351 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E8C7D14DB for ; Tue, 30 Jul 2019 17:34:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CAE19286B9 for ; Tue, 30 Jul 2019 17:34:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BC40B286BE; Tue, 30 Jul 2019 17:34:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3D755286B9 for ; Tue, 30 Jul 2019 17:34:52 +0000 (UTC) Received: from localhost ([::1]:35070 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsW1T-0002O0-E8 for patchwork-qemu-devel@patchwork.kernel.org; Tue, 30 Jul 2019 13:34:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33775) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsVsu-0007qL-Rx for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:26:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsVsr-0002wF-3L for qemu-devel@nongnu.org; Tue, 30 Jul 2019 13:26:00 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57500) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hsVsj-0002si-Ll; Tue, 30 Jul 2019 13:25:49 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0191A307D91E; Tue, 30 Jul 2019 17:25:48 +0000 (UTC) Received: from localhost (ovpn-116-164.ams2.redhat.com [10.36.116.164]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DA87C1001E91; Tue, 30 Jul 2019 17:25:47 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 30 Jul 2019 19:25:08 +0200 Message-Id: <20190730172508.19911-14-mreitz@redhat.com> In-Reply-To: <20190730172508.19911-1-mreitz@redhat.com> References: <20190730172508.19911-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Tue, 30 Jul 2019 17:25:49 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH for-4.2 13/13] iotests: Test qcow2's snapshot table handling X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Add a test how our qcow2 driver handles extra data in snapshot table entries, and how it repairs overly long snapshot tables. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/261 | 449 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/261.out | 321 ++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 771 insertions(+) create mode 100755 tests/qemu-iotests/261 create mode 100644 tests/qemu-iotests/261.out diff --git a/tests/qemu-iotests/261 b/tests/qemu-iotests/261 new file mode 100755 index 0000000000..c7c4a4742a --- /dev/null +++ b/tests/qemu-iotests/261 @@ -0,0 +1,449 @@ +#!/usr/bin/env bash +# +# Test case for qcow2's handling of extra data in snapshot table entries +# (and more generally, how certain cases of broken snapshot tables are +# handled) +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=mreitz@redhat.com + +seq=$(basename $0) +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + rm -f "$TEST_IMG".v{2,3}.orig + rm -f "$TEST_DIR"/sn{0,1,2}{,-pre,-extra,-post} +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +# This tests qocw2-specific low-level functionality +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux +# (1) We create a v2 image that supports nothing but refcount_bits=16 +# (2) We do some refcount management on our own which expects +# refcount_bits=16 +_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' + +# Parameters: +# $1: image filename +# $2: snapshot table entry offset in the image +snapshot_table_entry_size() +{ + id_len=$(peek_file_be "$1" $(($2 + 12)) 2) + name_len=$(peek_file_be "$1" $(($2 + 14)) 2) + extra_len=$(peek_file_be "$1" $(($2 + 36)) 4) + + full_len=$((40 + extra_len + id_len + name_len)) + if [ $((full_len % 8)) = 0 ]; then + echo $full_len + else + echo $((full_len + 8 - full_len % 8)) + fi +} + +# Parameter: +# $1: image filename +print_snapshot_table() +{ + nb_entries=$(peek_file_be "$1" 60 4) + offset=$(peek_file_be "$1" 64 8) + + echo "Snapshots in $1:" | _filter_testdir | _filter_imgfmt + + for ((i = 0; i < nb_entries; i++)); do + id_len=$(peek_file_be "$1" $((offset + 12)) 2) + name_len=$(peek_file_be "$1" $((offset + 14)) 2) + extra_len=$(peek_file_be "$1" $((offset + 36)) 4) + + extra_ofs=$((offset + 40)) + id_ofs=$((extra_ofs + extra_len)) + name_ofs=$((id_ofs + id_len)) + + echo " [$i]" + echo " ID: $(peek_file_raw "$1" $id_ofs $id_len)" + echo " Name: $(peek_file_raw "$1" $name_ofs $name_len)" + echo " Extra data size: $extra_len" + if [ $extra_len -ge 8 ]; then + echo " VM state size: $(peek_file_be "$1" $extra_ofs 8)" + fi + if [ $extra_len -ge 16 ]; then + echo " Disk size: $(peek_file_be "$1" $((extra_ofs + 8)) 8)" + fi + if [ $extra_len -gt 16 ]; then + echo ' Unknown extra data:' \ + "$(peek_file_raw "$1" $((extra_ofs + 16)) $((extra_len - 16)) \ + | tr -d '\0')" + fi + + offset=$((offset + $(snapshot_table_entry_size "$1" $offset))) + done +} + +# Mark clusters as allocated; works only in refblock 0 (i.e. before +# cluster #32768). +# Parameters: +# $1: Start offset of what to allocate +# $2: End offset (exclusive) +refblock0_allocate() +{ + reftable_ofs=$(peek_file_be "$TEST_IMG" 48 8) + refblock_ofs=$(peek_file_be "$TEST_IMG" $reftable_ofs 8) + + cluster=$(($1 / 65536)) + ecluster=$((($2 + 65535) / 65536)) + + while [ $cluster -lt $ecluster ]; do + if [ $cluster -ge 32768 ]; then + echo "*** Abort: Cluster $cluster exceeds refblock 0 ***" + exit 1 + fi + poke_file "$TEST_IMG" $((refblock_ofs + cluster * 2)) '\x00\x01' + cluster=$((cluster + 1)) + done +} + + +echo +echo '=== Create v2 template ===' +echo + +# Create v2 image with a snapshot table with three entries: +# [0]: No extra data (valid with v2, not valid with v3) +# [1]: Has extra data unknown to qemu +# [2]: Has the 64-bit VM state size, but not the disk size (again, +# valid with v2, not valid with v3) + +TEST_IMG="$TEST_IMG.v2.orig" IMGOPTS='compat=0.10' _make_test_img 64M +$QEMU_IMG snapshot -c sn0 "$TEST_IMG.v2.orig" +$QEMU_IMG snapshot -c sn1 "$TEST_IMG.v2.orig" +$QEMU_IMG snapshot -c sn2 "$TEST_IMG.v2.orig" + +# Copy out all existing snapshot table entries +sn_table_ofs=$(peek_file_be "$TEST_IMG.v2.orig" 64 8) + +# ofs: Snapshot table entry offset +# eds: Extra data size +# ids: Name + ID size +# len: Total entry length +sn0_ofs=$sn_table_ofs +sn0_eds=$(peek_file_be "$TEST_IMG.v2.orig" $((sn0_ofs + 36)) 4) +sn0_ids=$(($(peek_file_be "$TEST_IMG.v2.orig" $((sn0_ofs + 12)) 2) + + $(peek_file_be "$TEST_IMG.v2.orig" $((sn0_ofs + 14)) 2))) +sn0_len=$(snapshot_table_entry_size "$TEST_IMG.v2.orig" $sn0_ofs) +sn1_ofs=$((sn0_ofs + sn0_len)) +sn1_eds=$(peek_file_be "$TEST_IMG.v2.orig" $((sn1_ofs + 36)) 4) +sn1_ids=$(($(peek_file_be "$TEST_IMG.v2.orig" $((sn1_ofs + 12)) 2) + + $(peek_file_be "$TEST_IMG.v2.orig" $((sn1_ofs + 14)) 2))) +sn1_len=$(snapshot_table_entry_size "$TEST_IMG.v2.orig" $sn1_ofs) +sn2_ofs=$((sn1_ofs + sn1_len)) +sn2_eds=$(peek_file_be "$TEST_IMG.v2.orig" $((sn2_ofs + 36)) 4) +sn2_ids=$(($(peek_file_be "$TEST_IMG.v2.orig" $((sn2_ofs + 12)) 2) + + $(peek_file_be "$TEST_IMG.v2.orig" $((sn2_ofs + 14)) 2))) +sn2_len=$(snapshot_table_entry_size "$TEST_IMG.v2.orig" $sn2_ofs) + +# Data before extra data +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn0-pre" bs=1 skip=$sn0_ofs count=40 \ + &>/dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn1-pre" bs=1 skip=$sn1_ofs count=40 \ + &>/dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn2-pre" bs=1 skip=$sn2_ofs count=40 \ + &>/dev/null + +# Extra data +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn0-extra" bs=1 \ + skip=$((sn0_ofs + 40)) count=$sn0_eds &>/dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn1-extra" bs=1 \ + skip=$((sn1_ofs + 40)) count=$sn1_eds &>/dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn2-extra" bs=1 \ + skip=$((sn2_ofs + 40)) count=$sn2_eds &>/dev/null + +# Data after extra data +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn0-post" bs=1 \ + skip=$((sn0_ofs + 40 + sn0_eds)) count=$sn0_ids \ + &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn1-post" bs=1 \ + skip=$((sn1_ofs + 40 + sn1_eds)) count=$sn1_ids \ + &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn2-post" bs=1 \ + skip=$((sn2_ofs + 40 + sn2_eds)) count=$sn2_ids \ + &> /dev/null + +# Amend them, one by one +# Set sn0's extra data size to 0 +poke_file "$TEST_DIR/sn0-pre" 36 '\x00\x00\x00\x00' +truncate -s 0 "$TEST_DIR/sn0-extra" +# Grow sn0-post to pad +truncate -s $(($(snapshot_table_entry_size "$TEST_DIR/sn0-pre") - 40)) \ + "$TEST_DIR/sn0-post" + +# Set sn1's extra data size to 42 +poke_file "$TEST_DIR/sn1-pre" 36 '\x00\x00\x00\x2a' +truncate -s 42 "$TEST_DIR/sn1-extra" +poke_file "$TEST_DIR/sn1-extra" 16 'very important data' +# Grow sn1-post to pad +truncate -s $(($(snapshot_table_entry_size "$TEST_DIR/sn1-pre") - 82)) \ + "$TEST_DIR/sn1-post" + +# Set sn2's extra data size to 8 +poke_file "$TEST_DIR/sn2-pre" 36 '\x00\x00\x00\x08' +truncate -s 8 "$TEST_DIR/sn2-extra" +# Grow sn2-post to pad +truncate -s $(($(snapshot_table_entry_size "$TEST_DIR/sn2-pre") - 48)) \ + "$TEST_DIR/sn2-post" + +# Construct snapshot table +cat "$TEST_DIR"/sn0-{pre,extra,post} \ + "$TEST_DIR"/sn1-{pre,extra,post} \ + "$TEST_DIR"/sn2-{pre,extra,post} \ + | dd of="$TEST_IMG.v2.orig" bs=1 seek=$sn_table_ofs conv=notrunc \ + &> /dev/null + +# Done! +TEST_IMG="$TEST_IMG.v2.orig" _check_test_img +print_snapshot_table "$TEST_IMG.v2.orig" + +echo +echo '=== Upgrade to v3 ===' +echo + +cp "$TEST_IMG.v2.orig" "$TEST_IMG.v3.orig" +$QEMU_IMG amend -o compat=1.1 "$TEST_IMG.v3.orig" +TEST_IMG="$TEST_IMG.v3.orig" _check_test_img +print_snapshot_table "$TEST_IMG.v3.orig" + +echo +echo '=== Repair botched v3 ===' +echo + +# Force the v2 file to be v3. v3 requires each snapshot table entry +# to have at least 16 bytes of extra data, so it will not comply to +# the qcow2 v3 specification; but we can fix that. +cp "$TEST_IMG.v2.orig" "$TEST_IMG" + +# Set version +poke_file "$TEST_IMG" 4 '\x00\x00\x00\x03' +# Increase header length (necessary for v3) +poke_file "$TEST_IMG" 100 '\x00\x00\x00\x68' +# Set refcount order (necessary for v3) +poke_file "$TEST_IMG" 96 '\x00\x00\x00\x04' + +_check_test_img -r all +print_snapshot_table "$TEST_IMG" + + +# From now on, just test the qcow2 version we are supposed to test. +# (v3 by default, v2 by choice through $IMGOPTS.) +# That works because we always write all known extra data when +# updating the snapshot table, independent of the version. + +if echo "$IMGOPTS" | grep -q 'compat=\(0\.10\|v2\)' 2> /dev/null; then + subver=v2 +else + subver=v3 +fi + +echo +echo '=== Add new snapshot ===' +echo + +cp "$TEST_IMG.$subver.orig" "$TEST_IMG" +$QEMU_IMG snapshot -c sn3 "$TEST_IMG" +_check_test_img +print_snapshot_table "$TEST_IMG" + +echo +echo '=== Remove different snapshots ===' + +for sn in sn0 sn1 sn2; do + echo + echo "--- $sn ---" + + cp "$TEST_IMG.$subver.orig" "$TEST_IMG" + $QEMU_IMG snapshot -d $sn "$TEST_IMG" + _check_test_img + print_snapshot_table "$TEST_IMG" +done + +echo +echo '=== Reject too much unknown extra data ===' +echo + +cp "$TEST_IMG.$subver.orig" "$TEST_IMG" +$QEMU_IMG snapshot -c sn3 "$TEST_IMG" + +sn_table_ofs=$(peek_file_be "$TEST_IMG" 64 8) +sn0_ofs=$sn_table_ofs +sn1_ofs=$((sn0_ofs + $(snapshot_table_entry_size "$TEST_IMG" $sn0_ofs))) +sn2_ofs=$((sn1_ofs + $(snapshot_table_entry_size "$TEST_IMG" $sn1_ofs))) +sn3_ofs=$((sn2_ofs + $(snapshot_table_entry_size "$TEST_IMG" $sn2_ofs))) + +# 64 kB of extra data should be rejected +# (Note that this also induces a refcount error, because it spills +# over to the next cluster. That's a good way to test that we can +# handle simultaneous snapshot table and refcount errors.) +poke_file "$TEST_IMG" $((sn3_ofs + 36)) '\x00\x01\x00\x00' + +# Print error +_img_info +echo +_check_test_img +echo + +# Should be repairable +_check_test_img -r all + +echo +echo '=== Snapshot table too big ===' +echo + +sn_table_ofs=$(peek_file_be "$TEST_IMG.v3.orig" 64 8) + +# Fill a snapshot with 1 kB of extra data, a 65535-char ID, and a +# 65535-char name, and repeat it as many times as necessary to fill +# 64 MB (the maximum supported by qemu) + +touch "$TEST_DIR/sn0" + +# Full size (fixed + extra + ID + name + padding) +sn_size=$((40 + 1024 + 65535 + 65535 + 2)) + +# We only need the fixed part, though. +truncate -s 40 "$TEST_DIR/sn0" + +# 65535-char ID string +poke_file "$TEST_DIR/sn0" 12 '\xff\xff' +# 65535-char name +poke_file "$TEST_DIR/sn0" 14 '\xff\xff' +# 1 kB of extra data +poke_file "$TEST_DIR/sn0" 36 '\x00\x00\x04\x00' + +# Create test image +_make_test_img 64M + +# Hook up snapshot table somewhere safe (at 1 MB) +poke_file "$TEST_IMG" 64 '\x00\x00\x00\x00\x00\x10\x00\x00' + +offset=1048576 +size_written=0 +sn_count=0 +while [ $size_written -le $((64 * 1048576)) ]; do + dd if="$TEST_DIR/sn0" of="$TEST_IMG" bs=1 seek=$offset conv=notrunc \ + &> /dev/null + offset=$((offset + sn_size)) + size_written=$((size_written + sn_size)) + sn_count=$((sn_count + 1)) +done + +# Give the last snapshot (the one to be removed) an L1 table so we can +# see how that is handled when repairing the image +# (Put it two clusters before 1 MB, and one L2 table one cluster +# before 1 MB) +poke_file "$TEST_IMG" $((offset - sn-size + 0)) \ + '\x00\x00\x00\x00\x00\x0e\x00\x00' +poke_file "$TEST_IMG" $((offset - sn-size + 8)) \ + '\x00\x00\x00\x01' + +# Hook up the L2 table +poke_file "$TEST_IMG" $((1048576 - 2 * 65536)) \ + '\x80\x00\x00\x00\x00\x0f\x00\x00' + +# Make sure all of the clusters we just hooked up are allocated: +# - The snapshot table +# - The last snapshot's L1 and L2 table +refblock0_allocate $((1048576 - 2 * 65536)) $offset + +poke_file "$TEST_IMG" 60 \ + "$(printf '%08x' $sn_count | sed -e 's/\(..\)/\\x\1/g')" + +# Print error +_img_info +echo +_check_test_img +echo + +# Should be repairable +_check_test_img -r all + +echo +echo "$((sn_count - 1)) snapshots should remain:" +echo " qemu-img info reports $(_img_info | grep -c '^ \{34\}') snapshots" +echo " Image header reports $(peek_file_be "$TEST_IMG" 60 4) snapshots" + +echo +echo '=== Too many snapshots ===' +echo + +# Create a v2 image, for speeds' sake: All-zero snapshot table entries +# are only valid in v2. +IMGOPTS='compat=0.10' _make_test_img 64M + +# Hook up snapshot table somewhere safe (at 1 MB) +poke_file "$TEST_IMG" 64 '\x00\x00\x00\x00\x00\x10\x00\x00' +# "Create" more than 65536 snapshots (twice that many here) +poke_file "$TEST_IMG" 60 '\x00\x02\x00\x00' + +# 40-byte all-zero snapshot table entries are valid snapshots, but +# only in v2 (v3 needs 16 bytes of extra data, so we would have to +# write 131072x '\x10'). +truncate -s $((1048576 + 40 * 131072)) "$TEST_IMG" + +# But let us give one of the snapshots to be removed an L1 table so +# we can see how that is handled when repairing the image. +# (Put it two clusters before 1 MB, and one L2 table one cluster +# before 1 MB) +poke_file "$TEST_IMG" $((1048576 + 40 * 65536 + 0)) \ + '\x00\x00\x00\x00\x00\x0e\x00\x00' +poke_file "$TEST_IMG" $((1048576 + 40 * 65536 + 8)) \ + '\x00\x00\x00\x01' + +# Hook up the L2 table +poke_file "$TEST_IMG" $((1048576 - 2 * 65536)) \ + '\x80\x00\x00\x00\x00\x0f\x00\x00' + +# Make sure all of the clusters we just hooked up are allocated: +# - The snapshot table +# - The last snapshot's L1 and L2 table +refblock0_allocate $((1048576 - 2 * 65536)) $((1048576 + 40 * 131072)) + +# Print error +_img_info +echo +_check_test_img +echo + +# Should be repairable +_check_test_img -r all + +echo +echo '65536 snapshots should remain:' +echo " qemu-img info reports $(_img_info | grep -c '^ \{34\}') snapshots" +echo " Image header reports $(peek_file_be "$TEST_IMG" 60 4) snapshots" + +# success, all done +echo "*** done" +status=0 diff --git a/tests/qemu-iotests/261.out b/tests/qemu-iotests/261.out new file mode 100644 index 0000000000..75af8c98ac --- /dev/null +++ b/tests/qemu-iotests/261.out @@ -0,0 +1,321 @@ +QA output created by 261 + +=== Create v2 template === + +Formatting 'TEST_DIR/t.IMGFMT.v2.orig', fmt=IMGFMT size=67108864 +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT.v2.orig: + [0] + ID: 1 + Name: sn0 + Extra data size: 0 + [1] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + [2] + ID: 3 + Name: sn2 + Extra data size: 8 + VM state size: 0 + +=== Upgrade to v3 === + +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT.v3.orig: + [0] + ID: 1 + Name: sn0 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [1] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + [2] + ID: 3 + Name: sn2 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + +=== Repair botched v3 === + +Repairing snapshot table entry 0 is incomplete +Repairing snapshot table entry 2 is incomplete +The following inconsistencies were found and repaired: + + 0 leaked clusters + 2 corruptions + +Double checking the fixed image now... +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT: + [0] + ID: 1 + Name: sn0 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [1] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + [2] + ID: 3 + Name: sn2 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + +=== Add new snapshot === + +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT: + [0] + ID: 1 + Name: sn0 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [1] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + [2] + ID: 3 + Name: sn2 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [3] + ID: 4 + Name: sn3 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + +=== Remove different snapshots === + +--- sn0 --- +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT: + [0] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + [1] + ID: 3 + Name: sn2 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + +--- sn1 --- +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT: + [0] + ID: 1 + Name: sn0 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [1] + ID: 3 + Name: sn2 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + +--- sn2 --- +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT: + [0] + ID: 1 + Name: sn0 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [1] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + +=== Reject too much unknown extra data === + +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Too much extra metadata in snapshot table entry 3 +You can force-remove this extra metadata with qemu-img check -r all + +qemu-img: ERROR failed to read the snapshot table: Too much extra metadata in snapshot table entry 3 +You can force-remove this extra metadata with qemu-img check -r all +qemu-img: Check failed: File too large + +Discarding too much extra metadata in snapshot table entry 3 (65536 > 1024) +ERROR cluster 10 refcount=0 reference=1 +Rebuilding refcount structure +Repairing cluster 1 refcount=1 reference=0 +Repairing cluster 2 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 2 corruptions + +Double checking the fixed image now... +No errors were found on the image. + +=== Snapshot table too big === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Snapshot table is too big +You can force-remove all 1 overhanging snapshots with qemu-img check -r all + +qemu-img: ERROR failed to read the snapshot table: Snapshot table is too big +You can force-remove all 1 overhanging snapshots with qemu-img check -r all +qemu-img: Check failed: File too large + +Discarding 1 overhanging snapshots (snapshot table is too big) +Leaked cluster 14 refcount=1 reference=0 +Leaked cluster 15 refcount=1 reference=0 +Leaked cluster 1039 refcount=1 reference=0 +Leaked cluster 1040 refcount=1 reference=0 +Repairing cluster 14 refcount=1 reference=0 +Repairing cluster 15 refcount=1 reference=0 +Repairing cluster 1039 refcount=1 reference=0 +Repairing cluster 1040 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 4 leaked clusters + 1 corruptions + +Double checking the fixed image now... +No errors were found on the image. + +507 snapshots should remain: + qemu-img info reports 507 snapshots + Image header reports 507 snapshots + +=== Too many snapshots === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Snapshot table too large + +qemu-img: ERROR snapshot table too large +You can force-remove all 65536 overhanging snapshots with qemu-img check -r all +qemu-img: Check failed: File too large + +Discarding 65536 overhanging snapshots +Leaked cluster 14 refcount=1 reference=0 +Leaked cluster 15 refcount=1 reference=0 +Leaked cluster 56 refcount=1 reference=0 +Leaked cluster 57 refcount=1 reference=0 +Leaked cluster 58 refcount=1 reference=0 +Leaked cluster 59 refcount=1 reference=0 +Leaked cluster 60 refcount=1 reference=0 +Leaked cluster 61 refcount=1 reference=0 +Leaked cluster 62 refcount=1 reference=0 +Leaked cluster 63 refcount=1 reference=0 +Leaked cluster 64 refcount=1 reference=0 +Leaked cluster 65 refcount=1 reference=0 +Leaked cluster 66 refcount=1 reference=0 +Leaked cluster 67 refcount=1 reference=0 +Leaked cluster 68 refcount=1 reference=0 +Leaked cluster 69 refcount=1 reference=0 +Leaked cluster 70 refcount=1 reference=0 +Leaked cluster 71 refcount=1 reference=0 +Leaked cluster 72 refcount=1 reference=0 +Leaked cluster 73 refcount=1 reference=0 +Leaked cluster 74 refcount=1 reference=0 +Leaked cluster 75 refcount=1 reference=0 +Leaked cluster 76 refcount=1 reference=0 +Leaked cluster 77 refcount=1 reference=0 +Leaked cluster 78 refcount=1 reference=0 +Leaked cluster 79 refcount=1 reference=0 +Leaked cluster 80 refcount=1 reference=0 +Leaked cluster 81 refcount=1 reference=0 +Leaked cluster 82 refcount=1 reference=0 +Leaked cluster 83 refcount=1 reference=0 +Leaked cluster 84 refcount=1 reference=0 +Leaked cluster 85 refcount=1 reference=0 +Leaked cluster 86 refcount=1 reference=0 +Leaked cluster 87 refcount=1 reference=0 +Leaked cluster 88 refcount=1 reference=0 +Leaked cluster 89 refcount=1 reference=0 +Leaked cluster 90 refcount=1 reference=0 +Leaked cluster 91 refcount=1 reference=0 +Leaked cluster 92 refcount=1 reference=0 +Leaked cluster 93 refcount=1 reference=0 +Leaked cluster 94 refcount=1 reference=0 +Leaked cluster 95 refcount=1 reference=0 +Repairing cluster 14 refcount=1 reference=0 +Repairing cluster 15 refcount=1 reference=0 +Repairing cluster 56 refcount=1 reference=0 +Repairing cluster 57 refcount=1 reference=0 +Repairing cluster 58 refcount=1 reference=0 +Repairing cluster 59 refcount=1 reference=0 +Repairing cluster 60 refcount=1 reference=0 +Repairing cluster 61 refcount=1 reference=0 +Repairing cluster 62 refcount=1 reference=0 +Repairing cluster 63 refcount=1 reference=0 +Repairing cluster 64 refcount=1 reference=0 +Repairing cluster 65 refcount=1 reference=0 +Repairing cluster 66 refcount=1 reference=0 +Repairing cluster 67 refcount=1 reference=0 +Repairing cluster 68 refcount=1 reference=0 +Repairing cluster 69 refcount=1 reference=0 +Repairing cluster 70 refcount=1 reference=0 +Repairing cluster 71 refcount=1 reference=0 +Repairing cluster 72 refcount=1 reference=0 +Repairing cluster 73 refcount=1 reference=0 +Repairing cluster 74 refcount=1 reference=0 +Repairing cluster 75 refcount=1 reference=0 +Repairing cluster 76 refcount=1 reference=0 +Repairing cluster 77 refcount=1 reference=0 +Repairing cluster 78 refcount=1 reference=0 +Repairing cluster 79 refcount=1 reference=0 +Repairing cluster 80 refcount=1 reference=0 +Repairing cluster 81 refcount=1 reference=0 +Repairing cluster 82 refcount=1 reference=0 +Repairing cluster 83 refcount=1 reference=0 +Repairing cluster 84 refcount=1 reference=0 +Repairing cluster 85 refcount=1 reference=0 +Repairing cluster 86 refcount=1 reference=0 +Repairing cluster 87 refcount=1 reference=0 +Repairing cluster 88 refcount=1 reference=0 +Repairing cluster 89 refcount=1 reference=0 +Repairing cluster 90 refcount=1 reference=0 +Repairing cluster 91 refcount=1 reference=0 +Repairing cluster 92 refcount=1 reference=0 +Repairing cluster 93 refcount=1 reference=0 +Repairing cluster 94 refcount=1 reference=0 +Repairing cluster 95 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 42 leaked clusters + 65536 corruptions + +Double checking the fixed image now... +No errors were found on the image. + +65536 snapshots should remain: + qemu-img info reports 65536 snapshots + Image header reports 65536 snapshots +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index f13e5f2e23..d33ab9238e 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -271,3 +271,4 @@ 254 rw backing quick 255 rw quick 256 rw quick +261 rw