From patchwork Fri Dec 14 13:42:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10731179 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5E3AA109C for ; Fri, 14 Dec 2018 13:44:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D2E62C8AE for ; Fri, 14 Dec 2018 13:44:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 412652C904; Fri, 14 Dec 2018 13:44:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E6C032C8AE for ; Fri, 14 Dec 2018 13:44:29 +0000 (UTC) Received: from localhost ([::1]:33591 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXnlV-0001F6-3e for patchwork-qemu-devel@patchwork.kernel.org; Fri, 14 Dec 2018 08:44:29 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45480) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXnjv-0000uh-Eo for qemu-devel@nongnu.org; Fri, 14 Dec 2018 08:42:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXnju-0005jH-Nj for qemu-devel@nongnu.org; Fri, 14 Dec 2018 08:42:51 -0500 Received: from relay.sw.ru ([185.231.240.75]:36714) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXnjq-0005eL-IC; Fri, 14 Dec 2018 08:42:46 -0500 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1gXnjl-000251-TG; Fri, 14 Dec 2018 16:42:41 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 16:42:36 +0300 Message-Id: <20181214134240.217870-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181214134240.217870-1-vsementsov@virtuozzo.com> References: <20181214134240.217870-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 1/5] qcow2-refcount: fix check_oflag_copied X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Increase corruptions_fixed only after successful fix. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-refcount.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 1c63ac244a..b717fdecc6 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1822,7 +1822,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, for (i = 0; i < s->l1_size; i++) { uint64_t l1_entry = s->l1_table[i]; uint64_t l2_offset = l1_entry & L1E_OFFSET_MASK; - bool l2_dirty = false; + int l2_dirty = 0; if (!l2_offset) { continue; @@ -1884,8 +1884,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, l2_table[j] = cpu_to_be64(refcount == 1 ? l2_entry | QCOW_OFLAG_COPIED : l2_entry & ~QCOW_OFLAG_COPIED); - l2_dirty = true; - res->corruptions_fixed++; + l2_dirty++; } else { res->corruptions++; } @@ -1893,7 +1892,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, } } - if (l2_dirty) { + if (l2_dirty > 0) { ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L2, l2_offset, s->cluster_size); if (ret < 0) { @@ -1911,6 +1910,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, res->check_errors++; goto fail; } + res->corruptions_fixed += l2_dirty; } } From patchwork Fri Dec 14 13:42:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10731187 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 840D3109C for ; Fri, 14 Dec 2018 13:47:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 73F742D0C7 for ; Fri, 14 Dec 2018 13:47:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6798A2D1C5; Fri, 14 Dec 2018 13:47:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id ED36A2D0C7 for ; Fri, 14 Dec 2018 13:47:12 +0000 (UTC) Received: from localhost ([::1]:33612 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXno8-0004jj-9o for patchwork-qemu-devel@patchwork.kernel.org; Fri, 14 Dec 2018 08:47:12 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45506) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXnjv-0000vI-Vc for qemu-devel@nongnu.org; Fri, 14 Dec 2018 08:42:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXnjv-0005jm-0a for qemu-devel@nongnu.org; Fri, 14 Dec 2018 08:42:51 -0500 Received: from relay.sw.ru ([185.231.240.75]:36712) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXnjq-0005eH-GE; Fri, 14 Dec 2018 08:42:46 -0500 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1gXnjl-000251-V2; Fri, 14 Dec 2018 16:42:41 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 16:42:37 +0300 Message-Id: <20181214134240.217870-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181214134240.217870-1-vsementsov@virtuozzo.com> References: <20181214134240.217870-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 2/5] qcow2-refcount: avoid eating RAM X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP qcow2_inc_refcounts_imrt() (through realloc_refcount_array()) can eat an unpredictable amount of memory on corrupted table entries, which are referencing regions far beyond the end of file. Prevent this, by skipping such regions from further processing. Interesting that iotest 138 checks exactly the behavior which we fix here. So, change the test appropriately. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-refcount.c | 18 ++++++++++++++++++ tests/qemu-iotests/138 | 12 +++++------- tests/qemu-iotests/138.out | 5 ++++- 3 files changed, 27 insertions(+), 8 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index b717fdecc6..c41c1fbe00 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1505,12 +1505,30 @@ int qcow2_inc_refcounts_imrt(BlockDriverState *bs, BdrvCheckResult *res, { BDRVQcow2State *s = bs->opaque; uint64_t start, last, cluster_offset, k, refcount; + int64_t file_len; int ret; if (size <= 0) { return 0; } + file_len = bdrv_getlength(bs->file->bs); + if (file_len < 0) { + return file_len; + } + + /* Last cluster of qcow2 image may be semi-allocated, so it's may be OK to + * reference some space after file end but it should be less than one + * cluster. + */ + if (offset + size - file_len >= s->cluster_size) { + fprintf(stderr, "ERROR: counting reference for region exceeding the " + "end of the file by one cluster or more: offset 0x%" PRIx64 + " size 0x%" PRIx64 "\n", offset, size); + res->corruptions++; + return 0; + } + start = start_of_cluster(s, offset); last = start_of_cluster(s, offset + size - 1); for(cluster_offset = start; cluster_offset <= last; diff --git a/tests/qemu-iotests/138 b/tests/qemu-iotests/138 index eccbcae3a6..12e20762e5 100755 --- a/tests/qemu-iotests/138 +++ b/tests/qemu-iotests/138 @@ -54,15 +54,13 @@ $QEMU_IO -c 'write 0 512' "$TEST_IMG" | _filter_qemu_io # Put the data cluster at a multiple of 2 TB, resulting in the image apparently # having a multiple of 2^32 clusters # (To be more specific: It is at 32 PB) -poke_file "$TEST_IMG" 2048 "\x80\x80\x00\x00\x00\x00\x00\x00" +poke_file "$TEST_IMG" $((2048 + 8)) "\x00\x80\x00\x00\x00\x00\x00\x00" # An offset of 32 PB results in qemu-img check having to allocate an in-memory -# refcount table of 128 TB (16 bit refcounts, 512 byte clusters). -# This should be generally too much for any system and thus fail. -# What this test is checking is that the qcow2 driver actually tries to allocate -# such a large amount of memory (and is consequently aborting) instead of having -# truncated the cluster count somewhere (which would result in much less memory -# being allocated and then a segfault occurring). +# refcount table of 128 TB (16 bit refcounts, 512 byte clusters), if qemu-img +# don't check that referenced data cluster is far beyond the end of file. +# But starting from 4.0, qemu-img does this check, and instead of "Cannot +# allocate memory", we have an error showing that l2 entry is invalid. _check_test_img # success, all done diff --git a/tests/qemu-iotests/138.out b/tests/qemu-iotests/138.out index 3fe911f85a..aca7d47a80 100644 --- a/tests/qemu-iotests/138.out +++ b/tests/qemu-iotests/138.out @@ -5,5 +5,8 @@ QA output created by 138 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=512 wrote 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -qemu-img: Check failed: Cannot allocate memory +ERROR: counting reference for region exceeding the end of the file by one cluster or more: offset 0x80000000000000 size 0x200 + +1 errors were found on the image. +Data may be corrupted, or further writes to the image may corrupt it. *** done From patchwork Fri Dec 14 13:42:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10731185 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E42C109C for ; Fri, 14 Dec 2018 13:47:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3927B2D0C7 for ; Fri, 14 Dec 2018 13:47:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2D3D62D1C0; Fri, 14 Dec 2018 13:47:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C43402D0C7 for ; Fri, 14 Dec 2018 13:47:03 +0000 (UTC) Received: from localhost ([::1]:33614 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXnny-0005Jl-LN for patchwork-qemu-devel@patchwork.kernel.org; Fri, 14 Dec 2018 08:47:02 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45494) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXnjv-0000v6-On for qemu-devel@nongnu.org; Fri, 14 Dec 2018 08:42:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXnju-0005jc-TE for qemu-devel@nongnu.org; Fri, 14 Dec 2018 08:42:51 -0500 Received: from relay.sw.ru ([185.231.240.75]:36718) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXnjq-0005eI-G7; Fri, 14 Dec 2018 08:42:46 -0500 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1gXnjm-000251-2Y; Fri, 14 Dec 2018 16:42:42 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 16:42:38 +0300 Message-Id: <20181214134240.217870-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181214134240.217870-1-vsementsov@virtuozzo.com> References: <20181214134240.217870-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 3/5] qcow2-refcount: check_refcounts_l2: reduce ignored overlaps X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Reduce number of structures ignored in overlap check: when checking active table ignore active tables, when checking inactive table ignore inactive ones. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-refcount.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index c41c1fbe00..e76085d9aa 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1575,7 +1575,7 @@ enum { static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, void **refcount_table, int64_t *refcount_table_size, int64_t l2_offset, - int flags, BdrvCheckMode fix) + int flags, BdrvCheckMode fix, bool active) { BDRVQcow2State *s = bs->opaque; uint64_t *l2_table, l2_entry; @@ -1659,11 +1659,12 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, if (fix & BDRV_FIX_ERRORS) { uint64_t l2e_offset = l2_offset + (uint64_t)i * sizeof(uint64_t); + int ign = active ? QCOW2_OL_ACTIVE_L2 : + QCOW2_OL_INACTIVE_L2; l2_entry = QCOW_OFLAG_ZERO; l2_table[i] = cpu_to_be64(l2_entry); - ret = qcow2_pre_write_overlap_check(bs, - QCOW2_OL_ACTIVE_L2 | QCOW2_OL_INACTIVE_L2, + ret = qcow2_pre_write_overlap_check(bs, ign, l2e_offset, sizeof(uint64_t)); if (ret < 0) { fprintf(stderr, "ERROR: Overlap check failed\n"); @@ -1737,7 +1738,7 @@ static int check_refcounts_l1(BlockDriverState *bs, void **refcount_table, int64_t *refcount_table_size, int64_t l1_table_offset, int l1_size, - int flags, BdrvCheckMode fix) + int flags, BdrvCheckMode fix, bool active) { BDRVQcow2State *s = bs->opaque; uint64_t *l1_table = NULL, l2_offset, l1_size2; @@ -1793,7 +1794,7 @@ static int check_refcounts_l1(BlockDriverState *bs, /* Process and check L2 entries */ ret = check_refcounts_l2(bs, res, refcount_table, refcount_table_size, l2_offset, flags, - fix); + fix, active); if (ret < 0) { goto fail; } @@ -2079,7 +2080,7 @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res, /* current L1 table */ ret = check_refcounts_l1(bs, res, refcount_table, nb_clusters, s->l1_table_offset, s->l1_size, CHECK_FRAG_INFO, - fix); + fix, true); if (ret < 0) { return ret; } @@ -2102,7 +2103,8 @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res, continue; } ret = check_refcounts_l1(bs, res, refcount_table, nb_clusters, - sn->l1_table_offset, sn->l1_size, 0, fix); + sn->l1_table_offset, sn->l1_size, 0, fix, + false); if (ret < 0) { return ret; } From patchwork Fri Dec 14 13:42:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10731181 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9F1EF109C for ; Fri, 14 Dec 2018 13:44:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D7AE2C8AE for ; Fri, 14 Dec 2018 13:44:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 802962C904; Fri, 14 Dec 2018 13:44:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 322822C8AE for ; Fri, 14 Dec 2018 13:44:39 +0000 (UTC) Received: from localhost ([::1]:33599 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXnle-0003QU-Bd for patchwork-qemu-devel@patchwork.kernel.org; Fri, 14 Dec 2018 08:44:38 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45528) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXnk0-00014I-C6 for qemu-devel@nongnu.org; Fri, 14 Dec 2018 08:42:59 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXnju-0005jV-RN for qemu-devel@nongnu.org; Fri, 14 Dec 2018 08:42:56 -0500 Received: from relay.sw.ru ([185.231.240.75]:36724) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXnjq-0005eO-GC; Fri, 14 Dec 2018 08:42:46 -0500 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1gXnjm-000251-5u; Fri, 14 Dec 2018 16:42:42 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 16:42:39 +0300 Message-Id: <20181214134240.217870-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181214134240.217870-1-vsementsov@virtuozzo.com> References: <20181214134240.217870-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 4/5] qcow2-refcount: check_refcounts_l2: don't count fixed cluster as allocated X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Do not count a cluster which is fixed to be ZERO as allocated. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-refcount.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index e76085d9aa..8da0e91dd3 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1637,15 +1637,6 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, { uint64_t offset = l2_entry & L2E_OFFSET_MASK; - if (flags & CHECK_FRAG_INFO) { - res->bfi.allocated_clusters++; - if (next_contiguous_offset && - offset != next_contiguous_offset) { - res->bfi.fragmented_clusters++; - } - next_contiguous_offset = offset + s->cluster_size; - } - /* Correct offsets are cluster aligned */ if (offset_into_cluster(s, offset)) { if (qcow2_get_cluster_type(l2_entry) == @@ -1698,6 +1689,15 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, } } + if (flags & CHECK_FRAG_INFO) { + res->bfi.allocated_clusters++; + if (next_contiguous_offset && + offset != next_contiguous_offset) { + res->bfi.fragmented_clusters++; + } + next_contiguous_offset = offset + s->cluster_size; + } + /* Mark cluster as used */ ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, refcount_table_size, From patchwork Fri Dec 14 13:42:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10731183 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 73356109C for ; Fri, 14 Dec 2018 13:44:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 62D822C8BF for ; Fri, 14 Dec 2018 13:44:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5464E2CAE7; Fri, 14 Dec 2018 13:44:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E17992C8AE for ; Fri, 14 Dec 2018 13:44:41 +0000 (UTC) Received: from localhost ([::1]:33600 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXnlh-0003TC-5I for patchwork-qemu-devel@patchwork.kernel.org; Fri, 14 Dec 2018 08:44:41 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45503) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXnjv-0000vG-UO for qemu-devel@nongnu.org; Fri, 14 Dec 2018 08:42:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXnju-0005jh-V1 for qemu-devel@nongnu.org; Fri, 14 Dec 2018 08:42:51 -0500 Received: from relay.sw.ru ([185.231.240.75]:36732) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXnjq-0005eM-ID; Fri, 14 Dec 2018 08:42:46 -0500 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1gXnjm-000251-A2; Fri, 14 Dec 2018 16:42:42 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 16:42:40 +0300 Message-Id: <20181214134240.217870-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181214134240.217870-1-vsementsov@virtuozzo.com> References: <20181214134240.217870-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 5/5] qcow2-refcount: don't mask corruptions under internal errors X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP No reasons for not reporting found corruptions as corruptions in case of some internal errors, especially in case of just failed to fix l2 entry (and in this case, missed corruptions may influence comparing logic, when we calculate difference between corruptions fields of two results) Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 31 ++++++++++++++----------------- 1 file changed, 14 insertions(+), 17 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 8da0e91dd3..b0e006e628 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1639,6 +1639,8 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, /* Correct offsets are cluster aligned */ if (offset_into_cluster(s, offset)) { + res->corruptions++; + if (qcow2_get_cluster_type(l2_entry) == QCOW2_CLUSTER_ZERO_ALLOC) { @@ -1674,18 +1676,16 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, /* Do not abort, continue checking the rest of this * L2 table's entries */ } else { + res->corruptions--; res->corruptions_fixed++; /* Skip marking the cluster as used * (it is unused now) */ continue; } - } else { - res->corruptions++; } } else { fprintf(stderr, "ERROR offset=%" PRIx64 ": Data cluster is " "not properly aligned; L2 entry corrupted.\n", offset); - res->corruptions++; } } @@ -1854,6 +1854,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, continue; } if ((refcount == 1) != ((l1_entry & QCOW_OFLAG_COPIED) != 0)) { + res->corruptions++; fprintf(stderr, "%s OFLAG_COPIED L2 cluster: l1_index=%d " "l1_entry=%" PRIx64 " refcount=%" PRIu64 "\n", repair ? "Repairing" : "ERROR", i, l1_entry, refcount); @@ -1866,9 +1867,8 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, res->check_errors++; goto fail; } + res->corruptions--; res->corruptions_fixed++; - } else { - res->corruptions++; } } @@ -1899,19 +1899,16 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, fprintf(stderr, "%s OFLAG_COPIED data cluster: " "l2_entry=%" PRIx64 " refcount=%" PRIu64 "\n", repair ? "Repairing" : "ERROR", l2_entry, refcount); - if (repair) { - l2_table[j] = cpu_to_be64(refcount == 1 - ? l2_entry | QCOW_OFLAG_COPIED - : l2_entry & ~QCOW_OFLAG_COPIED); - l2_dirty++; - } else { - res->corruptions++; - } + l2_table[j] = cpu_to_be64(refcount == 1 + ? l2_entry | QCOW_OFLAG_COPIED + : l2_entry & ~QCOW_OFLAG_COPIED); + l2_dirty++; } } } - if (l2_dirty > 0) { + res->corruptions += l2_dirty; + if (repair && l2_dirty > 0) { ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L2, l2_offset, s->cluster_size); if (ret < 0) { @@ -1929,6 +1926,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, res->check_errors++; goto fail; } + res->corruptions -= l2_dirty; res->corruptions_fixed += l2_dirty; } } @@ -1967,6 +1965,7 @@ static int check_refblocks(BlockDriverState *bs, BdrvCheckResult *res, } if (cluster >= *nb_clusters) { + res->corruptions++; fprintf(stderr, "%s refcount block %" PRId64 " is outside image\n", fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR", i); @@ -2006,6 +2005,7 @@ static int check_refblocks(BlockDriverState *bs, BdrvCheckResult *res, goto resize_fail; } + res->corruptions--; res->corruptions_fixed++; ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, nb_clusters, @@ -2019,12 +2019,9 @@ static int check_refblocks(BlockDriverState *bs, BdrvCheckResult *res, continue; resize_fail: - res->corruptions++; *rebuild = true; fprintf(stderr, "ERROR could not resize image: %s\n", strerror(-ret)); - } else { - res->corruptions++; } continue; }