From patchwork Fri Oct 6 15:53:47 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 9989901 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D17476020F for ; Fri, 6 Oct 2017 15:57:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB9DD28E08 for ; Fri, 6 Oct 2017 15:57:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ADC0228E0A; Fri, 6 Oct 2017 15:57:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E475328E08 for ; Fri, 6 Oct 2017 15:57:01 +0000 (UTC) Received: from localhost ([::1]:45601 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uzl-0001BA-48 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 06 Oct 2017 11:57:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55781) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxq-0000TP-30 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxo-00034E-NJ for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:02 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36614) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxl-0002ve-2R; Fri, 06 Oct 2017 11:54:57 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0EDA8C047B74; Fri, 6 Oct 2017 15:54:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0EDA8C047B74 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BFB367598; Fri, 6 Oct 2017 15:54:54 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:47 +0200 Message-Id: <20171006155422.10135-20-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 06 Oct 2017 15:54:56 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 19/54] qcow2: Switch store_bitmap_data() to byte-based iteration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Eric Blake Now that we have adjusted the majority of the calls this function makes to be byte-based, it is easier to read the code if it makes passes over the image using bytes rather than sectors. iotests 165 was rather weak - on a default 64k-cluster image, where bitmap granularity also defaults to 64k bytes, a single cluster of the bitmap table thus covers (64*1024*8) bits which each cover 64k bytes, or 32G of image space. But the test only uses a 1G image, so it cannot trigger any more than one loop of the code in store_bitmap_data(); and it was writing to the first cluster. In order to test that we are properly aligning which portions of the bitmap are being written to the file, we really want to test a case where the first dirty bit returned by bdrv_dirty_iter_next() is not aligned to the start of a cluster, which we can do by modifying the test to write data that doesn't happen to fall in the first cluster of the image. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/qcow2-bitmap.c | 31 ++++++++++++++++--------------- tests/qemu-iotests/165 | 2 +- 2 files changed, 17 insertions(+), 16 deletions(-) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 02512a21f2..f45e46cfbd 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -1072,10 +1072,9 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs, { int ret; BDRVQcow2State *s = bs->opaque; - int64_t sector; - uint64_t limit, sbc; + int64_t offset; + uint64_t limit; uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap); - uint64_t bm_sectors = DIV_ROUND_UP(bm_size, BDRV_SECTOR_SIZE); const char *bm_name = bdrv_dirty_bitmap_name(bitmap); uint8_t *buf = NULL; BdrvDirtyBitmapIter *dbi; @@ -1100,18 +1099,22 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs, dbi = bdrv_dirty_iter_new(bitmap); buf = g_malloc(s->cluster_size); limit = bytes_covered_by_bitmap_cluster(s, bitmap); - sbc = limit >> BDRV_SECTOR_BITS; assert(DIV_ROUND_UP(bm_size, limit) == tb_size); - while ((sector = bdrv_dirty_iter_next(dbi) >> BDRV_SECTOR_BITS) >= 0) { - uint64_t cluster = sector / sbc; + while ((offset = bdrv_dirty_iter_next(dbi)) >= 0) { + uint64_t cluster = offset / limit; uint64_t end, write_size; int64_t off; - sector = cluster * sbc; - end = MIN(bm_sectors, sector + sbc); - write_size = bdrv_dirty_bitmap_serialization_size(bitmap, - sector * BDRV_SECTOR_SIZE, (end - sector) * BDRV_SECTOR_SIZE); + /* + * We found the first dirty offset, but want to write out the + * entire cluster of the bitmap that includes that offset, + * including any leading zero bits. + */ + offset = QEMU_ALIGN_DOWN(offset, limit); + end = MIN(bm_size, offset + limit); + write_size = bdrv_dirty_bitmap_serialization_size(bitmap, offset, + end - offset); assert(write_size <= s->cluster_size); off = qcow2_alloc_clusters(bs, s->cluster_size); @@ -1123,9 +1126,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs, } tb[cluster] = off; - bdrv_dirty_bitmap_serialize_part(bitmap, buf, - sector * BDRV_SECTOR_SIZE, - (end - sector) * BDRV_SECTOR_SIZE); + bdrv_dirty_bitmap_serialize_part(bitmap, buf, offset, end - offset); if (write_size < s->cluster_size) { memset(buf + write_size, 0, s->cluster_size - write_size); } @@ -1143,11 +1144,11 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs, goto fail; } - if (end >= bm_sectors) { + if (end >= bm_size) { break; } - bdrv_set_dirty_iter(dbi, end * BDRV_SECTOR_SIZE); + bdrv_set_dirty_iter(dbi, end); } *bitmap_table_size = tb_size; diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165 index 74d7b79a0b..a3932db3de 100755 --- a/tests/qemu-iotests/165 +++ b/tests/qemu-iotests/165 @@ -27,7 +27,7 @@ disk = os.path.join(iotests.test_dir, 'disk') disk_size = 0x40000000 # 1G # regions for qemu_io: (start, count) in bytes -regions1 = ((0, 0x100000), +regions1 = ((0x0fff00, 0x10000), (0x200000, 0x100000)) regions2 = ((0x10000000, 0x20000),