From patchwork Thu May 4 03:07:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Blake X-Patchwork-Id: 9710751 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1E76B60387 for ; Thu, 4 May 2017 03:14:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 023DD28667 for ; Thu, 4 May 2017 03:14:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EB1F728687; Thu, 4 May 2017 03:14:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 908E628667 for ; Thu, 4 May 2017 03:14:35 +0000 (UTC) Received: from localhost ([::1]:39530 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d67Du-0007wu-Oi for patchwork-qemu-devel@patchwork.kernel.org; Wed, 03 May 2017 23:14:34 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52637) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d677q-000365-61 for qemu-devel@nongnu.org; Wed, 03 May 2017 23:08:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d677n-0001LX-NN for qemu-devel@nongnu.org; Wed, 03 May 2017 23:08:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50934) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d677h-0001Gv-IW; Wed, 03 May 2017 23:08:09 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ABCA2C04BD26; Thu, 4 May 2017 03:08:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com ABCA2C04BD26 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com ABCA2C04BD26 Received: from red.redhat.com (ovpn-122-206.rdu2.redhat.com [10.10.122.206]) by smtp.corp.redhat.com (Postfix) with ESMTP id C767062923; Thu, 4 May 2017 03:08:07 +0000 (UTC) From: Eric Blake To: qemu-devel@nongnu.org Date: Wed, 3 May 2017 22:07:52 -0500 Message-Id: <20170504030755.1001-8-eblake@redhat.com> In-Reply-To: <20170504030755.1001-1-eblake@redhat.com> References: <20170504030755.1001-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 04 May 2017 03:08:08 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v12 07/10] iotests: Add test 179 to cover write zeroes with unmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP No tests were covering write zeroes with unmap. Additionally, I needed to prove that my previous patches for correct status reporting and write zeroes optimizations actually had an impact. The test works for cluster_size between 8k and 2M (for smaller sizes, it fails because our allocation patterns are not contiguous with small clusters - in part, the largest consecutive allocation we tend to get is often bounded by the size covered by one L2 table). Note that testing for zero clusters is tricky: 'qemu-io map' reports whether data comes from the current layer of the image (useful for sniffing out which regions of the file have QCOW_OFLAG_ZERO) - but doesn't show which clusters have mappings; while 'qemu-img map' sees "zero":true for both unallocated and zero clusters for any qcow2 with no backing layer (so less useful at detecting true zero clusters), but reliably shows mappings. So we have to rely on both queries side-by-side at each point of the test. Signed-off-by: Eric Blake Reviewed-by: Max Reitz --- v12: probe the map in more places, to make test easier to follow v11: reserved for blkdebug half of v10 v10: drop any changes to v2 files, rewrite test to work with updates earlier in the series, add a blkdebug probe v9: new patch --- tests/qemu-iotests/179 | 132 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/179.out | 160 +++++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 293 insertions(+) create mode 100755 tests/qemu-iotests/179 create mode 100644 tests/qemu-iotests/179.out diff --git a/tests/qemu-iotests/179 b/tests/qemu-iotests/179 new file mode 100755 index 0000000..124666f --- /dev/null +++ b/tests/qemu-iotests/179 @@ -0,0 +1,132 @@ +#!/bin/bash +# +# Test case for write zeroes with unmap +# +# Copyright (C) 2017 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=eblake@redhat.com + +seq="$(basename $0)" +echo "QA output created by $seq" + +here="$PWD" +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +# v2 images can't mark clusters as zero +_unsupported_imgopts compat=0.10 + +echo +echo '=== Testing write zeroes with unmap ===' +echo + +TEST_IMG="$TEST_IMG.base" _make_test_img 64M +_make_test_img -b "$TEST_IMG.base" + +# Offsets chosen at or near 2M boundaries so test works at any cluster size + +# Aligned writes to unallocated cluster should not allocate mapping, but must +# mark cluster as zero, whether or not unmap was requested +$QEMU_IO -c "write -z -u 2M 2M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 6M 2M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "map" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG.base" | _filter_qemu_img_map + +# Unaligned writes need not allocate mapping if the cluster already reads +# as zero, but must mark cluster as zero, whether or not unmap was requested +$QEMU_IO -c "write -z -u 10485761 2097150" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 14680065 2097150" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "map" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG.base" | _filter_qemu_img_map + +# Requesting unmap of normal data must deallocate; omitting unmap should +# preserve the mapping +$QEMU_IO -c "write 18M 14M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z -u 20M 2M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 24M 6M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "map" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG.base" | _filter_qemu_img_map + +# Likewise when writing on already-mapped zero data +$QEMU_IO -c "write -z -u 26M 2M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 28M 2M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "map" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG.base" | _filter_qemu_img_map + +# Writing on unmapped zeroes does not allocate +$QEMU_IO -c "write -z 32M 8M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z -u 34M 2M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 36M 2M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "map" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG.base" | _filter_qemu_img_map + +# Writing zero overrides a backing file, regardless of backing cluster type +$QEMU_IO -c "write -z 40M 8M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write 48M 8M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z -u 42M 2M" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z 44M 2M" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z -u 50M 2M" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z 52M 2M" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z -u 58M 2M" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z 60M 2M" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "map" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +# Final check that mappings are correct and images are still sane +TEST_IMG="$TEST_IMG.base" _check_test_img +_check_test_img + +echo +echo '=== Testing cache optimization ===' +echo + +BLKDBG_TEST_IMG="blkdebug:$TEST_DIR/blkdebug.conf:$TEST_IMG.base" + +cat > "$TEST_DIR/blkdebug.conf" <