From patchwork Fri Apr 19 14:05:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10908891 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0503D922 for ; Fri, 19 Apr 2019 14:11:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB0BD28B4D for ; Fri, 19 Apr 2019 14:11:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DF75928C6F; Fri, 19 Apr 2019 14:11:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7D9A928D0D for ; Fri, 19 Apr 2019 14:11:13 +0000 (UTC) Received: from localhost ([127.0.0.1]:57105 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHUES-0001MK-FM for patchwork-qemu-devel@patchwork.kernel.org; Fri, 19 Apr 2019 10:11:12 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56391) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHU9h-0005gL-2J for qemu-devel@nongnu.org; Fri, 19 Apr 2019 10:06:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hHU9f-00086Q-PN for qemu-devel@nongnu.org; Fri, 19 Apr 2019 10:06:16 -0400 Received: from relay.sw.ru ([185.231.240.75]:34414) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hHU9c-0007cu-CN; Fri, 19 Apr 2019 10:06:13 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hHU9N-0000CE-V2; Fri, 19 Apr 2019 17:05:58 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 19 Apr 2019 17:05:55 +0300 Message-Id: <20190419140557.102725-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190419140557.102725-1-vsementsov@virtuozzo.com> References: <20190419140557.102725-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 1/3] block/qcow2-refcount: add trace-point to qcow2_process_discards X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, den@virtuozzo.com, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Let's at least trace ignored failure. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- block/qcow2-refcount.c | 7 ++++++- block/trace-events | 3 +++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index e0fe322500..60284bcaac 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -30,6 +30,7 @@ #include "qemu/range.h" #include "qemu/bswap.h" #include "qemu/cutils.h" +#include "trace.h" static int64_t alloc_clusters_noref(BlockDriverState *bs, uint64_t size, uint64_t max); @@ -738,7 +739,11 @@ void qcow2_process_discards(BlockDriverState *bs, int ret) /* Discard is optional, ignore the return value */ if (ret >= 0) { - bdrv_pdiscard(bs->file, d->offset, d->bytes); + int r2 = bdrv_pdiscard(bs->file, d->offset, d->bytes); + if (r2 < 0) { + trace_qcow2_process_discards_failed_region(d->offset, d->bytes, + r2); + } } g_free(d); diff --git a/block/trace-events b/block/trace-events index 7335a42540..ea508f637e 100644 --- a/block/trace-events +++ b/block/trace-events @@ -91,6 +91,9 @@ qcow2_cache_get_done(void *co, int c, int i) "co %p is_l2_cache %d index %d" qcow2_cache_flush(void *co, int c) "co %p is_l2_cache %d" qcow2_cache_entry_flush(void *co, int c, int i) "co %p is_l2_cache %d index %d" +# qcow2-refcount.c +qcow2_process_discards_failed_region(uint64_t offset, uint64_t bytes, int ret) "offset 0x%" PRIx64 " bytes 0x%" PRIx64 " ret %d" + # qed-l2-cache.c qed_alloc_l2_cache_entry(void *l2_cache, void *entry) "l2_cache %p entry %p" qed_unref_l2_cache_entry(void *entry, int ref) "entry %p ref %d" From patchwork Fri Apr 19 14:05:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10908895 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E99C922 for ; Fri, 19 Apr 2019 14:12:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0F985286B2 for ; Fri, 19 Apr 2019 14:12:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 03FD528BDA; Fri, 19 Apr 2019 14:12:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9D7FE286B2 for ; Fri, 19 Apr 2019 14:12:51 +0000 (UTC) Received: from localhost ([127.0.0.1]:57119 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHUG2-0002Vq-T5 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 19 Apr 2019 10:12:50 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56429) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHU9i-0005hy-ID for qemu-devel@nongnu.org; Fri, 19 Apr 2019 10:06:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hHU9h-00089w-Eh for qemu-devel@nongnu.org; Fri, 19 Apr 2019 10:06:18 -0400 Received: from relay.sw.ru ([185.231.240.75]:34400) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hHU9h-0007ct-65; Fri, 19 Apr 2019 10:06:17 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hHU9O-0000CE-3u; Fri, 19 Apr 2019 17:05:58 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 19 Apr 2019 17:05:56 +0300 Message-Id: <20190419140557.102725-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190419140557.102725-1-vsementsov@virtuozzo.com> References: <20190419140557.102725-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 2/3] block/io: bdrv_pdiscard: support int64_t bytes parameter X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, den@virtuozzo.com, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This fixes at least one overflow in qcow2_process_discards, which passes 64bit region length to bdrv_pdiscard which bytes (or sectors in the past) parameter is int since it's introduction in 0b919fae. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- include/block/block.h | 4 ++-- block/io.c | 16 ++++++++-------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index c7a26199aa..69fa18867e 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -432,8 +432,8 @@ void bdrv_drain_all(void); AIO_WAIT_WHILE(bdrv_get_aio_context(bs_), \ cond); }) -int bdrv_pdiscard(BdrvChild *child, int64_t offset, int bytes); -int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int bytes); +int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes); +int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes); int bdrv_has_zero_init_1(BlockDriverState *bs); int bdrv_has_zero_init(BlockDriverState *bs); bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs); diff --git a/block/io.c b/block/io.c index dfc153b8d8..35c4157669 100644 --- a/block/io.c +++ b/block/io.c @@ -2653,7 +2653,7 @@ int bdrv_flush(BlockDriverState *bs) typedef struct DiscardCo { BdrvChild *child; int64_t offset; - int bytes; + int64_t bytes; int ret; } DiscardCo; static void coroutine_fn bdrv_pdiscard_co_entry(void *opaque) @@ -2664,14 +2664,15 @@ static void coroutine_fn bdrv_pdiscard_co_entry(void *opaque) aio_wait_kick(); } -int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int bytes) +int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, + int64_t bytes) { BdrvTrackedRequest req; int max_pdiscard, ret; int head, tail, align; BlockDriverState *bs = child->bs; - if (!bs || !bs->drv) { + if (!bs || !bs->drv || !bdrv_is_inserted(bs)) { return -ENOMEDIUM; } @@ -2679,9 +2680,8 @@ int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int bytes) return -EPERM; } - ret = bdrv_check_byte_request(bs, offset, bytes); - if (ret < 0) { - return ret; + if (offset < 0) { + return -EIO; } /* Do nothing if disabled. */ @@ -2716,7 +2716,7 @@ int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int bytes) assert(max_pdiscard >= bs->bl.request_alignment); while (bytes > 0) { - int num = bytes; + int64_t num = bytes; if (head) { /* Make small requests to get to alignment boundaries. */ @@ -2778,7 +2778,7 @@ out: return ret; } -int bdrv_pdiscard(BdrvChild *child, int64_t offset, int bytes) +int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes) { Coroutine *co; DiscardCo rwco = { From patchwork Fri Apr 19 14:05:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10908887 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15E8A922 for ; Fri, 19 Apr 2019 14:09:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 04E4B28CFA for ; Fri, 19 Apr 2019 14:09:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EBCDC28D00; Fri, 19 Apr 2019 14:09:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6A8A128CFA for ; Fri, 19 Apr 2019 14:09:06 +0000 (UTC) Received: from localhost ([127.0.0.1]:57035 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHUCP-0007zK-Mv for patchwork-qemu-devel@patchwork.kernel.org; Fri, 19 Apr 2019 10:09:05 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56373) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHU9g-0005fX-4F for qemu-devel@nongnu.org; Fri, 19 Apr 2019 10:06:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hHU9b-0007yQ-DT for qemu-devel@nongnu.org; Fri, 19 Apr 2019 10:06:13 -0400 Received: from relay.sw.ru ([185.231.240.75]:34406) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hHU9X-0007cv-JI; Fri, 19 Apr 2019 10:06:09 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hHU9O-0000CE-8R; Fri, 19 Apr 2019 17:05:58 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 19 Apr 2019 17:05:57 +0300 Message-Id: <20190419140557.102725-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190419140557.102725-1-vsementsov@virtuozzo.com> References: <20190419140557.102725-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 3/3] iotests: test big qcow2 shrink X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, den@virtuozzo.com, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This test checks bug in qcow2_process_discards, fixed by previous commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Tested-by: Eric Blake --- tests/qemu-iotests/250 | 72 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/250.out | 21 +++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 94 insertions(+) create mode 100755 tests/qemu-iotests/250 create mode 100644 tests/qemu-iotests/250.out diff --git a/tests/qemu-iotests/250 b/tests/qemu-iotests/250 new file mode 100755 index 0000000000..618ddef5e4 --- /dev/null +++ b/tests/qemu-iotests/250 @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# +# Test big discard in qcow2 shrink +# +# Copyright (c) 2019 Virtuozzo International GmbH. All rights reserved. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=vsementsov@virtuozzo.com + +seq=`basename $0` +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +# This test checks that qcow2_process_discards does not truncate a discard +# request > 2G. +# To reproduce bug we need to overflow int by one sequential discard, so we +# need size > 2G, bigger cluster size (as with default 64k we may have maximum +# of 512M sequential data, corresponding to one L1 entry), and we need some +# data of the beginning of the disk mapped to the end of file to prevent +# bdrv_co_truncate(bs->file) call in qcow2_co_truncate(), which may success +# anyway. + +size=2100M +IMGOPTS="cluster_size=1M,preallocation=metadata" + +_make_test_img $size +$QEMU_IO -c 'discard 0 10M' -c 'discard 2090M 10M' \ + -c 'write 2090M 10M' -c 'write 0 10M' "$TEST_IMG" | _filter_qemu_io + +# Check that our trick with swapping first and last 10M chunks succeeded. +# Otherwise test will may pass even if bdrv_pdiscard() fails in +# qcow2_process_discards() +$QEMU_IMG map "$TEST_IMG" | _filter_testdir +$QEMU_IMG info "$TEST_IMG" | grep size | _filter_testdir + +$QEMU_IMG -T 'qcow2_process_discards_failed*' resize --shrink "$TEST_IMG" 5M + +$QEMU_IMG info "$TEST_IMG" | grep size | _filter_testdir + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/250.out b/tests/qemu-iotests/250.out new file mode 100644 index 0000000000..37e37f0c9e --- /dev/null +++ b/tests/qemu-iotests/250.out @@ -0,0 +1,21 @@ +QA output created by 250 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 preallocation=metadata +discard 10485760/10485760 bytes at offset 0 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +discard 10485760/10485760 bytes at offset 2191523840 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 10485760/10485760 bytes at offset 2191523840 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 10485760/10485760 bytes at offset 0 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length Mapped to File +0 0xa00000 0x82f00000 TEST_DIR/t.qcow2 +0x82a00000 0xa00000 0x500000 TEST_DIR/t.qcow2 +virtual size: 2.1G (2202009600 bytes) +disk size: 24M +cluster_size: 1048576 +Image resized. +virtual size: 5.0M (5242880 bytes) +disk size: 9.0M +cluster_size: 1048576 +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index bae7718380..588ae8b8b1 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -248,3 +248,4 @@ 246 rw auto quick 247 rw auto quick 248 rw auto quick +250 rw auto quick