From patchwork Fri Apr 19 13:51:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10908885 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BEB5D922 for ; Fri, 19 Apr 2019 14:08:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ADE5F28CFA for ; Fri, 19 Apr 2019 14:08:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A1DA128D00; Fri, 19 Apr 2019 14:08:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5179D28CFA for ; Fri, 19 Apr 2019 14:08:20 +0000 (UTC) Received: from localhost ([127.0.0.1]:57026 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHUBf-0007MR-IU for patchwork-qemu-devel@patchwork.kernel.org; Fri, 19 Apr 2019 10:08:19 -0400 Received: from eggs.gnu.org ([209.51.188.92]:55602) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHU8A-0003ld-Mh for qemu-devel@nongnu.org; Fri, 19 Apr 2019 10:04:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hHTvg-000751-9D for qemu-devel@nongnu.org; Fri, 19 Apr 2019 09:51:49 -0400 Received: from relay.sw.ru ([185.231.240.75]:33720) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hHTvg-0006z6-1Z; Fri, 19 Apr 2019 09:51:48 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hHTvb-0008V2-2j; Fri, 19 Apr 2019 16:51:43 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 19 Apr 2019 16:51:40 +0300 Message-Id: <20190419135142.91810-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190419135142.91810-1-vsementsov@virtuozzo.com> References: <20190419135142.91810-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 1/3] block/qcow2-refcount: add trace-point to qcow2_process_discards X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, den@virtuozzo.com, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Let's at least trace ignored failure. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 7 ++++++- block/trace-events | 3 +++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index e0fe322500..60284bcaac 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -30,6 +30,7 @@ #include "qemu/range.h" #include "qemu/bswap.h" #include "qemu/cutils.h" +#include "trace.h" static int64_t alloc_clusters_noref(BlockDriverState *bs, uint64_t size, uint64_t max); @@ -738,7 +739,11 @@ void qcow2_process_discards(BlockDriverState *bs, int ret) /* Discard is optional, ignore the return value */ if (ret >= 0) { - bdrv_pdiscard(bs->file, d->offset, d->bytes); + int r2 = bdrv_pdiscard(bs->file, d->offset, d->bytes); + if (r2 < 0) { + trace_qcow2_process_discards_failed_region(d->offset, d->bytes, + r2); + } } g_free(d); diff --git a/block/trace-events b/block/trace-events index 7335a42540..ea508f637e 100644 --- a/block/trace-events +++ b/block/trace-events @@ -91,6 +91,9 @@ qcow2_cache_get_done(void *co, int c, int i) "co %p is_l2_cache %d index %d" qcow2_cache_flush(void *co, int c) "co %p is_l2_cache %d" qcow2_cache_entry_flush(void *co, int c, int i) "co %p is_l2_cache %d index %d" +# qcow2-refcount.c +qcow2_process_discards_failed_region(uint64_t offset, uint64_t bytes, int ret) "offset 0x%" PRIx64 " bytes 0x%" PRIx64 " ret %d" + # qed-l2-cache.c qed_alloc_l2_cache_entry(void *l2_cache, void *entry) "l2_cache %p entry %p" qed_unref_l2_cache_entry(void *entry, int ref) "entry %p ref %d" From patchwork Fri Apr 19 13:51:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10908889 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E3D371515 for ; Fri, 19 Apr 2019 14:10:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8884828BB9 for ; Fri, 19 Apr 2019 14:10:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7D1F028D09; Fri, 19 Apr 2019 14:10:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2032C28BB9 for ; Fri, 19 Apr 2019 14:10:09 +0000 (UTC) Received: from localhost ([127.0.0.1]:57066 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHUDQ-0000Sq-EM for patchwork-qemu-devel@patchwork.kernel.org; Fri, 19 Apr 2019 10:10:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:55568) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHU8A-0003kQ-M8 for qemu-devel@nongnu.org; Fri, 19 Apr 2019 10:04:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hHTvg-000750-99 for qemu-devel@nongnu.org; Fri, 19 Apr 2019 09:51:49 -0400 Received: from relay.sw.ru ([185.231.240.75]:33724) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hHTvg-0006z5-1Y; Fri, 19 Apr 2019 09:51:48 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hHTvb-0008V2-AC; Fri, 19 Apr 2019 16:51:43 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 19 Apr 2019 16:51:41 +0300 Message-Id: <20190419135142.91810-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190419135142.91810-1-vsementsov@virtuozzo.com> References: <20190419135142.91810-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 2/3] block/io: bdrv_pdiscard: support int64_t bytes parameter X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, den@virtuozzo.com, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This fixes at least one overflow in qcow2_process_discards, which passes 64bit region length to bdrv_pdiscard which bytes (or sectors in the past) parameter is int since it's introduction in 0b919fae. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 4 ++-- block/io.c | 16 ++++++++-------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index c7a26199aa..69fa18867e 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -432,8 +432,8 @@ void bdrv_drain_all(void); AIO_WAIT_WHILE(bdrv_get_aio_context(bs_), \ cond); }) -int bdrv_pdiscard(BdrvChild *child, int64_t offset, int bytes); -int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int bytes); +int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes); +int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes); int bdrv_has_zero_init_1(BlockDriverState *bs); int bdrv_has_zero_init(BlockDriverState *bs); bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs); diff --git a/block/io.c b/block/io.c index dfc153b8d8..35c4157669 100644 --- a/block/io.c +++ b/block/io.c @@ -2653,7 +2653,7 @@ int bdrv_flush(BlockDriverState *bs) typedef struct DiscardCo { BdrvChild *child; int64_t offset; - int bytes; + int64_t bytes; int ret; } DiscardCo; static void coroutine_fn bdrv_pdiscard_co_entry(void *opaque) @@ -2664,14 +2664,15 @@ static void coroutine_fn bdrv_pdiscard_co_entry(void *opaque) aio_wait_kick(); } -int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int bytes) +int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, + int64_t bytes) { BdrvTrackedRequest req; int max_pdiscard, ret; int head, tail, align; BlockDriverState *bs = child->bs; - if (!bs || !bs->drv) { + if (!bs || !bs->drv || !bdrv_is_inserted(bs)) { return -ENOMEDIUM; } @@ -2679,9 +2680,8 @@ int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int bytes) return -EPERM; } - ret = bdrv_check_byte_request(bs, offset, bytes); - if (ret < 0) { - return ret; + if (offset < 0) { + return -EIO; } /* Do nothing if disabled. */ @@ -2716,7 +2716,7 @@ int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int bytes) assert(max_pdiscard >= bs->bl.request_alignment); while (bytes > 0) { - int num = bytes; + int64_t num = bytes; if (head) { /* Make small requests to get to alignment boundaries. */ @@ -2778,7 +2778,7 @@ out: return ret; } -int bdrv_pdiscard(BdrvChild *child, int64_t offset, int bytes) +int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes) { Coroutine *co; DiscardCo rwco = { From patchwork Fri Apr 19 13:51:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 10908883 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4270D1515 for ; Fri, 19 Apr 2019 14:08:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A01A28CFA for ; Fri, 19 Apr 2019 14:08:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1E44428D00; Fri, 19 Apr 2019 14:08:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9DAC828CFA for ; Fri, 19 Apr 2019 14:08:05 +0000 (UTC) Received: from localhost ([127.0.0.1]:57024 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHUBQ-00079J-PF for patchwork-qemu-devel@patchwork.kernel.org; Fri, 19 Apr 2019 10:08:04 -0400 Received: from eggs.gnu.org ([209.51.188.92]:55832) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hHU8A-00048M-2U for qemu-devel@nongnu.org; Fri, 19 Apr 2019 10:04:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hHTvg-00075f-Dj for qemu-devel@nongnu.org; Fri, 19 Apr 2019 09:51:49 -0400 Received: from relay.sw.ru ([185.231.240.75]:33728) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hHTvg-0006z4-5D; Fri, 19 Apr 2019 09:51:48 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hHTvb-0008V2-Fg; Fri, 19 Apr 2019 16:51:43 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 19 Apr 2019 16:51:42 +0300 Message-Id: <20190419135142.91810-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190419135142.91810-1-vsementsov@virtuozzo.com> References: <20190419135142.91810-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 3/3] iotests: test big qcow2 shrink X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, den@virtuozzo.com, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This test checks bug in qcow2_process_discards, fixed by previous commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/250 | 72 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/250.out | 21 +++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 94 insertions(+) create mode 100755 tests/qemu-iotests/250 create mode 100644 tests/qemu-iotests/250.out diff --git a/tests/qemu-iotests/250 b/tests/qemu-iotests/250 new file mode 100755 index 0000000000..ab64019dd2 --- /dev/null +++ b/tests/qemu-iotests/250 @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# +# Test big discard in qcow2 shrink +# +# Copyright (c) 2019 Virtuozzo International GmbH. All rights reserved. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=vsementsov@virtuozzo.com + +seq=`basename $0` +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +# This test checks that qcow2_process_discards does not truncate a discard +# request > 2G. +# To reproduce bug we need to overflow int by one sequential discard, so we +# need size > 2G, bigger cluster size (as with default 64k we may have maximum +# of 512M sequential data, corresponding to one L1 entry), and we need some +# data of the beginning of the disk mapped to the end of file to prevent +# bdrv_co_truncate(bs->file) call in qcow2_co_truncate(), which may success +# anyway. + +size=2100M +IMGOPTS="cluster_size=1M,preallocation=metadata" + +_make_test_img $size +$QEMU_IO -c 'discard 0 10M' -c 'discard 2090M 10M' \ + -c 'write 2090M 10M' -c 'write 0 10M' "$TEST_IMG" | _filter_qemu_io + +# Check that our trick with swapping first and last 10M chunks succeeded. +# Otherwise test will may pass even if bdrv_pdiscard() fails in +# qcow2_process_discards() +$QEMU_IMG map "$TEST_IMG" | _filter_qemu_img_map +$QEMU_IMG info "$TEST_IMG" | grep size | _filter_testdir + +$QEMU_IMG -T 'qcow2_process_discards_failed*' resize --shrink "$TEST_IMG" 5M + +$QEMU_IMG info "$TEST_IMG" | grep size | _filter_testdir + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/250.out b/tests/qemu-iotests/250.out new file mode 100644 index 0000000000..62ab775e6c --- /dev/null +++ b/tests/qemu-iotests/250.out @@ -0,0 +1,21 @@ +QA output created by 250 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 preallocation=metadata +discard 10485760/10485760 bytes at offset 0 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +discard 10485760/10485760 bytes at offset 2191523840 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 10485760/10485760 bytes at offset 2191523840 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 10485760/10485760 bytes at offset 0 +10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0 0xa00000 TEST_DIR/t.IMGFMT +0x82a00000 0xa00000 TEST_DIR/t.IMGFMT +virtual size: 2.1G (2202009600 bytes) +disk size: 24M +cluster_size: 1048576 +Image resized. +virtual size: 5.0M (5242880 bytes) +disk size: 9.0M +cluster_size: 1048576 +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index bae7718380..588ae8b8b1 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -248,3 +248,4 @@ 246 rw auto quick 247 rw auto quick 248 rw auto quick +250 rw auto quick