From patchwork Wed Mar 27 22:39:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Blake X-Patchwork-Id: 10873993 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D79DB925 for ; Wed, 27 Mar 2019 22:43:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C2A4028A12 for ; Wed, 27 Mar 2019 22:43:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B6E9E28C07; Wed, 27 Mar 2019 22:43:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A5C0928A12 for ; Wed, 27 Mar 2019 22:43:13 +0000 (UTC) Received: from localhost ([127.0.0.1]:54834 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h9HGL-0003Bl-36 for patchwork-qemu-devel@patchwork.kernel.org; Wed, 27 Mar 2019 18:43:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50683) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h9HCw-0000gT-8g for qemu-devel@nongnu.org; Wed, 27 Mar 2019 18:39:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h9HCq-0006lk-Ej for qemu-devel@nongnu.org; Wed, 27 Mar 2019 18:39:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35766) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1h9HCi-0006jJ-0j; Wed, 27 Mar 2019 18:39:28 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 55377307D98D; Wed, 27 Mar 2019 22:39:27 +0000 (UTC) Received: from blue.redhat.com (ovpn-116-59.phx2.redhat.com [10.3.116.59]) by smtp.corp.redhat.com (Postfix) with ESMTP id AF3A45C23D; Wed, 27 Mar 2019 22:39:26 +0000 (UTC) From: Eric Blake To: qemu-devel@nongnu.org Date: Wed, 27 Mar 2019 17:39:19 -0500 Message-Id: <20190327223921.32601-2-eblake@redhat.com> In-Reply-To: <20190327223921.32601-1-eblake@redhat.com> References: <20190327223921.32601-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Wed, 27 Mar 2019 22:39:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 1/3] iotests: Add 241 to test NBD on unaligned images X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , vsementsov@virtuozzo.com, jsnow@redhat.com, qemu-block@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Add a test for the NBD client workaround in the previous patch. It's not really feasible for an iotest to assume a specific tracing engine, so we can't really probe for the new trace_nbd_parse_blockstatus_compliance to see if the server was fixed vs. whether the client just worked around the server (other than by rearranging order between code patches and this test). But having a successful exchange sure beats the previous state of an error message. Not tested yet, but worth adding to this test in future patches: an NBD server that can advertise a non-sector-aligned size (such as nbdkit) causes qemu as the NBD client to misbehave when it rounds the size up and accesses beyond the advertised size. Qemu as NBD server never advertises a non-sector-aligned size (since bdrv_getlength() currently rounds up to sector boundaries); until qemu can act as such a server, testing that flaw will have to rely on external binaries. Signed-off-by: Eric Blake --- tests/qemu-iotests/241 | 71 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/241.out | 7 ++++ tests/qemu-iotests/group | 1 + 3 files changed, 79 insertions(+) create mode 100755 tests/qemu-iotests/241 create mode 100644 tests/qemu-iotests/241.out diff --git a/tests/qemu-iotests/241 b/tests/qemu-iotests/241 new file mode 100755 index 00000000000..b1f6e6452de --- /dev/null +++ b/tests/qemu-iotests/241 @@ -0,0 +1,71 @@ +#!/bin/bash +# +# Test qemu-nbd vs. unaligned images +# +# Copyright (C) 2018-2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +seq="$(basename $0)" +echo "QA output created by $seq" + +status=1 # failure is the default! + +nbd_unix_socket=$TEST_DIR/test_qemu_nbd_socket +rm -f "${TEST_DIR}/qemu-nbd.pid" + +_cleanup() +{ + _cleanup_test_img + nbd_server_stop +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +. ./common.nbd + +_supported_fmt raw +_supported_proto nbd +_supported_os Linux +_require_command QEMU_NBD + +echo +echo "=== Exporting unaligned raw image ===" +echo + +# can't use _make_test_img, because qemu-img rounds image size up, +# and because we want to use Unix socket rather than TCP port. Likewise, +# we have to redirect TEST_IMG to our server. +# This tests that we can deal with the hole at the end of an unaligned +# raw file (either because the server doesn't advertise alignment too +# large, or because the client ignores the server's noncompliance). +printf %01000d 0 > "$TEST_IMG_FILE" +nbd_server_start_unix_socket -f $IMGFMT -e 42 -x '' "$TEST_IMG_FILE" +TEST_IMG="nbd:unix:$nbd_unix_socket" + +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map +$QEMU_IO -c map "$TEST_IMG" + +# Not tested yet: we also want to ensure that qemu as NBD client does +# not access beyond the end of a server's advertised unaligned size. +# However, since qemu as server always rounds up to a sector alignment, +# we would have to use nbdkit to provoke the current client failures. + +# success, all done +echo '*** done' +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/241.out b/tests/qemu-iotests/241.out new file mode 100644 index 00000000000..044afc0c6f8 --- /dev/null +++ b/tests/qemu-iotests/241.out @@ -0,0 +1,7 @@ +QA output created by 241 + +=== Exporting unaligned raw image === + +[{ "start": 0, "length": 1024, "depth": 0, "zero": false, "data": true}] +1 KiB (0x400) bytes allocated at offset 0 bytes (0x0) +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 41da10c6cf5..bae77183809 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -240,6 +240,7 @@ 238 auto quick 239 rw auto quick 240 auto quick +241 rw auto quick 242 rw auto quick 243 rw auto quick 244 rw auto quick From patchwork Wed Mar 27 22:39:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Blake X-Patchwork-Id: 10873991 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E4A9925 for ; Wed, 27 Mar 2019 22:41:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0AFC128A12 for ; Wed, 27 Mar 2019 22:41:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F135528B55; Wed, 27 Mar 2019 22:41:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E295228A12 for ; Wed, 27 Mar 2019 22:41:28 +0000 (UTC) Received: from localhost ([127.0.0.1]:54816 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h9HEe-000230-70 for patchwork-qemu-devel@patchwork.kernel.org; Wed, 27 Mar 2019 18:41:28 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50702) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h9HD2-0000mc-Bk for qemu-devel@nongnu.org; Wed, 27 Mar 2019 18:39:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h9HCy-0006nY-5v for qemu-devel@nongnu.org; Wed, 27 Mar 2019 18:39:48 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54920) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1h9HCi-0006jW-Te; Wed, 27 Mar 2019 18:39:29 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 38C67308FC20; Wed, 27 Mar 2019 22:39:28 +0000 (UTC) Received: from blue.redhat.com (ovpn-116-59.phx2.redhat.com [10.3.116.59]) by smtp.corp.redhat.com (Postfix) with ESMTP id 89E5C5C23D; Wed, 27 Mar 2019 22:39:27 +0000 (UTC) From: Eric Blake To: qemu-devel@nongnu.org Date: Wed, 27 Mar 2019 17:39:20 -0500 Message-Id: <20190327223921.32601-3-eblake@redhat.com> In-Reply-To: <20190327223921.32601-1-eblake@redhat.com> References: <20190327223921.32601-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Wed, 27 Mar 2019 22:39:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 2/3] block: Add bdrv_get_request_alignment() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , vsementsov@virtuozzo.com, jsnow@redhat.com, qemu-block@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP The next patch needs access to a device's minimum permitted alignment, since NBD wants to advertise this to clients. Add an accessor function, borrowing from blk_get_max_transfer() for accessing a backend's block limits. Signed-off-by: Eric Blake Message-Id: <20180802144834.520904-2-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy --- include/sysemu/block-backend.h | 1 + block/block-backend.c | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h index e2066eb06b2..3be05c2d689 100644 --- a/include/sysemu/block-backend.h +++ b/include/sysemu/block-backend.h @@ -177,6 +177,7 @@ bool blk_is_available(BlockBackend *blk); void blk_lock_medium(BlockBackend *blk, bool locked); void blk_eject(BlockBackend *blk, bool eject_flag); int blk_get_flags(BlockBackend *blk); +uint32_t blk_get_request_alignment(BlockBackend *blk); uint32_t blk_get_max_transfer(BlockBackend *blk); int blk_get_max_iov(BlockBackend *blk); void blk_set_guest_block_size(BlockBackend *blk, int align); diff --git a/block/block-backend.c b/block/block-backend.c index edad02a0f2a..f78e82a707f 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -1764,6 +1764,13 @@ int blk_get_flags(BlockBackend *blk) } } +/* Returns the minimum request alignment, in bytes; guaranteed nonzero */ +uint32_t blk_get_request_alignment(BlockBackend *blk) +{ + BlockDriverState *bs = blk_bs(blk); + return bs ? bs->bl.request_alignment : BDRV_SECTOR_SIZE; +} + /* Returns the maximum transfer length, in bytes; guaranteed nonzero */ uint32_t blk_get_max_transfer(BlockBackend *blk) { From patchwork Wed Mar 27 22:39:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Blake X-Patchwork-Id: 10873987 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3DF4015AC for ; Wed, 27 Mar 2019 22:41:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C4A328B55 for ; Wed, 27 Mar 2019 22:41:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1003328C0C; Wed, 27 Mar 2019 22:41:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7D11628B55 for ; Wed, 27 Mar 2019 22:41:13 +0000 (UTC) Received: from localhost ([127.0.0.1]:54812 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h9HEO-0001sD-4c for patchwork-qemu-devel@patchwork.kernel.org; Wed, 27 Mar 2019 18:41:12 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50627) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h9HCn-0000VH-F0 for qemu-devel@nongnu.org; Wed, 27 Mar 2019 18:39:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h9HCm-0006kf-DI for qemu-devel@nongnu.org; Wed, 27 Mar 2019 18:39:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36038) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1h9HCj-0006jm-Rz; Wed, 27 Mar 2019 18:39:30 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 115BE811DC; Wed, 27 Mar 2019 22:39:29 +0000 (UTC) Received: from blue.redhat.com (ovpn-116-59.phx2.redhat.com [10.3.116.59]) by smtp.corp.redhat.com (Postfix) with ESMTP id 66F0A5C23D; Wed, 27 Mar 2019 22:39:28 +0000 (UTC) From: Eric Blake To: qemu-devel@nongnu.org Date: Wed, 27 Mar 2019 17:39:21 -0500 Message-Id: <20190327223921.32601-4-eblake@redhat.com> In-Reply-To: <20190327223921.32601-1-eblake@redhat.com> References: <20190327223921.32601-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 27 Mar 2019 22:39:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 3/3] nbd/server: Advertise actual minimum block size X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , vsementsov@virtuozzo.com, jsnow@redhat.com, qemu-block@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Both NBD_CMD_BLOCK_STATUS and structured NBD_CMD_READ will split their reply according to bdrv_block_status() boundaries. If the block device has a request_alignment smaller than 512, but we advertise a block alignment of 512 to the client, then this can result in the server reply violating client expectations by reporting a smaller region of the export than what the client is permitted to address (although this is less of an issue for qemu 4.0 clients, given recent client patches to overlook our non-compliance). Since it's always better to be strict in what we send, it is worth advertising the actual minimum block limit rather than blindly rounding it up to 512. Note that this patch is not foolproof - it is still possible to provoke non-compliant server behavior using: $ qemu-nbd --image-opts driver=blkdebug,align=512,image.driver=file,image.filename=/path/to/non-aligned-file But as blkdebug is not normally used, and as this patch makes our server more interoperable with qemu 3.1 clients, it is worth applying now, even while we still work on a larger patch for the 4.1 timeframe to improve the block layer to prevent the mid-block status changes that can be observed now with blkdebug or with a backing layer with smaller alignment than the active layer. Note that the iotests output changes - both pre- and post-patch, the server is reporting a mid-sector hole; but pre-patch, the client was then rounding that up to a sector boundary as a workaround, while post-patch the client doesn't have to round because it sees the server's smaller advertised block size. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- nbd/server.c | 12 +++++++----- tests/qemu-iotests/241.out | 3 ++- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/nbd/server.c b/nbd/server.c index fd013a2817a..c76f32dbb50 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -607,13 +607,15 @@ static int nbd_negotiate_handle_info(NBDClient *client, uint16_t myflags, /* Send NBD_INFO_BLOCK_SIZE always, but tweak the minimum size * according to whether the client requested it, and according to * whether this is OPT_INFO or OPT_GO. */ - /* minimum - 1 for back-compat, or 512 if client is new enough. - * TODO: consult blk_bs(blk)->bl.request_alignment? */ - sizes[0] = - (client->opt == NBD_OPT_INFO || blocksize) ? BDRV_SECTOR_SIZE : 1; + /* minimum - 1 for back-compat, or actual if client will obey it. */ + if (client->opt == NBD_OPT_INFO || blocksize) { + sizes[0] = blk_get_request_alignment(exp->blk); + } else { + sizes[0] = 1; + } /* preferred - Hard-code to 4096 for now. * TODO: is blk_bs(blk)->bl.opt_transfer appropriate? */ - sizes[1] = 4096; + sizes[1] = MAX(4096, sizes[0]); /* maximum - At most 32M, but smaller as appropriate. */ sizes[2] = MIN(blk_get_max_transfer(exp->blk), NBD_MAX_BUFFER_SIZE); trace_nbd_negotiate_handle_info_block_size(sizes[0], sizes[1], sizes[2]); diff --git a/tests/qemu-iotests/241.out b/tests/qemu-iotests/241.out index 044afc0c6f8..a7f1e665a9a 100644 --- a/tests/qemu-iotests/241.out +++ b/tests/qemu-iotests/241.out @@ -2,6 +2,7 @@ QA output created by 241 === Exporting unaligned raw image === -[{ "start": 0, "length": 1024, "depth": 0, "zero": false, "data": true}] +[{ "start": 0, "length": 1000, "depth": 0, "zero": false, "data": true}, +{ "start": 1000, "length": 24, "depth": 0, "zero": true, "data": true}] 1 KiB (0x400) bytes allocated at offset 0 bytes (0x0) *** done