From patchwork Mon Mar 4 10:08:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Shinkevich X-Patchwork-Id: 10837519 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2684017E9 for ; Mon, 4 Mar 2019 10:12:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1105B2A00A for ; Mon, 4 Mar 2019 10:12:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 037F32A007; Mon, 4 Mar 2019 10:12:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 65F5E2A007 for ; Mon, 4 Mar 2019 10:12:28 +0000 (UTC) Received: from localhost ([127.0.0.1]:51257 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h0kaB-0008JH-Pj for patchwork-qemu-devel@patchwork.kernel.org; Mon, 04 Mar 2019 05:12:27 -0500 Received: from eggs.gnu.org ([209.51.188.92]:50089) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h0kWk-0005m3-Dc for qemu-devel@nongnu.org; Mon, 04 Mar 2019 05:08:55 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h0kWg-0005pr-Ou for qemu-devel@nongnu.org; Mon, 04 Mar 2019 05:08:54 -0500 Received: from relay.sw.ru ([185.231.240.75]:41440) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1h0kWe-0005h7-K1; Mon, 04 Mar 2019 05:08:50 -0500 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1h0kWX-0006jh-5M; Mon, 04 Mar 2019 13:08:41 +0300 From: Andrey Shinkevich To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 4 Mar 2019 13:08:38 +0300 Message-Id: <1551694120-768127-2-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1551694120-768127-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1551694120-768127-1-git-send-email-andrey.shinkevich@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 1/3] block: iterate_format with account of whitelisting X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, rkagan@virtuozzo.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP bdrv_iterate_format (which is currently only used for printing out the formats supported by the block layer) doesn't take format whitelisting into account. This creates a problem for tests: they enumerate supported formats to decide which tests to enable, but then discover that QEMU doesn't let them actually use some of those formats. To avoid that, exclude formats that are not whitelisted from enumeration, if whitelisting is in use. Since we have separate whitelists for r/w and r/o, take this a parameter to bdrv_iterate_format, and print two lists of supported formats (r/w and r/o) in main qemu. Signed-off-by: Roman Kagan Signed-off-by: Andrey Shinkevich --- block.c | 23 +++++++++++++++++++---- blockdev.c | 4 +++- include/block/block.h | 2 +- qemu-img.c | 2 +- 4 files changed, 24 insertions(+), 7 deletions(-) diff --git a/block.c b/block.c index 4ad0e90..1f0e23f 100644 --- a/block.c +++ b/block.c @@ -395,7 +395,7 @@ BlockDriver *bdrv_find_format(const char *format_name) return bdrv_do_find_format(format_name); } -int bdrv_is_whitelisted(BlockDriver *drv, bool read_only) +static int bdrv_format_is_whitelisted(const char *format_name, bool read_only) { static const char *whitelist_rw[] = { CONFIG_BDRV_RW_WHITELIST @@ -410,13 +410,13 @@ int bdrv_is_whitelisted(BlockDriver *drv, bool read_only) } for (p = whitelist_rw; *p; p++) { - if (!strcmp(drv->format_name, *p)) { + if (!strcmp(format_name, *p)) { return 1; } } if (read_only) { for (p = whitelist_ro; *p; p++) { - if (!strcmp(drv->format_name, *p)) { + if (!strcmp(format_name, *p)) { return 1; } } @@ -424,6 +424,11 @@ int bdrv_is_whitelisted(BlockDriver *drv, bool read_only) return 0; } +int bdrv_is_whitelisted(BlockDriver *drv, bool read_only) +{ + return bdrv_format_is_whitelisted(drv->format_name, read_only); +} + bool bdrv_uses_whitelist(void) { return use_bdrv_whitelist; @@ -4040,7 +4045,7 @@ static int qsort_strcmp(const void *a, const void *b) } void bdrv_iterate_format(void (*it)(void *opaque, const char *name), - void *opaque) + void *opaque, bool read_only) { BlockDriver *drv; int count = 0; @@ -4051,6 +4056,11 @@ void bdrv_iterate_format(void (*it)(void *opaque, const char *name), if (drv->format_name) { bool found = false; int i = count; + + if (use_bdrv_whitelist && !bdrv_is_whitelisted(drv, read_only)) { + continue; + } + while (formats && i && !found) { found = !strcmp(formats[--i], drv->format_name); } @@ -4069,6 +4079,11 @@ void bdrv_iterate_format(void (*it)(void *opaque, const char *name), bool found = false; int j = count; + if (use_bdrv_whitelist && + !bdrv_format_is_whitelisted(format_name, read_only)) { + continue; + } + while (formats && j && !found) { found = !strcmp(formats[--j], format_name); } diff --git a/blockdev.c b/blockdev.c index 8714ad2..292e121 100644 --- a/blockdev.c +++ b/blockdev.c @@ -531,7 +531,9 @@ static BlockBackend *blockdev_init(const char *file, QDict *bs_opts, if ((buf = qemu_opt_get(opts, "format")) != NULL) { if (is_help_option(buf)) { error_printf("Supported formats:"); - bdrv_iterate_format(bdrv_format_print, NULL); + bdrv_iterate_format(bdrv_format_print, NULL, false); + error_printf("\nSupported formats (read-only):"); + bdrv_iterate_format(bdrv_format_print, NULL, true); error_printf("\n"); goto early_err; } diff --git a/include/block/block.h b/include/block/block.h index 73357c6..0f98bc9 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -472,7 +472,7 @@ void bdrv_next_cleanup(BdrvNextIterator *it); BlockDriverState *bdrv_next_monitor_owned(BlockDriverState *bs); bool bdrv_is_encrypted(BlockDriverState *bs); void bdrv_iterate_format(void (*it)(void *opaque, const char *name), - void *opaque); + void *opaque, bool read_only); const char *bdrv_get_node_name(const BlockDriverState *bs); const char *bdrv_get_device_name(const BlockDriverState *bs); const char *bdrv_get_device_or_node_name(const BlockDriverState *bs); diff --git a/qemu-img.c b/qemu-img.c index 7853935..ccd3ba9 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -198,7 +198,7 @@ static void QEMU_NORETURN help(void) " 'skip=N' skip N bs-sized blocks at the start of input\n"; printf("%s\nSupported formats:", help_msg); - bdrv_iterate_format(format_print, NULL); + bdrv_iterate_format(format_print, NULL, false); printf("\n\n" QEMU_HELP_BOTTOM "\n"); exit(EXIT_SUCCESS); } From patchwork Mon Mar 4 10:08:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Shinkevich X-Patchwork-Id: 10837515 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E5DAC17E9 for ; Mon, 4 Mar 2019 10:10:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D2F0C29F1E for ; Mon, 4 Mar 2019 10:10:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C6D5729F67; Mon, 4 Mar 2019 10:10:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0BC0D29F1E for ; Mon, 4 Mar 2019 10:10:20 +0000 (UTC) Received: from localhost ([127.0.0.1]:51241 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h0kY7-0007CK-VU for patchwork-qemu-devel@patchwork.kernel.org; Mon, 04 Mar 2019 05:10:20 -0500 Received: from eggs.gnu.org ([209.51.188.92]:50088) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h0kWk-0005m2-DL for qemu-devel@nongnu.org; Mon, 04 Mar 2019 05:08:55 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h0kWg-0005pP-Ml for qemu-devel@nongnu.org; Mon, 04 Mar 2019 05:08:54 -0500 Received: from relay.sw.ru ([185.231.240.75]:41438) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1h0kWe-0005fN-K3; Mon, 04 Mar 2019 05:08:48 -0500 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1h0kWX-0006jh-BB; Mon, 04 Mar 2019 13:08:41 +0300 From: Andrey Shinkevich To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 4 Mar 2019 13:08:39 +0300 Message-Id: <1551694120-768127-3-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1551694120-768127-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1551694120-768127-1-git-send-email-andrey.shinkevich@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 2/3] iotests: ask QEMU for supported formats X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, rkagan@virtuozzo.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Supported formats listed by 'qemu' may differ from those listed by 'qemu-img' due to whitelists. Some test cases require specific formats that may be used with qemu. They can be inquired directly by running 'qemu -drive format=help'. The response takes whitelists into account. The method supported_formats() serves for that. The method decorator skip_if_unsupported() checks if all requested formats are whitelisted. If not, the test case will be skipped. That has been implemented in the 'check' file in the way similar to the 'test notrun' mechanism. Suggested-by: Roman Kagan Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/check | 16 +++++++++++++- tests/qemu-iotests/iotests.py | 50 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 895e1e3..b9d539c 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -25,6 +25,7 @@ try=0 n_bad=0 bad="" notrun="" +casenotrun="" interrupt=true # by default don't output timestamps @@ -664,6 +665,11 @@ END { if (NR > 0) { echo "Not run:$notrun" echo "Not run:$notrun" >>check.log fi + if [ ! -z "$casenotrun" ] + then + echo "Some cases not run in:$casenotrun" + echo "Some cases not run in:$casenotrun" >>check.log + fi if [ ! -z "$n_bad" -a $n_bad != 0 ] then echo "Failures:$bad" @@ -743,6 +749,10 @@ do printf " " # prettier output with timestamps. fi rm -f core $seq.notrun + if [ -f $seq.casenotrun ] + then + rm -f $seq.casenotrun + fi start=$(_wallclock) $timestamp && printf %s " [$(date "+%T")]" @@ -823,7 +833,11 @@ do fi fi fi - + if [ -f $seq.casenotrun ] + then + cat $seq.casenotrun + casenotrun="$casenotrun $seq" + fi fi # come here for each test, except when $showme is true diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index b461f53..8fe1620 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -728,6 +728,56 @@ def verify_quorum(): if not supports_quorum(): notrun('quorum support missing') +def qemu_pipe(*args): + '''Run qemu with an option to print something and exit (e.g. a help option), + and return its output''' + args = [qemu_prog] + qemu_opts + list(args) + subp = subprocess.Popen(args, stdout=subprocess.PIPE, + stderr=subprocess.STDOUT) + exitcode = subp.wait() + if exitcode < 0: + sys.stderr.write('qemu received signal %i: %s\n' % (-exitcode, + ' '.join(args))) + return subp.communicate()[0] + +def supported_formats(read_only=False): + '''Set 'read_only' to True to check ro-whitelist + Otherwise, rw-whitelist is checked''' + format_message = qemu_pipe("-drive", "format=help") + available = [] + + if read_only: + # Check read-only whitelist + available = format_message.splitlines()[1].split(":")[1].split() + else: + # Check read-write whitelist + available = format_message.splitlines()[0].split(":")[1].split() + + return available + +def case_notrun(reason): + '''Skip this test case''' + # Each test in qemu-iotests has a number ("seq") + seq = os.path.basename(sys.argv[0]) + + open('%s/%s.casenotrun' % (output_dir, seq), 'ab').write( + ' [case not run] ' + reason + '\n') + +def skip_if_unsupported(required_formats=[], read_only=False): + '''Skip Test Decorator + Runs the test if all the required formats are whitelisted''' + def skip_test_decorator(func): + def func_wrapper(*args, **kwargs): + usf_list = list(set(required_formats) - + set(supported_formats(read_only))) + if usf_list: + case_notrun('{}: formats {} are not whitelisted'.format( + args[0], usf_list)) + else: + return func(*args, **kwargs) + return func_wrapper + return skip_test_decorator + def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[], unsupported_fmts=[]): '''Run tests''' From patchwork Mon Mar 4 10:08:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Shinkevich X-Patchwork-Id: 10837517 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1A10917E9 for ; Mon, 4 Mar 2019 10:10:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0757C2A003 for ; Mon, 4 Mar 2019 10:10:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EF9B92A006; Mon, 4 Mar 2019 10:10:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A041E2A003 for ; Mon, 4 Mar 2019 10:10:54 +0000 (UTC) Received: from localhost ([127.0.0.1]:51249 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h0kYf-0007gQ-Tq for patchwork-qemu-devel@patchwork.kernel.org; Mon, 04 Mar 2019 05:10:53 -0500 Received: from eggs.gnu.org ([209.51.188.92]:50074) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h0kWi-0005k2-EK for qemu-devel@nongnu.org; Mon, 04 Mar 2019 05:08:53 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h0kWg-0005q2-Pd for qemu-devel@nongnu.org; Mon, 04 Mar 2019 05:08:52 -0500 Received: from relay.sw.ru ([185.231.240.75]:41436) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1h0kWe-0005h5-K4; Mon, 04 Mar 2019 05:08:50 -0500 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1h0kWX-0006jh-HH; Mon, 04 Mar 2019 13:08:41 +0300 From: Andrey Shinkevich To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 4 Mar 2019 13:08:40 +0300 Message-Id: <1551694120-768127-4-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1551694120-768127-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1551694120-768127-1-git-send-email-andrey.shinkevich@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 3/3] iotests: check whitelisted formats X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, rkagan@virtuozzo.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Some test cases require specific formats. The method decorator skip_if_unsupported() checks if requested formats are whitelisted. The test #139 was selected for a sample output: 137 3s ... 138 0s ... 139 2s ... [case not run] testBlkVerify (__main__.TestBlockdevDel): formats ['blkverify'] are not whitelisted [case not run] testQuorum (__main__.TestBlockdevDel): formats ['quorum'] are not whitelisted 140 0s ... Not run: 131 135 136 Some cases not run in: 139 Passed all 137 tests Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/139 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/139 b/tests/qemu-iotests/139 index 62402c1..cb97029 100755 --- a/tests/qemu-iotests/139 +++ b/tests/qemu-iotests/139 @@ -333,6 +333,7 @@ class TestBlockdevDel(iotests.QMPTestCase): self.delBlockDriverState('debug0') self.checkBlockDriverState('node0', False) + @iotests.skip_if_unsupported(['blkverify']) def testBlkVerify(self): self.addBlkVerify('verify0', 'node0', 'node1') # We cannot remove the children of a blkverify device @@ -343,6 +344,7 @@ class TestBlockdevDel(iotests.QMPTestCase): self.checkBlockDriverState('node0', False) self.checkBlockDriverState('node1', False) + @iotests.skip_if_unsupported(['quorum']) def testQuorum(self): if not iotests.supports_quorum(): return