From patchwork Mon Nov 12 07:06:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gonglei (Arei)\" via" X-Patchwork-Id: 10678235 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3985F1759 for ; Mon, 12 Nov 2018 07:08:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2951E29FB6 for ; Mon, 12 Nov 2018 07:08:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1CFC129FC1; Mon, 12 Nov 2018 07:08:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B165629FBC for ; Mon, 12 Nov 2018 07:08:41 +0000 (UTC) Received: from localhost ([::1]:45091 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gM6Ku-0002WV-Sz for patchwork-qemu-devel@patchwork.kernel.org; Mon, 12 Nov 2018 02:08:40 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50538) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gM6Jd-00085E-8R for qemu-devel@nongnu.org; Mon, 12 Nov 2018 02:07:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gM6JZ-0004DZ-Hl for qemu-devel@nongnu.org; Mon, 12 Nov 2018 02:07:18 -0500 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]:35598) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1gM6JT-0003kE-8j; Mon, 12 Nov 2018 02:07:11 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1542006431; x=1573542431; h=from:to:cc:subject:date:message-id; bh=yqww3xNb6FSa6LAXrNWtVnzGsfxec677dt8cxPzb3WM=; b=UzM23vRR/AYIplkbNSuQYZmov+D0wYm/ldCTfaxWYn7ZrkBNIV96l5JV iatOroZ6ELlpvhQ1bP74xHptn8KJw6uPeSwRWpUSAYV9A2N1LRuvT6iTq bllCHrtYS91V5ipMXP4RhxaIOZXqvwMysL7LYQVSB+9hmmXVwdh4nEgUH 0=; X-IronPort-AV: E=Sophos;i="5.54,494,1534809600"; d="scan'208";a="764611644" Received: from sea3-co-svc-lb6-vlan2.sea.amazon.com (HELO email-inbound-relay-1d-5dd976cd.us-east-1.amazon.com) ([10.47.22.34]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Nov 2018 07:07:07 +0000 Received: from ue4a4719f239757c4d2ec.ant.amazon.com (iad7-ws-svc-lb50-vlan3.amazon.com [10.0.93.214]) by email-inbound-relay-1d-5dd976cd.us-east-1.amazon.com (8.14.7/8.14.7) with ESMTP id wAC774ZU030082 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 12 Nov 2018 07:07:05 GMT Received: from ue4a4719f239757c4d2ec.ant.amazon.com (localhost [127.0.0.1]) by ue4a4719f239757c4d2ec.ant.amazon.com (8.15.2/8.15.2/Debian-3) with ESMTP id wAC773JS030100; Sun, 11 Nov 2018 23:07:03 -0800 Received: (from marcolso@localhost) by ue4a4719f239757c4d2ec.ant.amazon.com (8.15.2/8.15.2/Submit) id wAC772dw030095; Sun, 11 Nov 2018 23:07:02 -0800 To: qemu-devel@nongnu.org Date: Sun, 11 Nov 2018 23:06:36 -0800 Message-Id: <1542006398-30037-1-git-send-email-marcolso@amazon.com> X-Mailer: git-send-email 2.7.4 Precedence: Bulk X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 207.171.190.10 Subject: [Qemu-devel] [PATCH v3 1/3] blkdebug: fix one shot rule processing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Marc Olson via Qemu-devel From: "Gonglei (Arei)\" via" Reply-To: Marc Olson Cc: Kevin Wolf , Marc Olson , jsnow@redhat.com, qemu-block@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP If 'once' is specified, the rule should execute just once, regardless if it is supposed to return an error or not. Take the example where you want the first IO to an LBA to succeed, but subsequent IOs to fail. You could either use state transitions, or create two rules, one with error = 0 and once set to true, and one with a non-zero error. Signed-off-by: Marc Olson Reviewed-by: John Snow Reviewed-by: Max Reitz --- block/blkdebug.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index 0759452..327049b 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -488,7 +488,7 @@ static int rule_check(BlockDriverState *bs, uint64_t offset, uint64_t bytes) } } - if (!rule || !rule->options.inject.error) { + if (!rule) { return 0; } @@ -500,7 +500,7 @@ static int rule_check(BlockDriverState *bs, uint64_t offset, uint64_t bytes) remove_rule(rule); } - if (!immediately) { + if (error && !immediately) { aio_co_schedule(qemu_get_current_aio_context(), qemu_coroutine_self()); qemu_coroutine_yield(); } From patchwork Mon Nov 12 07:06:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gonglei (Arei)\" via" X-Patchwork-Id: 10678233 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1FDEF15A6 for ; Mon, 12 Nov 2018 07:08:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E99E29FBE for ; Mon, 12 Nov 2018 07:08:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0123529FC1; Mon, 12 Nov 2018 07:08:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 661D429FB6 for ; Mon, 12 Nov 2018 07:08:41 +0000 (UTC) Received: from localhost ([::1]:45090 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gM6Ku-0002Vj-9Q for patchwork-qemu-devel@patchwork.kernel.org; Mon, 12 Nov 2018 02:08:40 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50535) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gM6Ja-00084o-1k for qemu-devel@nongnu.org; Mon, 12 Nov 2018 02:07:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gM6JX-000449-8d for qemu-devel@nongnu.org; Mon, 12 Nov 2018 02:07:17 -0500 Received: from smtp-fw-2101.amazon.com ([72.21.196.25]:16459) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1gM6JS-0003dn-Jc; Mon, 12 Nov 2018 02:07:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1542006430; x=1573542430; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=WXKJ0EBT7WU6xkXtJMPb2w+1h0sMYTIuTSDi1ENC99I=; b=tAjZxaNcFropEA0O5tr5uTPRsFxjmrOvGiwG+m3ci6itTYOQCZf2L0fu AuT3uF/5/G2TWMhkzNBrDdA8sE+sS3a3xL1iMrTCcd3bkVoyERR/eIUK2 ChMd4HbLoxjh24RKnPDUh5ay7lGK97N7NmkZVrRwMQXO8irbNVj6kdkAi E=; X-IronPort-AV: E=Sophos;i="5.54,494,1534809600"; d="scan'208";a="703956363" Received: from iad6-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-1a-821c648d.us-east-1.amazon.com) ([10.124.125.2]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Nov 2018 07:07:07 +0000 Received: from ue4a4719f239757c4d2ec.ant.amazon.com (iad7-ws-svc-lb50-vlan3.amazon.com [10.0.93.214]) by email-inbound-relay-1a-821c648d.us-east-1.amazon.com (8.14.7/8.14.7) with ESMTP id wAC774q1023707 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 12 Nov 2018 07:07:05 GMT Received: from ue4a4719f239757c4d2ec.ant.amazon.com (localhost [127.0.0.1]) by ue4a4719f239757c4d2ec.ant.amazon.com (8.15.2/8.15.2/Debian-3) with ESMTP id wAC774WU030106; Sun, 11 Nov 2018 23:07:04 -0800 Received: (from marcolso@localhost) by ue4a4719f239757c4d2ec.ant.amazon.com (8.15.2/8.15.2/Submit) id wAC773pV030105; Sun, 11 Nov 2018 23:07:03 -0800 To: qemu-devel@nongnu.org Date: Sun, 11 Nov 2018 23:06:37 -0800 Message-Id: <1542006398-30037-2-git-send-email-marcolso@amazon.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1542006398-30037-1-git-send-email-marcolso@amazon.com> References: <1542006398-30037-1-git-send-email-marcolso@amazon.com> Precedence: Bulk X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 72.21.196.25 Subject: [Qemu-devel] [PATCH v3 2/3] blkdebug: Extend rule check for additional types X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Marc Olson via Qemu-devel From: "Gonglei (Arei)\" via" Reply-To: Marc Olson Cc: Kevin Wolf , Marc Olson , jsnow@redhat.com, qemu-block@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Break out the more common parts of the BlkdebugRule struct, and make rule_check() more explicit about operating only on error injection types so that additional rule types can be added in the future. Signed-off-by: Marc Olson Reviewed-By: John Snow Reviewed-by: Max Reitz --- block/blkdebug.c | 59 +++++++++++++++++++++++++++++--------------------------- 1 file changed, 31 insertions(+), 28 deletions(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index 327049b..7739849 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -73,13 +73,13 @@ typedef struct BlkdebugRule { BlkdebugEvent event; int action; int state; + int once; + int64_t offset; union { struct { int error; int immediately; - int once; - int64_t offset; - } inject; + } inject_error; struct { int new_state; } set_state; @@ -182,16 +182,16 @@ static int add_rule(void *opaque, QemuOpts *opts, Error **errp) .state = qemu_opt_get_number(opts, "state", 0), }; + rule->once = qemu_opt_get_bool(opts, "once", 0); + sector = qemu_opt_get_number(opts, "sector", -1); + rule->offset = sector == -1 ? -1 : sector * BDRV_SECTOR_SIZE; + /* Parse action-specific options */ switch (d->action) { case ACTION_INJECT_ERROR: - rule->options.inject.error = qemu_opt_get_number(opts, "errno", EIO); - rule->options.inject.once = qemu_opt_get_bool(opts, "once", 0); - rule->options.inject.immediately = + rule->options.inject_error.error = qemu_opt_get_number(opts, "errno", EIO); + rule->options.inject_error.immediately = qemu_opt_get_bool(opts, "immediately", 0); - sector = qemu_opt_get_number(opts, "sector", -1); - rule->options.inject.offset = - sector == -1 ? -1 : sector * BDRV_SECTOR_SIZE; break; case ACTION_SET_STATE: @@ -474,38 +474,41 @@ static int rule_check(BlockDriverState *bs, uint64_t offset, uint64_t bytes) { BDRVBlkdebugState *s = bs->opaque; BlkdebugRule *rule = NULL; + BlkdebugRule *error_rule = NULL; int error; bool immediately; + int ret = 0; QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) { - uint64_t inject_offset = rule->options.inject.offset; - - if (inject_offset == -1 || - (bytes && inject_offset >= offset && - inject_offset < offset + bytes)) + if (rule->offset == -1 || + (bytes && rule->offset >= offset && + rule->offset < offset + bytes)) { - break; + if (rule->action == ACTION_INJECT_ERROR) { + error_rule = rule; + break; + } } } - if (!rule) { - return 0; - } + if (error_rule) { + immediately = error_rule->options.inject_error.immediately; + error = error_rule->options.inject_error.error; - immediately = rule->options.inject.immediately; - error = rule->options.inject.error; + if (error_rule->once) { + QSIMPLEQ_REMOVE(&s->active_rules, error_rule, BlkdebugRule, active_next); + remove_rule(error_rule); + } - if (rule->options.inject.once) { - QSIMPLEQ_REMOVE(&s->active_rules, rule, BlkdebugRule, active_next); - remove_rule(rule); - } + if (error && !immediately) { + aio_co_schedule(qemu_get_current_aio_context(), qemu_coroutine_self()); + qemu_coroutine_yield(); + } - if (error && !immediately) { - aio_co_schedule(qemu_get_current_aio_context(), qemu_coroutine_self()); - qemu_coroutine_yield(); + ret = -error; } - return -error; + return ret; } static int coroutine_fn From patchwork Mon Nov 12 07:06:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gonglei (Arei)\" via" X-Patchwork-Id: 10678237 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB7AD14DB for ; Mon, 12 Nov 2018 07:10:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD36129FCE for ; Mon, 12 Nov 2018 07:10:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B154C29FD4; Mon, 12 Nov 2018 07:10:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6A69129FCE for ; Mon, 12 Nov 2018 07:10:28 +0000 (UTC) Received: from localhost ([::1]:45103 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gM6Md-0003ar-ER for patchwork-qemu-devel@patchwork.kernel.org; Mon, 12 Nov 2018 02:10:27 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50553) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gM6Jf-00087F-Jk for qemu-devel@nongnu.org; Mon, 12 Nov 2018 02:07:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gM6JZ-0004Dn-R6 for qemu-devel@nongnu.org; Mon, 12 Nov 2018 02:07:21 -0500 Received: from smtp-fw-4101.amazon.com ([72.21.198.25]:42735) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1gM6JS-0003jb-IY; Mon, 12 Nov 2018 02:07:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1542006430; x=1573542430; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=Gz3MDurgvrAQk8Cld7LcQ5jOMhuI0pQVEilH0mGkM3g=; b=KGTcY+Kp76ysgRl+dSNJVArotrZybpI3xEq4AwJ2OX1Y3CMIuJlvhEhU TfvvAmkrNOMTnN6azFM9l/zxImtbKqJPEWcCtCnKBjL6P97u0S6jFhd/c I3et1/IwDeBt1wpC+6exYZnUqwClfEWpP2uG+HoJNQFzalUwsMJJCgXGy I=; X-IronPort-AV: E=Sophos;i="5.54,494,1534809600"; d="scan'208";a="745202449" Received: from iad6-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1d-98acfc19.us-east-1.amazon.com) ([10.124.125.6]) by smtp-border-fw-out-4101.iad4.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Nov 2018 07:07:08 +0000 Received: from ue4a4719f239757c4d2ec.ant.amazon.com (iad7-ws-svc-lb50-vlan3.amazon.com [10.0.93.214]) by email-inbound-relay-1d-98acfc19.us-east-1.amazon.com (8.14.7/8.14.7) with ESMTP id wAC775Iw070689 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 12 Nov 2018 07:07:05 GMT Received: from ue4a4719f239757c4d2ec.ant.amazon.com (localhost [127.0.0.1]) by ue4a4719f239757c4d2ec.ant.amazon.com (8.15.2/8.15.2/Debian-3) with ESMTP id wAC774b8030112; Sun, 11 Nov 2018 23:07:04 -0800 Received: (from marcolso@localhost) by ue4a4719f239757c4d2ec.ant.amazon.com (8.15.2/8.15.2/Submit) id wAC774ta030111; Sun, 11 Nov 2018 23:07:04 -0800 To: qemu-devel@nongnu.org Date: Sun, 11 Nov 2018 23:06:38 -0800 Message-Id: <1542006398-30037-3-git-send-email-marcolso@amazon.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1542006398-30037-1-git-send-email-marcolso@amazon.com> References: <1542006398-30037-1-git-send-email-marcolso@amazon.com> Precedence: Bulk X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 72.21.198.25 Subject: [Qemu-devel] [PATCH v3 3/3] blkdebug: Add latency injection rule type X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Marc Olson via Qemu-devel From: "Gonglei (Arei)\" via" Reply-To: Marc Olson Cc: Kevin Wolf , qemu-block@nongnu.org, jsnow@redhat.com, Marc Olson , Markus Armbruster , Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Add a new rule type for blkdebug that instead of returning an error, can inject latency to an IO. Signed-off-by: Marc Olson Reviewed-by: John Snow --- block/blkdebug.c | 79 +++++++++++++++++++++++++++++++++++++++++++--- docs/devel/blkdebug.txt | 35 ++++++++++++++------ qapi/block-core.json | 31 ++++++++++++++++++ tests/qemu-iotests/071 | 63 ++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/071.out | 31 ++++++++++++++++++ 5 files changed, 226 insertions(+), 13 deletions(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index 7739849..6b1f2d6 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -65,6 +65,7 @@ typedef struct BlkdebugSuspendedReq { enum { ACTION_INJECT_ERROR, + ACTION_INJECT_DELAY, ACTION_SET_STATE, ACTION_SUSPEND, }; @@ -81,6 +82,9 @@ typedef struct BlkdebugRule { int immediately; } inject_error; struct { + int64_t latency; + } delay; + struct { int new_state; } set_state; struct { @@ -123,6 +127,34 @@ static QemuOptsList inject_error_opts = { }, }; +static QemuOptsList inject_delay_opts = { + .name = "inject-delay", + .head = QTAILQ_HEAD_INITIALIZER(inject_delay_opts.head), + .desc = { + { + .name = "event", + .type = QEMU_OPT_STRING, + }, + { + .name = "state", + .type = QEMU_OPT_NUMBER, + }, + { + .name = "latency", + .type = QEMU_OPT_NUMBER, + }, + { + .name = "sector", + .type = QEMU_OPT_NUMBER, + }, + { + .name = "once", + .type = QEMU_OPT_BOOL, + }, + { /* end of list */ } + }, +}; + static QemuOptsList set_state_opts = { .name = "set-state", .head = QTAILQ_HEAD_INITIALIZER(set_state_opts.head), @@ -145,6 +177,7 @@ static QemuOptsList set_state_opts = { static QemuOptsList *config_groups[] = { &inject_error_opts, + &inject_delay_opts, &set_state_opts, NULL }; @@ -194,6 +227,11 @@ static int add_rule(void *opaque, QemuOpts *opts, Error **errp) qemu_opt_get_bool(opts, "immediately", 0); break; + case ACTION_INJECT_DELAY: + rule->options.delay.latency = + qemu_opt_get_number(opts, "latency", 100) * SCALE_US; + break; + case ACTION_SET_STATE: rule->options.set_state.new_state = qemu_opt_get_number(opts, "new_state", 0); @@ -226,6 +264,12 @@ static void remove_rule(BlkdebugRule *rule) g_free(rule); } +static void remove_active_rule(BDRVBlkdebugState *s, BlkdebugRule *rule) +{ + QSIMPLEQ_REMOVE(&s->active_rules, rule, BlkdebugRule, active_next); + remove_rule(rule); +} + static int read_config(BDRVBlkdebugState *s, const char *filename, QDict *options, Error **errp) { @@ -264,6 +308,14 @@ static int read_config(BDRVBlkdebugState *s, const char *filename, goto fail; } + d.action = ACTION_INJECT_DELAY; + qemu_opts_foreach(&inject_delay_opts, add_rule, &d, &local_err); + if (local_err) { + error_propagate(errp, local_err); + ret = -EINVAL; + goto fail; + } + d.action = ACTION_SET_STATE; qemu_opts_foreach(&set_state_opts, add_rule, &d, &local_err); if (local_err) { @@ -275,6 +327,7 @@ static int read_config(BDRVBlkdebugState *s, const char *filename, ret = 0; fail: qemu_opts_reset(&inject_error_opts); + qemu_opts_reset(&inject_delay_opts); qemu_opts_reset(&set_state_opts); if (f) { fclose(f); @@ -474,7 +527,8 @@ static int rule_check(BlockDriverState *bs, uint64_t offset, uint64_t bytes) { BDRVBlkdebugState *s = bs->opaque; BlkdebugRule *rule = NULL; - BlkdebugRule *error_rule = NULL; + BlkdebugRule *error_rule = NULL, *delay_rule = NULL; + int64_t latency; int error; bool immediately; int ret = 0; @@ -484,20 +538,36 @@ static int rule_check(BlockDriverState *bs, uint64_t offset, uint64_t bytes) (bytes && rule->offset >= offset && rule->offset < offset + bytes)) { - if (rule->action == ACTION_INJECT_ERROR) { + if (!error_rule && rule->action == ACTION_INJECT_ERROR) { error_rule = rule; + } else if (!delay_rule && rule->action == ACTION_INJECT_DELAY) { + delay_rule = rule; + } + + if (error_rule && delay_rule) { break; } } } + if (delay_rule) { + latency = delay_rule->options.delay.latency; + + if (delay_rule->once) { + remove_active_rule(s, delay_rule); + } + + if (latency != 0) { + qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, latency); + } + } + if (error_rule) { immediately = error_rule->options.inject_error.immediately; error = error_rule->options.inject_error.error; if (error_rule->once) { - QSIMPLEQ_REMOVE(&s->active_rules, error_rule, BlkdebugRule, active_next); - remove_rule(error_rule); + remove_active_rule(s, error_rule); } if (error && !immediately) { @@ -697,6 +767,7 @@ static bool process_rule(BlockDriverState *bs, struct BlkdebugRule *rule, /* Take the action */ switch (rule->action) { case ACTION_INJECT_ERROR: + case ACTION_INJECT_DELAY: if (!injected) { QSIMPLEQ_INIT(&s->active_rules); injected = true; diff --git a/docs/devel/blkdebug.txt b/docs/devel/blkdebug.txt index 43d8e8f..1719835 100644 --- a/docs/devel/blkdebug.txt +++ b/docs/devel/blkdebug.txt @@ -24,7 +24,7 @@ This way, all error paths can be tested to make sure they are correct. Rules ----- The blkdebug block driver takes a list of "rules" that tell the error injection -engine when to fail an I/O request. +engine when to either fail or add latency to an I/O request. Each I/O request is evaluated against the rules. If a rule matches the request then its "action" is executed. @@ -33,24 +33,35 @@ Rules can be placed in a configuration file; the configuration file follows the same .ini-like format used by QEMU's -readconfig option, and each section of the file represents a rule. -The following configuration file defines a single rule: +The following configuration file defines multiple rules: $ cat blkdebug.conf [inject-error] event = "read_aio" errno = "28" -This rule fails all aio read requests with ENOSPC (28). Note that the errno -value depends on the host. On Linux, see + [inject-delay] + event = "read_aio" + sector = "2048" + latency = "500000" + +The error rule fails all aio read requests with ENOSPC (28). Note that the +errno value depends on the host. On Linux, see /usr/include/asm-generic/errno-base.h for errno values. +The delay rule adds 500 ms of latency to a read I/O request containing sector +2048. + +An error rule and a delay rule can overlap, and both will execute. Only one +rule of a given type will be executed for each I/O. + Invoke QEMU as follows: $ qemu-system-x86_64 -drive if=none,cache=none,file=blkdebug:blkdebug.conf:test.img,id=drive0 \ -device virtio-blk-pci,drive=drive0,id=virtio-blk-pci0 -Rules support the following attributes: +All rules support the following attributes: event - which type of operation to match (e.g. read_aio, write_aio, flush_to_os, flush_to_disk). See the "Events" section for @@ -60,21 +71,27 @@ Rules support the following attributes: rule to match. See the "State transitions" section for information on states. - errno - the numeric errno value to return when a request matches this rule. - The errno values depend on the host since the numeric values are not - standarized in the POSIX specification. - sector - (optional) a sector number that the request must overlap in order to match this rule once - (optional, default "off") only execute this action on the first matching request +Error injection rules support the following additional attributes: + + errno - the numeric errno value to return when a request matches this rule. + The errno values depend on the host since the numeric values are not + standarized in the POSIX specification. + immediately - (optional, default "off") return a NULL BlockAIOCB pointer and fail without an errno instead. This exercises the code path where BlockAIOCB fails and the caller's BlockCompletionFunc is not invoked. +Delay rules support the following additional attribute: + + latency - the delay to add to an I/O request, in microseconds. + Events ------ Block drivers provide information about the type of I/O request they are about diff --git a/qapi/block-core.json b/qapi/block-core.json index d4fe710..72f7861 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3057,6 +3057,34 @@ '*immediately': 'bool' } } ## +# @BlkdebugDelayOptions: +# +# Describes a single latency injection for blkdebug. +# +# @event: trigger event +# +# @state: the state identifier blkdebug needs to be in to +# actually trigger the event; defaults to "any" +# +# @latency: The delay to add to an I/O, in microseconds. +# +# @sector: specifies the sector index which has to be affected +# in order to actually trigger the event; defaults to "any +# sector" +# +# @once: disables further events after this one has been +# triggered; defaults to false +# +# Since: 3.1 +## +{ 'struct': 'BlkdebugDelayOptions', + 'data': { 'event': 'BlkdebugEvent', + '*state': 'int', + '*latency': 'int', + '*sector': 'int', + '*once': 'bool' } } + +## # @BlkdebugSetStateOptions: # # Describes a single state-change event for blkdebug. @@ -3115,6 +3143,8 @@ # # @inject-error: array of error injection descriptions # +# @inject-delay: array of delay injection descriptions +# # @set-state: array of state-change descriptions # # Since: 2.9 @@ -3126,6 +3156,7 @@ '*opt-write-zero': 'int32', '*max-write-zero': 'int32', '*opt-discard': 'int32', '*max-discard': 'int32', '*inject-error': ['BlkdebugInjectErrorOptions'], + '*inject-delay': ['BlkdebugDelayOptions'], '*set-state': ['BlkdebugSetStateOptions'] } } ## diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071 index 48b4955..976f747 100755 --- a/tests/qemu-iotests/071 +++ b/tests/qemu-iotests/071 @@ -100,6 +100,69 @@ $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.inject-error.event -c 'read -P 42 0x38000 512' echo +echo "=== Testing blkdebug latency through filename ===" +echo + +$QEMU_IO -c "open -o file.driver=blkdebug,file.inject-delay.event=write_aio,file.inject-delay.latency=10000 $TEST_IMG" \ + -c 'aio_write -P 42 0x28000 512' \ + -c 'aio_read -P 42 0x38000 512' \ + | _filter_qemu_io + +echo +echo "=== Testing blkdebug latency through file blockref ===" +echo + +$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.inject-delay.event=write_aio,file.inject-delay.latency=10000,file.image.filename=$TEST_IMG" \ + -c 'aio_write -P 42 0x28000 512' \ + -c 'aio_read -P 42 0x38000 512' \ + | _filter_qemu_io + +# Using QMP is synchronous by default, so even though we would +# expect reordering due to using the aio_* commands, they are +# not. The purpose of this test is to verify that the driver +# can be setup via QMP, and IO can complete. See the qemu-io +# test above to prove delay functionality +echo +echo "=== Testing blkdebug on existing block device ===" +echo + +run_qemu <