From patchwork Wed Jul 7 15:00:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 12362741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D591C07E9C for ; Wed, 7 Jul 2021 15:02:11 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EC07C61C7C for ; Wed, 7 Jul 2021 15:02:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC07C61C7C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:44248 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m193y-0005jG-4g for qemu-devel@archiver.kernel.org; Wed, 07 Jul 2021 11:02:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35528) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m192U-0003gX-Vp for qemu-devel@nongnu.org; Wed, 07 Jul 2021 11:00:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:37298) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m192S-0006Cl-Hx for qemu-devel@nongnu.org; Wed, 07 Jul 2021 11:00:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625670036; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e6+ZiMdDFv8oHA3dnDQn58wlQWvHhmXKtz6vfzKsitY=; b=b/iBPYQCOmBc19YHuQhyLo6NpvCnyWbkKN9UONiOHgq4zwP9TrdjqDp0K4h6ht0juS4Pgx f10lOhC0Paae+N9Yy39gwjIPRlms/DjQqxeyI1G5M6YeWsbiPpWikGk5hEKRlt/2V8+hGn kWvezY2zYxaS4e0se4OptxSqh30DYgA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-382-oN2ru85XNk-FMg-YV9GzWQ-1; Wed, 07 Jul 2021 11:00:34 -0400 X-MC-Unique: oN2ru85XNk-FMg-YV9GzWQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C7A835074B; Wed, 7 Jul 2021 15:00:33 +0000 (UTC) Received: from steredhat.tendawifi.com (ovpn-112-132.ams2.redhat.com [10.36.112.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0C73E5C1D5; Wed, 7 Jul 2021 15:00:30 +0000 (UTC) From: Stefano Garzarella To: qemu-devel@nongnu.org Subject: [PATCH 1/3] iothread: generalize iothread_set_param/iothread_get_param Date: Wed, 7 Jul 2021 17:00:17 +0200 Message-Id: <20210707150019.201442-2-sgarzare@redhat.com> In-Reply-To: <20210707150019.201442-1-sgarzare@redhat.com> References: <20210707150019.201442-1-sgarzare@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=sgarzare@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=sgarzare@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.439, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , =?utf-8?q?Dan?= =?utf-8?q?iel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , qemu-block@nongnu.org, Stefan Weil , Markus Armbruster , Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Eric Blake , "Dr. David Alan Gilbert" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Changes in preparation for next patches where we add a new parameter not related to the poll mechanism. Let's add two new generic functions (iothread_set_param and iothread_get_param) that we use to set and get IOThread parameters. Signed-off-by: Stefano Garzarella --- iothread.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/iothread.c b/iothread.c index 2c5ccd7367..103679a16b 100644 --- a/iothread.c +++ b/iothread.c @@ -213,7 +213,7 @@ static PollParamInfo poll_shrink_info = { "poll-shrink", offsetof(IOThread, poll_shrink), }; -static void iothread_get_poll_param(Object *obj, Visitor *v, +static void iothread_get_param(Object *obj, Visitor *v, const char *name, void *opaque, Error **errp) { IOThread *iothread = IOTHREAD(obj); @@ -223,7 +223,7 @@ static void iothread_get_poll_param(Object *obj, Visitor *v, visit_type_int64(v, name, field, errp); } -static void iothread_set_poll_param(Object *obj, Visitor *v, +static bool iothread_set_param(Object *obj, Visitor *v, const char *name, void *opaque, Error **errp) { IOThread *iothread = IOTHREAD(obj); @@ -232,17 +232,36 @@ static void iothread_set_poll_param(Object *obj, Visitor *v, int64_t value; if (!visit_type_int64(v, name, &value, errp)) { - return; + return false; } if (value < 0) { error_setg(errp, "%s value must be in range [0, %" PRId64 "]", info->name, INT64_MAX); - return; + return false; } *field = value; + return true; +} + +static void iothread_get_poll_param(Object *obj, Visitor *v, + const char *name, void *opaque, Error **errp) +{ + + iothread_get_param(obj, v, name, opaque, errp); +} + +static void iothread_set_poll_param(Object *obj, Visitor *v, + const char *name, void *opaque, Error **errp) +{ + IOThread *iothread = IOTHREAD(obj); + + if (!iothread_set_param(obj, v, name, opaque, errp)) { + return; + } + if (iothread->ctx) { aio_context_set_poll_params(iothread->ctx, iothread->poll_max_ns, From patchwork Wed Jul 7 15:00:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 12362743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03423C07E95 for ; Wed, 7 Jul 2021 15:02:48 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 67A8661CC0 for ; Wed, 7 Jul 2021 15:02:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 67A8661CC0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45376 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m194Y-0006T5-Kr for qemu-devel@archiver.kernel.org; Wed, 07 Jul 2021 11:02:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35600) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m192c-0003xq-CP for qemu-devel@nongnu.org; Wed, 07 Jul 2021 11:00:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:43571) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m192Z-0006FB-9L for qemu-devel@nongnu.org; Wed, 07 Jul 2021 11:00:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625670042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HHTqHtDAYGO8HY1kxDyuIi+mr/c2wOexrvV9Jc95EEA=; b=U6AALbTmIGIFz2V1grPlla+nOwSAownmkNvVWH2UDv6oDvxluydht9SH6AjGcbSCmB3KP1 vvL3kTGvwMQOP2N7qgfu5uNYfU7rXSDdtEbHgUQd6E+vr7JwcXDzqXXahESiNAcB7Jx0Ve QSuWrdc039yB5SW7R7VDchgAS7fA6PY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-479-fgNZvsAVOAS1MMrpLq0f7A-1; Wed, 07 Jul 2021 11:00:41 -0400 X-MC-Unique: fgNZvsAVOAS1MMrpLq0f7A-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1E01F100C664; Wed, 7 Jul 2021 15:00:40 +0000 (UTC) Received: from steredhat.tendawifi.com (ovpn-112-132.ams2.redhat.com [10.36.112.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 26AB45C1D5; Wed, 7 Jul 2021 15:00:33 +0000 (UTC) From: Stefano Garzarella To: qemu-devel@nongnu.org Subject: [PATCH 2/3] iothread: add aio-max-batch parameter Date: Wed, 7 Jul 2021 17:00:18 +0200 Message-Id: <20210707150019.201442-3-sgarzare@redhat.com> In-Reply-To: <20210707150019.201442-1-sgarzare@redhat.com> References: <20210707150019.201442-1-sgarzare@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=sgarzare@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=sgarzare@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.439, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , =?utf-8?q?Dan?= =?utf-8?q?iel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , qemu-block@nongnu.org, Stefan Weil , Markus Armbruster , Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Eric Blake , "Dr. David Alan Gilbert" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The `aio-max-batch` parameter will be propagated to AIO engines and it will be used to control the maximum number of queued requests. When there are in queue a number of requests equal to `aio-max-batch`, the engine invokes the system call to forward the requests to the kernel. This parameter allows us to control the maximum batch size to reduce the latency that requests might accumulate while queued in the AIO engine queue. If `aio-max-batch` is equal to 0 (default value), the AIO engine will use its default maximum batch size value. Signed-off-by: Stefano Garzarella --- qapi/misc.json | 6 ++++- qapi/qom.json | 7 ++++- include/block/aio.h | 12 +++++++++ include/sysemu/iothread.h | 3 +++ iothread.c | 55 +++++++++++++++++++++++++++++++++++---- monitor/hmp-cmds.c | 2 ++ util/aio-posix.c | 12 +++++++++ util/aio-win32.c | 5 ++++ util/async.c | 2 ++ qemu-options.hx | 8 ++++-- 10 files changed, 103 insertions(+), 9 deletions(-) diff --git a/qapi/misc.json b/qapi/misc.json index 156f98203e..f64bb69f74 100644 --- a/qapi/misc.json +++ b/qapi/misc.json @@ -86,6 +86,9 @@ # @poll-shrink: how many ns will be removed from polling time, 0 means that # it's not configured (since 2.9) # +# @aio-max-batch: maximum number of requests in a bacth for the AIO engine, +# 0 means that the engine will use its default (since 6.1) +# # Since: 2.0 ## { 'struct': 'IOThreadInfo', @@ -93,7 +96,8 @@ 'thread-id': 'int', 'poll-max-ns': 'int', 'poll-grow': 'int', - 'poll-shrink': 'int' } } + 'poll-shrink': 'int', + 'aio-max-batch': 'int' } } ## # @query-iothreads: diff --git a/qapi/qom.json b/qapi/qom.json index 652be317b8..23fd586614 100644 --- a/qapi/qom.json +++ b/qapi/qom.json @@ -516,12 +516,17 @@ # algorithm detects it is spending too long polling without # encountering events. 0 selects a default behaviour (default: 0) # +# @aio-max-batch: maximum number of requests in a bacth for the AIO engine, +# 0 means that the engine will use its default +# (default:0, since 6.1) +# # Since: 2.0 ## { 'struct': 'IothreadProperties', 'data': { '*poll-max-ns': 'int', '*poll-grow': 'int', - '*poll-shrink': 'int' } } + '*poll-shrink': 'int', + '*aio-max-batch': 'int' } } ## # @MemoryBackendProperties: diff --git a/include/block/aio.h b/include/block/aio.h index 10fcae1515..5b7983348f 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -232,6 +232,9 @@ struct AioContext { int64_t poll_grow; /* polling time growth factor */ int64_t poll_shrink; /* polling time shrink factor */ + /* AIO engine parameters */ + int64_t aio_max_batch; /* maximum number of requests in a batch */ + /* * List of handlers participating in userspace polling. Protected by * ctx->list_lock. Iterated and modified mostly by the event loop thread @@ -730,4 +733,13 @@ void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns, int64_t grow, int64_t shrink, Error **errp); +/** + * aio_context_set_aio_params: + * @ctx: the aio context + * @max_batch: maximum number of requests in a bacth, 0 means that the + * engine will use its default + */ +void aio_context_set_aio_params(AioContext *ctx, int64_t max_batch, + Error **errp); + #endif diff --git a/include/sysemu/iothread.h b/include/sysemu/iothread.h index f177142f16..7f714bd136 100644 --- a/include/sysemu/iothread.h +++ b/include/sysemu/iothread.h @@ -37,6 +37,9 @@ struct IOThread { int64_t poll_max_ns; int64_t poll_grow; int64_t poll_shrink; + + /* AioContext AIO engine parameters */ + int64_t aio_max_batch; }; typedef struct IOThread IOThread; diff --git a/iothread.c b/iothread.c index 103679a16b..ddbbde61f7 100644 --- a/iothread.c +++ b/iothread.c @@ -152,6 +152,24 @@ static void iothread_init_gcontext(IOThread *iothread) iothread->main_loop = g_main_loop_new(iothread->worker_context, TRUE); } +static void iothread_set_aio_context_params(IOThread *iothread, Error **errp) +{ + ERRP_GUARD(); + + aio_context_set_poll_params(iothread->ctx, + iothread->poll_max_ns, + iothread->poll_grow, + iothread->poll_shrink, + errp); + if (*errp) { + return; + } + + aio_context_set_aio_params(iothread->ctx, + iothread->aio_max_batch, + errp); +} + static void iothread_complete(UserCreatable *obj, Error **errp) { Error *local_error = NULL; @@ -171,11 +189,7 @@ static void iothread_complete(UserCreatable *obj, Error **errp) */ iothread_init_gcontext(iothread); - aio_context_set_poll_params(iothread->ctx, - iothread->poll_max_ns, - iothread->poll_grow, - iothread->poll_shrink, - &local_error); + iothread_set_aio_context_params(iothread, &local_error); if (local_error) { error_propagate(errp, local_error); aio_context_unref(iothread->ctx); @@ -212,6 +226,9 @@ static PollParamInfo poll_grow_info = { static PollParamInfo poll_shrink_info = { "poll-shrink", offsetof(IOThread, poll_shrink), }; +static PollParamInfo aio_max_batch_info = { + "aio-max-batch", offsetof(IOThread, aio_max_batch), +}; static void iothread_get_param(Object *obj, Visitor *v, const char *name, void *opaque, Error **errp) @@ -271,6 +288,29 @@ static void iothread_set_poll_param(Object *obj, Visitor *v, } } +static void iothread_get_aio_param(Object *obj, Visitor *v, + const char *name, void *opaque, Error **errp) +{ + + iothread_get_param(obj, v, name, opaque, errp); +} + +static void iothread_set_aio_param(Object *obj, Visitor *v, + const char *name, void *opaque, Error **errp) +{ + IOThread *iothread = IOTHREAD(obj); + + if (!iothread_set_param(obj, v, name, opaque, errp)) { + return; + } + + if (iothread->ctx) { + aio_context_set_aio_params(iothread->ctx, + iothread->aio_max_batch, + errp); + } +} + static void iothread_class_init(ObjectClass *klass, void *class_data) { UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass); @@ -288,6 +328,10 @@ static void iothread_class_init(ObjectClass *klass, void *class_data) iothread_get_poll_param, iothread_set_poll_param, NULL, &poll_shrink_info); + object_class_property_add(klass, "aio-max-batch", "int", + iothread_get_aio_param, + iothread_set_aio_param, + NULL, &aio_max_batch_info); } static const TypeInfo iothread_info = { @@ -337,6 +381,7 @@ static int query_one_iothread(Object *object, void *opaque) info->poll_max_ns = iothread->poll_max_ns; info->poll_grow = iothread->poll_grow; info->poll_shrink = iothread->poll_shrink; + info->aio_max_batch = iothread->aio_max_batch; QAPI_LIST_APPEND(*tail, info); return 0; diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 0942027208..e00255f7ee 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -1893,6 +1893,8 @@ void hmp_info_iothreads(Monitor *mon, const QDict *qdict) monitor_printf(mon, " poll-max-ns=%" PRId64 "\n", value->poll_max_ns); monitor_printf(mon, " poll-grow=%" PRId64 "\n", value->poll_grow); monitor_printf(mon, " poll-shrink=%" PRId64 "\n", value->poll_shrink); + monitor_printf(mon, " aio-max-batch=%" PRId64 "\n", + value->aio_max_batch); } qapi_free_IOThreadInfoList(info_list); diff --git a/util/aio-posix.c b/util/aio-posix.c index 30f5354b1e..2b86777e91 100644 --- a/util/aio-posix.c +++ b/util/aio-posix.c @@ -716,3 +716,15 @@ void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns, aio_notify(ctx); } + +void aio_context_set_aio_params(AioContext *ctx, int64_t max_batch, + Error **errp) +{ + /* + * No thread synchronization here, it doesn't matter if an incorrect value + * is used once. + */ + ctx->aio_max_batch = max_batch; + + aio_notify(ctx); +} diff --git a/util/aio-win32.c b/util/aio-win32.c index 168717b51b..d5b09a1193 100644 --- a/util/aio-win32.c +++ b/util/aio-win32.c @@ -440,3 +440,8 @@ void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns, error_setg(errp, "AioContext polling is not implemented on Windows"); } } + +void aio_context_set_aio_params(AioContext *ctx, int64_t max_batch, + Error **errp) +{ +} diff --git a/util/async.c b/util/async.c index 5d9b7cc1eb..f3510ea4f9 100644 --- a/util/async.c +++ b/util/async.c @@ -537,6 +537,8 @@ AioContext *aio_context_new(Error **errp) ctx->poll_grow = 0; ctx->poll_shrink = 0; + ctx->aio_max_batch = 0; + return ctx; fail: g_source_destroy(&ctx->source); diff --git a/qemu-options.hx b/qemu-options.hx index 8965dabc83..43c03a8e3c 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -5255,7 +5255,7 @@ SRST CN=laptop.example.com,O=Example Home,L=London,ST=London,C=GB - ``-object iothread,id=id,poll-max-ns=poll-max-ns,poll-grow=poll-grow,poll-shrink=poll-shrink`` + ``-object iothread,id=id,poll-max-ns=poll-max-ns,poll-grow=poll-grow,poll-shrink=poll-shrink,aio-max-batch=aio-max-batch`` Creates a dedicated event loop thread that devices can be assigned to. This is known as an IOThread. By default device emulation happens in vCPU threads or the main event loop thread. @@ -5291,7 +5291,11 @@ SRST the polling time when the algorithm detects it is spending too long polling without encountering events. - The polling parameters can be modified at run-time using the + The ``aio-max-batch`` parameter is the maximum number of requests + in a batch for the AIO engine, 0 means that the engine will use + its default. + + The IOThread parameters can be modified at run-time using the ``qom-set`` command (where ``iothread1`` is the IOThread's ``id``): From patchwork Wed Jul 7 15:00:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 12362747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85117C07E95 for ; Wed, 7 Jul 2021 15:04:21 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 512F861CC2 for ; Wed, 7 Jul 2021 15:04:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 512F861CC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52608 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1964-0002ps-Fb for qemu-devel@archiver.kernel.org; Wed, 07 Jul 2021 11:04:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35642) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m192f-00047a-RE for qemu-devel@nongnu.org; Wed, 07 Jul 2021 11:00:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:32019) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m192e-0006G7-B4 for qemu-devel@nongnu.org; Wed, 07 Jul 2021 11:00:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625670047; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xv6E6Wt35ZNlndkXnnhHZQ4H/tCu5PIDlmg+Hy6dgJk=; b=WtoSNtzyBneDB0uhQws7j9pBjx6Ifv/WMhLRmg1P0JT3S95Ezbx0hLQ3wNiaL6uW/sfhoV 3N7XwzPe0/qHy9zBGuDTdDPKP0c4rssNl22EKyRJQF7CxbXNqNJbWN12wpdV5MmpsNkd/c ndaSY74qFZdVqnrvBahWAuF0hxe1kmI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-236-U2ccUlGoP3mDCzoET_id3Q-1; Wed, 07 Jul 2021 11:00:44 -0400 X-MC-Unique: U2ccUlGoP3mDCzoET_id3Q-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 797A91934119; Wed, 7 Jul 2021 15:00:43 +0000 (UTC) Received: from steredhat.tendawifi.com (ovpn-112-132.ams2.redhat.com [10.36.112.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 720FA5FC07; Wed, 7 Jul 2021 15:00:40 +0000 (UTC) From: Stefano Garzarella To: qemu-devel@nongnu.org Subject: [PATCH 3/3] linux-aio: limit the batch size using `aio-max-batch` parameter Date: Wed, 7 Jul 2021 17:00:19 +0200 Message-Id: <20210707150019.201442-4-sgarzare@redhat.com> In-Reply-To: <20210707150019.201442-1-sgarzare@redhat.com> References: <20210707150019.201442-1-sgarzare@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=sgarzare@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=sgarzare@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.439, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , =?utf-8?q?Dan?= =?utf-8?q?iel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , qemu-block@nongnu.org, Stefan Weil , Markus Armbruster , Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Eric Blake , "Dr. David Alan Gilbert" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When there are multiple queues attached to the same AIO context, some requests may experience high latency, since in the worst case the AIO engine queue is only flushed when it is full (MAX_EVENTS) or there are no more queues plugged. Commit 2558cb8dd4 ("linux-aio: increasing MAX_EVENTS to a larger hardcoded value") changed MAX_EVENTS from 128 to 1024, to increase the number of in-flight requests. But this change also increased the potential maximum batch to 1024 elements. When there is a single queue attached to the AIO context, the issue is mitigated from laio_io_unplug() that will flush the queue every time is invoked since there can't be others queue plugged. Let's use the new `aio-max-batch` IOThread parameter to mitigate this issue, limiting the number of requests in a batch. We also define a default value (32): this value is obtained running some benchmarks and it represents a good tradeoff between the latency increase while a request is queued and the cost of the io_submit(2) system call. Signed-off-by: Stefano Garzarella --- block/linux-aio.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index 3c0527c2bf..8a7bb136fc 100644 --- a/block/linux-aio.c +++ b/block/linux-aio.c @@ -28,6 +28,9 @@ */ #define MAX_EVENTS 1024 +/* Maximum number of requests in a batch. (default value) */ +#define DEFAULT_MAX_BATCH 32 + struct qemu_laiocb { Coroutine *co; LinuxAioState *ctx; @@ -351,6 +354,7 @@ static int laio_do_submit(int fd, struct qemu_laiocb *laiocb, off_t offset, LinuxAioState *s = laiocb->ctx; struct iocb *iocbs = &laiocb->iocb; QEMUIOVector *qiov = laiocb->qiov; + int64_t max_batch = s->aio_context->aio_max_batch ?: DEFAULT_MAX_BATCH; switch (type) { case QEMU_AIO_WRITE: @@ -371,7 +375,7 @@ static int laio_do_submit(int fd, struct qemu_laiocb *laiocb, off_t offset, s->io_q.in_queue++; if (!s->io_q.blocked && (!s->io_q.plugged || - s->io_q.in_flight + s->io_q.in_queue >= MAX_EVENTS)) { + s->io_q.in_queue >= max_batch)) { ioq_submit(s); }