From patchwork Mon Jan 20 14:18:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 11342299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1EAB17EA for ; Mon, 20 Jan 2020 14:23:10 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B870422314 for ; Mon, 20 Jan 2020 14:23:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MTwT61vi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B870422314 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:37414 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1itXxN-0002l3-9i for patchwork-qemu-devel@patchwork.kernel.org; Mon, 20 Jan 2020 09:23:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39961) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1itXuk-0007Q5-Kt for qemu-devel@nongnu.org; Mon, 20 Jan 2020 09:20:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1itXug-0006Oz-KJ for qemu-devel@nongnu.org; Mon, 20 Jan 2020 09:20:26 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:58221 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1itXug-0006Oe-FR for qemu-devel@nongnu.org; Mon, 20 Jan 2020 09:20:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1579530022; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=viQq/3zVgkeAg3g8w0rxue+f/NvZPq/Ufyv3yRiAgzc=; b=MTwT61vi0xlIjcmnatGfKhyxeKg6AVHVU3Ge2I80DhTbEks/YxesoEGmUcUiYrNsRTxES6 DuA4wFatQLid+MqJH9bBpKF5ASJflEhpLgq5tST7OP6okKR6qsOkuTvUx+YQ6wvVZrcpv9 yzEah5CwGstkaDDwrPymYU89GNJgTzA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-310-1T2mc5ZRPruIlGaDz3WKBA-1; Mon, 20 Jan 2020 09:20:20 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 436FB8017CC; Mon, 20 Jan 2020 14:20:19 +0000 (UTC) Received: from localhost (ovpn-117-223.ams2.redhat.com [10.36.117.223]) by smtp.corp.redhat.com (Postfix) with ESMTP id A0AC460C05; Mon, 20 Jan 2020 14:20:11 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v5 08/15] block/file-posix.c: extend to use io_uring Date: Mon, 20 Jan 2020 14:18:51 +0000 Message-Id: <20200120141858.587874-9-stefanha@redhat.com> In-Reply-To: <20200120141858.587874-1-stefanha@redhat.com> References: <20200120141858.587874-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: 1T2mc5ZRPruIlGaDz3WKBA-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: oleksandr@redhat.com, Kevin Wolf , Maxim Levitsky , qemu-block@nongnu.org, Julia Suvorova , Markus Armbruster , Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Fam Zheng , Aarushi Mehta , Stefano Garzarella Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Aarushi Mehta Signed-off-by: Aarushi Mehta Reviewed-by: Maxim Levitsky Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- v5: * Add back BDRV_O_IO_URING check that was dropped accidentally in v3 [Kevin] --- block/file-posix.c | 98 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 79 insertions(+), 19 deletions(-) diff --git a/block/file-posix.c b/block/file-posix.c index 1b805bd938..f4e6e4d33e 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -156,6 +156,7 @@ typedef struct BDRVRawState { bool has_write_zeroes:1; bool discard_zeroes:1; bool use_linux_aio:1; + bool use_linux_io_uring:1; bool page_cache_inconsistent:1; bool has_fallocate; bool needs_alignment; @@ -444,7 +445,7 @@ static QemuOptsList raw_runtime_opts = { { .name = "aio", .type = QEMU_OPT_STRING, - .help = "host AIO implementation (threads, native)", + .help = "host AIO implementation (threads, native, io_uring)", }, { .name = "locking", @@ -503,9 +504,16 @@ static int raw_open_common(BlockDriverState *bs, QDict *options, goto fail; } - aio_default = (bdrv_flags & BDRV_O_NATIVE_AIO) - ? BLOCKDEV_AIO_OPTIONS_NATIVE - : BLOCKDEV_AIO_OPTIONS_THREADS; + if (bdrv_flags & BDRV_O_NATIVE_AIO) { + aio_default = BLOCKDEV_AIO_OPTIONS_NATIVE; +#ifdef CONFIG_LINUX_IO_URING + } else if (bdrv_flags & BDRV_O_IO_URING) { + aio_default = BLOCKDEV_AIO_OPTIONS_IO_URING; +#endif + } else { + aio_default = BLOCKDEV_AIO_OPTIONS_THREADS; + } + aio = qapi_enum_parse(&BlockdevAioOptions_lookup, qemu_opt_get(opts, "aio"), aio_default, &local_err); @@ -514,7 +522,11 @@ static int raw_open_common(BlockDriverState *bs, QDict *options, ret = -EINVAL; goto fail; } + s->use_linux_aio = (aio == BLOCKDEV_AIO_OPTIONS_NATIVE); +#ifdef CONFIG_LINUX_IO_URING + s->use_linux_io_uring = (aio == BLOCKDEV_AIO_OPTIONS_IO_URING); +#endif locking = qapi_enum_parse(&OnOffAuto_lookup, qemu_opt_get(opts, "locking"), @@ -600,6 +612,22 @@ static int raw_open_common(BlockDriverState *bs, QDict *options, } #endif /* !defined(CONFIG_LINUX_AIO) */ +#ifdef CONFIG_LINUX_IO_URING + if (s->use_linux_io_uring) { + if (!aio_setup_linux_io_uring(bdrv_get_aio_context(bs), errp)) { + error_prepend(errp, "Unable to use io_uring: "); + goto fail; + } + } +#else + if (s->use_linux_io_uring) { + error_setg(errp, "aio=io_uring was specified, but is not supported " + "in this build."); + ret = -EINVAL; + goto fail; + } +#endif /* !defined(CONFIG_LINUX_IO_URING) */ + s->has_discard = true; s->has_write_zeroes = true; if ((bs->open_flags & BDRV_O_NOCACHE) != 0) { @@ -1877,21 +1905,25 @@ static int coroutine_fn raw_co_prw(BlockDriverState *bs, uint64_t offset, return -EIO; /* - * Check if the underlying device requires requests to be aligned, - * and if the request we are trying to submit is aligned or not. - * If this is the case tell the low-level driver that it needs - * to copy the buffer. + * When using O_DIRECT, the request must be aligned to be able to use + * either libaio or io_uring interface. If not fail back to regular thread + * pool read/write code which emulates this for us if we + * set QEMU_AIO_MISALIGNED. */ - if (s->needs_alignment) { - if (!bdrv_qiov_is_aligned(bs, qiov)) { - type |= QEMU_AIO_MISALIGNED; + if (s->needs_alignment && !bdrv_qiov_is_aligned(bs, qiov)) { + type |= QEMU_AIO_MISALIGNED; +#ifdef CONFIG_LINUX_IO_URING + } else if (s->use_linux_io_uring) { + LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs)); + assert(qiov->size == bytes); + return luring_co_submit(bs, aio, s->fd, offset, qiov, type); +#endif #ifdef CONFIG_LINUX_AIO - } else if (s->use_linux_aio) { - LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs)); - assert(qiov->size == bytes); - return laio_co_submit(bs, aio, s->fd, offset, qiov, type); + } else if (s->use_linux_aio) { + LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs)); + assert(qiov->size == bytes); + return laio_co_submit(bs, aio, s->fd, offset, qiov, type); #endif - } } acb = (RawPosixAIOData) { @@ -1927,24 +1959,36 @@ static int coroutine_fn raw_co_pwritev(BlockDriverState *bs, uint64_t offset, static void raw_aio_plug(BlockDriverState *bs) { + BDRVRawState __attribute__((unused)) *s = bs->opaque; #ifdef CONFIG_LINUX_AIO - BDRVRawState *s = bs->opaque; if (s->use_linux_aio) { LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs)); laio_io_plug(bs, aio); } #endif +#ifdef CONFIG_LINUX_IO_URING + if (s->use_linux_io_uring) { + LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs)); + luring_io_plug(bs, aio); + } +#endif } static void raw_aio_unplug(BlockDriverState *bs) { + BDRVRawState __attribute__((unused)) *s = bs->opaque; #ifdef CONFIG_LINUX_AIO - BDRVRawState *s = bs->opaque; if (s->use_linux_aio) { LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs)); laio_io_unplug(bs, aio); } #endif +#ifdef CONFIG_LINUX_IO_URING + if (s->use_linux_io_uring) { + LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs)); + luring_io_unplug(bs, aio); + } +#endif } static int raw_co_flush_to_disk(BlockDriverState *bs) @@ -1964,14 +2008,20 @@ static int raw_co_flush_to_disk(BlockDriverState *bs) .aio_type = QEMU_AIO_FLUSH, }; +#ifdef CONFIG_LINUX_IO_URING + if (s->use_linux_io_uring) { + LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs)); + return luring_co_submit(bs, aio, s->fd, 0, NULL, QEMU_AIO_FLUSH); + } +#endif return raw_thread_pool_submit(bs, handle_aiocb_flush, &acb); } static void raw_aio_attach_aio_context(BlockDriverState *bs, AioContext *new_context) { + BDRVRawState __attribute__((unused)) *s = bs->opaque; #ifdef CONFIG_LINUX_AIO - BDRVRawState *s = bs->opaque; if (s->use_linux_aio) { Error *local_err = NULL; if (!aio_setup_linux_aio(new_context, &local_err)) { @@ -1981,6 +2031,16 @@ static void raw_aio_attach_aio_context(BlockDriverState *bs, } } #endif +#ifdef CONFIG_LINUX_IO_URING + if (s->use_linux_io_uring) { + Error *local_err; + if (!aio_setup_linux_io_uring(new_context, &local_err)) { + error_reportf_err(local_err, "Unable to use linux io_uring, " + "falling back to thread pool: "); + s->use_linux_io_uring = false; + } + } +#endif } static void raw_close(BlockDriverState *bs)