From patchwork Wed Aug 14 10:45:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13763315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30C58C3DA4A for ; Wed, 14 Aug 2024 10:45:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B81706B0083; Wed, 14 Aug 2024 06:45:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B32306B0085; Wed, 14 Aug 2024 06:45:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 984AF6B0088; Wed, 14 Aug 2024 06:45:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7EA566B0083 for ; Wed, 14 Aug 2024 06:45:26 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3FDD2160E11 for ; Wed, 14 Aug 2024 10:45:26 +0000 (UTC) X-FDA: 82450519452.30.1404088 Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) by imf05.hostedemail.com (Postfix) with ESMTP id 53F38100004 for ; Wed, 14 Aug 2024 10:45:24 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=W6YM2Ybt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723632244; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oGCUwVa5Kyp0Pg2+kUxIsl/LPJjPrI+WqiYx+qtKk3Q=; b=nUupxzzttWXQuT1oTiD0c5PbkSbd/WVocmP3EFM+ZN370u1PxuJ7TeeZf6r6AJiDAhg5RB FefHxYEs6FHEgrQHBrPMfMR60T/qLs9KoJlA3XcsU+mAf59NLF6nO0/Soeafl+iisqRAzo 6huRhQ4WkUoXChIwGVLVd1epcXdyPbw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723632244; a=rsa-sha256; cv=none; b=aXSKy2r3tR9ZRqL/icR6aK4q+EKWXfVN2HB4XZjlBtPuz77zFlN8i/oUd6cwlLpyxL/033 DLbW3YbNR5Wkq3P2c9/FMwt0c2CofByMAKLpcg30iuRN+asTrBme3NtX1wAwx/mCroDB5i 4/X6LVB+ScbO0KojewHTiNQ89YeAzys= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=W6YM2Ybt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com Received: by mail-lf1-f51.google.com with SMTP id 2adb3069b0e04-52f01993090so8796282e87.2 for ; Wed, 14 Aug 2024 03:45:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723632323; x=1724237123; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oGCUwVa5Kyp0Pg2+kUxIsl/LPJjPrI+WqiYx+qtKk3Q=; b=W6YM2Ybt7xh5hSff/dEzZS9Eqaj+VaaL4za6zDVh+Q97v2gNb4bNAmL4IF14K2GgpE QoPikXYeREIKyd2sBS2Ia1vn8EMeo2+1KV/Yat4iU8YG2Pbt9DXQZqhYdyWVZ5j7fhfk O3xMn603zAMH0lWfLGy+6xhXHS+vW+O3NI+w05ji16XA9F78OgzpuOYoOPOR6hNxHaMs LNBSNvnz39QRf1UnEq6scwNxRi7ANxsOoraU8wISeiSyDC3OHyGvvY/mJf3t/PtS9l6P piWCTFpVRRLAHgpbyivzv58kXdx2i3xcGEMdZt4KxC9VKYwWTp9NRUTErYMppV7uAA9O y56A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723632323; x=1724237123; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oGCUwVa5Kyp0Pg2+kUxIsl/LPJjPrI+WqiYx+qtKk3Q=; b=dn3RHDEYJ3uKg4HGhCsSwyoFWPa2uZQ1lr8dPfiKO2hM8bKBDjKr8iLeLTcHNNeMB+ lrMbR1orw2N2hESP0/5KIqLiKcORyhdLsxy7NPxjq0sm7UybKz023nt3OR+qvp9Xa8G9 fveuFrJfQhwfHuDMEvVVfQ/175hCeogM42Bl+Ok6/DsZ9woUm8CJb4w9Ltd+GF6C/zib wxJNemJNMb86UoLEGOglRW7P9xh20XmqBeoq3k85KLZecmqLb9Hb7YkijJOcJGukFnyn rGa7ZLHZftvsq3T6kxmzHVch+Ga3yPr/hTgtc2y8QCcb07ZPZZngbFofcHvfVLHsd7uY tFBw== X-Forwarded-Encrypted: i=1; AJvYcCUvbnQ2H+mxZyoGyutCxw/xxJFF4VTIU1EuYV5DXWwQIU/0fWM47qGMyvNTZR6gSx8ouH+5yYJQuAtd8/7bmYpmXD0= X-Gm-Message-State: AOJu0YyAjGaG9DzBjUVB7+vBmQKKG8IcznfQXS8DzM3CJ8e5G4IlvKGQ FFVM7yI56lEkIUTZgZi9abrNhszonSlvIXcOaebqHjIalwWq13Sv X-Google-Smtp-Source: AGHT+IHhdpdAhXAFGJZHpDxWh8B1Ms2Ng2AgCkdonuHrS94jDeJvM4fNY2m5pVGUUhNgCJJpgyCaZQ== X-Received: by 2002:a05:6512:1591:b0:52e:be50:9c66 with SMTP id 2adb3069b0e04-532edbbe548mr1283237e87.53.1723632322432; Wed, 14 Aug 2024 03:45:22 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.132.251]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a80f418692asm157212766b.224.2024.08.14.03.45.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Aug 2024 03:45:22 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [RFC 1/5] io_uring/cmd: expose iowq to cmds Date: Wed, 14 Aug 2024 11:45:50 +0100 Message-ID: <0e94fb94c3d627dbd5c210d00bd6cc3e644d00ff.1723601134.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 53F38100004 X-Stat-Signature: bcpfdpdbsinz7t7aswg41w7iwbcme3a3 X-Rspam-User: X-HE-Tag: 1723632324-873147 X-HE-Meta: U2FsdGVkX18+36dz8ofGHue5bm5I9aA0h+b2EkFI2YmRhl4qrSgUYp1W4Alxf5oYxEif8GWTF/1763lkq0/hkoaxV7BADf1SGlVp8nMcEaoa7tMT5QhRxHXzOa7MGvO5lU6WEUTDalXttBuMYdXIAoz/qD20QulaW04MwJfeqElpG09ti9mCF3v3g1RfFBMYgPXlA6TE5PeQUpBZ0cKKJSg5fuXMezKFuME9HHZD8ZqSmydcUvceLQRJ/Iv8Qacystggx8u9xBTN5qEA1NsBBMFeb9VT0tJ9B0QguYDeBzGLWNT6vlFzoiyyJ1W5JgWQBENYh4DPFsLJkWShVxEYQ8A7kFzMK03UfoCwMv8s/x75D5ShnH9zjwDbVZhdvunxPpYl5t4OYLr8PjOzGLgDzkTte58YquX8dI74dCynxiTeESEy800w8ozHBR9IJ42oWI96sk9y0jQeniyT7qztWyifHdbOtWtmfx7+DA3i5xBVAE4N3L6AsLWIin3UpIll8rENLMR4a6KjyNJeOmIeJnG4555QonVf4irww4ZwWCBFyd+UA5P4quMdxjTlCWirJNfj2MjV6HKXmPIpnYEocX5I7Wp2m14yXi2vs12PQXO2zYeMekjD+myj74ZtAr4BfVhkDFBo5z7euc4C4c5P6cO8GNX7ZF0mF1zafOloF0OyfWq1tvV0XBKthuRPTQgqSiAhXLPgPc47MxBNKU12ogPY8ZRIVy00VvlaftBZRcQQOjt3mmFK9PM04XNVZBsnaNAVvF6uZlZxXaE76Vwd2EW3qi39Ms2cAqz7JqE4oH7Kw9pGbsJ/9U6iUHkVU851JKJZZ+u77lkc0TqigTYuoMLurZWrDz5uAGoM3PLcONn6LKJX/IL1uKkBfj2xMsw7ZpWzybm45il2IpG2nGSD36HkztdG++B1p5vP68j8+UUmQN30XaJTaAnnqE/3F2rOQDEpCAPLptShu97ytj4 JZG/a3YB bWJGP+V9/aPp9DIv1lK65KbVefDjE0K63XAB4+oqG6i75SrwWnIxDigvTT8hiJiTT9kHMQygBMDYOP3YgajSJGbpS9CxITdMlqyBwm3dh5aCXlTXRmYkeVTzwBluXj8J0qReZTAmi67fD8Vy2gnwwG94CcW6n6fwbQ3P+LHbGnhpFCnPqeaMHoZOlIeI3R645vcHnzivdVq1ug9PkSKXgvGq+B3QyfniOtgAjtp16BAvcYtYy2b5xrOkgl8Mz2EE6ellblK2/3SxSR0jXbqYt+8UCvSRTExc9vAuiTj2g4/ctg2TmVkb5e7nZs0TZpZjO41NGST7uixexdJJodua/O2lN+XVSdMhCBV0wC11/5nxfeAHcYSUEklHUtDmMxVgLnb8qE0k5bJEa3ccO7xL5C4FLBE1dM1Cm04sWw/Ygmy9K1pKtlQ2Xgcy9orQhP6fp5mS3GHZMxRBLkwWg82obHNhjg+GUcaeKqS0TcQPZCWbviPkRzzkZp0oEHg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When an io_uring request needs blocking context we offload it to the io_uring's thread pool called io-wq. We can get there off ->uring_cmd by returning -EAGAIN, but there is no straightforward way of doing that from an asynchronous callback. Add a helper that would transfer a command to a blocking context. Note, we do an extra hop via task_work before io_queue_iowq(), that's a limitation of io_uring infra we have that can likely be lifted later if that would ever become a problem. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 6 ++++++ io_uring/io_uring.c | 11 +++++++++++ io_uring/io_uring.h | 1 + io_uring/uring_cmd.c | 7 +++++++ 4 files changed, 25 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 447fbfd32215..86ceb3383e49 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -48,6 +48,9 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); +/* Execute the request from a blocking context */ +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd); + #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd) @@ -67,6 +70,9 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } +static inline void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ +} #endif /* diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 3942db160f18..da819018088d 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -533,6 +533,17 @@ static void io_queue_iowq(struct io_kiocb *req) io_queue_linked_timeout(link); } +static void io_req_queue_iowq_tw(struct io_kiocb *req, struct io_tw_state *ts) +{ + io_queue_iowq(req); +} + +void io_req_queue_iowq(struct io_kiocb *req) +{ + req->io_task_work.func = io_req_queue_iowq_tw; + io_req_task_work_add(req); +} + static __cold void io_queue_deferred(struct io_ring_ctx *ctx) { while (!list_empty(&ctx->defer_list)) { diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index c2acf6180845..2896587a5221 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -90,6 +90,7 @@ int io_uring_alloc_task_context(struct task_struct *task, int io_ring_add_registered_file(struct io_uring_task *tctx, struct file *file, int start, int end); +void io_req_queue_iowq(struct io_kiocb *req); int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts); int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8391c7c7c1ec..39c3c816ec78 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -277,6 +277,13 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, } EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ + struct io_kiocb *req = cmd_to_io_kiocb(ioucmd); + + io_req_queue_iowq(req); +} + static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd, unsigned int issue_flags)