From patchwork Fri Sep 6 22:57:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13794831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC381E6FE4B for ; Fri, 6 Sep 2024 22:56:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6EFA46B0088; Fri, 6 Sep 2024 18:56:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A0BE6B0089; Fri, 6 Sep 2024 18:56:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51A7B6B008A; Fri, 6 Sep 2024 18:56:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1AFB86B0088 for ; Fri, 6 Sep 2024 18:56:59 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B1BD712125B for ; Fri, 6 Sep 2024 22:56:58 +0000 (UTC) X-FDA: 82535825316.09.7FA3899 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) by imf23.hostedemail.com (Postfix) with ESMTP id CE49314000C for ; Fri, 6 Sep 2024 22:56:56 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RRjdYXje; spf=pass (imf23.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725663286; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=XJUQTCjMsnqkF4XJBgKWfj76h9NgdidLqvuF52YMg8cy3QWnAGONp61GbO+aa96exbderF vWTCeRXhc9RIhIN+SqA3t3RNRSTmQ0+qnOCfWFYVfnwaPrddpdZYGOTQy2O8Z7JqMNJ0A+ +qIX/nW2V/W9eTKPzXtysbCBIBVkSoQ= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RRjdYXje; spf=pass (imf23.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725663286; a=rsa-sha256; cv=none; b=I/aQJIuJ/RKjSGVlv2eyd/sOmaUO/Z9K/DMnPlDkKWiNg0zIpNUMale2GOZiQnxFJkevRJ 7ItBzGo55LcdA7P5/feBmVB5AwkLgkX8Gj8M4iFkycULR+gghKbovw5jK80P6X8n0h1JeD v3FwSKJqvQsvuPRVViO0JfEDwxCxsuw= Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-a8a789c4fc5so348860766b.0 for ; Fri, 06 Sep 2024 15:56:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725663415; x=1726268215; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=RRjdYXje0r5Wzc+v+i94lD8njLJBMv8P02QMYYf0DddZPIfClMWSGAf4eficwtIVGc NXiLcur/OmkEn/ueFkStDkIBsCchy3+6bb85/bxR4pnQ6BLbn0t3r/tDRdipMLyVnFpA uGzV6DVIBpXDQXCk9FENM93pQnxQXZf6sdNp15U4/Y9pc/KOdgaNtVm0iZEKEDhQrYjL y/DesX0P3dZiX4SNAok0jhELrgiXgpA+vuC01Fh238X6fe9ezDXmmElLgpZErmXcjt1j 0fNxN/1004WZ48BkNs3qqc0rWUZdntvMBTSb9yDWCEg/N/6EL7xjfQkGJouXMVoRIvZJ IEcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725663415; x=1726268215; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=ELlQg/5766RdQ/OLuyeFxbMvK55aH3MyZ4EEKJbtkt4BerCKDWGC86lgw5k33FGsF2 en+YV5pnOi3iGL3VADnc363tQ10I2HeMLPpQ528hpLPMJ4GDKR4NoxDHF26gVxf+dl/m pQrMAhsh2j7a9i7vAlYv2hSiwxToiWDYlmoJ/3dsSUlY1IU+4F8SmWtcTaMkbt8lupeP 9foDp6tSYTxq0MdRA7aIQsJAbQFGNdK3mTe5Mq9z07/oix116RWQkoxMv6+til+9C26V mwfuOf7Lpfl3dROEy/HSzxfB4dk6LQnDu+flvEBGykOTKhC0DY0CpfQTn8CYvk+S1ken 4r+Q== X-Forwarded-Encrypted: i=1; AJvYcCV6TycbqKi9JclImc8aTsbnwTgq/iLrxrQJCRg18L0a72NFqlKiyeOIa9OcgaTlzl0ebYvbOe4fVA==@kvack.org X-Gm-Message-State: AOJu0YyIfCH97euresK51sX/ywovX81xGT3JlDN/g39HSsrIjs5wMDmf 28QWtGDLi+rzVGWOiS/dn5wGydJd0A5FU3qWD7/Ufe2Qj3jh7nlCCm7PGZcc X-Google-Smtp-Source: AGHT+IHyXvN1ue7B3ODRoTm6YInUy1vTPF5mt5XLGTBYTGrDKLCROG5nKHfbHCNql8pi+pUWW0OV0g== X-Received: by 2002:a17:907:2d2b:b0:a8b:58e3:ac1f with SMTP id a640c23a62f3a-a8b595ddc9bmr313555066b.12.1725663415255; Fri, 06 Sep 2024 15:56:55 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.146.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25d54978sm2679566b.199.2024.09.06.15.56.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Sep 2024 15:56:54 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v4 1/8] io_uring/cmd: expose iowq to cmds Date: Fri, 6 Sep 2024 23:57:18 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: CE49314000C X-Stat-Signature: nqs1msyqimksarberwxzcmaydu3ymwwd X-Rspam-User: X-HE-Tag: 1725663416-484298 X-HE-Meta: U2FsdGVkX1/GQn96aUfN4O68pmzin0TzhlSMQvoG8YXSDBYE95hGewTY/Kn4WTFOia1LaVgONJ08sVmkhXKI46MPn2yyGZuWB1Q0iDDJdhtwaT3IGRgzRWF4eIzep7Ysz+eRNKtTP1sdSUECx9Z1jsd/FBhYZjRRq7MZ6ztNSb8E/0gyqomi4yy3910WGhrYzq5/Gk4qUM0aqfjJGxUnh8vx6JAFQT0pfwWFuXjtidhvnd2FIlL2C8ZHyJqEUhfv1L8+mHtWz8bnob4QIjwjLkXS6e78Ue+47Ch2+p2M7nNvVHr2cNeRiED+R1KUH1h+LslPQIKky6VIrjMsR9na2uZxTQikq0xmkMd7dt+SgZGWSbiMxEdjsa3v/bqxvIlGCTjO1+VN9ZfLTCFLaGSk9KiNfygNyNno5Kn0GbIdG1aObuT5H0BseFIRzEzUoygajs3EiBkZea7n0plBhZP3es1NZi7DPiyPJ84Wv69sFdUu+iCXW3PL8NReUGSVV9HRPe68JMAIsOsnxTpEshMBfeAwnLEusPQoTO9PQz35JiHckoVde1B99Hnzw3chmEiFKvT/E9jn6ckm9T9lM9kt3T79dqrMSXStVfxydnAI0gq4cC7246VsT5lyG+URSUV4NLqte+Gb4K3HBG9FjnHCu5S6lme3TqXsOgf+9mlLAFoaDVGrOJAamnPclNKHCXSiQkHLS7AUuOxWwWmbqfTIs2Ewq1W4JTnMYU4bvdDbHFxe5zozBzNgxDQJBo07A/UWS6SvlHpd7XBW2z0LsAFuoE8w++WcVxTaQxjGCpcjqDY2tPMmmF7ypdCrF9Nkd12hNGrUNVYFWPdyT4jhJ69kVWHy0A6FwghKT9ED4X7AXPahTBef+eB72+9F/ZIh18YsBgQ8Ock4jcHN2A1PpIQdO34X76l6YBvumGqgR0NgRwgxBWBH9YfaeJVbhjDGIEduIFhWOpTyngBN7ATfTQo 2hog6bLy BcJ2IzDDSNrM3+q/eP9NT1iT0pRrHRQn3gjDyZxMrpikR9L9Ioj9nURi3AbjICBO1hRQ1gHdBtisYdFNQNiGtsIKt2K91jKDSWwWO2ROaDGA/ZxTV1SLUAVTSPI0Ux2Do+o9GGqDGN8ZfUBARP2eCZ+lQVIVGcHDFDMlJ3HBDcAvVbyPwMeyLKCrOZV5yExXKBYwZwMmzwEjMbsW14bkiLWkdGy5uLKTI9E9aG+2IpPwTu3vum8qePO+bVUingROVsnrJe1JJMIjWB2eLQUBtpON6UH2BarGP63KZrWBBviHBYaDc8My4qP0bf+bIR4xPVZVn56BWU8MXejkk8z7h2iyKU2K/N6gxOZZRUIEYNS9pl9WoBmTA8RgdBWrH2STqlY70VnzStSkGHBYBNs7SzbC3mmwLAWo2IQ6CJgnblVPWVDhEFaTNu1s3AG/5CH+O0KSGtnyLVqQpLJw/IOViaPsQGgRhRSWHRvh7R0vmapD2izlncg1GdtRECQSIy7W1etp6BFtVIWFs+MzsNeUhAFrGgw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When an io_uring request needs blocking context we offload it to the io_uring's thread pool called io-wq. We can get there off ->uring_cmd by returning -EAGAIN, but there is no straightforward way of doing that from an asynchronous callback. Add a helper that would transfer a command to a blocking context. Note, we do an extra hop via task_work before io_queue_iowq(), that's a limitation of io_uring infra we have that can likely be lifted later if that would ever become a problem. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 6 ++++++ io_uring/io_uring.c | 11 +++++++++++ io_uring/io_uring.h | 1 + io_uring/uring_cmd.c | 7 +++++++ 4 files changed, 25 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 447fbfd32215..86ceb3383e49 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -48,6 +48,9 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); +/* Execute the request from a blocking context */ +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd); + #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd) @@ -67,6 +70,9 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } +static inline void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ +} #endif /* diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 1aca501efaf6..86cf31902841 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -533,6 +533,17 @@ static void io_queue_iowq(struct io_kiocb *req) io_queue_linked_timeout(link); } +static void io_req_queue_iowq_tw(struct io_kiocb *req, struct io_tw_state *ts) +{ + io_queue_iowq(req); +} + +void io_req_queue_iowq(struct io_kiocb *req) +{ + req->io_task_work.func = io_req_queue_iowq_tw; + io_req_task_work_add(req); +} + static __cold void io_queue_deferred(struct io_ring_ctx *ctx) { while (!list_empty(&ctx->defer_list)) { diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 65078e641390..9d70b2cf7b1e 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -94,6 +94,7 @@ int io_uring_alloc_task_context(struct task_struct *task, int io_ring_add_registered_file(struct io_uring_task *tctx, struct file *file, int start, int end); +void io_req_queue_iowq(struct io_kiocb *req); int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts); int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8391c7c7c1ec..39c3c816ec78 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -277,6 +277,13 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, } EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ + struct io_kiocb *req = cmd_to_io_kiocb(ioucmd); + + io_req_queue_iowq(req); +} + static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd, unsigned int issue_flags)