From patchwork Thu Aug 22 03:35:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13772574 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4872C52D7C for ; Thu, 22 Aug 2024 03:35:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A29D6B0253; Wed, 21 Aug 2024 23:35:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 454D594000B; Wed, 21 Aug 2024 23:35:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 259CB6B0258; Wed, 21 Aug 2024 23:35:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EA7BB6B0253 for ; Wed, 21 Aug 2024 23:35:40 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 921111404B8 for ; Thu, 22 Aug 2024 03:35:40 +0000 (UTC) X-FDA: 82478466840.08.F8044BE Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by imf02.hostedemail.com (Postfix) with ESMTP id B68AC80011 for ; Thu, 22 Aug 2024 03:35:38 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bNR1LNJJ; spf=pass (imf02.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724297658; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Oq1dmJmhMQ/ONS/9FcWt2dsBOZ/rQa7CJ2HEombNkGY=; b=uM2w/1bu1daBVGseTXpl/yZabRedvvX3eWrHq0wLZy8MOLinjtVQbkSLK2wpa+u0WWUr+/ FhdYN809nKxYB8N+bTiDYGeO76Smm4r/e2IF/bdYQ3E25WuLjDsdH6rzmQTnEUZ6Dzio1L +ammJljv/RqyejugtrFlMtkougJEgMQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724297658; a=rsa-sha256; cv=none; b=cb2A+jxljpQomF0HD6OcF79AOwgrt77DVpN91lCuw0BvA1I+rDBfaPtkyXojdki9mjfdTW +XuqSME/YWD7CMKSp55HlkqnCAm+OtWvp6mnLhRoAoWdvCoQ/5I02mOvBupDImv2FcDUd+ 8T+EAobpDtji363Ayp4toW2PSlQa228= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bNR1LNJJ; spf=pass (imf02.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-42816ca797fso1974625e9.2 for ; Wed, 21 Aug 2024 20:35:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724297737; x=1724902537; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Oq1dmJmhMQ/ONS/9FcWt2dsBOZ/rQa7CJ2HEombNkGY=; b=bNR1LNJJXG0gaqdVXFbOiJVBRz9MwOHA6+f2hcuhO0VId2jLT16be4Mex4nVdYCsT2 wjr2DMl5Q1QlMqs6RmO2vW+7aUubqDPh1ajOjBxywkK8Zb+90Bfu48AKN2qwg4bKoeYo GF0T7acGTQlBYQvOy1YZDQxgoVq2ToOAmwB99/yEiVyVSFgAQckpVwKNQUhZgLD0LYh8 cYm18FF08FC7KwjPu0cEvBF7PIy1YeF1C2lLftOfWy/rQgll3+ZXoS2Oz5B2vvfASh59 UWs3TslymR6axCPJJCBhQnw+UAL2YO85VAghgPXP1RQNnmNaGFThEAWu1jKVJt6Oxas2 B63A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724297737; x=1724902537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Oq1dmJmhMQ/ONS/9FcWt2dsBOZ/rQa7CJ2HEombNkGY=; b=ZxmpOeQhNxbEZ9XP8Q9jAKFHwStd4hgluRGy9lCbJOWKucDTWNAezJHggRlEiQyKoc NAiqFoy01lDiVN9UgOkPeTc6/ZS5RSkYlz+mMRGauDenDHvm2jwCRaz2RAXg6QEnKT51 +PqTYa6U9iJCLrngTrzwCHYEC8VFBfXzXSORkZiPVN35a7c/r9GSmfgejyZW0tUnCSef 3IUbseGusnwzNFkfS8lLCI14zIUlNNp9kGKW83SkKYk5gxO2ld8SSIJqhyIeW+vllokb tgU8ksvEw9hx+8kAklLiJcrYOT8b0El8KaZvLYMDClKtJKrDVShvcz0nGWsaD4uNa34v 3y2g== X-Forwarded-Encrypted: i=1; AJvYcCWdHWZNZ1bBWHw3k3vulTtMzmtA/7fPYtQwMUt3igaq6xG2o4Y51WQmh3/5sAegQ8w9IuMB4za89A==@kvack.org X-Gm-Message-State: AOJu0YwA9ctQ4MpFdLVOpw8xD7gzBpNmziClwroS3x+3Zod8kU++Y3PB v7bTiwJnZkhaxL+MzZ1y74bH1aY4bei7UFEQPfdNRPhnkrcyg88Y X-Google-Smtp-Source: AGHT+IF1YE9cTCYPQ6lqJcA/bVj+/okCLizbljJ5bkv4iDV+o0PRhQhkc+Im1+k1WARHaBsjsN01GQ== X-Received: by 2002:a05:600c:5486:b0:429:e637:959e with SMTP id 5b1f17b1804b1-42abd2125d2mr33760875e9.10.1724297736489; Wed, 21 Aug 2024 20:35:36 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.128.6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42abefc626fsm45491995e9.31.2024.08.21.20.35.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 20:35:35 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 1/7] io_uring/cmd: expose iowq to cmds Date: Thu, 22 Aug 2024 04:35:51 +0100 Message-ID: <55ce8b5b813a7ca40597457b9afc8fd17d4ff11b.1724297388.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B68AC80011 X-Stat-Signature: cmgeqmxz6ra4x6o3ambezztniiowu5cr X-HE-Tag: 1724297738-988223 X-HE-Meta: U2FsdGVkX1+1D5gaVycxucIs75z1YQvG383sM88sq2nIb+xIglg/WfqDDFg2V9j82UbXTrwavqPj5c039kZU2h3S8+Q+TDxDphdQ11lA7JAq2tXk7bZ2p3XfizncXBa3u41fb+J0R9yJzl2nX+PZMVtrHRqAdgVuWH65lY1op40kEF5uibKnpoTYGP/WVca6T35upCnGijqsOPV9lfzaf+ILjirpm6b9Im2MusMuhIRg0jMG0xj7TZJcrg2uUEbuyXXzCBv5hDOaNIks38bGbTGgEegILbne87yy8DMy/D1SxlAepuInzGVVRNPWlpV4b2/982F/WpHB67QrkuTxriyvHUzUL0pTg7HMUuVR0Gh055iZVhk5wXpqSpbfKPmsCPeBftPg4oQFnbuy417Asl9cocQSsaEQcRVc7H0Bo5sETGaGD3xjjHRo+mZFuZqKsWGXdRd2DUQy7nLtixwt8KGyI7xXeVIuDq/BB67Us8kJqJh3RnKifNAfre4ouLCLd+4mg63SsYVL1lB30BSxAVCAqWPv1nHZHSXxNY7mBQsRCAE1S0WUA6JqXC4TfIfxv4CqA7+W8Jd8feXLD5S5TKm8KEfo6LuP/HuC8Rx5TiEsHxBk+3OyUHS3cpo4t74mPm/5CCQdiPTimf+aaqLmgGCEp7tH/gdWodfFgcOSWMeu2JXv6HNET3tZsZpkJ1lXjaM+Kif47cjH3NIwTaHywcWn5YGpEnC4sgTvNefxhr4zRFPW8XfL/rykkpuYLP4vyKoN4KffFI2jwhXLgatJC/KjnHx3c4dNPIULDzq9bHwh/HkG/JoIDte4gdiobKeJ0enR58CzAKNcKpqA1EPb0OUrx9ts1PJpLqPER1jsmB8dDAV4/rGFaV5ZTf1JL48QUgPSANR7PUOpmlXZEJ8kTP0edqqVr58lvAJK/e9zDqnnK3ZCCvPQIinITW4cphaGB3iQlYW34fP2QyfIjGA sAMzzCCw PucyZAQKjHDayKjmtQjULvABvHakZWQsuYfrL2aQpnH9Af6uToELnpVmtRvi1o0Fd66htgC0aTVXQZqhUMmC/7OdZfVxh1ceyI/nffE9AIdkiSnA6ci+o9Ek/TeGHEUpSd975kuXPo6kIsfkpRciPjvH0QhgUice632Ucm5ujVDRqQkP/l9TGBrEkRir5vkIQrWMkrCVVCAHZ7rmSIlTu6oWgLdZ21c/0SFsbkf5OmWC3dFmP63gl3TlLW2Dze2z/Qu0HPxCLaGGGlSg8rWXyODJ1+hrZxzKPRzcvcq/XM18D0dErXVV4mPcyuUDlAzEBOSyMjX1KEFV7MyUAGwceBF0HG2GNYbtNF67/OhwQbA5rMLfbMqtMoC+BcIUYy2Dr8nNvDemseLvwJgbicCz1voAQiD/wNCqV6LSk2f2S+yjpa2/Lp+f8TymE5Ie3PmJa/VQgNX7Tbt/nvDnal1nNZKQVeu1HRBexX0wMXnCB7hodkJVd+XtYCx36lA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000078, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When an io_uring request needs blocking context we offload it to the io_uring's thread pool called io-wq. We can get there off ->uring_cmd by returning -EAGAIN, but there is no straightforward way of doing that from an asynchronous callback. Add a helper that would transfer a command to a blocking context. Note, we do an extra hop via task_work before io_queue_iowq(), that's a limitation of io_uring infra we have that can likely be lifted later if that would ever become a problem. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 6 ++++++ io_uring/io_uring.c | 11 +++++++++++ io_uring/io_uring.h | 1 + io_uring/uring_cmd.c | 7 +++++++ 4 files changed, 25 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 447fbfd32215..86ceb3383e49 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -48,6 +48,9 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); +/* Execute the request from a blocking context */ +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd); + #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd) @@ -67,6 +70,9 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } +static inline void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ +} #endif /* diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index a53f2f25a80b..323cad8175e9 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -533,6 +533,17 @@ static void io_queue_iowq(struct io_kiocb *req) io_queue_linked_timeout(link); } +static void io_req_queue_iowq_tw(struct io_kiocb *req, struct io_tw_state *ts) +{ + io_queue_iowq(req); +} + +void io_req_queue_iowq(struct io_kiocb *req) +{ + req->io_task_work.func = io_req_queue_iowq_tw; + io_req_task_work_add(req); +} + static __cold void io_queue_deferred(struct io_ring_ctx *ctx) { while (!list_empty(&ctx->defer_list)) { diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 65078e641390..9d70b2cf7b1e 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -94,6 +94,7 @@ int io_uring_alloc_task_context(struct task_struct *task, int io_ring_add_registered_file(struct io_uring_task *tctx, struct file *file, int start, int end); +void io_req_queue_iowq(struct io_kiocb *req); int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts); int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8391c7c7c1ec..39c3c816ec78 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -277,6 +277,13 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, } EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ + struct io_kiocb *req = cmd_to_io_kiocb(ioucmd); + + io_req_queue_iowq(req); +} + static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd, unsigned int issue_flags)