From patchwork Wed Sep 4 14:18:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13790931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BA6CCD37B4 for ; Wed, 4 Sep 2024 14:17:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9806E6B013B; Wed, 4 Sep 2024 10:17:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 906CE6B013D; Wed, 4 Sep 2024 10:17:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 732296B04D2; Wed, 4 Sep 2024 10:17:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4AEEC6B013B for ; Wed, 4 Sep 2024 10:17:47 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id EAE251215C7 for ; Wed, 4 Sep 2024 14:17:46 +0000 (UTC) X-FDA: 82527259332.21.8328F81 Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) by imf24.hostedemail.com (Postfix) with ESMTP id 149D6180003 for ; Wed, 4 Sep 2024 14:17:44 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BmGDRBdH; spf=pass (imf24.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.208.42 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725459441; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=7i1IN3gtI/Pm9aIbqA5+eol8MKbseexeRCjkJFIv42dJG9DuIk8lHVTOdbmVYlcGnhM3+S hVCbwbiubCSeH5UT15x2uJdeg9bqezAcJ+yTGoa/6pxRreanSnXnzsibKyx0iIBiPR6PoS EWFus0zxPucYOfHCCdcqh5y4XK/BJt0= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BmGDRBdH; spf=pass (imf24.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.208.42 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725459441; a=rsa-sha256; cv=none; b=WV7yy4p0++xLsC+ko4vxmtcrmjv1BvRyxDAoVK68MLRShhMDdPzc61aPRBmG7f9ij2PzlK Pi0FkCTCJBXdzM+rukrLghlDhzYM3aeR+w62HcoxQOPTruqT+pnWud01jDTGG0acborc9v o8vUhHoDqLH/abdDYj/1qpZPZInjRRg= Received: by mail-ed1-f42.google.com with SMTP id 4fb4d7f45d1cf-5c245c62362so4454370a12.0 for ; Wed, 04 Sep 2024 07:17:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725459464; x=1726064264; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=BmGDRBdHJpYoF+FIgjgnit6TMm/URSEL2pxJ+lrccszoF+8ymQ1t2ljZHJwA1by5Vm Cl5vHN+d78VVyspiFJAD0K2kWd0ibqxQSNmjd1/0XrH4wD9HcE8ngdyVXUrSg//hhqOo tZUho0i0exXB/YBx6L5KtCXqeQOWDpG9wJWQbps8mao28pT0QVwFg1qLvhLPcNCqnUoB /zq5j4oKl1oO5kDmpNWttzWVgLbfwY7OQCLp0R6YO682kvukQvsCVbXeNRv7EAycPLod 9hXiVRSUCuNidxBPx7e4vB1VN0EAkgCcBj6LG8eZ9PeoqCZ/NNVPIj3jWHvRgWOn/FDf rRPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725459464; x=1726064264; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=K+gtlmeJoZOY0RzAw8NPkP1CEFtKcaQnIp6rwkfLY+hU8APyBhNr+obU3kJPhTj5dn WSfsTl6pqvyJzga7ASbaaOb5nAiy3UpoTtXQN6DX19QWjgkFAgeDOnvddkUeGI+1HfgJ rfdxHX2L8NxV+mIcm998xgMOkWxrQ86rUGN17fC/MUJt4TynaC/oWkvNLmEnNbJIAXO/ K/NK6Fi8spOFdeFrvNLdSJPoipF5+Jx3bVWjCldEfGTN86f8eOIo9QD9i82I5HYjkLcn Hxx7ZsB7+LhPzrryDzA8JkpAzp6fwYReh9hwwATJX6QwWIesEz5EUcqXgqftsmsfxXA/ fOuQ== X-Forwarded-Encrypted: i=1; AJvYcCUqVkf6GCRP3mwe7IHA+QRNCul75wsHDQOg78weRgyRtT+0UbpmL+cgQPOR9OW1FcSNVx6dVGD+ng==@kvack.org X-Gm-Message-State: AOJu0YxU7+dpBc/Gizpm1b8Jz/zeoiVqs7WFSDHbJFKvI1S8eTcUjG7S NnltoqjeDwDePVqJyt+3pYmeahOVKr8WgX7KrJFwEwS7aGRoaYSw X-Google-Smtp-Source: AGHT+IH097ivB/9WxgCVpcO8B8ihxC+WO/gvLLOLQq9DQZmLBk0QFnqSJuSVFdgCDV3eF0HI2K1RcQ== X-Received: by 2002:a17:907:7289:b0:a86:9f56:21e6 with SMTP id a640c23a62f3a-a8a32ed6109mr297583066b.33.1725459462834; Wed, 04 Sep 2024 07:17:42 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8989196c88sm811160766b.102.2024.09.04.07.17.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 07:17:42 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v3 1/8] io_uring/cmd: expose iowq to cmds Date: Wed, 4 Sep 2024 15:18:00 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: mp3o6dm7jjqyqcouew576gsgs179pbwd X-Rspamd-Queue-Id: 149D6180003 X-Rspamd-Server: rspam11 X-HE-Tag: 1725459464-551763 X-HE-Meta: U2FsdGVkX1+AV5JwEAw+l7BLzsz+f+N/nwqmpMAJ2B8/mQXdNjNEZhStcmVWs+4IVwTLbT9pXmNARhM+1g4/Je7QulsLUfb8TIJp8gLIY+FmhuHDPkENTXRnVKkjGhD2+lh3nfBFH87sgnGRrQMApgfHYtlmHWH7fFZPKjc7pt+YJ7Yt39xzrFNcQjJidQJsEMAcBmDxOePmOR2dg6CKRMAW2PXOe5RnJ9xCI+YXrLaAptb6hxlu1C9cySai8WP9Gmn8788OTQYgd35O11Uy30qqj8rOiexcGTrTUajR87MySOkJJ0GTIbwFp92s4Tj77Fe9Io+w8zilQD1a/immdafk3TtyyrjWuydhPhUTfz8vmJkuhDF+V/zz5PrhL3k9viy74adCr2gdyIhj2gHtRN7VQRZ5Nb7WoyMfZ1WoJnPjKym+k1F06NZrk9kuxYHeFoC4whZNpJbwKTW6yKME0vOXyfV8wI17G9DtoQYbjsJ1N3gIlfemwNRlNBfrHHsjbVlZHT432MSq7AD1X/8PcHFV8rLVWwY61MkJ2QHFnTUncG07WxAPynso8333idKumeGodqPG+ZyFv3EkYF2NqALWydCZ57HRY/w02NVATXz19NZwsrySvzU1nlITPLzXUf54o8IpOXS3jMMbBVnOUppcdTde/HVhj3vb6DRVhQYNiw9E8P2wr6U5Sk+jW5VeitbM5hv1xJNvRnIY6w2W/GuXXucdGrlwdLIJIh4bSJcbyOWWcaVNkNT3Rt2S+uAy2XYbFwjAlyfXeMP7VRcub5bjxmvdQ9ieHDtLVWbn6ZmWsCt6Qq5O59c4BpSDaw5NzAXe3qvdEWgDPuvbxBEoAbbQ3/lpClmHRh+OLhBkCS1XPa0x14nkNOFjCi84264FevhhFwG2Qi+q1S+d++6nnpxymn0hJ9atUbReH3Z6GHvco+i5kEs/eMlxbwDu/vMx9WomsbxCvwCDKeh0Mi8 72sqpAKh q4mPvmNY26wRU14JRa3ftxGeXmhfKo6SVlcGB6G+SRFOCtbUnswJhu1LqLBqbg0yUC6deEdPfYyYXvAhsmGIZT8fUjb6E1N/yyXPZCkNcNNJqcJNhGV3yfNiJiI7Efcex/9BGbrOGPS+fLYZmTZILSGwCHKIHHec1mTqOvgziAjKYeqC/M+p9vRVBtWmUC+C7WEJW+mHSWKkBZk/iWLWFQ9bouIQUtCxyoMN/BoJlfBPjAtvRVa65rIkWk3k3fC5Xzh2RizZX48Myq3gNUOuJhjx4atLWKw06unM1t0/NnYeufNdeTZ/poKSnFE36h/pSYChKFqjq0ApnLMtuKT+IBVNe0uaTy24Y6Ys5GjaPAwMP754Z6BNj3wCcUB6Ks7oI2YcUST7MZ6qSVYW5gHgTGb7M/bKmAUvNZRf31gkcqdXtr1IOCbEPgCuWNuWvZh1KUm3iSnPcaNIzGUHC1TeIVj+CtHSlXIkWFjiGWxrh9FzHsjwqRqVcod5Sa2icW6hXqctwP0quz7zw4qXlYptW//oaGg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When an io_uring request needs blocking context we offload it to the io_uring's thread pool called io-wq. We can get there off ->uring_cmd by returning -EAGAIN, but there is no straightforward way of doing that from an asynchronous callback. Add a helper that would transfer a command to a blocking context. Note, we do an extra hop via task_work before io_queue_iowq(), that's a limitation of io_uring infra we have that can likely be lifted later if that would ever become a problem. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 6 ++++++ io_uring/io_uring.c | 11 +++++++++++ io_uring/io_uring.h | 1 + io_uring/uring_cmd.c | 7 +++++++ 4 files changed, 25 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 447fbfd32215..86ceb3383e49 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -48,6 +48,9 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); +/* Execute the request from a blocking context */ +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd); + #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd) @@ -67,6 +70,9 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } +static inline void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ +} #endif /* diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 1aca501efaf6..86cf31902841 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -533,6 +533,17 @@ static void io_queue_iowq(struct io_kiocb *req) io_queue_linked_timeout(link); } +static void io_req_queue_iowq_tw(struct io_kiocb *req, struct io_tw_state *ts) +{ + io_queue_iowq(req); +} + +void io_req_queue_iowq(struct io_kiocb *req) +{ + req->io_task_work.func = io_req_queue_iowq_tw; + io_req_task_work_add(req); +} + static __cold void io_queue_deferred(struct io_ring_ctx *ctx) { while (!list_empty(&ctx->defer_list)) { diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 65078e641390..9d70b2cf7b1e 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -94,6 +94,7 @@ int io_uring_alloc_task_context(struct task_struct *task, int io_ring_add_registered_file(struct io_uring_task *tctx, struct file *file, int start, int end); +void io_req_queue_iowq(struct io_kiocb *req); int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts); int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8391c7c7c1ec..39c3c816ec78 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -277,6 +277,13 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, } EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ + struct io_kiocb *req = cmd_to_io_kiocb(ioucmd); + + io_req_queue_iowq(req); +} + static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd, unsigned int issue_flags)