From patchwork Wed Nov 22 16:01:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13465087 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Tl4Yoaw5" Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D781A1B9; Wed, 22 Nov 2023 08:02:49 -0800 (PST) Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-a00ac0101d9so382976466b.0; Wed, 22 Nov 2023 08:02:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700668968; x=1701273768; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5dSrUq+ohe+2QxG5hUZzM2C3AjiOuQC9HjD2I4wQZ4U=; b=Tl4Yoaw5VZWNQwD+t+rbHbsfX3BJpNidWe6vYBgx0NDWPWo5jFm+jZ0nxOTkQxrKwG bnrImZOYh6w68+o2cmBrxEqc69VscODx7tCApOnsgSKL8ko8uRNicStU+OW/XzaZwUvq P3hzeSLaWHQ40ZlGGnvAD1TSjuY/3wlb9O9pYynk9hSqWSsbJRVBE9ZuwYGOeOJXrbbO D2XhNblQnHzRzwkHhHBT9e6J5UQBc4/G0sCn8ikBQyO1iE9f1yqnvCZW3OrGi9cNbCuS dmO3rFJZN8TzMM6h5jMbOwvS0LmjHgliHmgySPpRgn8+FvrB73gaQGLCL+FYqCjhC/mv 79QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700668968; x=1701273768; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5dSrUq+ohe+2QxG5hUZzM2C3AjiOuQC9HjD2I4wQZ4U=; b=vwb0t0xf7gXte9N/w2TRoKSpB9ZjZsdb740gooQCqwME0gjVU5j3mIjeGabS0mH8O9 ksSq4ahaN+jxZQBSWxGi/8mCLuXg5xLTr7Qj0asv+0Y+1/A0b89UNuCKtqxPylUC1fLW b5InNP+ZlsGnIiue0dexZY6ItsYXygCwbO1/6mTK61GMMAq1b6n5NwbU9OpbMlbsfVZy a/YoCWNvpE6YW7XMpQYgh1yTdrY2wunfNEQ2S62JP3TyJ/oDAFeVXw1+3eE2OVppxYTn upe+VV8TUbZUZyoSo4lcZ4lpPb5r61APl8JeJnME0pWyg9on56qn0xX5StoX6M6h64q1 8JiQ== X-Gm-Message-State: AOJu0Yz0YsnvCDnZh0QEarufsuNTnDbLPW4rgmpwu6D89GG9oEJvMTfT 5eLl9yqcxppwy4lGhllmPLmLxi+TLYQ= X-Google-Smtp-Source: AGHT+IEsW15KaW8QYJ4q6C6qtAKSP2U9DyVTlX09iq1I4wkEcXMFV20v+wCRwqxjp9zoulScYHPIbQ== X-Received: by 2002:a17:906:68c9:b0:9d2:20ee:b18b with SMTP id y9-20020a17090668c900b009d220eeb18bmr735631ejr.42.1700668967478; Wed, 22 Nov 2023 08:02:47 -0800 (PST) Received: from 127.com ([2620:10d:c092:600::2:fba6]) by smtp.gmail.com with ESMTPSA id x26-20020a1709065ada00b009fd04a1a1dfsm4541805ejs.40.2023.11.22.08.02.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Nov 2023 08:02:46 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, ming.lei@redhat.com, joshi.k@samsung.com Subject: [PATCH 1/3] io_uring: split out cmd api into a separate header Date: Wed, 22 Nov 2023 16:01:09 +0000 Message-ID: <547e56560b97cd66f00bfc5b53db24f2fa1a8852.1700668641.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.42.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 linux/io_uring.h is slowly becoming a rubbish bin where we put anything exposed to other subsystems. For instance, the task exit hooks and io_uring cmd infra are completely orthogonal and don't need each other's definitions. Start cleaning it up by splitting out all command bits into a new header file. Signed-off-by: Pavel Begunkov --- drivers/block/ublk_drv.c | 2 +- drivers/nvme/host/ioctl.c | 2 +- include/linux/io_uring.h | 89 +--------------------------------- include/linux/io_uring/cmd.h | 81 +++++++++++++++++++++++++++++++ include/linux/io_uring_types.h | 20 ++++++++ io_uring/io_uring.c | 1 + io_uring/rw.c | 2 +- io_uring/uring_cmd.c | 2 +- 8 files changed, 107 insertions(+), 92 deletions(-) create mode 100644 include/linux/io_uring/cmd.h diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index 83600b45e12a..909377068a87 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -36,7 +36,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 529b9954d2b8..6864a6eeee93 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -5,7 +5,7 @@ */ #include /* for force_successful_syscall_return */ #include -#include +#include #include "nvme.h" enum { diff --git a/include/linux/io_uring.h b/include/linux/io_uring.h index aefb73eeeebf..d8fc93492dc5 100644 --- a/include/linux/io_uring.h +++ b/include/linux/io_uring.h @@ -6,71 +6,13 @@ #include #include -enum io_uring_cmd_flags { - IO_URING_F_COMPLETE_DEFER = 1, - IO_URING_F_UNLOCKED = 2, - /* the request is executed from poll, it should not be freed */ - IO_URING_F_MULTISHOT = 4, - /* executed by io-wq */ - IO_URING_F_IOWQ = 8, - /* int's last bit, sign checks are usually faster than a bit test */ - IO_URING_F_NONBLOCK = INT_MIN, - - /* ctx state flags, for URING_CMD */ - IO_URING_F_SQE128 = (1 << 8), - IO_URING_F_CQE32 = (1 << 9), - IO_URING_F_IOPOLL = (1 << 10), - - /* set when uring wants to cancel a previously issued command */ - IO_URING_F_CANCEL = (1 << 11), - IO_URING_F_COMPAT = (1 << 12), -}; - -/* only top 8 bits of sqe->uring_cmd_flags for kernel internal use */ -#define IORING_URING_CMD_CANCELABLE (1U << 30) -#define IORING_URING_CMD_POLLED (1U << 31) - -struct io_uring_cmd { - struct file *file; - const struct io_uring_sqe *sqe; - union { - /* callback to defer completions to task context */ - void (*task_work_cb)(struct io_uring_cmd *cmd, unsigned); - /* used for polled completion */ - void *cookie; - }; - u32 cmd_op; - u32 flags; - u8 pdu[32]; /* available inline for free use */ -}; - -static inline const void *io_uring_sqe_cmd(const struct io_uring_sqe *sqe) -{ - return sqe->cmd; -} - #if defined(CONFIG_IO_URING) -int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, - struct iov_iter *iter, void *ioucmd); -void io_uring_cmd_done(struct io_uring_cmd *cmd, ssize_t ret, ssize_t res2, - unsigned issue_flags); struct sock *io_uring_get_socket(struct file *file); void __io_uring_cancel(bool cancel_all); void __io_uring_free(struct task_struct *tsk); void io_uring_unreg_ringfd(void); const char *io_uring_get_opcode(u8 opcode); -void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, - void (*task_work_cb)(struct io_uring_cmd *, unsigned), - unsigned flags); -/* users should follow semantics of IOU_F_TWQ_LAZY_WAKE */ -void io_uring_cmd_do_in_task_lazy(struct io_uring_cmd *ioucmd, - void (*task_work_cb)(struct io_uring_cmd *, unsigned)); - -static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, - void (*task_work_cb)(struct io_uring_cmd *, unsigned)) -{ - __io_uring_cmd_do_in_task(ioucmd, task_work_cb, 0); -} +int io_uring_cmd_sock(struct io_uring_cmd *cmd, unsigned int issue_flags); static inline void io_uring_files_cancel(void) { @@ -89,28 +31,7 @@ static inline void io_uring_free(struct task_struct *tsk) if (tsk->io_uring) __io_uring_free(tsk); } -int io_uring_cmd_sock(struct io_uring_cmd *cmd, unsigned int issue_flags); -void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, - unsigned int issue_flags); -struct task_struct *io_uring_cmd_get_task(struct io_uring_cmd *cmd); #else -static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, - struct iov_iter *iter, void *ioucmd) -{ - return -EOPNOTSUPP; -} -static inline void io_uring_cmd_done(struct io_uring_cmd *cmd, ssize_t ret, - ssize_t ret2, unsigned issue_flags) -{ -} -static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, - void (*task_work_cb)(struct io_uring_cmd *, unsigned)) -{ -} -static inline void io_uring_cmd_do_in_task_lazy(struct io_uring_cmd *ioucmd, - void (*task_work_cb)(struct io_uring_cmd *, unsigned)) -{ -} static inline struct sock *io_uring_get_socket(struct file *file) { return NULL; @@ -133,14 +54,6 @@ static inline int io_uring_cmd_sock(struct io_uring_cmd *cmd, { return -EOPNOTSUPP; } -static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, - unsigned int issue_flags) -{ -} -static inline struct task_struct *io_uring_cmd_get_task(struct io_uring_cmd *cmd) -{ - return NULL; -} #endif #endif diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h new file mode 100644 index 000000000000..62fcfaf6fcc9 --- /dev/null +++ b/include/linux/io_uring/cmd.h @@ -0,0 +1,81 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +#ifndef _LINUX_IO_URING_CMD_H +#define _LINUX_IO_URING_CMD_H + +#include +#include + +/* only top 8 bits of sqe->uring_cmd_flags for kernel internal use */ +#define IORING_URING_CMD_CANCELABLE (1U << 30) +#define IORING_URING_CMD_POLLED (1U << 31) + +struct io_uring_cmd { + struct file *file; + const struct io_uring_sqe *sqe; + union { + /* callback to defer completions to task context */ + void (*task_work_cb)(struct io_uring_cmd *cmd, unsigned); + /* used for polled completion */ + void *cookie; + }; + u32 cmd_op; + u32 flags; + u8 pdu[32]; /* available inline for free use */ +}; + +static inline const void *io_uring_sqe_cmd(const struct io_uring_sqe *sqe) +{ + return sqe->cmd; +} + +#if defined(CONFIG_IO_URING) +int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, + struct iov_iter *iter, void *ioucmd); +void io_uring_cmd_done(struct io_uring_cmd *cmd, ssize_t ret, ssize_t res2, + unsigned issue_flags); +void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, + void (*task_work_cb)(struct io_uring_cmd *, unsigned), + unsigned flags); +/* users should follow semantics of IOU_F_TWQ_LAZY_WAKE */ +void io_uring_cmd_do_in_task_lazy(struct io_uring_cmd *ioucmd, + void (*task_work_cb)(struct io_uring_cmd *, unsigned)); + +static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, + void (*task_work_cb)(struct io_uring_cmd *, unsigned)) +{ + __io_uring_cmd_do_in_task(ioucmd, task_work_cb, 0); +} + +void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, + unsigned int issue_flags); +struct task_struct *io_uring_cmd_get_task(struct io_uring_cmd *cmd); + +#else +static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, + struct iov_iter *iter, void *ioucmd) +{ + return -EOPNOTSUPP; +} +static inline void io_uring_cmd_done(struct io_uring_cmd *cmd, ssize_t ret, + ssize_t ret2, unsigned issue_flags) +{ +} +static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, + void (*task_work_cb)(struct io_uring_cmd *, unsigned)) +{ +} +static inline void io_uring_cmd_do_in_task_lazy(struct io_uring_cmd *ioucmd, + void (*task_work_cb)(struct io_uring_cmd *, unsigned)) +{ +} +static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, + unsigned int issue_flags) +{ +} +static inline struct task_struct *io_uring_cmd_get_task(struct io_uring_cmd *cmd) +{ + return NULL; +} +#endif + +#endif /* _LINUX_IO_URING_CMD_H */ diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h index d3009d56af0b..0bcecb734af3 100644 --- a/include/linux/io_uring_types.h +++ b/include/linux/io_uring_types.h @@ -7,6 +7,26 @@ #include #include +enum io_uring_cmd_flags { + IO_URING_F_COMPLETE_DEFER = 1, + IO_URING_F_UNLOCKED = 2, + /* the request is executed from poll, it should not be freed */ + IO_URING_F_MULTISHOT = 4, + /* executed by io-wq */ + IO_URING_F_IOWQ = 8, + /* int's last bit, sign checks are usually faster than a bit test */ + IO_URING_F_NONBLOCK = INT_MIN, + + /* ctx state flags, for URING_CMD */ + IO_URING_F_SQE128 = (1 << 8), + IO_URING_F_CQE32 = (1 << 9), + IO_URING_F_IOPOLL = (1 << 10), + + /* set when uring wants to cancel a previously issued command */ + IO_URING_F_CANCEL = (1 << 11), + IO_URING_F_COMPAT = (1 << 12), +}; + struct io_wq_work_node { struct io_wq_work_node *next; }; diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index ed254076c723..6ffd7216393b 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -70,6 +70,7 @@ #include #include #include +#include #include #include #include diff --git a/io_uring/rw.c b/io_uring/rw.c index 64390d4e20c1..4943d683508b 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -10,7 +10,7 @@ #include #include #include -#include +#include #include diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index acbc2924ecd2..4ed0c66e3aae 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -2,7 +2,7 @@ #include #include #include -#include +#include #include #include From patchwork Wed Nov 22 16:01:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13465085 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ktaoQ/Ou" Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3EB31BF; Wed, 22 Nov 2023 08:02:50 -0800 (PST) Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-a00b01955acso410310066b.1; Wed, 22 Nov 2023 08:02:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700668968; x=1701273768; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YiiLGrpZSDIC4Qc1OGZlXR+FMZWiJkQTWTZS1aN/KEA=; b=ktaoQ/OuKb7W/v9ZY8ffw8Ns2sJbglUnzqGnCtkscifLEe7UIQ0uP/YWcj4QrPmxfF VMOt/3WBi2WfKJcOTfC6EHpcVFZnRHWVFR1uC/zX6LFWfDr1JNLZdcqsJrro2I5ZlVkl 76s2jtNAwcr5F0pPCuf9My10yqAY8C75ttS4J4N6jU4bBNQunrXLsvVOXAAPKXbLBsf2 qsWZZ7hXI+2q2HPVpFICY+SrW86in3ALxM8Ci9iqKdJ+oI7NwcQ5ntXjbzABEGsnbxlF 5+jVHQCDIuTf7pwfpSwmRVkYxBGZYBJQlR7lysif5DU67oZ55cK/8TC13jA/EqZwIAzy UHJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700668968; x=1701273768; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YiiLGrpZSDIC4Qc1OGZlXR+FMZWiJkQTWTZS1aN/KEA=; b=D88TwKv+Kc8RonWBwCLCl98lkyLT/QaJIB/87kn7bWUkGY49JBQdsCv/OkZlLq/G+b B+yA6jfKTIoHcjge/mWYnJxJ1ur9+5oydw8Or7mSX6ENpWwhurOK9mTuWVEFi33TFKrR 8qUqPKsvwDhpAgaJuAiT6rXH8W/Se+Z7xWQYPNP686r8f8UgoBDdX3TQQFPvScAI+qiI 6QwEKnkKusLM8QYmHJB3yY5CwJxKV1Fi7IsflhdEfPbxjVhRoppS19rwMqtAO9BXm2Ry kKEWNQ7Ot9sV19mIb6EQNHgK5KRtFjbPhTsxlD+Ts1cGuTFPHuDtwND8zHGNmv5LUIjb 3kKg== X-Gm-Message-State: AOJu0YwvelUrvyryjtDxFZ+2fCpcCx3NdkH2uppLLACg2WR1iizQxQZ3 GvBYNiy1+XZq/rijOg3twLoHa9Ys+AY= X-Google-Smtp-Source: AGHT+IGFH0tW3t94HXFAM07g1SB+FjrzNCWiCzSDz1uOZHDp0U0s9sHudsY7p7+bVKYZydXprueerA== X-Received: by 2002:a17:907:cbc9:b0:a00:1c9a:a472 with SMTP id vk9-20020a170907cbc900b00a001c9aa472mr2730748ejc.7.1700668968118; Wed, 22 Nov 2023 08:02:48 -0800 (PST) Received: from 127.com ([2620:10d:c092:600::2:fba6]) by smtp.gmail.com with ESMTPSA id x26-20020a1709065ada00b009fd04a1a1dfsm4541805ejs.40.2023.11.22.08.02.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Nov 2023 08:02:47 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, ming.lei@redhat.com, joshi.k@samsung.com Subject: [PATCH 2/3] io_uring/cmd: inline io_uring_cmd_do_in_task_lazy Date: Wed, 22 Nov 2023 16:01:10 +0000 Message-ID: <9af6ade1052b26c69a1d12dad3fa5d05126578b3.1700668641.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.42.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Now as we can easily include io_uring_types.h, move IOU_F_TWQ_LAZY_WAKE and inline io_uring_cmd_do_in_task_lazy(). Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 31 ++++++++++++++++--------------- include/linux/io_uring_types.h | 11 +++++++++++ io_uring/io_uring.h | 10 ---------- io_uring/uring_cmd.c | 7 ------- 4 files changed, 27 insertions(+), 32 deletions(-) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 62fcfaf6fcc9..ee9b3bc3a4af 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -36,15 +36,6 @@ void io_uring_cmd_done(struct io_uring_cmd *cmd, ssize_t ret, ssize_t res2, void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void (*task_work_cb)(struct io_uring_cmd *, unsigned), unsigned flags); -/* users should follow semantics of IOU_F_TWQ_LAZY_WAKE */ -void io_uring_cmd_do_in_task_lazy(struct io_uring_cmd *ioucmd, - void (*task_work_cb)(struct io_uring_cmd *, unsigned)); - -static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, - void (*task_work_cb)(struct io_uring_cmd *, unsigned)) -{ - __io_uring_cmd_do_in_task(ioucmd, task_work_cb, 0); -} void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); @@ -60,12 +51,9 @@ static inline void io_uring_cmd_done(struct io_uring_cmd *cmd, ssize_t ret, ssize_t ret2, unsigned issue_flags) { } -static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, - void (*task_work_cb)(struct io_uring_cmd *, unsigned)) -{ -} -static inline void io_uring_cmd_do_in_task_lazy(struct io_uring_cmd *ioucmd, - void (*task_work_cb)(struct io_uring_cmd *, unsigned)) +static inline void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, + void (*task_work_cb)(struct io_uring_cmd *, unsigned), + unsigned flags) { } static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, @@ -78,4 +66,17 @@ static inline struct task_struct *io_uring_cmd_get_task(struct io_uring_cmd *cmd } #endif +/* users must follow the IOU_F_TWQ_LAZY_WAKE semantics */ +static inline void io_uring_cmd_do_in_task_lazy(struct io_uring_cmd *ioucmd, + void (*task_work_cb)(struct io_uring_cmd *, unsigned)) +{ + __io_uring_cmd_do_in_task(ioucmd, task_work_cb, IOU_F_TWQ_LAZY_WAKE); +} + +static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, + void (*task_work_cb)(struct io_uring_cmd *, unsigned)) +{ + __io_uring_cmd_do_in_task(ioucmd, task_work_cb, 0); +} + #endif /* _LINUX_IO_URING_CMD_H */ diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h index 0bcecb734af3..8ff4642dc6e3 100644 --- a/include/linux/io_uring_types.h +++ b/include/linux/io_uring_types.h @@ -7,6 +7,17 @@ #include #include +enum { + /* + * A hint to not wake right away but delay until there are enough of + * tw's queued to match the number of CQEs the task is waiting for. + * + * Must not be used wirh requests generating more than one CQE. + * It's also ignored unless IORING_SETUP_DEFER_TASKRUN is set. + */ + IOU_F_TWQ_LAZY_WAKE = 1, +}; + enum io_uring_cmd_flags { IO_URING_F_COMPLETE_DEFER = 1, IO_URING_F_UNLOCKED = 2, diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index dc6d779b452b..48ffdb156f73 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -15,16 +15,6 @@ #include #endif -enum { - /* - * A hint to not wake right away but delay until there are enough of - * tw's queued to match the number of CQEs the task is waiting for. - * - * Must not be used wirh requests generating more than one CQE. - * It's also ignored unless IORING_SETUP_DEFER_TASKRUN is set. - */ - IOU_F_TWQ_LAZY_WAKE = 1, -}; enum { IOU_OK = 0, diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 4ed0c66e3aae..94e9de1c24aa 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -78,13 +78,6 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, } EXPORT_SYMBOL_GPL(__io_uring_cmd_do_in_task); -void io_uring_cmd_do_in_task_lazy(struct io_uring_cmd *ioucmd, - void (*task_work_cb)(struct io_uring_cmd *, unsigned)) -{ - __io_uring_cmd_do_in_task(ioucmd, task_work_cb, IOU_F_TWQ_LAZY_WAKE); -} -EXPORT_SYMBOL_GPL(io_uring_cmd_do_in_task_lazy); - static inline void io_req_set_cqe32_extra(struct io_kiocb *req, u64 extra1, u64 extra2) { From patchwork Wed Nov 22 16:01:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13465086 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fBfqv5A+" Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7673FD56; Wed, 22 Nov 2023 08:02:52 -0800 (PST) Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-50943ccbbaeso9875236e87.2; Wed, 22 Nov 2023 08:02:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700668970; x=1701273770; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gkK7PO601H1RBbgbf5l3iMvM54jLAc0y16CY4Tdw43g=; b=fBfqv5A+aBI0uKbB1gCZKc8VrNhf0t8pOPBH2JXnK9iOg9U/uzcMwai8NLUnKQr+iU 3pVXQiTqQeabAQ0UU6qQAiw5dyChq73ZcT7DzKV5UsFELUCfGvcd9Wbn2GXfm/rYyMVy jb8wPpxNx7eDavz5Fk/hCp5IEES8c5p8s6ZgZmPCZuP5ZD8/TbGeu+PJ+XkmENFDJali f41s+GyOwk9J+VsC3Dl8g5eY1Vhph7aFTeZv+0b1jQzq5jGDZ1ryMjq3bVu0F50FR1/x Xt0POqBshhJjQxQSJa+yKPH30lThUNy8zowDUUz3AA1Ftl5hNWuGRVe5ncjvWOqxsNzR PN8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700668970; x=1701273770; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gkK7PO601H1RBbgbf5l3iMvM54jLAc0y16CY4Tdw43g=; b=X9fT/6d7KSiLLOKFqGmbB6RzvpDOIAZjyOdYocvT+aRNO64G/tIwXASQiqIXEFS5Zc 7r7eSwa4JBVpAc4RKv5dl7h6s/IPljMJwmOo6eIUWTzKOoRWwaD+dZ98HVvhAYUqviIU nZrGQbU9Br6M+xFyQ1zubWdB3WyPsFv0lOcyadfYUQTbPEifpOr9i47v18rbhbiOkbWS KCSKzWhwLCV/clUy/VGaoxBOMVHGmrEJh1FygVtlBPgJnANPqiYI9loAhHnCWkwgiCcj q+NSmBVK4i52xzakLEzCi3KPZU2V1eVAIcXaXpa8q8mIJoJzSD5rGCaHr7JbHhpxcGif 8ycA== X-Gm-Message-State: AOJu0YzP9pk1acLxPPTykfL6fmD44fQ8naTdHXYm8LPcq33yIM7xAq8T dRYwiOoTJJlJaRv+je2nIc2KvOL2i+4= X-Google-Smtp-Source: AGHT+IHuSF3RLTzru/3I6Fp13VVWBGT+O0N5315u+wAyv+X/KenGfAao3fBZEy+WEbF3PsgesVP9Jg== X-Received: by 2002:ac2:5e9b:0:b0:4ff:7f7f:22e7 with SMTP id b27-20020ac25e9b000000b004ff7f7f22e7mr2367670lfq.17.1700668969705; Wed, 22 Nov 2023 08:02:49 -0800 (PST) Received: from 127.com ([2620:10d:c092:600::2:fba6]) by smtp.gmail.com with ESMTPSA id x26-20020a1709065ada00b009fd04a1a1dfsm4541805ejs.40.2023.11.22.08.02.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Nov 2023 08:02:48 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, ming.lei@redhat.com, joshi.k@samsung.com Subject: [PATCH 3/3] io_uring/cmd: inline io_uring_cmd_get_task Date: Wed, 22 Nov 2023 16:01:11 +0000 Message-ID: <037278f2e69a0333d33d81864e3556ba54a76240.1700668641.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.42.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 With io_uring_types.h we see all required definitions to inline io_uring_cmd_get_task(). Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 10 +++++----- io_uring/uring_cmd.c | 6 ------ 2 files changed, 5 insertions(+), 11 deletions(-) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index ee9b3bc3a4af..d69b4038aa3e 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -39,7 +39,6 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); -struct task_struct *io_uring_cmd_get_task(struct io_uring_cmd *cmd); #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, @@ -60,10 +59,6 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } -static inline struct task_struct *io_uring_cmd_get_task(struct io_uring_cmd *cmd) -{ - return NULL; -} #endif /* users must follow the IOU_F_TWQ_LAZY_WAKE semantics */ @@ -79,4 +74,9 @@ static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, __io_uring_cmd_do_in_task(ioucmd, task_work_cb, 0); } +static inline struct task_struct *io_uring_cmd_get_task(struct io_uring_cmd *cmd) +{ + return cmd_to_io_kiocb(cmd)->task; +} + #endif /* _LINUX_IO_URING_CMD_H */ diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 94e9de1c24aa..34030583b9b2 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -52,12 +52,6 @@ void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, } EXPORT_SYMBOL_GPL(io_uring_cmd_mark_cancelable); -struct task_struct *io_uring_cmd_get_task(struct io_uring_cmd *cmd) -{ - return cmd_to_io_kiocb(cmd)->task; -} -EXPORT_SYMBOL_GPL(io_uring_cmd_get_task); - static void io_uring_cmd_work(struct io_kiocb *req, struct io_tw_state *ts) { struct io_uring_cmd *ioucmd = io_kiocb_to_cmd(req, struct io_uring_cmd);