From patchwork Wed Sep 11 16:34:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13800877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42726EE57C2 for ; Wed, 11 Sep 2024 16:34:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B176A940064; Wed, 11 Sep 2024 12:34:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AC70094004F; Wed, 11 Sep 2024 12:34:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9679A940064; Wed, 11 Sep 2024 12:34:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 792D394004F for ; Wed, 11 Sep 2024 12:34:27 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2CF39A7D30 for ; Wed, 11 Sep 2024 16:34:27 +0000 (UTC) X-FDA: 82553005374.27.F6B6D33 Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) by imf30.hostedemail.com (Postfix) with ESMTP id 371F48001C for ; Wed, 11 Sep 2024 16:34:24 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Jw+nc4/o"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726072460; a=rsa-sha256; cv=none; b=IeSODn4g9yb6Qt98ko5N40SU7PFT1QlPErHaafa4BaUSojPbe/N5A1pYNjgGZ+ZKqGgGAe jX1w+FEkav2crmfdKMzLRuke43CpHG7C4WRlfIo7tmxvqBr1laYHC8WASYy8gCRfmOUl7f p8tAjanTetJ5bZErTT5VAWvdF8b8X+8= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Jw+nc4/o"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726072460; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=llZ1YZ9ap1afUzL+DfM2yTRgS2V2EngOmOI+C+adjbrwoMdxmiX7uXgfqh3/vaiGzJyh7u wokV5jCpYQbNXuAFrStVtgBL98PgJBqqpNGNlqGn8h4xqIDwDvoKk6Qm9C8vOwcDesvNDN 1WwlQqCa1fpOAtWERrMKeLdDzgScPDY= Received: by mail-lf1-f51.google.com with SMTP id 2adb3069b0e04-5356aa9a0afso11833312e87.2 for ; Wed, 11 Sep 2024 09:34:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726072463; x=1726677263; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=Jw+nc4/ocbjRsZQLM5FO6tibb0WIl9azDyo1suIkoSU8WASi5hvZm+2ZYbtqO3Ai5N pqrbeADqgQ1zuNH7+56fWidSvsOe1lRAbfYtuqYO3VvBMnOQXxEZ72lELXB7UE36k0Ms cCN38MVSPo/c1xS3mrG5OlyvGSJRBLg6tlEtTaHEroJxwtvknrimsRtc6GvTzbTANwF0 J9TGVKJmTLpN5gwweEV1yhXgdhWJIET2POx9Z0oMfNMFvMZX097o7X+vtNEkGU0WaQyo KStiasho1QPPzwJOmX4nNQEM7vCv54f+ZQOVo9a8aG5SRyO6UabGq3tHc/ZO+0+WYNJL 96BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726072463; x=1726677263; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=ODibERZ4SraSA6X2zqbDM30GuR7BOacvIN8KDCwHJLv9+rh9k7vTTRaCxEAUR4VM1c qAa2pDFvogw/L65mq0WzkIQCPxUWiOuwr43Zc5SU9ShBA/jKg5Mexkr63A1s0N0E+AL8 wHOwpG+WBcH7YHek7ND1Xd4VZYZQeMuoKsS1M1/4ZIN2gvs7wvmRjxFxUpxlKEaSBHQN 8PsowKb+ZgSOSF3esINhntv8iyksO7j4QoncB2zQPoEv/XI9gxuBgMhPVwsJN+M4BVyt cjKQ4W+5GOFoOw4mrdFAWDOm+ka7tikXpCglcA0+O9FgRzfL5jNUb3QngZOEsdKBxqtV UQug== X-Forwarded-Encrypted: i=1; AJvYcCWGNPaul/+YTyzHyG90XXGTazJ2sF+GIdzO0YxYfvQjg/L6HLj6mWeA75MmOsJHfrvv6/0VK/cLeg==@kvack.org X-Gm-Message-State: AOJu0Yx6INztZ4wQ94TNAQhaZpdubhkcN0y4zJyLV0vYRKRKUVIWqjdd V3YoppVlgV46AFlBH0WjN0m7IE/HwVVgrylnDFPhtjo0iQ2uljnI X-Google-Smtp-Source: AGHT+IEG2kHGTzNGAdSjpEaXDSIgfMbjRxkjuMwMgmsa9Moi42gUvNZP3GJN98tnR/Gf0UwGcIEW0Q== X-Received: by 2002:a05:6512:10d5:b0:535:681d:34b0 with SMTP id 2adb3069b0e04-5365880bc04mr17121594e87.47.1726072462342; Wed, 11 Sep 2024 09:34:22 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25c72ed3sm631820866b.135.2024.09.11.09.34.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 09:34:21 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v5 1/8] io_uring/cmd: expose iowq to cmds Date: Wed, 11 Sep 2024 17:34:37 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: toro6jzu695xzwcgqtzu4gbxjxg4e7s4 X-Rspamd-Queue-Id: 371F48001C X-Rspamd-Server: rspam02 X-HE-Tag: 1726072464-740438 X-HE-Meta: U2FsdGVkX1+vFb0fmOeUycbS2hXj+5XLzoXem1qQTcLuvtUDfjJhgQQgduBIYwKK4LOXXsRbpgoTPKeD4gWyBs72km7ikUGYVA1DG5T9UCkoXyARFktw+b7aey88EPsmnWr6LyGOcwh+GORSCC19pRBWGUwJjrWCp0/hVGpGmCeVJvthi1roQtVeO9eWLPKqjbzlUlS/ipXAAnVPZq8O21qa65JvwHJ30UUD8ijy5h0AxLWsAdX/peietClSLAhqHAT5kwx4DrYOoeXNHn5n6yAElZcVMMihUC6MBvFz/Xl7lrbErJ20SgXQP/F4SmuaaN4/m3f6Bg2SMZJ0jDbVn4a2YP+1+9tPHG1U5Hd2ci5nJ66ZeQuIllD6UupkTi9oUaX7djzUmQnwgzcX3vB2ARfHeSYPQdLJNOdEPxARr9ZKnqk7AJ/UILy5Q/x1IzoY2Kam4YkEIfpkwZ3rvwSTJtGZTOobCIFvuMousbgvcg87iBcajsw4SNT247gICvwH1IHEKI68qDYG2ca2uN6mZ7hdULb+bMh5NwF6D0lDaDy5o39A7swt2SZuWMqz+gIboZX7Z4IyI3SmHitWRiEo3v/eaRfGpF8PUP7J06e1oO05FvktxBxYrSCP157i82aAz+Sbfnj/EhfSljLsJqjF6ohfo0k1o+TXn+T4UzI/6I47VnPDeqHd0BC2nQ+dHtHaeRa4CQdy/nKuG7Sq5SETy7uByRFvN1TF82PXlwKEaen+f6WtsjTumidGhqmvA8A8AnRvLHnu4r3jIaabg0ZtAS9689gBLGyHCepSfOipTsXsAmvTwmM7J5Uv0krdQfqqLpiAsEZT3hceHWk3oIk80oJFpdDqQbCjv4sUwAt5P8w/LBrpvUEWMMTlh333nzQ2zePRthB1C6gjX8a3AQstomy8X46C0uD7TF9lMrR3Rr/8sMZd0P3F4j0/oABLfIv+D1TakQ10kKNso4aFO4R bpf5dhTT 4dBhobLyzrIFClX3aedBLXxh3leToFIWx8skjt149SHy/d+NbIPN9+g4CVgBjwY2m+sDU+0mA1sQ+LQec4R1dsVtxB6kC50jbm0mnkAotzw/kNjDTgqa5DtNLtFovWifCeBJvvjVCPn27CqPydNOC+qVf90O02KBcr4VEkNxKhlXqI+mXeQFm5i2sHuAeZIqcGlAHuyN8Y5tVOPEUKrPe4dlbil41pYL1JtqspvILN11i5KZiThDiOKmq9v25kSItjQf5bmQH8uJBH3vZYCH55NUtHQf18tF7geUbBoRyV+W1z46d1Tz5Af1ceaxU8ssfKEcXZwml8FNQSkaL+V30i/CtyDU0WeS9X+2pbJB3Fms41qB2S4+rxxlLgDjMmHSgNzYOWofvkUYv6B1+Qbnyu07fh3QYofvVFli8LrPLQYrqZ+2i9NYXs5Up5FCmNg0AZjqM0Vc+fDSZiktYt7MqASsmf/vzLsilETt/SgEhd4Ocz2gIgx06sMDqrLEW7Qj5bv/XlfPweo6F2c8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When an io_uring request needs blocking context we offload it to the io_uring's thread pool called io-wq. We can get there off ->uring_cmd by returning -EAGAIN, but there is no straightforward way of doing that from an asynchronous callback. Add a helper that would transfer a command to a blocking context. Note, we do an extra hop via task_work before io_queue_iowq(), that's a limitation of io_uring infra we have that can likely be lifted later if that would ever become a problem. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 6 ++++++ io_uring/io_uring.c | 11 +++++++++++ io_uring/io_uring.h | 1 + io_uring/uring_cmd.c | 7 +++++++ 4 files changed, 25 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 447fbfd32215..86ceb3383e49 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -48,6 +48,9 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); +/* Execute the request from a blocking context */ +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd); + #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd) @@ -67,6 +70,9 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } +static inline void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ +} #endif /* diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 1aca501efaf6..86cf31902841 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -533,6 +533,17 @@ static void io_queue_iowq(struct io_kiocb *req) io_queue_linked_timeout(link); } +static void io_req_queue_iowq_tw(struct io_kiocb *req, struct io_tw_state *ts) +{ + io_queue_iowq(req); +} + +void io_req_queue_iowq(struct io_kiocb *req) +{ + req->io_task_work.func = io_req_queue_iowq_tw; + io_req_task_work_add(req); +} + static __cold void io_queue_deferred(struct io_ring_ctx *ctx) { while (!list_empty(&ctx->defer_list)) { diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 65078e641390..9d70b2cf7b1e 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -94,6 +94,7 @@ int io_uring_alloc_task_context(struct task_struct *task, int io_ring_add_registered_file(struct io_uring_task *tctx, struct file *file, int start, int end); +void io_req_queue_iowq(struct io_kiocb *req); int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts); int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8391c7c7c1ec..39c3c816ec78 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -277,6 +277,13 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, } EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ + struct io_kiocb *req = cmd_to_io_kiocb(ioucmd); + + io_req_queue_iowq(req); +} + static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd, unsigned int issue_flags)