From patchwork Wed Aug 14 10:45:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13763315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30C58C3DA4A for ; Wed, 14 Aug 2024 10:45:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B81706B0083; Wed, 14 Aug 2024 06:45:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B32306B0085; Wed, 14 Aug 2024 06:45:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 984AF6B0088; Wed, 14 Aug 2024 06:45:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7EA566B0083 for ; Wed, 14 Aug 2024 06:45:26 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3FDD2160E11 for ; Wed, 14 Aug 2024 10:45:26 +0000 (UTC) X-FDA: 82450519452.30.1404088 Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) by imf05.hostedemail.com (Postfix) with ESMTP id 53F38100004 for ; Wed, 14 Aug 2024 10:45:24 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=W6YM2Ybt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723632244; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oGCUwVa5Kyp0Pg2+kUxIsl/LPJjPrI+WqiYx+qtKk3Q=; b=nUupxzzttWXQuT1oTiD0c5PbkSbd/WVocmP3EFM+ZN370u1PxuJ7TeeZf6r6AJiDAhg5RB FefHxYEs6FHEgrQHBrPMfMR60T/qLs9KoJlA3XcsU+mAf59NLF6nO0/Soeafl+iisqRAzo 6huRhQ4WkUoXChIwGVLVd1epcXdyPbw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723632244; a=rsa-sha256; cv=none; b=aXSKy2r3tR9ZRqL/icR6aK4q+EKWXfVN2HB4XZjlBtPuz77zFlN8i/oUd6cwlLpyxL/033 DLbW3YbNR5Wkq3P2c9/FMwt0c2CofByMAKLpcg30iuRN+asTrBme3NtX1wAwx/mCroDB5i 4/X6LVB+ScbO0KojewHTiNQ89YeAzys= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=W6YM2Ybt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com Received: by mail-lf1-f51.google.com with SMTP id 2adb3069b0e04-52f01993090so8796282e87.2 for ; Wed, 14 Aug 2024 03:45:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723632323; x=1724237123; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oGCUwVa5Kyp0Pg2+kUxIsl/LPJjPrI+WqiYx+qtKk3Q=; b=W6YM2Ybt7xh5hSff/dEzZS9Eqaj+VaaL4za6zDVh+Q97v2gNb4bNAmL4IF14K2GgpE QoPikXYeREIKyd2sBS2Ia1vn8EMeo2+1KV/Yat4iU8YG2Pbt9DXQZqhYdyWVZ5j7fhfk O3xMn603zAMH0lWfLGy+6xhXHS+vW+O3NI+w05ji16XA9F78OgzpuOYoOPOR6hNxHaMs LNBSNvnz39QRf1UnEq6scwNxRi7ANxsOoraU8wISeiSyDC3OHyGvvY/mJf3t/PtS9l6P piWCTFpVRRLAHgpbyivzv58kXdx2i3xcGEMdZt4KxC9VKYwWTp9NRUTErYMppV7uAA9O y56A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723632323; x=1724237123; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oGCUwVa5Kyp0Pg2+kUxIsl/LPJjPrI+WqiYx+qtKk3Q=; b=dn3RHDEYJ3uKg4HGhCsSwyoFWPa2uZQ1lr8dPfiKO2hM8bKBDjKr8iLeLTcHNNeMB+ lrMbR1orw2N2hESP0/5KIqLiKcORyhdLsxy7NPxjq0sm7UybKz023nt3OR+qvp9Xa8G9 fveuFrJfQhwfHuDMEvVVfQ/175hCeogM42Bl+Ok6/DsZ9woUm8CJb4w9Ltd+GF6C/zib wxJNemJNMb86UoLEGOglRW7P9xh20XmqBeoq3k85KLZecmqLb9Hb7YkijJOcJGukFnyn rGa7ZLHZftvsq3T6kxmzHVch+Ga3yPr/hTgtc2y8QCcb07ZPZZngbFofcHvfVLHsd7uY tFBw== X-Forwarded-Encrypted: i=1; AJvYcCUvbnQ2H+mxZyoGyutCxw/xxJFF4VTIU1EuYV5DXWwQIU/0fWM47qGMyvNTZR6gSx8ouH+5yYJQuAtd8/7bmYpmXD0= X-Gm-Message-State: AOJu0YyAjGaG9DzBjUVB7+vBmQKKG8IcznfQXS8DzM3CJ8e5G4IlvKGQ FFVM7yI56lEkIUTZgZi9abrNhszonSlvIXcOaebqHjIalwWq13Sv X-Google-Smtp-Source: AGHT+IHhdpdAhXAFGJZHpDxWh8B1Ms2Ng2AgCkdonuHrS94jDeJvM4fNY2m5pVGUUhNgCJJpgyCaZQ== X-Received: by 2002:a05:6512:1591:b0:52e:be50:9c66 with SMTP id 2adb3069b0e04-532edbbe548mr1283237e87.53.1723632322432; Wed, 14 Aug 2024 03:45:22 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.132.251]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a80f418692asm157212766b.224.2024.08.14.03.45.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Aug 2024 03:45:22 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [RFC 1/5] io_uring/cmd: expose iowq to cmds Date: Wed, 14 Aug 2024 11:45:50 +0100 Message-ID: <0e94fb94c3d627dbd5c210d00bd6cc3e644d00ff.1723601134.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 53F38100004 X-Stat-Signature: bcpfdpdbsinz7t7aswg41w7iwbcme3a3 X-Rspam-User: X-HE-Tag: 1723632324-873147 X-HE-Meta: U2FsdGVkX18+36dz8ofGHue5bm5I9aA0h+b2EkFI2YmRhl4qrSgUYp1W4Alxf5oYxEif8GWTF/1763lkq0/hkoaxV7BADf1SGlVp8nMcEaoa7tMT5QhRxHXzOa7MGvO5lU6WEUTDalXttBuMYdXIAoz/qD20QulaW04MwJfeqElpG09ti9mCF3v3g1RfFBMYgPXlA6TE5PeQUpBZ0cKKJSg5fuXMezKFuME9HHZD8ZqSmydcUvceLQRJ/Iv8Qacystggx8u9xBTN5qEA1NsBBMFeb9VT0tJ9B0QguYDeBzGLWNT6vlFzoiyyJ1W5JgWQBENYh4DPFsLJkWShVxEYQ8A7kFzMK03UfoCwMv8s/x75D5ShnH9zjwDbVZhdvunxPpYl5t4OYLr8PjOzGLgDzkTte58YquX8dI74dCynxiTeESEy800w8ozHBR9IJ42oWI96sk9y0jQeniyT7qztWyifHdbOtWtmfx7+DA3i5xBVAE4N3L6AsLWIin3UpIll8rENLMR4a6KjyNJeOmIeJnG4555QonVf4irww4ZwWCBFyd+UA5P4quMdxjTlCWirJNfj2MjV6HKXmPIpnYEocX5I7Wp2m14yXi2vs12PQXO2zYeMekjD+myj74ZtAr4BfVhkDFBo5z7euc4C4c5P6cO8GNX7ZF0mF1zafOloF0OyfWq1tvV0XBKthuRPTQgqSiAhXLPgPc47MxBNKU12ogPY8ZRIVy00VvlaftBZRcQQOjt3mmFK9PM04XNVZBsnaNAVvF6uZlZxXaE76Vwd2EW3qi39Ms2cAqz7JqE4oH7Kw9pGbsJ/9U6iUHkVU851JKJZZ+u77lkc0TqigTYuoMLurZWrDz5uAGoM3PLcONn6LKJX/IL1uKkBfj2xMsw7ZpWzybm45il2IpG2nGSD36HkztdG++B1p5vP68j8+UUmQN30XaJTaAnnqE/3F2rOQDEpCAPLptShu97ytj4 JZG/a3YB bWJGP+V9/aPp9DIv1lK65KbVefDjE0K63XAB4+oqG6i75SrwWnIxDigvTT8hiJiTT9kHMQygBMDYOP3YgajSJGbpS9CxITdMlqyBwm3dh5aCXlTXRmYkeVTzwBluXj8J0qReZTAmi67fD8Vy2gnwwG94CcW6n6fwbQ3P+LHbGnhpFCnPqeaMHoZOlIeI3R645vcHnzivdVq1ug9PkSKXgvGq+B3QyfniOtgAjtp16BAvcYtYy2b5xrOkgl8Mz2EE6ellblK2/3SxSR0jXbqYt+8UCvSRTExc9vAuiTj2g4/ctg2TmVkb5e7nZs0TZpZjO41NGST7uixexdJJodua/O2lN+XVSdMhCBV0wC11/5nxfeAHcYSUEklHUtDmMxVgLnb8qE0k5bJEa3ccO7xL5C4FLBE1dM1Cm04sWw/Ygmy9K1pKtlQ2Xgcy9orQhP6fp5mS3GHZMxRBLkwWg82obHNhjg+GUcaeKqS0TcQPZCWbviPkRzzkZp0oEHg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When an io_uring request needs blocking context we offload it to the io_uring's thread pool called io-wq. We can get there off ->uring_cmd by returning -EAGAIN, but there is no straightforward way of doing that from an asynchronous callback. Add a helper that would transfer a command to a blocking context. Note, we do an extra hop via task_work before io_queue_iowq(), that's a limitation of io_uring infra we have that can likely be lifted later if that would ever become a problem. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 6 ++++++ io_uring/io_uring.c | 11 +++++++++++ io_uring/io_uring.h | 1 + io_uring/uring_cmd.c | 7 +++++++ 4 files changed, 25 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 447fbfd32215..86ceb3383e49 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -48,6 +48,9 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); +/* Execute the request from a blocking context */ +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd); + #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd) @@ -67,6 +70,9 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } +static inline void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ +} #endif /* diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 3942db160f18..da819018088d 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -533,6 +533,17 @@ static void io_queue_iowq(struct io_kiocb *req) io_queue_linked_timeout(link); } +static void io_req_queue_iowq_tw(struct io_kiocb *req, struct io_tw_state *ts) +{ + io_queue_iowq(req); +} + +void io_req_queue_iowq(struct io_kiocb *req) +{ + req->io_task_work.func = io_req_queue_iowq_tw; + io_req_task_work_add(req); +} + static __cold void io_queue_deferred(struct io_ring_ctx *ctx) { while (!list_empty(&ctx->defer_list)) { diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index c2acf6180845..2896587a5221 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -90,6 +90,7 @@ int io_uring_alloc_task_context(struct task_struct *task, int io_ring_add_registered_file(struct io_uring_task *tctx, struct file *file, int start, int end); +void io_req_queue_iowq(struct io_kiocb *req); int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts); int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8391c7c7c1ec..39c3c816ec78 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -277,6 +277,13 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, } EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ + struct io_kiocb *req = cmd_to_io_kiocb(ioucmd); + + io_req_queue_iowq(req); +} + static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd, unsigned int issue_flags) From patchwork Wed Aug 14 10:45:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13763316 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8D0CC52D7B for ; Wed, 14 Aug 2024 10:45:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 718456B0088; Wed, 14 Aug 2024 06:45:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 58D696B0089; Wed, 14 Aug 2024 06:45:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B8A16B008A; Wed, 14 Aug 2024 06:45:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1CE176B0088 for ; Wed, 14 Aug 2024 06:45:28 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A48D7A7F9A for ; Wed, 14 Aug 2024 10:45:27 +0000 (UTC) X-FDA: 82450519494.13.65B60A1 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf06.hostedemail.com (Postfix) with ESMTP id B042218001E for ; Wed, 14 Aug 2024 10:45:25 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=epbn5fB0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723632313; a=rsa-sha256; cv=none; b=i8zEqz/L25n7nxTHTNx36s8e0OcXvXnPHA49yV39HL7NPiO9CWuCfQ2q1pfN6Bcui77L7v mWiZu2KwNa7D8o/P5eVNTYrp1fMLUS9GL+F/QwZEkjc8rM6lFmmTuNNz3HS3uEGkciMioG W0BYHDdFUKC/0CZYUs0f7VJvRmAykAE= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=epbn5fB0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723632313; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=XtlsJmwbrNbycm+jZArzojaOteA0Nc04vfLFK8zQ8m6fbQQQla72UUg8dz89wEBLIR2Ayb ZrwQF2JoG1HZVHGpWK5SRGAjTwooRnXY5tJb6yZ+ULAvrtNkHtrNnYRMl/4Ul13xwSacFT RnNGsD2uy4fA8st42COJv2PyUVqgepI= Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-a7aac70e30dso720812266b.1 for ; Wed, 14 Aug 2024 03:45:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723632324; x=1724237124; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=epbn5fB0CF+0IThePn3cXuB58q/ziwtEUbsrtaPTXQl+1kuZdvy+op53FCqbKjNnCy 3l3q0DBBociXAp64bYlx5z/8lunxkbD7Yj+EDbpQF35GlgzKFGSj52De2ozOGnfFfTZo rA00+ZcZCXeuATWye5/SktiocSAoQQgLCpvUokOxQLkmJEPBgwujsvh/XHHAYkiwyASc 5K5f48H76uT9NI/c+G+tHuorJyocU4ld/Xs246NJ9y7rzggORkVD+0Ny+D1A21jflgFd 4yBEjoqMhq9+haUUT7FGSNBkB3EpEfbZ+EOTAC+crt14unbwNUWh8ESgybLvINgfkg+M +lEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723632324; x=1724237124; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=tY1YHdoHnixuXv43NGDahoRuvf2/fMK6rfCjJjJArVUReV4xjcv4dE1HMV4yJAOiZ3 eGrcYqExYMZir32NONoswMTi3QnZRWVP8Pu3kgF8FJLbWE4W+4ASezCvdDWdi2G8s5dT 4he/bNWWH3ArNB8/t0gG3X2LLvLSVWKlIsfNn9S53DFZsK26RAtADxagVHhJG6BTjYP1 rvHhjdwaljkRpPSwv5uduWyIGLxFzR3FwOvlNplBIEZGE7TcY86xCFEnLcrfuH+KRGIy 9XTfI1n2I0mw6RAm4hxHK3Cd9qWq27unZV8Gr6cp4Tfzd7H0m/syE6K4OLaLLkj6b4wX ffDw== X-Forwarded-Encrypted: i=1; AJvYcCULz3IxzmleY9u1fWyKQukC2dE9bnn0EiAThyBHuWdoueNZxnurSe+244tqyzmW+psP+uqPlTfx48Q/zsOcuBRb1QI= X-Gm-Message-State: AOJu0Yx0FkttvRSvEiyshK8UONTr5/Z9u9Y1bHjoOzfgZIggcOXxqZY5 URqwVjrOHuatsNO1JE0ksaYgcaVmE9dr4NjDmfSIJ6q1oSRIBjbX X-Google-Smtp-Source: AGHT+IF0FJ+sTAvpOE9fJ0iyXXmALD7st6qSqWtNfVEYLYX577G+yNnVuj6lODfTWpASdYrnqFEhoA== X-Received: by 2002:a17:907:e6d1:b0:a77:cf9d:f495 with SMTP id a640c23a62f3a-a8366d658cfmr180030066b.40.1723632324005; Wed, 14 Aug 2024 03:45:24 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.132.251]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a80f418692asm157212766b.224.2024.08.14.03.45.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Aug 2024 03:45:23 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [RFC 2/5] io_uring/cmd: give inline space in request to cmds Date: Wed, 14 Aug 2024 11:45:51 +0100 Message-ID: <86ae0042c57c5904b40e19f6171f47fd3e8c6126.1723601134.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: B042218001E X-Rspamd-Server: rspam01 X-Stat-Signature: 8iy5j3u5iugwc6d4r634pk6tftcmd7g4 X-HE-Tag: 1723632325-905249 X-HE-Meta: U2FsdGVkX18o0iPnjzT05y5KSNBbX3XaFPnpvLXHynfKAADQ51NUgDu7gFsfU8oLbKlKiJ6gb0XO22Uxo4zI88m4O/L7H5AmiLkhb1m1ONvU3CKLkTedIjK4EYB47lgVrVM3vv38zgpWp6Qeua1YveQnjT5AGLeW5MD5WHyVWFdVDFb7H1rAkMrcVYeR6z2zCn9/VCmPmPGdVAWvmSKIDfXEc0hqxIPHPz3o3Ri/IigxgIBQPiIqNSQJQkzumjG1UZvWdUgMb78QeP8zG5qWervkAqRfkQdtVgG9rMvu957SlI8W3LIkLuhVfDkzFDEHgihWOlKsHZ2RqMhmSOZalwfbL5bFOK/fEH3+r0iVIVRatxa33eZthOyHgmEWq3NkKu9XJ+qVmcQSiwsEfd1rQSZb6aOcMhWyMNbpKBbMbPOPJWrYwT2XQsRrKhDRkiuUSLoxTlNNZVCxGvsGOA/BAw34vhkJ2O+BFXCbevzxlO73ms/z7vJwaTJgl4759sDYqLKX5yeeTqnpwnw5nrJhp3zfvWCVIic1tRfodKWkyyuZMWJOhoNjBii3eCzNNDtymC/0rL+U06XAbELGQ0ouachQBgL+KqPKAMEbVl8ETtcmbY4h2GS63xZTww11wAcEI13DLPS0L+MYbF73KRC2ytJeG1Ooy8er4GbCXHJ5UbATP7D+r+4XGbzUZg/Tl7Tw1wqs7nJf9hNmR+8snnY9OvPE9dPZsfDUwEDIk/yufQAgDZaMGYkOqR4gkHEWpPcf6BYqsJb8d2OQrtjbMrtrHVHbCAV5uiUkRLhEBMk6ulLM2JbnzQxPjoKUI0LdbG5U9eBwi4EjyHkaX8I++DVwDNxLMSkQnsLqR2LpLzsMrDO1YOPOPoR9NAM/E0WN50m5h9ib6OjJPDMIr0HCtpVEZXIhx/fmzPQThxrqYGiHJ6x8X6yLB0nsuti4fISDcthK6Ul86W+XaZuyr+Q9FvE YnVhbKsT eNUqaUls3jGEau8IOTXvdo+/hfmBlEjmMnSwMQlF+BdnB4z/gDI2Rmy1WlwAOeUPaiH1SIrzYUkQ4kohXdVyf2eJQrm/W8IEmf4XzbjwBftm62d4M3IvkXYI+C5OGTTF0hI4TqDF1O9fLIiHiXYVRpRhaZcDuZtPYko7iVj4IVcFHfWnbw530uXrqx7cdDbOQDx5oH/7NdOBuerMbjKhOvIBd3Sd3TWy1Qtg4GL9XiLI9cZoaeDuTQUFQULduwoGu18D0QEhw0V5o+gxFl3NZlE3wZ/ea5l1OwR/6a/chyU/HjWtbjbnXJkM1bXBksJwtT7TKrb38a7eRe6QHGeciMPvMuE3opm/CMyMhvvn/Osv/TE3KyOqe3Zfpx6cvzDbRnGj6qEbPtnMkM+VUcnvt7mC9AqxZM4ZT4ksTehc71gVDy/fdfwDDb9M282Usei58Z4cFkCMCpIa5WbSr+uLtCQgvJ2SaNdNr1htSO1NjY6Lmm+zS7I0F2rjYlg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.001439, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some io_uring commands can use some inline space in io_kiocb. We have 32 bytes in struct io_uring_cmd, expose it. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 86ceb3383e49..c189d36ad55e 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -23,6 +23,15 @@ static inline const void *io_uring_sqe_cmd(const struct io_uring_sqe *sqe) return sqe->cmd; } +static inline void io_uring_cmd_private_sz_check(size_t cmd_sz) +{ + BUILD_BUG_ON(cmd_sz > sizeof_field(struct io_uring_cmd, pdu)); +} +#define io_uring_cmd_to_pdu(cmd, pdu_type) ( \ + io_uring_cmd_private_sz_check(sizeof(pdu_type)), \ + ((pdu_type *)&(cmd)->pdu) \ +) + #if defined(CONFIG_IO_URING) int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd); From patchwork Wed Aug 14 10:45:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13763317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89BF6C3DA4A for ; Wed, 14 Aug 2024 10:45:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 911816B0089; Wed, 14 Aug 2024 06:45:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89B796B008A; Wed, 14 Aug 2024 06:45:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 676126B0092; Wed, 14 Aug 2024 06:45:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 4359D6B0089 for ; Wed, 14 Aug 2024 06:45:29 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 07C4740E8E for ; Wed, 14 Aug 2024 10:45:29 +0000 (UTC) X-FDA: 82450519578.01.CD0FB64 Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) by imf22.hostedemail.com (Postfix) with ESMTP id 230A4C000B for ; Wed, 14 Aug 2024 10:45:26 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=VhpfWCAc; spf=pass (imf22.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.44 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723632256; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=coYbv4PcaCiw3s7iKLGpr0JKeDXM/58rHgT2Op2hXlo=; b=PPg+lK6Y8tw+FdBAbHCV6zhdgKF87NxKSWW22w5pcnCTcQVGg2JvQSQB9PswKjtBAHeRLP wOw8cKYE5SjI9s3gaugV6xnLsLoz0zVLoiSUPSdZC2SV9bUsBaFOWq8bjyS0+QqCUxJiqS fj2lclMKt/Gy51p6WDP+/WDNbqAyap4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723632256; a=rsa-sha256; cv=none; b=cAOLBxuaWiDazN6aleubdA9PwJUGCVO/Q8QKNERyEOZXMXPCP1roGYsQQF803U2unD7GFb oITnYoww6dmKT0YBQUaqbbdQScXb7XOoCcDaS/OsvJ8Re/C0re+tVg2jvIdf87sOAZ9esV 1GfRuyXJxlNlAyaB9mccSQGPSRxKjGU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=VhpfWCAc; spf=pass (imf22.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.44 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-a7a81bd549eso468129466b.3 for ; Wed, 14 Aug 2024 03:45:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723632325; x=1724237125; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=coYbv4PcaCiw3s7iKLGpr0JKeDXM/58rHgT2Op2hXlo=; b=VhpfWCAck5QNRGYT0+TImvNzMLikRF21rNh/oe5cUTNiJpfc0ZutBBH83skDbtxx8o BJf9/LAsr2/pwzAw4zW5xHryRC4x1m99rIVu/vVVWZNM6TIFJFuo5TjtfLWAIC+5O3FB Dcm6BqSwVAfl/LH2Li9QjiUfLUt3JocH8M3ROemG8TM9WES0tkxDSTRyQn9lfp31txD7 yI7NtAYXruIzrFvrUgPLtu7MLEYe6Y3ykG4VoEy+/qCc75Cl+gQHEoy8Eqe0Y95nsWLn tQuriCd/klJ88j58sdYxL8NXW/c+KMxkGNr/KbuPFlm23st7zQEQPeR98CXiKL0hHF8G II1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723632325; x=1724237125; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=coYbv4PcaCiw3s7iKLGpr0JKeDXM/58rHgT2Op2hXlo=; b=NE51JJNIZK9xrKSMR561YQ9G9YRE4kpX/nXjKGspBdvxKZ2fSaUWBxDxQytPnUq62i zoX6hzkKFOzzR8BDgwMQDqGtYofwJCiP3eeFmXY3Le/Uo5q2xhjFfIyC7/geF+12nENZ iItG/y9eTYkaqhl2ywfiMM66BP87z1rl8XUyV4+E6iIeFogjxUQz3PWY8tTh3J6OdkS0 ZAhYeS8GIAb3OgEmog1HCXuZGp0YbwZ9OutVF/yZupX9kic0x//lL42jIIAlU3HIFztN EciuTvlGOxLl6YVrJokMSo3/Pzk+vZHtyMdxQ5ncxzE1q5U9/Zuy/5skhZ2qQIOJanpO +m5w== X-Forwarded-Encrypted: i=1; AJvYcCXl0NrthaDcU8v0pNA1A5mnq/O/d1EFI/6O4j2tfU/bHbLVxsI8P4wl3xmxEl0sK441GNROBwqc0GbToA9H1KJNrbA= X-Gm-Message-State: AOJu0Yxcqcz/7us34KhhGfnec/SWokruqnyZvywsKmsQWhUJarXIQfHA g/D/OBnT9Wc9H/wuPfhHKKesk4BeGiZ2uawSqxwcWXZbAzOQKCujqlU1ZVwR X-Google-Smtp-Source: AGHT+IGfriIepvv++2PrjZ9Q9AjLepaO3ymcXMwFRj3SpDze6Sv3EXtnN23wF2rBhfZ8mcwRYC92+g== X-Received: by 2002:a17:906:cade:b0:a7a:1cfe:a262 with SMTP id a640c23a62f3a-a8367009c0emr168159466b.55.1723632325229; Wed, 14 Aug 2024 03:45:25 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.132.251]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a80f418692asm157212766b.224.2024.08.14.03.45.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Aug 2024 03:45:24 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [RFC 3/5] filemap: introduce filemap_invalidate_pages Date: Wed, 14 Aug 2024 11:45:52 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 230A4C000B X-Stat-Signature: oeihjzccxamhr5roxoz3xpgfumrquoo8 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1723632326-159376 X-HE-Meta: U2FsdGVkX1/9RKRmxy/YAV+6EezqiSkVxf099duVj7GM92OQU5Okm0VjdhcHLXjR2FiKY4/Z/DuXV4iQ6mHjten7UTCa64o8o4cW6EAeEhOfhBdIN2byV+wvh+g3t1bznmg01i25DGhDFI4J9nP3wVMdHvkxBVLp8Tys0JhFlPUXII0qiPBkvlB9lq07qrTqbQ143ZWGuoT3CvKJPgjfFNOgZX50J3jsLdtfhGX/FJwdJuMMs1i3R09CDpf6Lnc3XUPx0n86au/7LZcNjubgcvkVoXNMhIvfJ4f06zh36SiGDkhupLo6APC0KWIEb7AJLTD+ZhCpMoiYFDuDxLN3bKYkTNy3wAoGEd+ZWZd5EakLQ2MQY3Vfdu89GXvYTRxf5ZCnK1P82vOLxsABoxDIwwERk7AbRx3Ox5yu6UtfK3Wh4imaG0Wl3Ab1xsagKdNKxGCBibt/dS9IyoDbBktm5NEomfzg2DMOj3inE/PQwCEEKc3N1B//sIfMC/RUKagOjBJId724z0UsyK5XgRPXnFdtcKcOoKkL7i4LIylHfkIzElTRmClPBR3STYFpYE9xkGNVApj8Gh/rAq26I9Wia7pSRkbiDGGvVGv1Ya1dbnuPBHWNGp5K/c6e0no99J68Zmicw2WY3G4ts2xZXWOl16+ifVz+9JpDzjVjuHly/YfLl+TRkqDbs7PWq56ZQxi0zbw6C7qBIMV8p9dWUajxS7du3KZQwP26hm6ntwIad416hNdvWV+AhhiPXAjntrjfsjHCfNUUCV2AEDTG59xZUkJs1B3PQmb/RTZ9sRMSA9aD09HdrI05+/L9ZqltbuV8/3hl2hCr+7Z1atVNrJyI7WR5r72H/xkyUwsF8+YQhEzDe9V6va5KX/feFgst3Tpm069RGAHd7dshCzOJkr/PniZJyhBf5xbw70aQShE+CyWzU+1TMZSjLuEjM1H8QgrRg30K3sbEhbHxh4gYLaP hz1Ab6IL h7331QG5gbLhPIzrHCL2INVT4dFiiD/GugxKGNJD1Hu3SsPoaMw10+6Au/+yI87Bw9491tLlxFGHf5qS1RHFL4148m7PP4YWh25NUCZxxR/d7Ed0Lm2gsEY8KnGLgehpN6JGnEVqEk1rp229NQBImCOlK+0hXAh7LaGSaT/lFZwTK0Vrpx9svTF5naz25Zz+lYzWxr14v7hGWpObZxxVulJWASzbGqWC1dDF3PEVZMRzWUzycN5Lr5t7KUiYYf8oUzIYogr2uYOG5CWfSEhui2xPs/+igQ8VLUXZIR3UFyc3iyF4jihL3b059fwckGLhXSbvMSPOiy3m8X5iNhkjuUGUdWSSyWWzo48+mmYEDS9mWZMCvuGERFJKW/Ubpo+tvwKHQYjPGvvuM0JrDWTP3pTU9vm3pBeYhChATnmGHT+9ofVnJQqU3EZpYizfN+pzYbMTesVDJX7WxRFMpmRpxVAtbR7AfVWd2hQThKA9RMLyf/zcyBM0d+IfNWQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000021, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kiocb_invalidate_pages() is useful for the write path, however not everything is backed by kiocb and we want to reuse the function for bio based discard implementation. Extract and and reuse a new helper called filemap_invalidate_pages(), which takes a argument indicating whether it should be non-blocking and might return -EAGAIN. Signed-off-by: Pavel Begunkov --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 18 +++++++++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index d9c7edb6422b..e39c3a7ce33c 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -32,6 +32,8 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pgoff_t start, pgoff_t end); int kiocb_invalidate_pages(struct kiocb *iocb, size_t count); void kiocb_invalidate_post_direct_write(struct kiocb *iocb, size_t count); +int filemap_invalidate_pages(struct address_space *mapping, + loff_t pos, loff_t end, bool nowait); int write_inode_now(struct inode *, int sync); int filemap_fdatawrite(struct address_space *); diff --git a/mm/filemap.c b/mm/filemap.c index d62150418b91..74baec119239 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2712,14 +2712,12 @@ int kiocb_write_and_wait(struct kiocb *iocb, size_t count) } EXPORT_SYMBOL_GPL(kiocb_write_and_wait); -int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) +int filemap_invalidate_pages(struct address_space *mapping, + loff_t pos, loff_t end, bool nowait) { - struct address_space *mapping = iocb->ki_filp->f_mapping; - loff_t pos = iocb->ki_pos; - loff_t end = pos + count - 1; int ret; - if (iocb->ki_flags & IOCB_NOWAIT) { + if (nowait) { /* we could block if there are any pages in the range */ if (filemap_range_has_page(mapping, pos, end)) return -EAGAIN; @@ -2738,6 +2736,16 @@ int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) return invalidate_inode_pages2_range(mapping, pos >> PAGE_SHIFT, end >> PAGE_SHIFT); } + +int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) +{ + struct address_space *mapping = iocb->ki_filp->f_mapping; + loff_t pos = iocb->ki_pos; + loff_t end = pos + count - 1; + + return filemap_invalidate_pages(mapping, pos, end, + iocb->ki_flags & IOCB_NOWAIT); +} EXPORT_SYMBOL_GPL(kiocb_invalidate_pages); /** From patchwork Wed Aug 14 10:45:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13763318 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B54CC52D7F for ; Wed, 14 Aug 2024 10:45:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A041E6B0092; Wed, 14 Aug 2024 06:45:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 987B26B0093; Wed, 14 Aug 2024 06:45:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7AE9C6B0095; Wed, 14 Aug 2024 06:45:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5B94E6B0092 for ; Wed, 14 Aug 2024 06:45:30 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0F2C980E0E for ; Wed, 14 Aug 2024 10:45:30 +0000 (UTC) X-FDA: 82450519620.22.29A05CF Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) by imf18.hostedemail.com (Postfix) with ESMTP id 2EF3E1C0008 for ; Wed, 14 Aug 2024 10:45:27 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RgzEeoAa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf18.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.42 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723632270; a=rsa-sha256; cv=none; b=AYrAv6GIcHCLUYOtjJUSkyUwbRFrjwVma4NrMFPJzM7apW1WpbwuQy811U55BOO8nemm0s yPiuL1kCVZlfZL3kap/gL984O4zchPptVozVbWn08Mivebj+dnCD9sv5eFHjV+l80At1W1 GHAMxqfYK+wtCJ0fCLOMOuVA4j1AMSo= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RgzEeoAa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf18.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.42 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723632270; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fvJpr7p454Y/UfNPCoQA/axDAfLDDtHMeofcmIJVphY=; b=zFsQWpjBmYMHVDGXa/+GqUfi6pvbFTrulFphUnCY275KxMhJ2njI6gH4wfAyr4051JsJ5E CgF/aoPU4IR/1JsMG1aFuok1oANMSAK2TcMFX+6ZDw/qDoeqXdrlANckfnhRPAszHXgNSv K9+4c1cLtDI2veWYGHU7LXDJO5N3w5k= Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-a7aac70e30dso720819766b.1 for ; Wed, 14 Aug 2024 03:45:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723632327; x=1724237127; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fvJpr7p454Y/UfNPCoQA/axDAfLDDtHMeofcmIJVphY=; b=RgzEeoAaFHH35+m5+o+uK9rZlf3jJ/7OqRBss8eSdReNGq6xLsQZSVwDyR98p+rd1C ekicxv6rowXJYwNW8fE7Qe82ycSrYwDle6GFOrnYUFBRRuKe4FqzjzZZ1v1Z+Rfb2Oaz 5rQ2bjvX58Abh6yUWUPK+FubQrqNyLUadLshYVgv/rOCPkmcYzILR+f0zdDfdf57veTV 7Nxha5jCN6P+oqFiNHm+PBn0P/MrEZY/V3Q0vHH1o+7rR7uiGLWhAhJysJzZcI3PAZk7 mM7zuoqzamYdByxVYdGv7xyr9pIEOXMNhNNdmyHhKf488odIoDDvJr5zzFM6cECSnTsY LnQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723632327; x=1724237127; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fvJpr7p454Y/UfNPCoQA/axDAfLDDtHMeofcmIJVphY=; b=mjn30vFSdZjj8nZ5eKdyrGW+1u09ys1UmMWo0BYSR9n092eZsmdyKBr21S8SA77jn0 U62cPKCOGllYZhY6IK9X+Vig5n2UA7CPOjuyGxLM62C8hJcyIKD5V/k7EjP9iKK/L238 rUgs1KGNVipKNhHtZLdWu4pCwD9d/sDrRc49DpwBhVO1zHmbe44jg0g7w/KLr0+Esah9 nEA+1zqh9Kzb8hDoDnP19tSwe8YViLHYSY+McwCL+keuHATIm/le1RbmeJnyVzGYEusg I0d9LRnOTQ6S5xYb9U0w78b5DXXPwp7fwoeMsIpWiKhyKin6mLxLDPfYsN0hWWbT4JF+ hGfQ== X-Forwarded-Encrypted: i=1; AJvYcCWldDo6YNuu5JY+t1VLi9K+We5T9faW0CezVo2igSgQhEvQ5o5nx5UQAr1u9jEpfnXIEvY7zEKiqY8ft6SgEMlxHqc= X-Gm-Message-State: AOJu0YzPBLIHaGSJY8qS3TixcssKvqDSD/WGJZB5OpHR4vJxhIjR5f8f yZHE1LtcxYzWuHlMdv9/usEJrIrbd4Nfdc9ghrczTuF216CLGsju X-Google-Smtp-Source: AGHT+IF5x7vVq+APZiVW0Zf/kYmxy4TtcaaAPS1Fj1piCtyS5MQvRUzOqgbF+0Kt0wNs5ja3bp6E1A== X-Received: by 2002:a17:907:e601:b0:a7a:c197:8701 with SMTP id a640c23a62f3a-a8366d5b86cmr180254766b.31.1723632326621; Wed, 14 Aug 2024 03:45:26 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.132.251]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a80f418692asm157212766b.224.2024.08.14.03.45.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Aug 2024 03:45:25 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [RFC 4/5] block: introduce blk_validate_discard() Date: Wed, 14 Aug 2024 11:45:53 +0100 Message-ID: <41ac9ff000cbd47fd8e386ee70e8049c3ac80ead.1723601134.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2EF3E1C0008 X-Stat-Signature: sfesmcycp66wjjrew87y4qxtja5j5kjh X-Rspam-User: X-HE-Tag: 1723632327-312273 X-HE-Meta: U2FsdGVkX1+KDTppNTGhlwa0C1HX3cp6HFaWHqB+vFfn4hFbS1jroySpEJeVu8Mnb4qvjQZL5JCT3mJ8BFCNZcc916vCAfXEzBLa/lLIjDrz4MlAu/h2dkA1/FYusM5XkMZOEc8Md4RuaCdpNWvBN/Kp5Wy/ouD9EMektx+9scONXHOXLcM5fl+Mi/jQ8Mlz+4/egmzDO7Ujxghc0I+gdCqP9H/NkGk1pIC+auns+EdXNWsW4MPuiUTO9UAk5E1MiU5yAfHVZa+7ARBqDnv6hhNI2RqeTYN2idwp07zeeUPdkuHCpXZvfH2syyCo4YqyFP8hKN7cULP3BgR0sTBrasnNkKaqx8umF8NYBlY+6H+kWzJa1R7rcKLjEcuuUud1vf0fRSsVs1iojoxx/pVLw829yxZ6Z4RzaI2n5PpYhVwsRmFB0E4Ht/GKWC2nIs/7WkM57Ez9HSOb/KU8CcUyn00xHIgou3wnliWDq0YFmE8XZzk4yxT/OKlDd8U6Aar1cmxq9/Xo6S9245FrVCe9l9+0Zj2fFzWun3t7OSImjh25Kq0xchPrztnO6C3WJwKTRlstC1AulS+nbeQdY82Koe4Z2UKjJlXLFTkackSX92LZ1l7DStYGbS+GuyCA7FHucDjQSlp5JlaEHMHcQUWFX0uK4ZtTq5Qiq/g3gnS+gBJQzyoxJMc9b/rPFmxD6L/AR+EZ4SaHP37Zn9iMmD+psUzoUYDtfJoFLE1n/kWVfdjWOKV2GA104Z5Z0Eap/hFy0CSqUJksfArsNXQS7iPJoR4q2lTzml/tdBODIqGF+aYn4S7wcWhKotSyfNM/0W2Sta/oorLy3eOjUI3xOIVaBzU67TRvjHHYQ3eLK6ePS8f9zUatLuwQE5H2m+CsWaNIBCWJcDyuZNgp0LjSDzSIHp1BcABxDaGBGPQ0zux9l15ljEA6c4PrF650+sLIHUT0QTpSQIEg1FTWYxfXJqv zGg1iD8j mkrX3jy19ddeOKCp8xB/Cmd0Ibpu71wn1+RWojlupcPhHa7xclzMdTTGf1RUpUtWsepStqLnW6dfv8rNP1gkRjKbaSQWSlZilrQk31kuVK8yrrz4tvwDOwGe8EawKgv2UlEXiY7f8WSDtNGInyLiB8wx4Ds9LPoxnkod8cPcCAT+lVT5WX7OZrS6aGF2IGJVcymhhim+PkOcLVtCqvLgVqa+9RJOSD3orrujhpDJzx1onZrREoVb8rXGkKK8LFNjAJS+66CuKIGdZA/oJMCg3dNc+DeZSFgyyHf5opTV/1e5YzQ+7u0X6jqfLKPP8J7+aB2gv8ajxkHwD6NskbXGvNBegl5IVv9iLvyak96kx426cGhccjoPSzHgdb0Q6LQbxJrGqoSgFA0pf9k21xyQUMNUodb5WP1Ey8POvV6m5xdH20NsFFKT88idDm87R44KLKNW4sCPy0iuoaPOqLlHlFpb5OKNE1Q3AAeSuyVjn/strE/J0f0J47piUvA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000280, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation to further changes extract a helper function out of blk_ioctl_discard() that validates discard arguments. Signed-off-by: Pavel Begunkov --- block/ioctl.c | 45 ++++++++++++++++++++++++++++----------------- 1 file changed, 28 insertions(+), 17 deletions(-) diff --git a/block/ioctl.c b/block/ioctl.c index e8e4a4190f18..c7a3e6c6f5fa 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -92,39 +92,50 @@ static int compat_blkpg_ioctl(struct block_device *bdev, } #endif -static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, - unsigned long arg) +static int blk_validate_discard(struct block_device *bdev, blk_mode_t mode, + uint64_t start, uint64_t len) { - unsigned int bs_mask = bdev_logical_block_size(bdev) - 1; - uint64_t range[2], start, len, end; - struct bio *prev = NULL, *bio; - sector_t sector, nr_sects; - struct blk_plug plug; - int err; + unsigned int bs_mask; + uint64_t end; if (!(mode & BLK_OPEN_WRITE)) return -EBADF; - if (!bdev_max_discard_sectors(bdev)) return -EOPNOTSUPP; if (bdev_read_only(bdev)) return -EPERM; - if (copy_from_user(range, (void __user *)arg, sizeof(range))) - return -EFAULT; - - start = range[0]; - len = range[1]; - - if (!len) - return -EINVAL; + bs_mask = bdev_logical_block_size(bdev) - 1; if ((start | len) & bs_mask) return -EINVAL; + if (!len) + return -EINVAL; if (check_add_overflow(start, len, &end) || end > bdev_nr_bytes(bdev)) return -EINVAL; + return 0; +} + +static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, + unsigned long arg) +{ + uint64_t range[2], start, len; + struct bio *prev = NULL, *bio; + sector_t sector, nr_sects; + struct blk_plug plug; + int err; + + if (copy_from_user(range, (void __user *)arg, sizeof(range))) + return -EFAULT; + start = range[0]; + len = range[1]; + + err = blk_validate_discard(bdev, mode, start, len); + if (err) + return err; + filemap_invalidate_lock(bdev->bd_mapping); err = truncate_bdev_range(bdev, mode, start, start + len - 1); if (err) From patchwork Wed Aug 14 10:45:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13763319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E714C3DA4A for ; Wed, 14 Aug 2024 10:45:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5DFAE6B0093; Wed, 14 Aug 2024 06:45:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F0786B0095; Wed, 14 Aug 2024 06:45:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36C066B0096; Wed, 14 Aug 2024 06:45:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 094CE6B0093 for ; Wed, 14 Aug 2024 06:45:32 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BB8DC160E12 for ; Wed, 14 Aug 2024 10:45:31 +0000 (UTC) X-FDA: 82450519662.23.163EBE5 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf10.hostedemail.com (Postfix) with ESMTP id D4A9AC0016 for ; Wed, 14 Aug 2024 10:45:29 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=khXPOoEG; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723632250; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Tuo5pTUVgRqd7+Wu/p/7uLxEPhnfCWa8o0PxSIfrqWo=; b=NQnA1TC0RnpEM3AK8a1eo8/BAiI1pTRymq5drgWjldVdeEp3YANLdgRjYk7wddfT+VljeC lCIPpzaZ4m/5KoxyT2PN3lQKEDrqtMXMbg4HuPoe5x3sORZpFQ5oa0m4eg3mpEnyk+l3bY ODVYASxfmL2Koqo/lk5YHweFxej/KXg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723632250; a=rsa-sha256; cv=none; b=pW9F5QKeKhojSdxeSI4MJXV8xlYn+tZ3fsfipUHZ/UlcFETDmzItdKpIXAXwgHQYkI48gn 59eFSGNj7NYbEO4iHH2oAUyCWIJWoJuhA8JaIxQ3pWYAe7shaWs6x4ZyQ05FvGGqH8J7gh dzFehsMy/iFC4JyD1Vvshia01lk93iA= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=khXPOoEG; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=asml.silence@gmail.com Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-a7d89bb07e7so701689166b.3 for ; Wed, 14 Aug 2024 03:45:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723632328; x=1724237128; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tuo5pTUVgRqd7+Wu/p/7uLxEPhnfCWa8o0PxSIfrqWo=; b=khXPOoEG1rrHH+OHWyrz+KlWx3qV4ypDKHQPlJSZKev9h9UkGQNQh0O7HcU1luQ05g pODe/6QsXpqvzGuG3A5/wot42dmvVhrjpVT0kkAMmP8JDXH2ulOxPUIeAlRncts/z9ZY u7HLnTwctYQM9gWpgt/nUuQWQmcr67lFWmIEzHd/p4t6Q57+YRv6g9QYIQTUw7etFjcz +46fpPFhh7L/739V7am1Yxk+XasAvi4ZQS67uOYM5e+AX0smDE9Se3EFmHvmahOD6meL qWmIglIZr7u6I29aFwvLvxn+1oJ5/g10+uxKi+T0rM4tGc1me3MWheqyuCnFUyaxbW9C IJrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723632328; x=1724237128; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tuo5pTUVgRqd7+Wu/p/7uLxEPhnfCWa8o0PxSIfrqWo=; b=nt3FiFN+PrsN1+y8WFyp/Le7jcETpfDhiYh3uHi0EfKjn+CkMv5H55FCLdml59bKOo +UUfBq5YjB/goT06Pecf9fi2B3B87l9R4bC/1KaKyjkpsN/Mckuho3CXBoVDVEscn70j EWKv5V/JlW+Awn5tW0B5VWP+kUpd53k52o8FavGsGlVMdiKA0d9vUafJXLzKsOTNUYGy qkIKFi1TNCNwR1VzKvUlh+UVDdMNH1g3rqKG822GcCHdHCSGs6JaPkaL5SVDu/AW4GVL H+qXdyShJ1fc4/vf61ljhnrnBRwXTadmK0anQJYcvkdsTao2iwmUMNmKoE9PP0GkZsc/ PIUA== X-Forwarded-Encrypted: i=1; AJvYcCVBN/3vJzvR276wWTmWEYy+LgL5lAGtmHIPJz2kjDRIb0tienk8fuAGMJjCdouK/Ew8PTRqeqwI6M5VM5JcnVMPt58= X-Gm-Message-State: AOJu0Yw3H47CxsjivhGf/iJ2eUs7bJSV7UFnfRJ0CgtNOng7N/bJgAgi d897saAzYI8PA1Z5PbeBvgBqxdZ89ETylz6KBYCcb4XnikLUO3Lvyj1HZldp X-Google-Smtp-Source: AGHT+IG1KkX6wMcxGFIa+KqlcGfRZzjy0lHz5phQrjiJN6wASSWoiIqHMLk5BznC58MpKrZaWW6grA== X-Received: by 2002:a17:907:84c:b0:a7a:bc34:a4c9 with SMTP id a640c23a62f3a-a83670bf16cmr128595066b.69.1723632328176; Wed, 14 Aug 2024 03:45:28 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.132.251]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a80f418692asm157212766b.224.2024.08.14.03.45.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Aug 2024 03:45:27 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [RFC 5/5] block: implement io_uring discard cmd Date: Wed, 14 Aug 2024 11:45:54 +0100 Message-ID: <6ecd7ab3386f63f1656dc766c1b5b038ff5353c2.1723601134.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D4A9AC0016 X-Stat-Signature: x88e9f4qqku8d7m168wan1fnbfs4ceub X-Rspam-User: X-HE-Tag: 1723632329-655574 X-HE-Meta: U2FsdGVkX1+O3SnKDJRZZhSBDoPixVvemDQzLChE8s4uN918UvUxNsXYMahxdU/JT5IiuRZ5fGER6FLufwceyrxUufVugWtAXTnVlhmYryKkKDyq89m3CaOdMOqAnhAb0V0YcY2CMOQYmNbwtTasIhisx4gaBUV/myVPADuNMXJNSRGxBm54i//MXXnJN7dygA5x0WK6iUUM5k8yQUDo8gy1cCcuH2Ws5MF8o6jlaPw9k+cEI2fq7ETt3CIEg1ra/TRlqoMbdPk/9AhnRRI1sL6JppOVJq5V3Cgk5IaAeXNMX7Ycj4ttaVzAmndD4NuI0E8Da9CnWRcZ0XjSaTYV5esiNg/IzzXQEHwSzKfDLusHxh81dqSKSHd1dob4fFhJ9va8OP3BaUyXbXi5EkL6JP8R4II0y8qJTgXDIQwZPNq9tDuGsHFyh/6d6kAG3EILw5Um9/kTMhWew/MSoyF6rSMwcTzS/06pxHtVDxrpbzq6bvEYdp3R8q2Qiyo8wjhoiTZzis45Ze7SoEV3Wy1uqFF2f3qHagS87ke/bo6XhoffWGAQuvTHj7gOAu/bJ7CHOVPzgj//T+7oZvaB0A3sjnwGuQuGKMBqcLn+1PHD4yv/YjBkcRoNomAZcdKClu9uoEI0kfs6Ng4j39Qtade0m2rvN3+E7fQAdgi1Y5A6Es7S+JIdapHZ1UXCENwIhRw1e8v+FD2T8/Nap2+GKylNpb4kmRqSpdzR6Jg1xprRqwheMvrTHC4B79Mxq96qnL0WeuxHm3jLUUSvhTmLiMDr/Xytb8D1lVZz4G+tN44vkuli/6rbtVOhfY1nP5aZv8c4ze6RQgCk/vILfsilNM1Ave89l0dq305asfgWUOq+syywEKJJb0H8tI9htzL3BJKyHidWz8u/RpOdnwKeAVdDM/HVhBI7tUPBYNEmJmtoDT6kMc4OJ1xpjF0jDgGEDjOk+gRKF0u0S1TAj0WFy3d XnWt6D/1 mjrBYcVowKHQJGd27Saz91UQLyIJWbBVarDj+qeldKDuRw5DjojYR07fN0zNCCdXvPYUvJcoC/9FIcqcNQSQYz0DqEdVzXa9WJ9LKbp4scChNlJGJUKa96Rf9NO0bYYvaIxl9Gon//DNKwlu528B5uBJGECnTCwqaVl2YH8hEpFtdhTU4i4RJMlZhmwS5ul9okS7kKWgFOnEZ/SKhFE6PE0dfqnEBLIO4N/80Em5WVHgJHrS23HbU7TofVrDTlE2Fu8YMcLmme5gUcwD9/Rl76b/f/snSlc6jeIpnLyJqupqDZsgrFQhAP87UZoxHxMfPXNg1YY2vHycZ0Ev2XzB0ToJKGfyq4FpnG95QJzbQAn05ctMwT0r+QH6LdDQS564XrD2Tq3zX9/T8K/0d2ipCVlelf5urnn4Ebam0oH66q7z6yAiXq+quiruImOazZD2qVFn2xqrVsV0xoClaRN1YeQHgQMGTJCcJ87ZOCNoE4RYUqUtZ4M5iyWIaqScxYhfn+PywdNfEkSKB+Si9PQgb2fFxaI9NPDsBm7PEtKRPrSWtGjpoIF9beCGkgQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add ->uring_cmd callback for block device files and use it to implement asynchronous discard. Normally, it first tries to execute the command from non-blocking context, which we limit to a single bio because otherwise one of sub-bios may need to wait for other bios, and we don't want to deal with partial IO. If non-blocking attempt fails, we'll retry it in a blocking context. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov --- block/blk.h | 1 + block/fops.c | 2 + block/ioctl.c | 94 +++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/fs.h | 2 + 4 files changed, 99 insertions(+) diff --git a/block/blk.h b/block/blk.h index e180863f918b..5178c5ba6852 100644 --- a/block/blk.h +++ b/block/blk.h @@ -571,6 +571,7 @@ blk_mode_t file_to_blk_mode(struct file *file); int truncate_bdev_range(struct block_device *bdev, blk_mode_t mode, loff_t lstart, loff_t lend); long blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg); +int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags); long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg); extern const struct address_space_operations def_blk_aops; diff --git a/block/fops.c b/block/fops.c index 9825c1713a49..8154b10b5abf 100644 --- a/block/fops.c +++ b/block/fops.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "blk.h" static inline struct inode *bdev_file_inode(struct file *file) @@ -873,6 +874,7 @@ const struct file_operations def_blk_fops = { .splice_read = filemap_splice_read, .splice_write = iter_file_splice_write, .fallocate = blkdev_fallocate, + .uring_cmd = blkdev_uring_cmd, .fop_flags = FOP_BUFFER_RASYNC, }; diff --git a/block/ioctl.c b/block/ioctl.c index c7a3e6c6f5fa..f7f9c4c6d6b5 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -11,6 +11,8 @@ #include #include #include +#include +#include #include "blk.h" static int blkpg_do_ioctl(struct block_device *bdev, @@ -744,4 +746,96 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg) return ret; } + +struct blk_cmd { + blk_status_t status; + bool nowait; +}; + +static void blk_cmd_complete(struct io_uring_cmd *cmd, unsigned int issue_flags) +{ + struct blk_cmd *bc = io_uring_cmd_to_pdu(cmd, struct blk_cmd); + int res = blk_status_to_errno(bc->status); + + if (res == -EAGAIN && bc->nowait) + io_uring_cmd_issue_blocking(cmd); + else + io_uring_cmd_done(cmd, res, 0, issue_flags); +} + +static void bio_cmd_end(struct bio *bio) +{ + struct io_uring_cmd *cmd = bio->bi_private; + struct blk_cmd *bc = io_uring_cmd_to_pdu(cmd, struct blk_cmd); + + if (unlikely(bio->bi_status) && !bc->status) + bc->status = bio->bi_status; + + io_uring_cmd_do_in_task_lazy(cmd, blk_cmd_complete); + bio_put(bio); +} + +static int blkdev_cmd_discard(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + int err; + + err = blk_validate_discard(bdev, file_to_blk_mode(cmd->file), + start, len); + if (err) + return err; + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, nowait); + if (err) + return err; + + while ((bio = blk_alloc_discard_bio(bdev, §or, &nr_sects, + GFP_KERNEL))) { + if (nowait) { + if (unlikely(nr_sects)) { + bio_put(bio); + return -EAGAIN; + } + bio->bi_opf |= REQ_NOWAIT; + } + prev = bio_chain_and_submit(prev, bio); + } + if (!prev) + return -EFAULT; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_end; + submit_bio(prev); + return -EIOCBQUEUED; +} + +int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) +{ + struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host); + struct blk_cmd *bc = io_uring_cmd_to_pdu(cmd, struct blk_cmd); + const struct io_uring_sqe *sqe = cmd->sqe; + u32 cmd_op = cmd->cmd_op; + uint64_t start, len; + + if (unlikely(sqe->ioprio || sqe->__pad1 || sqe->len || + sqe->rw_flags || sqe->file_index)) + return -EINVAL; + + bc->status = BLK_STS_OK; + bc->nowait = issue_flags & IO_URING_F_NONBLOCK; + + start = READ_ONCE(sqe->addr); + len = READ_ONCE(sqe->addr3); + + switch (cmd_op) { + case BLOCK_URING_CMD_DISCARD: + return blkdev_cmd_discard(cmd, bdev, start, len, bc->nowait); + } + return -EINVAL; +} + #endif diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..0016e38ed33c 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -208,6 +208,8 @@ struct fsxattr { * (see uapi/linux/blkzoned.h) */ +#define BLOCK_URING_CMD_DISCARD 0 + #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */ #define FIGETBSZ _IO(0x00,2) /* get the block size used for bmap */