From patchwork Fri Sep 6 22:57:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13794831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC381E6FE4B for ; Fri, 6 Sep 2024 22:56:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6EFA46B0088; Fri, 6 Sep 2024 18:56:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A0BE6B0089; Fri, 6 Sep 2024 18:56:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51A7B6B008A; Fri, 6 Sep 2024 18:56:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1AFB86B0088 for ; Fri, 6 Sep 2024 18:56:59 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B1BD712125B for ; Fri, 6 Sep 2024 22:56:58 +0000 (UTC) X-FDA: 82535825316.09.7FA3899 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) by imf23.hostedemail.com (Postfix) with ESMTP id CE49314000C for ; Fri, 6 Sep 2024 22:56:56 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RRjdYXje; spf=pass (imf23.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725663286; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=XJUQTCjMsnqkF4XJBgKWfj76h9NgdidLqvuF52YMg8cy3QWnAGONp61GbO+aa96exbderF vWTCeRXhc9RIhIN+SqA3t3RNRSTmQ0+qnOCfWFYVfnwaPrddpdZYGOTQy2O8Z7JqMNJ0A+ +qIX/nW2V/W9eTKPzXtysbCBIBVkSoQ= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RRjdYXje; spf=pass (imf23.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725663286; a=rsa-sha256; cv=none; b=I/aQJIuJ/RKjSGVlv2eyd/sOmaUO/Z9K/DMnPlDkKWiNg0zIpNUMale2GOZiQnxFJkevRJ 7ItBzGo55LcdA7P5/feBmVB5AwkLgkX8Gj8M4iFkycULR+gghKbovw5jK80P6X8n0h1JeD v3FwSKJqvQsvuPRVViO0JfEDwxCxsuw= Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-a8a789c4fc5so348860766b.0 for ; Fri, 06 Sep 2024 15:56:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725663415; x=1726268215; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=RRjdYXje0r5Wzc+v+i94lD8njLJBMv8P02QMYYf0DddZPIfClMWSGAf4eficwtIVGc NXiLcur/OmkEn/ueFkStDkIBsCchy3+6bb85/bxR4pnQ6BLbn0t3r/tDRdipMLyVnFpA uGzV6DVIBpXDQXCk9FENM93pQnxQXZf6sdNp15U4/Y9pc/KOdgaNtVm0iZEKEDhQrYjL y/DesX0P3dZiX4SNAok0jhELrgiXgpA+vuC01Fh238X6fe9ezDXmmElLgpZErmXcjt1j 0fNxN/1004WZ48BkNs3qqc0rWUZdntvMBTSb9yDWCEg/N/6EL7xjfQkGJouXMVoRIvZJ IEcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725663415; x=1726268215; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=ELlQg/5766RdQ/OLuyeFxbMvK55aH3MyZ4EEKJbtkt4BerCKDWGC86lgw5k33FGsF2 en+YV5pnOi3iGL3VADnc363tQ10I2HeMLPpQ528hpLPMJ4GDKR4NoxDHF26gVxf+dl/m pQrMAhsh2j7a9i7vAlYv2hSiwxToiWDYlmoJ/3dsSUlY1IU+4F8SmWtcTaMkbt8lupeP 9foDp6tSYTxq0MdRA7aIQsJAbQFGNdK3mTe5Mq9z07/oix116RWQkoxMv6+til+9C26V mwfuOf7Lpfl3dROEy/HSzxfB4dk6LQnDu+flvEBGykOTKhC0DY0CpfQTn8CYvk+S1ken 4r+Q== X-Forwarded-Encrypted: i=1; AJvYcCV6TycbqKi9JclImc8aTsbnwTgq/iLrxrQJCRg18L0a72NFqlKiyeOIa9OcgaTlzl0ebYvbOe4fVA==@kvack.org X-Gm-Message-State: AOJu0YyIfCH97euresK51sX/ywovX81xGT3JlDN/g39HSsrIjs5wMDmf 28QWtGDLi+rzVGWOiS/dn5wGydJd0A5FU3qWD7/Ufe2Qj3jh7nlCCm7PGZcc X-Google-Smtp-Source: AGHT+IHyXvN1ue7B3ODRoTm6YInUy1vTPF5mt5XLGTBYTGrDKLCROG5nKHfbHCNql8pi+pUWW0OV0g== X-Received: by 2002:a17:907:2d2b:b0:a8b:58e3:ac1f with SMTP id a640c23a62f3a-a8b595ddc9bmr313555066b.12.1725663415255; Fri, 06 Sep 2024 15:56:55 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.146.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25d54978sm2679566b.199.2024.09.06.15.56.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Sep 2024 15:56:54 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v4 1/8] io_uring/cmd: expose iowq to cmds Date: Fri, 6 Sep 2024 23:57:18 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: CE49314000C X-Stat-Signature: nqs1msyqimksarberwxzcmaydu3ymwwd X-Rspam-User: X-HE-Tag: 1725663416-484298 X-HE-Meta: U2FsdGVkX1/GQn96aUfN4O68pmzin0TzhlSMQvoG8YXSDBYE95hGewTY/Kn4WTFOia1LaVgONJ08sVmkhXKI46MPn2yyGZuWB1Q0iDDJdhtwaT3IGRgzRWF4eIzep7Ysz+eRNKtTP1sdSUECx9Z1jsd/FBhYZjRRq7MZ6ztNSb8E/0gyqomi4yy3910WGhrYzq5/Gk4qUM0aqfjJGxUnh8vx6JAFQT0pfwWFuXjtidhvnd2FIlL2C8ZHyJqEUhfv1L8+mHtWz8bnob4QIjwjLkXS6e78Ue+47Ch2+p2M7nNvVHr2cNeRiED+R1KUH1h+LslPQIKky6VIrjMsR9na2uZxTQikq0xmkMd7dt+SgZGWSbiMxEdjsa3v/bqxvIlGCTjO1+VN9ZfLTCFLaGSk9KiNfygNyNno5Kn0GbIdG1aObuT5H0BseFIRzEzUoygajs3EiBkZea7n0plBhZP3es1NZi7DPiyPJ84Wv69sFdUu+iCXW3PL8NReUGSVV9HRPe68JMAIsOsnxTpEshMBfeAwnLEusPQoTO9PQz35JiHckoVde1B99Hnzw3chmEiFKvT/E9jn6ckm9T9lM9kt3T79dqrMSXStVfxydnAI0gq4cC7246VsT5lyG+URSUV4NLqte+Gb4K3HBG9FjnHCu5S6lme3TqXsOgf+9mlLAFoaDVGrOJAamnPclNKHCXSiQkHLS7AUuOxWwWmbqfTIs2Ewq1W4JTnMYU4bvdDbHFxe5zozBzNgxDQJBo07A/UWS6SvlHpd7XBW2z0LsAFuoE8w++WcVxTaQxjGCpcjqDY2tPMmmF7ypdCrF9Nkd12hNGrUNVYFWPdyT4jhJ69kVWHy0A6FwghKT9ED4X7AXPahTBef+eB72+9F/ZIh18YsBgQ8Ock4jcHN2A1PpIQdO34X76l6YBvumGqgR0NgRwgxBWBH9YfaeJVbhjDGIEduIFhWOpTyngBN7ATfTQo 2hog6bLy BcJ2IzDDSNrM3+q/eP9NT1iT0pRrHRQn3gjDyZxMrpikR9L9Ioj9nURi3AbjICBO1hRQ1gHdBtisYdFNQNiGtsIKt2K91jKDSWwWO2ROaDGA/ZxTV1SLUAVTSPI0Ux2Do+o9GGqDGN8ZfUBARP2eCZ+lQVIVGcHDFDMlJ3HBDcAvVbyPwMeyLKCrOZV5yExXKBYwZwMmzwEjMbsW14bkiLWkdGy5uLKTI9E9aG+2IpPwTu3vum8qePO+bVUingROVsnrJe1JJMIjWB2eLQUBtpON6UH2BarGP63KZrWBBviHBYaDc8My4qP0bf+bIR4xPVZVn56BWU8MXejkk8z7h2iyKU2K/N6gxOZZRUIEYNS9pl9WoBmTA8RgdBWrH2STqlY70VnzStSkGHBYBNs7SzbC3mmwLAWo2IQ6CJgnblVPWVDhEFaTNu1s3AG/5CH+O0KSGtnyLVqQpLJw/IOViaPsQGgRhRSWHRvh7R0vmapD2izlncg1GdtRECQSIy7W1etp6BFtVIWFs+MzsNeUhAFrGgw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When an io_uring request needs blocking context we offload it to the io_uring's thread pool called io-wq. We can get there off ->uring_cmd by returning -EAGAIN, but there is no straightforward way of doing that from an asynchronous callback. Add a helper that would transfer a command to a blocking context. Note, we do an extra hop via task_work before io_queue_iowq(), that's a limitation of io_uring infra we have that can likely be lifted later if that would ever become a problem. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 6 ++++++ io_uring/io_uring.c | 11 +++++++++++ io_uring/io_uring.h | 1 + io_uring/uring_cmd.c | 7 +++++++ 4 files changed, 25 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 447fbfd32215..86ceb3383e49 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -48,6 +48,9 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); +/* Execute the request from a blocking context */ +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd); + #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd) @@ -67,6 +70,9 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } +static inline void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ +} #endif /* diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 1aca501efaf6..86cf31902841 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -533,6 +533,17 @@ static void io_queue_iowq(struct io_kiocb *req) io_queue_linked_timeout(link); } +static void io_req_queue_iowq_tw(struct io_kiocb *req, struct io_tw_state *ts) +{ + io_queue_iowq(req); +} + +void io_req_queue_iowq(struct io_kiocb *req) +{ + req->io_task_work.func = io_req_queue_iowq_tw; + io_req_task_work_add(req); +} + static __cold void io_queue_deferred(struct io_ring_ctx *ctx) { while (!list_empty(&ctx->defer_list)) { diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 65078e641390..9d70b2cf7b1e 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -94,6 +94,7 @@ int io_uring_alloc_task_context(struct task_struct *task, int io_ring_add_registered_file(struct io_uring_task *tctx, struct file *file, int start, int end); +void io_req_queue_iowq(struct io_kiocb *req); int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts); int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8391c7c7c1ec..39c3c816ec78 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -277,6 +277,13 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, } EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ + struct io_kiocb *req = cmd_to_io_kiocb(ioucmd); + + io_req_queue_iowq(req); +} + static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd, unsigned int issue_flags) From patchwork Fri Sep 6 22:57:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13794832 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E7BDE6FE49 for ; Fri, 6 Sep 2024 22:57:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8A08A6B008A; Fri, 6 Sep 2024 18:57:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7FF4B6B008C; Fri, 6 Sep 2024 18:57:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62B306B0092; Fri, 6 Sep 2024 18:57:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4276C6B008A for ; Fri, 6 Sep 2024 18:57:02 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B5A121611A5 for ; Fri, 6 Sep 2024 22:57:01 +0000 (UTC) X-FDA: 82535825442.19.96A08E9 Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) by imf02.hostedemail.com (Postfix) with ESMTP id DBAFA8000E for ; Fri, 6 Sep 2024 22:56:59 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=KLu9uuQZ; spf=pass (imf02.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.43 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725663370; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=Mlc9mARmezwImZp5W4m09oVNuZf+CSyX7yilNyuCb1JquiPiw5rsiZD3urDcIh+X3U1SfE yCGRLs6OZ92PaVKIbTwkBai/2qPEyNnD8fBY+k28Y5S0vAu3Q/9yvCaiy+jpYFOq9oylrK JtpL/6M9OKwBruqzvU1/+vHbLuTCKMU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=KLu9uuQZ; spf=pass (imf02.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.43 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725663370; a=rsa-sha256; cv=none; b=DDET5dbMLn7Rh0RrSWYPJ9lUAahD2LfNK1vnZEwXMUS8oQMYmczAdUeVi8ZB+K2ZtuGEpI qwoHIb2m/Hl8YnKxwFoFF4Fbs96+0G5161MjEuc7uEcklzV9RN8Qel9SKxDLlOFKz1H4KS zxVD274URZosUyFKln7avOQ8AZ5N8CA= Received: by mail-lf1-f43.google.com with SMTP id 2adb3069b0e04-5353cd2fa28so3004924e87.3 for ; Fri, 06 Sep 2024 15:56:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725663418; x=1726268218; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=KLu9uuQZ0u2h9qVWiRscki6SURUEut3+pZEO93I02PU0SNzq7R00r3nykW+BcmH8xc FB+Bg8PVo+fXsSRt4SmF0aTDTbAS//5yEdLA89BHaA8/LTBVxGlzH3976v95ZQgHPDTW NtoOcEymFKLn0s8A72YFsuYiPxmX3ptIUPGkNwSrvaTmd1hORmN+/oCFmt6wG6x1stYm Nb7teliREyvwfULi+yR/gt2znPDRx507tsmByFjhxpFyDf0mVsq5vfrqTUKk3Yy/DHa0 juQGw4ZZsARofKKnWbGeh8Vbf0xWGX+6AFhBGCPw7554WUvKSNQIxbHXVrkm0DpAJy94 gPVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725663418; x=1726268218; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=T8UNPCVwiNVk7/EebL1XjwBFoRV+J4EtmR3wNY/o3uL7HPi8OOo+hefkTDBUgKQiw8 ZljVdBI3obUeOP4T1kCfSUe7Kj2Bsk4FO9z14Egj+KYu4kSDVyIhc4ZFqPF9KFQ0+G27 28Z3OUDrRa8jqCsvF1VdV92zLSsSyl0b8AN3rHEv6mFGpBU2P/voBV26WRzwPr/7gaO7 JTqtayrgH46N4SRuD4c4zGzXSvMP4RMTQU1x3MYtFuPKNCcWRKpSAFN55qH5kTBSeVsi /+nVUVrtHvGjUN58xdtuUTBH+d3NKUO9ENwoOJyuv8AWXhZ2joJa9yNTfGt9w0/KWC4v jwDQ== X-Forwarded-Encrypted: i=1; AJvYcCV+7ogp9ARiSbYGYpw80P5NZyoLQIe8uhPTNHiiERYith70TPvFEIntkzA5ULtsGG6OsAihlmMPBw==@kvack.org X-Gm-Message-State: AOJu0YwYd8mdM5ArHD1sCuDKdDec/rPGBH56HAQ+8MRXSS1V93jj3dk/ T3sQn9dQWdUBfkMxeIRzwNTzbdmVounzwD/yg7YMfO8GgsIA4kYC X-Google-Smtp-Source: AGHT+IHXnXfbkaF7u3VX1p6hb15jHuT8Sz6pGMKxR+NPrjTVAbIbg44Z7wW3wwKJC6QGwAd3IjyH4Q== X-Received: by 2002:a05:6512:e88:b0:52e:7542:f471 with SMTP id 2adb3069b0e04-536587b8ec7mr3170548e87.29.1725663416985; Fri, 06 Sep 2024 15:56:56 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.146.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25d54978sm2679566b.199.2024.09.06.15.56.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Sep 2024 15:56:56 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v4 2/8] io_uring/cmd: give inline space in request to cmds Date: Fri, 6 Sep 2024 23:57:19 +0100 Message-ID: <7ca779a61ee5e166e535d70df9c7f07b15d8a0ce.1725621577.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: DBAFA8000E X-Stat-Signature: mdtfa6swmuik9cyc8tdinx8swfqjy47p X-HE-Tag: 1725663419-529889 X-HE-Meta: U2FsdGVkX184fRIJFc6yzx5nuqAC8SNKXsDNSAHlSE/0JJZi2KWdKwo3TFepNjyKeAcjY9aw2Nd+2GX55/So5a1At79pyldYdZQLJ3xLuIAE/9oXscX9CP7Fb42GykX0/wbDNo/uq1BivSGsPk1q5YrVYDh6rBUWIY+pCOFZMF6mSosZ4FhoMHiGwc74ijJWZ7tsyGVI+0LGNndJN7wjX4cOPhl1jSf9RyrjTxHmY/btjkO8ZM4HvBYJQw5rCXqJA9NZUFeQqh+Aq9VtqmRWBt6+YLV9EQ743MGJya1mgUuLmbJTFIROtGsWBSXpWmDgvtkFFHKxc4SY1V/tbYWrUMX7nPzqruCHjNJPI1wr9mIsDAXwgxihXXh2q8Yk4kFCdsXd0Pcl0ZcRk3tEYnAViaJeKumXs4R/pkpw175iL3c8GOFWDiwxpg/k1FCcHtcxmnQLaeRTL5FMnmLgGdfOEseICo2NmvIzByQtEOCVaQsDC17Kpbdui+k+P6clsp0rid7XQzQ07h126zc/88gy5lI4ZgODKBXqt+Mdg686GY3TsSFJYxQAsdcqfArn//Q5G0rnmTx+4B8co9CgKC/JixTHSvqrlvGXEILfq3eX4YDRBCSVfWnsk2i8pgAeJ6SFTdPUQegZRXwtCIKyfRPEugytoUp3rsUNDYlGe1Kx5dlp4/u71Nm+T+ERI4r7mAm9npvcZ+Mazng5OmC8ZQO5VCuWjas2aO5AF2qo6MDCNHhIshaKv94LTsEapoSNnfKaE3tGp6K0qsZfsPw3gPd0qkbbuKMxxJPMl4rIt8EpzUayVxQIQWedUlbCWreLFOL4jgLMPBy5ulZBRw0NwYoulbwg2E+GNEssbbMrNtse5+z8q5Xr2w9SiHZnP2YOmUJajdubOPBqACnuXqZRHVnk0hAMfdGBr2X4VOh6ooPr9oLVoCVldUIQRLb5devDvSPFnEcImuPN0GGxL7U9+Ba hqIksCjA aXk0P1ohQtOfiMtee2f1tmC93Cfq5GsDcdDW1tk7Bi5wVN720eFo67lbNs0xc7fki+fAn55TH7YmGHBQ1L0EhPi6sT7t0wIbWnKSVruPqdo2jmE7D7ZE6PkaYOS2S0xlauSNrxgKOkvJaY83ziR6xC+rYjesxxhpWpUgLbz26rSO/OJiThMLMioRmtJFhKX2GOTE+1q3xBySez9okD7kgbDJeqfmm11ixkEcejw4mK2EeqGTC304PWxQYRrvS6zQ6BOBWFJfMOYnrS+EVOgiR1S9j++MiXNmnwzAOJDkVLW3WX4gWS5jJJ1wSQxphRsRpx20ydT+DOVlMWNQ6PmPW+ODEXoJJELYygx++sCqJg87OTvliK7Amk8TslVARLh7BQNgWr75kT+mJq9Bp8HEg5vKqknQJCmC6yT8w+EjabeOlBUm3bGPCNraRm4AZFGkE2qyQnsWleSyaXXXHT4OucB8HOvlhn2sIK+/aRoZkYiWqwdziBxiw8qG3kA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000010, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some io_uring commands can use some inline space in io_kiocb. We have 32 bytes in struct io_uring_cmd, expose it. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 86ceb3383e49..c189d36ad55e 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -23,6 +23,15 @@ static inline const void *io_uring_sqe_cmd(const struct io_uring_sqe *sqe) return sqe->cmd; } +static inline void io_uring_cmd_private_sz_check(size_t cmd_sz) +{ + BUILD_BUG_ON(cmd_sz > sizeof_field(struct io_uring_cmd, pdu)); +} +#define io_uring_cmd_to_pdu(cmd, pdu_type) ( \ + io_uring_cmd_private_sz_check(sizeof(pdu_type)), \ + ((pdu_type *)&(cmd)->pdu) \ +) + #if defined(CONFIG_IO_URING) int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd); From patchwork Fri Sep 6 22:57:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13794833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1620EE6FE4C for ; Fri, 6 Sep 2024 22:57:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C1F876B0092; Fri, 6 Sep 2024 18:57:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B599A6B0093; Fri, 6 Sep 2024 18:57:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D3BF6B0095; Fri, 6 Sep 2024 18:57:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7D8866B0092 for ; Fri, 6 Sep 2024 18:57:03 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 261931C5E3E for ; Fri, 6 Sep 2024 22:57:03 +0000 (UTC) X-FDA: 82535825526.13.607AFBD Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by imf04.hostedemail.com (Postfix) with ESMTP id 4969340006 for ; Fri, 6 Sep 2024 22:57:01 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ci92UBGT; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725663348; a=rsa-sha256; cv=none; b=ErK8Vwp6ybOW/8R1Po7v6S7Z/ZWumlgDAjf/KznUTF+qyQb370WJzalP9/FgIoo8Z4HnPY bO3MNoCpTewni02u3rItfTjRA4F70uK9oa6/mkgAEE+SKdebix1o8h4yx5wBQ6R/SqPHyb Hpf3jp79/tiFQZBpHt0ygCotzr0hEAs= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ci92UBGT; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725663348; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2XjW/1z5M+e5L1HlSBo37JIC76JYeYPBxc+HZavMul8=; b=QKXgo6Wd58l956KDWoPssKWLE2sE1H4pNLC62sk4XtsQ8ib9PKhCViCfMQaDxQxarduz0j zew/cd6aF+hpUKzriJsfiyv5oUR9cQWafHL81anxPKYDgYmzbv/Udj+dd5bUOzd9qbj1NT GlZUukNasnDo2HNITVbzjaNVL/88iX8= Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-a8a7929fd64so286558166b.0 for ; Fri, 06 Sep 2024 15:57:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725663420; x=1726268220; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2XjW/1z5M+e5L1HlSBo37JIC76JYeYPBxc+HZavMul8=; b=ci92UBGTS+FDr7aU7Z4a/i9KSiv1sM8IZlTBkGDm5Rs/gzJC82XkjY+G4LQusffuE8 ZNM6cnuFMVlrkjop2XXWWofSf1IFCLscQunyl5IansE8ZrmRvd4s8pbXCqF1WED2Ex5P Lfb82sRWV+ZL9+X5LslcSp+FvRcXLyNAnRykdxP8149tioLrIeoEaCWUIYWM7SbHi3QK zACvKBtPLmpTXarfogblWsgbMswt6ZPqUxQjSpwg98pqH7dDDUeu20E5/L7K61tRFqYB ek9Xv3RMxFF+CRBC0ynhvMWoO4zvagGrRaEXAOPLIKxwC7Annnmfu0l0uhZER3oM/Zge a+uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725663420; x=1726268220; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2XjW/1z5M+e5L1HlSBo37JIC76JYeYPBxc+HZavMul8=; b=qzPk3MoFu8cOiy2IpDmiddky0la+H4n53yKEzLayntJo/5d7gN4otGZtzCXfe+L+3b 0H9M5dcrCY2/HlkgapTC6+tCjNbUiYHPMG4+yxuR9/SVzPi4UZgN/8qr6D8DuCGj+Q5c rl1NvZX9uSA/+nLDijZ9LGXTJSLMdEeH+uxW7epkNDKHfhzvolCd1hztH7x/Nf+Vl0PL WxUZjaRfbUFgvg3JT1y43hEiq4i/lfCTLfQMLubA18pywtcZTfNsGNlzz8p/vFO1+M0h lZpPMRwh939T+uUv2aV9C5Uii3JKtQzPPnOEB1p5Rr7PV6H/LwFQ5OLkW+IaY/Lp3AFe L41w== X-Forwarded-Encrypted: i=1; AJvYcCVzp82eni7AxK5SzUX+qwZsMVPAL8zEYPMUmUYdBggvfMCgtEO0vPoPoxISxLNry/uvL/6cyc8fFw==@kvack.org X-Gm-Message-State: AOJu0YzOd95rORlIy2fei1Bg62NNPXn5M68q1HiGDdwrzkWrBoJvmQ6P eKeyCGpVNJR+f2Sh+FNcP1rtgWvtl0QUz8kFqCAzQtewbICIQeTD X-Google-Smtp-Source: AGHT+IElFIAZCS4JhFrJt5ylxD7009i496r/jtdVHC3NBh65yo7l8qVQPQsC4GCBaH1S1XbN+0jHTA== X-Received: by 2002:a17:906:f59f:b0:a86:b32f:eee6 with SMTP id a640c23a62f3a-a8a88858b21mr338191766b.54.1725663419302; Fri, 06 Sep 2024 15:56:59 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.146.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25d54978sm2679566b.199.2024.09.06.15.56.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Sep 2024 15:56:58 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v4 3/8] filemap: introduce filemap_invalidate_pages Date: Fri, 6 Sep 2024 23:57:20 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4969340006 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: o5nqqnznbhooaoionqr4k75y6658zkme X-HE-Tag: 1725663421-469559 X-HE-Meta: U2FsdGVkX18qodL7XqKVRpAooBQrGEaqNEMCj2AODPz8tj9BomHfWte2IbN/WTII2ozG7PexmqfgQtDJWfCO843qXT8AdSle/PIB2p9a4q0MEsTO2LqyJ8c/9IJ5H/o4bIgGZcFFw8pYCpcE0pFA/HqyyTTfm5SaXBc+S4LsKwfwjYcT7HDHV2Dtzp1YCF67LRdI+d4yBRbjj+l5Ok4YBgl+mVkPQ2LahtrCIgHNGRqMtR+AGDbSZDJ0QJnYYw2dvYRRvQQylqH2DDcO8Vjsp+DXZbJuhDl38mpWCZZ90dpA5uzdXP7pdHnyCDbV9Ov/rY89Qj5xenFqzeHF86D+NSU56KCcGGyNPX2Fng2ksKv9T+yGKfeEexjQoR2317Q08vH6nPj+SPSMi5dKH+dEmtEicnf8+HN+WiE8EoKNGvFsc31hxlM3iXKCvDN8V0rBJwqgwVEe26zQxU2+EOaWgvnhAyHWxoEHZgP/0/gG1wsvzVR+pYseqgA+7ti++ePWFw0BpMEkUAXz1Su7UVxboX1PW8ObHZ/GT9BrnmjCgLneSomIbrF+2dt6Ua5Gm4T9RcJsDd3tPipiZAy6lH334HhcV5QIg7OH/GTXYoySWBNQVxl0gTBOEQWeeaW4v3xGZWD7MiH+Gx50LSe7CaV4fMIbV48OfIy3P7RV/N8dMdTP4P5kUeX2QKFCu0DYmirkqy8IqGG3gtIZZ/Sp7yrbQyALoBKOpLfhn8g3a6egT0qS6jd/F4gD4KDzBWredEU0knBpLwMmFSGF3LViy/iyTLIP8FxMCD5tCyZ0wjjPUyi2HqyqS9uCRNjdK4g7SOVaMRY7gMreTlVv6BVbprt2y6WRnrc41xUdtljd/IsXI+R0+8qsVceJZr1ThmdcYWa8X/3eo0RgZsg5UwxOt5z8Nv+diun0u11VumL3BuzWVTFcTH/onQUrMP8YpiVSsY19FGEItptQTGYyng/IIAF Ho1pcyBc 4LvghkNOQCVW/Ofdr8b+i2KshXai8d6aqdwA0+miOTVJitq4iYQ+5Xyexnc/AIJDE3SHV2oUy/pQwmKk67nuK7L5INZk1MkqkYsSszFj6LknnNrlhrj3yclFZCVMb401lkQFX7tYisfgtr/bXYxDt9lMN2hBrn6H1cjyrcDpFUpKxOP8A8+b7tv2Cw+PaBoaGuDNXjhKCO4Hamg4HgHNS5dtVgXU2ProdyEdAGIOuASGFNqDFe3k6eVUhIn12xcwKc+ssKwadiCLSELsV7rTUjN0AiD026tw2L9uBtAl5l/v5c23+cmWDfGfw3EvBaM2lwGTFnSC8HKM5APcuw1lK9ldfr69/ULpluyZ9HPaVZdcVOYEq8fC32xRbDh1MCUNnZfJka+ZN6LbAs3HRA8+GUuALMcXd9P0JPlS5pKsvLdGTKV+x/xmxn7TJ4AZg7HBPeLRh4Wl7RiIIpqtJAZBDBD1Q265xRdCewJ/WqifcefgAW2QkHfnz4lWgWi9LzJ6ieo22+nr710039+Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000178, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kiocb_invalidate_pages() is useful for the write path, however not everything is backed by kiocb and we want to reuse the function for bio based discard implementation. Extract and and reuse a new helper called filemap_invalidate_pages(), which takes a argument indicating whether it should be non-blocking and might return -EAGAIN. Signed-off-by: Pavel Begunkov --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 17 ++++++++++++----- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index d9c7edb6422b..e39c3a7ce33c 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -32,6 +32,8 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pgoff_t start, pgoff_t end); int kiocb_invalidate_pages(struct kiocb *iocb, size_t count); void kiocb_invalidate_post_direct_write(struct kiocb *iocb, size_t count); +int filemap_invalidate_pages(struct address_space *mapping, + loff_t pos, loff_t end, bool nowait); int write_inode_now(struct inode *, int sync); int filemap_fdatawrite(struct address_space *); diff --git a/mm/filemap.c b/mm/filemap.c index d62150418b91..6843ed4847d4 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2712,14 +2712,12 @@ int kiocb_write_and_wait(struct kiocb *iocb, size_t count) } EXPORT_SYMBOL_GPL(kiocb_write_and_wait); -int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) +int filemap_invalidate_pages(struct address_space *mapping, + loff_t pos, loff_t end, bool nowait) { - struct address_space *mapping = iocb->ki_filp->f_mapping; - loff_t pos = iocb->ki_pos; - loff_t end = pos + count - 1; int ret; - if (iocb->ki_flags & IOCB_NOWAIT) { + if (nowait) { /* we could block if there are any pages in the range */ if (filemap_range_has_page(mapping, pos, end)) return -EAGAIN; @@ -2738,6 +2736,15 @@ int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) return invalidate_inode_pages2_range(mapping, pos >> PAGE_SHIFT, end >> PAGE_SHIFT); } + +int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) +{ + struct address_space *mapping = iocb->ki_filp->f_mapping; + + return filemap_invalidate_pages(mapping, iocb->ki_pos, + iocb->ki_pos + count - 1, + iocb->ki_flags & IOCB_NOWAIT); +} EXPORT_SYMBOL_GPL(kiocb_invalidate_pages); /** From patchwork Fri Sep 6 22:57:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13794834 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3173E6FE49 for ; Fri, 6 Sep 2024 22:57:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9AFFC6B0095; Fri, 6 Sep 2024 18:57:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 95EE36B0098; Fri, 6 Sep 2024 18:57:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 763CD6B0096; Fri, 6 Sep 2024 18:57:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 513E66B0093 for ; Fri, 6 Sep 2024 18:57:05 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 070A9A98D9 for ; Fri, 6 Sep 2024 22:57:05 +0000 (UTC) X-FDA: 82535825610.07.C155338 Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by imf20.hostedemail.com (Postfix) with ESMTP id 340561C0002 for ; Fri, 6 Sep 2024 22:57:02 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Wpq5taj6; spf=pass (imf20.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725663292; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gCQ1sFR6q4N5GvvlFRQ23FRxGveAlTZ5FLsIiX1fOqY=; b=l/QExP9rtpAjraqmL9xmD+2XZMvf4DMx+NJNkM2aT6DuuWVedXiQvQLUjSf4b9ojdMBLRe mGt0Jz+CZyj0lAVTwDA+t+L2g9nSxY/3oDAZ6Xnm1YdY4c/5hveMsvz8QEzkMxG6PVmnmg p3BbG2d2T1EcQw7mNqP1/ck7cKh+eMU= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Wpq5taj6; spf=pass (imf20.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725663292; a=rsa-sha256; cv=none; b=zMBiUNY0WYSrQU3JyfhXQqhOIQOyYHNDgbX8VKp301s9OThB7pK/ZIzbihzoTkD7RrwhGp x9IbXIzMsvTsxQKxHKn2093Zo/khZEvBHwksGLeCqdLXef1uo/B9Dfg4qYU+ZHcf+DR+RO 1MRHbPy+Z5HEGVjiCjNPbJwBMYk+CqM= Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-a7aa086b077so261311866b.0 for ; Fri, 06 Sep 2024 15:57:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725663422; x=1726268222; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gCQ1sFR6q4N5GvvlFRQ23FRxGveAlTZ5FLsIiX1fOqY=; b=Wpq5taj6vUEiIR2/NoZgHgS4VfiTorpxiGn7AkDD2QO/5I0wOtLflYXBDbzzzWkmYW G/jjfpalTUqmLVaqiKALE422ajRlgAvwvP/eJu4YN1PqCn2BTtyGS/HqTVF2Wds3DD6j xJuyuhxojc38FlnzH+Gd6I/R/QRYlU82wHtkW4DR5xWz5E0B5RHJkqnjWoMK5FlILCBk 111x5vUPxWoNjTjjbn/h28CvGz9RjQMl5K8zy4faTcBD1WkHqj0ybhbDi7McPokuGOWt /ma4lyD6VEOkE9xv/32Or4dEL+Vc3Ef0jEOQWYDyL+if2y4FZ4ZhZNON965mxouZNcO+ zK7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725663422; x=1726268222; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gCQ1sFR6q4N5GvvlFRQ23FRxGveAlTZ5FLsIiX1fOqY=; b=wUbrfM+q2yfF8cRi3+e7SxJHaJDy2YuNXtuy/mbZK9oyTz0gUqVtHrLtxMVkDLhFsq 10XVs0q0laRNjSxehutf8cMziKxFUD1j0JT6fd4gW9QS7eGyIqxm+0Rc8euuQHKXn6ox INhhsfReLyB2b3P1vtt65tMnZ+OUwp53XGVSqfoTf+Ujo9IlroXsfynMCYvkEDI2WKxr V3zezGNdizqkbvgAW/FbzY4tY0SjEx995nqRXQP9IDCHp6qYG4Yf3j4XiC5yG1+fDDS1 cVpIUR2289KDdyTaJgdK8YZs8FMB3ztNPF4lz11T+eNgtRKxUekMongK9w4QuuaPquMG sR/g== X-Forwarded-Encrypted: i=1; AJvYcCULrNMRhivLnsHJ87rHqBWgCPrcSYlNP3oIX9wmw0GV6GERm7ND0Q9hMwEAPiO7O31u73VEUCJx9A==@kvack.org X-Gm-Message-State: AOJu0YwkxCP0v/vRBJJGGQ6bRuLXB95+VvbSrcoTvuYFAy1y6Q2R9qvs M0Cww8WNAcUpSKz04gqEGNY7DAiC5C3SMO2knHP5JQ9p2KHztF0W X-Google-Smtp-Source: AGHT+IGJ6gzQkF2S9wNkzcDzctZY5nEoDtVuCYWlHsBqBW5skC54eWB3Kt2NmCi50PZKCSauBvcXGg== X-Received: by 2002:a17:907:7b92:b0:a7a:9a78:4b5a with SMTP id a640c23a62f3a-a8a887fcdb3mr350202066b.52.1725663421631; Fri, 06 Sep 2024 15:57:01 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.146.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25d54978sm2679566b.199.2024.09.06.15.56.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Sep 2024 15:57:01 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v4 4/8] block: introduce blk_validate_byte_range() Date: Fri, 6 Sep 2024 23:57:21 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 340561C0002 X-Stat-Signature: cb8s3irzmzxcrr549exq9tsn89t4r18i X-Rspam-User: X-HE-Tag: 1725663422-604172 X-HE-Meta: U2FsdGVkX19kvbQLS6EwjnYYDf3fYVZ7NODPvtR6jCZie6MeXNBersy9wpbt8jFBb+PQknWxFTqnbdPccVGMgeoP51OMok4657fqEPhu5Q4vy/7wKNAoQyjwcue5skJgJ1JXC0DIMqK9kJA2Q3CUYeTwLzOj+X3SISBYyBvxCNgsM+1rs1AzeLczgrGnEJFnNRW5ywkE6hmIRgkBjB9Ldmj/gJTWTsP5HJxFAxE7fB4ralKy+L69F3WPgpsN/9ArkjKI6UBcm6FAUQh2TTwea0Qyyr2uM8Ok9QM3zoJeM8wGGY2e/mrQn+3SyOZl4+AH9Sf1SDb/08Ohx0tPQ/ZdSqa8x+EqBVxIXkivL3qy0qRT+UHsk0KmIOERMN1G3txPYGKkAt9Ry9cIHOR3XdB8mDvjg231jpTDvIx9R0EXL2Z+UMEgr27yfvNw8Tk9tUmKJo+LH1DQo8hm0MqZ+iXgnITnKCpUtI9KJVG94xP0GLIYgex3wNS1+usPuMFcOa6HfDy/LS5tnT6FK9Fu/nEcKZgFC4os+gCw8d9Yht7w2bnhFv3CHbCJFucDidpfd9MVOzJFIRE7VMvS7JUkm0bfOagGJ5cem8qukq8an9NYBXSjc6zU9qaDhshdH5k/DHBM5vuan30qDjnv+pJTgb8IfU8C4pwlpB6WYEqEKL60s0Oa3t+IfcKEVeJemPeVpjfBT0yEe4TuiP5rumw7A9gksXXJaoqmaldepysok9gA/tWIax357wF0A+H9nQsdMB+eJdQHEwTXY+df5/6MTPAsMGDxUFcnMyM/iQU8WFWbnoo20ENhmPLI2Z6OevMnQHHR2EVN3p1eE0YRRyYdiUOi8Cvb1a+oo5tcgRPb/0+4s+R/aVBPdktwDJQYo/t0asAHWTpL2fzYxjFb57hq672vURY3FWNWNlo39VfgpYz5Dj+YKDefH3j8kx2NnIkHDCYkRsscOtbzT+ymOKZW9c8 g/i5TohI DSrrJPSoG2cCo4tjS2yw+yCIKE+DvoTNwhuqDHKv/JW13IGkXfCHcR6llzRMrqF1FHTK8GZNtExuPgVvrLdVh7c1CUTgXmM8CQK30psIKdJhDrzlYEcV5s/+TbdXoqu3FHR6cBTdjpgZ85RfWR7x44lnbCr+9H/7cZLnB1RYDxb1uq7DBhQmXCRcu+6af8F3Ay3ehds5KhRsmvWUNaLYX8eI2o9zhCsBPyn8CCiO5Z275/SNobjqpzrZcn8UAvS1ecD1q7IKEN1JIBy3e+J7x98PUWrgVlhUC/yvkEpInJxmM+8tI4LVwfcwkWZyuWJ3SIrRltSuAaxdwM37N93oO5GKGoksr2WTItM6hMYdrh6sP9WT6OnPcvrOQFX5IeLQbFwJehyxmOsx7+D8zdIP1vaRyFsoZP9drWTq6zAWM6VU7H6VJsU7SfS48dEFFK72ZLyWESi8oaCbGbqNexSEJQWTj0uznhur6i4hY2/n0xlGn6xwFOb1lmVbQ6CVG1+IfMev56ntaGI1HJ8o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000006, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation to further changes extract a helper function out of blk_ioctl_discard() that validates if we can do IO against the given range of disk byte addresses. Signed-off-by: Pavel Begunkov --- block/ioctl.c | 44 ++++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 18 deletions(-) diff --git a/block/ioctl.c b/block/ioctl.c index e8e4a4190f18..a820f692dd1c 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -92,38 +92,46 @@ static int compat_blkpg_ioctl(struct block_device *bdev, } #endif +static int blk_validate_byte_range(struct block_device *bdev, + uint64_t start, uint64_t len) +{ + unsigned int bs_mask = bdev_logical_block_size(bdev) - 1; + uint64_t end; + + if ((start | len) & bs_mask) + return -EINVAL; + if (!len) + return -EINVAL; + if (check_add_overflow(start, len, &end) || end > bdev_nr_bytes(bdev)) + return -EINVAL; + + return 0; +} + static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, unsigned long arg) { - unsigned int bs_mask = bdev_logical_block_size(bdev) - 1; - uint64_t range[2], start, len, end; + uint64_t range[2], start, len; struct bio *prev = NULL, *bio; sector_t sector, nr_sects; struct blk_plug plug; int err; - if (!(mode & BLK_OPEN_WRITE)) - return -EBADF; - - if (!bdev_max_discard_sectors(bdev)) - return -EOPNOTSUPP; - if (bdev_read_only(bdev)) - return -EPERM; - if (copy_from_user(range, (void __user *)arg, sizeof(range))) return -EFAULT; - start = range[0]; len = range[1]; - if (!len) - return -EINVAL; - if ((start | len) & bs_mask) - return -EINVAL; + if (!bdev_max_discard_sectors(bdev)) + return -EOPNOTSUPP; - if (check_add_overflow(start, len, &end) || - end > bdev_nr_bytes(bdev)) - return -EINVAL; + if (!(mode & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + err = blk_validate_byte_range(bdev, start, len); + if (err) + return err; filemap_invalidate_lock(bdev->bd_mapping); err = truncate_bdev_range(bdev, mode, start, start + len - 1); From patchwork Fri Sep 6 22:57:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13794835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E111E6FE49 for ; Fri, 6 Sep 2024 22:57:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A28B86B0098; Fri, 6 Sep 2024 18:57:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 93B3A6B009A; Fri, 6 Sep 2024 18:57:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7661D6B0099; Fri, 6 Sep 2024 18:57:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 54B3F6B0096 for ; Fri, 6 Sep 2024 18:57:08 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CF668A134F for ; Fri, 6 Sep 2024 22:57:07 +0000 (UTC) X-FDA: 82535825694.19.710DF5A Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) by imf22.hostedemail.com (Postfix) with ESMTP id D8B51C0007 for ; Fri, 6 Sep 2024 22:57:05 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OWN26DZA; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.46 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725663353; a=rsa-sha256; cv=none; b=kBN/euSN1zG14xBNKhVAddQ3WcbT88j7NDMwM80Nvi6pYnYyP9HTK+oUamwuV7pf58m5pl i5yJ+W0j6uBcttx5J2HTsXOcnT6DwWIEEWUGKlxqlr7RByRLoCGxjcvCc2q97aUtfcCCKH 6NpdWbz7an2Fgy8TNA30GIlWt8aUZKg= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OWN26DZA; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.46 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725663353; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=v7dFMMDbVFAIV5pvquHCkne/t7NzjwwosACmczIEiUs=; b=5bx/QYMCS4hFTxltncUoJaJpiyQufnnFsVOGK6T2QN+wSN+qhJrzt3RJYpXGHRzmRjNQAE 834Eihp6/63axNOQh+QjIq6chD2bxErbKfDILy6cpCDlDLq677Un9DI1SsbEL6DCYgIEkT 1yuZ0UjIDW7UCXCWO7PZa8BKb7/pi40= Received: by mail-lf1-f46.google.com with SMTP id 2adb3069b0e04-5365a9574b6so1532263e87.1 for ; Fri, 06 Sep 2024 15:57:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725663424; x=1726268224; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v7dFMMDbVFAIV5pvquHCkne/t7NzjwwosACmczIEiUs=; b=OWN26DZA5yBzndqpXA9Lk7D8sXURH8LfNn7rB2zeE5kJKrmuscz40RDXWBSWWRuJ1C VttvLuHXepNzIPjfeAdUo81WkzEf5h3zoO5aqUoJ/wlSizQp9BlLopHIExUBKt61SIkP +UkRKcuKRcFVK09xaTJ441bL4CAQCEsIzzGQBxAKYKs5vmaF26ZDP4HEzTf74xGAGZuS jHAslcxhhZURj8bxPZQ5hkxgXcR4ieetEbMMiO2ud51khrNyDL6dJmlPtiEcm9LnEEWk poxkS3SWMOXvk52ytCL6XjNtEh/q8BfewyBqb69gKDaXpl9IizJoixB7uoPbDk7yjg9Z KPfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725663424; x=1726268224; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v7dFMMDbVFAIV5pvquHCkne/t7NzjwwosACmczIEiUs=; b=HIayw2T52Cn3nH85qc1Y9i0u2AgOyR42fL7HXGCWJeiG1mM7H4ARATusg5jCAj39PO 6thSOMgJWUArwd8jHW6chq6vmJlqirH50lcluKEy1POiHSYSqPwpSF0F+l5hzzeR0hab 4pCEfVv5aJWS3OEi9sChuKdHDC8QHfLA5fShq29NjxKq8BOQMs35wgr2kUN5H/OoVEfY PZZbcL4ZhpfmCBtC259mTQxUal9p94gnbp7Z7/RnCjNCuMhwDwK+C0ym8UFz8jMU5jT0 M30h1cDwrqjBX+krR9UoFsmTNoOcQL0Hpgogsz2SEYJJkk7iZ8BURO3Xh1+CDt9Nr6SD vCtw== X-Forwarded-Encrypted: i=1; AJvYcCX9JnkzCDhCXTkXI7vSA0NFt9r7UCiX6Nk8Hrnd7psQiIXl0wI9evtoo29sEFqWR1e1GJMp351cQA==@kvack.org X-Gm-Message-State: AOJu0YxZVjlcMkzKsSoGX9ELmpw5stf8j+vCSV0zXDN7iWzIBZLAyuI6 QXE0FV/Z6XPXua2ThrjYZBiUm89mER40IqNxnbrcpUNTXzgLZrqo X-Google-Smtp-Source: AGHT+IHt+QYHmRu0nquydclberbtpO0LZ0vL2z0xJXuhII8HVhOpdcMo8Pgh0RIOwsY85+0VzuhDqw== X-Received: by 2002:a05:6512:b88:b0:536:53d1:850d with SMTP id 2adb3069b0e04-536587f848emr3490089e87.39.1725663423587; Fri, 06 Sep 2024 15:57:03 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.146.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25d54978sm2679566b.199.2024.09.06.15.57.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Sep 2024 15:57:03 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v4 5/8] block: implement async discard as io_uring cmd Date: Fri, 6 Sep 2024 23:57:22 +0100 Message-ID: <7fc0a61ae29190a42e958eddfefd6d44cdf372ad.1725621577.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: D8B51C0007 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: fbakeesbpa6tyju6np1sde71dwnkgf1y X-HE-Tag: 1725663425-563009 X-HE-Meta: U2FsdGVkX195NlmikL/ZGdV40W9ykCq1msAsNkSOV6jUeY2O0qszqD83C7f9FTWj0Q3b9XMlbWi4kmlBdq8xJT2Lu+/IwJCsQIE2P9JTu+31VN8zETdkgzlBetjkH623EPCV/pTEEz7tdhh00LEWfi7cvKmVuTyrGoeBLKG+HWnelOWo5VUgykZetgTTp5iRmeYBGoM/zSj/RQLHuIKbkGV4CVeqHitYq8v3AtxnVpcNwTtqpj/kdBqRIDFCpiNDEztZvZ3CGBv406eBDQkkakBxr71P5p3W2PPSikKcv6ST2sQ9i7xV2T5IJhjFyVJPN1novevOainXRkHevXRJJKYriV4oVt8iSwNrcETNuxWJ7TRYqvZJ2zsKUa6mpaqasuxXDhZlaqTjy1iqrXPaueiwK8dhxjDdLXggagzcsvE+GPgGPheMpziTSkiQe51veyv1CnevHSazKYzGQnxREuT19mZ2/v6xo0FdVRK5UIDB5CbarZwuxIAY2NPFZiAgl0tSN40q1arhiP42+fI6IJvYxpEFF2TfFkZ4G7T9oi4K81RejEPjZUq0wpzxNgYZYczwR03SfW9DyZBf1uQSBIol96p1Hrg3jCC56OrvAUtRhqrL7a9xBh6NbfbUyeVRW7NWFDOLnmByCNOoFHCV4ifi9ifDKia7mWPhX82iIjim9uuOWb2s3XSkcK2HPCXfGBlHizxWKP1GBBCs+1ri/a/lXavDiDT3L4EHXSSnIsTXXIt5Q9rcgobjJ7+Q4SYuRXzgrGgmuZdZ0xgbq5iPzo98CNIAfHrdxSqjs8XnCT6VxAcL84gThz7/4etjBTFAlK2KGNPDp3FSflY+S2tYfyLlDul5Qhdqge8mz2BuPz4Y4gVwhu0MWai2V5+enxLS7nqVGgz9YcnnfIYEVD4pWTqCLj6mPgl6M+o2nYldeqAR9Zm377QkHw2mHWml23ipz8i/jyYUROwTWGASere r3QPCqZD xgzWHBByBQ4zVCgUYP7Gj+p3Fa6/UfFjb4n/7wtPPhS+Im5+gKr+Twye74NITAFUDU/GEWuMdB3JRS4MOGy5BID1YdSwsUsWUYx2CMB/jor61BfL3htOiTxP67yRpDg8kiTOheIKadr1DKR3DdH7pWL3RyGY5laJte+nmMpAXdeplkp01wHHQWbPpP8TB5PLsiiZmGJF08aeCWtnZNqO1bDSNWhqz3h07Ozsn9beVjDwciH8ZfuWnD9gRbHxLbGrbGWN4v1Rkum6uuKitclUEwwxmT0oiyZL8E7A81n1t+6wdPSeECk9u6DFy8Ml60Y9cI2HIj72Qh4Er1KIsa18P6lOOivayemkF6MUQlucPgQce/To5FbI2Qwbpo4g/yUCrtSkTEitaJKv5PYexvxnNFi9vlOHwk640fgYS4/uBkn680uZhv/aCLbBUFg0JbRwh4FnuAyEF/cZXOfpeAwAfYKwBnuSu25xBfJt7JNrQztRIhP+2IijLSvZARbI6imYUfcYS5p5ak7v65EHr30eVABKveRxKUT3+bvq1vm2nNTqnhlM6dKpn9zE4dWVIlR0x8eYu9GE4/+FThWg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: io_uring allows to implement custom file specific operations via fops->uring_cmd callback. Use it to wire up asynchronous discard commands. Normally, first it tries to do a non-blocking issue, and if fails we'd retry from a blocking context by returning -EAGAIN to core io_uring. Note, unlike ioctl(BLKDISCARD) with stronger guarantees against races, we only do a best effort attempt to invalidate page cache, and it can race with any writes and reads and leave page cache stale. It's the same kind of races we allow to direct writes. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov --- block/blk-lib.c | 3 +- block/blk.h | 1 + block/fops.c | 2 + block/ioctl.c | 102 ++++++++++++++++++++++++++++++++++++++++ include/linux/bio.h | 2 + include/uapi/linux/fs.h | 2 + 6 files changed, 111 insertions(+), 1 deletion(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index 83eb7761c2bf..c94c67a75f7e 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -10,7 +10,8 @@ #include "blk.h" -static sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) +/* The maximum size of a discard that can be issued from a given sector. */ +sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) { unsigned int discard_granularity = bdev_discard_granularity(bdev); sector_t granularity_aligned_sector; diff --git a/block/blk.h b/block/blk.h index 32f4e9f630a3..1a1a18d118f7 100644 --- a/block/blk.h +++ b/block/blk.h @@ -605,6 +605,7 @@ blk_mode_t file_to_blk_mode(struct file *file); int truncate_bdev_range(struct block_device *bdev, blk_mode_t mode, loff_t lstart, loff_t lend); long blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg); +int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags); long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg); extern const struct address_space_operations def_blk_aops; diff --git a/block/fops.c b/block/fops.c index 9825c1713a49..8154b10b5abf 100644 --- a/block/fops.c +++ b/block/fops.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "blk.h" static inline struct inode *bdev_file_inode(struct file *file) @@ -873,6 +874,7 @@ const struct file_operations def_blk_fops = { .splice_read = filemap_splice_read, .splice_write = iter_file_splice_write, .fallocate = blkdev_fallocate, + .uring_cmd = blkdev_uring_cmd, .fop_flags = FOP_BUFFER_RASYNC, }; diff --git a/block/ioctl.c b/block/ioctl.c index a820f692dd1c..19fba8332eee 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -11,6 +11,8 @@ #include #include #include +#include +#include #include "blk.h" static int blkpg_do_ioctl(struct block_device *bdev, @@ -742,3 +744,103 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg) return ret; } #endif + +struct blk_iou_cmd { + int res; + bool nowait; +}; + +static void blk_cmd_complete(struct io_uring_cmd *cmd, unsigned int issue_flags) +{ + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + + if (bic->res == -EAGAIN && bic->nowait) + io_uring_cmd_issue_blocking(cmd); + else + io_uring_cmd_done(cmd, bic->res, 0, issue_flags); +} + +static void bio_cmd_bio_end_io(struct bio *bio) +{ + struct io_uring_cmd *cmd = bio->bi_private; + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + + if (unlikely(bio->bi_status) && !bic->res) + bic->res = blk_status_to_errno(bio->bi_status); + + io_uring_cmd_do_in_task_lazy(cmd, blk_cmd_complete); + bio_put(bio); +} + +static int blkdev_cmd_discard(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + gfp_t gfp = nowait ? GFP_NOWAIT : GFP_KERNEL; + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + int err; + + if (!bdev_max_discard_sectors(bdev)) + return -EOPNOTSUPP; + + if (!(file_to_blk_mode(cmd->file) & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + err = blk_validate_byte_range(bdev, start, len); + if (err) + return err; + + /* + * Don't allow multi-bio non-blocking submissions as subsequent bios + * may fail but we won't get a direct indication of that. Normally, + * the caller should retry from a blocking context. + */ + if (nowait && nr_sects > bio_discard_limit(bdev, sector)) + return -EAGAIN; + + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, nowait); + if (err) + return err; + + while ((bio = blk_alloc_discard_bio(bdev, §or, &nr_sects, gfp))) { + if (nowait) + bio->bi_opf |= REQ_NOWAIT; + prev = bio_chain_and_submit(prev, bio); + } + if (!prev) + return -EAGAIN; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_bio_end_io; + submit_bio(prev); + return -EIOCBQUEUED; +} + +int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) +{ + struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host); + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + const struct io_uring_sqe *sqe = cmd->sqe; + u32 cmd_op = cmd->cmd_op; + uint64_t start, len; + + if (unlikely(sqe->ioprio || sqe->__pad1 || sqe->len || + sqe->rw_flags || sqe->file_index)) + return -EINVAL; + + bic->res = 0; + bic->nowait = issue_flags & IO_URING_F_NONBLOCK; + + start = READ_ONCE(sqe->addr); + len = READ_ONCE(sqe->addr3); + + switch (cmd_op) { + case BLOCK_URING_CMD_DISCARD: + return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); + } + return -EINVAL; +} diff --git a/include/linux/bio.h b/include/linux/bio.h index faceadb040f9..78ead424484c 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -684,4 +684,6 @@ struct bio *bio_chain_and_submit(struct bio *prev, struct bio *new); struct bio *blk_alloc_discard_bio(struct block_device *bdev, sector_t *sector, sector_t *nr_sects, gfp_t gfp_mask); +sector_t bio_discard_limit(struct block_device *bdev, sector_t sector); + #endif /* __LINUX_BIO_H */ diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..7ea41ca97158 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -208,6 +208,8 @@ struct fsxattr { * (see uapi/linux/blkzoned.h) */ +#define BLOCK_URING_CMD_DISCARD _IO(0x12,137) + #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */ #define FIGETBSZ _IO(0x00,2) /* get the block size used for bmap */ From patchwork Fri Sep 6 22:57:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13794836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D985E6FE4B for ; Fri, 6 Sep 2024 22:57:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5C8C6B0099; Fri, 6 Sep 2024 18:57:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AE7B36B009A; Fri, 6 Sep 2024 18:57:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 913356B009B; Fri, 6 Sep 2024 18:57:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 71E376B0099 for ; Fri, 6 Sep 2024 18:57:09 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1B4601C5E11 for ; Fri, 6 Sep 2024 22:57:09 +0000 (UTC) X-FDA: 82535825778.14.6092425 Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) by imf20.hostedemail.com (Postfix) with ESMTP id 2DE3C1C0009 for ; Fri, 6 Sep 2024 22:57:06 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Vzn20Iqd; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.208.50 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725663401; a=rsa-sha256; cv=none; b=RBF56qj5m/HUsQEasa+HeJmO8Q+ILwTr7YbvbzsrcdYE8OjrcyG/fawxwaivGbMUJfatiT rFrq0trVl1jaRxq5oZmn+Em+8DthByi4Ar5+wLsCHCVdYehR/0m7ykTyxVCt4kID4HC+Lm q6Vdj/xEa6PUtwL/jlkErRZv+hJoCe4= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Vzn20Iqd; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.208.50 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725663401; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sVNRBhtechMDGkg+NH1UKI/XUcjXx3fSa2oGaERaZjA=; b=OmWgVJIDPmIThlp0MynhFwx87UMO7KCui+lMbfwnd/es0UYpPGpKtdMfbvUePMGtIGmMl/ PpNJHjODRKHs0C8a2wwQydYH6f/gJqtPxaFo3n4EFVkws6HisU/DLjthSxNdU87kiRpz3w 7IPBllo8ZrTM6owIGxKnABluuSSH6WU= Received: by mail-ed1-f50.google.com with SMTP id 4fb4d7f45d1cf-5c25554ec1eso2739555a12.1 for ; Fri, 06 Sep 2024 15:57:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725663426; x=1726268226; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sVNRBhtechMDGkg+NH1UKI/XUcjXx3fSa2oGaERaZjA=; b=Vzn20IqdI3dzQLBg2EBqECi78xXjWD9EA/zpjtS2Drf2uO6InY6TQC5A0+mrpq3gXg Y+tYjQnqbNKQ8Yj9jB2aGPlIOIrYh72lND61m8Gz7AMAyKclbHGz+bSMMxhaQH9uyNQB NZPk59x0+ZyWvSlGTBgMIvUORuzN3f+Ob50BYQCLomLjWokjX/GfIXQ+DC4q0DgwCnI3 ijQYFWu7v4cWwAI+uyLqAwSLVpj2pQWMU4UbOxiOWHA9kxMsxYSJWqCp70L2T4I8mrYd IE5DDspof2j2CfCcU+ZGxs/jTWiuwyzo8fzI3YbBIsKlmdeGtlvgHY/Zb/mwV8aJtjX5 RvPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725663426; x=1726268226; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sVNRBhtechMDGkg+NH1UKI/XUcjXx3fSa2oGaERaZjA=; b=AFdo28O4HhxU9qUQklBYdm9w/ovBDgUyTvMCsfNJw/jmOiNkqf/OjcTyJEdtcToSBp ItpB9iOnHeTCCsmYR+2NksvihlWmC8lemNx4tpDydSPhWOTxihvcqLkhblWT6obyOYxI PDqvReKeIJ+bF4mrVuRk2HMu0Wk6eyCawYIZrDHNV7Cy7hnkLGQZhMLETfyBfw5T9Wkw qvX0lwgJ9KSzs8zzCo3fEgAWgVMR3g1zUy9gCDOoblVJrSaXccGItwQfvfEu0AnD8j+W Il1QzRVZvDPTh8IT/Kj8MMoXyOWIEJWgcBLQe51NF4IcqtRnTaGJ1XK0UrAHFo/xxPyA R7/Q== X-Forwarded-Encrypted: i=1; AJvYcCX+r+GI3W0P8h3JkO89NVVmVSMuTxtSCcZ6tKVDTXnZWebtVy8C5EkSuVWxTbqdWwNrTVQnNaY6LA==@kvack.org X-Gm-Message-State: AOJu0Ywt5pYEO6s33Ox0hVEEsWtjX8JJJ4qcAhVNxesRfGVzlgsn1gvC x4mGI5RO6AIBBsQMuJ1DAqWT6N4+oOIfF9pzupH4bmcpkyZu1bod X-Google-Smtp-Source: AGHT+IGwtkNX+b7B+F+E65xCqdvf83CYwy/QQoEkioql8r4UWUgpbreyku1f9KDVfE9jAVw9Or4nJQ== X-Received: by 2002:a17:907:968b:b0:a86:b46b:860a with SMTP id a640c23a62f3a-a8d1c738eedmr53296966b.54.1725663425531; Fri, 06 Sep 2024 15:57:05 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.146.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25d54978sm2679566b.199.2024.09.06.15.57.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Sep 2024 15:57:05 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v4 6/8] block: implement async write zeroes command Date: Fri, 6 Sep 2024 23:57:23 +0100 Message-ID: <2a99dd14c2e0c1fced433822a13ff00735a84816.1725621577.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 2DE3C1C0009 X-Rspamd-Server: rspam01 X-Stat-Signature: rkhnhcskew3ueg81k5jz5fbhc7hwahmh X-HE-Tag: 1725663426-312056 X-HE-Meta: U2FsdGVkX199ECdiMF5fFtldTt/BDachs8JbIquqwGhnoMruWwI4c1Aszg/qvpKWzoL8jhj/0PHc3ghw8GuvxAbMJFJu03oUS1mCSuGC8+x6s7wsnvaxEZYs/uo1bfUZRSQ21HhcsASnhc8AHhMXbYjqY23xfBP3GMXYZDnlNDf2QMiXRIhi7eq9XSESvg5hP34x79hq2Sx4Jo8hVHAsB1k8yA4R+mE3mUMGPD3k2sFIRG/b4FdeD+LBr7aN5n27tup5rNkGaonqsDGb/ZpJt42+/fP8JsZqMC95xhwHcaGspNAV+y6ouNxAYXerTz7lxR4c4bfVBhbmYCy3I5jt0Dv/yBZ54kguV15xbdBwqo5g7JNmseKt/UZ80aOkcAimuke2XO4SDWqRkcWYS5sxUrIrENDtpLB0hHo0Vxa7qq3V8kocAkCHAG2UMQV6pje3Ghmfx9GXzgJBfgBMlDt9H4zpQWFAPEQNkozbKCMwOw2VmIQ7TbPcKEtuVszLNBaJygeM3pLOJLzXbXRH+6cflzg0l0JtpBWa7/L3xpSl3dvy9AW9adLLEOeihaN0OB5gkalDrvoce9pBZLmz2umIjadoswTSsk/Cm9sNNkBc1j/v7L3qwGU0zAHPywVteKmG0M+PfBb0RWWpo1yrTOy6vo6Xf6Whi1mO1wxq0rs+WM/M+FqkDN4iXL0KTxMz++3ipigk8/GenTHaL4HYsl9UUqo+KFOM5oWa6YefQeSxPgN6MLMOrfKxh4aXUeysddIR0TMk66rbgOfUkAAHnleMB2IA9EzYm8vbzcMNtA5xwfV7fWcqAzJS66eDInFGkP5hLMciRjjwhnP1XMJaeTh4MjTRd/mAR6jq9h3YqVQHoLxxtJYdISCq8NVG2uvFaWMnzIm5jKvq8DO5tDdwbhQwIJAXekpOZoXGE96ijsixfKeLFqTFTxXLBta24EGR+f2TM+ZSeb3CpUXUGNidRv+ qI5xalS5 0dy8RWZYeTiL7dSfA9+WObh1J3BF5YPi2rxa0WmzGP6PW/LWif0PJW72bBSiwu0S/jhLiGlpdXv0UFAZajTVCVQRKjdxdjFMZ9AVcdtHI1NmS5apCa5HpxoSbJL0RBOK5SQZVk9ZD5EvtmA+WtNQKPGJg1Siwd2Njn5R/CG2uhnN0lCYb7CDbTHviOZ0jNVXjR7Dcbp4D6tAqax8P4GGseR/w+oM/sn1Hrz8Vcx/1JkIDb2WLR5OBHHYCn1i0uGhE4MmHEVIxj+c5Dw4u204aUPx+nigcHXZ9Py+5BAslOIx850E3Md9cq3xt+ZCd0A07rUFDrI3H1rLIMpr+vgY7bRBBVP4vX2I0pvQN9ErhefNiUYgAZVNxeSkXHfAy2Eg65Wu3x79rSy9Pc3Je27F1834ETOme8SwFQc1HNMKnOlR8ThzgSTHZBKvNds7QmyyasWPegRLwyQZOUo+EGiNY/uWl7a8FY6Ztvu59RaTuJK2XmQTX8hbt7P1UQUMSrDlcykR+3SM1IBMro7OcHXW+6Lv+hSj/+j6iGfZ1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add another io_uring cmd for block layer implementing asynchronous write zeroes. It reuses helpers we've added for async discards, and inherits the code structure as well as all considerations in regards to page cache races. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov --- block/ioctl.c | 64 +++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/fs.h | 1 + 2 files changed, 65 insertions(+) diff --git a/block/ioctl.c b/block/ioctl.c index 19fba8332eee..ef4b2a90ad79 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -772,6 +772,67 @@ static void bio_cmd_bio_end_io(struct bio *bio) bio_put(bio); } +static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + + sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; + sector_t limit = bdev_write_zeroes_sectors(bdev); + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + gfp_t gfp = nowait ? GFP_NOWAIT : GFP_KERNEL; + int err; + + if (!(file_to_blk_mode(cmd->file) & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + err = blk_validate_byte_range(bdev, start, len); + if (err) + return err; + + if (!limit) + return -EOPNOTSUPP; + /* + * Don't allow multi-bio non-blocking submissions as subsequent bios + * may fail but we won't get a direct indication of that. Normally, + * the caller should retry from a blocking context. + */ + if (nowait && nr_sects > limit) + return -EAGAIN; + + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, nowait); + if (err) + return err; + + limit = min(limit, (UINT_MAX >> SECTOR_SHIFT) & ~bs_mask); + while (nr_sects) { + sector_t bio_sects = min(nr_sects, limit); + + bio = bio_alloc(bdev, 0, REQ_OP_WRITE_ZEROES|REQ_NOUNMAP, gfp); + if (!bio) + break; + if (nowait) + bio->bi_opf |= REQ_NOWAIT; + bio->bi_iter.bi_sector = sector; + bio->bi_iter.bi_size = bio_sects << SECTOR_SHIFT; + sector += bio_sects; + nr_sects -= bio_sects; + + prev = bio_chain_and_submit(prev, bio); + } + if (!prev) + return -EAGAIN; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_bio_end_io; + submit_bio(prev); + return -EIOCBQUEUED; +} + static int blkdev_cmd_discard(struct io_uring_cmd *cmd, struct block_device *bdev, uint64_t start, uint64_t len, bool nowait) @@ -841,6 +902,9 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) switch (cmd_op) { case BLOCK_URING_CMD_DISCARD: return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); + case BLOCK_URING_CMD_WRITE_ZEROES: + return blkdev_cmd_write_zeroes(cmd, bdev, start, len, + bic->nowait); } return -EINVAL; } diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 7ea41ca97158..68b0fccebf92 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -209,6 +209,7 @@ struct fsxattr { */ #define BLOCK_URING_CMD_DISCARD _IO(0x12,137) +#define BLOCK_URING_CMD_WRITE_ZEROES _IO(0x12,138) #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */ From patchwork Fri Sep 6 22:57:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13794837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D889E6FE49 for ; Fri, 6 Sep 2024 22:57:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A264F6B009D; Fri, 6 Sep 2024 18:57:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9D4926B009C; Fri, 6 Sep 2024 18:57:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 827E46B009D; Fri, 6 Sep 2024 18:57:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5FD806B009B for ; Fri, 6 Sep 2024 18:57:11 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D580641167 for ; Fri, 6 Sep 2024 22:57:10 +0000 (UTC) X-FDA: 82535825820.21.5A14B74 Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) by imf18.hostedemail.com (Postfix) with ESMTP id EA0921C0005 for ; Fri, 6 Sep 2024 22:57:08 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=QXyvQ51B; spf=pass (imf18.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.44 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725663330; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jYezJHCo7v489eblMcj1bTcVFXeBrX/vk5R7qG2A1Yk=; b=SeZuAg5kt5YGd+nYnwvdCw71S91miXTCHyLrMWyQ5yM5XzhGfLUkJvYvGusLvOPhCTVyBG I6MfkYl5eE0KVqrtkoAFJjqbgTZx2sEl5igt63URSbqEX14YMXl7iiAO5z6UWrGl3Lejkj 4nkVIGpZv7y8LI2e3y5WHja65O5pDms= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725663330; a=rsa-sha256; cv=none; b=IN5cExh1RAtVtghh9E6Kq6i4+bxMPu+iHbeK4fwCvgq9+9IEbA7+O1C8ocCrRtz/WyH707 VaBz9lSv8zpyX/aYW63AR1W0ulQ0bHmFnd3VFsn2Y8RD4rwNB85qWOCHgImdwee0F+iIbz Z42kWqgsYZG2DcECYef6K0/NsxbFfpE= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=QXyvQ51B; spf=pass (imf18.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.44 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-a8696e9bd24so326090566b.0 for ; Fri, 06 Sep 2024 15:57:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725663427; x=1726268227; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jYezJHCo7v489eblMcj1bTcVFXeBrX/vk5R7qG2A1Yk=; b=QXyvQ51BX5LcqUY9vC4wrA3UNxapiMRYjAYwY1nAopCTksLH1+H/31zrtSAL10NN+B bJ+VZu1wYcHflbxq2s7qO8bvCZzlzRbbOuhgZihQKS9VjSPK7JxKt//qSs4ztcNqzAWU cUyy8423tOPKDqujLGIn8dW7Qq/pzzokbhuias6hqOn2c9RqoORwNvf7keMNLt/2w87a Zlpb0/1F70ebobZZiBWVbMRiV4JMeVPh8TlI9vPnmIi1qWlk/BhA5ewlU/p/uKQDnjPG TiwQEOKwIDOUktNMu634o69qN8P5WmEA+JrA+rYZNIkkhemkjXmbI0VYbzEPCzYyTjyY X3SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725663427; x=1726268227; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jYezJHCo7v489eblMcj1bTcVFXeBrX/vk5R7qG2A1Yk=; b=XB4Q+uwtN0ov7WUitgO057pyXRetL3UltLWMeW0bIdG2HINjKwiqxcrSwRd4CIVrZD 1qvt88oA8y2xY+QZQ3fmsusEQpfLo1wsDopj8c4JREyX//HYYqvoaW0nzusxPvUX7Ln0 ezU6XlXebisJhQCiMnPkZEV5eyCd755QRxZuaXTE3XVNakiEzYRXcg98ccmt5Mei11Vh LsONLeOdRrBUMpk3lqIcd1wqadoOIxNdqWqvZxCZb1w49LosQiJHgDcNVgqce0DXIg+B 17P9IYuYd3DVvlRWNSUWeTVVR3Ph19XQph89IXI5F5K+EI7j+j9HZ46285gOMgGODr+y YhtQ== X-Forwarded-Encrypted: i=1; AJvYcCXkdNLS1IORaNztrQkcJ1Rqz3rXcvpn+5ssxdJCIHXrn/O7geYrvNjCt9T+l91Buzvd+ObDX5EweQ==@kvack.org X-Gm-Message-State: AOJu0YyGQSRzzwhlhR8S+rtOzUfjIigUuyof7eGCQ8jEX11STGfrFB1j rpadalxIb+e+/LiKQ1xVipXLO17gDaikUlJFybkhOGYNdv6aLC3S X-Google-Smtp-Source: AGHT+IE/3M7b/K3pyseCsX+NiZT9J56A5VyhoP1o7o5GWtiHew5dHLC0T6ZxFj2yq0dOHss7K5NZbA== X-Received: by 2002:a17:907:3f20:b0:a8d:caa:7fee with SMTP id a640c23a62f3a-a8d1bf75ec5mr67443566b.7.1725663427142; Fri, 06 Sep 2024 15:57:07 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.146.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25d54978sm2679566b.199.2024.09.06.15.57.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Sep 2024 15:57:06 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v4 7/8] block: add nowait flag for __blkdev_issue_zero_pages Date: Fri, 6 Sep 2024 23:57:24 +0100 Message-ID: <387caa9b7a23061f19034a8afd1dfcf017d8fe35.1725621577.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EA0921C0005 X-Stat-Signature: 1akcje6rwj1419msfcikepnnfru89awb X-HE-Tag: 1725663428-879437 X-HE-Meta: U2FsdGVkX18ofkvbCVKaLsnD7d4i5tI52z9l2R1y5mWOrbJ8awyXXonj7O/D6WVVPnxvZJQ9v6ZkAVd8ZvWeecRZpUdXQt14EcHTbV3UHf3ElnpaShCg2h4bKD2JK2HN1AhdDKrhvTMZ37/9t6tk9ehYxSr9m+QWauNukDw+GCn4Q5mMm+jw8mnA8KsTd2mLgdZHK+kljTfVLL62mAL3FbPpWnYzC6twy9L+9k8z4H5n7A3VcWv6Fy3PD34Rib10j7KFyKuSqAtt7MxvWmsFfDYb9wiJawqEN9Wfp0hfmQKYCOSzr6UmAQycRpeezI10YSp/sIh7uOrJ9mRvSmta040fAIyrt6PUnT7f/W3TkJFBAVxGS7uQgs0Gtoj7rHTpMzxCD7ooPReoDwO977EmGme+yRNLDuXdk9/MYp/It8KHj4+18cbJtLVTgC3G0XJ68YHjZAagmwVg3UeDn4dFQ9vBUHCuilSutIJGl2UrMKpj0zwn3I5lf9d2+WARQny6YFXVvzOo79X8I8x+/BsB0lph6c/hc36rvIlDBPd1eKIfVNQQ7Km3WIlATYhAvOyja5E6GSWDe1rzrct1PAzo7yImkNstO/YqIdEUAbkS+uxDWb8iTsgSTwk6veygHI1sxtNzsuq13kwLXFHQOIwBO8UodV1E9iID3uhbTZ5PiVAr+o9J/MoQS8o48RIs+DBA8Of4l7xAbM0sOmqKPLv248ctwJaSqw5CA/lGHENPzHlAlklykb58i841xqb9DUXV36lloNsE2KO/XNXEeLvkjGWHt/SDkpm0/ZkBxiy87gSbhmb9w4poWlTstEySt86tzqYiXXhD7gFG8ZfswQijJD3BQsRigFvTFteedDGFFVDnOoVED2BMYyxZLmHBIXjb5dr8tRSqHlAu44ebOdW2jfpoPiRqFIYMOKgVZORQ4t00gK5OaBjhdFVW3qch/8HV+zWpojaZJOoqHWZDu6Z xAtQ7V/e ycGQ9j4zCGdqJbYFTprc4WiTRruAgX4lFuP8jsxszuR+xwEC3Pksi13OdjgPZYACpVUpV2kpMxsbikDs2I/2KWUcCWWXeGm62CndnmnVR1ByfIVnsjYoQy16tkPHHzsGSCEl+SJdiDEWZsBA6azcu5KZ+KjH7qBWVhVM34fXj42HW/eiJHJQtvQcC/pHJa0GlgsiFYDLU3ArY1eVhJF1QApBtvLE2zmCX+0mHzr3o/kMmY3aDkuymXOtzYyWKHp9VfoQeLfVudit+x/Vi1KulqyPxDYhbcyPmMub9U+HpsbP3hW0wsPdHEqBHDI1TGwar+HX1qGy/OyD6ukbgpmQFYbhL5miHEQ6bEgW5fDa58zclKwayadWsMzzd+v0uqPrp3gVuGCd70AwG47dcbAS2t6y7eVsp5kGJyY51YvMd9Vt2cXTwbrtUvLKToRXqeWA6nc/D1XOmEZGEHdPwznsPWBq8RNwc93Gv6oEAQQG17fz/++BYmtz9B0+MYnvXS/hKewgdo1cB9ep1skEvaPHGeqzINw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To reuse __blkdev_issue_zero_pages() in the following patch, we need to make it work with non-blocking requests. Add a new nowait flag we can pass inside. Return errors if something went wrong, and check bio_alloc() for failures, which wasn't supposed to happen before because of what gfp flags the callers are passing. Note that there might be a bio passed back even when the function returned an error. To limit the scope of the patch, don't add return code handling to callers, that can be deferred to a follow up. Signed-off-by: Pavel Begunkov --- block/blk-lib.c | 24 +++++++++++++++++++----- include/linux/bio.h | 4 ++++ include/linux/blkdev.h | 1 + 3 files changed, 24 insertions(+), 5 deletions(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index c94c67a75f7e..0d8f1b93b4c3 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -193,20 +193,32 @@ static unsigned int __blkdev_sectors_to_bio_pages(sector_t nr_sects) return min(pages, (sector_t)BIO_MAX_VECS); } -static void __blkdev_issue_zero_pages(struct block_device *bdev, +int blkdev_issue_zero_pages_bio(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned int flags) { + blk_opf_t opf = REQ_OP_WRITE; + + if (flags & BLKDEV_ZERO_PAGES_NOWAIT) { + sector_t max_bio_sectors = BIO_MAX_VECS << PAGE_SECTORS_SHIFT; + + if (nr_sects > max_bio_sectors) + return -EAGAIN; + opf |= REQ_NOWAIT; + } + while (nr_sects) { unsigned int nr_vecs = __blkdev_sectors_to_bio_pages(nr_sects); struct bio *bio; - bio = bio_alloc(bdev, nr_vecs, REQ_OP_WRITE, gfp_mask); + bio = bio_alloc(bdev, nr_vecs, opf, gfp_mask); + if (!bio) + return -ENOMEM; bio->bi_iter.bi_sector = sector; if ((flags & BLKDEV_ZERO_KILLABLE) && fatal_signal_pending(current)) - break; + return -EINTR; do { unsigned int len, added; @@ -223,6 +235,8 @@ static void __blkdev_issue_zero_pages(struct block_device *bdev, *biop = bio_chain_and_submit(*biop, bio); cond_resched(); } + + return 0; } static int blkdev_issue_zero_pages(struct block_device *bdev, sector_t sector, @@ -236,7 +250,7 @@ static int blkdev_issue_zero_pages(struct block_device *bdev, sector_t sector, return -EOPNOTSUPP; blk_start_plug(&plug); - __blkdev_issue_zero_pages(bdev, sector, nr_sects, gfp, &bio, flags); + blkdev_issue_zero_pages_bio(bdev, sector, nr_sects, gfp, &bio, flags); if (bio) { if ((flags & BLKDEV_ZERO_KILLABLE) && fatal_signal_pending(current)) { @@ -286,7 +300,7 @@ int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector, } else { if (flags & BLKDEV_ZERO_NOFALLBACK) return -EOPNOTSUPP; - __blkdev_issue_zero_pages(bdev, sector, nr_sects, gfp_mask, + blkdev_issue_zero_pages_bio(bdev, sector, nr_sects, gfp_mask, biop, flags); } return 0; diff --git a/include/linux/bio.h b/include/linux/bio.h index 78ead424484c..87d85b326e1e 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -686,4 +686,8 @@ struct bio *blk_alloc_discard_bio(struct block_device *bdev, sector_t bio_discard_limit(struct block_device *bdev, sector_t sector); +int blkdev_issue_zero_pages_bio(struct block_device *bdev, + sector_t sector, sector_t nr_sects, gfp_t gfp_mask, + struct bio **biop, unsigned int flags); + #endif /* __LINUX_BIO_H */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 643c9020a35a..bf1aa951fda2 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1098,6 +1098,7 @@ int blkdev_issue_secure_erase(struct block_device *bdev, sector_t sector, #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ #define BLKDEV_ZERO_KILLABLE (1 << 2) /* interruptible by fatal signals */ +#define BLKDEV_ZERO_PAGES_NOWAIT (1 << 3) /* non-blocking submission */ extern int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, From patchwork Fri Sep 6 22:57:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13794838 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BA35E6FE49 for ; Fri, 6 Sep 2024 22:57:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F1696B009B; Fri, 6 Sep 2024 18:57:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A41A6B009C; Fri, 6 Sep 2024 18:57:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BA1A6B009E; Fri, 6 Sep 2024 18:57:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 18EC46B009B for ; Fri, 6 Sep 2024 18:57:13 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id CD5DCC05B8 for ; Fri, 6 Sep 2024 22:57:12 +0000 (UTC) X-FDA: 82535825904.09.E12BF6B Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by imf16.hostedemail.com (Postfix) with ESMTP id E996F18000C for ; Fri, 6 Sep 2024 22:57:10 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=QytKF2Kw; spf=pass (imf16.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.52 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725663405; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=llWa+KY1CsE699NO6/Qz9OdhajA1Gh72FplwVQHN8Os=; b=6fPflszuRPhcTeFOcA3lccJXUbIIsOaE4vln3gCTemn+UBdDXh3tt9w9Yswbti736EV6hy Z9EAF/KYuBneseiiIqIQ7xYANTB/gdrHrAZJbhlt1UuWDhjJQS6B48Il+HZlW5p8XyhuPL X93uLNz17SHnnbSUkmGkYlWCMPAArI0= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=QytKF2Kw; spf=pass (imf16.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.52 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725663405; a=rsa-sha256; cv=none; b=BbSxk1YpCKQclbwEvJgrcUzRlJw6Q0/RQnD7c/Aqe7g8uVP+mEAU3QVovKpCMSLkidlGUi ePsXyXAWjeLygB9Ffh6OfvUmCdSv1+pWNVdgDLPTaHK1H7meQaC5mZUBTnogGw78Nx0xSZ K7LtFqk9sKPEZutL5/5X+xNZc8OTNj0= Received: by mail-lf1-f52.google.com with SMTP id 2adb3069b0e04-535be093a43so3180053e87.3 for ; Fri, 06 Sep 2024 15:57:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725663429; x=1726268229; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=llWa+KY1CsE699NO6/Qz9OdhajA1Gh72FplwVQHN8Os=; b=QytKF2KwklOJq6NOhHtImwB54K+K9k+jatEGP9Fwrykjh9OwK5Zo5GE0Hpye3BzK3C Fa5Md9zyD7r/Ly1JCOg6iRpQRkOv21lVOg12LWNKiJ/9CizUv5neANjr9yAk8Gu0mje5 XPhy+6X5pwrTh/4eHwk5HJZcg++twUTvobxyRRNaWACnlFfd6X3kmPYfVdECb6YdMLV9 R1Ax9xgbQL4BdzVf/Ebti1/udBNJiBhsfNEVhpeflx6+W/ciVEjDy8qMKdPhzcGiBknR MorTRlbpI6srbp2A818PR7I9UDbQxl96mLzAU92LoaSH9W8B9d6U3sNPcg157CFUI1UC yldQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725663429; x=1726268229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=llWa+KY1CsE699NO6/Qz9OdhajA1Gh72FplwVQHN8Os=; b=CBjMmq5lK5sRYvqWpIYHl9wQ7fDd6J1rAG9CUqCIDqkHBL3CaxDfENmvG6RU6BeUO1 ZlI2Ili3sXpXtGekTuX+ygc5U+ns4CfHfcRiUXe+5bkez4U9428LmQdd9PeIZ16sEvUa oRNTlJyaBx33vJ2H+uo8KUVAs9iu5tgLaA+PIWyJUqpbsZQxHj2mP3kuCTm4qmCrkvLD 4TCfDoyPu2F6L8CRubxcivgzLQpPBFQ/h9QzqNw+ksyaM6T/0iSUU6+0lnpuN3ALnAWf vl5ElOYz/KeW8bGrYhG72YT3viVDuESBcjF9M4AZ6PCqN19/o7Dhfu3bnEwm27zIaY8/ lMeA== X-Forwarded-Encrypted: i=1; AJvYcCWZ3NDgcrxcUPev/PPbqIJZzUGIeke1k8fOMzt48MGfEYZIyqjYUQwnl7HBPTsbzusB2/UcfdRfZg==@kvack.org X-Gm-Message-State: AOJu0YwQxrU3VKgXS+lbZpdkaYsQ6J6NI5EOSAGoKDP4n04/0Bq3F5xb V6wQi1LLR3QrtOh2pyc9y6yaK3uhSpUr9UTwGvkElP+LmVa1zwhb X-Google-Smtp-Source: AGHT+IF7dy6Fwnb2jmgqk1DE1zvsMyOI+fNBEs5xezgv4QpyDJkwby70G9EHhdLSnvBeBbSkACl5HA== X-Received: by 2002:a05:6512:10d0:b0:52c:d904:d26e with SMTP id 2adb3069b0e04-536587abee5mr2790225e87.21.1725663429138; Fri, 06 Sep 2024 15:57:09 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.146.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25d54978sm2679566b.199.2024.09.06.15.57.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Sep 2024 15:57:08 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v4 8/8] block: implement async write zero pages command Date: Fri, 6 Sep 2024 23:57:25 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: pgdwqx49uq8toiwi8w41tera9u8uinga X-Rspamd-Queue-Id: E996F18000C X-Rspamd-Server: rspam11 X-HE-Tag: 1725663430-726855 X-HE-Meta: U2FsdGVkX19owr+FigYWLGSm/KFjNvhKQ31r/v1ubWpZVErwGc9pLWFwSzjs0whLheClpKJcdxC83oxd8JIkGqDEvnhSV3xKW/wclDVDtpwhmnb5PEvSZkvkp6z0x76WcuiqAxdMKexkJ7qND6Xx39NCkG8wfP36HugpT1HRj9chPlla8KF66s2Ra8M+wtXrE+sK/Inm4/3awYt5Ybm60jCNfxgyXcxu1PKiFe3k15TDtz+F2Djfek3WqvDV3WvRCW/uEMFZSu6HxABCs7W3zIz+TkKOWFr4Z/z4HnEi507jPO3xRRyq+Y+OV1fYMlZttzAHB/x6F4jGAssoqfROOGxb+yjlCv+73a+x/zW+Qt/byy68OSJCorGvziShYurYZ8LywgBCFvBs/pBd47UpzZBfKqDOwxIhu0tWj00qhoUrZYGeUfCeXLcC0MwYQL9eZoC1eJwZFwhA6OR40Rfhbs41XTfKOkK1n111uEZofFT3vOMjbxF2/TjOETQSgCA+nncs68BrNaLajtPtJJC+uh4IqhY4MC4vH7RS8kj3VpSKedF+zWwmz9uG52xybv67WT/q1a51QgqSjcOraZTcmxj3WSrHJ5O3fSuk3HMFPhvJj8cHJyrs35m+DF3yc9F4oLgMyvSF24/18r8DxUzZJfVOdbkuzN8ktIL61hxdcePe0m3x4JSTDsPavSfVbVvLnQ2o6qwB2BDN4wT7RvGCFRMaZTq8xRYsilksMvdrcxs4gFdd50tg1lORGUBaBwABAePCyGkyKwrTT1dMq/bsJwqg+YB0JvQM8q2zuNX1BpPMMCBu3PcEIjumz+2ZmZLecg4ikdXi3IuBBgwyg/wf8xoa0DM2NBVE1AWaKYHPOutM85xJ7ZJDI11eSBRdRnxMnv12Ccs9aV7krJ+QFILzKMvLJR3GeZ5J04/JgKJXIuEPa/D9zPjZSLB/ntR9j4u+fbNwCEO12sfdjZDrZSL ro0kVtfx UgwQ2V+f2JAdKt1wmG7GMtb1UWBIM2rLEhkmwvV3sRsw4N8nMjkGTwAffqOLEPE448YppYFtbdl2cCw0kMjtmqUUG9UqEBVtqmaSDosYv5983NGjhxoiNFT8Zf8fEF8jozfRqF+OhHjdcanp6+fkA9za11vPSuWA1JGCapB8MZXq67nQGrA+yNTo7Is1VVf3QlpaVKYuCN4luisAabyPTWG5KRSNg0XtE3vWioJKD3ZZGaD9b8859mdnLLRCKlxzXA4R6AFuhsFWS8BvLZ7gQlI8MPI3jvytC//FGfNR9CtfldSdA7nUbREVDwA9p7LBErgat54l8H9E6Vq5ykbY1pbkKFAWQshbrrr1y3TZUs1inQWtyLlBWOYkrC1p9qGkKa4awKnvADdh6VyCuD4wydDyavEE+phrafIhn3IL32YCZmxtwDu7d4giVdyK3mxECB0t3XZXqR0jQw7hNdhiW2mo1bMXEfohTXYdK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a command that writes the zero page to the drive. Apart from passing the zero page instead of actual data it uses the normal write path and doesn't do any further acceleration, nor it requires any special hardware support. The indended use is to have a fallback when BLOCK_URING_CMD_WRITE_ZEROES is not supported. Signed-off-by: Pavel Begunkov --- block/ioctl.c | 24 +++++++++++++++++++++--- include/uapi/linux/fs.h | 1 + 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/block/ioctl.c b/block/ioctl.c index ef4b2a90ad79..3cb479192023 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -774,7 +774,8 @@ static void bio_cmd_bio_end_io(struct bio *bio) static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, struct block_device *bdev, - uint64_t start, uint64_t len, bool nowait) + uint64_t start, uint64_t len, + bool nowait, bool zero_pages) { sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; @@ -793,6 +794,20 @@ static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, if (err) return err; + if (zero_pages) { + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, + struct blk_iou_cmd); + + err = blkdev_issue_zero_pages_bio(bdev, sector, nr_sects, + gfp, &prev, + BLKDEV_ZERO_PAGES_NOWAIT); + if (!prev) + return -EAGAIN; + if (err) + bic->res = err; + goto out_submit; + } + if (!limit) return -EOPNOTSUPP; /* @@ -826,7 +841,7 @@ static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, } if (!prev) return -EAGAIN; - +out_submit: prev->bi_private = cmd; prev->bi_end_io = bio_cmd_bio_end_io; submit_bio(prev); @@ -904,7 +919,10 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); case BLOCK_URING_CMD_WRITE_ZEROES: return blkdev_cmd_write_zeroes(cmd, bdev, start, len, - bic->nowait); + bic->nowait, false); + case BLOCK_URING_CMD_WRITE_ZERO_PAGE: + return blkdev_cmd_write_zeroes(cmd, bdev, start, len, + bic->nowait, true); } return -EINVAL; } diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 68b0fccebf92..f4337b87d846 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -210,6 +210,7 @@ struct fsxattr { #define BLOCK_URING_CMD_DISCARD _IO(0x12,137) #define BLOCK_URING_CMD_WRITE_ZEROES _IO(0x12,138) +#define BLOCK_URING_CMD_WRITE_ZERO_PAGE _IO(0x12,139) #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */