From patchwork Wed Sep 4 14:18:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13790931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BA6CCD37B4 for ; Wed, 4 Sep 2024 14:17:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9806E6B013B; Wed, 4 Sep 2024 10:17:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 906CE6B013D; Wed, 4 Sep 2024 10:17:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 732296B04D2; Wed, 4 Sep 2024 10:17:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4AEEC6B013B for ; Wed, 4 Sep 2024 10:17:47 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id EAE251215C7 for ; Wed, 4 Sep 2024 14:17:46 +0000 (UTC) X-FDA: 82527259332.21.8328F81 Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) by imf24.hostedemail.com (Postfix) with ESMTP id 149D6180003 for ; Wed, 4 Sep 2024 14:17:44 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BmGDRBdH; spf=pass (imf24.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.208.42 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725459441; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=7i1IN3gtI/Pm9aIbqA5+eol8MKbseexeRCjkJFIv42dJG9DuIk8lHVTOdbmVYlcGnhM3+S hVCbwbiubCSeH5UT15x2uJdeg9bqezAcJ+yTGoa/6pxRreanSnXnzsibKyx0iIBiPR6PoS EWFus0zxPucYOfHCCdcqh5y4XK/BJt0= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BmGDRBdH; spf=pass (imf24.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.208.42 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725459441; a=rsa-sha256; cv=none; b=WV7yy4p0++xLsC+ko4vxmtcrmjv1BvRyxDAoVK68MLRShhMDdPzc61aPRBmG7f9ij2PzlK Pi0FkCTCJBXdzM+rukrLghlDhzYM3aeR+w62HcoxQOPTruqT+pnWud01jDTGG0acborc9v o8vUhHoDqLH/abdDYj/1qpZPZInjRRg= Received: by mail-ed1-f42.google.com with SMTP id 4fb4d7f45d1cf-5c245c62362so4454370a12.0 for ; Wed, 04 Sep 2024 07:17:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725459464; x=1726064264; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=BmGDRBdHJpYoF+FIgjgnit6TMm/URSEL2pxJ+lrccszoF+8ymQ1t2ljZHJwA1by5Vm Cl5vHN+d78VVyspiFJAD0K2kWd0ibqxQSNmjd1/0XrH4wD9HcE8ngdyVXUrSg//hhqOo tZUho0i0exXB/YBx6L5KtCXqeQOWDpG9wJWQbps8mao28pT0QVwFg1qLvhLPcNCqnUoB /zq5j4oKl1oO5kDmpNWttzWVgLbfwY7OQCLp0R6YO682kvukQvsCVbXeNRv7EAycPLod 9hXiVRSUCuNidxBPx7e4vB1VN0EAkgCcBj6LG8eZ9PeoqCZ/NNVPIj3jWHvRgWOn/FDf rRPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725459464; x=1726064264; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=K+gtlmeJoZOY0RzAw8NPkP1CEFtKcaQnIp6rwkfLY+hU8APyBhNr+obU3kJPhTj5dn WSfsTl6pqvyJzga7ASbaaOb5nAiy3UpoTtXQN6DX19QWjgkFAgeDOnvddkUeGI+1HfgJ rfdxHX2L8NxV+mIcm998xgMOkWxrQ86rUGN17fC/MUJt4TynaC/oWkvNLmEnNbJIAXO/ K/NK6Fi8spOFdeFrvNLdSJPoipF5+Jx3bVWjCldEfGTN86f8eOIo9QD9i82I5HYjkLcn Hxx7ZsB7+LhPzrryDzA8JkpAzp6fwYReh9hwwATJX6QwWIesEz5EUcqXgqftsmsfxXA/ fOuQ== X-Forwarded-Encrypted: i=1; AJvYcCUqVkf6GCRP3mwe7IHA+QRNCul75wsHDQOg78weRgyRtT+0UbpmL+cgQPOR9OW1FcSNVx6dVGD+ng==@kvack.org X-Gm-Message-State: AOJu0YxU7+dpBc/Gizpm1b8Jz/zeoiVqs7WFSDHbJFKvI1S8eTcUjG7S NnltoqjeDwDePVqJyt+3pYmeahOVKr8WgX7KrJFwEwS7aGRoaYSw X-Google-Smtp-Source: AGHT+IH097ivB/9WxgCVpcO8B8ihxC+WO/gvLLOLQq9DQZmLBk0QFnqSJuSVFdgCDV3eF0HI2K1RcQ== X-Received: by 2002:a17:907:7289:b0:a86:9f56:21e6 with SMTP id a640c23a62f3a-a8a32ed6109mr297583066b.33.1725459462834; Wed, 04 Sep 2024 07:17:42 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8989196c88sm811160766b.102.2024.09.04.07.17.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 07:17:42 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v3 1/8] io_uring/cmd: expose iowq to cmds Date: Wed, 4 Sep 2024 15:18:00 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: mp3o6dm7jjqyqcouew576gsgs179pbwd X-Rspamd-Queue-Id: 149D6180003 X-Rspamd-Server: rspam11 X-HE-Tag: 1725459464-551763 X-HE-Meta: U2FsdGVkX1+AV5JwEAw+l7BLzsz+f+N/nwqmpMAJ2B8/mQXdNjNEZhStcmVWs+4IVwTLbT9pXmNARhM+1g4/Je7QulsLUfb8TIJp8gLIY+FmhuHDPkENTXRnVKkjGhD2+lh3nfBFH87sgnGRrQMApgfHYtlmHWH7fFZPKjc7pt+YJ7Yt39xzrFNcQjJidQJsEMAcBmDxOePmOR2dg6CKRMAW2PXOe5RnJ9xCI+YXrLaAptb6hxlu1C9cySai8WP9Gmn8788OTQYgd35O11Uy30qqj8rOiexcGTrTUajR87MySOkJJ0GTIbwFp92s4Tj77Fe9Io+w8zilQD1a/immdafk3TtyyrjWuydhPhUTfz8vmJkuhDF+V/zz5PrhL3k9viy74adCr2gdyIhj2gHtRN7VQRZ5Nb7WoyMfZ1WoJnPjKym+k1F06NZrk9kuxYHeFoC4whZNpJbwKTW6yKME0vOXyfV8wI17G9DtoQYbjsJ1N3gIlfemwNRlNBfrHHsjbVlZHT432MSq7AD1X/8PcHFV8rLVWwY61MkJ2QHFnTUncG07WxAPynso8333idKumeGodqPG+ZyFv3EkYF2NqALWydCZ57HRY/w02NVATXz19NZwsrySvzU1nlITPLzXUf54o8IpOXS3jMMbBVnOUppcdTde/HVhj3vb6DRVhQYNiw9E8P2wr6U5Sk+jW5VeitbM5hv1xJNvRnIY6w2W/GuXXucdGrlwdLIJIh4bSJcbyOWWcaVNkNT3Rt2S+uAy2XYbFwjAlyfXeMP7VRcub5bjxmvdQ9ieHDtLVWbn6ZmWsCt6Qq5O59c4BpSDaw5NzAXe3qvdEWgDPuvbxBEoAbbQ3/lpClmHRh+OLhBkCS1XPa0x14nkNOFjCi84264FevhhFwG2Qi+q1S+d++6nnpxymn0hJ9atUbReH3Z6GHvco+i5kEs/eMlxbwDu/vMx9WomsbxCvwCDKeh0Mi8 72sqpAKh q4mPvmNY26wRU14JRa3ftxGeXmhfKo6SVlcGB6G+SRFOCtbUnswJhu1LqLBqbg0yUC6deEdPfYyYXvAhsmGIZT8fUjb6E1N/yyXPZCkNcNNJqcJNhGV3yfNiJiI7Efcex/9BGbrOGPS+fLYZmTZILSGwCHKIHHec1mTqOvgziAjKYeqC/M+p9vRVBtWmUC+C7WEJW+mHSWKkBZk/iWLWFQ9bouIQUtCxyoMN/BoJlfBPjAtvRVa65rIkWk3k3fC5Xzh2RizZX48Myq3gNUOuJhjx4atLWKw06unM1t0/NnYeufNdeTZ/poKSnFE36h/pSYChKFqjq0ApnLMtuKT+IBVNe0uaTy24Y6Ys5GjaPAwMP754Z6BNj3wCcUB6Ks7oI2YcUST7MZ6qSVYW5gHgTGb7M/bKmAUvNZRf31gkcqdXtr1IOCbEPgCuWNuWvZh1KUm3iSnPcaNIzGUHC1TeIVj+CtHSlXIkWFjiGWxrh9FzHsjwqRqVcod5Sa2icW6hXqctwP0quz7zw4qXlYptW//oaGg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When an io_uring request needs blocking context we offload it to the io_uring's thread pool called io-wq. We can get there off ->uring_cmd by returning -EAGAIN, but there is no straightforward way of doing that from an asynchronous callback. Add a helper that would transfer a command to a blocking context. Note, we do an extra hop via task_work before io_queue_iowq(), that's a limitation of io_uring infra we have that can likely be lifted later if that would ever become a problem. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 6 ++++++ io_uring/io_uring.c | 11 +++++++++++ io_uring/io_uring.h | 1 + io_uring/uring_cmd.c | 7 +++++++ 4 files changed, 25 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 447fbfd32215..86ceb3383e49 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -48,6 +48,9 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); +/* Execute the request from a blocking context */ +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd); + #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd) @@ -67,6 +70,9 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } +static inline void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ +} #endif /* diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 1aca501efaf6..86cf31902841 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -533,6 +533,17 @@ static void io_queue_iowq(struct io_kiocb *req) io_queue_linked_timeout(link); } +static void io_req_queue_iowq_tw(struct io_kiocb *req, struct io_tw_state *ts) +{ + io_queue_iowq(req); +} + +void io_req_queue_iowq(struct io_kiocb *req) +{ + req->io_task_work.func = io_req_queue_iowq_tw; + io_req_task_work_add(req); +} + static __cold void io_queue_deferred(struct io_ring_ctx *ctx) { while (!list_empty(&ctx->defer_list)) { diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 65078e641390..9d70b2cf7b1e 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -94,6 +94,7 @@ int io_uring_alloc_task_context(struct task_struct *task, int io_ring_add_registered_file(struct io_uring_task *tctx, struct file *file, int start, int end); +void io_req_queue_iowq(struct io_kiocb *req); int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts); int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8391c7c7c1ec..39c3c816ec78 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -277,6 +277,13 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, } EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ + struct io_kiocb *req = cmd_to_io_kiocb(ioucmd); + + io_req_queue_iowq(req); +} + static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd, unsigned int issue_flags) From patchwork Wed Sep 4 14:18:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13790932 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEC14CD484A for ; Wed, 4 Sep 2024 14:17:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6243F8D0258; Wed, 4 Sep 2024 10:17:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5866F8D0253; Wed, 4 Sep 2024 10:17:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 400A18D0258; Wed, 4 Sep 2024 10:17:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1E26B8D0253 for ; Wed, 4 Sep 2024 10:17:50 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BDA5D815CC for ; Wed, 4 Sep 2024 14:17:49 +0000 (UTC) X-FDA: 82527259458.18.FBD9EC6 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by imf24.hostedemail.com (Postfix) with ESMTP id C74B9180004 for ; Wed, 4 Sep 2024 14:17:47 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=A0tc6jLy; spf=pass (imf24.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725459371; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=0C3cPvtqnOq44CFLnblCHAjTMfsstoShyAQFIXZSatWvUahinkP4YMZUJ/vBrd8SzDXKnr weJRn4oDOz3YH/f4RkhbZ76Y3cbsjPTd8ny//iIzqNkmjwsfb+UMSkce5Uxkuuu7ekt5Lz FFljKGZ+7oMaI2OJ3SkaqPKQD3Kzmc4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725459371; a=rsa-sha256; cv=none; b=m5tnl6o6eGyt5kDfNJXoK+SYfaLsqXTgYL9Ddz4rSH0WBn+bkoUr8/0SawydmgHMlMzWMo qIrEmWM0aMqy1nE3gvWCn1P5+AJE/NCkhWvbTsTiJzg2mhMtXp57st077+Okl6Sbf8CItl Zr0sGj4jnFDVE68tBkQnAoZNashvcP4= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=A0tc6jLy; spf=pass (imf24.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a86883231b4so814828466b.3 for ; Wed, 04 Sep 2024 07:17:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725459466; x=1726064266; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=A0tc6jLycDvUOeJh1sNbEEZFb/Y091x2EtanpqXU5PMMqhH3zRRiE4rtLWtOgZG3hX uKGBEZRnYIiEwu9UZN6T7h1FJzfo4XuINl4DbXJAm/dI52YzmJhCJaq0dlUDkp38yNNi OFx6oqmg6c2wTiBNxH+1jBNLeX8k0H5sf7X1BmkfkP+724wbHsaCVahcGokg7bTlKFW9 kgMK36EZrz3RwiuD6Hu1FdjlpDlS7om7a+tQhEq9/pdRaIth55zTJBe89tbfL1xs+tHv v1dbKavVup9WKOhH0ITZ0IdUDEDPOc1j99b1VEiqt4ZcvLSAKGwhc1emtrLAt0UEyLoZ UYDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725459466; x=1726064266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=pzIRWcdp0jzq0T7NnmXkcz9I93JYaVJUQ+Yyg3SNpd7I36jUD8f1xLSpKi86H3n0Uy 6R5MoAEWChDQCrrcQH16vuas4PJqzeX6kg91jtSuYRdPWWPkGowG7zvFx38Bo1LMU8lH saP6Sv3k7j5QATSs8HvyOsuGy3yjBp2e0gvNrCNxsRUM9iFNjejyqgDBxsdZlEnyKEor +q5D/olASyyHL0pw6umxKyNCHabXPr7FXtkgLBTRt+FweYo/aFB6Ow4PzamCGrwUoF8u fkztdo+fKkNbXULJyF0od8PQVyoi8PvGhbJKXseRzpqEgcROxenCxvyHrN8cts3wAKkD mEJA== X-Forwarded-Encrypted: i=1; AJvYcCUmrg5l1vh6yDb8DCHZaCP99/iMMRlJ6JFtYdUvgdq9UNEzfPeMAvClNrda/aRClfjO0b5eD0r+Sw==@kvack.org X-Gm-Message-State: AOJu0YwrHhgi4nvUDzlz9bioBAt6Ur0W9MoD85mKk4sbUDxEfqP9fz/Z Y1Gk32s8veqsc5+doWOXDNv16yedordlTzXzXmZQQg3qGpnFrCmrf+47bg== X-Google-Smtp-Source: AGHT+IElud/tGXrs7q4hPENcWMYIZSj7RTtRqWMNx0QoFf5mcRK9CPIkT2ZU7thyj+ZNDXYdgCuEaw== X-Received: by 2002:a17:907:a07:b0:a7d:e84c:a9e7 with SMTP id a640c23a62f3a-a89d8848bd3mr1056922166b.53.1725459466090; Wed, 04 Sep 2024 07:17:46 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8989196c88sm811160766b.102.2024.09.04.07.17.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 07:17:43 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v3 2/8] io_uring/cmd: give inline space in request to cmds Date: Wed, 4 Sep 2024 15:18:01 +0100 Message-ID: <7ca779a61ee5e166e535d70df9c7f07b15d8a0ce.1725459175.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: y9k1mgfg8g8r9hnz4gzz1tj43xt44hgw X-Rspamd-Queue-Id: C74B9180004 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1725459467-375975 X-HE-Meta: U2FsdGVkX1+uFxnhDeod5t/OG2OB18qwOdjams+83bD089KW8rEee7kwJCuzTrLwIc+omuYdT+rm5gNXCnbOzqp60SIvpiMjNYhpu7ZWaQ7bi8cDNc7cd79Xls0+ozrYYvHtM5rJ12UhLMwEz4LgSxCU8ezw7tad8dWFWc/jybGzsvLdoXnTEtHxxhY49zs5yCPupFDhsk270kkvu7EX3Xb0+9Vo+jRl72XUCxto/stl+vaVLh/sDB0WImAgGgiP3Bg+vdEMHGgXjQXL91WVhG2iPvyBnD5UjJ1/6tL1dkTSWwxLc0KYPga1RqVDvKzoq6ADIe6chZgH/f65s+8Z7AiZW9wkPG+mClh3tQ/NR6C66AccBiBQr5HKNFgqo6thbsRxRZ4997w0VqVTwGl4NsntgTgBCzGmxAPv0a27Iqnr40PHtzZAYxI6jXX/Jn2LDnxW0Ct0vHztkAtQcA9AqX3+5cwtklkwH04IdcG96Kt9x2RmsvqabR55KDV/qRSoix1rfm436nzxLjfJodKi9Y1IRJGidMdtXXW3CpLIErgvhZKpnWN57srrd/wUDwBtEnHEdIb4944Uw8PahU2EU49H9TmhLk9Trrv7VI/AGGJlmTTbp5jhstzl6Q+FmqNr/uqzlW7TanYNLHMbVLxbK+pz0mjZQWy/Sxx+h3ZsrSTdAj2Sf+TAc9hpcin9WgvA/ghdnsHB059liYEf01/4SXYF7uTUOn05qhgiyLW2u/5/QiVQwmpi+Bsfjl4e00HNudcrRuxyfar8cEGzf+zopPhpjzXkTl7zc2PFHWcmnuJUqcc2jJaD9wV3DNkW/Q16efbZ4bL1/2sny5pi31n/vYL+X+MkRYsPfWcGOS8q7/MGdjwffEVRpoHV5rmcr1/zpmB1Q6xmJGLBJdnrIsl3QwYp4Irocp+Shb9FOhZG3l/FWSsHZ9gApuIrf9Gt8L50lElvEstUtb4nz1yrKOh k2a01F5I bpCFzZkv28gPR6/faCfDPiVkR9vxG03Eues9Tm5GIbbkvbplzyWiDehQhTNyPyQSZp7o8L+Kius4wxuV12bsr+NV4sFwcfPvCghJMV5p/Qm1eASveZR5kT1Ius+3Svv3/WgVNCDTycDT9JiBl1VM/ucleoNZKzfxZzktrhd/F6aQqrqTZwIQQzktpppElIeEpbgYiWswTk8kXr3XJjDWCo1XnyAvlBPZEOQkwtk76x8IS3JoFjy/4xbyAM48PZaMdzivrys7MSDqcyBrSCRHAmT9lilY7LAW76gbJj8Rj8/xg+txRk0Xo4WZxnlso+sFmfMMCvBOri7NZLjbYIySvCqotU4pS/3v/Z75htxKE86VW+xeSfFj3u6s0QxRvqIYD/oCgMsQAHzDED2HOTuLM0ZAcgYsZujfPbbdGwA683fl/Siv7+CqxuP/tJw/bxKBpwHeHY00+xfojb881jFiaCJxr6amMT2epV+dcI/rFS4bWWaNlglVryx/d6zbhv2w0994eS/QpuABcmZ4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000011, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some io_uring commands can use some inline space in io_kiocb. We have 32 bytes in struct io_uring_cmd, expose it. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 86ceb3383e49..c189d36ad55e 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -23,6 +23,15 @@ static inline const void *io_uring_sqe_cmd(const struct io_uring_sqe *sqe) return sqe->cmd; } +static inline void io_uring_cmd_private_sz_check(size_t cmd_sz) +{ + BUILD_BUG_ON(cmd_sz > sizeof_field(struct io_uring_cmd, pdu)); +} +#define io_uring_cmd_to_pdu(cmd, pdu_type) ( \ + io_uring_cmd_private_sz_check(sizeof(pdu_type)), \ + ((pdu_type *)&(cmd)->pdu) \ +) + #if defined(CONFIG_IO_URING) int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd); From patchwork Wed Sep 4 14:18:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13790933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AF10CD484A for ; Wed, 4 Sep 2024 14:17:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D28F38D0259; Wed, 4 Sep 2024 10:17:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD8088D0253; Wed, 4 Sep 2024 10:17:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A8DA18D0259; Wed, 4 Sep 2024 10:17:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 89C508D0253 for ; Wed, 4 Sep 2024 10:17:51 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1E879161611 for ; Wed, 4 Sep 2024 14:17:51 +0000 (UTC) X-FDA: 82527259542.05.67C15A7 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf05.hostedemail.com (Postfix) with ESMTP id 36E3D100008 for ; Wed, 4 Sep 2024 14:17:48 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=D4WSbmTc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725459445; a=rsa-sha256; cv=none; b=xeCzNQG+25szIa9868MfGh82R20WIBa0nHlR6BIubwLDF/dBUep3Uozkh2WQqDg/QxCbBX xFlYe2RDb9u6a6Sux1D0zn+g2YsoLjiJzxfJCoKzQ4MXLAS/+16SNWHpNA6yalqT+iVq/1 ysPrvZ2uWQjc6HrVaoho9bmVWsW4rBo= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=D4WSbmTc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725459445; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2XjW/1z5M+e5L1HlSBo37JIC76JYeYPBxc+HZavMul8=; b=pjYA929/RYUIe30bSJovFELR6prJj1vi6P+luEkzhCYotMI8En8loxwV+LT2U3ocb/maFW RGCyFCtZIpZ7REiJyglE2LK2uNCgZEu1laN0QLI2nas7XeZAvGcbh5dPdOsKbyE1IXY18m U7ixB6jGgw3YtfEnENpA61wXLFid8Fc= Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-a7a81bd549eso611147966b.3 for ; Wed, 04 Sep 2024 07:17:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725459468; x=1726064268; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2XjW/1z5M+e5L1HlSBo37JIC76JYeYPBxc+HZavMul8=; b=D4WSbmTcGiMn0qDdrp2rBv9E/fiMIeuViCB3i//BfE3ckn7ROO5Fyi1C+wZkDEjNvO DFRO3hp0wrzeK6lm0i/9vUPu7rWwKjJ/U3FitnZeH4ywMTEL7SRyDee6q03Bt/+30k8l +6cEdckiPO8pPZT/EwOskr3IRHwa8WHBxPcWahqcpiggoAKQ3m+BcrW9BT8CLwGsQYp0 qoY3sOPLNOQCSytnNUwPzQzQk39PJbsiNcU0P79NqOm5c2PexYKJ14+/qy57dkpjCjuK q9lAmeWiqP6znUqzEhBUunm+eIgze3ewAlIoIvAf4X4B3eS4uLAWk7efG1yZ3iLEQ0fy b1CQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725459468; x=1726064268; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2XjW/1z5M+e5L1HlSBo37JIC76JYeYPBxc+HZavMul8=; b=ugAGfxuulCqWPYMeBGBAcytyc3E37Ag6REWznM3iG3Qr5LKJEDSuwaUe1neXplwAyp 5FW7kkA9XPh8tc62uuecQFylNF1MQJsZUVZkmCmwapBXB76Y+wdFlIXQQrPSmIaj42vF A8IJm+U2Q7umN1HM95yezDX1pUIwj1EGVsSe4kXxNXU4wuKfxT2YCvyXrhxfy/ch6peb 7COdld1ZdeqGVQavHkNzBJbGukfdExPXUJts87FzaoPzRuxGEa6C9QjqUZZ0j6Ez0XiA kYelH/S30g+CdKZz3zELLaAnmY2nwmEBgmFPrx2qOYBLJObL3h8ShWEdja7saAJyxs2G 2NDA== X-Forwarded-Encrypted: i=1; AJvYcCXeDry9SK5wvVE8QDKpGsnKtSJYeZOW5dhsWu2QL8vupImNGKF13I7AngZCiEZlmqB0a/KTH0Fv1Q==@kvack.org X-Gm-Message-State: AOJu0YzCtbBOKrESlT8CslAmuyoL8L8DcpG3Y3M79WeFDQ2IsXh8sjH2 NKfhbSfuGmdvUs9GzvZkVOblMtJwQygqc7IX6X/jRVNFihoAXhFO X-Google-Smtp-Source: AGHT+IEYnlE2jE9AeI30LzbXw8Pi928CHHZ2cfjLj3+Sl10dnyCa2YAie/Zh3flyVcceULOyKSQ8nA== X-Received: by 2002:a17:907:1ca4:b0:a77:c080:11fa with SMTP id a640c23a62f3a-a8a1d4d2201mr484969666b.48.1725459467509; Wed, 04 Sep 2024 07:17:47 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8989196c88sm811160766b.102.2024.09.04.07.17.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 07:17:46 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v3 3/8] filemap: introduce filemap_invalidate_pages Date: Wed, 4 Sep 2024 15:18:02 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 36E3D100008 X-Rspamd-Server: rspam01 X-Stat-Signature: sq1pw5rdkehgkxpo9brdxw5sb9cfed9u X-HE-Tag: 1725459468-384231 X-HE-Meta: U2FsdGVkX1+O3BVyJI6VwHF0S6utNmW733o5+fYOnrQdZQmUSSPz/43sLyLMfnvDgofO0C3RVQaSgrrJuddNuvvJ01ZRPBQcctItMkAcD76C53NJYyKsMNw2msL0uFlsog34jA8gjDdOaDHgtg6oSc8wy+qCupM5n39bEkhgOdc9noSxGcvZioPnmL0w/DMwQ7XkJxI47uYmlESMa8Lei313LS6YJ+eawGqmqNcIooWEL+PHH64We3vyZrV6BUBRgZTZWA1QL7JTu8i7rb7kZ4Moeudz2fRbHvPT2oi8Th/7bwhXmQaG9VSSe4pyE/3opNG9La3/6Z1I4Fad/dWqE3qk2MaL4LjHobdF/bmXc+YAGhZL/YmEu7wbuOvHOCd0elwy+s10iWHOltUzuPoPUE6udFotGIW+rZ12rEgPDCGgDkbqQ77zLFPzWlkprll5y+E+LrdlaSujIKpEovDkR1GkEVaogTioJg7qNwgXkJiQ7AU2frP6WSZ47Ky+Gmz8h1JoaUDjGhbUEvONqy8kBcbThaBgpdOjEn/IfviLC1qg+R6Ev49kMZA/Pa0aKKoYTPZQasAJ+OEB9gBSjmYbTDeuVklzZBW8pePFR0hjxp2c40ogjPXZAzm/Oh2qdRmDljLJrXuQBsTeIM+yIeNI2bP6qD1sHz7JbLT+sZ+t4T80KoLTafLGZ05t0poSssB9sURvxGi8q7L8+iEM0YhHyxmzDcCGvsAR+KIZKximq/abIOdGjvP3TJjGEXy2RnQWUjb4Yz9RalBMy4dvXNWwLnkziYdYsvJKuhjGL4NpN/UnhyISg/xW2J/egU/mn6EHT+83+QuhkKvFnxuHLT5iwve/jbDSl48b0xKLeJLqkD4wD8f1hUTXYzV09pPqUJnpvCIz+Xw0R9+Si+XH87Y2IiCcu47O7LgFQjBFCnaXT5eJCYSdQrW4S25NlQE7xc4qwtzJO5aXYWK5ui/NM+e k9/T7YC7 tKmHWuWBR1d/cAyOr+ykoClkhvJKagl3AZiDznLTEU931FPl4EUoLnwnBUzwyuottkvq9FUCOz2ZfMrPlz1V6E/6i6Z/3S3jqqzoSgbNbdgDDb1gOb800/uqWJTKoWUZ32Dw48hsg2WWPA+PtX5tUG9sQdufFlB3IPI/hAlM6dlpDUjTNXHhtdetTcsbnS1X/UVq1bg9OtxSvqnK+nVYdUeP20Hn/A1975u20aZcqvM0xQhEuz0PBypxAmhV8s4/dm5b/p2mIwTW/gjxqva0o6p4Z1X52Ikkad9ob6szqN2KdF+NiaHD7h8vK8QPuYKhHf0SF5xckZSWdSfaXKjl9inqTIXUKybKY+chFjCYm1i9iK1xfaFN83WRneVnCWczcVvK84PxKy9PtAGtM22QgIJciu/AnSKpYKM8Jo7fJfiqbpbNsCEM7Dyd2KWwiRwpuk8UiHVYAdAKgZ/jmedruf19Zns8lSHw52XZgslcdr4JaxamUD16VlTQguUezf6cWe5ofY/N1dy1XqR8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kiocb_invalidate_pages() is useful for the write path, however not everything is backed by kiocb and we want to reuse the function for bio based discard implementation. Extract and and reuse a new helper called filemap_invalidate_pages(), which takes a argument indicating whether it should be non-blocking and might return -EAGAIN. Signed-off-by: Pavel Begunkov --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 17 ++++++++++++----- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index d9c7edb6422b..e39c3a7ce33c 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -32,6 +32,8 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pgoff_t start, pgoff_t end); int kiocb_invalidate_pages(struct kiocb *iocb, size_t count); void kiocb_invalidate_post_direct_write(struct kiocb *iocb, size_t count); +int filemap_invalidate_pages(struct address_space *mapping, + loff_t pos, loff_t end, bool nowait); int write_inode_now(struct inode *, int sync); int filemap_fdatawrite(struct address_space *); diff --git a/mm/filemap.c b/mm/filemap.c index d62150418b91..6843ed4847d4 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2712,14 +2712,12 @@ int kiocb_write_and_wait(struct kiocb *iocb, size_t count) } EXPORT_SYMBOL_GPL(kiocb_write_and_wait); -int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) +int filemap_invalidate_pages(struct address_space *mapping, + loff_t pos, loff_t end, bool nowait) { - struct address_space *mapping = iocb->ki_filp->f_mapping; - loff_t pos = iocb->ki_pos; - loff_t end = pos + count - 1; int ret; - if (iocb->ki_flags & IOCB_NOWAIT) { + if (nowait) { /* we could block if there are any pages in the range */ if (filemap_range_has_page(mapping, pos, end)) return -EAGAIN; @@ -2738,6 +2736,15 @@ int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) return invalidate_inode_pages2_range(mapping, pos >> PAGE_SHIFT, end >> PAGE_SHIFT); } + +int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) +{ + struct address_space *mapping = iocb->ki_filp->f_mapping; + + return filemap_invalidate_pages(mapping, iocb->ki_pos, + iocb->ki_pos + count - 1, + iocb->ki_flags & IOCB_NOWAIT); +} EXPORT_SYMBOL_GPL(kiocb_invalidate_pages); /** From patchwork Wed Sep 4 14:18:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13790934 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8822CA0ED3 for ; Wed, 4 Sep 2024 14:17:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E13908D025A; Wed, 4 Sep 2024 10:17:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D25658D0253; Wed, 4 Sep 2024 10:17:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA19B8D025A; Wed, 4 Sep 2024 10:17:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9A27F8D0253 for ; Wed, 4 Sep 2024 10:17:52 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5CB93AADB0 for ; Wed, 4 Sep 2024 14:17:52 +0000 (UTC) X-FDA: 82527259584.29.9C19CD9 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) by imf16.hostedemail.com (Postfix) with ESMTP id 6DDBB18000C for ; Wed, 4 Sep 2024 14:17:50 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=M7RqykQA; spf=pass (imf16.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.47 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725459343; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gCQ1sFR6q4N5GvvlFRQ23FRxGveAlTZ5FLsIiX1fOqY=; b=S45T4hIiNpKxCQF0woi6DrrvEVaAJn9/tndym3cDOuiBHI49lC4sVpRxcdcE0WKktzP8aO rSJndxa+9EIx1TBVSEFKhFByNuyt19t2zIIzCRAIOXdPIl0GO2a7KBdBl1EA0tdXKW+ZTK G8BtyG/Nf+bilhjTeHVQ1by/THCqw9g= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=M7RqykQA; spf=pass (imf16.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.47 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725459343; a=rsa-sha256; cv=none; b=wgC2TyhAiHrfW3ymIVguSAOm63XnzbTfFIMZgIwyqlLKbzt9zXkLhwhnAoef81uRpNZtDI 3uiUqPWpHs1aeAAFXp+tNJWpRrc7bXaPKJgVI89ZlhcT3281RdVV/H47AbRCbVCqQy460m hDRzxSEJxqEeqf72ROxCUq2l7+xphnM= Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-a868831216cso789456166b.3 for ; Wed, 04 Sep 2024 07:17:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725459469; x=1726064269; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gCQ1sFR6q4N5GvvlFRQ23FRxGveAlTZ5FLsIiX1fOqY=; b=M7RqykQAvpXMxiq4tyWPRGbmC5nZyFVzpWR/OzoMQ/ctNMMdaY1UL3JXZ6cC0nOIzC 9YpZIFozh6RVj/VhhJ+BNNpyLXq/r9MrlHdlctwyoFsJNfktLezJbeTX3Oh3a544ehKM AwOP/SEjcyugoJ/V+Wlry0cuZObfZbpE8i63t2BP9sGANePtdwURBMD4A12pweS9tR4t rFTZgDQLaIQ8QNSDsrFYaNFRxseS12pVYfR9Ix3p99CrbUDxpfSfUFsgsamhJjM3fFeV BzG51ZEIEGgQHJj19oWHcl4Bp+Z3kjPcB+mdqAHraVorBoEnIzayPdil4YitH7Y+RXpx an2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725459469; x=1726064269; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gCQ1sFR6q4N5GvvlFRQ23FRxGveAlTZ5FLsIiX1fOqY=; b=SguobQYf0wsANswuxZX076rb7hWKqpyBSkNYWdTAaSTAB1brkrd36vI8re4LQgztxt FcjwAmEI4MmglDk7E3g0w4PGF6AMstBrLxUFwB/fR3jYO5kXTsieF8LS0Nb2rhLPcOik YnYWvLiEzS4QA+GqP9geY3MGsPZ5J+7Fs7tRHvfVc78YoXk1Qqy91QOKs8tjYOao4343 YtoP3/K2/dNVSaHOyVa99ERYN7lVv4aSof23g6+8b81DAWYaYPCr0vTbU0yBeDFjxx+I H4+8DCfg97rmvWZOdG+TLrIANaioM48qDYnzf4zlLR9TQljz65RTHhm+0fXA8hntSFkb j3aw== X-Forwarded-Encrypted: i=1; AJvYcCXeAvJH/xfnoeeurGmo24bd84TzLF/CT0lVxn4vw+IJPkFEY858FuUx0fWnK6esNdjJLTBtlhBMug==@kvack.org X-Gm-Message-State: AOJu0Yxvg+C8yDB2Vndq7y/+FdLT6fyxeSJ+PAzRtxH+3hkjSOZl9cyK ES+FgAykAXn7Rsjas6qwLdVSB1edps6cAESCwtvc4rsqAgON8rhn X-Google-Smtp-Source: AGHT+IEHqBvvipKDt3pVc4/T+6L4m8BSlDWTikVIuwjQS1tXdrsvfLbUudOAA51rWESeq0to4VuuuA== X-Received: by 2002:a17:907:6d02:b0:a86:a866:9e25 with SMTP id a640c23a62f3a-a89b9727ea8mr1148556666b.56.1725459468821; Wed, 04 Sep 2024 07:17:48 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8989196c88sm811160766b.102.2024.09.04.07.17.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 07:17:47 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v3 4/8] block: introduce blk_validate_byte_range() Date: Wed, 4 Sep 2024 15:18:03 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 6DDBB18000C X-Stat-Signature: nr5nuhzj99n46tibue8n9n83r54b3wdz X-Rspam-User: X-HE-Tag: 1725459470-501906 X-HE-Meta: U2FsdGVkX1+2V9Zm/mZRELjRRJ21ke9cdwvX5/GF2nlIOj0PqZeO5qQfOM+BbRqWlb15GCCUPaMhHZKLqbJPduCgTvDrZ69EALKxF65Xsa92ARYr+uWDXNCZ1aYwc/QG2n4APZZ239QqY4JgrcgJKOcyQDVrbB0dy1ntHiIdKtvP4h47IbzrMC+rMAUxtIqp+SteNY4uVmYb1szpLEHyEFKsRb3nIhJtFzSj4DeJGmxoPMenSuGYlC2Swl0jDbK7Hhv37t4PJ8XKwhCiu2jUc9S/3Cw181MvWwqR6XJdHAn0kKdXXYf9DkebxJUc79CQx03Hl9MQZxzEw7isAvv+hNXWfwml5F74vVvlduS9BWhVIvofOgbDvsdZXby7T8MzujVA9G7xwULXFBtALf3HArtT44oYBWHuLwLQuaM9+YuO84x8YqQiYpoF8W3YbGhyIvMo3u0tiYkA9j3QIxpFzEmBzuLacUJ0TiAYuj2oVugWie1O3bvyIg/EXO2+H5XOmq9BBzV6LZHudODl1j+qLyVSOdkLv7S0mCOx1LgK4lWj820P7d5O/N/yYMfQDgA6eDWtXdR3Deo14LeB0/BYplOUhVywfiCkYWI0o395aE/WRL3FR5Gk/6HLY3yEpRBTnSruLpPeNRjjEDCYoYS4kNKAD/qBiHJNG3W4X+vyaV9Oa5M8HRZvWFxvkdlYk1tTe7dU5cyFXmWsc8YNv6q35QvNWF8iCdYrE2gSNvn0SUBclfAwFFfVxPkfJsAeF3NkSw7bJti0Ce9bgt/3XxXris5HFhDGddP2m0KtzVWZns3D5tvZXwWoQq+L1pDataUSQlvnQxuGafKatv0IkbElnUlbovR/uTh+iisy7J7AsZatkhrdd0PuMhqtIbSIogJeu4bcAdZq1L7/kTIoeknLdd+fwqiPC4dqiEVCy66JPc2By8jyT8eFcYCkCynqhyJkBsIKiu7bl9nWTQkoczS NqmIef/j QnaDc2k4IrW4JSBnW0xJiGgFoGkScdXNBete5EDGxY8jr6E5ZZpqQYEgPRjbZAOd3z79OvTXiqOotqV+V7ps/bXGY8Ng5p8rsOCmP2A/a2m3bY2I/JqT8ieO9VxE685EGdeeAddp9Lba9KmMvLLNbT1MiakqdRSRNTTk99SBLAI/vQtL0/l0DTJHKMBCRai2CLWrvjCyU/6APHxsJX1k+XYGkl8ietykmDIJeIdBh+DBQSsa+f3l6JOIgQQToa3qcvW/v7zRxho5M35b2/1t+brbFIIoyRUXQQ8Bi28H3raoWrxvmEnQ4i7zdwM66kt2y75E7v1MFPYChne7CsD4EL/7tzqJanzOfpXlTv4mQhWlZoi8O98n1zsKrqmsOzmt73zmTnBFsqt9yUXgfMcaB/iMPHanWnI+6B65zGfE4ELQtbV09i9zvNMHA0COK3bJ+AezNDMaX6JhQ26MIiGrd9DS6qgFCL9PNzRmx9ZIslaXIEknq49R3AIkbae/zkZZy+63heGNRyNX+LTg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000262, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation to further changes extract a helper function out of blk_ioctl_discard() that validates if we can do IO against the given range of disk byte addresses. Signed-off-by: Pavel Begunkov --- block/ioctl.c | 44 ++++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 18 deletions(-) diff --git a/block/ioctl.c b/block/ioctl.c index e8e4a4190f18..a820f692dd1c 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -92,38 +92,46 @@ static int compat_blkpg_ioctl(struct block_device *bdev, } #endif +static int blk_validate_byte_range(struct block_device *bdev, + uint64_t start, uint64_t len) +{ + unsigned int bs_mask = bdev_logical_block_size(bdev) - 1; + uint64_t end; + + if ((start | len) & bs_mask) + return -EINVAL; + if (!len) + return -EINVAL; + if (check_add_overflow(start, len, &end) || end > bdev_nr_bytes(bdev)) + return -EINVAL; + + return 0; +} + static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, unsigned long arg) { - unsigned int bs_mask = bdev_logical_block_size(bdev) - 1; - uint64_t range[2], start, len, end; + uint64_t range[2], start, len; struct bio *prev = NULL, *bio; sector_t sector, nr_sects; struct blk_plug plug; int err; - if (!(mode & BLK_OPEN_WRITE)) - return -EBADF; - - if (!bdev_max_discard_sectors(bdev)) - return -EOPNOTSUPP; - if (bdev_read_only(bdev)) - return -EPERM; - if (copy_from_user(range, (void __user *)arg, sizeof(range))) return -EFAULT; - start = range[0]; len = range[1]; - if (!len) - return -EINVAL; - if ((start | len) & bs_mask) - return -EINVAL; + if (!bdev_max_discard_sectors(bdev)) + return -EOPNOTSUPP; - if (check_add_overflow(start, len, &end) || - end > bdev_nr_bytes(bdev)) - return -EINVAL; + if (!(mode & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + err = blk_validate_byte_range(bdev, start, len); + if (err) + return err; filemap_invalidate_lock(bdev->bd_mapping); err = truncate_bdev_range(bdev, mode, start, start + len - 1); From patchwork Wed Sep 4 14:18:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13790935 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAEB2CD37B4 for ; Wed, 4 Sep 2024 14:17:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2080E8D025B; Wed, 4 Sep 2024 10:17:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B5EE8D0253; Wed, 4 Sep 2024 10:17:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 007138D025B; Wed, 4 Sep 2024 10:17:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C85698D0253 for ; Wed, 4 Sep 2024 10:17:53 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 932C51A1580 for ; Wed, 4 Sep 2024 14:17:53 +0000 (UTC) X-FDA: 82527259626.09.848399D Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) by imf07.hostedemail.com (Postfix) with ESMTP id 89B7640029 for ; Wed, 4 Sep 2024 14:17:51 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WiXUzoJ+; spf=pass (imf07.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.47 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725459423; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=v7dFMMDbVFAIV5pvquHCkne/t7NzjwwosACmczIEiUs=; b=FaoWkaeo5BX0aYiCgNDy4JrynRoDS0IL9fWuDXAbrfN7e4UQipv+i7H38XAEVbOughxwrg L5wQyJhEXe+RosaIH6OSzyuuEciykfY1Y632XbPPk4cur0IUJaYvwG5si2PjV50DGsxmb5 7wzVWUroLu5mejcSefGghXcLg78k3Ig= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WiXUzoJ+; spf=pass (imf07.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.47 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725459423; a=rsa-sha256; cv=none; b=8XNcGGE5t9uWNp214+t5g4A7tK/y3Vv++7iGwU65OtH1S02FCkupFMRBDTHG0MrnK2p4NJ 4XcokV/o2fdahaQaoxt0FInlJlJepIkthTH6yCr1Q4WVdqbH1oLljdCOsXy8DKMgB0Ghaz kanHMo5ULvNrSw4GZp0LPR4an5uhO3U= Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-a8a1acb51a7so305592966b.2 for ; Wed, 04 Sep 2024 07:17:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725459470; x=1726064270; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v7dFMMDbVFAIV5pvquHCkne/t7NzjwwosACmczIEiUs=; b=WiXUzoJ+uVEgqGuNSPYMjTdGXGOZ6oTlGyV0xj17FwinUd5MzCeDsejm+ixQLuvQdU lGGczB1Ut1FlR9s50GjKChuMiSi8fnxgC/AZgG9oRmcpLFJHzO7hCBZFc52eLlp9g1hg lQCXEt2/aqno8rf1ed05ZckOnPWo39bz0fZRY1EZ/Q3Wew3Xzpo90Q0QUXczTMwZacGN IhoojpC4rTgOjFCs6vqsqZrzIrmh/RaKehkYZYaY1Q4zRDpcZULmdtnb5kTHlk56OPKb N3SfyjlJbfUTN7BCoWYzTx4S93W4SXjH5zXaA6o3Vt29mvN1HLrbtvvMfX9sPysdL0/b i5hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725459470; x=1726064270; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v7dFMMDbVFAIV5pvquHCkne/t7NzjwwosACmczIEiUs=; b=WYZaZROaz+Pb+FZrgaOgxFTmybCvCxanV2fl5bdcLnSBnY0cIiRWMPdP5JNI5qXj5O 3U45EW8oVJuMJcPGIpabR8U8/yXHn57uC8ZgJPSXq/0+hJUzqzjLiR48Sx+dfEuEbsQA 1PBriSdGuxFendg3otDy9ztcLmq07HHQfF9B2PLhMq6Hj5Zj7NeD2KihvPwUs7s+uZDO Do0RbQLWdkscZ/EiWkeK7cPSEIYZ4RnX+tKVrsrRllwYtP0gNqoTtCdR9yCAiWSEor87 HFKSS7mQf2kNBkmfvmq1+5T1mvJbD4se26Wy1CGOheuLyCU9FGNkg56m47gwGcD/KwIy zyZQ== X-Forwarded-Encrypted: i=1; AJvYcCVS9EZkf/OITCgpoWe1u5/99eDb1JB1j/OkY+sZs1CPdti9qzek9mpQXB57Wdr7YeT43AoHco7wLQ==@kvack.org X-Gm-Message-State: AOJu0YwQHgd+TyAVpS3iP5RJuf/lAmO5vWUnMMg9HvVjMszpNsOj5TX6 JWXMp4Ztxe1PPvg8qtg6UWXHtQEdLi9Kq5p8Z9K1uH0Epku0Lgql X-Google-Smtp-Source: AGHT+IE26wVQR2LgzuFQ4epQHPbTKn3Zvnc693bAIlnah2akuYCekc7BKlc1sYCNhwYDpPPSQe9AfA== X-Received: by 2002:a17:907:7e9b:b0:a86:6cb1:4d49 with SMTP id a640c23a62f3a-a8a1d29b736mr490505366b.13.1725459469967; Wed, 04 Sep 2024 07:17:49 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8989196c88sm811160766b.102.2024.09.04.07.17.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 07:17:49 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v3 5/8] block: implement async discard as io_uring cmd Date: Wed, 4 Sep 2024 15:18:04 +0100 Message-ID: <7fc0a61ae29190a42e958eddfefd6d44cdf372ad.1725459175.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 89B7640029 X-Stat-Signature: qw7gner1hkrckch3rmdg933ibyorrkf5 X-HE-Tag: 1725459471-645567 X-HE-Meta: U2FsdGVkX18hp+ESCTUuZGsn91IavyvM9gtqoTS2Y4WIajhDOT9EGVTMUFMFRnr8hVyhlRMf2Er1PrjWXaLZSR2X8xKlxoCxK2rHmgrk7LjFd8DOFB0RZT1jVvshh8+571CZyS/5NMGqFIhCT80EOPS5vUtgVkG4lBHgt3WDpaXegm2V/c/sC0EmxI0V6f/cL7g4fse2OZAJV8/+r8r9H9jU8bzlOHU9LXXCxSa+BngD/hteCwqApWnLBv0KUPt0uHzXbL3+BgG0gYFAFH5tnnG0Ma89IVof/UvBPADQdDnBYSEzBbm/Qlfwp/eM8HIV5Oyu9ivrhqq5zXyUkU1cDmkP4+H9q5yxiygOXgEFW9FWRQJMNTNEWyPRRy57BDS/dGhg0dDvOYC9RKzq+J0wN4wEFcBUaoYQtka2qY0XeEXu1F6JDM/MP90pdYZqgq1lm/n4im8VJSaBo3htXpQ+eZScORZGf5+wccHodA5caRJGo6VDZrdherME6aRIAGfMGE5hP2lU+nrbZbEL5r+vCS1Nb3YTv2n5MIcTfLhIZq6QdQ0HbRVCJr4cXkiYgvURCgh9bdueifaQ0dnXowBNRKaEkzubNxtQTqWbKqzPCKGKfgZWxbOTDCJbadlkb6mul7WGVkKL5XO/HR1rNOZnSWFMvHvkMGHvO85Vz6DaL88Lc9LVdkV6WZHAcIK3CWTg6tE1OgGCROx/gOwCECM1WWYGGEoUHw+xNRXZ8NyY2PrWoLNoUDl4KdhPvO7Jyz8KZMnESeLVKTBAyjdZ8t1aLzDfTEMay30bRFwPy8neSl8x2IPHSOKmaahtQtpE6fSHr05zOXuEYfixBGmoj+kdkmzypwL5EYvi4ZgPpdK+UEjr9+9nizBSe9VttHKGWOb3DrqViSOTCpOdRkcHwx0gvRx6MhLM6rVkGaY+R1juKlq5PlGdTKDIRNMTg2npu/a/gyb81I843ty/VqtCM83 4BBpCJTJ im+6e8oS1cFNZUij01mB/q5H9V9ZWJXs8DZdmWn6bNpHvg865sck+jg/LxQsJm00tSN9SKyGAkHRB72TxPVQIIHihfO31zI/81dgRrmj9fGC/2BogkBpdkWeSvvefZ9b+NpHcG/lmzeRVHgac33uhir7VQhBYCZtFt9Zz2V9n3V9vbyyik9Yv8A78yyIquMM54RtNaMj1pKeaVWATSZhjA1L3KjVMLj5aUdTpxfqE5UsxO8cP5aMm+Y+VY5qCFzRQuFYvpuFwBNZhLENqU66BEASsPmYxa2cnpyDStqEIQZ4gAwoEncDqyabXgdW3r5YDMcxXWu92etANLxkjbzDVF4DbxpsjyjjzN5UTJ/pFZt+Rt9hw7hDZYK8unzQS4RCzv2s5w5FsWD8vW360cUaqRM8yAT60fBZXfP6oTTNGWmFjA6F6HFEvGCcGRxA7Q+2J9yTaiPf+y05Gd7nT/Ggh1X5sGES9pPeOThR6K1HPXToOBOGSzwt5KQn0Y8PUOJbdgWcXN5AARDRq3BT40Gmkb49EPECLnuJPqBCvIKAW2SNC4wPJqtvN3IHgo2icmaHZKg4OXHmSeolISY0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: io_uring allows to implement custom file specific operations via fops->uring_cmd callback. Use it to wire up asynchronous discard commands. Normally, first it tries to do a non-blocking issue, and if fails we'd retry from a blocking context by returning -EAGAIN to core io_uring. Note, unlike ioctl(BLKDISCARD) with stronger guarantees against races, we only do a best effort attempt to invalidate page cache, and it can race with any writes and reads and leave page cache stale. It's the same kind of races we allow to direct writes. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov --- block/blk-lib.c | 3 +- block/blk.h | 1 + block/fops.c | 2 + block/ioctl.c | 102 ++++++++++++++++++++++++++++++++++++++++ include/linux/bio.h | 2 + include/uapi/linux/fs.h | 2 + 6 files changed, 111 insertions(+), 1 deletion(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index 83eb7761c2bf..c94c67a75f7e 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -10,7 +10,8 @@ #include "blk.h" -static sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) +/* The maximum size of a discard that can be issued from a given sector. */ +sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) { unsigned int discard_granularity = bdev_discard_granularity(bdev); sector_t granularity_aligned_sector; diff --git a/block/blk.h b/block/blk.h index 32f4e9f630a3..1a1a18d118f7 100644 --- a/block/blk.h +++ b/block/blk.h @@ -605,6 +605,7 @@ blk_mode_t file_to_blk_mode(struct file *file); int truncate_bdev_range(struct block_device *bdev, blk_mode_t mode, loff_t lstart, loff_t lend); long blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg); +int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags); long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg); extern const struct address_space_operations def_blk_aops; diff --git a/block/fops.c b/block/fops.c index 9825c1713a49..8154b10b5abf 100644 --- a/block/fops.c +++ b/block/fops.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "blk.h" static inline struct inode *bdev_file_inode(struct file *file) @@ -873,6 +874,7 @@ const struct file_operations def_blk_fops = { .splice_read = filemap_splice_read, .splice_write = iter_file_splice_write, .fallocate = blkdev_fallocate, + .uring_cmd = blkdev_uring_cmd, .fop_flags = FOP_BUFFER_RASYNC, }; diff --git a/block/ioctl.c b/block/ioctl.c index a820f692dd1c..19fba8332eee 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -11,6 +11,8 @@ #include #include #include +#include +#include #include "blk.h" static int blkpg_do_ioctl(struct block_device *bdev, @@ -742,3 +744,103 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg) return ret; } #endif + +struct blk_iou_cmd { + int res; + bool nowait; +}; + +static void blk_cmd_complete(struct io_uring_cmd *cmd, unsigned int issue_flags) +{ + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + + if (bic->res == -EAGAIN && bic->nowait) + io_uring_cmd_issue_blocking(cmd); + else + io_uring_cmd_done(cmd, bic->res, 0, issue_flags); +} + +static void bio_cmd_bio_end_io(struct bio *bio) +{ + struct io_uring_cmd *cmd = bio->bi_private; + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + + if (unlikely(bio->bi_status) && !bic->res) + bic->res = blk_status_to_errno(bio->bi_status); + + io_uring_cmd_do_in_task_lazy(cmd, blk_cmd_complete); + bio_put(bio); +} + +static int blkdev_cmd_discard(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + gfp_t gfp = nowait ? GFP_NOWAIT : GFP_KERNEL; + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + int err; + + if (!bdev_max_discard_sectors(bdev)) + return -EOPNOTSUPP; + + if (!(file_to_blk_mode(cmd->file) & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + err = blk_validate_byte_range(bdev, start, len); + if (err) + return err; + + /* + * Don't allow multi-bio non-blocking submissions as subsequent bios + * may fail but we won't get a direct indication of that. Normally, + * the caller should retry from a blocking context. + */ + if (nowait && nr_sects > bio_discard_limit(bdev, sector)) + return -EAGAIN; + + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, nowait); + if (err) + return err; + + while ((bio = blk_alloc_discard_bio(bdev, §or, &nr_sects, gfp))) { + if (nowait) + bio->bi_opf |= REQ_NOWAIT; + prev = bio_chain_and_submit(prev, bio); + } + if (!prev) + return -EAGAIN; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_bio_end_io; + submit_bio(prev); + return -EIOCBQUEUED; +} + +int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) +{ + struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host); + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + const struct io_uring_sqe *sqe = cmd->sqe; + u32 cmd_op = cmd->cmd_op; + uint64_t start, len; + + if (unlikely(sqe->ioprio || sqe->__pad1 || sqe->len || + sqe->rw_flags || sqe->file_index)) + return -EINVAL; + + bic->res = 0; + bic->nowait = issue_flags & IO_URING_F_NONBLOCK; + + start = READ_ONCE(sqe->addr); + len = READ_ONCE(sqe->addr3); + + switch (cmd_op) { + case BLOCK_URING_CMD_DISCARD: + return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); + } + return -EINVAL; +} diff --git a/include/linux/bio.h b/include/linux/bio.h index faceadb040f9..78ead424484c 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -684,4 +684,6 @@ struct bio *bio_chain_and_submit(struct bio *prev, struct bio *new); struct bio *blk_alloc_discard_bio(struct block_device *bdev, sector_t *sector, sector_t *nr_sects, gfp_t gfp_mask); +sector_t bio_discard_limit(struct block_device *bdev, sector_t sector); + #endif /* __LINUX_BIO_H */ diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..7ea41ca97158 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -208,6 +208,8 @@ struct fsxattr { * (see uapi/linux/blkzoned.h) */ +#define BLOCK_URING_CMD_DISCARD _IO(0x12,137) + #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */ #define FIGETBSZ _IO(0x00,2) /* get the block size used for bmap */ From patchwork Wed Sep 4 14:18:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13790936 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2125CA0ED3 for ; Wed, 4 Sep 2024 14:18:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 674D76B0085; Wed, 4 Sep 2024 10:17:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5FE3B6B00A8; Wed, 4 Sep 2024 10:17:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FE9B6B00CD; Wed, 4 Sep 2024 10:17:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1F34C6B0085 for ; Wed, 4 Sep 2024 10:17:55 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CA54581433 for ; Wed, 4 Sep 2024 14:17:54 +0000 (UTC) X-FDA: 82527259668.30.547F7C6 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) by imf24.hostedemail.com (Postfix) with ESMTP id E105018001E for ; Wed, 4 Sep 2024 14:17:52 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=MGQKIrx8; spf=pass (imf24.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.47 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725459345; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sVNRBhtechMDGkg+NH1UKI/XUcjXx3fSa2oGaERaZjA=; b=RaqDIJsS4hcLtAQ5xqVy2TTCTVSiz/Oo36py9Yl3bjIXwtkQtU6R+AzuHtrkkzk06lD7Pw 4oZmwXg0qLWS0mdOyvXJTS94AZ4nmKFoNEwBqJR7e9ylfj800V/brSnK16BDW5qT/UMPUN epqJ1Shcso76mfsJ2CeH4JDBewFjAsg= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=MGQKIrx8; spf=pass (imf24.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.47 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725459345; a=rsa-sha256; cv=none; b=t8Mlfi4WOmN3B41rfpqF9DIM08vr+UIRQczHEzR/lc7lyjVdcVacexvHmpSYVsdYaV4nkC 7Xf5gkXIqXnyEFdtz7UaRMCLsMciLBsBkHE39emzQzE9hL2tFLGlDBJyc9JYb9FLXlPSYI deQrXUkv5T708QZELT0ZZKdevJj5vUM= Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-a868831216cso789463166b.3 for ; Wed, 04 Sep 2024 07:17:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725459471; x=1726064271; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sVNRBhtechMDGkg+NH1UKI/XUcjXx3fSa2oGaERaZjA=; b=MGQKIrx8vrw8S2pu1pVGZQBpMF54/5fNzf8Fzai6VeK4ZCzZyRIINAXfrKVrobm3+t j5QrFvGp6J1jXY7u6iLHQVdVno5Qy3V/EloS+9/dHoLZSy+oPCTbgbc0znrnSp/7XGAO B7sg04ZnnhvlSSAsbTvPkoe52Hqju8vap2GGX1FUWMMC4sCezPc7FG70uF40M6GTBqqp 3z/+PEALG4wYizG6s6f7UTW2zroqVUDlbIFZV5rZQ/JPRxC4n+A92GyRkyxLMpZMG/St 3DO6lD6N9fXahZOm6YBHGx9S3oi5FLrxGbeTA6c1V3sUAqzfvuLQc6ZZzwIHx6noN61r rgVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725459471; x=1726064271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sVNRBhtechMDGkg+NH1UKI/XUcjXx3fSa2oGaERaZjA=; b=mMpkVqgfW/x49cI+oUdUqOjRWcVk9IzpjrqRHUHnRV9YG8SpMZRCgUG4GHHs6Pq8Ky lHPk+maP8sNdZz6I23vuLrAZaMjUt9BLFwS6TfX/06x8lti0ei2NcOCYhMdnyo7BThTX 82hx68Uq6CvSAbes5oTpYwrQP8zIyiVN/6z+YtCv12RYxaWDJKqVovq/Csq74cwZ3H6z bVRA6R4mKEStBo6KqMKLUqY22flYlwmB07dAqnXq3eUYfSs0TMbDo6j61nR9FxZv3k8t 59JtQutXhvYVvDXNOzCDqeXtsWueNfwsqfRH9Lh/m9mHhQh7uUIhWtnL0pg+J0puic9z HYyw== X-Forwarded-Encrypted: i=1; AJvYcCVMscGxoJ8Gl6Ji30bV6x7NDQGcgg/Jn1eGr7iaYK4CoQ28zuwj2L/UcJpzTNndvFFuhzZmeCIMDQ==@kvack.org X-Gm-Message-State: AOJu0Yz9eZkPuq1eDfos4JfzfgZzZslKORKbwXCmjorIR4k/l8ieOuMQ DGfQ37at2612KL1qxxt/waQ/95Ks+pYQKCnlk6w+nUExWxctnIB3 X-Google-Smtp-Source: AGHT+IHb0jlP4IioizAxaoRrXvgoAIk/f8ER3xkL75iNWvRggyazpylQKgjr9C/Pc8pltG39ZfMk2Q== X-Received: by 2002:a17:906:730a:b0:a86:a4b1:d2b8 with SMTP id a640c23a62f3a-a89b93da137mr930271466b.4.1725459471238; Wed, 04 Sep 2024 07:17:51 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8989196c88sm811160766b.102.2024.09.04.07.17.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 07:17:50 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v3 6/8] block: implement async write zeroes command Date: Wed, 4 Sep 2024 15:18:05 +0100 Message-ID: <2a99dd14c2e0c1fced433822a13ff00735a84816.1725459175.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: E105018001E X-Stat-Signature: d1nhroxhg8on8ojqbu7cmhtxj78nwyx9 X-Rspam-User: X-HE-Tag: 1725459472-898985 X-HE-Meta: U2FsdGVkX1+WyYiu0bZm/JJIUSGx8bCjN9Q5Mab7+b7549EInIWc7k60jXs9nH1j6vJYJRd5B81hLdkISJmVjORa0SauwTp2R1NAJaoEpLCnRiGY7b4jcTATHd5YkHe+qK0p45hquKKpAzxu0758nE7JwzNq6KzK0KIFwLrGG7sEPre6hJjjjvvjMUA+yrH1WfAljKeDaxh3kOpEJHxplZIoDEQQQ5cVSq0t/phMAn7mS4jmvXq8VrZJi+Fdztsn/fAylycdjD5PYbrKXKW9SktbWypKv3i+3C+e4FylOJtsf9EHZj3CpxBsRGNGKJr2SmBOQ530+bXBusW+eG8vFLzpgxyg/XfFp/vqaM7B2vyggFW+Pd83zEzQ/WaKaz/uOUKDxjo/Sudt2uSJJgIYNCT+UHslZNOXUmzhMiVm0qYNSevIuobHcWWtf2bQSFUqi47ZYTO/o0nIES0klxhuj6C77JUst1Ctun1v2LA9C8YjHJ44Att1RSwcQ4aSTGNV7S5/FT91/P7VuvVXcI5qDmYGYB6cFUXK7YkoEsOSKkUvFlvkpHg+nFBIok0udGiEUmH/GsKu/FZA3nro3TMFVhQZ28dw4Eme0R4uV/Qr5DH/I0NAzwBpTAuP8w3XG7kEkIVj7nIDPeBJd3DrXm2DzxJz6/qsomBDPkh0xOfwD2vdQi9+zDHwgzg3yMfwyeZSeL/mmN6DAp5yktYcGy/QD7bgRGF32W0Sc2Kwg7ddI4XCT9yzOJ3RKxHWFDhfkLjDXE5+9keZfIEvkE05NolTIgAoAYzJPl2DdJILrDSvCyPs2wZir1a4FOcUtyrispdDPVTTun4WqXhJZSmxAizBe7TYHAx0DYxAHejE1VMV7tgogUa5AHKD0xs8VfdTIkoYh9e8vWaCAIfPswss7YfUOrLFYHwMo8kluBtqIdZw22ZJAYVZLVF0LE0lrJH4/uuJgTcEtNgtf/S8rtN6+Aj +vFmg1tU CeOfBmcwB+AKSNy8IVKcVDAg8tAwetQDKxHAH2mirX1jo6WHG+9T9SSzyuiodPP0l87FlXWwTJG98J2rOfGI+ZPXNoaAikMGSihA2zAKbGNKCkgaUKRRrztbyukco3IZ9HnRU7O+cfTQlc7tYgfxtgoDX/FfPoyLQPSeRwr8M8lS5/TfrH/nKC8Cj0/MUieL3njMrdrcLp07NLFr+4oFqg8iZgKxq+Y9KBJvndVMEvxI/rxOOEvP5srodEllzFdvHJ1QmDi0dybEE1MeCtKSdPZdW2tkNBhyaHE7B2XBYLYhJeluiBNJsHnEZGeTWWOm0TgoNDw9YxCJZk3ReLN1lHacSpqXXBg1Vn4Y8etfklxo1f17BKi37wHE3munw90dz/O6og9XHj1mBWFrbGPOo43Kk/RR22w+2ihej3ZAOi+pqd6vgQWa/6NHPg+7z+VHpRBd88uSOchbOuJMb+sTJHMR8MHeWN9nfcexPWdYBHQibrp7JQtE0oBXyKP+R5n3czGnVktR7TgtEUQF7j8nE15fUcARR4y3BcOJQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add another io_uring cmd for block layer implementing asynchronous write zeroes. It reuses helpers we've added for async discards, and inherits the code structure as well as all considerations in regards to page cache races. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov --- block/ioctl.c | 64 +++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/fs.h | 1 + 2 files changed, 65 insertions(+) diff --git a/block/ioctl.c b/block/ioctl.c index 19fba8332eee..ef4b2a90ad79 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -772,6 +772,67 @@ static void bio_cmd_bio_end_io(struct bio *bio) bio_put(bio); } +static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + + sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; + sector_t limit = bdev_write_zeroes_sectors(bdev); + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + gfp_t gfp = nowait ? GFP_NOWAIT : GFP_KERNEL; + int err; + + if (!(file_to_blk_mode(cmd->file) & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + err = blk_validate_byte_range(bdev, start, len); + if (err) + return err; + + if (!limit) + return -EOPNOTSUPP; + /* + * Don't allow multi-bio non-blocking submissions as subsequent bios + * may fail but we won't get a direct indication of that. Normally, + * the caller should retry from a blocking context. + */ + if (nowait && nr_sects > limit) + return -EAGAIN; + + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, nowait); + if (err) + return err; + + limit = min(limit, (UINT_MAX >> SECTOR_SHIFT) & ~bs_mask); + while (nr_sects) { + sector_t bio_sects = min(nr_sects, limit); + + bio = bio_alloc(bdev, 0, REQ_OP_WRITE_ZEROES|REQ_NOUNMAP, gfp); + if (!bio) + break; + if (nowait) + bio->bi_opf |= REQ_NOWAIT; + bio->bi_iter.bi_sector = sector; + bio->bi_iter.bi_size = bio_sects << SECTOR_SHIFT; + sector += bio_sects; + nr_sects -= bio_sects; + + prev = bio_chain_and_submit(prev, bio); + } + if (!prev) + return -EAGAIN; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_bio_end_io; + submit_bio(prev); + return -EIOCBQUEUED; +} + static int blkdev_cmd_discard(struct io_uring_cmd *cmd, struct block_device *bdev, uint64_t start, uint64_t len, bool nowait) @@ -841,6 +902,9 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) switch (cmd_op) { case BLOCK_URING_CMD_DISCARD: return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); + case BLOCK_URING_CMD_WRITE_ZEROES: + return blkdev_cmd_write_zeroes(cmd, bdev, start, len, + bic->nowait); } return -EINVAL; } diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 7ea41ca97158..68b0fccebf92 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -209,6 +209,7 @@ struct fsxattr { */ #define BLOCK_URING_CMD_DISCARD _IO(0x12,137) +#define BLOCK_URING_CMD_WRITE_ZEROES _IO(0x12,138) #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */ From patchwork Wed Sep 4 14:18:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13790937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4C23CD484A for ; Wed, 4 Sep 2024 14:18:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B6136B00CF; Wed, 4 Sep 2024 10:17:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 363926B00D2; Wed, 4 Sep 2024 10:17:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B5C16B00D6; Wed, 4 Sep 2024 10:17:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id EBDA36B00CF for ; Wed, 4 Sep 2024 10:17:56 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7122AC1342 for ; Wed, 4 Sep 2024 14:17:56 +0000 (UTC) X-FDA: 82527259752.11.1F1C338 Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) by imf18.hostedemail.com (Postfix) with ESMTP id 805231C0027 for ; Wed, 4 Sep 2024 14:17:54 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Xca3ZfHJ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf18.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.44 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725459450; a=rsa-sha256; cv=none; b=cVoiVTJBesTGOmPa3N52MpkUZIyPFdKBH58y4JsxHVtHxlwjbXz72S6gpdRV1Smx6qmOMP p7+y1w9oQmlwBQQpyVJYGzLZjwdhajETlzobawcAjYy4BPbJEg75RdG2AJaLegYSu+Sn+O v+S5zU/SYpFuGqyjyom3fNOsGhKk+Do= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Xca3ZfHJ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf18.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.44 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725459450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/3weTv0JDO1olCzxF1klLqGqviZ0xpQPSzNHS+GA/qM=; b=IoW4gdhvRxa0rk8y6IDzpI3zpXxqDYTSiF7HrV+C7pPAX23hec9MLCzJJ+CailOPU0Laco Cx5+OcqbX67Jv11gY5D21Hn9rypED5WUF0IvKdp6H5abe0PmphadEeUjWd8WfzWJ3QwVki j/rAOjfTKqmvjoGlNiZ9ct/cg42OAuk= Received: by mail-lf1-f44.google.com with SMTP id 2adb3069b0e04-533de5a88f8so6869809e87.3 for ; Wed, 04 Sep 2024 07:17:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725459472; x=1726064272; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/3weTv0JDO1olCzxF1klLqGqviZ0xpQPSzNHS+GA/qM=; b=Xca3ZfHJyap+EADU7jX2ISPugr2YcZqWZTb0YLvbpKc8ltHumbczAextV6cxtRZKWz aOsaGBBJJZEDyCDRrfta0VmYop/uulOP0H/VaxI3OzRHWt93zGc4tvbV6FMOqdV0aTIv fuM7IaBEU3gKGML/roQpyVAf/2wOvxEf74TjBjA6eb2sWLUlB35QwejBMsnMTzikLvwY Fl0MGM8MgtMJ3fxDsZz5dzeGWHBcDpz9SmhGUkt6+8FH/wevMMHIewU6F5e4zFmP+349 xarqRKlW2NFHYsQtnQqPi9DYWfXLpjZbHTKIRZ4+1XHcUvu+lUl4O4vcyJYXLBOXKWIH 85Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725459472; x=1726064272; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/3weTv0JDO1olCzxF1klLqGqviZ0xpQPSzNHS+GA/qM=; b=NEao5pT0YvuhpKET8ol9ShzY94mfsg73KbuckoKG7WOKX+NVY5Wl7P21Lq7eRDePqm 8oeqkvawrZHt+3ArnMVDeOdWw+Cq1RiprXNLrJuqDfiPWTLTR6R/f3T0jheYdwwhWKN6 tQg3jUWYzxkNKCGrnP6UXNDlu9hl7Aaz4IaONrXuRE6n6X7raf4cKkzB5VRfqW+BQDk5 gsQqx6z1n9mk7av+AJxPYMmIgQTDsigTcy1FGZWOdldgRcMdHU14j3ev7aWPpSnx15tb v3EBWn69ZAG2ljyad688pB6JJXYZ77kc8rhY0zySF3i5K+ZJwnWjsyh7y+q2/bDHltLe 074Q== X-Forwarded-Encrypted: i=1; AJvYcCWAViwsGPoZRu2/r3GCV5lggNU2AGvi/SEPZ2XzXUs7pipwlzY0IPRvZhSvffocxxt2q1G35MSdXg==@kvack.org X-Gm-Message-State: AOJu0YxNqrQj1UcR8Dk3tWh9PvcETkhGUgUF8U0e5ZgzEndlXlwZYn0Q xRU3UKmO5vBsbSIvK8Is65nMx+NLJ0hfuGGKqEwBrB2G6lRZbHYL X-Google-Smtp-Source: AGHT+IHXEXHSZq9e6Nc/W/109+8Kx4Ik97uxMWG+wzLcTtKwZnA/1xjIlHHukneB6BfUKC9IqvzHsA== X-Received: by 2002:ac2:4c4f:0:b0:533:42ae:c985 with SMTP id 2adb3069b0e04-53546b2df7emr12335403e87.25.1725459472360; Wed, 04 Sep 2024 07:17:52 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8989196c88sm811160766b.102.2024.09.04.07.17.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 07:17:51 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v3 7/8] block: add nowait flag for __blkdev_issue_zero_pages Date: Wed, 4 Sep 2024 15:18:06 +0100 Message-ID: <292fa1c611adb064efe16ab741aad65c2128ada8.1725459175.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 805231C0027 X-Rspamd-Server: rspam01 X-Stat-Signature: rddkfwwdgfzmt1tgqnkfimgnyj8g1ix3 X-HE-Tag: 1725459474-877005 X-HE-Meta: U2FsdGVkX19e1bDuCJSmZI6uH5rHKSr3ITgRsXLj6VjRPDBHE/uBllbzXvx3IMLiD+R9ARBsoX/obrlMXRIPp97kf1E5ObB2DGz35l0Qul6EXNr0gf4xtGp30jJraXYUqutpppqgjLqLqpwThVSGHVicGWdPl9YZvHwMmbrYBOpvoxMPD0WtfLaQa/4OiIgJgI1ynLCyySAzYjXWZgM2k+gUHuJgW5srMfya/9nwS12c6C310coGZx760pUQ9CuRwjh1xHuxHxcntMMvwJrj3ev6S5E6ChvhAKVOE30rETe9gfmDOm3j65g9i+SC+y66Y/Ih6+eXA+XfFc2loFnNaxHVu1roCPZfVAQQmm0g8CEH3XDSfibBCSbrR1jP158PH6zqYSjoyCiKSsQGp/Bc0xfQwPRfNMxO34/CVvM/Cm0wPkql/R+z+eeDP+MmaS9x7tBiw38Xr2wJ2WXKflqTzspFtiZaZ+Xoa+A+DnbjGQzHY2sHS+rx0yp+Dxdi5WGApxAzw/rzS5Xg+/n8Pf8D2VElt7Yl6UPrjxLgG6ioOKo1ugHEWBjMd7n2AvUc0Ql4jRCwh1dpbx+4K3o0494qm+xjQRz+FSBurXOCp1N9drbno2Yd9iwnKUgypQh7Gltu0wuyf7OFU0SvqTdohBm/wyndeowpCjhPOn7K00c7yAzZivCMP9mpZlkJ8bqj324NRiiL1XhSq4jl2tC326hfN7FXgZMzhE9Ny36z6+upNQP4Wign4aCTWfs97LL4oAlRMrc61NbXkHpN95tTaDq923ZG8q3fcH5+huVaWn7RMjLIazLOUtnOLzDwmSnOkDBalWpat1v47ZHa1dXJ/BPvdapEH48O7e3G8D/O99Dw5AoY8fK2Tx7ipZ2Qy4llu/BxnBXtjfRm1YerfzRoP8sfoF59SxFlzDIK7c5be7IjbhtfOSEomB8OP+zzWBPv2TBNKgLPYVX6uSyScWH7K13 nz6y2gDy OViuXOF3woYeTOEQS+XYR9tTjB4X04CsM/9GBX39on9AKwM9GxrjlK1hLyT4LLf8CKqlYMsZcypf+he7KX8NwxHnJ4uW6GNlvfkUJg+YsH+WhtLwqTN3SYWuVDgETMZDu479DbitS2tZWifrGaee4DIjexd5rGCyqj4vgTUYKnGujESPHa9g8cM3Lfa9x66hTUWiDw78EaqVf0mVOtM5UpZWQnCv7bwt22CTC+4MSv+sh/BviSwhrpWEG4zlnrLp3AkaOZY/AjJO5BEWEXCFQesFnii9TRrHSpaDAXXxhIN0ckJW9YRtnkQxMFf0++Xre+GP1CXllrM5ow26ajN5Fg7K4YquQ/hQvc6QdwK5Nk89IJWUDmIe7rwFhl/AScUGGXwTZEDCFrELCEuITyVGdC9wGpOod+m9CIG6DdkeHOU52aixzW5NrhaqratRYAAJGKV+1tZQ7hDlv+WCaGxo2NGIcb1lRXKl7BkRgtHnxfPYMsbQlfHi0djB7FEp4RJWd1+p3wy8RUTnZGxoODWTL4Lxm7A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To reuse __blkdev_issue_zero_pages() in the following patch, we need to make it work with non-blocking requests. Add a new nowait flag we can pass inside. Return errors if something went wrong, and check bio_alloc() for failures, which wasn't supposed to happen before because of what gfp flags the callers are passing. Note that there might be a bio passed back even when the function returned an error. To limit the scope of the patch, don't add return code handling to callers, that can be deferred to a follow up. Signed-off-by: Pavel Begunkov --- block/blk-lib.c | 22 ++++++++++++++++++---- include/linux/bio.h | 4 ++++ include/linux/blkdev.h | 1 + 3 files changed, 23 insertions(+), 4 deletions(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index c94c67a75f7e..a16b7c7965e8 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -193,20 +193,32 @@ static unsigned int __blkdev_sectors_to_bio_pages(sector_t nr_sects) return min(pages, (sector_t)BIO_MAX_VECS); } -static void __blkdev_issue_zero_pages(struct block_device *bdev, +int blkdev_issue_zero_pages_bio(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned int flags) { + blk_opf_t opf = REQ_OP_WRITE; + + if (flags & BLKDEV_ZERO_PAGES_NOWAIT) { + sector_t max_bio_sectors = BIO_MAX_VECS << PAGE_SECTORS_SHIFT; + + if (nr_sects > max_bio_sectors) + return -EAGAIN; + opf |= REQ_NOWAIT; + } + while (nr_sects) { unsigned int nr_vecs = __blkdev_sectors_to_bio_pages(nr_sects); struct bio *bio; bio = bio_alloc(bdev, nr_vecs, REQ_OP_WRITE, gfp_mask); + if (!bio) + return -ENOMEM; bio->bi_iter.bi_sector = sector; if ((flags & BLKDEV_ZERO_KILLABLE) && fatal_signal_pending(current)) - break; + return -EINTR; do { unsigned int len, added; @@ -223,6 +235,8 @@ static void __blkdev_issue_zero_pages(struct block_device *bdev, *biop = bio_chain_and_submit(*biop, bio); cond_resched(); } + + return 0; } static int blkdev_issue_zero_pages(struct block_device *bdev, sector_t sector, @@ -236,7 +250,7 @@ static int blkdev_issue_zero_pages(struct block_device *bdev, sector_t sector, return -EOPNOTSUPP; blk_start_plug(&plug); - __blkdev_issue_zero_pages(bdev, sector, nr_sects, gfp, &bio, flags); + blkdev_issue_zero_pages_bio(bdev, sector, nr_sects, gfp, &bio, flags); if (bio) { if ((flags & BLKDEV_ZERO_KILLABLE) && fatal_signal_pending(current)) { @@ -286,7 +300,7 @@ int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector, } else { if (flags & BLKDEV_ZERO_NOFALLBACK) return -EOPNOTSUPP; - __blkdev_issue_zero_pages(bdev, sector, nr_sects, gfp_mask, + blkdev_issue_zero_pages_bio(bdev, sector, nr_sects, gfp_mask, biop, flags); } return 0; diff --git a/include/linux/bio.h b/include/linux/bio.h index 78ead424484c..87d85b326e1e 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -686,4 +686,8 @@ struct bio *blk_alloc_discard_bio(struct block_device *bdev, sector_t bio_discard_limit(struct block_device *bdev, sector_t sector); +int blkdev_issue_zero_pages_bio(struct block_device *bdev, + sector_t sector, sector_t nr_sects, gfp_t gfp_mask, + struct bio **biop, unsigned int flags); + #endif /* __LINUX_BIO_H */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 643c9020a35a..bf1aa951fda2 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1098,6 +1098,7 @@ int blkdev_issue_secure_erase(struct block_device *bdev, sector_t sector, #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ #define BLKDEV_ZERO_KILLABLE (1 << 2) /* interruptible by fatal signals */ +#define BLKDEV_ZERO_PAGES_NOWAIT (1 << 3) /* non-blocking submission */ extern int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, From patchwork Wed Sep 4 14:18:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13790938 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31FD7CA0ED3 for ; Wed, 4 Sep 2024 14:18:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 141856B00D2; Wed, 4 Sep 2024 10:17:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F1646B00D6; Wed, 4 Sep 2024 10:17:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC2F56B00DC; Wed, 4 Sep 2024 10:17:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C921A6B00D2 for ; Wed, 4 Sep 2024 10:17:57 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5CA0E1A15F3 for ; Wed, 4 Sep 2024 14:17:57 +0000 (UTC) X-FDA: 82527259794.12.60E993A Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by imf23.hostedemail.com (Postfix) with ESMTP id 5EAE9140018 for ; Wed, 4 Sep 2024 14:17:55 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TaN2CGMQ; spf=pass (imf23.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.208.49 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725459380; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=llWa+KY1CsE699NO6/Qz9OdhajA1Gh72FplwVQHN8Os=; b=0KHOfpOSvFEK7x+noNeVMJOoM4Kq6yXoCuMi/qta8ZiMSXU+tPzG9QGqukmgtFwyhYFHOi bLQNHEBfOs7HuJKhXlue2ryAU6XQcCx6s8dvAUwFccHLo3f7p+yKWslIflMJl6uRT93eLv GmajNTKoRavcYnUuDOs5C30Mzydggog= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725459380; a=rsa-sha256; cv=none; b=ScUV/TJi7w8NohaHJ1cBaA1+3jRxcSBDxzp178nAcG68vnNB3KbVfIqMvVw7ldD5yZt29s wzhmN0bBt+0SjC5XL/kQiOKk85HObg+dWEKqSjXmwJgH+spjmpJZEEzkTNJEXvc3Cgrw4j v2tme5LXaOAunEgI46yyeqZ+NfzzN7I= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TaN2CGMQ; spf=pass (imf23.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.208.49 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-5c24ebaa427so1253755a12.1 for ; Wed, 04 Sep 2024 07:17:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725459474; x=1726064274; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=llWa+KY1CsE699NO6/Qz9OdhajA1Gh72FplwVQHN8Os=; b=TaN2CGMQqLQcUmcyZyIEW/nuy5z1SjHhj7fQlCaGrgSOGcvycT/7rvuBCzYjFTRPQu 8H5gFO+TMJdWv6wl98GNrP+pApQpXh4fn8L6FpB5F4wfv1YpsnC8Dz55SxEjceWSiLQy 588fz/plplDXfUSIH2WBtMCYPZ//pRT3zwpcY52KSEcl66wh6YjpptcUKPojqEpM4MdO qYyGbXaZUjjEiJqTEv8lfLmNqTUxJ6rbd3iP0KhvKvIKasmWXSqJlDk+gf9XhB9FKVe/ i8RkqhlLfbuYNdE1hJ1NMt7BzEwYHpVKOvOFrY9o/Zk2rIuXpLEQ3oFyNJFlTavx4GAA s+sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725459474; x=1726064274; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=llWa+KY1CsE699NO6/Qz9OdhajA1Gh72FplwVQHN8Os=; b=Ccms4wYZXx7tJPls16pCYtfpZZyQGmt8Bc4xp6hxB3Ti7iCGok+qGES0EB3U670Pdf qqKiAx1aS7Ze4LFiWv5srH+2bWob1upJuqq14HEdWNo+pWqW2KxffVa2PMqXAmmAuUQM /38QvLypbbz15KMjTbkCUQSoZiYMMSS7vgGOMNZXgI8p2flwZ9JOFsUXn51NkMh/PyZL toD1uJZSK++Yq8uYCXlMSJ72qLeL1D68HN5ZjeCLDaryhpdhwQSCjNcBEYIRMyN0rg0/ 8vnA3GpA5TmmqWGMMkWEM1SKXrkok6+ZEb7u9rrvBWmhZcSJBLu7zlgo8Ivb80gcKkjF JtUA== X-Forwarded-Encrypted: i=1; AJvYcCVUWidYz8M3ms2Dq7GbHXOB+c+/gmeCENREhm6UWBY+DssbylV6b9zg+Aqp0FDR/Mko0zHGuRO1oQ==@kvack.org X-Gm-Message-State: AOJu0Yw0fVi6cFzIPTN6dK4xPGyO/VFmqBHX6aDMsbkJNknIN2Vd2ztJ N8l5CpvSwNdeBUoDS2pBTaHkm+nxXXTGF7rWTblk1Lxx9p0sHG4+ X-Google-Smtp-Source: AGHT+IFWD5NeSx4L5OBZcmDku0R0EHp0Mplv+v0QTrmPwvlzQ40tRMtryzZM6Z0h0n/l6uV8EPBdaQ== X-Received: by 2002:a17:907:3fa4:b0:a7a:8dcd:ffb4 with SMTP id a640c23a62f3a-a8a430913f9mr290483266b.17.1725459473656; Wed, 04 Sep 2024 07:17:53 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8989196c88sm811160766b.102.2024.09.04.07.17.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 07:17:52 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v3 8/8] block: implement async write zero pages command Date: Wed, 4 Sep 2024 15:18:07 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 5EAE9140018 X-Stat-Signature: 9g8p1nucgxazsw4q5ds7eakkkf9e4hf6 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1725459475-336516 X-HE-Meta: U2FsdGVkX18dSXyTiWhzroFbI9IDLO2WlOwYWMNH25PbTO/V5JA2Ib75fDat4KsH7SDfqzTFzyYvoJbzHcVDjZTA3BdIPhnselfOiLrmTYcyPoRZGvNPdp5ctoLa7BSw1a9nIYyIUfl83ihrwRUjAN8XHS5kdJRHsx46rfCalD1VBxHV3MxeljCiLqi/+DT4ujbyoJCt3Ky9vrCS2TjF4e/cdQbiztdMyhcsjherTtZe/Af2/SCvBoZCTwY9RcZ/hlfgYkp7g65Qs+USC9khn7Lr03AbWbPHAp+kbbn0nwyOOlZowyrK1KoxvY23TiiIDzip78fbLeDMSjQs4aGt4XSKQEP3zqpZCRZJRZkIg1FPtN4C/EmOHPWi7/nYRA3C81r2WGL/cJdX3BWb+jkYiPqdiSDF6TY+sjPli2lw0FTnUM07m51GieL57Zd53K9eSb6tb7SMkgOzKskUK35txQZtL1o8OyBiStoVS5sL+JNRBvlkAuE4ugCnJ03y0P10Xv7aMhuDjP5IPkR9MgVsTfcKr53oErKdqCrR50cGRlVqobQYuOrgqFr/pf7jmSZYAV0p8vk/lRI8EOw+n5i9bMTR2ekgxRW4nMObRFnrFJD268+pmWfgPdSEdhiG0+2z0FtRW/nLFsKsTi+0wC2Is+fc8Zgih1JJG6TQJsdZWJLY1KS1ejTSlKFzmhkMLhpBLV4sDh+kNfVRr/8LTDEqzNwaKylOhsjm9bke/JSx2/KgThf6hXN7peZ43zH6spxs94+EcKYLqqHL2FlOq/xZcCxxaQZrcIU1w7vhVzWAWSGqbQPU43nlcB5yYO3C4JY0k74PRG+I3E0YTc4jeBHWRxrYLkYpIdZEEaLoGBEXivRrnvgYM3mAWnOfUpF8r/PCa7K3CxcbRKCBBpBRbQB6YEq+h9bcCWvkU/pqRjUp92bWuK8Mhvs+0Mj0XSHe4rNlee8KiV9sXq5VGWNqM96 lbANNPxA lhOmvs9ExRgOER/191/DgbfkZZ4cjnUdahSlg/4XilAaXcDk9919S2oG1Ye0ep84mDHm4hI7/ejQxs3rhzY6Fr4kwmWtBVb2T3Y8Sg2OVqzHaO3XIJYcUaQ81YYxHNDHv0Nds4RNR/sz+w+u0WezJCXW7CKCAGY2Fr89ToUT/1FnvKfNN8xYkmkINRuC/qsL3pU7J4AiIVK4YWS2pm7mpNEC0qQrxahNE1azswkJDlV8ttao1iNQPBOMYArXXlITRQ07VNDWEhlElCTi6ntyHPwbn0IuP4h9SGLGcM7gJR5M8h1c5K8+AVkf4bXf8SVVNEacXFSYqRuOjKvc3E3CMsDeddYvBVhNSyledPIp7o0Som6nm8HEKUR9K8DNTBkyIczM7K52vZwoz3sw+kZ28zvMCDVOU71QDlO8INArDqVA4hd7ejunnm5uTu+wQm8hp+wV7y0Q4KiJKJj4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a command that writes the zero page to the drive. Apart from passing the zero page instead of actual data it uses the normal write path and doesn't do any further acceleration, nor it requires any special hardware support. The indended use is to have a fallback when BLOCK_URING_CMD_WRITE_ZEROES is not supported. Signed-off-by: Pavel Begunkov --- block/ioctl.c | 24 +++++++++++++++++++++--- include/uapi/linux/fs.h | 1 + 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/block/ioctl.c b/block/ioctl.c index ef4b2a90ad79..3cb479192023 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -774,7 +774,8 @@ static void bio_cmd_bio_end_io(struct bio *bio) static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, struct block_device *bdev, - uint64_t start, uint64_t len, bool nowait) + uint64_t start, uint64_t len, + bool nowait, bool zero_pages) { sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; @@ -793,6 +794,20 @@ static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, if (err) return err; + if (zero_pages) { + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, + struct blk_iou_cmd); + + err = blkdev_issue_zero_pages_bio(bdev, sector, nr_sects, + gfp, &prev, + BLKDEV_ZERO_PAGES_NOWAIT); + if (!prev) + return -EAGAIN; + if (err) + bic->res = err; + goto out_submit; + } + if (!limit) return -EOPNOTSUPP; /* @@ -826,7 +841,7 @@ static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, } if (!prev) return -EAGAIN; - +out_submit: prev->bi_private = cmd; prev->bi_end_io = bio_cmd_bio_end_io; submit_bio(prev); @@ -904,7 +919,10 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); case BLOCK_URING_CMD_WRITE_ZEROES: return blkdev_cmd_write_zeroes(cmd, bdev, start, len, - bic->nowait); + bic->nowait, false); + case BLOCK_URING_CMD_WRITE_ZERO_PAGE: + return blkdev_cmd_write_zeroes(cmd, bdev, start, len, + bic->nowait, true); } return -EINVAL; } diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 68b0fccebf92..f4337b87d846 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -210,6 +210,7 @@ struct fsxattr { #define BLOCK_URING_CMD_DISCARD _IO(0x12,137) #define BLOCK_URING_CMD_WRITE_ZEROES _IO(0x12,138) +#define BLOCK_URING_CMD_WRITE_ZERO_PAGE _IO(0x12,139) #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */