From patchwork Thu Aug 22 03:35:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13772582 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 114C623741; Thu, 22 Aug 2024 03:35:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297740; cv=none; b=EliF+lJJXRSXIh0J7eu3nkdRZWgJiHdnOfKN3UlfEO3wJV+HcBFluKOCmcncizYRuf6DvmTfPaijfQcj7q2QjJ9X6m2Kl8Bfqzs/w3wCwNNptj1BoYHO2HmEV3UDT2IEUk/AmLq1321hm1VCZzZF2BpaKF6XRDRwYqn0XkhlT+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297740; c=relaxed/simple; bh=GlHe35fp35rXqkAwWGeMGxR0hGwDzfbkl8e7VR9Dt9E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=enCKtCtUTP9Sl3G2s8tsDZT9z01uabBPOnodhSh2yh+G5nZ6BWAt24RTwR7p+a8c/W+sAdrP1Aae7ibDI6lQ54+Pko0ImyUimAcPf5C+dGRBLMJwI1tcwEQbGIJEKfFXh3CQxPZvBd38t6wRCFV8a0uGqepiS2Vooa3myNbqr08= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MIxi7/wJ; arc=none smtp.client-ip=209.85.128.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MIxi7/wJ" Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-428e3129851so1961795e9.3; Wed, 21 Aug 2024 20:35:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724297737; x=1724902537; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Oq1dmJmhMQ/ONS/9FcWt2dsBOZ/rQa7CJ2HEombNkGY=; b=MIxi7/wJBUOD5XF+n4Q4NHTIjuhgWpMP4Pj3RbPN+QU9lq7leVEpx396brHuNbKChL 7z5pHeQEYoMi/gqh7qRScRYG7T+tC80N3dBkZlp82fSc8ckdGCzB159ltEnsysLn9ctA PHEq5cuEpRe/ux0WPgZUXxdefzqfNQRcwzH/OUa2o6oNZ+dTJhtyZ/SfIJix34YrfeOz vODuP8nsRDylIXFfPi7q3dZVC85/c0Z8NM4jX/+b0Y4gOUHx/tjhV/+u6fbcG8qUnm/j A1gTRGmGDIWcznfrq0Gic/IUH8lb3Rk0kHBnXPm5gbp8pdGBFi8f9Qv91W2EVJv9aC1a w/zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724297737; x=1724902537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Oq1dmJmhMQ/ONS/9FcWt2dsBOZ/rQa7CJ2HEombNkGY=; b=R7WdS6AGmYPIlOn6l9l6+5xJZ4cFuq9PL5Y0Mz2IYMeFlgmx5abHGGxQL/yNGZd+SE K1USNIx7WXQGub9B+xwPcJziUt5Um0bfNNC2KP3M6ij0DRbXxQtIwStSPsdwTCgaGbfc 5ECF5fc+anFgYDMzG29vCdwTbSbcDbPaAMVnqT+Bc28pFBzYoY5rY57IS3rOEwg+HuMC +6CanRxcN6ABdRyHhTjQl39cXtdrlSyMKiiK9pKMERK+oGMSW1IoFz9at3a1bEPrNpMS dWgqQi1c14JK20pw5Y/BwbffhPyeQRC1K4ClUuXszP+8T3wYFdwXtaAnX9S/O51CJrX4 sokw== X-Forwarded-Encrypted: i=1; AJvYcCXegY05eHPAZKdieE/kqos6PSfeg55Bimd4NT4LhjMvdV7nk5TcyxWMi2CxlETluOo6k+cAzC+Bdi8Aog==@vger.kernel.org X-Gm-Message-State: AOJu0Yw8degUNnrZKp2V9BmJ/Nc9X/QnkhuuOe0zCENFbAuk+Qf9Q28b Rse3zvf7xpSUh+Yj54cd9eGx32hnVo5Wl5sT5JY6w2Sz9S5ryKsjs1v3jw== X-Google-Smtp-Source: AGHT+IF1YE9cTCYPQ6lqJcA/bVj+/okCLizbljJ5bkv4iDV+o0PRhQhkc+Im1+k1WARHaBsjsN01GQ== X-Received: by 2002:a05:600c:5486:b0:429:e637:959e with SMTP id 5b1f17b1804b1-42abd2125d2mr33760875e9.10.1724297736489; Wed, 21 Aug 2024 20:35:36 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.128.6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42abefc626fsm45491995e9.31.2024.08.21.20.35.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 20:35:35 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 1/7] io_uring/cmd: expose iowq to cmds Date: Thu, 22 Aug 2024 04:35:51 +0100 Message-ID: <55ce8b5b813a7ca40597457b9afc8fd17d4ff11b.1724297388.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When an io_uring request needs blocking context we offload it to the io_uring's thread pool called io-wq. We can get there off ->uring_cmd by returning -EAGAIN, but there is no straightforward way of doing that from an asynchronous callback. Add a helper that would transfer a command to a blocking context. Note, we do an extra hop via task_work before io_queue_iowq(), that's a limitation of io_uring infra we have that can likely be lifted later if that would ever become a problem. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 6 ++++++ io_uring/io_uring.c | 11 +++++++++++ io_uring/io_uring.h | 1 + io_uring/uring_cmd.c | 7 +++++++ 4 files changed, 25 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 447fbfd32215..86ceb3383e49 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -48,6 +48,9 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); +/* Execute the request from a blocking context */ +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd); + #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd) @@ -67,6 +70,9 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } +static inline void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ +} #endif /* diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index a53f2f25a80b..323cad8175e9 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -533,6 +533,17 @@ static void io_queue_iowq(struct io_kiocb *req) io_queue_linked_timeout(link); } +static void io_req_queue_iowq_tw(struct io_kiocb *req, struct io_tw_state *ts) +{ + io_queue_iowq(req); +} + +void io_req_queue_iowq(struct io_kiocb *req) +{ + req->io_task_work.func = io_req_queue_iowq_tw; + io_req_task_work_add(req); +} + static __cold void io_queue_deferred(struct io_ring_ctx *ctx) { while (!list_empty(&ctx->defer_list)) { diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 65078e641390..9d70b2cf7b1e 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -94,6 +94,7 @@ int io_uring_alloc_task_context(struct task_struct *task, int io_ring_add_registered_file(struct io_uring_task *tctx, struct file *file, int start, int end); +void io_req_queue_iowq(struct io_kiocb *req); int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts); int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8391c7c7c1ec..39c3c816ec78 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -277,6 +277,13 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, } EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ + struct io_kiocb *req = cmd_to_io_kiocb(ioucmd); + + io_req_queue_iowq(req); +} + static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd, unsigned int issue_flags) From patchwork Thu Aug 22 03:35:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13772583 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D02F5364A4; Thu, 22 Aug 2024 03:35:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297741; cv=none; b=H5+dO3NX5rUzm07zOMVKvjYWemHhYsy8isRDuipEQvsJcspdnPEHFc1/K5b6orF/537PI40IpTaJriBv6p5uCnRao/WtGF60/4q33/avr8NmCtFff5tiKgXIH/z9mC/NP+2AAkJ812ha/npUSVyXnp+tf6nNFrgg07VkrtCqnW8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297741; c=relaxed/simple; bh=xgZK/jw8ZlQXj8cIxR0W7ybGVnlcP1lJNS+9Q/l9a0s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Srd/86hSJC6CH93D7J02oGR8iEeRmy5FyqnQxqyvd39FPe1/muHmmuIG0UIZzQFX7iXAL8d/SSlENPLuVy6sUZH7FE/yVYBWL0CYxlpa1q8wF+KzNXcuL2MWp1wr6Uw3NBk0AvUj/lFX05BWcq07w0bKK++pM7+b0luRAEguuI8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hvJ7vKGk; arc=none smtp.client-ip=209.85.128.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hvJ7vKGk" Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-4280ee5f1e3so2096315e9.0; Wed, 21 Aug 2024 20:35:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724297738; x=1724902538; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=hvJ7vKGk4ZpP6Bs7IeSiD16vSHOOntiAyv2KLQxnC4dHCJny/wQ4JjOSGZiLA6soXB 54DhrDODRDlhpu4mWBRzRSByI87qBkA1EjQoVNBtJ19TtJPexNS9bpGASefvPLovvV0H /d0EG5pfSiI5fKTj26QqQMc+/HOcGjfXII5tMAoAYGZTzOrq3zP4YPBXIGMFv9wM8P4O OZjp3FvhcvcKsMpwZCrUXWPhyGF+9PumTq/Wm+Rk+CHV6ez8d4sqA2PZ7KFi1s6e4U2S fboKVd7pBOMrM7gJUcqFR4c3wSzo+QNxX8lJDgZFXIEmKBjjYocog/LUN7NVlYrP/D0o 1FZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724297738; x=1724902538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=jd7qnnBhqYOtTVusmKO5Jo6bppMVyjtQLSwhsPZ1uualrSlzUVpvjWBmBR6SQoaawD nMMEgso6iEL7SeSaCOEO0iZPAxJoeWzaFf4sbiUuMu14mY4Cfq5EPpxdJI/WmfUsiw3U BpwZdArJf2dv1qMVb85m17YKaWMHA0bE3yufrmClnvzMb2tCcDDczBRZHg2DA2WOxaol cOlYzFlHTa3tr638F08ss2lJfi/HzGVUjSmVQAluvpyMajZK8zOIq1Ey5C7FrIShf8+7 cd+e4iqRiNpSlfwtsnjr0ugVTPXUnYaklLVlMH3cT0TTe5iJ9MpHlGKimvdSO5mUdPRl 0qhQ== X-Forwarded-Encrypted: i=1; AJvYcCUghpOCW+Ma2yQKLmW3qUmM4sYQ25F77urGfpetu2Bg8YC/ErFT7hzXLKQf491qCYNBrsersxId/iMpAw==@vger.kernel.org X-Gm-Message-State: AOJu0Yyf76i/gpH6Pr54S/P55Gs/pT7iCff5FeZ2OpZkXtwCyaKch4eo d3EVOXxpc7W6F6ovalN6+ec/p++G4RZbXWKx8Qghn3MNrVxX8uuKl34SRQ== X-Google-Smtp-Source: AGHT+IHDULdLquHnuYJwxgO4fQ1bzZiZlm+YOsZaNNDGdxrpbgmJfo4P3H5hO/VE8ik0Fzd4TfAAUQ== X-Received: by 2002:a05:600c:45cf:b0:428:fb7f:c831 with SMTP id 5b1f17b1804b1-42abd253e34mr32708315e9.32.1724297737617; Wed, 21 Aug 2024 20:35:37 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.128.6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42abefc626fsm45491995e9.31.2024.08.21.20.35.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 20:35:37 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 2/7] io_uring/cmd: give inline space in request to cmds Date: Thu, 22 Aug 2024 04:35:52 +0100 Message-ID: <54b2273600884f077b1564d4353c77ce2d9b1051.1724297388.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Some io_uring commands can use some inline space in io_kiocb. We have 32 bytes in struct io_uring_cmd, expose it. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 86ceb3383e49..c189d36ad55e 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -23,6 +23,15 @@ static inline const void *io_uring_sqe_cmd(const struct io_uring_sqe *sqe) return sqe->cmd; } +static inline void io_uring_cmd_private_sz_check(size_t cmd_sz) +{ + BUILD_BUG_ON(cmd_sz > sizeof_field(struct io_uring_cmd, pdu)); +} +#define io_uring_cmd_to_pdu(cmd, pdu_type) ( \ + io_uring_cmd_private_sz_check(sizeof(pdu_type)), \ + ((pdu_type *)&(cmd)->pdu) \ +) + #if defined(CONFIG_IO_URING) int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd); From patchwork Thu Aug 22 03:35:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13772584 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F36C39FDD; Thu, 22 Aug 2024 03:35:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297742; cv=none; b=GYRrkBPqxtIPA0QTYQjzqqDtfhdoRZfAQu6bd4/nDAGoD/AuTGOhuFumFPZuJt41dw+ei4OiFHJjNzpeKbr+FGZRHhOhTnFiV5jYzE4kuqJ8mAEVlH0Z04Rgid+1VczJOnt2u2OH2kD792kz7ZrzcevOua67SESRT8TtQAiuXBM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297742; c=relaxed/simple; bh=w2+Gx2llTSCIEW4z0w0ChEqAXKpMZsr4Hk2SFKO+bsU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iI3CkwoI+RAme40MoIoGcxhoKjGd0vP5QezZ9UCRnKczW7S0pHTTGuTLcVD5r8t/vWlelCkGwRyw3xam76ttHkvCEJ90gQ3qOLlcYrM/9PzQNEQpM4CvBU6Mcn6KCzyu9jqAVKcWLOiamGYpyBK8mMcSDTci5PmR7bda4TfgDus= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iwVF/m/v; arc=none smtp.client-ip=209.85.221.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iwVF/m/v" Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-3719896b7c8so107862f8f.3; Wed, 21 Aug 2024 20:35:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724297739; x=1724902539; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=coYbv4PcaCiw3s7iKLGpr0JKeDXM/58rHgT2Op2hXlo=; b=iwVF/m/vaGMWqIWVxUKFgzwYj3eQiqAD4eAlZv77LMFB8QcYEbIyLHUIgGRnHSGZ7G YP7xVgISsA8iZ2rGx+WIRLmNKimlvXr2/j4etDl00yTvvVIqSuVGj4ZUM4gE618RWo+m mGfy+XKGiZ5jSaY3AnXCg3+XN+l09ILNbEAV/M+s+8AQrhcOlVV8ImtdQy3en1nwGuK/ iUA+4YrX0iIkMj+LfYoAXPr+Z8pgX0gNT/Hrktun+0DhNanI2Anc+hx84RfaHFssp0gA 2V5VN7VgB1Xh6+66RyTtFrXiomd/qW6tVs4IP/1xi12JpOGtn93H1JPWL5V0g1VEI35a BisQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724297739; x=1724902539; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=coYbv4PcaCiw3s7iKLGpr0JKeDXM/58rHgT2Op2hXlo=; b=l2bKP73EePGnG6jhT6k02X96HiCXtHS0Yk6zLZZwu6PsNU8YCI4hMO6Aojj0L+buiL idG5H3/VghON1B+bxOp8Y0bJeg8a2IaiwkjcNttIVgfeJ/5fVjZFzvtTt4DxlPxGiAqS R+V+18Iaektnpw2EDKn7eBGdn+dwe/6ZWiOZGmnkDHWa64+Zp8OjUJyLAd93CVmqgcdD dqsiXWtrMENIIS7x1f2ZjIyi4rNuptaUpflTZ31KacI6grQvCDqmiLRZF4ZjPSc59oQx o5ydbzJ7RQPoWxI02lErcqpJo9V1dXuNAj7IZPxIK2Ew/ERGaP7EKXyAcUK18XJRXZA1 D6+w== X-Forwarded-Encrypted: i=1; AJvYcCUbbScAvFAhe0FANwt4UZk/h/b/fSoRtAatn/F8RfIc+oucjDIZqFfezqBUfxAv92PcoGP13PEiM0Ba5Q==@vger.kernel.org X-Gm-Message-State: AOJu0YzLlm99QjyZUwgB/5UoCH5HLIj310JkUOA8y1X/4EsAUfJiolw9 XAxJZd/jf3MCZdg9pncw9kF5tbydhg45fPDOLsfejMf0h3mO3q1cqt93WQ== X-Google-Smtp-Source: AGHT+IHP7tOkU4byOXFqX59kqg4UjLfQGE6iQhOnMRCrdta+ni4FkcLFL3OtNWEW+YJSDoGtQ2G8Gw== X-Received: by 2002:adf:ef0b:0:b0:371:8763:763c with SMTP id ffacd0b85a97d-372fd5a45d5mr2872795f8f.33.1724297738908; Wed, 21 Aug 2024 20:35:38 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.128.6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42abefc626fsm45491995e9.31.2024.08.21.20.35.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 20:35:38 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 3/7] filemap: introduce filemap_invalidate_pages Date: Thu, 22 Aug 2024 04:35:53 +0100 Message-ID: <5bf2b0f08ec25fa649f04c0847722c6b028f660c.1724297388.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 kiocb_invalidate_pages() is useful for the write path, however not everything is backed by kiocb and we want to reuse the function for bio based discard implementation. Extract and and reuse a new helper called filemap_invalidate_pages(), which takes a argument indicating whether it should be non-blocking and might return -EAGAIN. Signed-off-by: Pavel Begunkov --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 18 +++++++++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index d9c7edb6422b..e39c3a7ce33c 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -32,6 +32,8 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pgoff_t start, pgoff_t end); int kiocb_invalidate_pages(struct kiocb *iocb, size_t count); void kiocb_invalidate_post_direct_write(struct kiocb *iocb, size_t count); +int filemap_invalidate_pages(struct address_space *mapping, + loff_t pos, loff_t end, bool nowait); int write_inode_now(struct inode *, int sync); int filemap_fdatawrite(struct address_space *); diff --git a/mm/filemap.c b/mm/filemap.c index d62150418b91..74baec119239 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2712,14 +2712,12 @@ int kiocb_write_and_wait(struct kiocb *iocb, size_t count) } EXPORT_SYMBOL_GPL(kiocb_write_and_wait); -int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) +int filemap_invalidate_pages(struct address_space *mapping, + loff_t pos, loff_t end, bool nowait) { - struct address_space *mapping = iocb->ki_filp->f_mapping; - loff_t pos = iocb->ki_pos; - loff_t end = pos + count - 1; int ret; - if (iocb->ki_flags & IOCB_NOWAIT) { + if (nowait) { /* we could block if there are any pages in the range */ if (filemap_range_has_page(mapping, pos, end)) return -EAGAIN; @@ -2738,6 +2736,16 @@ int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) return invalidate_inode_pages2_range(mapping, pos >> PAGE_SHIFT, end >> PAGE_SHIFT); } + +int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) +{ + struct address_space *mapping = iocb->ki_filp->f_mapping; + loff_t pos = iocb->ki_pos; + loff_t end = pos + count - 1; + + return filemap_invalidate_pages(mapping, pos, end, + iocb->ki_flags & IOCB_NOWAIT); +} EXPORT_SYMBOL_GPL(kiocb_invalidate_pages); /** From patchwork Thu Aug 22 03:35:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13772585 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97134364A4; Thu, 22 Aug 2024 03:35:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297744; cv=none; b=SkOCK08jF6XesaFbzn8kMiEcVSSSJWlaesYdIdLQ9SmuegsUxcpxgidZFXsJNYl5C5xz3CuzoIxln1Cl5ZCyiG9KF9bAf+yhT8hHH7+xU2c1nmCuHExEE3nNvFw+U/URKAW2sO6rm2dWPigq+mAPfcljGt5AW3K5f50yNK1yCh4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297744; c=relaxed/simple; bh=eor95KyuHkxSooEEY/l0Xm4XcRE0Im5ZycLWR04WdVo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KTp5irfDlezLeOvvHCE772ptHV2o7/7Zvb/dPwbVc+h0XT56GSy8GXa33Er0hu3qYpqh4SukqBgg489dMk/m5sI7S0V1BEn5Bgp5qp+7eWJC/HCNx6wTLHUIRllNUtKp7zaChSodL6FTWuMCmVvq+44R9jOqH6Ak+dpL6gO+rTw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XyPh/weC; arc=none smtp.client-ip=209.85.221.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XyPh/weC" Received: by mail-wr1-f44.google.com with SMTP id ffacd0b85a97d-37182eee02dso163163f8f.1; Wed, 21 Aug 2024 20:35:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724297740; x=1724902540; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G7yUgVOy5MpRKfji6spWllEFui/I+HP/hnabDQ6U+Y0=; b=XyPh/weC9qV9n9T8M+iLxbtd+ztGWBiqkNFa643i4LsCQ0IfZIDFvcqqc3z58/5sH9 YpQ+LrnLjo2P3JLwv0hNO0+VuhsC8ZaylbBEuVXQQ71kV1ZRtOOjXjFsXFyKh95YmZG5 MKGHughWbjMfoMxP1blpH7+PqptE4dWI7UH5NRrmd4Q9l+lcPgf4JYWT2/0jlmKLfvB/ lBnNB6I3wWTefzm5i7kdSSv55PPvQa1JmhiJcAPZgNM9PJ3GunP1oDwF7/yZLpou3fPN mOCgGpkDVl7af6Xx28BGTJeszrF2VfCWUEbSPOYQwu4bg3LfXXZyu9QLP7clODz7/QJ/ 904w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724297740; x=1724902540; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G7yUgVOy5MpRKfji6spWllEFui/I+HP/hnabDQ6U+Y0=; b=VK0wmcnEox/t6bmoX/JBfKWQHOoGpG6oxwBsxru7YcGjWj/nGHBFO+oz8FJK0rWNhg MMzqO06dgsjSWZ1DrLwO2nDIEKaRaF4PnHfW2ru6r3rwvgWXXv/n+Fu/dQ3rB8Y8BKVM GTn5IKSE/CEGVX2351R9eEM1DBlUOICMgb5CLYAG5mIyJiAYFlKzGkStJgr76+UL3lEf ji/mTJYx6ILXDFXCLMkkXqVgsH8XCTH9fcRTQNTCZvz/f+xYV3q9M1jc3KqucIKjqmxY MS4OUQwWJXvl7Egvh1JEj3HJZBT8YXBzVkFePl7hUk6w7dI42WlytVjXtkiLxsCB2XzE 5Otw== X-Forwarded-Encrypted: i=1; AJvYcCV+WGoVOTNLs5yMqtus5bF/PUl7HHQUf7rAwbNrWrHaOwNn9CBiuC3xgM/dEc1RUIkhEL5h+N8wPJv8yQ==@vger.kernel.org X-Gm-Message-State: AOJu0YxPnPf00X1puP7fOsZ999E/ctP5+IToeSucXxTQlv1MGN7K8Mdf CfnWNujdmHVC1hjyMb6GbUX48KT3niiD6VB80+kuNdgh48suZZVLQe6JJw== X-Google-Smtp-Source: AGHT+IFHlvWFFCxl8YjFlJaGHJvsG1W8kc3fcAPScv4ZmRwi4I8WRIgqdUzhutwlfp/55qdQx9eetw== X-Received: by 2002:a5d:4487:0:b0:365:aec0:e191 with SMTP id ffacd0b85a97d-373052b4b25mr815314f8f.21.1724297740108; Wed, 21 Aug 2024 20:35:40 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.128.6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42abefc626fsm45491995e9.31.2024.08.21.20.35.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 20:35:39 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 4/7] block: introduce blk_validate_write() Date: Thu, 22 Aug 2024 04:35:54 +0100 Message-ID: <2ef85c782997ad40e923e7640039e0c7795e19da.1724297388.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In preparation to further changes extract a helper function out of blk_ioctl_discard() that validates if it's allowed to do a write-like operation for the given range. Signed-off-by: Pavel Begunkov --- block/ioctl.c | 47 +++++++++++++++++++++++++++++------------------ 1 file changed, 29 insertions(+), 18 deletions(-) diff --git a/block/ioctl.c b/block/ioctl.c index e8e4a4190f18..8df0bc8002f5 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -92,38 +92,49 @@ static int compat_blkpg_ioctl(struct block_device *bdev, } #endif +static int blk_validate_write(struct block_device *bdev, blk_mode_t mode, + uint64_t start, uint64_t len) +{ + unsigned int bs_mask; + uint64_t end; + + if (!(mode & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + + bs_mask = bdev_logical_block_size(bdev) - 1; + if ((start | len) & bs_mask) + return -EINVAL; + if (!len) + return -EINVAL; + + if (check_add_overflow(start, len, &end) || end > bdev_nr_bytes(bdev)) + return -EINVAL; + + return 0; +} + static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, unsigned long arg) { - unsigned int bs_mask = bdev_logical_block_size(bdev) - 1; - uint64_t range[2], start, len, end; + uint64_t range[2], start, len; struct bio *prev = NULL, *bio; sector_t sector, nr_sects; struct blk_plug plug; int err; - if (!(mode & BLK_OPEN_WRITE)) - return -EBADF; - - if (!bdev_max_discard_sectors(bdev)) - return -EOPNOTSUPP; - if (bdev_read_only(bdev)) - return -EPERM; - if (copy_from_user(range, (void __user *)arg, sizeof(range))) return -EFAULT; - start = range[0]; len = range[1]; - if (!len) - return -EINVAL; - if ((start | len) & bs_mask) - return -EINVAL; + if (!bdev_max_discard_sectors(bdev)) + return -EOPNOTSUPP; - if (check_add_overflow(start, len, &end) || - end > bdev_nr_bytes(bdev)) - return -EINVAL; + err = blk_validate_write(bdev, mode, start, len); + if (err) + return err; filemap_invalidate_lock(bdev->bd_mapping); err = truncate_bdev_range(bdev, mode, start, start + len - 1); From patchwork Thu Aug 22 03:35:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13772586 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 529813B79C; Thu, 22 Aug 2024 03:35:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297745; cv=none; b=VCCKHQ9TVmdGawxmZ/PTMFobeAh2PhWrL9Q3m1KeNQg8UuO6TgvypKTH82Mzoz2zFP6mo+qwdOD1dzBHPDgsQfSPtH9DCLRH/NUgCl61ro4XBcGGza/zzBhQyT26WSbUBDE5KkpdDrDuYprtmACdcB8YBxT4uQZqpWZEmv6GpLw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297745; c=relaxed/simple; bh=gBDpG6pGeKhN8A/poyP7gUhVt5GhvHtARNXkIfHOrwY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B5+wSETxwR3yZ0hLekOe4xVShvUl5BEdtGqO5xr6hn2vnyrjh4FbsIZz159ecsut5f2H9SAx3q+IiHQqBjdldqSxzSXMxFZ4ulD7KoRJfd7Zj1gkNRRemcCQY13tT9iBitlLu+3JYCCBBxduAe/trhF7lUpnj/krzjWQzUz5osA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kSnjNSH8; arc=none smtp.client-ip=209.85.128.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kSnjNSH8" Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-428141be2ddso1724415e9.2; Wed, 21 Aug 2024 20:35:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724297741; x=1724902541; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qGj49BU9lgDQwY8mNGLmILE2ejA/5ZP33Z20PIIjLDo=; b=kSnjNSH8Wt6C062y8Jp5cIR0L2l4VSGDaGk8hvMzJVzAWZG098iw6kUfVsyvpthnl4 UUMtJD3TmkTkoDp8HQIpkvKbcHe0M0Jar3cfTFoxnuwgAZQQrYBKyyG8UnXj4bge3EPb MNlBxxiGLUWCvx1ylC4MAMELurq8gj58Wfh2JTb4F3qDgmg6RovyUW+25Y4yGHfyhHwp wYGevoCoKR9pDoMdJsvOQZiuv5XHr9QeA3NUPMfbJ2Z/7+NNTgzjxxSuTTwGf7H0c339 H2fuw9RCOstGSlwZVvZZB2J2wKd5YhOIVlknwNuQBoKW84pvXTglIAg2c3e0tqcu2WT5 lBjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724297741; x=1724902541; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qGj49BU9lgDQwY8mNGLmILE2ejA/5ZP33Z20PIIjLDo=; b=jo+pRlwkUfwoIcHpVMruy/sjQuUk2Xa9np15d37meu7tG4R5os3HsyD/FDG7UCsQaH XyvdtKjBAzZbAVO0PcFrYpPhUnjRSoZ8YQIpTwTeitaNAM4oNeDaUlglDaLeQm2Ii7Nj 89aORTkwoXq/63Rpy6zElp679uBycW/kI11xH38595fkurOngx12n63WKEyc8m9kiKx7 Kw/DWvUWpNJswbbziG1deyxSCpQdRT1gUZr6hAnQqul7xCX67btGY76qpA3jOwAw4E5w 7EOXQcQoGWHqNmXAQjfof3BSt5Pfb6v2JG1HZyZBHI3IZ3+EAIKH+tfsTq5ivBSIYbGq o34Q== X-Forwarded-Encrypted: i=1; AJvYcCW54HzlEsLl1Hk+VilgKXr7SfUsxd4DKSGIbynVLZeeDa3NcYCUWgDHViJ1kZnH1eB7ZG9nfCwdOGg8QQ==@vger.kernel.org X-Gm-Message-State: AOJu0Yw54woWCfrpOylyy6N6TJgjBhcdiDRpXCOW1leUPKx8d6qamThC /C516z5b1ZewQ8okWDv+8GMF1aWfO1mz3fL831xezj0IST+VdkYOGSu5Xw== X-Google-Smtp-Source: AGHT+IEgmvrrHLtdXgyQgbBcddGUU2deWM/qRpnDz7MB2b3ub/DdWnFYjjz5OkIhJ7k2yrrWHOHjOg== X-Received: by 2002:adf:cd8a:0:b0:368:310d:c383 with SMTP id ffacd0b85a97d-37308c08adcmr230595f8f.5.1724297741164; Wed, 21 Aug 2024 20:35:41 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.128.6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42abefc626fsm45491995e9.31.2024.08.21.20.35.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 20:35:40 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 5/7] block: implement async discard as io_uring cmd Date: Thu, 22 Aug 2024 04:35:55 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 io_uring allows to implement custom file specific operations via fops->uring_cmd callback. Use it to wire up asynchronous discard commands. Normally, first it tries to do a non-blocking issue, and if fails we'd retry from a blocking context by returning -EAGAIN to core io_uring. Note, unlike ioctl(BLKDISCARD) with stronger guarantees against races, we only do a best effort attempt to invalidate page cache, and it can race with any writes and reads and leave page cache stale. It's the same kind of races we allow to direct writes. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov --- block/blk.h | 1 + block/fops.c | 2 + block/ioctl.c | 101 ++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/fs.h | 2 + 4 files changed, 106 insertions(+) diff --git a/block/blk.h b/block/blk.h index e180863f918b..5178c5ba6852 100644 --- a/block/blk.h +++ b/block/blk.h @@ -571,6 +571,7 @@ blk_mode_t file_to_blk_mode(struct file *file); int truncate_bdev_range(struct block_device *bdev, blk_mode_t mode, loff_t lstart, loff_t lend); long blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg); +int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags); long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg); extern const struct address_space_operations def_blk_aops; diff --git a/block/fops.c b/block/fops.c index 9825c1713a49..8154b10b5abf 100644 --- a/block/fops.c +++ b/block/fops.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "blk.h" static inline struct inode *bdev_file_inode(struct file *file) @@ -873,6 +874,7 @@ const struct file_operations def_blk_fops = { .splice_read = filemap_splice_read, .splice_write = iter_file_splice_write, .fallocate = blkdev_fallocate, + .uring_cmd = blkdev_uring_cmd, .fop_flags = FOP_BUFFER_RASYNC, }; diff --git a/block/ioctl.c b/block/ioctl.c index 8df0bc8002f5..a9aaa7cb7f73 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -11,6 +11,8 @@ #include #include #include +#include +#include #include "blk.h" static int blkpg_do_ioctl(struct block_device *bdev, @@ -745,3 +747,102 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg) return ret; } #endif + +struct blk_cmd { + blk_status_t status; + bool nowait; +}; + +static void blk_cmd_complete(struct io_uring_cmd *cmd, unsigned int issue_flags) +{ + struct blk_cmd *bc = io_uring_cmd_to_pdu(cmd, struct blk_cmd); + int res = blk_status_to_errno(bc->status); + + if (res == -EAGAIN && bc->nowait) + io_uring_cmd_issue_blocking(cmd); + else + io_uring_cmd_done(cmd, res, 0, issue_flags); +} + +static void bio_cmd_end(struct bio *bio) +{ + struct io_uring_cmd *cmd = bio->bi_private; + struct blk_cmd *bc = io_uring_cmd_to_pdu(cmd, struct blk_cmd); + + if (unlikely(bio->bi_status) && !bc->status) + bc->status = bio->bi_status; + + io_uring_cmd_do_in_task_lazy(cmd, blk_cmd_complete); + bio_put(bio); +} + +static int blkdev_cmd_discard(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + int err; + + if (!bdev_max_discard_sectors(bdev)) + return -EOPNOTSUPP; + + err = blk_validate_write(bdev, file_to_blk_mode(cmd->file), start, len); + if (err) + return err; + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, nowait); + if (err) + return err; + + while ((bio = blk_alloc_discard_bio(bdev, §or, &nr_sects, + GFP_KERNEL))) { + if (nowait) { + /* + * Don't allow multi-bio non-blocking submissions as + * subsequent bios may fail but we won't get direct + * feedback about that. Normally, the caller should + * retry from a blocking context. + */ + if (unlikely(nr_sects)) { + bio_put(bio); + return -EAGAIN; + } + bio->bi_opf |= REQ_NOWAIT; + } + prev = bio_chain_and_submit(prev, bio); + } + if (!prev) + return -EFAULT; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_end; + submit_bio(prev); + return -EIOCBQUEUED; +} + +int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) +{ + struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host); + struct blk_cmd *bc = io_uring_cmd_to_pdu(cmd, struct blk_cmd); + const struct io_uring_sqe *sqe = cmd->sqe; + u32 cmd_op = cmd->cmd_op; + uint64_t start, len; + + if (unlikely(sqe->ioprio || sqe->__pad1 || sqe->len || + sqe->rw_flags || sqe->file_index)) + return -EINVAL; + + bc->status = BLK_STS_OK; + bc->nowait = issue_flags & IO_URING_F_NONBLOCK; + + start = READ_ONCE(sqe->addr); + len = READ_ONCE(sqe->addr3); + + switch (cmd_op) { + case BLOCK_URING_CMD_DISCARD: + return blkdev_cmd_discard(cmd, bdev, start, len, bc->nowait); + } + return -EINVAL; +} diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..0016e38ed33c 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -208,6 +208,8 @@ struct fsxattr { * (see uapi/linux/blkzoned.h) */ +#define BLOCK_URING_CMD_DISCARD 0 + #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */ #define FIGETBSZ _IO(0x00,2) /* get the block size used for bmap */ From patchwork Thu Aug 22 03:35:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13772587 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A52822612; Thu, 22 Aug 2024 03:35:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297746; cv=none; b=ZHLTWNgQwZYdQ8B+3soVv3JJQG7VKC+dc5cKDDoArUW0VT5F4XO2mcZHRou8Sm9HoVwa8fNLNYfPnKNOLZ9GRxo9s7UCK/vfwLbZ9+EUwaatAKMwdEA+aAhHViA8b79cyMhfkMzSXrJI+Ji4yJjDd+mc2T0qCgx/d9N2jpUU80Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297746; c=relaxed/simple; bh=IoTc07K0AVIjxOTptHn5WO5OuyWZ6NvBayNrDkbYFzY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XF/SVNwlDKt91zzuPt1ADMcMmCfo5WbbPATrzo5OeHz34MiY7PJjmqoAq17fMDtONAx1L3o4nvpVklbxzjFRrAvzJ+obobo6KIZ3qKXwGQbpF0gZPcdfXuJrGQjRJaoMd8kuxNOsnKlnkOXciMCQ6AiEEUNw0sNfWjAgZ0HMKmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NGEc46nA; arc=none smtp.client-ip=209.85.128.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NGEc46nA" Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-42809d6e719so1726655e9.3; Wed, 21 Aug 2024 20:35:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724297743; x=1724902543; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PVhT9OqiSkRhkZBmbxRRu3VwPAT0F6C3XigyZkc8g4k=; b=NGEc46nAsTOscibMBcN3CBf9bGe8MKjlFOOjLL2yL3osGzg8dUDVAOQUXMlozsEZgl n8aocM1E8P8eTGX1zeEXKyqos/Rf7aTnOVgq1k1/6r/jss+8I2fNY0qYKatLD3dGc5qr 2kzX8NtXAByNcNWPCBckw9gMytI9wI5XAFJqBdsWLG5YMbdefXTHWZhnFHvoMben5gyD gYE6BUPsLI8AzH3nm4uJHdkmptajSFHPnR7u/oARNQEv3vHtnQE9nLZFacLLybPYR17g B1YXGUITGADH9UaO66kiLcj9mxoldP+TYtGiagGoACivlK5aXCKrVI9AJvPp89zCs8QM JYSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724297743; x=1724902543; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PVhT9OqiSkRhkZBmbxRRu3VwPAT0F6C3XigyZkc8g4k=; b=P/yUCmxe+mjhDp8hJfaR3YGPeM4vFEdJJU6xMsNPo3a67XQOptk+yIFZwxpicoG0MR HCQ/iwX+sGEMyo0kSGzU884iGBfRlM0MyonktRN1WjHHOgvHkQW2JTKIaroZl5VWgVvo 02eRx0u+XqBCAdq2PhzvZvqbpxzPtn7knDFL5C+IOx0KDzro1f6UtS6++TLtM/wxEKNP lOiMA6EBgSSbLHw34TNfDddts8i2Cq/rl9TsPL4fsWo9/dkOP0h4vLXO/5sY33Xfxuwy KmlodguLns9N8Ejq2oUj5g3xu3IHeE8y6O9a4cOEmi5DqlpA/ZdUXH7CKAGZ0ORqIwtN hWTA== X-Forwarded-Encrypted: i=1; AJvYcCX7hoIMlo89k4qYxEkbL+j36cJQX/GWZy5YxqwvNQ0PlcDE0sPlFZo3SXcaXyu3+xUjMGZiHWmnF/ACgw==@vger.kernel.org X-Gm-Message-State: AOJu0Yxem83suy07Xv9nUxxBF/39ztn0MiKsCxYWzktEdjrn3DSVIo6o +BeyiaFWVqe+tMdq+txZj3AUomzFpAl3Ljcecbl0VHc8fuk+kB6dsWvqCA== X-Google-Smtp-Source: AGHT+IHqvEt/onVyk5pfq2MvnE1mDsVk/iF7QUbNipDxcqrMgadYUmb8t6sc6QQq2b3aRAiZt2CaSw== X-Received: by 2002:a05:600c:4ed2:b0:426:5440:854a with SMTP id 5b1f17b1804b1-42abd21f727mr28548505e9.1.1724297742468; Wed, 21 Aug 2024 20:35:42 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.128.6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42abefc626fsm45491995e9.31.2024.08.21.20.35.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 20:35:41 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 6/7] block: implement async wire write zeroes Date: Thu, 22 Aug 2024 04:35:56 +0100 Message-ID: <09c5ef75c04c17ee2fd551da50fc9aae3bfce50a.1724297388.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add another io_uring cmd for block layer implementing asynchronous write zeroes. It reuses helpers we've added for async discards, and inherits the code structure as well as all considerations in regards to page cache races. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov --- block/ioctl.c | 68 +++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 4 +++ include/uapi/linux/fs.h | 1 + 3 files changed, 73 insertions(+) diff --git a/block/ioctl.c b/block/ioctl.c index a9aaa7cb7f73..6f0676f21e7b 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -776,6 +776,71 @@ static void bio_cmd_end(struct bio *bio) bio_put(bio); } +static int blkdev_queue_cmd(struct io_uring_cmd *cmd, struct block_device *bdev, + uint64_t start, uint64_t len, sector_t limit, + blk_opf_t opf) +{ + sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + int err; + + if (!limit) + return -EOPNOTSUPP; + + err = blk_validate_write(bdev, file_to_blk_mode(cmd->file), start, len); + if (err) + return err; + + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, opf & REQ_NOWAIT); + if (err) + return err; + + limit = min(limit, (UINT_MAX >> SECTOR_SHIFT) & ~bs_mask); + while (nr_sects) { + sector_t bio_sects = min(nr_sects, limit); + + /* + * Don't allow multi-bio non-blocking submissions as subsequent + * bios may fail but we won't get direct feedback about that. + * Normally, the caller should retry from a blocking context. + */ + if ((opf & REQ_NOWAIT) && bio_sects != nr_sects) + return -EAGAIN; + + bio = bio_alloc(bdev, 0, opf, GFP_KERNEL); + if (!bio) + break; + bio->bi_iter.bi_sector = sector; + bio->bi_iter.bi_size = bio_sects << SECTOR_SHIFT; + sector += bio_sects; + nr_sects -= bio_sects; + + prev = bio_chain_and_submit(prev, bio); + } + if (!prev) + return -EFAULT; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_end; + submit_bio(prev); + return -EIOCBQUEUED; +} + +static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + blk_opf_t opf = REQ_OP_WRITE_ZEROES | REQ_NOUNMAP; + + if (nowait) + opf |= REQ_NOWAIT; + return blkdev_queue_cmd(cmd, bdev, start, len, + bdev_write_zeroes_sectors(bdev), opf); +} + static int blkdev_cmd_discard(struct io_uring_cmd *cmd, struct block_device *bdev, uint64_t start, uint64_t len, bool nowait) @@ -843,6 +908,9 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) switch (cmd_op) { case BLOCK_URING_CMD_DISCARD: return blkdev_cmd_discard(cmd, bdev, start, len, bc->nowait); + case BLOCK_URING_CMD_WRITE_ZEROES: + return blkdev_cmd_write_zeroes(cmd, bdev, start, len, + bc->nowait); } return -EINVAL; } diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index e85ec73a07d5..82bbe1e3e278 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1095,6 +1095,10 @@ int __blkdev_issue_discard(struct block_device *bdev, sector_t sector, int blkdev_issue_secure_erase(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp); +struct bio *blk_alloc_write_zeroes_bio(struct block_device *bdev, + sector_t *sector, sector_t *nr_sects, + gfp_t gfp_mask); + #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ #define BLKDEV_ZERO_KILLABLE (1 << 2) /* interruptible by fatal signals */ diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 0016e38ed33c..b9e20ce57a28 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -209,6 +209,7 @@ struct fsxattr { */ #define BLOCK_URING_CMD_DISCARD 0 +#define BLOCK_URING_CMD_WRITE_ZEROES 1 #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */ From patchwork Thu Aug 22 03:35:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13772588 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8762E41A92; Thu, 22 Aug 2024 03:35:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297747; cv=none; b=kgc6lhdbfsI2OInldYUZ/USnzU4M7jndeApLS01gPegpPaYha0x75Nm5RxSSa1fPQ4aXz+qPTHtVpUpHsvIgbY3NNq+zL5mkYsSDdU97je9eHza7OYOh2Nrd8Xu+m8YljnP+s+u/p9AZLPpDOIKxkOzDJp5GRrB6CWv14iqsYmw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724297747; c=relaxed/simple; bh=6fn6CsIOzyK68TmwcPPLBT3BLJJOWUnLS3iRmcSIu8E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=goI03oY+tkmZvkrAWlvc8uH3TH3MHBf/yuuc9zb+m/brIs8AbEKjSBeBVtPewvhHjjtJkK5CTPAzDuJfTDhQHLnrZxLxbXwsqmyZjAEPSu4ZAf3k+b4ka3+WAwA5o33rNT/6/ZgAjTaGykCSQl5B7DI4ArjJ3VWtdzgX0NxgdMM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kfM1TmAB; arc=none smtp.client-ip=209.85.128.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kfM1TmAB" Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-429d2d7be1eso1490095e9.1; Wed, 21 Aug 2024 20:35:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724297744; x=1724902544; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lswIDLrIj7CXYuFfzAb9ZDGauPy9RK0EG0P/gPYjyms=; b=kfM1TmABqcPDc+nGmjNYkQxkJbrJCxj7Jjww0Q7BljtKy5TBfYCJjPjfsLjX2nGxS4 oUhF1Ma9+Bvf6Dqfug7sHEvfSh7UgGqnojkJ0xGpN4aLz8qcqtlQbtxs5akiwmHgngFt 0EBSoYt0slJ0BtFVMCcbXDH1uNb7p+983i7m++NtXjc6U/yOtrLejpqD4NHmXYvgid+0 y1+36ZKBAeA4mK70thdOF6hUOS+DhhChTx4nfymAr2pIlyxJLgBd9UbXT1mgIIRJCO0f km2Rom1K3ZoEUAmL+fpsXHI6z/V8y0o/Rzno7oNXL2uaVcI4+xAK+im2HwX8obzlwGcP Tfyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724297744; x=1724902544; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lswIDLrIj7CXYuFfzAb9ZDGauPy9RK0EG0P/gPYjyms=; b=ERFTEabT8XurIkSP74WiiHGMdw2xlMWJLVy4dpd8dPIlWmDgc/90lhyxhqn929u52W yW9rW+LqwzlsLk2sjxOQHAfA245rhU9XzTI2FXtAyeZb8xSSa8o8IeLQjJWThFtHkePl VsGrxcOyX3IN3gZcTkI01KwglD8zAHQfbiky5/qMzltZllf07LoNr+CmaKF45rs++8BZ aLdKOF465AUuMttVGrXieSXPZF9ZTEEAFWkQHhRupi6hT4fPOlfMFFRoASKfhrksjH9b 8Hxv+zPubEBfgws1B9YSOalVl05DQoWbJp6JwO1hEyj1Rl5IvpPql64wX7wR2myx9i2L ZTPA== X-Forwarded-Encrypted: i=1; AJvYcCUFCC5sHbt9MeAGbr4GuUlK6GDYoynIO1YQgRB6BnT7LWmGPyaVMPt+QLUhxejB9KUHXNkMcsSmejiySg==@vger.kernel.org X-Gm-Message-State: AOJu0Yx4JEPCj4jS10sbimRAqNR/j4qFrrIVvDZ1nw5BjbMEZHQSwzUA Z3wmu4DphZ29hkQqdg67VJ7m2SAqYiUgSuku8uD9RQqUo+61bhzT48RgRg== X-Google-Smtp-Source: AGHT+IG8hgkDMcLSSneo44THzR5IGgkExEMFh9iktxvoExcbZbFlYFw8o0WlWu1YjpgQLAsbGqHOpQ== X-Received: by 2002:a05:600c:3c9d:b0:428:6ac:426e with SMTP id 5b1f17b1804b1-42ac3899e82mr7892605e9.5.1724297743449; Wed, 21 Aug 2024 20:35:43 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.128.6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42abefc626fsm45491995e9.31.2024.08.21.20.35.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 20:35:43 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 7/7] block: implement async secure erase Date: Thu, 22 Aug 2024 04:35:57 +0100 Message-ID: <5ee52b6cc60fb3d4ecc3d689a3b30eabf4359dba.1724297388.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add yet another io_uring cmd implementing async secure erases. It has same page cache races as async discards and write zeroes and reuses the common paths in general. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov --- block/ioctl.c | 15 +++++++++++++++ include/uapi/linux/fs.h | 1 + 2 files changed, 16 insertions(+) diff --git a/block/ioctl.c b/block/ioctl.c index 6f0676f21e7b..ab8bab6ee806 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -841,6 +841,18 @@ static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, bdev_write_zeroes_sectors(bdev), opf); } +static int blkdev_cmd_secure_erase(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + blk_opf_t opf = REQ_OP_SECURE_ERASE; + + if (nowait) + opf |= REQ_NOWAIT; + return blkdev_queue_cmd(cmd, bdev, start, len, + bdev_max_secure_erase_sectors(bdev), opf); +} + static int blkdev_cmd_discard(struct io_uring_cmd *cmd, struct block_device *bdev, uint64_t start, uint64_t len, bool nowait) @@ -911,6 +923,9 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) case BLOCK_URING_CMD_WRITE_ZEROES: return blkdev_cmd_write_zeroes(cmd, bdev, start, len, bc->nowait); + case BLOCK_URING_CMD_SECURE_ERASE: + return blkdev_cmd_secure_erase(cmd, bdev, start, len, + bc->nowait); } return -EINVAL; } diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index b9e20ce57a28..425957589bdf 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -210,6 +210,7 @@ struct fsxattr { #define BLOCK_URING_CMD_DISCARD 0 #define BLOCK_URING_CMD_WRITE_ZEROES 1 +#define BLOCK_URING_CMD_SECURE_ERASE 2 #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */