From patchwork Wed Sep 11 16:34:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13800877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42726EE57C2 for ; Wed, 11 Sep 2024 16:34:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B176A940064; Wed, 11 Sep 2024 12:34:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AC70094004F; Wed, 11 Sep 2024 12:34:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9679A940064; Wed, 11 Sep 2024 12:34:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 792D394004F for ; Wed, 11 Sep 2024 12:34:27 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2CF39A7D30 for ; Wed, 11 Sep 2024 16:34:27 +0000 (UTC) X-FDA: 82553005374.27.F6B6D33 Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) by imf30.hostedemail.com (Postfix) with ESMTP id 371F48001C for ; Wed, 11 Sep 2024 16:34:24 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Jw+nc4/o"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726072460; a=rsa-sha256; cv=none; b=IeSODn4g9yb6Qt98ko5N40SU7PFT1QlPErHaafa4BaUSojPbe/N5A1pYNjgGZ+ZKqGgGAe jX1w+FEkav2crmfdKMzLRuke43CpHG7C4WRlfIo7tmxvqBr1laYHC8WASYy8gCRfmOUl7f p8tAjanTetJ5bZErTT5VAWvdF8b8X+8= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Jw+nc4/o"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726072460; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=llZ1YZ9ap1afUzL+DfM2yTRgS2V2EngOmOI+C+adjbrwoMdxmiX7uXgfqh3/vaiGzJyh7u wokV5jCpYQbNXuAFrStVtgBL98PgJBqqpNGNlqGn8h4xqIDwDvoKk6Qm9C8vOwcDesvNDN 1WwlQqCa1fpOAtWERrMKeLdDzgScPDY= Received: by mail-lf1-f51.google.com with SMTP id 2adb3069b0e04-5356aa9a0afso11833312e87.2 for ; Wed, 11 Sep 2024 09:34:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726072463; x=1726677263; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=Jw+nc4/ocbjRsZQLM5FO6tibb0WIl9azDyo1suIkoSU8WASi5hvZm+2ZYbtqO3Ai5N pqrbeADqgQ1zuNH7+56fWidSvsOe1lRAbfYtuqYO3VvBMnOQXxEZ72lELXB7UE36k0Ms cCN38MVSPo/c1xS3mrG5OlyvGSJRBLg6tlEtTaHEroJxwtvknrimsRtc6GvTzbTANwF0 J9TGVKJmTLpN5gwweEV1yhXgdhWJIET2POx9Z0oMfNMFvMZX097o7X+vtNEkGU0WaQyo KStiasho1QPPzwJOmX4nNQEM7vCv54f+ZQOVo9a8aG5SRyO6UabGq3tHc/ZO+0+WYNJL 96BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726072463; x=1726677263; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KZ8pkxfeOzRC4kA801xaO2cK98vlGCE9d9luTt+J2Kg=; b=ODibERZ4SraSA6X2zqbDM30GuR7BOacvIN8KDCwHJLv9+rh9k7vTTRaCxEAUR4VM1c qAa2pDFvogw/L65mq0WzkIQCPxUWiOuwr43Zc5SU9ShBA/jKg5Mexkr63A1s0N0E+AL8 wHOwpG+WBcH7YHek7ND1Xd4VZYZQeMuoKsS1M1/4ZIN2gvs7wvmRjxFxUpxlKEaSBHQN 8PsowKb+ZgSOSF3esINhntv8iyksO7j4QoncB2zQPoEv/XI9gxuBgMhPVwsJN+M4BVyt cjKQ4W+5GOFoOw4mrdFAWDOm+ka7tikXpCglcA0+O9FgRzfL5jNUb3QngZOEsdKBxqtV UQug== X-Forwarded-Encrypted: i=1; AJvYcCWGNPaul/+YTyzHyG90XXGTazJ2sF+GIdzO0YxYfvQjg/L6HLj6mWeA75MmOsJHfrvv6/0VK/cLeg==@kvack.org X-Gm-Message-State: AOJu0Yx6INztZ4wQ94TNAQhaZpdubhkcN0y4zJyLV0vYRKRKUVIWqjdd V3YoppVlgV46AFlBH0WjN0m7IE/HwVVgrylnDFPhtjo0iQ2uljnI X-Google-Smtp-Source: AGHT+IEG2kHGTzNGAdSjpEaXDSIgfMbjRxkjuMwMgmsa9Moi42gUvNZP3GJN98tnR/Gf0UwGcIEW0Q== X-Received: by 2002:a05:6512:10d5:b0:535:681d:34b0 with SMTP id 2adb3069b0e04-5365880bc04mr17121594e87.47.1726072462342; Wed, 11 Sep 2024 09:34:22 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25c72ed3sm631820866b.135.2024.09.11.09.34.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 09:34:21 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v5 1/8] io_uring/cmd: expose iowq to cmds Date: Wed, 11 Sep 2024 17:34:37 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: toro6jzu695xzwcgqtzu4gbxjxg4e7s4 X-Rspamd-Queue-Id: 371F48001C X-Rspamd-Server: rspam02 X-HE-Tag: 1726072464-740438 X-HE-Meta: U2FsdGVkX1+vFb0fmOeUycbS2hXj+5XLzoXem1qQTcLuvtUDfjJhgQQgduBIYwKK4LOXXsRbpgoTPKeD4gWyBs72km7ikUGYVA1DG5T9UCkoXyARFktw+b7aey88EPsmnWr6LyGOcwh+GORSCC19pRBWGUwJjrWCp0/hVGpGmCeVJvthi1roQtVeO9eWLPKqjbzlUlS/ipXAAnVPZq8O21qa65JvwHJ30UUD8ijy5h0AxLWsAdX/peietClSLAhqHAT5kwx4DrYOoeXNHn5n6yAElZcVMMihUC6MBvFz/Xl7lrbErJ20SgXQP/F4SmuaaN4/m3f6Bg2SMZJ0jDbVn4a2YP+1+9tPHG1U5Hd2ci5nJ66ZeQuIllD6UupkTi9oUaX7djzUmQnwgzcX3vB2ARfHeSYPQdLJNOdEPxARr9ZKnqk7AJ/UILy5Q/x1IzoY2Kam4YkEIfpkwZ3rvwSTJtGZTOobCIFvuMousbgvcg87iBcajsw4SNT247gICvwH1IHEKI68qDYG2ca2uN6mZ7hdULb+bMh5NwF6D0lDaDy5o39A7swt2SZuWMqz+gIboZX7Z4IyI3SmHitWRiEo3v/eaRfGpF8PUP7J06e1oO05FvktxBxYrSCP157i82aAz+Sbfnj/EhfSljLsJqjF6ohfo0k1o+TXn+T4UzI/6I47VnPDeqHd0BC2nQ+dHtHaeRa4CQdy/nKuG7Sq5SETy7uByRFvN1TF82PXlwKEaen+f6WtsjTumidGhqmvA8A8AnRvLHnu4r3jIaabg0ZtAS9689gBLGyHCepSfOipTsXsAmvTwmM7J5Uv0krdQfqqLpiAsEZT3hceHWk3oIk80oJFpdDqQbCjv4sUwAt5P8w/LBrpvUEWMMTlh333nzQ2zePRthB1C6gjX8a3AQstomy8X46C0uD7TF9lMrR3Rr/8sMZd0P3F4j0/oABLfIv+D1TakQ10kKNso4aFO4R bpf5dhTT 4dBhobLyzrIFClX3aedBLXxh3leToFIWx8skjt149SHy/d+NbIPN9+g4CVgBjwY2m+sDU+0mA1sQ+LQec4R1dsVtxB6kC50jbm0mnkAotzw/kNjDTgqa5DtNLtFovWifCeBJvvjVCPn27CqPydNOC+qVf90O02KBcr4VEkNxKhlXqI+mXeQFm5i2sHuAeZIqcGlAHuyN8Y5tVOPEUKrPe4dlbil41pYL1JtqspvILN11i5KZiThDiOKmq9v25kSItjQf5bmQH8uJBH3vZYCH55NUtHQf18tF7geUbBoRyV+W1z46d1Tz5Af1ceaxU8ssfKEcXZwml8FNQSkaL+V30i/CtyDU0WeS9X+2pbJB3Fms41qB2S4+rxxlLgDjMmHSgNzYOWofvkUYv6B1+Qbnyu07fh3QYofvVFli8LrPLQYrqZ+2i9NYXs5Up5FCmNg0AZjqM0Vc+fDSZiktYt7MqASsmf/vzLsilETt/SgEhd4Ocz2gIgx06sMDqrLEW7Qj5bv/XlfPweo6F2c8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When an io_uring request needs blocking context we offload it to the io_uring's thread pool called io-wq. We can get there off ->uring_cmd by returning -EAGAIN, but there is no straightforward way of doing that from an asynchronous callback. Add a helper that would transfer a command to a blocking context. Note, we do an extra hop via task_work before io_queue_iowq(), that's a limitation of io_uring infra we have that can likely be lifted later if that would ever become a problem. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 6 ++++++ io_uring/io_uring.c | 11 +++++++++++ io_uring/io_uring.h | 1 + io_uring/uring_cmd.c | 7 +++++++ 4 files changed, 25 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 447fbfd32215..86ceb3383e49 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -48,6 +48,9 @@ void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags); +/* Execute the request from a blocking context */ +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd); + #else static inline int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd) @@ -67,6 +70,9 @@ static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { } +static inline void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ +} #endif /* diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 1aca501efaf6..86cf31902841 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -533,6 +533,17 @@ static void io_queue_iowq(struct io_kiocb *req) io_queue_linked_timeout(link); } +static void io_req_queue_iowq_tw(struct io_kiocb *req, struct io_tw_state *ts) +{ + io_queue_iowq(req); +} + +void io_req_queue_iowq(struct io_kiocb *req) +{ + req->io_task_work.func = io_req_queue_iowq_tw; + io_req_task_work_add(req); +} + static __cold void io_queue_deferred(struct io_ring_ctx *ctx) { while (!list_empty(&ctx->defer_list)) { diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 65078e641390..9d70b2cf7b1e 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -94,6 +94,7 @@ int io_uring_alloc_task_context(struct task_struct *task, int io_ring_add_registered_file(struct io_uring_task *tctx, struct file *file, int start, int end); +void io_req_queue_iowq(struct io_kiocb *req); int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts); int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8391c7c7c1ec..39c3c816ec78 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -277,6 +277,13 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, } EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); +void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd) +{ + struct io_kiocb *req = cmd_to_io_kiocb(ioucmd); + + io_req_queue_iowq(req); +} + static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd, unsigned int issue_flags) From patchwork Wed Sep 11 16:34:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13800878 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49BD3EE57C0 for ; Wed, 11 Sep 2024 16:34:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E15E940065; Wed, 11 Sep 2024 12:34:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9688994004F; Wed, 11 Sep 2024 12:34:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 76DF5940065; Wed, 11 Sep 2024 12:34:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4AE9794004F for ; Wed, 11 Sep 2024 12:34:28 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 007B91C405F for ; Wed, 11 Sep 2024 16:34:27 +0000 (UTC) X-FDA: 82553005374.08.9F11550 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) by imf16.hostedemail.com (Postfix) with ESMTP id 20E23180003 for ; Wed, 11 Sep 2024 16:34:24 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Z7sSPBXH; spf=pass (imf16.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.45 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726072412; a=rsa-sha256; cv=none; b=k7UPkHIjZzXoEOcQXaUlIwXs+axdtJjnYlwbwqVhDPzX2oC9i+OVv/VekFeNCrWjYZ7stg fG//5lGsGJys9ghfinLV5LA6YqZiOwZsyI5Du9jDs8o4rpKfIe9dwtop1xha8IB0X1UKsL KJGD2m8F9SEcoPDIi4AydoKLpqtJA8k= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Z7sSPBXH; spf=pass (imf16.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.45 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726072412; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=rzhnzOvUgQZ6hXDVL/EExhok//J2ZbJ53NMBPA3XrBwAcXavsn25QqX4NNIQmG+YHSAXhc pEBEv/TZ7KxGZENI0B2ghr+IvWp3Xx8PBWFw7Qx4rFR4EiyKL9jHHygtM5V0h4xjfhjy8U zh0x+K3nPbgeIZkJuehJFyCtd6KA8Z4= Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-a8ce5db8668so5403866b.1 for ; Wed, 11 Sep 2024 09:34:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726072463; x=1726677263; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=Z7sSPBXHV8GNXyLn3Qi6gg56CILK1Iy+LqhJ4kTL/zKwdSt3XJGdTZmSxC0HJCUB6L ZvtGyI5Pa2+Pv6vwDwaG5LlFWLhndUe67+gD39+JqI7g+/znXDw+GWqBqa3tYGyXDacY wIVy0UH7bGWuRC33OVi3HWnLudv6cuontC369argqFbIoGIpGmzMcD/QXXzRm/1wNV/u rV0CePGg1H0gTDRnycoBZmTMnx9koCkKB2UiOPBXnerzli14SOYcSih5MNXn+Em4be7j wf/BiKRLFElHKFWIcdzvnmvn5HYCWW+NeonhFTkvcjtRu23IIRUYONv/0ctOU2PIbO4l kmlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726072463; x=1726677263; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yyZRXxiAI12SAMIbkwT8bn9l7S8e/TKEaLOeKBVDL1c=; b=eNs4nu64XTnCuIGRUbyGrF6tDr8lwaB75uJDHl1yxFZLU6BdrYJpD796SrT+8zDBi3 31KpPwVE7CKFTj4035PMeDodi6ZLAku8O85cRvopvm7B8xARLNLAHRV48lju1VAcTsNV Su4KT0bAVoo9j2v79NC7eUWHIShU/ihh97ErgUqODqKJ0rGRIcBqcx9pCOJCL7bM7H70 uCuE79a2Fb1J9m88MuxAU10Md2eFMWmPIYG8sAwtDYaBe3RNL6sjBk5p/dZvACBOtI0r HSlMTwUQptM2LiihYa+1FNaRWei2rDVLKjED2ON79y1jHxHDb4CNjrQjjoFxt+2xUIiw r1ag== X-Forwarded-Encrypted: i=1; AJvYcCViX3orsQR5sRFU23Cj5GYLAOrczroGUVZkCZ87y19UEhNzD90rNwqaaxBH++1BdiPEeTE5g2cDIQ==@kvack.org X-Gm-Message-State: AOJu0YzN+hFHeCW22HMaxavNbFIr0b+Y8hioYYD+dvpC7foN3ap0+c9X eDcicOesdWuODKZ8iacRAPHohb3MmphSkEu9v4JgHehxafSCgXDzONuX2fqM X-Google-Smtp-Source: AGHT+IELcXHRTUSdLyLG5jnLYvW4/fqCNo1EAgwvWA0FaGjFDUj4/njuzwhpx/wmLaysANwmyDI4Hw== X-Received: by 2002:a17:907:2da3:b0:a7d:9f92:9107 with SMTP id a640c23a62f3a-a9029690793mr5422666b.58.1726072463487; Wed, 11 Sep 2024 09:34:23 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25c72ed3sm631820866b.135.2024.09.11.09.34.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 09:34:22 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v5 2/8] io_uring/cmd: give inline space in request to cmds Date: Wed, 11 Sep 2024 17:34:38 +0100 Message-ID: <7ca779a61ee5e166e535d70df9c7f07b15d8a0ce.1726072086.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: 6oodc43o5s89udh87now44nrw78cngf7 X-Rspamd-Queue-Id: 20E23180003 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1726072464-974554 X-HE-Meta: U2FsdGVkX1+S8mk+KzNTivCR2mrj07sf9WY54XPZZ/mzRbh3qzdZT06E3eTTpXF8jkxWqJ+hR5JAqF00hAxV/+zIypx0I8K8Yx/38BVTk10frb0IXdIXu+cyQlRmB0HyVM2tl9q8t3357erKKftV4Gsu/X6OvPGWigEGyvPE/9cK21QtG1hoK85+SRt7XgK5EUFmqDleELxjTv7nk6N8T10HaE/Ezory18L9ZtfbWteFSAPMtJmUNKxb4FXldn3UtEQw+6hHSfJSiZfXFOQqOl/vL0hb9EAk3e/oc0RyBTcQTHNREYxG5kTbLlyaDUS3pg6KU7gMqecu6Idn4lBqSwdBu7VYbZhNhXEHGm6yikc0aSt9/fYzH/pRY9INXq0ZY+/PAkaSP5fbf1RYEeaU46DvBdo8gocrKjnJSOefjE/Sq/NylUURhwNo/twu4mBnveaXQ4xKLoRcL/UFQMlr/12H8VzYAqvURgFB5LTme7xiusmjTlUdtjLKlW9rrM9O6200xf4HYjmh/eZwYJz90qLuTwh8rPZ3HKiYTPiBjvADjqcvZmAwg6mb0Gn/6Ls/RGMrqWlnq0JFXLVnQntK3eNUu5Fxc4irSFxi5i6GXeDJ8rFAmlP0fYbCvI3mgQbKXVnliLxG3zMLiQUcdNMGfsIaQ+R64S3a35Om5ncUZCYNPb9WLo4RVqBKLvqXYi3Izdu+xgBMjOKbKsYnNpDdzWnVBJkI9jMPHOrisn++NjobBhP1DbjaOrmIcYnPzvcTRaHYSxBx+ueNAsRKA2rpA0STNRlHbcS4L2wqfD1tFN/96z0OKBTGG4mzLtZxWFA9wRgmTCil3wwzIX9FIdcnr3Sg+1Y1IT5PuM+BBJswjgYidYXkCAq1CICvKeNTpfNXqgWLOQ5PSpi2aayqQ2fCdxDCMv/wxC31K2FrSBm0R0tlfEaf2toUS8T/PbYQNVW8yxuwgc7GujcViKMolRA XHtCdZGF muTi/4ctN0kQcLZ5gro9muS3hJUhQpmye2j5AjmayRP65kKkIJoCGe2bjZQHrnT9eYWlRPeKTq4J7CLjPYJQxaEkIfzv2ehRhsBRZwMece1l02zZxjs1VzKRRmFunEvDKNQ5JO5aWZE5kdixUW7ygfE6dnbBNdJa2lZjXPJ35xVrzJNCrAwHZCpfKXUuX2JE6GuHRKhpsjqsabmY+cynB/9Q7RS6o5nkbQ1oqRLOSGnaVttUqTOMpmTm8OejINSxiH6hFyy1OCvucC0yM27RAfs+VJayqIz4suYRzoCtg2GXpgXwuCYPqiUkCjoeg590NIKyHMYPXycLecpHZ33elPT7+hGWwxa7P0sl6amCnxre7ijvykZbdUdzvZpbjeiRgpYCDho4v4klc2AVxv+HqeqwIlCxaDJck35zHyGFkpCOYomdeqQ3u98sLBQ1AMLLEkeIkJURyTT7QXA7mJcUXaKLL4GIIsjaqLDv+slA4H5vmf6OCA+994X/U6Aa9VZnlueig20OHI5pRrLU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some io_uring commands can use some inline space in io_kiocb. We have 32 bytes in struct io_uring_cmd, expose it. Signed-off-by: Pavel Begunkov --- include/linux/io_uring/cmd.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 86ceb3383e49..c189d36ad55e 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -23,6 +23,15 @@ static inline const void *io_uring_sqe_cmd(const struct io_uring_sqe *sqe) return sqe->cmd; } +static inline void io_uring_cmd_private_sz_check(size_t cmd_sz) +{ + BUILD_BUG_ON(cmd_sz > sizeof_field(struct io_uring_cmd, pdu)); +} +#define io_uring_cmd_to_pdu(cmd, pdu_type) ( \ + io_uring_cmd_private_sz_check(sizeof(pdu_type)), \ + ((pdu_type *)&(cmd)->pdu) \ +) + #if defined(CONFIG_IO_URING) int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw, struct iov_iter *iter, void *ioucmd); From patchwork Wed Sep 11 16:34:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13800879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEA65EE57C0 for ; Wed, 11 Sep 2024 16:34:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A96F940068; Wed, 11 Sep 2024 12:34:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D9EA940066; Wed, 11 Sep 2024 12:34:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD8E7940067; Wed, 11 Sep 2024 12:34:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9938A94004F for ; Wed, 11 Sep 2024 12:34:29 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 112381C42CE for ; Wed, 11 Sep 2024 16:34:29 +0000 (UTC) X-FDA: 82553005458.14.205CFF9 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by imf29.hostedemail.com (Postfix) with ESMTP id 33C76120020 for ; Wed, 11 Sep 2024 16:34:26 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OWNqOaq3; spf=pass (imf29.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.52 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726072439; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2XjW/1z5M+e5L1HlSBo37JIC76JYeYPBxc+HZavMul8=; b=WWV8Op/YB3uJjGwuSR7IUofijmZ3W4Bn027y2K1G0IpwWK66wdTFx4tTarhgXEMeLB4lJM BmlSKJgMulDKsWWRfe8WjGWu0gdOY0MDQ5NhuH0gvXGfqeH/Fux+qQfpB+4VMg1dOcnWhC XiTbeosHiY1n25m3do79Lgi/PR5snA0= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OWNqOaq3; spf=pass (imf29.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.52 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726072439; a=rsa-sha256; cv=none; b=OglogBiWGFDdz+mS7yDaolwh2+geY+Uv7BpQyqSlcLuiEFVOZkeBPZNonAQ3UOu473UWkk JIWGETvFIdeXXO9cdVJRuTjTHKMCsjMsMDrM01Irf86er7IHdW1uTQIxGd5MsFbi5sGSj7 egAi6QVJv4xuJyTH350AaRgkArIlHnk= Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a8a789c4fc5so190240266b.0 for ; Wed, 11 Sep 2024 09:34:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726072466; x=1726677266; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2XjW/1z5M+e5L1HlSBo37JIC76JYeYPBxc+HZavMul8=; b=OWNqOaq3GweDp4e2lNgTvcpN2/ibO+dCe7sw2QAxDWgzkwW10xCaUGaR2wzBFigP2M kn6FGk9HP25c1206tXWW+uVPem1n97KUm7sZFNHSkTqKp5K9GDiNMQ7tuhZRcxGqdguH Kl8ntj6UvXgzfrRkC9MPdYJO4a5Bl8CtAf9J8tYftl7jfyFvk91doKyvm5GeH/M2IBiI SYUeHFAdwIL8n0po4GjHKQ0EX/mO5fdLL9uAw+55+QlfjWxV+uNMKQXPh4W4AN8AYREA u4lqFK6W7dhftxo0/TbF+1V/2Vybus27W/0n3LgGviIc2twip9FeKE2MRuRkSXRQok+I a3Zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726072466; x=1726677266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2XjW/1z5M+e5L1HlSBo37JIC76JYeYPBxc+HZavMul8=; b=dSc5TVoDPJhaH6hSqJgYG1/Yca5yIHq3vkDdeVcN9PjBvfqriuglNTbxSfURMUInAA W6TClAlqJ9oMCxQi98LXs3V5+iVkhXkaQ2R+3FJH0kIQJCJaOz0T1Vbwz1KkUPHaP7PO UjS/itiXNMa65ZLZKNEIRMzY8HwVcuigQ7FFc3LGqChs2yxXWWeQbzVlf+TzAPo2LUUQ DizqRhcALg5pctviCxuZ+e3/g0aqNns4tCNpnc14zaK4Eqnr9B690BaK4TEYVBzdkt2N 2KoB+qxgcHQrnmqe6e/B/KIVYkvkN74gijA5DVWUlcEvO+gTsSZ9MioBKxrEwOTvVYVZ 74fQ== X-Forwarded-Encrypted: i=1; AJvYcCXMmADLV1cntXa/6OvFsGaNEcAeX921G/T5owy7zzbfBbTe2+BFefWTVMUe1Or+9EWW/LtcPYomkg==@kvack.org X-Gm-Message-State: AOJu0YxPTZmKFpZOZ+nb0cITaG+7y4JR6khSoatadRwWUrlIRYH9kW8u mQs+cuAPePUl2cq/gKaj1YrTco5R+JfxTjDFMHVbLhAOfbuNtFik X-Google-Smtp-Source: AGHT+IFMn90AnKpQjlSSfbL+WudXYP6lMWX8+dJQnqeRgseHvIS6DUoUvcDEA12mRbA7+KIVujg4Mw== X-Received: by 2002:a17:907:94d4:b0:a8d:141a:87cb with SMTP id a640c23a62f3a-a8ffb2459a6mr490300366b.18.1726072464867; Wed, 11 Sep 2024 09:34:24 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25c72ed3sm631820866b.135.2024.09.11.09.34.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 09:34:23 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v5 3/8] filemap: introduce filemap_invalidate_pages Date: Wed, 11 Sep 2024 17:34:39 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: pi86gkkrie8wu5r9eaqgypbf6b1qkxn4 X-Rspamd-Queue-Id: 33C76120020 X-Rspamd-Server: rspam11 X-HE-Tag: 1726072466-24392 X-HE-Meta: U2FsdGVkX18od/fDLPNZFyNFy9PHrjLGy5EpkthoJULpXuxFxJzNL52NaNg75tOledARxPdX3gqR3icR9KYLOh5adH/Ex4V0sijj1AUVCBlAXF3VxZ2eUzQ1nhz3gV1MU6YaFhrKXbUWhaypg8CkbI8YePBMSmxHz4/RrvHQWYor4XD+YLC7LSqnAkUufE/eBSFCUsdgJI8+oS82U+RnzLMyRdqrIwWQr7XdTLRn+miGJrTZZ+BQaLyQfgySgExhTCplVBKM8ARQq+HwRtvIw7EHUEgQgHAaAgC4oG8DrzmMLZwjK+KDCOGJA9N36RypJE9Dg51HtjMaQa4tPyuxWIAQ3fJsLuVakmL9EWFR/BQ+AMshADQodSKkX4IJi1xNTAk4SGbJGPmulwA4/MD3XQIZzyUYVh2xWttlZG+T6EbFJ0A4phwGIXVbwqQsDbXV/5Ih3AGvlOIroMZeLa0mcbgMcmOxCoRZIwMVxKy05ESTOq5jOiw3+fDH0sJM0sZUJlCIKj6OtXIMc93hCiSMFOHpAFgEXfHk3uWGZN4CBfLG2ka93qT52ORfrcpHaipxHWzwyc5ikIu+//LUj4j0JcLVbGWXJJQYIAF4bg7NKKXLr505Tc107zyOUs6ZhHFUDzxEzKPhrH+ip7ClweALHeqeNT52EnDrkI/VudwmHJNbLwrGBBvZYulfTuVlV1SFXUeiAub7bNrm9lGhFC8WLC+1Z+qoFrexVywewVsomEwW1YNcmCre2J/NgOVu6lJgWHhIoaKFomY0p6V0c+9tx83O5nxDvZrHeuOP5hyvRfHqJmKNvXAw1G1ZmhYRAR8leMGk++EdRvx4PNWhbTh1nQYfH6pABJ7X8EzfhqWCyoKNjU/y8th3Tz0MGURrHEiITOPpkY33m715DqwS/0hzqgzINCOJpnynSHKVTaTNpmU3QWj+lUBd2SAqbUKr5wsRpMmR1D1PckUZ3IKyJxi xkQHmSUY ndKfRTN49YtqomBxQOuY4ObDf4DpToSJcISagoTXm/Z6mJMyEZueH1os5RtHfoIGQ6WghntjzMfd52hKDzf/f1cE6dWEuO+R3HzOMSs40MldI/ZDbIoBGjiA8S17GpuZ5w1G4u1xSDewOfoyQUYSHvJe2qBDokWs2R/Res4ugg8OwZK3AlhESUr7ReBrRv2TQZjP9lr+sHqajV+6fzQ1svehwNb+FGofp/ZmTi7wDmNbTlUOWXefdrqPn5hnB/laviBGeDejbQlMZuKyt4Ut2pcF59ql18dXcmlFi1moZMBU8pV3VL1ut3Y2nN6vxMeFCMobYWNWxVXl4MGin3yn0Y5pE2cOMs0eGIQi5OquipwSs66ZG4sxkRrTPpabrwb6eBOx1UuLIZHGtwSwKZG/45QZgi1ARZMn/6w+R5YX+TdmBrZ3YpZPg+iqZWBNQLdmvjkVWT9af5YsUN9h+94MnWSW20RnT9CZc7Li8FKHi9ZQjdRka1oIy3gUmGN35croBGfMlG700LqnYER8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kiocb_invalidate_pages() is useful for the write path, however not everything is backed by kiocb and we want to reuse the function for bio based discard implementation. Extract and and reuse a new helper called filemap_invalidate_pages(), which takes a argument indicating whether it should be non-blocking and might return -EAGAIN. Signed-off-by: Pavel Begunkov --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 17 ++++++++++++----- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index d9c7edb6422b..e39c3a7ce33c 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -32,6 +32,8 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pgoff_t start, pgoff_t end); int kiocb_invalidate_pages(struct kiocb *iocb, size_t count); void kiocb_invalidate_post_direct_write(struct kiocb *iocb, size_t count); +int filemap_invalidate_pages(struct address_space *mapping, + loff_t pos, loff_t end, bool nowait); int write_inode_now(struct inode *, int sync); int filemap_fdatawrite(struct address_space *); diff --git a/mm/filemap.c b/mm/filemap.c index d62150418b91..6843ed4847d4 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2712,14 +2712,12 @@ int kiocb_write_and_wait(struct kiocb *iocb, size_t count) } EXPORT_SYMBOL_GPL(kiocb_write_and_wait); -int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) +int filemap_invalidate_pages(struct address_space *mapping, + loff_t pos, loff_t end, bool nowait) { - struct address_space *mapping = iocb->ki_filp->f_mapping; - loff_t pos = iocb->ki_pos; - loff_t end = pos + count - 1; int ret; - if (iocb->ki_flags & IOCB_NOWAIT) { + if (nowait) { /* we could block if there are any pages in the range */ if (filemap_range_has_page(mapping, pos, end)) return -EAGAIN; @@ -2738,6 +2736,15 @@ int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) return invalidate_inode_pages2_range(mapping, pos >> PAGE_SHIFT, end >> PAGE_SHIFT); } + +int kiocb_invalidate_pages(struct kiocb *iocb, size_t count) +{ + struct address_space *mapping = iocb->ki_filp->f_mapping; + + return filemap_invalidate_pages(mapping, iocb->ki_pos, + iocb->ki_pos + count - 1, + iocb->ki_flags & IOCB_NOWAIT); +} EXPORT_SYMBOL_GPL(kiocb_invalidate_pages); /** From patchwork Wed Sep 11 16:34:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13800880 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4597EEE57C0 for ; Wed, 11 Sep 2024 16:34:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3946594004F; Wed, 11 Sep 2024 12:34:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A855940067; Wed, 11 Sep 2024 12:34:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EE77194004F; Wed, 11 Sep 2024 12:34:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AE245940066 for ; Wed, 11 Sep 2024 12:34:29 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5CD01417C9 for ; Wed, 11 Sep 2024 16:34:29 +0000 (UTC) X-FDA: 82553005458.10.85EA9B6 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf11.hostedemail.com (Postfix) with ESMTP id 6942140006 for ; Wed, 11 Sep 2024 16:34:27 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ZibtDetm; spf=pass (imf11.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726072363; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j1ilgt3VEuuXKV+jcKbpBuwPBu7e3oK2wW8bPbjcXIU=; b=8r7AkqJBtcVhJK20F6NMMbJWFfvC+dX3DCiFE8BIKLTs9nub0u3DyX0Qey1AX9XHBXHEVB IVDB+371O71xsfb2IstIfvfxWrnSZAAX56YhUocc4JJtXsuWCf+ipe6Sl+0e24HpdsgZtx D89JRDJA2M/Lm8yedZJCalsSDuZh44Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726072363; a=rsa-sha256; cv=none; b=vDTZlrEfu91GaCJrXZ8cft2AC+2t+/Rpw2nW6rc9iwc6fi6bgBonTsXPtpDaRmSk1EPgeI 7BQWyjaYj6xGNyOlGTctKFVGzPpP4CWBxkBfZCgvJ9ZT5ZdSM4QMn4B4k2T2iIYBcs4W1r bhaqJd+gGyV+m+DwduvalsLfpAxUhwc= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ZibtDetm; spf=pass (imf11.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-a90188ae58eso3536566b.1 for ; Wed, 11 Sep 2024 09:34:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726072466; x=1726677266; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=j1ilgt3VEuuXKV+jcKbpBuwPBu7e3oK2wW8bPbjcXIU=; b=ZibtDetmSeUS9bGIKlXFJ547JZcukEcFTVc7LcC1Lbm+eHihjVNAOAohzGW1vAs6QK +yHNuzlQgSag22I0TOOJSDcp/ReTu0qS02ksK5wlKfUuIw8HZZS99Tiv8zZbMv5778dS Wq0Lu2HgS1fQrhBKZEov3hdUXsMnUTj+zLxVqRVjpz5sz1sD07kHpptUYCCebgPP88Pu Ll7mGP4cWjiC7s2VTStwlWi1z+6BtXh4cVbHiF40vCFa1tfQihp9qxaUnnoMa7oQMl5O 363b8TpV/gwsjUSy+NZa1O5BwIxtjxWWWtIY3p/Ro1Uw3gPsEl9g3/5yKcnskeOudQ5g 0q4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726072466; x=1726677266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j1ilgt3VEuuXKV+jcKbpBuwPBu7e3oK2wW8bPbjcXIU=; b=cJgNq7iIl6l973isgI13juqDCFGWnOpZSMu5uc/+j1+vCwRJbjZvgOK000MSeNW4e1 4rkLRUleNcRCmXvtXLA6aChWbvwoKA1bwLkV+pmbxe9rQl2PGtswpxN1iG/ZZZdOsVxW KOg1IeEuKz0d7L0fjC/b0V/9iOcbZEATunWnLVxOrBz2gAzv6AtW5HkPcbW3c9WUE7ul 1vZwPwNOnVHvMYGE/8EXaMjYEqZxkZ09hBnYS+DFCVFp99sTHlhiXLdk4QAswelQVO+n 0K9GWwdCK/W6BPNqK1NFY6aI2Z7TjsE2ODbyRgSb5wuVcS1IYTxkl2ewYs2VTJYOYwJF yx7A== X-Forwarded-Encrypted: i=1; AJvYcCUpYEfch3Ebtbohp9yXn3mJFjXS6E7s1MRbmy28/xd5TWhM/0uOULW1HufkBFxh6aTDmpYOxzuGEA==@kvack.org X-Gm-Message-State: AOJu0Yx9FHl1tFrv7u9cRyr6dqxBAJDid6JzRhpFz5JLlUsMaRiep+e9 3e7p87uHPC+JE7LetB++m0h5KpVRjE2FaR+R4nFK/9laKqXYGrsG X-Google-Smtp-Source: AGHT+IFotWTb4zxEeL7XJDWd5BKSyUFpJRfVXT5ql2T0pZNPygKiYk4YlGgrsyP58zfDitbg48lEeQ== X-Received: by 2002:a17:907:d3cf:b0:a8d:2d35:3dc6 with SMTP id a640c23a62f3a-a902945f284mr8613966b.26.1726072465802; Wed, 11 Sep 2024 09:34:25 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25c72ed3sm631820866b.135.2024.09.11.09.34.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 09:34:25 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v5 4/8] block: introduce blk_validate_byte_range() Date: Wed, 11 Sep 2024 17:34:40 +0100 Message-ID: <19a7779323c71e742a2f511e4cf49efcfd68cfd4.1726072086.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: txc5sc7s859k9b16b9tkyfuoh6un4pbg X-Rspamd-Queue-Id: 6942140006 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1726072467-187844 X-HE-Meta: U2FsdGVkX18+xAnMVtVDWDOKowRzzKSIJHbtABeTN0dP3ri7q8xyiey1qhAxgUtpvPdBdEnD5IlxJBW282jFZ9rtJCvFpAzihxy1sV4QZ4QP2AgHeXB2GtGD3xOudz2h0aIO1hn8pWMYuaNi64VzSc83tGIim/Ru2wE4DW/+TQC0oH5ozZSUh+olbvVqLTOo/GCpfemLJlpWfLupBWTL449enLF8yeyH24Yb0A/WgDOSPv+XPSAPuBBATcJ9XiteJDW5+0/Gx05O6B0hRIrGquzISN6D8oS9STPEkcXIeVPUNHxsNHY8uixoF3BpmJkRf05l3KyqASmGB+n0oo5R+JnuDTToQNKeP1JnC3L56NQnbnlEFNTyZeHKfFZF7c2wPb+pnsTQUcK5z9nPcmMoZ0oz79fAgARX2EfiEFHzAdO8li/SQP3wesBQGrv+JbaOWYDlFkDHh1rcrBEzqMJD33ldPdlbNFmVrxLo0bjVxEX38LSH1XQOzs4fHQFCYfShT/6Juj5Ba35UIBpeh9WeamMwxlkUWlQMRyGaA79lddFzI0lFa86tfGDJvlW03UtbNLyMKgtqVsecnOjun1TkDqcBlDfZHX3ANBWVd8ZGdV7FRYxhX/WmUUm/eWGBiJ5HAqW+Q1V1691dnugNb/gvu3yas92vsHXY9RfAeUfbQyN+RWl4/qD3bdHC5UbtVDeQxX6WaVDuW7VMFahShnj9ZJvOT23JKEcCmj2s2JL1kX9bAKarVULMX6+7zxtCY+90C7a8AhOOOX3hjopLAlt0x18zj8CXN4JkbewyXAfG1ryJpDLW8+3OD4MIf86uM8iq3o+pXt9kJhjKgHWmX/cwE4Da5ATvQKXm0CgLwNfj/6OJOAqtGyT2NDWVMlMnI1VYSaVwQMNviQOQ+nMT/9KIQgFeLrTRyR+RkP7slZ2cJOLs48u3wkWg00FFme6LQAJC7pb7azlg+Om68XWT1NC H5vcURUm QrIIBRcJDcFNknFBafDRLeXx+bqVxnXge8KCy8RyxOdatAiJHVYDbFlSAGb4q6KqUGkQISE1zAjDOkyxZum3JDgyFHm3C36TzkZVm9vQXCua6gk1bx/YwIiuP+hBIDkJ5lBjSvuJUUWJIxPtCG7crlAnkysZiVbNjJHcfQcT2Kyt1oUnNLXUnw6TeKT5qjvdusnhKmjxC+9jJwxtAwahXD0ntgXjVYLM4JZuQ4/kYd+D7zxMO0893SKobtTDlk8KiYBLNTKae5NNXpsmhgHU25X3NKwwkwatVQFjBBvMDYLuOvJTu6po09o/fZkY1iahkrJySIWVxI0U0QiIOgj/VC9urBftRVwawQAKCLMxu5RKtTZuI5Prvu6rUFOgBQuej/s+IHPxLnTeRTGAJDsQqTvEBpw0lXu1X/FRXAv5DXwXmt+LXf5OExQ0Rn50+hK74pW40ZduYTss6HkF01Wdc1Gj7K17cgOzpiBoezoshmzhcR9jHCeaKiPhAcnQciDFZXad5Y3D9BzDUOYc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation to further changes extract a helper function out of blk_ioctl_discard() that validates if we can do IO against the given range of disk byte addresses. Signed-off-by: Pavel Begunkov Reviewed-by: Christoph Hellwig --- block/ioctl.c | 49 +++++++++++++++++++++++++++++++------------------ 1 file changed, 31 insertions(+), 18 deletions(-) diff --git a/block/ioctl.c b/block/ioctl.c index e8e4a4190f18..6d663d6ae036 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -92,38 +92,51 @@ static int compat_blkpg_ioctl(struct block_device *bdev, } #endif +/* + * Check that [start, start + len) is a valid range from the block device's + * perspective, including verifying that it can be correctly translated into + * logical block addresses. + */ +static int blk_validate_byte_range(struct block_device *bdev, + uint64_t start, uint64_t len) +{ + unsigned int bs_mask = bdev_logical_block_size(bdev) - 1; + uint64_t end; + + if ((start | len) & bs_mask) + return -EINVAL; + if (!len) + return -EINVAL; + if (check_add_overflow(start, len, &end) || end > bdev_nr_bytes(bdev)) + return -EINVAL; + + return 0; +} + static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, unsigned long arg) { - unsigned int bs_mask = bdev_logical_block_size(bdev) - 1; - uint64_t range[2], start, len, end; + uint64_t range[2], start, len; struct bio *prev = NULL, *bio; sector_t sector, nr_sects; struct blk_plug plug; int err; - if (!(mode & BLK_OPEN_WRITE)) - return -EBADF; - - if (!bdev_max_discard_sectors(bdev)) - return -EOPNOTSUPP; - if (bdev_read_only(bdev)) - return -EPERM; - if (copy_from_user(range, (void __user *)arg, sizeof(range))) return -EFAULT; - start = range[0]; len = range[1]; - if (!len) - return -EINVAL; - if ((start | len) & bs_mask) - return -EINVAL; + if (!bdev_max_discard_sectors(bdev)) + return -EOPNOTSUPP; - if (check_add_overflow(start, len, &end) || - end > bdev_nr_bytes(bdev)) - return -EINVAL; + if (!(mode & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + err = blk_validate_byte_range(bdev, start, len); + if (err) + return err; filemap_invalidate_lock(bdev->bd_mapping); err = truncate_bdev_range(bdev, mode, start, start + len - 1); From patchwork Wed Sep 11 16:34:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13800881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C1FDEE57C2 for ; Wed, 11 Sep 2024 16:34:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 65C8F940067; Wed, 11 Sep 2024 12:34:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 60CBA940066; Wed, 11 Sep 2024 12:34:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46232940067; Wed, 11 Sep 2024 12:34:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1DEBC940066 for ; Wed, 11 Sep 2024 12:34:31 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id CD121161979 for ; Wed, 11 Sep 2024 16:34:30 +0000 (UTC) X-FDA: 82553005500.02.E76AAE6 Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) by imf28.hostedemail.com (Postfix) with ESMTP id DE453C001D for ; Wed, 11 Sep 2024 16:34:28 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bHP7apIy; spf=pass (imf28.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.53 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726072416; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kF1so/Smn2x5rpzm017teUbERvLKVA5uX9EtHevfvOw=; b=G94NxZ3QUuCyyj+lGquHU+z9xHcmvDcrUgfl8TGRWTILlWQxkc0YxAr9RgEA4JH0LiK9u/ epri8RU9moKsGQcHgohecU5ntQZrybFkvVxeEusxzzWfRqXIB4SSuocCmOz78xNNrQotzZ Vy6I4BS090vm2UeqRwQASVJ49Pg+yFc= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bHP7apIy; spf=pass (imf28.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.167.53 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726072416; a=rsa-sha256; cv=none; b=yHdNMKl+FBHb4pSiaFqOl9roYeaM7cJXuJSYycaz+AKll6fB5nhDAwN1Xrv/Rwt2osfe08 +Ei/4xsbEXjHl7tZp96ejH1K84t5sG0eg2TwZoUgdsxPv4VZCum2U7gh+s6J7qZTo9vruq +1WAkIcv1DNxnzxJAwYVs3RXVP1wGY8= Received: by mail-lf1-f53.google.com with SMTP id 2adb3069b0e04-536562739baso669e87.1 for ; Wed, 11 Sep 2024 09:34:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726072467; x=1726677267; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kF1so/Smn2x5rpzm017teUbERvLKVA5uX9EtHevfvOw=; b=bHP7apIywI6I3bZQ/mM+IRJxkD8lNhf4M/cKvHKs4sRCSTAIEQNaKy6Px9Z484gV9Z /wybtozMRNRoNAcjeRZIF804ATHSV2PqbnRwYU8Zxm1GyUy5T9D3IWz5LxQ4jiIRYYIj xo8+yr6OIOkkFj1yE418GRxPVjMbRjko1W3WOntXxjpB69RjCcDRAT8ew7L5Nn64j7pW 2fg9Auvx/UKaXIAyWKOqhgrpG3ckS0gvJ5eh9skU1zvbSOwbypC/nN4F4kNpaNr6Ismd SAT7nV2lRhCCEihiC1ihO18XI/fNdR8Nc4TTto+Rir0PfE34zogqP5H00B/Vdlmyk10j XmZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726072467; x=1726677267; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kF1so/Smn2x5rpzm017teUbERvLKVA5uX9EtHevfvOw=; b=T4yxn2IkNMmSGLECl10L6U9KG9iFh0NIXiO8a2MAviW07afyFGxVfvx8toke8INwKQ E6xTsp9VdROTtOVfTCcOmeUNb/QOrYcjHdd4I55LmdcMiGKE4dsFc3OJ1src7fOSi6Y8 DxDiC4v1vRer2LAQ2iuuEOlQKppwCUxJNUqquVv3v3mvyXYq1cnyWg8+p9czUIJiaOaz IDkCi6UbDehq2YzjOmHM9OvwjX6nKMXFyrP7I6QHdIR4QM8ppRpxKw1ILeI54OHKMEzM c3s8u6NzXCDcJgbcHqHAT2z/eVwO48uv+Gj9+w3l2mHfaXLH0LaNIzVnRkAbzxiOCx0u mXcw== X-Forwarded-Encrypted: i=1; AJvYcCXbS4Ug9Qcy7rswAzAM9DT5cd37Sa9SOphrVw7ym0604hQmiKL3juWis/vuS/7IY7aelgxMg7184w==@kvack.org X-Gm-Message-State: AOJu0Yzv7M8v+F4f4IVzhnW+lRHegz/JfK9yVqFZordDv/njT9BSIsVZ bRGXzNWqQsYfgUEeZJZkizcobG6de1I/xovEZhqp4y3lAFkgfE8q X-Google-Smtp-Source: AGHT+IHLIiqymTGBUF58wtBP7zqzsomLZiPzbl4k1nvQcQaXuT38DwgIAaWI4pNf9MD33TwjU3QnoA== X-Received: by 2002:a05:6512:68b:b0:536:53fc:e8f5 with SMTP id 2adb3069b0e04-5365880a221mr13417594e87.55.1726072467080; Wed, 11 Sep 2024 09:34:27 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25c72ed3sm631820866b.135.2024.09.11.09.34.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 09:34:26 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Conrad Meyer Subject: [PATCH v5 5/8] block: implement async io_uring discard cmd Date: Wed, 11 Sep 2024 17:34:41 +0100 Message-ID: <2b5210443e4fa0257934f73dfafcc18a77cd0e09.1726072086.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: DE453C001D X-Stat-Signature: 5d81raa13cjkicp8j1irejk4ax838p1m X-HE-Tag: 1726072468-247761 X-HE-Meta: U2FsdGVkX1/OTADu7M9BHidaJTcP0qCymA6WOoYDpvTCS1Bxi6C/riacRg/YZD+2b/FC/giXfIH7qss0Y7Ooucmixhi1foB6ctrXWb3CBz2WLWGQr/6C0rQc922ZQEdGeLW5DaNvseIZOsVvVwgf9pAeWVWxSM60fMIYy5Lb4Ge9miaL/ddUjJspgkUvd/S9o71ADOwiZPY6bDnuxH3DQM0XAJf2jaecSHoXOfu7Zzw5hGi0kRKo8H5AZyRkWKckpyV7oTQ+BhmV28q6c2tqGvRLeza/mKxkDtCW/Um6zmhxgRYO+be74QgPseoSfOku9mv1hwqwMjcWEQcb2NVv+RLhjKWyABFpBmfSbNt7AGggjDIWox9e8HKchsMKXHc2Dk3okSqERAh5PtOJTP0zEdoz0MUIc85kJPLr99D3F92VV+N9TwIDdQGXGCslXD9uwZJk3VUGxgN+qjQCOhuxIlVODJzB5tUmr94PF3hlIoITJWg5Qhokb7jjit4CWb2Y/fJwnhDMU53PJpMGFFP5vIhWHiwGByq9X5iyjQd3Fth6UB82bHKTn+sQCMY6breJcNCctB/bIsBE82aHfA5pqX5U7HD1/C494VRxrtKNHjyE2+3waQPqY3evNaZnWtjjgSo+fzmo+CLWdnb8C7kk8wySFOCLX/PWvC2V+EMmIjw88K0+sOeD+WsP29N+dAB3W+Fuw/I6pX10Dq2H+chyRrXyxrKDuxc2fTuX4NsKvEtBel1aKpAQiHRpchZoDG2cWyx2iMIpdtSjONhOMWUoibChodseGgsid3YNgZTrDMx1lkpD8pjDQqQaM8xFufnYuQdk1jEhqskUSUyYEI98o9k1ZFC7OrUL6ti+3SWX5EBGccu8AkUGZw1/e9S57T+QohdaJrEPPVHpUTjymTGqe7nttGr/AhiH9pI2iyymY+w5Xf/y7zTt0+PKxyFKGMqcWZs+XHHLCEd2vEbUZrd o32aHE9e db39smcEGMVsFkjsUIruheS1fXQ98ewBpeDcT+z1mIeiBfTvs2nqYjqMo/9WmVTjrAzVzAxGoxoOzRKz6NI9XgpJkOG91Vk7ClHn12Pe/FidN8mPBclzCfln8n2udTkPgHn2IgL3yl2A0Gb4P9aG8iBbSysbZ0PkhdtmwnNGnJxr3Oi75foNNuKXP4YH8DICNSDoO3jx9a9SZ8wHLAKBt3eq4/cJ1dTsKKKAEnR6EkmQJDu+4bzhszb3c6mw3A7wDHrdWPFZpjrHIpYMRnOfKMvcWigSl51EZd+ADKDkc3XLFGegtLmyBP2ujpGyxIWhEtj1pQWUfNNyxcN9wixzSNmSPpOZO3SD65JVcx/YFFEC6fWX11jKIews3grnZVcGDHiTWuyewMjizESly5Wf7VlEdd7AqPYvnixALG/yWTN//Kj/m8erTXizMwABi8Z+xLFjpDRzUUHzf/Pflj+M56BezXunGhh9BZofBqeI5iQZKnz2WzKrkZCfXo1bbh4/IY1O0Dp/o/b+wBxOeVR1uqU3goyB7E7aZ5ttlz9X2cyYo/OByX8muwviLzw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: io_uring allows implementing custom file specific asynchronous operations via the fops->uring_cmd callback, a.k.a. IORING_OP_URING_CMD requests or just io_uring commands. Use it to add support for async discards. Normally, it first tries to queue up bios in a non-blocking context, and if that fails, we'd retry from a blocking context by returning -EAGAIN to the core io_uring. We always get the result from bios asynchronously by setting a custom bi_end_io callback, at which point we drag the request into the task context to either reissue or complete it and post a completion to the user. Unlike ioctl(BLKDISCARD) with stronger guarantees against races, we only do a best effort attempt to invalidate page cache, and it can race with any writes and reads and leave page cache stale. It's the same kind of races we allow to direct writes. Also, apart from cases where discarding is not allowed at all, e.g. discards are not supported or the file/device is read only, the user should assume that the sector range on disk is not valid anymore, even when an error was returned to the user. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov --- block/blk.h | 1 + block/fops.c | 2 + block/ioctl.c | 112 ++++++++++++++++++++++++++++++++++++ include/uapi/linux/blkdev.h | 14 +++++ 4 files changed, 129 insertions(+) create mode 100644 include/uapi/linux/blkdev.h diff --git a/block/blk.h b/block/blk.h index 32f4e9f630a3..1a1a18d118f7 100644 --- a/block/blk.h +++ b/block/blk.h @@ -605,6 +605,7 @@ blk_mode_t file_to_blk_mode(struct file *file); int truncate_bdev_range(struct block_device *bdev, blk_mode_t mode, loff_t lstart, loff_t lend); long blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg); +int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags); long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg); extern const struct address_space_operations def_blk_aops; diff --git a/block/fops.c b/block/fops.c index 9825c1713a49..8154b10b5abf 100644 --- a/block/fops.c +++ b/block/fops.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "blk.h" static inline struct inode *bdev_file_inode(struct file *file) @@ -873,6 +874,7 @@ const struct file_operations def_blk_fops = { .splice_read = filemap_splice_read, .splice_write = iter_file_splice_write, .fallocate = blkdev_fallocate, + .uring_cmd = blkdev_uring_cmd, .fop_flags = FOP_BUFFER_RASYNC, }; diff --git a/block/ioctl.c b/block/ioctl.c index 6d663d6ae036..007f6399de66 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -11,6 +11,9 @@ #include #include #include +#include +#include +#include #include "blk.h" static int blkpg_do_ioctl(struct block_device *bdev, @@ -747,3 +750,112 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg) return ret; } #endif + +struct blk_iou_cmd { + int res; + bool nowait; +}; + +static void blk_cmd_complete(struct io_uring_cmd *cmd, unsigned int issue_flags) +{ + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + + if (bic->res == -EAGAIN && bic->nowait) + io_uring_cmd_issue_blocking(cmd); + else + io_uring_cmd_done(cmd, bic->res, 0, issue_flags); +} + +static void bio_cmd_bio_end_io(struct bio *bio) +{ + struct io_uring_cmd *cmd = bio->bi_private; + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + + if (unlikely(bio->bi_status) && !bic->res) + bic->res = blk_status_to_errno(bio->bi_status); + + io_uring_cmd_do_in_task_lazy(cmd, blk_cmd_complete); + bio_put(bio); +} + +static int blkdev_cmd_discard(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + gfp_t gfp = nowait ? GFP_NOWAIT : GFP_KERNEL; + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + int err; + + if (!bdev_max_discard_sectors(bdev)) + return -EOPNOTSUPP; + if (!(file_to_blk_mode(cmd->file) & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + err = blk_validate_byte_range(bdev, start, len); + if (err) + return err; + + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, nowait); + if (err) + return err; + + while (true) { + bio = blk_alloc_discard_bio(bdev, §or, &nr_sects, gfp); + if (!bio) + break; + if (nowait) { + /* + * Don't allow multi-bio non-blocking submissions as + * subsequent bios may fail but we won't get a direct + * indication of that. Normally, the caller should + * retry from a blocking context. + */ + if (unlikely(nr_sects)) { + bio_put(bio); + return -EAGAIN; + } + bio->bi_opf |= REQ_NOWAIT; + } + + prev = bio_chain_and_submit(prev, bio); + } + if (unlikely(!prev)) + return -EAGAIN; + if (unlikely(nr_sects)) + bic->res = -EAGAIN; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_bio_end_io; + submit_bio(prev); + return -EIOCBQUEUED; +} + +int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) +{ + struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host); + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + const struct io_uring_sqe *sqe = cmd->sqe; + u32 cmd_op = cmd->cmd_op; + uint64_t start, len; + + if (unlikely(sqe->ioprio || sqe->__pad1 || sqe->len || + sqe->rw_flags || sqe->file_index)) + return -EINVAL; + + bic->res = 0; + bic->nowait = issue_flags & IO_URING_F_NONBLOCK; + + start = READ_ONCE(sqe->addr); + len = READ_ONCE(sqe->addr3); + + switch (cmd_op) { + case BLOCK_URING_CMD_DISCARD: + return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); + } + return -EINVAL; +} diff --git a/include/uapi/linux/blkdev.h b/include/uapi/linux/blkdev.h new file mode 100644 index 000000000000..66373cd1a83a --- /dev/null +++ b/include/uapi/linux/blkdev.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _UAPI_LINUX_BLKDEV_H +#define _UAPI_LINUX_BLKDEV_H + +#include +#include + +/* + * io_uring block file commands, see IORING_OP_URING_CMD. + * It's a different number space from ioctl(), reuse the block's code 0x12. + */ +#define BLOCK_URING_CMD_DISCARD _IO(0x12, 0) + +#endif From patchwork Wed Sep 11 16:34:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13800882 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF05CEE57C0 for ; Wed, 11 Sep 2024 16:34:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28D0F940069; Wed, 11 Sep 2024 12:34:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23BFE940066; Wed, 11 Sep 2024 12:34:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B58A940069; Wed, 11 Sep 2024 12:34:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D9A77940066 for ; Wed, 11 Sep 2024 12:34:31 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7DE7E121891 for ; Wed, 11 Sep 2024 16:34:31 +0000 (UTC) X-FDA: 82553005542.19.C1529C6 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) by imf17.hostedemail.com (Postfix) with ESMTP id 9650F4000A for ; Wed, 11 Sep 2024 16:34:29 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=JAAaKOuT; spf=pass (imf17.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.49 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726072331; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=scYTie9WkhCZlLX/43ma4VJqtQTMJpI9jsSYXTTakWw=; b=f5v1hPsQQN06HLUFI/4xOESFxnvL4xBN0uTQwN8gKRepPx4ha1u/E59jk3hJTeKUidKza1 5/U38QE4HjiAo9Wb4bfUbaPPE7MBQYoWWTh/oyrfwDmC9g3MEu8iUQq1MCnkPuulZOQwud ICbWF0dgTl2eKi8lrm58X0p+kqVj9P4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=JAAaKOuT; spf=pass (imf17.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.49 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726072331; a=rsa-sha256; cv=none; b=2zmSNeV/77GZPceXaBBGofmAiFD7zLT3lhN6cmG1S9mO1qqLX3sQnPhwKVjnFDVLh54MpA iz5VnZ9oOVpfSW/XSo9fDrW1A+zRseLbpL0K9NeH4AZgC61r9EWf5038H/xcgdjrFMdZbZ 7LbE2C/HZW9GhLjUZcZ7TqmWBdWqpm0= Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-a8a789c4fc5so190250466b.0 for ; Wed, 11 Sep 2024 09:34:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726072468; x=1726677268; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=scYTie9WkhCZlLX/43ma4VJqtQTMJpI9jsSYXTTakWw=; b=JAAaKOuTSdrsfKrUxa26kl+DoYqzyuDejs9ci/GzEHVHJBtlYp5REX1RNGpGa7s0uM GRiMlQKnH0PVUOgiKtHpuy4+zSNrrVw4NsE6YPXipwZBNZ+YoXCnhZODegefi8FTbwIK EMHTzPqsGC0QnxVSKpLJVXuiJpLHKa7Kd2ntV73a0x4FjxEOu9fzP3Dcv2qstSBMlbkp 5zE1bBDtlxzHlnvQ+W7FSvFUxM54Jnms0noRmaUsm3kwpkM/tnNU/7NvK38Avz6zVxtW PgNft1qNekt1wTMtFurApPE+kxihXK/HgfBU6gLmJla9CCGsprboJfXzzYeXcDFKmMBg 1MPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726072468; x=1726677268; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=scYTie9WkhCZlLX/43ma4VJqtQTMJpI9jsSYXTTakWw=; b=lwVWw/QZad5rMfzo9uWlKwqSt/ZdxS/UGl6uyO1OaggLim9Eh/06pZJV1Wki/YhC9V 7Sjy5Oo+5W6jayYU2FsqN8CgVQyzvp4/LL1iSR7BFFGpn1DZz6sTvSMhB6UXms01utrJ Pk5QJm3fBTjO+y3xehjgUQ9ARiJX47CTd38nZWSdjrR1zzddJFddlyrg/Z/CWqjh5jWc RPx8fIkHFM9LTcj2P/nXeicT23MV8okJxaULxUXUMybJZgKjAeOGl3Pn7D8kEvVJ/8yj P2OhUZHC8Nll7F6CdlWL9Ryl4s05cUdwPdbFNgDT+Up+0MLD64Lwv0lo3pGW24fedYIC 9PkA== X-Forwarded-Encrypted: i=1; AJvYcCXJjlrhwLPwOb56lqNobdTff+pp1e1ALyQzKS0E/vSZGYLTHNQfW7j7Wq0/igTA8AjDEeBKv+iOaw==@kvack.org X-Gm-Message-State: AOJu0YwmS6i+n7OApXXYiw9aD0ZTlyWNr1+A/cMFoH7pO3BYdT9+14cu EiG4pHc4hW2OHfvkN5q9oBtCDdndqp1GvfR5I5xm1lN7tQbaBovZ X-Google-Smtp-Source: AGHT+IFtkK/R+pcDbDqSnBe+yOdtrkxtJbf96YiBq1nB7U6U1178eNxdDg00TSuprdAdoC34frzF6g== X-Received: by 2002:a17:907:944b:b0:a8a:87d5:2f49 with SMTP id a640c23a62f3a-a8ffae3a20cmr485726666b.28.1726072468161; Wed, 11 Sep 2024 09:34:28 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25c72ed3sm631820866b.135.2024.09.11.09.34.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 09:34:27 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Conrad Meyer Subject: [PATCH v5 6/8] block: implement write zeroes io_uring cmd Date: Wed, 11 Sep 2024 17:34:42 +0100 Message-ID: <8e7975e44504d8371d716167face2bc8e248f7a4.1726072086.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9650F4000A X-Stat-Signature: 1o6xxd5jo54m1kqms317zjrrpngb4x4y X-Rspam-User: X-HE-Tag: 1726072469-784951 X-HE-Meta: U2FsdGVkX1+RvwlN2A3e5As4oMHS2Q3MgmreKraTjmTlPwuotbs2yO16qqPH4gvdYY4xHafL0nK2xhBMi6Rk6rzTdaP7tyLDheycfP81dP4mTnXc9yFzi7c9t2smUNy/+Me7PK+4Y6jTioBMykn3crY23+2sJCSVp2AN9yhDtG/Xy+iC9itlL0y+iyde5yP4laS+wSw3ByYrNKzKvAPm5B9f9tpKzBjfNl+gId0kit8F5d7TiDuDeLAp1Zk+mq2WAkFlSLtVT8CjJmyUwCNw/a3rzE3PVW0lianRNRCF2he+O3y3Il9b/pxlSKd52xwSwYw3+MlhvHjFOLcz3hJ9hyo8qiLaagxKVrDDJRi5ioQRuTXEYnLgC7iX2bh7uadvPJJfhbuRKIqIrGIpWlcoGJ54P7QnZcBYOi2Zdveup8oQejorY8J0v1rm950BTu20dRQKtjlbe63SgXcExS4Y4C23YKatQ21yM1KZfezbKv5zUZdC/qSAvDndVwajjekTTQ1wnWuCPVan0fhR0Myxz3BRABk3TguSNio8s70nC2p0ve66/s6aFKSU6so5jmNAc6t81CII5VKC53fFdAAln6RdsPgUd15RVgANvEaz3UWUmGyDqlfp29vabVwRaCwpLCoHaAbBpu9tQn/cV+mcAe8CjB1cxPOU2YxCnLGBeszfggBFGLI/JIccbIyqX2tLBjnBbOwl2jA/sYkCfg/RozEoPtkw3BpnzQwrdjkUNDgcI2UCSAAjCaaAFtTGfJcGt8sL5n4dx23Kvnwh2v9X/oDm2xygFzkEX008+qQpZOWs+DW8LSci+V4UCksNoq1TBSey67oQT0p8yUWv1LZjkv01vL+x6/uimaoJGmeFTae3t7Ym+JBc668uo2Ma31cFzfSYlj9DyhvitUUv0sYh8gzE2QQAqO6PB/k+wr/qpS4hNCx9W10cuowuInTWxDtIwygL796RN7t4JxMNhGF 0bEWmk7n 4K6ERpXYTlHdoNNkT9i82OsRGcPIH6h8JgbpYV+hQy3XdnL8cLpHGac7lg2slfcTcGO7n+9beZZo1Nntt+qZGlAXXghue2jgS7cn4Z98pIQzHKtZTcz/S5ZrFWz8x/qwaL9D8ZefUggT5RSZt0vtTAyh6nRpwqwWzp+xFo5z2TXvDEo5SWDdTH8jF0VpSINoblhsiIogaj/bokueusfTRMQ3Vk6OfhU+PFWHLSlpvZ9yyUa2W/7lcr9vXdeEDIK42PORStiOMEvD0kk5oFD/N1iobcAUb5U8Ll5zn8WLp0QUPAnm9sEtXQYTWDPJVi+GNL5mNXhY0Ly8/PV+wjXkvlCeT3fUTXGnY8FCzmEJ8OrVko8BIDmYoiFt6bBpGko1zvzU469BUQI96a5x+59yyIlZ32J6zHCzvgLLGgJ8ZLb5fTPVYfqWuOkxF2JQiVFtld2YxYYlkoIgNqOwW1CPatFr7YXUor0vAfj2WYCAnOxAlzI9WpqajMrXY6dmRh/pye9zC9dVAs1h5W51gujicUHTFiNdV89dYwbJYwMx8NDkS/OrWce4go8LwIePl8U4Nq+DCgzpXjSQAzZ0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a second io_uring cmd for block layer implementing asynchronous write zeroes. It reuses helpers we've added for async discards, and inherits the code structure as well as all considerations in regards to page cache races. It has to be supported by underlying hardware to be used, otherwise the request will fail. A fallback version is implemented separately in a later patch. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov Nacked-by: Christoph Hellwig --- block/ioctl.c | 66 +++++++++++++++++++++++++++++++++++++ include/uapi/linux/blkdev.h | 1 + 2 files changed, 67 insertions(+) diff --git a/block/ioctl.c b/block/ioctl.c index 007f6399de66..f8f6d6a71ff0 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -778,6 +778,69 @@ static void bio_cmd_bio_end_io(struct bio *bio) bio_put(bio); } +static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; + sector_t limit = bdev_write_zeroes_sectors(bdev); + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + gfp_t gfp = nowait ? GFP_NOWAIT : GFP_KERNEL; + int err; + + if (!(file_to_blk_mode(cmd->file) & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + err = blk_validate_byte_range(bdev, start, len); + if (err) + return err; + + if (!limit) + return -EOPNOTSUPP; + /* + * Don't allow multi-bio non-blocking submissions as subsequent bios + * may fail but we won't get a direct indication of that. Normally, + * the caller should retry from a blocking context. + */ + if (nowait && nr_sects > limit) + return -EAGAIN; + + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, nowait); + if (err) + return err; + + limit = min(limit, (UINT_MAX >> SECTOR_SHIFT) & ~bs_mask); + while (nr_sects) { + sector_t bio_sects = min(nr_sects, limit); + + bio = bio_alloc(bdev, 0, REQ_OP_WRITE_ZEROES|REQ_NOUNMAP, gfp); + if (!bio) + break; + if (nowait) + bio->bi_opf |= REQ_NOWAIT; + bio->bi_iter.bi_sector = sector; + bio->bi_iter.bi_size = bio_sects << SECTOR_SHIFT; + sector += bio_sects; + nr_sects -= bio_sects; + + prev = bio_chain_and_submit(prev, bio); + } + if (unlikely(!prev)) + return -EAGAIN; + if (unlikely(nr_sects)) + bic->res = -EAGAIN; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_bio_end_io; + submit_bio(prev); + return -EIOCBQUEUED; +} + static int blkdev_cmd_discard(struct io_uring_cmd *cmd, struct block_device *bdev, uint64_t start, uint64_t len, bool nowait) @@ -856,6 +919,9 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) switch (cmd_op) { case BLOCK_URING_CMD_DISCARD: return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); + case BLOCK_URING_CMD_WRITE_ZEROES: + return blkdev_cmd_write_zeroes(cmd, bdev, start, len, + bic->nowait); } return -EINVAL; } diff --git a/include/uapi/linux/blkdev.h b/include/uapi/linux/blkdev.h index 66373cd1a83a..b4664139a82a 100644 --- a/include/uapi/linux/blkdev.h +++ b/include/uapi/linux/blkdev.h @@ -10,5 +10,6 @@ * It's a different number space from ioctl(), reuse the block's code 0x12. */ #define BLOCK_URING_CMD_DISCARD _IO(0x12, 0) +#define BLOCK_URING_CMD_WRITE_ZEROES _IO(0x12, 1) #endif From patchwork Wed Sep 11 16:34:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13800883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62CFEEE57C2 for ; Wed, 11 Sep 2024 16:34:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 667DA94006A; Wed, 11 Sep 2024 12:34:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E8DB940066; Wed, 11 Sep 2024 12:34:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 413A094006A; Wed, 11 Sep 2024 12:34:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0DF2E940066 for ; Wed, 11 Sep 2024 12:34:33 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B5B24161997 for ; Wed, 11 Sep 2024 16:34:32 +0000 (UTC) X-FDA: 82553005584.17.DC5466F Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by imf02.hostedemail.com (Postfix) with ESMTP id DF61680011 for ; Wed, 11 Sep 2024 16:34:30 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OnJ2vqNf; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.52 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726072394; a=rsa-sha256; cv=none; b=bfAjCdZilMC7ZXycp6ug3P6DC9dL3bxojxjSo5+IM4CyLXbCV5Z5htePZNdabkwOsUy99s kSHY7ccqJ3QhHZr+/qJGjhjHED2+Pav4/tLzDEiEgHhk49VDi7fZvc1gtAgg5oZCYfJFSo RhFW714r6nmwxPvvg1xHf0tIV3DsLik= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OnJ2vqNf; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.52 as permitted sender) smtp.mailfrom=asml.silence@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726072394; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Jv/xRgK93eDswhXa+H3bNlv587Pzbd+1FrWwfpAcjww=; b=4nvCNylF+b6qAHvANIzK3J1iHXpyvflfshfKDBbsDD7LThzXDOlupjExFpYVUiFHfVdEdj ZKZNCMRG0ww201fRfjFq++TtsBdFh4SfrHZx6aaVY33zu8DwPUsegrV4/tQpEgmxSDxKo9 txouM/qFcRDw4Ci7nUh4U5RpOAzh9ZE= Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a8a7cdfdd80so7192966b.0 for ; Wed, 11 Sep 2024 09:34:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726072469; x=1726677269; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jv/xRgK93eDswhXa+H3bNlv587Pzbd+1FrWwfpAcjww=; b=OnJ2vqNfqa8AW2WRd1NrwCRj4LM8Tzl/L1MfrnfPcAD5MqaMZousMPXC+W1ytLYzhv kiRRwT8CnycIRwxWTpZ8PBo+XWzEUVJ9LpFw2juR2HJzkJehXRWq/pnNr/VUC0e39m2p M7ZFzbDyEu5FsWZhX4jsZ4SiMgz80DlkJcDDlNzcV/ZmVgB3keCWDbMYDYhGuZRJJ+wx yOd46RomEZJsf1QdI1+EdM1jlKl88inZeNNRiOUHPPHYp9/G8ASRB9tLJwBlWRkwG4fy Nzypi0TuTmGgevpZCmfqpoOpAgsS7zlAseVseAySrsek0EC5b2e7GYsQQOVvl0k/SpK5 VoZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726072469; x=1726677269; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jv/xRgK93eDswhXa+H3bNlv587Pzbd+1FrWwfpAcjww=; b=Vf8WaL72vmfe7BsehjuNw4eOGaFlc0tz1mcGyO8lVfz3mTaxAclPtxzRdCYVHsSphe FPrw6qek8BWUjs9aXUXP3k/6lPChzdAYfbDQ4g3Xg1mrsGdiq873kZL6m3cWPFEYXKOv /Umz1B/v6tvaAOj5WpSjkYUwPOR+5PnMuvJmkN8kfSostj3OYt5xfrWUPbVXO5FU0wf2 p5Dl8Gvd+7R/DqPNK7DkHP+4Bq9vKcwtX0t6j64jXoiQ9ZA53In4vF+T+ukEYbC+iJ4q NBGuBKh+nWCvcBD3sZnVH/+wVGIx8TcCZIcUy5XYpaS2Fy/HzC5bd7ON/VA/S8PDf4GN q/PQ== X-Forwarded-Encrypted: i=1; AJvYcCVEBirHoRHfwCEby6hlLQ/rwIe0R0j2HOXPzklUlOYlbQAhxDBS72ZWPhs+WKdgg7BxoPTsQk+h/w==@kvack.org X-Gm-Message-State: AOJu0YzpVJHEnj1dkM4zePnbKhT293vgHgTvfQ8mBxamuw7r8s4WWmP+ WjVAzOZNZFR0CEr6kTW1ajcr/MWpCT2fyxOAGe5FtqxUC3uWpPhs X-Google-Smtp-Source: AGHT+IGuiTLA2mNX0YCzkTRNyEFx9mbWso8US7cNHDcH1lN98BWCM89d0c663QhibBxvnr/nZH5TzQ== X-Received: by 2002:a17:907:f186:b0:a86:f960:411d with SMTP id a640c23a62f3a-a9029409aedmr9838766b.2.1726072469422; Wed, 11 Sep 2024 09:34:29 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25c72ed3sm631820866b.135.2024.09.11.09.34.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 09:34:28 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v5 7/8] block: add nowait flag for __blkdev_issue_zero_pages Date: Wed, 11 Sep 2024 17:34:43 +0100 Message-ID: <0a4e17bf70d5fa035329664b25a780bf4f53e38f.1726072086.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: DF61680011 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ou9cruspuwuqbgcgosrxhb13tgtiaz7e X-HE-Tag: 1726072470-84347 X-HE-Meta: U2FsdGVkX1/CTwPt6l+aYNIWSFVvD7EknxAO4qBAqPVxN2GfwC2LMtEKPOeKF14Kw5nV0P0SEy4Nr+QabZKFUloZwYqd0yacOMyOKzNPNfA+BO698YpjSb6SFeuXKBYAUit9V1dyxkdmvVerdbNMItSqTNquKj3ufMlJTm+2cFB4AA+DaEPSX3JBknmwsufBUibMFKBKo7W05B74v84+1+XCSP7t9KiCVMl6RAdQkC9Px/S+UgZ57QkN2S0iLUiL5yPstX60W0UMkgoDSlqqzFvtkVeaSTlMhKl59psIP2FTWyehcNjTgZ6UmFqcYIlsTqSimnz3rKh96aZMryGzdcGsOYQrPrhJly2Zyw3M2KWdWSLULG0nFsKVU/o0CFAPK0TaG957v14dESNychvo5EPMk10WMZd2WCOXEw/H9bu1yiVs3DkZxD7nqZIo3ECs1VQvsA+9oHZXZ7qkIBajv9lowWjx2fl9RNlVQ1eEK/VIeK6HXSiIcRrfXcm9iC+VWWloUjdt5vELggy78qh2SJ7BUVaQjuJnv+Of7P5G56YW9/pyXyu9265ypDOq6YJfkpPm/DHmmigZsk4l+ddyHHEV0VRBGTOY2Kbff4Td/eCeyQ6sLuze9bMM4nCs5tDfQ1CoUpa/Uh+Ufd7Oiiyu0kipwJuUaD3SCW72U0HBlfPPO5IIH165m2eXV4tHIXFzy/bLogcOjS/NpYlAbLLiU3/FEvx5OYw+6yYhYxns1n7Z9aLDvbk1GDWFvXYhHaJ//I4b8Bgsa44jrqeGS3v/R4GZ0HoHcb+cB39ffSlyu/M263/etMHk02RfwdsOLAyxSHdQ+cWAuJbwkOFTacpMqkZ6+6keytm82IYEp/GZza9OesMPPGKfuRCxGccxrNASrCz6KR+sg1/En0gDrns04IzndhD+Dz8pwJQmb5iHymVhGatKDGSdKb7eYaoCvVwruI9OMRcfDgZGYL7/kMU AzAcXnnv i15j1dnR2fi3JUsLn7dmnKK6ABdAOJ2lbq25Z/26LDXgYEcXoT9cUTm1q2WiXfSvwlFkisQJ5P1yVj2Bx/m4ebFfacz412WLos5zy/QddafiKQVrvCjTpCxjau8JehciLM9Zb5cTIRTi0yVKaXXwcS2j+gXglbkGfkT8hUEeja/9+F0reUuv1K3aeIgUddpMq/x5FTkG23ldKRr5z3OFJ5dO0QPIsj/tx4/j23GmzJ4h+1oX6a4PK2NxMcTQkg4u0cLIzn6SLYUe7SAAqylGPE3Zzw1kGn8aEuZWpB2OkQejN+8GyfI4MOM2pLBXIYSMmdEhVSj+8xCG1iR5rAVDpz6W4T43WDnrYLCzj6vUo9b2MUrJmd+IAiplbRq5VK3e/UiB8qNnh4kApzb/q+BYmS52cQQhqbnylygApLP7TQopTbPSUb+DkE1UhaFveDSQPLqtFahOCfYFggT4ib1F79PcgUA9Om8LXw6aNe4BI8WGSGX0gjKgr4ZNLe/LosAcagpad/ZJYuyezZkbNfl3FXuYhneJw01yMK/pUVZibHC1gv594tWItJiby8F202ysk/hg5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To reuse __blkdev_issue_zero_pages() in the following patch, we need to make it work with non-blocking requests. Add a new nowait flag we can pass inside. Return errors if something went wrong, and check bio_alloc() for failures, which wasn't supposed to happen before because of what gfp flags the callers are passing. Note that there might be a bio passed back even when the function returned an error. To limit the scope of the patch, don't add return code handling to callers, that can be deferred to a follow up. Signed-off-by: Pavel Begunkov --- block/blk-lib.c | 22 ++++++++++++++++++---- include/linux/bio.h | 4 ++++ include/linux/blkdev.h | 1 + 3 files changed, 23 insertions(+), 4 deletions(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index 83eb7761c2bf..40bb59f583ee 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -192,20 +192,32 @@ static unsigned int __blkdev_sectors_to_bio_pages(sector_t nr_sects) return min(pages, (sector_t)BIO_MAX_VECS); } -static void __blkdev_issue_zero_pages(struct block_device *bdev, +int blkdev_issue_zero_pages_bio(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned int flags) { + blk_opf_t opf = REQ_OP_WRITE; + + if (flags & BLKDEV_ZERO_PAGES_NOWAIT) { + sector_t max_bio_sectors = BIO_MAX_VECS << PAGE_SECTORS_SHIFT; + + if (nr_sects > max_bio_sectors) + return -EAGAIN; + opf |= REQ_NOWAIT; + } + while (nr_sects) { unsigned int nr_vecs = __blkdev_sectors_to_bio_pages(nr_sects); struct bio *bio; bio = bio_alloc(bdev, nr_vecs, REQ_OP_WRITE, gfp_mask); + if (!bio) + return -ENOMEM; bio->bi_iter.bi_sector = sector; if ((flags & BLKDEV_ZERO_KILLABLE) && fatal_signal_pending(current)) - break; + return -EINTR; do { unsigned int len, added; @@ -222,6 +234,8 @@ static void __blkdev_issue_zero_pages(struct block_device *bdev, *biop = bio_chain_and_submit(*biop, bio); cond_resched(); } + + return 0; } static int blkdev_issue_zero_pages(struct block_device *bdev, sector_t sector, @@ -235,7 +249,7 @@ static int blkdev_issue_zero_pages(struct block_device *bdev, sector_t sector, return -EOPNOTSUPP; blk_start_plug(&plug); - __blkdev_issue_zero_pages(bdev, sector, nr_sects, gfp, &bio, flags); + blkdev_issue_zero_pages_bio(bdev, sector, nr_sects, gfp, &bio, flags); if (bio) { if ((flags & BLKDEV_ZERO_KILLABLE) && fatal_signal_pending(current)) { @@ -285,7 +299,7 @@ int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector, } else { if (flags & BLKDEV_ZERO_NOFALLBACK) return -EOPNOTSUPP; - __blkdev_issue_zero_pages(bdev, sector, nr_sects, gfp_mask, + blkdev_issue_zero_pages_bio(bdev, sector, nr_sects, gfp_mask, biop, flags); } return 0; diff --git a/include/linux/bio.h b/include/linux/bio.h index faceadb040f9..e4e495a4a95b 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -684,4 +684,8 @@ struct bio *bio_chain_and_submit(struct bio *prev, struct bio *new); struct bio *blk_alloc_discard_bio(struct block_device *bdev, sector_t *sector, sector_t *nr_sects, gfp_t gfp_mask); +int blkdev_issue_zero_pages_bio(struct block_device *bdev, + sector_t sector, sector_t nr_sects, gfp_t gfp_mask, + struct bio **biop, unsigned int flags); + #endif /* __LINUX_BIO_H */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 643c9020a35a..bf1aa951fda2 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1098,6 +1098,7 @@ int blkdev_issue_secure_erase(struct block_device *bdev, sector_t sector, #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ #define BLKDEV_ZERO_KILLABLE (1 << 2) /* interruptible by fatal signals */ +#define BLKDEV_ZERO_PAGES_NOWAIT (1 << 3) /* non-blocking submission */ extern int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, From patchwork Wed Sep 11 16:34:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13800884 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0B57EE57C0 for ; Wed, 11 Sep 2024 16:34:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E252694006B; Wed, 11 Sep 2024 12:34:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DBBBA940066; Wed, 11 Sep 2024 12:34:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD9F994006B; Wed, 11 Sep 2024 12:34:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8E112940066 for ; Wed, 11 Sep 2024 12:34:34 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 46390C19A3 for ; Wed, 11 Sep 2024 16:34:34 +0000 (UTC) X-FDA: 82553005668.10.114475B Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) by imf23.hostedemail.com (Postfix) with ESMTP id 0CA5C14000E for ; Wed, 11 Sep 2024 16:34:31 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=DgnkOgEP; spf=pass (imf23.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726072444; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tt1Sdp5HcYzfSkBqCxePJrZcv+DNb0SwUWNG181PEJo=; b=TFOIh9wtXAXBq12qIdMYNT3O8Pwv6i+Wys6ohUNSDHgtnP5MxbWcKw9oH8sDkHyVznOEyJ JRci/p8MZwV35FipYktts0O0CKhA5GVMviH/EzH8DxpZNXiQ9uhmOPVUShTmJgjqEYPj+n +cG8lWhL6Iq4ZKuNOu5NCXBMRX8Fgn0= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=DgnkOgEP; spf=pass (imf23.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726072444; a=rsa-sha256; cv=none; b=gcPCOm4RuD6w+6rJQy+qhPRe4bkoIe3BZn0u4BPDqj2k5IhLUubhZ/OBnw4+JlizYssOIO z56dATDvxNvdWUIQskWSK2CveOV/oFScq7V7y+J8kORRwY36SSIKM3YbDqsepqQIJG3lkY oT9KzIkQcEye3eLlr6DDK70DXbIcMGs= Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-a8a7929fd64so6665666b.0 for ; Wed, 11 Sep 2024 09:34:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726072470; x=1726677270; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tt1Sdp5HcYzfSkBqCxePJrZcv+DNb0SwUWNG181PEJo=; b=DgnkOgEPV3QXGTbsDOXNZ+LtJeVGgevmEvoGUYWjdiVYkRNKzKU7FldSw2At/MNPv/ V5+s/4JS1mKyhFaPioogdA4Ftimhd9OGyteoSBhu6dMYjvstqJOKyaWcm+naPlhy7CzH Y3mipIzB5zjx9n07TR9jUSypkydmf2hsrkJdOfXimY3s9pXa7ehnEXF364icCX4zPTcf GuDyWUFODJa5HtwFpfxZEWn776mKLTdVLQnDj4p7eUiOGv0ivhW9ZRw5tjUJwkO6rS+f C01uv4PVuvyZD5oq3xs4KRyBAY3ocUGspscVU072kqjTNv2NQd6lj8o9uSKaUEanN70m E92Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726072470; x=1726677270; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tt1Sdp5HcYzfSkBqCxePJrZcv+DNb0SwUWNG181PEJo=; b=pXM0cJrpjuZT/7VjR9r0rvYWlL7m5PVYZUWC54+BIyGSpWAa7R+WUL/Ha1/St0IKW1 fdpj62/fHxUaapE5lELuDpj9JYz/EtoorcDovFNPX3FBFIbVSgydk3AY3Yg2EOzBL95r NBFQpce85/YDUd/Wf/Qmya6y7hXshEP0S2pQKYq64zE52ZEm3tfnrdgM9wVFI81NKwuT kudjczmlXtFIJsnE350x2nf6l5qOAC6u4IxKIROjfOd5aH8kL0xrv6pP0RuNH98XczqE pGdqqeRSp6NPlc6ZH1MU/XFuAARbjJM8uwSvELmpGJlopINLI9wyaiss0z2L70tU/J4b GyFw== X-Forwarded-Encrypted: i=1; AJvYcCV1pSYsl4rwbPtZFa8UFAjS8RiN7NYhGAv92OT+WGpDGabOtRpPgztmowA6w6qtZWukR0VQUMcA/w==@kvack.org X-Gm-Message-State: AOJu0Yztwv165ShrHB6d9MZ/azlnO+09iuos+W9xO4w30ezFI0A7m+FN oRXvODqKa4nNWJTave3w0p0kxt9VW6MWEOY7J+ZiBfvVR9Yj71+G X-Google-Smtp-Source: AGHT+IGzBI9+dMWJvPlTYbOfk11QQ+FrGskd3fYscDMuABz9+Q22m/SZtfuiKraKUWCJKvHI3YqGqw== X-Received: by 2002:a17:907:d2ce:b0:a8d:3f6a:99ce with SMTP id a640c23a62f3a-a902961a769mr5471566b.49.1726072470444; Wed, 11 Sep 2024 09:34:30 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25c72ed3sm631820866b.135.2024.09.11.09.34.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 09:34:29 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v5 8/8] block: implement write zero pages cmd Date: Wed, 11 Sep 2024 17:34:44 +0100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: ybo85afi353ki9atxdgzg8pggfa636pn X-Rspamd-Queue-Id: 0CA5C14000E X-Rspamd-Server: rspam11 X-HE-Tag: 1726072471-234729 X-HE-Meta: U2FsdGVkX18aT02obgRlKxOszwAc5pjKJiHMgTgjOr8nODssZ5jg/GutRoCJ9kivDI1Pc7jjKNbvE1TCm7W+FgJbxgmqXWWht267HY+BJUDJqwxurkPpLa+XKKdN1TT12jAEwOvrwIa7HTr5sCNTA3O24mgiPnqlhTGqSUzg+vL0jh/a04YD/s9V9+46vAQ+wMDE0W2d+R6TqM2JnxRNxEU0xvlC8DU3svwYmI3x9jQrI5jvxWnEGWBJs/x14OQZzH0MT9CeWek+DnPvUdYOwV/W7/dRj6ZeF7dB79kWJ71VBqlbwzRViv08Kit1VMl7/6nMm2O2fvfu4Ywfj+HH8c3d39Vezza7ilqp3QPeIbWevEEm3vTVdcl/g3TO+PCb33LFDHR1SfxpSanHl6xkqBOJGDWH2eGl5PaHaxrC/nT1GAL/D47ui93OqKZmIJsrUCmK2xJfrdY81YCbbInzhkkmzvOUI6ShxKvHp5kCQX+bSkJiYOt6CPp2dtNkTx4m24VxAbDFrB8unNu0ypocynsZbytg8N7IXdf4uzGIfCWxKeYgiypMLOBAkvWVyNu0929irw0Ba93/29C0pUlMai3sF6GgP5bSDomFZCSy/sU2/ZKN052Si/8doGlYV6ukydKc/nczzPEw7KpNfZEuj6yH5DOWXeBZPmeouQnWztHflh23G7v5MxGQa3EHefCh3/rSPBz7gbjIQaMVW7hPnyNGO5A4ngdG2ORWjY/lZJ2+KUv0aPctoGzVVBAeLscSSrbGYyIAEM6dvJqaWg7VnahMehn5wHIpnJXpS2+ucQ8QCXuaJPtO0PqyZ/2C+ehkKjOeuiDQedsNCcQpWFihbVM0A1n1OE/e9Kyt9m7tLAimn2xoFOqvPGGAi4Ty+HRD4gtBxEx55IRrPpKthsD7B98P58lbTWw9Y8Z7AmKj4X17ZoG55tGWrOzDq76xRXKmpl5Md4PJQ4D7WKrol0z fICQ3X9v gBYI90bhpY7VMOYTRB6LF730O840AB4gMH2/sz52PCx5dTlC5JUNYItopeuGzEeObW3rAz8vZvTJ0flJJYGBk24Rlc2AtpOHukpqyZpwew5kM5Mifbe+H0rRAaV5As8WyvfHvX5WKPEI1Yb+pYlAB2Ri/+E55AEYPqsD3LWuEO0U/P492A9pzBmHaZRHmIspT5VZYkR+zA3SRAfPZO5/Hh+tYMQsiMCkUIQVxLAcH1hWaGjk9pw6k7Cw7KmIZ/kTgHtX8r4sGo5RFLf9rVgMppZqz6RhoY6fldoIkqwgKqo3bvr33w8ea1HxcYSgMy1cCjuv3S2YoqEPT/XRYpKeggSkTbj48kzHM7Q73kfJRpCGHiomCBCOUgi6CCfC63tHx9VK/W8DCHIhPXknQX76WFbhWZhj8y2UAo7SdFOlY9PlQ5WfINXymm9REljxwxVIeR3KlEto58ZkEmSI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a command that writes the zero page to the drive. Apart from passing the zero page instead of actual data it uses the normal write path and doesn't do any further acceleration. Subsequently, it doesn't requires any special hardware support, and for example can be used as a fallback option for BLOCK_URING_CMD_WRITE_ZEROES. Signed-off-by: Pavel Begunkov --- block/ioctl.c | 21 ++++++++++++++++++--- include/uapi/linux/blkdev.h | 1 + 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/block/ioctl.c b/block/ioctl.c index f8f6d6a71ff0..08e0d2d52b06 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -780,7 +780,8 @@ static void bio_cmd_bio_end_io(struct bio *bio) static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, struct block_device *bdev, - uint64_t start, uint64_t len, bool nowait) + uint64_t start, uint64_t len, + bool nowait, bool zero_pages) { struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; @@ -799,6 +800,17 @@ static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, if (err) return err; + if (zero_pages) { + err = blkdev_issue_zero_pages_bio(bdev, sector, nr_sects, + gfp, &prev, + BLKDEV_ZERO_PAGES_NOWAIT); + if (!prev) + return -EAGAIN; + if (err) + bic->res = err; + goto out_submit; + } + if (!limit) return -EOPNOTSUPP; /* @@ -834,7 +846,7 @@ static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, return -EAGAIN; if (unlikely(nr_sects)) bic->res = -EAGAIN; - +out_submit: prev->bi_private = cmd; prev->bi_end_io = bio_cmd_bio_end_io; submit_bio(prev); @@ -921,7 +933,10 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); case BLOCK_URING_CMD_WRITE_ZEROES: return blkdev_cmd_write_zeroes(cmd, bdev, start, len, - bic->nowait); + bic->nowait, false); + case BLOCK_URING_CMD_WRITE_ZERO_PAGE: + return blkdev_cmd_write_zeroes(cmd, bdev, start, len, + bic->nowait, true); } return -EINVAL; } diff --git a/include/uapi/linux/blkdev.h b/include/uapi/linux/blkdev.h index b4664139a82a..880b292d2d0d 100644 --- a/include/uapi/linux/blkdev.h +++ b/include/uapi/linux/blkdev.h @@ -11,5 +11,6 @@ */ #define BLOCK_URING_CMD_DISCARD _IO(0x12, 0) #define BLOCK_URING_CMD_WRITE_ZEROES _IO(0x12, 1) +#define BLOCK_URING_CMD_WRITE_ZERO_PAGE _IO(0x12, 2) #endif