From patchwork Wed Sep 11 16:34:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13800882 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF05CEE57C0 for ; Wed, 11 Sep 2024 16:34:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28D0F940069; Wed, 11 Sep 2024 12:34:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23BFE940066; Wed, 11 Sep 2024 12:34:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B58A940069; Wed, 11 Sep 2024 12:34:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D9A77940066 for ; Wed, 11 Sep 2024 12:34:31 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7DE7E121891 for ; Wed, 11 Sep 2024 16:34:31 +0000 (UTC) X-FDA: 82553005542.19.C1529C6 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) by imf17.hostedemail.com (Postfix) with ESMTP id 9650F4000A for ; Wed, 11 Sep 2024 16:34:29 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=JAAaKOuT; spf=pass (imf17.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.49 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726072331; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=scYTie9WkhCZlLX/43ma4VJqtQTMJpI9jsSYXTTakWw=; b=f5v1hPsQQN06HLUFI/4xOESFxnvL4xBN0uTQwN8gKRepPx4ha1u/E59jk3hJTeKUidKza1 5/U38QE4HjiAo9Wb4bfUbaPPE7MBQYoWWTh/oyrfwDmC9g3MEu8iUQq1MCnkPuulZOQwud ICbWF0dgTl2eKi8lrm58X0p+kqVj9P4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=JAAaKOuT; spf=pass (imf17.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.49 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726072331; a=rsa-sha256; cv=none; b=2zmSNeV/77GZPceXaBBGofmAiFD7zLT3lhN6cmG1S9mO1qqLX3sQnPhwKVjnFDVLh54MpA iz5VnZ9oOVpfSW/XSo9fDrW1A+zRseLbpL0K9NeH4AZgC61r9EWf5038H/xcgdjrFMdZbZ 7LbE2C/HZW9GhLjUZcZ7TqmWBdWqpm0= Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-a8a789c4fc5so190250466b.0 for ; Wed, 11 Sep 2024 09:34:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726072468; x=1726677268; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=scYTie9WkhCZlLX/43ma4VJqtQTMJpI9jsSYXTTakWw=; b=JAAaKOuTSdrsfKrUxa26kl+DoYqzyuDejs9ci/GzEHVHJBtlYp5REX1RNGpGa7s0uM GRiMlQKnH0PVUOgiKtHpuy4+zSNrrVw4NsE6YPXipwZBNZ+YoXCnhZODegefi8FTbwIK EMHTzPqsGC0QnxVSKpLJVXuiJpLHKa7Kd2ntV73a0x4FjxEOu9fzP3Dcv2qstSBMlbkp 5zE1bBDtlxzHlnvQ+W7FSvFUxM54Jnms0noRmaUsm3kwpkM/tnNU/7NvK38Avz6zVxtW PgNft1qNekt1wTMtFurApPE+kxihXK/HgfBU6gLmJla9CCGsprboJfXzzYeXcDFKmMBg 1MPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726072468; x=1726677268; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=scYTie9WkhCZlLX/43ma4VJqtQTMJpI9jsSYXTTakWw=; b=lwVWw/QZad5rMfzo9uWlKwqSt/ZdxS/UGl6uyO1OaggLim9Eh/06pZJV1Wki/YhC9V 7Sjy5Oo+5W6jayYU2FsqN8CgVQyzvp4/LL1iSR7BFFGpn1DZz6sTvSMhB6UXms01utrJ Pk5QJm3fBTjO+y3xehjgUQ9ARiJX47CTd38nZWSdjrR1zzddJFddlyrg/Z/CWqjh5jWc RPx8fIkHFM9LTcj2P/nXeicT23MV8okJxaULxUXUMybJZgKjAeOGl3Pn7D8kEvVJ/8yj P2OhUZHC8Nll7F6CdlWL9Ryl4s05cUdwPdbFNgDT+Up+0MLD64Lwv0lo3pGW24fedYIC 9PkA== X-Forwarded-Encrypted: i=1; AJvYcCXJjlrhwLPwOb56lqNobdTff+pp1e1ALyQzKS0E/vSZGYLTHNQfW7j7Wq0/igTA8AjDEeBKv+iOaw==@kvack.org X-Gm-Message-State: AOJu0YwmS6i+n7OApXXYiw9aD0ZTlyWNr1+A/cMFoH7pO3BYdT9+14cu EiG4pHc4hW2OHfvkN5q9oBtCDdndqp1GvfR5I5xm1lN7tQbaBovZ X-Google-Smtp-Source: AGHT+IFtkK/R+pcDbDqSnBe+yOdtrkxtJbf96YiBq1nB7U6U1178eNxdDg00TSuprdAdoC34frzF6g== X-Received: by 2002:a17:907:944b:b0:a8a:87d5:2f49 with SMTP id a640c23a62f3a-a8ffae3a20cmr485726666b.28.1726072468161; Wed, 11 Sep 2024 09:34:28 -0700 (PDT) Received: from 127.0.0.1localhost ([163.114.131.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25c72ed3sm631820866b.135.2024.09.11.09.34.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 09:34:27 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Conrad Meyer Subject: [PATCH v5 6/8] block: implement write zeroes io_uring cmd Date: Wed, 11 Sep 2024 17:34:42 +0100 Message-ID: <8e7975e44504d8371d716167face2bc8e248f7a4.1726072086.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9650F4000A X-Stat-Signature: 1o6xxd5jo54m1kqms317zjrrpngb4x4y X-Rspam-User: X-HE-Tag: 1726072469-784951 X-HE-Meta: U2FsdGVkX1+RvwlN2A3e5As4oMHS2Q3MgmreKraTjmTlPwuotbs2yO16qqPH4gvdYY4xHafL0nK2xhBMi6Rk6rzTdaP7tyLDheycfP81dP4mTnXc9yFzi7c9t2smUNy/+Me7PK+4Y6jTioBMykn3crY23+2sJCSVp2AN9yhDtG/Xy+iC9itlL0y+iyde5yP4laS+wSw3ByYrNKzKvAPm5B9f9tpKzBjfNl+gId0kit8F5d7TiDuDeLAp1Zk+mq2WAkFlSLtVT8CjJmyUwCNw/a3rzE3PVW0lianRNRCF2he+O3y3Il9b/pxlSKd52xwSwYw3+MlhvHjFOLcz3hJ9hyo8qiLaagxKVrDDJRi5ioQRuTXEYnLgC7iX2bh7uadvPJJfhbuRKIqIrGIpWlcoGJ54P7QnZcBYOi2Zdveup8oQejorY8J0v1rm950BTu20dRQKtjlbe63SgXcExS4Y4C23YKatQ21yM1KZfezbKv5zUZdC/qSAvDndVwajjekTTQ1wnWuCPVan0fhR0Myxz3BRABk3TguSNio8s70nC2p0ve66/s6aFKSU6so5jmNAc6t81CII5VKC53fFdAAln6RdsPgUd15RVgANvEaz3UWUmGyDqlfp29vabVwRaCwpLCoHaAbBpu9tQn/cV+mcAe8CjB1cxPOU2YxCnLGBeszfggBFGLI/JIccbIyqX2tLBjnBbOwl2jA/sYkCfg/RozEoPtkw3BpnzQwrdjkUNDgcI2UCSAAjCaaAFtTGfJcGt8sL5n4dx23Kvnwh2v9X/oDm2xygFzkEX008+qQpZOWs+DW8LSci+V4UCksNoq1TBSey67oQT0p8yUWv1LZjkv01vL+x6/uimaoJGmeFTae3t7Ym+JBc668uo2Ma31cFzfSYlj9DyhvitUUv0sYh8gzE2QQAqO6PB/k+wr/qpS4hNCx9W10cuowuInTWxDtIwygL796RN7t4JxMNhGF 0bEWmk7n 4K6ERpXYTlHdoNNkT9i82OsRGcPIH6h8JgbpYV+hQy3XdnL8cLpHGac7lg2slfcTcGO7n+9beZZo1Nntt+qZGlAXXghue2jgS7cn4Z98pIQzHKtZTcz/S5ZrFWz8x/qwaL9D8ZefUggT5RSZt0vtTAyh6nRpwqwWzp+xFo5z2TXvDEo5SWDdTH8jF0VpSINoblhsiIogaj/bokueusfTRMQ3Vk6OfhU+PFWHLSlpvZ9yyUa2W/7lcr9vXdeEDIK42PORStiOMEvD0kk5oFD/N1iobcAUb5U8Ll5zn8WLp0QUPAnm9sEtXQYTWDPJVi+GNL5mNXhY0Ly8/PV+wjXkvlCeT3fUTXGnY8FCzmEJ8OrVko8BIDmYoiFt6bBpGko1zvzU469BUQI96a5x+59yyIlZ32J6zHCzvgLLGgJ8ZLb5fTPVYfqWuOkxF2JQiVFtld2YxYYlkoIgNqOwW1CPatFr7YXUor0vAfj2WYCAnOxAlzI9WpqajMrXY6dmRh/pye9zC9dVAs1h5W51gujicUHTFiNdV89dYwbJYwMx8NDkS/OrWce4go8LwIePl8U4Nq+DCgzpXjSQAzZ0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a second io_uring cmd for block layer implementing asynchronous write zeroes. It reuses helpers we've added for async discards, and inherits the code structure as well as all considerations in regards to page cache races. It has to be supported by underlying hardware to be used, otherwise the request will fail. A fallback version is implemented separately in a later patch. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov Nacked-by: Christoph Hellwig --- block/ioctl.c | 66 +++++++++++++++++++++++++++++++++++++ include/uapi/linux/blkdev.h | 1 + 2 files changed, 67 insertions(+) diff --git a/block/ioctl.c b/block/ioctl.c index 007f6399de66..f8f6d6a71ff0 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -778,6 +778,69 @@ static void bio_cmd_bio_end_io(struct bio *bio) bio_put(bio); } +static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; + sector_t limit = bdev_write_zeroes_sectors(bdev); + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + gfp_t gfp = nowait ? GFP_NOWAIT : GFP_KERNEL; + int err; + + if (!(file_to_blk_mode(cmd->file) & BLK_OPEN_WRITE)) + return -EBADF; + if (bdev_read_only(bdev)) + return -EPERM; + err = blk_validate_byte_range(bdev, start, len); + if (err) + return err; + + if (!limit) + return -EOPNOTSUPP; + /* + * Don't allow multi-bio non-blocking submissions as subsequent bios + * may fail but we won't get a direct indication of that. Normally, + * the caller should retry from a blocking context. + */ + if (nowait && nr_sects > limit) + return -EAGAIN; + + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, nowait); + if (err) + return err; + + limit = min(limit, (UINT_MAX >> SECTOR_SHIFT) & ~bs_mask); + while (nr_sects) { + sector_t bio_sects = min(nr_sects, limit); + + bio = bio_alloc(bdev, 0, REQ_OP_WRITE_ZEROES|REQ_NOUNMAP, gfp); + if (!bio) + break; + if (nowait) + bio->bi_opf |= REQ_NOWAIT; + bio->bi_iter.bi_sector = sector; + bio->bi_iter.bi_size = bio_sects << SECTOR_SHIFT; + sector += bio_sects; + nr_sects -= bio_sects; + + prev = bio_chain_and_submit(prev, bio); + } + if (unlikely(!prev)) + return -EAGAIN; + if (unlikely(nr_sects)) + bic->res = -EAGAIN; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_bio_end_io; + submit_bio(prev); + return -EIOCBQUEUED; +} + static int blkdev_cmd_discard(struct io_uring_cmd *cmd, struct block_device *bdev, uint64_t start, uint64_t len, bool nowait) @@ -856,6 +919,9 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) switch (cmd_op) { case BLOCK_URING_CMD_DISCARD: return blkdev_cmd_discard(cmd, bdev, start, len, bic->nowait); + case BLOCK_URING_CMD_WRITE_ZEROES: + return blkdev_cmd_write_zeroes(cmd, bdev, start, len, + bic->nowait); } return -EINVAL; } diff --git a/include/uapi/linux/blkdev.h b/include/uapi/linux/blkdev.h index 66373cd1a83a..b4664139a82a 100644 --- a/include/uapi/linux/blkdev.h +++ b/include/uapi/linux/blkdev.h @@ -10,5 +10,6 @@ * It's a different number space from ioctl(), reuse the block's code 0x12. */ #define BLOCK_URING_CMD_DISCARD _IO(0x12, 0) +#define BLOCK_URING_CMD_WRITE_ZEROES _IO(0x12, 1) #endif