From patchwork Thu Aug 22 03:35:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13772579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2881AC3DA4A for ; Thu, 22 Aug 2024 03:35:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E8E566B0298; Wed, 21 Aug 2024 23:35:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E3F466B029A; Wed, 21 Aug 2024 23:35:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C423A6B029B; Wed, 21 Aug 2024 23:35:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A262B6B0298 for ; Wed, 21 Aug 2024 23:35:46 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5F1F2406A3 for ; Thu, 22 Aug 2024 03:35:46 +0000 (UTC) X-FDA: 82478467092.26.5A50CC5 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf09.hostedemail.com (Postfix) with ESMTP id 84464140017 for ; Thu, 22 Aug 2024 03:35:44 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="dEdh/28j"; spf=pass (imf09.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724297728; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PVhT9OqiSkRhkZBmbxRRu3VwPAT0F6C3XigyZkc8g4k=; b=Z2/FQ6Z/tLZhutWiReS6BqcK5QPnHElbhwQUUOaQ6UQQTFDfVloQxJpIRyccNa4PxJ9poe fSt7vnxuHenuEX9JHPAicfV6pd/yX7EWdRwW52aKJ7zpYugFZUa2pFaS3Rngx/hh5KcJP7 3xghrkju89PjdWPEXA73EDsr/VRhiIs= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="dEdh/28j"; spf=pass (imf09.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724297728; a=rsa-sha256; cv=none; b=51X48Nk7JLDjRwsjonwcvs7bnDoJPYN7erekkgTd6VH9bo75V9XoSl9NRaptezhL8m6zon 1Jk9lisLANDxusnO9OgKrqSPbMW7FiWhJ9cWvu6eFBIUjG35h9R2OAb44bj84WmxKAcsIN hLx6YCB6OAsznSaeJsN1cVuTzY+5TqE= Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-42816ca782dso1737395e9.2 for ; Wed, 21 Aug 2024 20:35:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724297743; x=1724902543; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PVhT9OqiSkRhkZBmbxRRu3VwPAT0F6C3XigyZkc8g4k=; b=dEdh/28jEeuPmlWp8R4w2Tj6LeB8ub9i3qg1hsy4PxPjUA7K4IMSAWWlXOiqQqeZmb ehcq9Xw/6SSX55KQb/yaKNOUUHzmVOANITzfN+2mgfr7o+M4GIEe8GDQ7IamP+Ppj/lJ 6BUsl8Go7KxeEBEcj+gIJDCW2a+iW2goADnC6/MkIJZRgx8qEo1+dZdiGbsCQssBGpK6 kbqvCsWTwpQYR+vtU7QlTcFCl3vXJv0Ocx0PjzP9tnzh4VguNEqdcXzHSEMj+hzmcsyl /L9GSnK4q81kCikfj85PhFfCmNQXWlpJm9iNjwpXWHvKcqNaVqKBRJiP8T0u/4oxnFEc ETTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724297743; x=1724902543; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PVhT9OqiSkRhkZBmbxRRu3VwPAT0F6C3XigyZkc8g4k=; b=MtOg8REYymDxWiXloFYYQUofsecRkuzbDEOj0vj7UWzrBRMfCcsdRRtB3J94PZD4k+ a6lsDUaLMZPiGul0t+R4hi81AOmn1Xjg4seLClKV9zdxDmfousgkpuC2a8qeO6by70cU B8jZweBjaL5Z9o1e9azpnuSedh3402cA4roSt2LLGvTC0oLoPv+kohRihIQlahGZOQH8 QZtQSm85Go3d8+3ht1dfO3KDyS+25vrJwQdza63Xky7CNWaE+foSU37BiQ9rMRwOpFeg Txw12X0XoMUiml4FixzljTqTmB5dOuAyWFCOfsU3jXyPhPnopJW+uiD7eMqFlfgr1M4R UMpw== X-Forwarded-Encrypted: i=1; AJvYcCVTNN5fxUa+/Xjeafy29ilW0fuDk4LxPHeE+D7V2ndpKN8TCaHhf8zBPSHpBijUn0b8e8IPwsyq5Q==@kvack.org X-Gm-Message-State: AOJu0YylA+Y1NY3Sqolza35JORNZoxQiSJuyowLxCsCHruRETGFVplHw +ddRXk+ULQFeFu4y2lZa7bS/wOuroC6Do3u3hWUeM1lTLAo8AB30 X-Google-Smtp-Source: AGHT+IHqvEt/onVyk5pfq2MvnE1mDsVk/iF7QUbNipDxcqrMgadYUmb8t6sc6QQq2b3aRAiZt2CaSw== X-Received: by 2002:a05:600c:4ed2:b0:426:5440:854a with SMTP id 5b1f17b1804b1-42abd21f727mr28548505e9.1.1724297742468; Wed, 21 Aug 2024 20:35:42 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.128.6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42abefc626fsm45491995e9.31.2024.08.21.20.35.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 20:35:41 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com, Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 6/7] block: implement async wire write zeroes Date: Thu, 22 Aug 2024 04:35:56 +0100 Message-ID: <09c5ef75c04c17ee2fd551da50fc9aae3bfce50a.1724297388.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 8p6rte9twnb6jmxcopgri7meubit9bst X-Rspamd-Queue-Id: 84464140017 X-Rspamd-Server: rspam11 X-HE-Tag: 1724297744-6112 X-HE-Meta: U2FsdGVkX1+Hj1C2638BW5HHz5o544A61gthkzAAtc0eRbspORDMQbXDTMguGoh6/DRK8+PUAdoFQTB1a4CQnWD9PQPASt9P1801lvIBYBdZpyukXTcEp0i0hgGngBoPWog3eoBX7GRVIrYIP0SIpnlVlBlHUXCIXcyry/2KmJF+SYW92RBtqqWIGGUe76/AqcLZSSAwj9/Wai8tCn94JfTWM0D6NxTyyhOuS5H+vrRY81LW0NUIkATBCS7QyvRsvNXx3JqhcHjZJbuOv30PMSubGYVnCc9zvjTebm9L3Jlfb2zXI2J67wTvlgpvU6YS8jsutQhkgbt3J1XCdcTN0FtEdNJGypmfCnALh1NY+poXTqpiioFAnrwLJcEEvtAUeYcxTrf50hTBRQjj/AqZcKic3yiVDd6qUg5rOKxIjS5i44J/iHu5LP1gkzkdgrTG7hc0n5GIDUslKOv48i4oKLUGc7y2+KBaKJW8twq4l775lK29RnxymjqCwZWjkPoUM6VmUBEoMk+DFAcNMHUbtACVBvEAHZAlld3JqR4CLSy36FBzJeg3h5LiRuGCw9w/aGvWuhCVa7WKIXbXPfQvvcgg0ugOcIPTTYrR+2txY1xrIvOM5rMF7U3dwk8S0IHD7xWTKmlU5I4JA24rt/GX7MJQUejldD3DCTH9mxnWS5ph1ngvA/tAb76g1mkxFZialAmXR/mF3Vkxn6q9fIfTFiTXvjBaQPq7NbKQyQGfFGmGV5pdKwyssbeEWZxv/Jp4CuVJTUzpvxI8+SA8RjltTj+R5riIEU1pRcGoATKvSE+GKTSjOrH1/+UHCmcGedRWI2o605MiN+XKKFDHnta/9oYzXeqc3tF13pbPviF79M2clKkTsIyQSNlKzHnc8VhcjhX3tKxxxVeLA0gWepnPjZSTmzUyrqW47QIvN9bHkRPwCSEwbsPH5RqZruf70Mi5SeJ1E9En1Dhy9qE0Eof FbBNKY3o 3bN1uEh+CNJ+e8tRrms6/FewwmuHrJ1VVv2ZE0+xTPzaSMELShC0Cb//FgihGGud27RNukzjoC5+iej2LxG8oBCz5NUzqPv9YbPgR7/iSJZAYFzGjbplvd7T/TCszhIu1shLkzxYFbW3xIJU4SXC74MioEdSUPFR1CWAFkySEu2t1871xydeoqJutakD1eRU3asdLI6+7D7X2j73BrYF/8VwMnnNTHxkrkBf8wC15r3uyRm4NXCoCX0qPux47u58AATlcTT6TneUlXI/R0ykyyFa7wsfw7imN62Qsp1Jjt00wQKIJFyb2JL/ZReMa3suQq8mBu8mqIn1m4znyunqPF1PmvkZ4b7hHK8v3jMwtbjQv6YVaP62QfeW9yZ4GehG4IX7vKHQs4+zscpfPdHRdn4sWOaO9J1e2NLMipNzXAClqF35Au9DpX0icppPG++Ym5+x9sJqUqmPzlIKExAkwSiQONIXdzhLiE9Lj1bALwyl2mCvKil9ZtCrLM19yiZ1yM4u5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add another io_uring cmd for block layer implementing asynchronous write zeroes. It reuses helpers we've added for async discards, and inherits the code structure as well as all considerations in regards to page cache races. Suggested-by: Conrad Meyer Signed-off-by: Pavel Begunkov --- block/ioctl.c | 68 +++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 4 +++ include/uapi/linux/fs.h | 1 + 3 files changed, 73 insertions(+) diff --git a/block/ioctl.c b/block/ioctl.c index a9aaa7cb7f73..6f0676f21e7b 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -776,6 +776,71 @@ static void bio_cmd_end(struct bio *bio) bio_put(bio); } +static int blkdev_queue_cmd(struct io_uring_cmd *cmd, struct block_device *bdev, + uint64_t start, uint64_t len, sector_t limit, + blk_opf_t opf) +{ + sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; + sector_t sector = start >> SECTOR_SHIFT; + sector_t nr_sects = len >> SECTOR_SHIFT; + struct bio *prev = NULL, *bio; + int err; + + if (!limit) + return -EOPNOTSUPP; + + err = blk_validate_write(bdev, file_to_blk_mode(cmd->file), start, len); + if (err) + return err; + + err = filemap_invalidate_pages(bdev->bd_mapping, start, + start + len - 1, opf & REQ_NOWAIT); + if (err) + return err; + + limit = min(limit, (UINT_MAX >> SECTOR_SHIFT) & ~bs_mask); + while (nr_sects) { + sector_t bio_sects = min(nr_sects, limit); + + /* + * Don't allow multi-bio non-blocking submissions as subsequent + * bios may fail but we won't get direct feedback about that. + * Normally, the caller should retry from a blocking context. + */ + if ((opf & REQ_NOWAIT) && bio_sects != nr_sects) + return -EAGAIN; + + bio = bio_alloc(bdev, 0, opf, GFP_KERNEL); + if (!bio) + break; + bio->bi_iter.bi_sector = sector; + bio->bi_iter.bi_size = bio_sects << SECTOR_SHIFT; + sector += bio_sects; + nr_sects -= bio_sects; + + prev = bio_chain_and_submit(prev, bio); + } + if (!prev) + return -EFAULT; + + prev->bi_private = cmd; + prev->bi_end_io = bio_cmd_end; + submit_bio(prev); + return -EIOCBQUEUED; +} + +static int blkdev_cmd_write_zeroes(struct io_uring_cmd *cmd, + struct block_device *bdev, + uint64_t start, uint64_t len, bool nowait) +{ + blk_opf_t opf = REQ_OP_WRITE_ZEROES | REQ_NOUNMAP; + + if (nowait) + opf |= REQ_NOWAIT; + return blkdev_queue_cmd(cmd, bdev, start, len, + bdev_write_zeroes_sectors(bdev), opf); +} + static int blkdev_cmd_discard(struct io_uring_cmd *cmd, struct block_device *bdev, uint64_t start, uint64_t len, bool nowait) @@ -843,6 +908,9 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) switch (cmd_op) { case BLOCK_URING_CMD_DISCARD: return blkdev_cmd_discard(cmd, bdev, start, len, bc->nowait); + case BLOCK_URING_CMD_WRITE_ZEROES: + return blkdev_cmd_write_zeroes(cmd, bdev, start, len, + bc->nowait); } return -EINVAL; } diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index e85ec73a07d5..82bbe1e3e278 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1095,6 +1095,10 @@ int __blkdev_issue_discard(struct block_device *bdev, sector_t sector, int blkdev_issue_secure_erase(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp); +struct bio *blk_alloc_write_zeroes_bio(struct block_device *bdev, + sector_t *sector, sector_t *nr_sects, + gfp_t gfp_mask); + #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ #define BLKDEV_ZERO_KILLABLE (1 << 2) /* interruptible by fatal signals */ diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 0016e38ed33c..b9e20ce57a28 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -209,6 +209,7 @@ struct fsxattr { */ #define BLOCK_URING_CMD_DISCARD 0 +#define BLOCK_URING_CMD_WRITE_ZEROES 1 #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */