From patchwork Mon Jul 24 16:54:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13325026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D13EC04A6A for ; Mon, 24 Jul 2023 16:54:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231251AbjGXQyh (ORCPT ); Mon, 24 Jul 2023 12:54:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231221AbjGXQyg (ORCPT ); Mon, 24 Jul 2023 12:54:36 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1C8410F8 for ; Mon, 24 Jul 2023 09:54:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=/L4Vf6XgX1i8Sa3+yr/s3kpmCh1ZQ3gtttexj4gOiSA=; b=if925Q/XjllAAUlGwWCFx9Bepm H8yI+sClIxfqCTajsyyicRcuZb5IVPSLmT+Lt19eNw1Pd79ftNTAHxctpymfEvrho2YNMIfRadJZ2 gvtqyHsRh+Vs0LkPzApeC3YxZWD/gHH8bbX1eimpw4Xt0rKUxmvO63dPHj0AgtImp05J+7AuJwPgW BwDyZWbB+kRQ+LywIi761lpnfjboLKhWFgz4/g3Ro3ia+naKkwolBybGn8XRmPK6sC3kn81QvwXhs rple+7tvKM07c3JBC8ioIf0J6/PLGe+OedRaYrz+PPy5zx6sK4i9STJdZ7HHfRM7fZ/pIckI33gzC bMgoh2bg==; Received: from 67-207-104-238.static.wiline.com ([67.207.104.238] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qNypN-004vtx-1q; Mon, 24 Jul 2023 16:54:33 +0000 From: Christoph Hellwig To: Jens Axboe Cc: Jinyoung Choi , linux-block@vger.kernel.org Subject: [PATCH 1/8] block: tidy up the bio full checks in bio_add_hw_page Date: Mon, 24 Jul 2023 09:54:26 -0700 Message-Id: <20230724165433.117645-2-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230724165433.117645-1-hch@lst.de> References: <20230724165433.117645-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org bio_add_hw_page already checks if the number of bytes trying to be added even fit into max_hw_sectors limit of the queue. Remove the call to bio_full and just do a check for the smaller of the number of segments in the bio and the queue max segments limit, and do this cheap check before the more expensive gap to previous check. Signed-off-by: Christoph Hellwig Reviewed-by: Jinyoung Choi --- block/bio.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/block/bio.c b/block/bio.c index 8672179213b939..72488ecea47acf 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1014,6 +1014,10 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, if (bio_try_merge_hw_seg(q, bio, page, len, offset, same_page)) return len; + if (bio->bi_vcnt >= + min(bio->bi_max_vecs, queue_max_segments(q))) + return 0; + /* * If the queue doesn't support SG gaps and adding this segment * would create a gap, disallow it. @@ -1023,12 +1027,6 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, return 0; } - if (bio_full(bio, len)) - return 0; - - if (bio->bi_vcnt >= queue_max_segments(q)) - return 0; - bvec_set_page(&bio->bi_io_vec[bio->bi_vcnt], page, len, offset); bio->bi_vcnt++; bio->bi_iter.bi_size += len; From patchwork Mon Jul 24 16:54:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13325025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB1CEC0015E for ; Mon, 24 Jul 2023 16:54:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229558AbjGXQyg (ORCPT ); Mon, 24 Jul 2023 12:54:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231202AbjGXQyf (ORCPT ); Mon, 24 Jul 2023 12:54:35 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C16F410F6 for ; Mon, 24 Jul 2023 09:54:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=fo891t+MRuPQ6hqtNNoXRls0WMaXaEefJnnr0O2/hDE=; b=t77oPWR7xbM1JPbK11oQp8uXr+ tNUCxletgrisw9YU+H+/VU974ujGueouskQW5SDrdn4NuB7ptzILcuVkFKI94SatkS2z1yw4MiB7N 2AYKtKCfE4pN6dLuW4NUNZd0fzy/VVR9+bC/cOYUtYg8PJTfEa3XONRiNZZ/3ec/g70dxzfHtatIE eT6gPxoXnrMSzXRgqNOhPKLSBhVP7Bqd8aGkQPj+Kbauv123pD2PhUwW25bYdkxjB5LFUs3esNPQa PXAZsOOCu/URwckGMf4n3W03j5GwzcZ4IfXKgUhpxOL5Tavf+W2mGsBOn4aiuFvg8T/YkkuhVvJMy Xwo2jebw==; Received: from 67-207-104-238.static.wiline.com ([67.207.104.238] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qNypN-004vu4-2X; Mon, 24 Jul 2023 16:54:33 +0000 From: Christoph Hellwig To: Jens Axboe Cc: Jinyoung Choi , linux-block@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 2/8] block: use SECTOR_SHIFT bio_add_hw_page Date: Mon, 24 Jul 2023 09:54:27 -0700 Message-Id: <20230724165433.117645-3-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230724165433.117645-1-hch@lst.de> References: <20230724165433.117645-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Use the SECTOR_SHIFT magic constant instead of the magic number. Signed-off-by: Christoph Hellwig Reviewed-by: Jinyoung Choi Reviewed-by: Johannes Thumshirn --- block/bio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/bio.c b/block/bio.c index 72488ecea47acf..445be4bdd99bdb 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1007,7 +1007,7 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED))) return 0; - if (((bio->bi_iter.bi_size + len) >> 9) > max_sectors) + if (((bio->bi_iter.bi_size + len) >> SECTOR_SHIFT) > max_sectors) return 0; if (bio->bi_vcnt > 0) { From patchwork Mon Jul 24 16:54:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13325027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CEDDC00528 for ; Mon, 24 Jul 2023 16:54:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231180AbjGXQyh (ORCPT ); Mon, 24 Jul 2023 12:54:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231213AbjGXQyf (ORCPT ); Mon, 24 Jul 2023 12:54:35 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C177610F7 for ; Mon, 24 Jul 2023 09:54:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=sUJrGMRt14c1CZmZUs2jOn1hnGR381PymfqIOp2Vw7I=; b=2oXihKJV/8rHUHFkmRdAzdEgvJ g8SVfBYuMYZRK3HUvhLhB8jnIL3f20371LEbbY6Pr0KC6TQ7IUFV7LQqaXO2za0qkFLZ4/QsXbI1H ihDgENIW+K0aB/DBbk2mKG+fkh5gtkxzlUnCcwoQImheIBLRbjxdUy9nqpKr9ZOB6YlqEo40yovNJ BgHfYlaGkvhQsU/6zl0cpC77OmBWYSyceP72e031xoDrFF+2w0atHrdk28rKtXv9ppLloVNJzHNUl 7sc+HJjNNH1gztqZxsxmKI9vD7nlcfhRD38UURR5TulCAeLSS/z/+BLlb+w2r3IioisKqw8XcsSaI be9MZwyw==; Received: from 67-207-104-238.static.wiline.com ([67.207.104.238] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qNypN-004vuB-39; Mon, 24 Jul 2023 16:54:34 +0000 From: Christoph Hellwig To: Jens Axboe Cc: Jinyoung Choi , linux-block@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 3/8] block: move the BIO_CLONED checks out of __bio_try_merge_page Date: Mon, 24 Jul 2023 09:54:28 -0700 Message-Id: <20230724165433.117645-4-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230724165433.117645-1-hch@lst.de> References: <20230724165433.117645-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org __bio_try_merge_page is a way too low-level helper to assert that the bio is not cloned. Move the check into bio_add_page and bio_iov_iter_get_pages instead, which are the high level entry points that should enforce this variant. bio_add_hw_page already this check, coverig the third (indirect) caller of __bio_try_merge_page. Signed-off-by: Christoph Hellwig Reviewed-by: Jinyoung Choi Reviewed-by: Johannes Thumshirn --- block/bio.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/block/bio.c b/block/bio.c index 445be4bdd99bdb..3ac72e60e7f11c 100644 --- a/block/bio.c +++ b/block/bio.c @@ -945,9 +945,6 @@ static inline bool page_is_mergeable(const struct bio_vec *bv, static bool __bio_try_merge_page(struct bio *bio, struct page *page, unsigned int len, unsigned int off, bool *same_page) { - if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED))) - return false; - if (bio->bi_vcnt > 0) { struct bio_vec *bv = &bio->bi_io_vec[bio->bi_vcnt - 1]; @@ -1127,6 +1124,9 @@ int bio_add_page(struct bio *bio, struct page *page, { bool same_page = false; + if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED))) + return 0; + if (!__bio_try_merge_page(bio, page, len, offset, &same_page)) { if (bio_full(bio, len)) return 0; @@ -1335,6 +1335,9 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) { int ret = 0; + if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED))) + return -EIO; + if (iov_iter_is_bvec(iter)) { bio_iov_bvec_set(bio, iter); iov_iter_advance(iter, bio->bi_iter.bi_size); From patchwork Mon Jul 24 16:54:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13325030 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 114A8C001DF for ; Mon, 24 Jul 2023 16:54:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231202AbjGXQyi (ORCPT ); Mon, 24 Jul 2023 12:54:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231230AbjGXQyg (ORCPT ); Mon, 24 Jul 2023 12:54:36 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 27F71B1 for ; Mon, 24 Jul 2023 09:54:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=vetIgRUHufwzeRnjuJRycXGyA7SLucCmRaJ2A15U1u8=; b=xSkP9s5cnMw2rm25qOatBP0QlI se9gm7sy3SBWozr3rlLVXieAGQnHKRnzID/YgcLLs/b1VLKHs5RCZDiZBT+H+xLxrfS/JSU9zGjbV 2idcuJfhVgMBqd7wAav2OynKlQ+ERyubjCWToA1xvVnZLKCn5Y61e1iQofYI4yV8KiGnxv27Om1Lb 97/CwqLkAxINcCaJbp1fiO2V3N/NOftuJXkdkCbVSD9EI/C2N6zEHDSk5UjAGnu1Fo0CXEXTN+KEX z2oICr2dnUENcr8QgixWbjXQxS12gfWlnWOKMM5q47Pe9ffs+ior/+l+GPrsbs2LSVdtnur37gVPz HrHfSk4g==; Received: from 67-207-104-238.static.wiline.com ([67.207.104.238] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qNypO-004vuJ-0a; Mon, 24 Jul 2023 16:54:34 +0000 From: Christoph Hellwig To: Jens Axboe Cc: Jinyoung Choi , linux-block@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 4/8] block: move the bi_vcnt check out of __bio_try_merge_page Date: Mon, 24 Jul 2023 09:54:29 -0700 Message-Id: <20230724165433.117645-5-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230724165433.117645-1-hch@lst.de> References: <20230724165433.117645-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Move the bi_vcnt out of __bio_try_merge_page and into the two callers that don't already have it in preparation for additional changes to __bio_try_merge_page. Signed-off-by: Christoph Hellwig Reviewed-by: Jinyoung Choi Reviewed-by: Johannes Thumshirn --- block/bio.c | 45 ++++++++++++++++++++++----------------------- 1 file changed, 22 insertions(+), 23 deletions(-) diff --git a/block/bio.c b/block/bio.c index 3ac72e60e7f11c..3dcbe98580dcef 100644 --- a/block/bio.c +++ b/block/bio.c @@ -945,20 +945,17 @@ static inline bool page_is_mergeable(const struct bio_vec *bv, static bool __bio_try_merge_page(struct bio *bio, struct page *page, unsigned int len, unsigned int off, bool *same_page) { - if (bio->bi_vcnt > 0) { - struct bio_vec *bv = &bio->bi_io_vec[bio->bi_vcnt - 1]; - - if (page_is_mergeable(bv, page, len, off, same_page)) { - if (bio->bi_iter.bi_size > UINT_MAX - len) { - *same_page = false; - return false; - } - bv->bv_len += len; - bio->bi_iter.bi_size += len; - return true; - } + struct bio_vec *bv = &bio->bi_io_vec[bio->bi_vcnt - 1]; + + if (!page_is_mergeable(bv, page, len, off, same_page)) + return false; + if (bio->bi_iter.bi_size > UINT_MAX - len) { + *same_page = false; + return false; } - return false; + bv->bv_len += len; + bio->bi_iter.bi_size += len; + return true; } /* @@ -1127,11 +1124,13 @@ int bio_add_page(struct bio *bio, struct page *page, if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED))) return 0; - if (!__bio_try_merge_page(bio, page, len, offset, &same_page)) { - if (bio_full(bio, len)) - return 0; - __bio_add_page(bio, page, len, offset); - } + if (bio->bi_vcnt > 0 && + __bio_try_merge_page(bio, page, len, offset, &same_page)) + return len; + + if (bio_full(bio, len)) + return 0; + __bio_add_page(bio, page, len, offset); return len; } EXPORT_SYMBOL(bio_add_page); @@ -1205,13 +1204,13 @@ static int bio_iov_add_page(struct bio *bio, struct page *page, { bool same_page = false; - if (!__bio_try_merge_page(bio, page, len, offset, &same_page)) { - __bio_add_page(bio, page, len, offset); + if (bio->bi_vcnt > 0 && + __bio_try_merge_page(bio, page, len, offset, &same_page)) { + if (same_page) + bio_release_page(bio, page); return 0; } - - if (same_page) - bio_release_page(bio, page); + __bio_add_page(bio, page, len, offset); return 0; } From patchwork Mon Jul 24 16:54:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13325029 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82588C04A94 for ; Mon, 24 Jul 2023 16:54:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231230AbjGXQyj (ORCPT ); Mon, 24 Jul 2023 12:54:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231247AbjGXQyg (ORCPT ); Mon, 24 Jul 2023 12:54:36 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5980DB7 for ; Mon, 24 Jul 2023 09:54:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=h8zEt1XgMZMmMv7pSGycCAsQPgI06zFL2S70tYLRonI=; b=cuZgzE+A90s6er4YKGdICwMPt9 E6qe+5XLgr4mAKvWDTuacVuy9DidlF9hS6u4pSiZg7Jt7GZkieWI8ej+kOk7wIuUesSip6yxZPZr1 XG+A0neKHWNwMOk99AOLefrft5Pc7npGHwLCVpnuMTF4VtqNntKpubDo8LauCF77dw090+4SfnwIt LZzYK6U6jEV5EXs/5UejkoOOyhLtONyp04rZXuF3oGiRNBxAsLlhHBxuz3JM/IhCit95eiRkgCvYZ g0SZG3pdFCxK77L/2cNAcU7iBaDX1q997RehOiLvl1AR3pY/ce5o73jib5RFltkWC06gaO7cmYQT+ SI5TXNAw==; Received: from 67-207-104-238.static.wiline.com ([67.207.104.238] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qNypO-004vuS-1D; Mon, 24 Jul 2023 16:54:34 +0000 From: Christoph Hellwig To: Jens Axboe Cc: Jinyoung Choi , linux-block@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 5/8] block: move the bi_size overflow check in __bio_try_merge_page Date: Mon, 24 Jul 2023 09:54:30 -0700 Message-Id: <20230724165433.117645-6-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230724165433.117645-1-hch@lst.de> References: <20230724165433.117645-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Checking for availability in bi_size in a function that attempts to merge into an existing segment is a bit odd, as the limit also applies when adding a new segment. This code works fine as we always call __bio_try_merge_page, but contributes to sub-optimal calling conventions and doesn't lead to clear code. Move it to two of the callers instead, the third one already has a more strict check that includes max_hw_segments anyway. Signed-off-by: Christoph Hellwig Reviewed-by: Jinyoung Choi Reviewed-by: Johannes Thumshirn --- block/bio.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/bio.c b/block/bio.c index 3dcbe98580dcef..17f57fd2cff2f1 100644 --- a/block/bio.c +++ b/block/bio.c @@ -949,10 +949,6 @@ static bool __bio_try_merge_page(struct bio *bio, struct page *page, if (!page_is_mergeable(bv, page, len, off, same_page)) return false; - if (bio->bi_iter.bi_size > UINT_MAX - len) { - *same_page = false; - return false; - } bv->bv_len += len; bio->bi_iter.bi_size += len; return true; @@ -1123,6 +1119,8 @@ int bio_add_page(struct bio *bio, struct page *page, if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED))) return 0; + if (bio->bi_iter.bi_size > UINT_MAX - len) + return 0; if (bio->bi_vcnt > 0 && __bio_try_merge_page(bio, page, len, offset, &same_page)) @@ -1204,6 +1202,9 @@ static int bio_iov_add_page(struct bio *bio, struct page *page, { bool same_page = false; + if (WARN_ON_ONCE(bio->bi_iter.bi_size > UINT_MAX - len)) + return -EIO; + if (bio->bi_vcnt > 0 && __bio_try_merge_page(bio, page, len, offset, &same_page)) { if (same_page) From patchwork Mon Jul 24 16:54:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13325028 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAF34C41513 for ; Mon, 24 Jul 2023 16:54:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231221AbjGXQyi (ORCPT ); Mon, 24 Jul 2023 12:54:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231231AbjGXQyg (ORCPT ); Mon, 24 Jul 2023 12:54:36 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 77088102 for ; Mon, 24 Jul 2023 09:54:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=vsmlaB3gAZ5AWEXt1JWDSCXwf4hG6CBi27SCZNx8+W8=; b=yhLOLGW3H705wbWExbRIjFFArQ FXOvNWn4HSqkQAFBt4QXI5uXUHiHQ8gPzzBZcgv87Vc0budcu3njdEEfq4Wp8BjzuVlE/ZX23XEVb 8q7AP1pZfF54jWoHdE0CngiEdQYxrHaqG5LouXeBAuhhnAXKtRK93qbkSOzsm9J6wIb8ubUngvsO3 Edw+ym5Qbw+kAjcaXeNBSkludx8lsQpmjhDQA0B8dbbwImbrvXz0Z4ulljNSrnkpFzpq7ctKLq+Uu vprVgoqQttqNaVH/4Z5Tkt9SK5sMQqumQ1XkB3LRtGzXHVPG17kl9yntApHy8auUI4gLrq4oVCDov +2aNZ4AQ==; Received: from 67-207-104-238.static.wiline.com ([67.207.104.238] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qNypO-004vuU-1q; Mon, 24 Jul 2023 16:54:34 +0000 From: Christoph Hellwig To: Jens Axboe Cc: Jinyoung Choi , linux-block@vger.kernel.org Subject: [PATCH 6/8] block: downgrade a bio_full call in bio_add_page Date: Mon, 24 Jul 2023 09:54:31 -0700 Message-Id: <20230724165433.117645-7-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230724165433.117645-1-hch@lst.de> References: <20230724165433.117645-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org bio_add_page already checks that there is space in bi_size a little earlier. So after we failed to add to an existing segment, just check that there is another one available instead of duplicating the bi_size check. Signed-off-by: Christoph Hellwig Reviewed-by: Jinyoung Choi --- block/bio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/bio.c b/block/bio.c index 17f57fd2cff2f1..d8e0e8de8cf4f6 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1126,7 +1126,7 @@ int bio_add_page(struct bio *bio, struct page *page, __bio_try_merge_page(bio, page, len, offset, &same_page)) return len; - if (bio_full(bio, len)) + if (bio->bi_vcnt >= bio->bi_max_vecs) return 0; __bio_add_page(bio, page, len, offset); return len; From patchwork Mon Jul 24 16:54:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13325032 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC834C04E69 for ; Mon, 24 Jul 2023 16:54:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231247AbjGXQyj (ORCPT ); Mon, 24 Jul 2023 12:54:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231250AbjGXQyh (ORCPT ); Mon, 24 Jul 2023 12:54:37 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38A6310F4 for ; Mon, 24 Jul 2023 09:54:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=jxj6RABkTFnEGOc2Iav3sSoTjFYz8cIIVXLuIca+dqc=; b=ed5kr1AfAQZgzGovJopAlUOnsr Qpcfez44A2faZvauFx9TqU8YFBrqu1FYIchBTmPZz1eEWtBmXqyxUt2uk75OYPu5rUF00EVAvdRoZ imqJMtihIp4SyvoWmZXjsL8AEpwNgxk0QdH4umBYfjWJ+9Nihr2KCMwRB0HZrZ92BTnQk4SDwIfty rW3ZsudQ3pofb1ZdIgkutvktzAYy/+4qQF/3t/oYKdyvFdzkZJS7BtfmtmprJQ1whvs1qVxjEQXtv srAZ2UleQEwA06J59v9vsVZxdOb8hrbcYj45u+63uqJ4O+Aq8Ug3DgrAUS+6q4MXvQsDSTdttIBAb LAe1yjgg==; Received: from 67-207-104-238.static.wiline.com ([67.207.104.238] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qNypO-004vv3-2b; Mon, 24 Jul 2023 16:54:34 +0000 From: Christoph Hellwig To: Jens Axboe Cc: Jinyoung Choi , linux-block@vger.kernel.org Subject: [PATCH 7/8] block: move the bi_size update out of __bio_try_merge_page Date: Mon, 24 Jul 2023 09:54:32 -0700 Message-Id: <20230724165433.117645-8-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230724165433.117645-1-hch@lst.de> References: <20230724165433.117645-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org The update of bi_size is the only thing in __bio_try_merge_page that needs a bio. Move it to the callers, and merge __bio_try_merge_page and page_is_mergeable into a single bvec_try_merge_page that only takes the current bvec instead of a full bio. This will allow reusing this function for supporting multi-page integrity payload bvecs. Signed-off-by: Christoph Hellwig Reviewed-by: Jinyoung Choi --- block/bio.c | 57 +++++++++++++++++++---------------------------------- 1 file changed, 20 insertions(+), 37 deletions(-) diff --git a/block/bio.c b/block/bio.c index d8e0e8de8cf4f6..23b7a001b5005d 100644 --- a/block/bio.c +++ b/block/bio.c @@ -903,9 +903,8 @@ static inline bool bio_full(struct bio *bio, unsigned len) return false; } -static inline bool page_is_mergeable(const struct bio_vec *bv, - struct page *page, unsigned int len, unsigned int off, - bool *same_page) +static bool bvec_try_merge_page(struct bio_vec *bv, struct page *page, + unsigned int len, unsigned int off, bool *same_page) { size_t bv_end = bv->bv_offset + bv->bv_len; phys_addr_t vec_end_addr = page_to_phys(bv->bv_page) + bv_end - 1; @@ -919,38 +918,14 @@ static inline bool page_is_mergeable(const struct bio_vec *bv, return false; *same_page = ((vec_end_addr & PAGE_MASK) == page_addr); - if (*same_page) - return true; - else if (IS_ENABLED(CONFIG_KMSAN)) - return false; - return (bv->bv_page + bv_end / PAGE_SIZE) == (page + off / PAGE_SIZE); -} - -/** - * __bio_try_merge_page - try appending data to an existing bvec. - * @bio: destination bio - * @page: start page to add - * @len: length of the data to add - * @off: offset of the data relative to @page - * @same_page: return if the segment has been merged inside the same page - * - * Try to add the data at @page + @off to the last bvec of @bio. This is a - * useful optimisation for file systems with a block size smaller than the - * page size. - * - * Warn if (@len, @off) crosses pages in case that @same_page is true. - * - * Return %true on success or %false on failure. - */ -static bool __bio_try_merge_page(struct bio *bio, struct page *page, - unsigned int len, unsigned int off, bool *same_page) -{ - struct bio_vec *bv = &bio->bi_io_vec[bio->bi_vcnt - 1]; + if (!*same_page) { + if (IS_ENABLED(CONFIG_KMSAN)) + return false; + if (bv->bv_page + bv_end / PAGE_SIZE != page + off / PAGE_SIZE) + return false; + } - if (!page_is_mergeable(bv, page, len, off, same_page)) - return false; bv->bv_len += len; - bio->bi_iter.bi_size += len; return true; } @@ -972,7 +947,7 @@ static bool bio_try_merge_hw_seg(struct request_queue *q, struct bio *bio, return false; if (bv->bv_len + len > queue_max_segment_size(q)) return false; - return __bio_try_merge_page(bio, page, len, offset, same_page); + return bvec_try_merge_page(bv, page, len, offset, same_page); } /** @@ -1001,8 +976,11 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, return 0; if (bio->bi_vcnt > 0) { - if (bio_try_merge_hw_seg(q, bio, page, len, offset, same_page)) + if (bio_try_merge_hw_seg(q, bio, page, len, offset, + same_page)) { + bio->bi_iter.bi_size += len; return len; + } if (bio->bi_vcnt >= min(bio->bi_max_vecs, queue_max_segments(q))) @@ -1123,8 +1101,11 @@ int bio_add_page(struct bio *bio, struct page *page, return 0; if (bio->bi_vcnt > 0 && - __bio_try_merge_page(bio, page, len, offset, &same_page)) + bvec_try_merge_page(&bio->bi_io_vec[bio->bi_vcnt - 1], + page, len, offset, &same_page)) { + bio->bi_iter.bi_size += len; return len; + } if (bio->bi_vcnt >= bio->bi_max_vecs) return 0; @@ -1206,7 +1187,9 @@ static int bio_iov_add_page(struct bio *bio, struct page *page, return -EIO; if (bio->bi_vcnt > 0 && - __bio_try_merge_page(bio, page, len, offset, &same_page)) { + bvec_try_merge_page(&bio->bi_io_vec[bio->bi_vcnt - 1], + page, len, offset, &same_page)) { + bio->bi_iter.bi_size += len; if (same_page) bio_release_page(bio, page); return 0; From patchwork Mon Jul 24 16:54:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13325031 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77AC2C0015E for ; Mon, 24 Jul 2023 16:54:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231292AbjGXQyk (ORCPT ); Mon, 24 Jul 2023 12:54:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231228AbjGXQyj (ORCPT ); Mon, 24 Jul 2023 12:54:39 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4ED0F10F5 for ; Mon, 24 Jul 2023 09:54:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=+wZ6RZIImLBGbn4gu+hmurcknkaqcxp6cNAZ4SE8xjs=; b=bwaSONhL3b8YYZoKbwAnmCQFB7 bHxzp2mzcJauWLwMxetkGm8b0fUrNX6RRgW+bA+vBlUJUlHH23p8K2lB66gk+t72T3Gy2I5DRQgjw nTeJukBiqZ1HOVIQF2mTiACya0e68CT/sStfUAbb1WERFG1xdue/jyiTGfT3ziq3ZhcCjigxHiI4l /AHbJWGqVFmrltEUVEoOKSUC75LpRAWNKS77zeCv+4OUhC4y1MnuTJ4AgpqPsLJga4OMoMHYVr+57 xjqTWJIH/Ejok6W/+NC1xnatVWxy9tlB/1z8eDGBNYs5XkEew15BkJZxoulbp5ch15WRLsNJUX1Bh SBu4JmMQ==; Received: from 67-207-104-238.static.wiline.com ([67.207.104.238] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qNypP-004vvB-0B; Mon, 24 Jul 2023 16:54:37 +0000 From: Christoph Hellwig To: Jens Axboe Cc: Jinyoung Choi , linux-block@vger.kernel.org Subject: [PATCH 8/8] block: don't pass a bio to bio_try_merge_hw_seg Date: Mon, 24 Jul 2023 09:54:33 -0700 Message-Id: <20230724165433.117645-9-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230724165433.117645-1-hch@lst.de> References: <20230724165433.117645-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org There is no good reason to pass the bio to bio_try_merge_hw_seg. Just pass the current bvec and rename the function to bvec_try_merge_hw_page. This will allow reusing this function for supporting multi-page integrity payload bvecs. Signed-off-by: Christoph Hellwig Reviewed-by: Jinyoung Choi --- block/bio.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/block/bio.c b/block/bio.c index 23b7a001b5005d..c92dda962449b0 100644 --- a/block/bio.c +++ b/block/bio.c @@ -934,11 +934,10 @@ static bool bvec_try_merge_page(struct bio_vec *bv, struct page *page, * size limit. This is not for normal read/write bios, but for passthrough * or Zone Append operations that we can't split. */ -static bool bio_try_merge_hw_seg(struct request_queue *q, struct bio *bio, - struct page *page, unsigned len, - unsigned offset, bool *same_page) +static bool bvec_try_merge_hw_page(struct request_queue *q, struct bio_vec *bv, + struct page *page, unsigned len, unsigned offset, + bool *same_page) { - struct bio_vec *bv = &bio->bi_io_vec[bio->bi_vcnt - 1]; unsigned long mask = queue_segment_boundary(q); phys_addr_t addr1 = page_to_phys(bv->bv_page) + bv->bv_offset; phys_addr_t addr2 = page_to_phys(page) + offset + len - 1; @@ -967,8 +966,6 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, struct page *page, unsigned int len, unsigned int offset, unsigned int max_sectors, bool *same_page) { - struct bio_vec *bvec; - if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED))) return 0; @@ -976,7 +973,9 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, return 0; if (bio->bi_vcnt > 0) { - if (bio_try_merge_hw_seg(q, bio, page, len, offset, + struct bio_vec *bv = &bio->bi_io_vec[bio->bi_vcnt - 1]; + + if (bvec_try_merge_hw_page(q, bv, page, len, offset, same_page)) { bio->bi_iter.bi_size += len; return len; @@ -990,8 +989,7 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, * If the queue doesn't support SG gaps and adding this segment * would create a gap, disallow it. */ - bvec = &bio->bi_io_vec[bio->bi_vcnt - 1]; - if (bvec_gap_to_prev(&q->limits, bvec, offset)) + if (bvec_gap_to_prev(&q->limits, bv, offset)) return 0; }