From patchwork Fri Jul 28 07:57:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinyoung Choi X-Patchwork-Id: 13331346 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CBC0C0015E for ; Fri, 28 Jul 2023 07:57:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234507AbjG1H5L (ORCPT ); Fri, 28 Jul 2023 03:57:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40906 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234516AbjG1H5K (ORCPT ); Fri, 28 Jul 2023 03:57:10 -0400 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 006A03585 for ; Fri, 28 Jul 2023 00:57:08 -0700 (PDT) Received: from epcas2p1.samsung.com (unknown [182.195.41.53]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20230728075705epoutp02aebabc8f7db6e8bf251bd059f58eb0fc~1_QLR5kSP2977629776epoutp02d for ; Fri, 28 Jul 2023 07:57:05 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20230728075705epoutp02aebabc8f7db6e8bf251bd059f58eb0fc~1_QLR5kSP2977629776epoutp02d DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1690531025; bh=fEBI3jFG7Yl1AWZ7NNNtJEMNDWilG4sULmX3Oko94OE=; h=Subject:Reply-To:From:To:In-Reply-To:Date:References:From; b=o/6Abm2zLDWk8oTBJ9gMmOehqk2OirOD1tmtGyvRKwm4noiKn2E1AEc0kG79ILPZR GBZ5gSj3JQqvBDJv6uo1Mz2YiayUa/cr4c/PfHeDSV1W1vBTLpiqUIbjpwM4KqPdaI 6i8xf81HDexicAEaA2cBWQlCprJsoVuvpMqh2QmY= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas2p2.samsung.com (KnoxPortal) with ESMTP id 20230728075705epcas2p2624e302aa77c663146ee97f7418e8739~1_QK4C-ot3006630066epcas2p2S; Fri, 28 Jul 2023 07:57:05 +0000 (GMT) Received: from epsmges2p1.samsung.com (unknown [182.195.36.90]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4RC0N04msqz4x9Q3; Fri, 28 Jul 2023 07:57:04 +0000 (GMT) X-AuditID: b6c32a45-e43e0a800000c2f9-7c-64c374d0b755 Received: from epcas2p3.samsung.com ( [182.195.41.55]) by epsmges2p1.samsung.com (Symantec Messaging Gateway) with SMTP id DA.B2.49913.0D473C46; Fri, 28 Jul 2023 16:57:04 +0900 (KST) Mime-Version: 1.0 Subject: [PATCH 1/2] block: make bvec_try_merge_hw_page() non-static Reply-To: j-young.choi@samsung.com Sender: Jinyoung Choi From: Jinyoung Choi To: "hch@lst.de" , "martin.petersen@oracle.com" , "axboe@kernel.dk" , "sagi@grimberg.me" , "kbusch@kernel.org" , "chaitanya.kulkarni@wdc.com" , "linux-block@vger.kernel.org" , "linux-kernel@vger.kernel.org" X-Priority: 3 X-Content-Kind-Code: NORMAL In-Reply-To: <20230728075537epcms2p194154023a4cdbe37c0346ef1102d1d63@epcms2p1> X-CPGS-Detection: blocking_info_exchange X-Drm-Type: N,general X-Msg-Generator: Mail X-Msg-Type: PERSONAL X-Reply-Demand: N Message-ID: <20230728075704epcms2p1fec48202cbda466411c44437cf788dea@epcms2p1> Date: Fri, 28 Jul 2023 16:57:04 +0900 X-CMS-MailID: 20230728075704epcms2p1fec48202cbda466411c44437cf788dea X-Sendblock-Type: AUTO_CONFIDENTIAL CMS-TYPE: 102P X-CPGSPASS: Y X-CPGSPASS: Y X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprJJsWRmVeSWpSXmKPExsWy7bCmue6FksMpBjc2qlisvtvPZjHr9msW i5eHNC1Wrj7KZDHp0DVGi723tC0u75rDZrH8+D8mi3Wv37M4cHqcv7eRxePy2VKPTas62Tx2 32xg8/j49BaLR9+WVYwenzfJebQf6GYK4IjKtslITUxJLVJIzUvOT8nMS7dV8g6Od443NTMw 1DW0tDBXUshLzE21VXLxCdB1y8wBOk5JoSwxpxQoFJBYXKykb2dTlF9akqqQkV9cYquUWpCS U2BeoFecmFtcmpeul5daYmVoYGBkClSYkJ3xZ+529oL73BVPP4Y3MD7m7GLk5JAQMJE4/PkQ axcjF4eQwA5GibcLdzN2MXJw8AoISvzdIQxSIyzgIjHz8jU2EFtIQEni3JpZjBBxA4mW220s IDabgJ7Ejue72UHmiAhMYJbYvOk2M8QCXokZ7U9ZIGxpie3Lt4I1cwr4Saxf+5IVIq4h8WNZ L1S9qMTN1W/ZYez3x+YzQtgiEq33zkLVCEo8+LkbKi4pcejQVzaQmyUE8iU2HAiECNdItP16 D1WuL3GtYyPYCbwCvhIXrhwGs1kEVCXe3+5jg6hxkVh07DLYWmYBeYntb+cwg4xkFtCUWL9L H2K6ssSRWywQFXwSHYf/ssM82LDxN1b2jnlPmCBa1SQWNRlBhGUkvh6ezz6BUWkWIphnIVk7 C2HtAkbmVYxiqQXFuempxUYFhvCITc7P3cQITqNarjsYJ7/9oHeIkYmD8RCjBAezkgjvqYBD KUK8KYmVValF+fFFpTmpxYcYTYEensgsJZqcD0zkeSXxhiaWBiZmZobmRqYG5krivPda56YI CaQnlqRmp6YWpBbB9DFxcEo1MJ1ZafXy0fPpR2e1vfrwX3ubu75S/3u1NezeQTPCVkjIVJeu c53pUfLp3VU5t7qA9O7r0blWn3YtNPir9OFk97Wo+bNNVzx0VPm6JMNSrCXzZLDSZdVKiytm 0zI2PDYXMomS3uLzqv/GC+XGyqy8vTzrbNcf7e7+8tDwua3pg9XXjG8wb4i6msTZsov3yK/1 +zUnu952rLj1Lldd4UaRVGR2ZNBqtq60eLbtVssY7qXm2Wd2TCttc11YOm9Xe3hPwlW/DcaH JqRmG2j8n5uYdzZPOp5X6sfjufUHPDtW6rRznJm0+NLvWMm5C67O/Jd84ueM9XcF3hy8l2dk v//slfmtLbLnCxtDIwV32GRq9iqxFGckGmoxFxUnAgBSZj6jLAQAAA== DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230728075537epcms2p194154023a4cdbe37c0346ef1102d1d63 References: <20230728075537epcms2p194154023a4cdbe37c0346ef1102d1d63@epcms2p1> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This will be used for multi-page configuration for integrity payload. Cc: Christoph Hellwig Cc: Martin K. Petersen Signed-off-by: Jinyoung Choi Reviewed-by: Christoph Hellwig --- block/bio.c | 2 +- block/blk.h | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/block/bio.c b/block/bio.c index c92dda962449..8d1533af7c60 100644 --- a/block/bio.c +++ b/block/bio.c @@ -934,7 +934,7 @@ static bool bvec_try_merge_page(struct bio_vec *bv, struct page *page, * size limit. This is not for normal read/write bios, but for passthrough * or Zone Append operations that we can't split. */ -static bool bvec_try_merge_hw_page(struct request_queue *q, struct bio_vec *bv, +bool bvec_try_merge_hw_page(struct request_queue *q, struct bio_vec *bv, struct page *page, unsigned len, unsigned offset, bool *same_page) { diff --git a/block/blk.h b/block/blk.h index 686712e13835..9d22ec3a53bc 100644 --- a/block/blk.h +++ b/block/blk.h @@ -75,6 +75,10 @@ struct bio_vec *bvec_alloc(mempool_t *pool, unsigned short *nr_vecs, gfp_t gfp_mask); void bvec_free(mempool_t *pool, struct bio_vec *bv, unsigned short nr_vecs); +bool bvec_try_merge_hw_page(struct request_queue *q, struct bio_vec *bv, + struct page *page, unsigned len, unsigned offset, + bool *same_page); + static inline bool biovec_phys_mergeable(struct request_queue *q, struct bio_vec *vec1, struct bio_vec *vec2) { From patchwork Fri Jul 28 07:57:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinyoung Choi X-Patchwork-Id: 13331347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFCB8C41513 for ; Fri, 28 Jul 2023 07:58:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234534AbjG1H6Q (ORCPT ); Fri, 28 Jul 2023 03:58:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234554AbjG1H56 (ORCPT ); Fri, 28 Jul 2023 03:57:58 -0400 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 749A23585 for ; Fri, 28 Jul 2023 00:57:56 -0700 (PDT) Received: from epcas2p4.samsung.com (unknown [182.195.41.56]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20230728075754epoutp0106ca83cdd765880516eedb2382fdcc1a~1_Q4nAb-V0322103221epoutp01U for ; Fri, 28 Jul 2023 07:57:54 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20230728075754epoutp0106ca83cdd765880516eedb2382fdcc1a~1_Q4nAb-V0322103221epoutp01U DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1690531074; bh=IDkiw2vf88CcoMUHaM6eAT6Gr0C3s0HY76p/6GA6/wU=; h=Subject:Reply-To:From:To:In-Reply-To:Date:References:From; b=cA2PBxKQX8jix6HceWy7+QLtcYJuYG7KA31as9cTtiRay7XO7GyjfNhbrsbuAn6+v /fqxhE7wMBaPMUEHdtnPUC7KEmWQzLt/tgEjh6YidyClfNlhRb0nUoFRPlHXix0zTE suEKFRjcdrNX4IdJ5PMzSw4x4kILS89WYg5An1LA= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas2p3.samsung.com (KnoxPortal) with ESMTP id 20230728075753epcas2p380d26b48e4c405d19a16f046682176d9~1_Q4J1Q8m1230112301epcas2p3w; Fri, 28 Jul 2023 07:57:53 +0000 (GMT) Received: from epsmgec2p1.samsung.com (unknown [182.195.36.70]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4RC0Nx2Ctzz4x9Q0; Fri, 28 Jul 2023 07:57:53 +0000 (GMT) X-AuditID: b6c32a43-557fb7000001d7ef-99-64c37501a6d9 Received: from epcas2p2.samsung.com ( [182.195.41.54]) by epsmgec2p1.samsung.com (Symantec Messaging Gateway) with SMTP id 41.A2.55279.10573C46; Fri, 28 Jul 2023 16:57:53 +0900 (KST) Mime-Version: 1.0 Subject: [PATCH 2/2] bio-integrity: create multi-page bvecs in bio_integrity_add_page() Reply-To: j-young.choi@samsung.com Sender: Jinyoung Choi From: Jinyoung Choi To: "hch@lst.de" , "martin.petersen@oracle.com" , "axboe@kernel.dk" , "sagi@grimberg.me" , "kbusch@kernel.org" , "chaitanya.kulkarni@wdc.com" , "linux-block@vger.kernel.org" , "linux-kernel@vger.kernel.org" X-Priority: 3 X-Content-Kind-Code: NORMAL In-Reply-To: <20230728075537epcms2p194154023a4cdbe37c0346ef1102d1d63@epcms2p1> X-CPGS-Detection: blocking_info_exchange X-Drm-Type: N,general X-Msg-Generator: Mail X-Msg-Type: PERSONAL X-Reply-Demand: N Message-ID: <20230728075753epcms2p7354d445f1888ab3942a16e26a13d5bbd@epcms2p7> Date: Fri, 28 Jul 2023 16:57:53 +0900 X-CMS-MailID: 20230728075753epcms2p7354d445f1888ab3942a16e26a13d5bbd X-Sendblock-Type: AUTO_CONFIDENTIAL CMS-TYPE: 102P X-CPGSPASS: Y X-CPGSPASS: Y X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprJJsWRmVeSWpSXmKPExsWy7bCmmS5j6eEUg2eL+C1W3+1ns5h1+zWL xctDmhYrVx9lsph06Bqjxd5b2haXd81hs1h+/B+TxbrX71kcOD3O39vI4nH5bKnHplWdbB67 bzaweXx8eovFo2/LKkaPz5vkPNoPdDMFcERl22SkJqakFimk5iXnp2TmpdsqeQfHO8ebmhkY 6hpaWpgrKeQl5qbaKrn4BOi6ZeYAHaekUJaYUwoUCkgsLlbSt7Mpyi8tSVXIyC8usVVKLUjJ KTAv0CtOzC0uzUvXy0stsTI0MDAyBSpMyM44NPEZY8EbyYrOFc4NjJtFuhg5OSQETCRO77jP 3sXIxSEksINRYsvfw0xdjBwcvAKCEn93CIPUCAtESjR0n2ECsYUElCTOrZnFCBE3kGi53cYC YrMJ6EnseL4bbI6IwARmic2bbjNDLOCVmNH+lAXClpbYvnwrWDOngJ/E+rUvWSHiGhI/lvVC 1YtK3Fz9lh3Gfn9sPiOELSLReu8sVI2gxIOfu6HikhKHDn1lA7lZQiBfYsOBQIhwjUTbr/dQ 5foS1zo2gp3AK+ArceUGxP0sAqoS/VOWQ9W4SHSd/w12DrOAvMT2t3OYQUYyC2hKrN+lDzFd WeLILRaICj6JjsN/2WEebNj4Gyt7x7wnTBCtahKLmowgwjISXw/PZ5/AqDQLEcyzkKydhbB2 ASPzKkax1ILi3PTUZKMCQ3jEJufnbmIEp1Et5x2MV+b/0zvEyMTBeIhRgoNZSYT3VMChFCHe lMTKqtSi/Pii0pzU4kOMpkAPT2SWEk3OBybyvJJ4QxNLAxMzM0NzI1MDcyVx3nutc1OEBNIT S1KzU1MLUotg+pg4OKUamPL/PJpvoPUrb+p677UfraR1lhTIzN52T85EjiU0pH9mfTX7hdnf FucJfDniVyioeudXycUGrzs9WWXPbr2onnpRN9xM67L6ZNmUa9bMciFHvrp+/ZmacdD9sn7o 7eM5uhMe9Ji+k676tfLqeibf/jKnM+nWf7/x2T2/cMbp2dQ4TqP7j+fMe+5/5Kra78nTV9t6 bD7P66jGyBn4zUBh7jFbxakTN9yKzvz152tgwINLmivcVFmPfP2o4B4d/EXRJNvz3d6uC0u/ fDj24fti7fX9b7YXFpZzpjrcVJB0CnTXtj3+38pjyxSh3MXxTp9MCvjXJOUUnNIUi7v1Zl1g zIqog4o7fMxuBtlGiPJvs1RiKc5INNRiLipOBAB+wR8KLAQAAA== DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230728075537epcms2p194154023a4cdbe37c0346ef1102d1d63 References: <20230728075537epcms2p194154023a4cdbe37c0346ef1102d1d63@epcms2p1> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Allow bio_integrity_add_page to create multi-page bvecs, just like the bio payloads. This simplifies adding larger payloads, and fixes support for non-tiny workloads with nvme, which stopped using scatterlist for metadata a while ago Cc: Christoph Hellwig Cc: Martin K. Petersen Fixes: 783b94bd9250 ("nvme-pci: do not build a scatterlist to map metadata") Signed-off-by: Jinyoung Choi Reviewed-by: Christoph Hellwig --- block/bio-integrity.c | 50 ++++++++++++++++++++++++------------------- 1 file changed, 28 insertions(+), 22 deletions(-) diff --git a/block/bio-integrity.c b/block/bio-integrity.c index 045553a164e0..ae054c6a06cb 100644 --- a/block/bio-integrity.c +++ b/block/bio-integrity.c @@ -123,21 +123,38 @@ void bio_integrity_free(struct bio *bio) int bio_integrity_add_page(struct bio *bio, struct page *page, unsigned int len, unsigned int offset) { + struct request_queue *q = bdev_get_queue(bio->bi_bdev); struct bio_integrity_payload *bip = bio_integrity(bio); - if (bip->bip_vcnt >= bip->bip_max_vcnt) { - printk(KERN_ERR "%s: bip_vec full\n", __func__); + if (((bip->bip_iter.bi_size + len) >> SECTOR_SHIFT) > + queue_max_hw_sectors(q)) return 0; - } - if (bip->bip_vcnt && - bvec_gap_to_prev(&bdev_get_queue(bio->bi_bdev)->limits, - &bip->bip_vec[bip->bip_vcnt - 1], offset)) - return 0; + if (bip->bip_vcnt > 0) { + struct bio_vec *bv = &bip->bip_vec[bip->bip_vcnt - 1]; + bool same_page = false; + + if (bvec_try_merge_hw_page(q, bv, page, len, offset, + &same_page)) { + bip->bip_iter.bi_size += len; + return len; + } + + if (bip->bip_vcnt >= + min(bip->bip_max_vcnt, queue_max_integrity_segments(q))) + return 0; + + /* + * If the queue doesn't support SG gaps and adding this segment + * would create a gap, disallow it. + */ + if (bvec_gap_to_prev(&q->limits, bv, offset)) + return 0; + } bvec_set_page(&bip->bip_vec[bip->bip_vcnt], page, len, offset); bip->bip_vcnt++; - + bip->bip_iter.bi_size += len; return len; } EXPORT_SYMBOL(bio_integrity_add_page); @@ -244,7 +261,6 @@ bool bio_integrity_prep(struct bio *bio) } bip->bip_flags |= BIP_BLOCK_INTEGRITY; - bip->bip_iter.bi_size = len; bip_set_seed(bip, bio->bi_iter.bi_sector); if (bi->flags & BLK_INTEGRITY_IP_CHECKSUM) @@ -252,27 +268,18 @@ bool bio_integrity_prep(struct bio *bio) /* Map it */ offset = offset_in_page(buf); - for (i = 0 ; i < nr_pages ; i++) { - int ret; + for (i = 0; i < nr_pages && len > 0; i++) { bytes = PAGE_SIZE - offset; - if (len <= 0) - break; - if (bytes > len) bytes = len; - ret = bio_integrity_add_page(bio, virt_to_page(buf), - bytes, offset); - - if (ret == 0) { + if (bio_integrity_add_page(bio, virt_to_page(buf), + bytes, offset) < bytes) { printk(KERN_ERR "could not attach integrity payload\n"); goto err_end_io; } - if (ret < bytes) - break; - buf += bytes; len -= bytes; offset = 0; @@ -291,7 +298,6 @@ bool bio_integrity_prep(struct bio *bio) bio->bi_status = BLK_STS_RESOURCE; bio_endio(bio); return false; - } EXPORT_SYMBOL(bio_integrity_prep);