From patchwork Wed Sep 20 08:07:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392306 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 496DECE79AF for ; Wed, 20 Sep 2023 08:58:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233885AbjITI6Z (ORCPT ); Wed, 20 Sep 2023 04:58:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233871AbjITI6W (ORCPT ); Wed, 20 Sep 2023 04:58:22 -0400 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF84BF2 for ; Wed, 20 Sep 2023 01:58:13 -0700 (PDT) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20230920085812epoutp02e06cb42d34e1316182d9a4090f2a6553~Gj68tQEK_2629026290epoutp02k for ; Wed, 20 Sep 2023 08:58:12 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20230920085812epoutp02e06cb42d34e1316182d9a4090f2a6553~Gj68tQEK_2629026290epoutp02k DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200292; bh=Pnjs48GpM2fNI5vQ32r+zx38m0Ym/uFN2UPMQUh9Xn4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aiSVNHIDKf7BCpaQa9RNAVJVW1wKDG9eF94qpVZ2K8G8iqZMtGvZkSIAk1Y5MmKvx fT1tkBTAMJbc7n7P2EELbJylqckg5iBzINbBOin4ztWXhs8sn52/P/J0w5uXNU46L1 j4VFTtcUBhlqHiABCiozoin1TBavv9uxmlJHR74M= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20230920085811epcas5p282b9d9c0b16a7c30dff7c6d563b1b7dd~Gj68DxkQu2273322733epcas5p2G; Wed, 20 Sep 2023 08:58:11 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.183]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4RrC9Y6VxCz4x9Q1; Wed, 20 Sep 2023 08:58:09 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id CD.74.09949.124BA056; Wed, 20 Sep 2023 17:58:09 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20230920081425epcas5p407a3b1ee776082798adb92e8025699f1~GjUuhAu0T2321223212epcas5p40; Wed, 20 Sep 2023 08:14:25 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20230920081425epsmtrp226bf5e4a93a025c4257d97c0743b3497~GjUuf60FK0164101641epsmtrp2a; Wed, 20 Sep 2023 08:14:25 +0000 (GMT) X-AuditID: b6c32a49-98bff700000026dd-5e-650ab4212680 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 32.FE.08742.1E9AA056; Wed, 20 Sep 2023 17:14:25 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081422epsmtip135b046fe83b44edbfd4fed015cd575c5~GjUrI0Vcg2886728867epsmtip1k; Wed, 20 Sep 2023 08:14:21 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Kanchan Joshi , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 01/12] block: Introduce queue limits and sysfs for copy-offload support Date: Wed, 20 Sep 2023 13:37:38 +0530 Message-Id: <20230920080756.11919-2-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Ta1CUZRTu/b7l2wsufoLiyzrJtk3EJS6bsLyIVBZDH+QPZoia0RxY2W+A 2Bt7AaOyVSFAWyDRkoVAFEvAuIOLuMUl3AQNaQcIZgARqBkYQWQ1GkRiWSj/Pec55zzPOeed l4U7f8vksZLlGlolF0sFBIfR0uXl6ftSE4cOyFvwRbU9N3G08HiZgU4UrOCoejSfQLNdjwCa as8GyDRX7ICG21sxdOPiGQxVVndjqHv1AYHOdA4CND1gwJBpxAeVf1nBQDdMtxjIcr2EQGXf TzPR6SEjgX4wP8PQHwXTABmnjgPUslyGo5rZeQb6dWQX6lsxO7zFo1oNo0yqb6yeQVnuaKmG qlyCaqz4gmob1hHUpbxCB0p/co6gFqZHGNT8TwMElddUBajG3k+pxYbdVMPUAyza6WDKviRa LKFVfFqeoJAkyxPDBO/FxL0TFyQKEPoKQ1CwgC8Xy+gwQfiBaN+IZOnaNQT8NLFUu0ZFi9Vq gf8b+1QKrYbmJynUmjABrZRIlYFKP7VYptbKE/3ktGavMCDg9aC1wviUJGve58rssKOWQiOm A017TgE2C5KB8PeZx+AU4LCcyTYA6zqGGfbgEYCTq5aNzBMAB8/nOWy2PCmadrAnTAA2l7QT 9iALg/UXf8ROARaLIH1g7yrLxm8ndTisa7u0LoWTXTjsLr9P2KRcyMPwztJfmA0zyFfgUr1l 3YJL7oUT/RWETQiS/jB/fJuNZpOhcLFiiGkv2QZvFU0xbBgn3eHJ5mLcpg/Ja2y4YJ3A7KOG w4HhmY2xXeCMuYlpxzy4OGci7DgdVp69QtibMwE0DBmAPfEmzOrJx21D4KQXrL3ub6dfhOd6 ajC7sRPUL09teHGhsXQTvwyv1l7Y0HeDg38f39iFgh16jv1YeQBWG78CBYBveG4fw3P7GP53 vgDwKuBGK9WyRFodpBTK6fT/XjlBIWsA6z/EO9IIRu899OsEGAt0AsjCBdu5Mg8O7cyViD/J oFWKOJVWSqs7QdDavb/GeTsSFGtfTK6JEwaGBASKRKLAkD0ioWAndzbrO4kzmSjW0Ck0raRV m30Yi83TYVsdTGWLd397NbJS4Xa+QzPG3WKNcaozn20+nDK5oyZV8PTIsdxD2pgrL7QXZbL/ CWkfKU7sL2G4p2abjWUSLj7RveID1O69HzoOeI7Tjt+wbqfqdOzY9ys5XaLxgoctCUuvwWuW n11je/bn9rR4B98uP+Qu+1O/ag5a9jiSeG5rbYYfs2S3PiLn7RJT5GW+9ZfPovCrnBxltrfX 0fvvcvodo7ZIL0d9wAtodR/fv7Nw/t4uVZvVwzoa+FGaa5rE7+7BsczgeEV0qOSpy+rNTI5e Vq/1dj1RxystHZmv/rhPj0yNIRmmiKK6Z7GTnTlQVHjAEH4sNLsiKx00d3hmxzsKGOoksdAb V6nF/wJYbdxeqgQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrLIsWRmVeSWpSXmKPExsWy7bCSnO7DlVypBovaFSzWnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFnvfzWa1uHlgJ5PFnkWTmCxWrj7KZHH0/1s2i0mHrjFa PL06i8li7y1ti4VtS1gs9uw9yWJxedccNov5y56yW3Rf38Fmsfz4PyaLGxOeMlrseNLIaLHt 93xmi3Wv37NYnLglbXH+73FWBymPnbPusnucv7eRxePy2VKPTas62Tw2L6n32H2zgc1jcd9k Vo/e5ndsHh+f3mLxeL/vKptH35ZVjB6bT1d7fN4k57HpyVumAL4oLpuU1JzMstQifbsErowv fbUF7bYVlyfvYGpg3GLcxcjJISFgIvFt5lPWLkYuDiGB3YwSK16sZIVISEos+3uEGcIWllj5 7zk7RFEzk8T9BZOAEhwcbALaEqf/c4DERQS6mCU6d75jAXGYBS4zS0xbeIQRpFtYIFpiRlsH mM0ioCrxY+NlsA28AlYSDy8uYQMZJCGgL9F/XxAkzClgLfF5yXV2EFsIqOT0hfnsEOWCEidn PmEBsZkF5CWat85mnsAoMAtJahaS1AJGplWMkqkFxbnpucWGBYZ5qeV6xYm5xaV56XrJ+bmb GMExrqW5g3H7qg96hxiZOBgPMUpwMCuJ8OaqcaUK8aYkVlalFuXHF5XmpBYfYpTmYFES5xV/ 0ZsiJJCeWJKanZpakFoEk2Xi4JRqYOLjU7zw7/eeOwa75SvSzibN35V+SeFcZd6rvW4hHYEJ 01uM669PmSqzMky8qUklSk8w2qq0vif3VP+LrzWRsR57Xr3dsS124kKXhUozJ8TGh9+YuenR ycwv/F61decVJ9pMnrki801f9f9Ax6lHZ/cfk/584ggHS/upaSFmSnvjN/yrnfnw2J0wcZ4c 7TeXJ/0/2pAV27FD9qbcXf/s7l2/vO30r/DP5A8PfJAdVu1Q+ary5++nzMduTFt83ayB18Nn yU5BWY/+Mw/X2JqYlbHsOZc79YlFTonwpvDFFSvzbpzTtY4/W2G8f164/UsRrdTQxftv/fQV 6ZY8e8d5QvqcmKxfJ/pLFtWdu5N9+4kSS3FGoqEWc1FxIgBsDchvYAMAAA== X-CMS-MailID: 20230920081425epcas5p407a3b1ee776082798adb92e8025699f1 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081425epcas5p407a3b1ee776082798adb92e8025699f1 References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add device limits as sysfs entries, - copy_max_bytes (RW) - copy_max_hw_bytes (RO) Above limits help to split the copy payload in block layer. copy_max_bytes: maximum total length of copy in single payload. copy_max_hw_bytes: Reflects the device supported maximum limit. Reviewed-by: Hannes Reinecke Reviewed-by: Luis Chamberlain Signed-off-by: Nitesh Shetty Signed-off-by: Kanchan Joshi Signed-off-by: Anuj Gupta --- Documentation/ABI/stable/sysfs-block | 23 ++++++++++++++++++ block/blk-settings.c | 24 +++++++++++++++++++ block/blk-sysfs.c | 36 ++++++++++++++++++++++++++++ include/linux/blkdev.h | 13 ++++++++++ 4 files changed, 96 insertions(+) diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stable/sysfs-block index 1fe9a553c37b..96ba701e57da 100644 --- a/Documentation/ABI/stable/sysfs-block +++ b/Documentation/ABI/stable/sysfs-block @@ -155,6 +155,29 @@ Description: last zone of the device which may be smaller. +What: /sys/block//queue/copy_max_bytes +Date: August 2023 +Contact: linux-block@vger.kernel.org +Description: + [RW] This is the maximum number of bytes that the block layer + will allow for a copy request. This is always smaller or + equal to the maximum size allowed by the hardware, indicated by + 'copy_max_hw_bytes'. An attempt to set a value higher than + 'copy_max_hw_bytes' will truncate this to 'copy_max_hw_bytes'. + Writing '0' to this file will disable offloading copies for this + device, instead copy is done via emulation. + + +What: /sys/block//queue/copy_max_hw_bytes +Date: August 2023 +Contact: linux-block@vger.kernel.org +Description: + [RO] This is the maximum number of bytes that the hardware + will allow for single data copy request. + A value of 0 means that the device does not support + copy offload. + + What: /sys/block//queue/crypto/ Date: February 2022 Contact: linux-block@vger.kernel.org diff --git a/block/blk-settings.c b/block/blk-settings.c index 0046b447268f..4441711ac364 100644 --- a/block/blk-settings.c +++ b/block/blk-settings.c @@ -59,6 +59,8 @@ void blk_set_default_limits(struct queue_limits *lim) lim->zoned = BLK_ZONED_NONE; lim->zone_write_granularity = 0; lim->dma_alignment = 511; + lim->max_copy_hw_sectors = 0; + lim->max_copy_sectors = 0; } /** @@ -82,6 +84,8 @@ void blk_set_stacking_limits(struct queue_limits *lim) lim->max_dev_sectors = UINT_MAX; lim->max_write_zeroes_sectors = UINT_MAX; lim->max_zone_append_sectors = UINT_MAX; + lim->max_copy_hw_sectors = UINT_MAX; + lim->max_copy_sectors = UINT_MAX; } EXPORT_SYMBOL(blk_set_stacking_limits); @@ -183,6 +187,22 @@ void blk_queue_max_discard_sectors(struct request_queue *q, } EXPORT_SYMBOL(blk_queue_max_discard_sectors); +/* + * blk_queue_max_copy_hw_sectors - set max sectors for a single copy payload + * @q: the request queue for the device + * @max_copy_sectors: maximum number of sectors to copy + */ +void blk_queue_max_copy_hw_sectors(struct request_queue *q, + unsigned int max_copy_sectors) +{ + if (max_copy_sectors > (BLK_COPY_MAX_BYTES >> SECTOR_SHIFT)) + max_copy_sectors = BLK_COPY_MAX_BYTES >> SECTOR_SHIFT; + + q->limits.max_copy_hw_sectors = max_copy_sectors; + q->limits.max_copy_sectors = max_copy_sectors; +} +EXPORT_SYMBOL_GPL(blk_queue_max_copy_hw_sectors); + /** * blk_queue_max_secure_erase_sectors - set max sectors for a secure erase * @q: the request queue for the device @@ -578,6 +598,10 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b, t->max_segment_size = min_not_zero(t->max_segment_size, b->max_segment_size); + t->max_copy_sectors = min(t->max_copy_sectors, b->max_copy_sectors); + t->max_copy_hw_sectors = min(t->max_copy_hw_sectors, + b->max_copy_hw_sectors); + t->misaligned |= b->misaligned; alignment = queue_limit_alignment_offset(b, start); diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 63e481262336..4840e21adefa 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -199,6 +199,37 @@ static ssize_t queue_discard_zeroes_data_show(struct request_queue *q, char *pag return queue_var_show(0, page); } +static ssize_t queue_copy_hw_max_show(struct request_queue *q, char *page) +{ + return sprintf(page, "%llu\n", (unsigned long long) + q->limits.max_copy_hw_sectors << SECTOR_SHIFT); +} + +static ssize_t queue_copy_max_show(struct request_queue *q, char *page) +{ + return sprintf(page, "%llu\n", (unsigned long long) + q->limits.max_copy_sectors << SECTOR_SHIFT); +} + +static ssize_t queue_copy_max_store(struct request_queue *q, const char *page, + size_t count) +{ + unsigned long max_copy; + ssize_t ret = queue_var_store(&max_copy, page, count); + + if (ret < 0) + return ret; + + if (max_copy & (queue_logical_block_size(q) - 1)) + return -EINVAL; + + max_copy >>= SECTOR_SHIFT; + q->limits.max_copy_sectors = min_t(unsigned int, max_copy, + q->limits.max_copy_hw_sectors); + + return count; +} + static ssize_t queue_write_same_max_show(struct request_queue *q, char *page) { return queue_var_show(0, page); @@ -517,6 +548,9 @@ QUEUE_RO_ENTRY(queue_nr_zones, "nr_zones"); QUEUE_RO_ENTRY(queue_max_open_zones, "max_open_zones"); QUEUE_RO_ENTRY(queue_max_active_zones, "max_active_zones"); +QUEUE_RO_ENTRY(queue_copy_hw_max, "copy_max_hw_bytes"); +QUEUE_RW_ENTRY(queue_copy_max, "copy_max_bytes"); + QUEUE_RW_ENTRY(queue_nomerges, "nomerges"); QUEUE_RW_ENTRY(queue_rq_affinity, "rq_affinity"); QUEUE_RW_ENTRY(queue_poll, "io_poll"); @@ -633,6 +667,8 @@ static struct attribute *queue_attrs[] = { &queue_discard_max_entry.attr, &queue_discard_max_hw_entry.attr, &queue_discard_zeroes_data_entry.attr, + &queue_copy_hw_max_entry.attr, + &queue_copy_max_entry.attr, &queue_write_same_max_entry.attr, &queue_write_zeroes_max_entry.attr, &queue_zone_append_max_entry.attr, diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index eef450f25982..7548f1685ee9 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -309,6 +309,9 @@ struct queue_limits { unsigned int discard_alignment; unsigned int zone_write_granularity; + unsigned int max_copy_hw_sectors; + unsigned int max_copy_sectors; + unsigned short max_segments; unsigned short max_integrity_segments; unsigned short max_discard_segments; @@ -893,6 +896,8 @@ extern void blk_queue_chunk_sectors(struct request_queue *, unsigned int); extern void blk_queue_max_segments(struct request_queue *, unsigned short); extern void blk_queue_max_discard_segments(struct request_queue *, unsigned short); +extern void blk_queue_max_copy_hw_sectors(struct request_queue *q, + unsigned int max_copy_sectors); void blk_queue_max_secure_erase_sectors(struct request_queue *q, unsigned int max_sectors); extern void blk_queue_max_segment_size(struct request_queue *, unsigned int); @@ -1211,6 +1216,14 @@ static inline unsigned int bdev_discard_granularity(struct block_device *bdev) return bdev_get_queue(bdev)->limits.discard_granularity; } +/* maximum copy offload length, this is set to 128MB based on current testing */ +#define BLK_COPY_MAX_BYTES (1 << 27) + +static inline unsigned int bdev_max_copy_sectors(struct block_device *bdev) +{ + return bdev_get_queue(bdev)->limits.max_copy_sectors; +} + static inline unsigned int bdev_max_secure_erase_sectors(struct block_device *bdev) { From patchwork Wed Sep 20 08:07:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B18CCE79B1 for ; Wed, 20 Sep 2023 08:58:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233951AbjITI6k (ORCPT ); Wed, 20 Sep 2023 04:58:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233912AbjITI6j (ORCPT ); Wed, 20 Sep 2023 04:58:39 -0400 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72EE3E4 for ; Wed, 20 Sep 2023 01:58:18 -0700 (PDT) Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20230920085816epoutp049058fc1e543beb4f2597682bfd1eabee~Gj7A99yAX1440014400epoutp04M for ; Wed, 20 Sep 2023 08:58:16 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20230920085816epoutp049058fc1e543beb4f2597682bfd1eabee~Gj7A99yAX1440014400epoutp04M DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200296; bh=A08fzSS5yF5x8HdE6Ju9IHdNmMRpxoEL3T7ujwB/X78=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YKTNdVK5gWkcM9zy7ju2woeguidYmpUwzB0HcsodNrsILb3g2fUOgbFder49MGFwN rC6sAs+izNya+FsMdGQzrspGeUquEXcbRBe3ueFlYcsdzPIzcZjBd4eNZ59LVhDx18 q3G8rxZISl8KBh5GSWSFeFqHnPYBKUjmc0FHY66g= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20230920085816epcas5p17b3562069a838dadb4a226894d1be337~Gj7AZHHlU3216432164epcas5p1n; Wed, 20 Sep 2023 08:58:16 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.183]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4RrC9f3y3Bz4x9Pt; Wed, 20 Sep 2023 08:58:14 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id 70.D1.19094.624BA056; Wed, 20 Sep 2023 17:58:14 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20230920081435epcas5p48932d9fc9ae0016b1edb2694d63fd885~GjU4ANEUu0845408454epcas5p4F; Wed, 20 Sep 2023 08:14:35 +0000 (GMT) Received: from epsmgmcp1.samsung.com (unknown [182.195.42.82]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20230920081435epsmtrp2167d7cef5c04ab05508bf1abc12a80ae~GjU3-EuLP0164101641epsmtrp2h; Wed, 20 Sep 2023 08:14:35 +0000 (GMT) X-AuditID: b6c32a50-39fff70000004a96-7d-650ab4260bf5 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgmcp1.samsung.com (Symantec Messaging Gateway) with SMTP id 1E.00.18916.BE9AA056; Wed, 20 Sep 2023 17:14:35 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081432epsmtip16afa5a70149a342ef8c197c1cbd17ffc~GjU0zuW3m0186201862epsmtip1G; Wed, 20 Sep 2023 08:14:32 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 02/12] Add infrastructure for copy offload in block and request layer. Date: Wed, 20 Sep 2023 13:37:39 +0530 Message-Id: <20230920080756.11919-3-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TbUxTVxjOuff2tiXrdimQndUJTXUk4AqtAjso4NyY3gRYGtAlyhxr4AYY /Vo/ZE4TytcMOL4EXCgqENgGLQGGyAABocgQCBLFipAw5wQ0EiGIE4Hh1lLY/Pc873mf5/04 eTk4P4ct4CSr9IxWJVeISBeitc/HR+zd4sJIlm3bUOPQbzha/GuNQBmF6ziyTBWQaK7vGUDT PWcA6povZ6GJnnYM1Vn6MXTOehegGZsJQ12Tu1DVdzUE6uwaJNBYxwUSVfw0w0Znx9tI9PPA KwzdK5wBqG06HaDWtQocNcwtEOjG5DY0uj7A+hDS7aYpNj36+y8EPTZioJvNOSR9uSaNvjph JOnq/GIWnZc5T9KLM5MEvdBtI+n8FjOgl5o96ebpp5iMdywlJImRJzBaIaOKVyckqxJDRREx cR/HBQZJpGJpMPpAJFTJlUyoKDxSJj6YrLDPLxKekCsM9pBMrtOJ/MNCtGqDnhEmqXX6UBGj SVBoAjR+OrlSZ1Al+qkY/V6pRLI70J74ZUpSX04hS2MM+aam+AxhBLckuYDLgVQAfHGvgMgF Lhw+1QlgfdMrtpM8A7B8fYTtyNog5t7YLUX96txmUjuABY8rSSfJxuBqnc3uxeGQ1C44/A/H EXenjDhsuloNHASn/sbguUdWlsPKjfocZqQvYQ5MUO/B7tvnWQ4xj9oLh7MTHRBS/rDgvqsj g0vtg0s14xsN8ShXOFg2TTgwTnnBzCvluMMeUj9w4YOKX3GnNhyaroudTbvBJwMtbCcWwKX5 LtKJU2FdSS3p1GYBaBo3AefDfpg9VLDhg1M+sLHD3xneDkuHGjBn3Tdh3to05ozzYNulLbwD 1jdWbvq/A+8up29iGlp6+oFzV/kAll3oZBUCoem1eUyvzWP6v3QlwM1AwGh0ykQmPlAjFauY 1P9+OV6tbAYbN+ErawOWpnU/K8A4wAogBxe585TeLgyflyA/+S2jVcdpDQpGZwWB9n0X4QKP eLX9qFT6OGlAsCQgKCgoIHhPkFT0Nm8u+2ICn0qU65kUhtEw2i0dxuEKjNjFhdDzfg9tvYWV BnhkJPyR72cvyN5o+ZGhaw8/yjzsOt4JDsV6Hj/h8dXLcG9ulGA04lDuzrlZ/aph5/p24mlD pJDPvS47/fxO/Xy/ZfHowdIDX5stoOt2zlCH263xCbbgjfQdxeTo1EssLyMtSul+P5KqTZUY Yk5N9D+I9iq9YhjzM4ftqXr3mmL4+aSHuChaiVxv3nyc23dpgIn9E/f8foXfmqY8dfQLsyTf JplEd2qDS4dDS6K0QGZ+ktVdzRafrRosWymKyTWenPWW/DGmWK6KCD+8mvLJaUXNsdlPV+IH mAOXf1zIeL9olF48/pZYOJLmldVovbF/n09uSdjuGBGhS5JLfXGtTv4vatcJMpwEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrFIsWRmVeSWpSXmKPExsWy7bCSnO7rlVypBp9+6VisP3WM2eLj198s Fk0T/jJbrL7bz2bx+vAnRosnB9oZLfa+m81qcfPATiaLlauPMllMOnSN0eLp1VlMFntvaVss bFvCYrFn70kWi8u75rBZzF/2lN2i+/oONovlx/8xWdyY8JTRYseTRkaLbb/nM1use/2exeLE LWmL83+PszpIeOycdZfd4/y9jSwel8+Wemxa1cnmsXlJvcfumw1sHov7JrN69Da/Y/P4+PQW i8f7fVfZPPq2rGL0+LxJzmPTk7dMAbxRXDYpqTmZZalF+nYJXBmHOyewFjTYVCyZ3M7SwHjR oIuRk0NCwERiza/X7F2MXBxCAtsZJW5O/8YKkZCUWPb3CDOELSyx8t9zdhBbSKCZSeLeT/Mu Rg4ONgFtidP/OUB6RQS6mCU6d75jAXGYQZxJ//+CNQgLREncf97BAmKzCKhK7Ls0jRWkmVfA SuJ0azqIKSGgL9F/XxCkglPAWuLzkutQq4AqLswHs3kFBCVOznwCNoVZQF6ieets5gmMArOQ pGYhSS1gZFrFKJpaUJybnptcYKhXnJhbXJqXrpecn7uJERy9WkE7GJet/6t3iJGJg/EQowQH s5IIb64aV6oQb0piZVVqUX58UWlOavEhRmkOFiVxXuWczhQhgfTEktTs1NSC1CKYLBMHp1QD k1tUwMPKoj2Tz7FqNJ6ZPYfb6+EBmZuGilXft2k2WHmXyTvtXbbnmPT2uZE5J1f9W5m42urx x5eP+Pfa7Uwx6L2YnePLE75Y71f+vza5RScnKX1eG13y/7vc4ha1Ko8A1Tl5Ce5FedzWm3Vm tnPZzC5P0rWfrnno0Q4V3d0Ozl5z9HQsvjx/5xH9RXxLXwW7SS6jXJPIYZYHRYnC+QeyJxSt 3Pu1jjlh9uFjBWp3WneXWu21U/Zw8fqrUqxSpekp9u/cf9bVoXJlp1NE9+wRlDqwXVpedKLl feb57k1zpi0RithswyW28GCQarPxiyjB4NBHM26ltfh/KCgLm3zKr2VCEq9VgvN/c0H1Ge+U WIozEg21mIuKEwGguSQkTQMAAA== X-CMS-MailID: 20230920081435epcas5p48932d9fc9ae0016b1edb2694d63fd885 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081435epcas5p48932d9fc9ae0016b1edb2694d63fd885 References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org We add two new opcode REQ_OP_COPY_SRC, REQ_OP_COPY_DST. Since copy is a composite operation involving src and dst sectors/lba, each needs to be represented by a separate bio to make it compatible with device mapper. We expect caller to take a plug and send bio with source information, followed by bio with destination information. Once the src bio arrives we form a request and wait for destination bio. Upon arrival of destination we merge these two bio's and send corresponding request down to device driver. Merging non copy offload bio is avoided by checking for copy specific opcodes in merge function. Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta --- block/blk-core.c | 7 +++++++ block/blk-merge.c | 41 +++++++++++++++++++++++++++++++++++++++ block/blk.h | 16 +++++++++++++++ block/elevator.h | 1 + include/linux/bio.h | 6 +----- include/linux/blk_types.h | 10 ++++++++++ 6 files changed, 76 insertions(+), 5 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 9d51e9894ece..33aadafdb7f9 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -121,6 +121,8 @@ static const char *const blk_op_name[] = { REQ_OP_NAME(ZONE_FINISH), REQ_OP_NAME(ZONE_APPEND), REQ_OP_NAME(WRITE_ZEROES), + REQ_OP_NAME(COPY_SRC), + REQ_OP_NAME(COPY_DST), REQ_OP_NAME(DRV_IN), REQ_OP_NAME(DRV_OUT), }; @@ -792,6 +794,11 @@ void submit_bio_noacct(struct bio *bio) if (!q->limits.max_write_zeroes_sectors) goto not_supported; break; + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: + if (!q->limits.max_copy_sectors) + goto not_supported; + break; default: break; } diff --git a/block/blk-merge.c b/block/blk-merge.c index 65e75efa9bd3..bcb55ba48107 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -158,6 +158,20 @@ static struct bio *bio_split_write_zeroes(struct bio *bio, return bio_split(bio, lim->max_write_zeroes_sectors, GFP_NOIO, bs); } +static struct bio *bio_split_copy(struct bio *bio, + const struct queue_limits *lim, + unsigned int *nsegs) +{ + *nsegs = 1; + if (bio_sectors(bio) <= lim->max_copy_sectors) + return NULL; + /* + * We don't support splitting for a copy bio. End it with EIO if + * splitting is required and return an error pointer. + */ + return ERR_PTR(-EIO); +} + /* * Return the maximum number of sectors from the start of a bio that may be * submitted as a single request to a block device. If enough sectors remain, @@ -366,6 +380,12 @@ struct bio *__bio_split_to_limits(struct bio *bio, case REQ_OP_WRITE_ZEROES: split = bio_split_write_zeroes(bio, lim, nr_segs, bs); break; + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: + split = bio_split_copy(bio, lim, nr_segs); + if (IS_ERR(split)) + return NULL; + break; default: split = bio_split_rw(bio, lim, nr_segs, bs, get_max_io_size(bio, lim) << SECTOR_SHIFT); @@ -922,6 +942,9 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio) if (!rq_mergeable(rq) || !bio_mergeable(bio)) return false; + if (blk_copy_offload_mergable(rq, bio)) + return true; + if (req_op(rq) != bio_op(bio)) return false; @@ -951,6 +974,8 @@ enum elv_merge blk_try_merge(struct request *rq, struct bio *bio) { if (blk_discard_mergable(rq)) return ELEVATOR_DISCARD_MERGE; + else if (blk_copy_offload_mergable(rq, bio)) + return ELEVATOR_COPY_OFFLOAD_MERGE; else if (blk_rq_pos(rq) + blk_rq_sectors(rq) == bio->bi_iter.bi_sector) return ELEVATOR_BACK_MERGE; else if (blk_rq_pos(rq) - bio_sectors(bio) == bio->bi_iter.bi_sector) @@ -1053,6 +1078,20 @@ static enum bio_merge_status bio_attempt_discard_merge(struct request_queue *q, return BIO_MERGE_FAILED; } +static enum bio_merge_status bio_attempt_copy_offload_merge(struct request *req, + struct bio *bio) +{ + if (req->__data_len != bio->bi_iter.bi_size) + return BIO_MERGE_FAILED; + + req->biotail->bi_next = bio; + req->biotail = bio; + req->nr_phys_segments++; + req->__data_len += bio->bi_iter.bi_size; + + return BIO_MERGE_OK; +} + static enum bio_merge_status blk_attempt_bio_merge(struct request_queue *q, struct request *rq, struct bio *bio, @@ -1073,6 +1112,8 @@ static enum bio_merge_status blk_attempt_bio_merge(struct request_queue *q, break; case ELEVATOR_DISCARD_MERGE: return bio_attempt_discard_merge(q, rq, bio); + case ELEVATOR_COPY_OFFLOAD_MERGE: + return bio_attempt_copy_offload_merge(rq, bio); default: return BIO_MERGE_NONE; } diff --git a/block/blk.h b/block/blk.h index 08a358bc0919..b0c17ad635a5 100644 --- a/block/blk.h +++ b/block/blk.h @@ -159,6 +159,20 @@ static inline bool blk_discard_mergable(struct request *req) return false; } +/* + * Copy offload sends a pair of bio with REQ_OP_COPY_SRC and REQ_OP_COPY_DST + * operation by taking a plug. + * Initially SRC bio is sent which forms a request and + * waits for DST bio to arrive. Once DST bio arrives + * we merge it and send request down to driver. + */ +static inline bool blk_copy_offload_mergable(struct request *req, + struct bio *bio) +{ + return (req_op(req) == REQ_OP_COPY_SRC && + bio_op(bio) == REQ_OP_COPY_DST); +} + static inline unsigned int blk_rq_get_max_segments(struct request *rq) { if (req_op(rq) == REQ_OP_DISCARD) @@ -300,6 +314,8 @@ static inline bool bio_may_exceed_limits(struct bio *bio, case REQ_OP_DISCARD: case REQ_OP_SECURE_ERASE: case REQ_OP_WRITE_ZEROES: + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: return true; /* non-trivial splitting decisions */ default: break; diff --git a/block/elevator.h b/block/elevator.h index 7ca3d7b6ed82..eec442bbf384 100644 --- a/block/elevator.h +++ b/block/elevator.h @@ -18,6 +18,7 @@ enum elv_merge { ELEVATOR_FRONT_MERGE = 1, ELEVATOR_BACK_MERGE = 2, ELEVATOR_DISCARD_MERGE = 3, + ELEVATOR_COPY_OFFLOAD_MERGE = 4, }; struct blk_mq_alloc_data; diff --git a/include/linux/bio.h b/include/linux/bio.h index 41d417ee1349..ed746738755a 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -53,11 +53,7 @@ static inline unsigned int bio_max_segs(unsigned int nr_segs) */ static inline bool bio_has_data(struct bio *bio) { - if (bio && - bio->bi_iter.bi_size && - bio_op(bio) != REQ_OP_DISCARD && - bio_op(bio) != REQ_OP_SECURE_ERASE && - bio_op(bio) != REQ_OP_WRITE_ZEROES) + if (bio && (bio_op(bio) == REQ_OP_READ || bio_op(bio) == REQ_OP_WRITE)) return true; return false; diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index d5c5e59ddbd2..78624e8f4ab4 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -393,6 +393,10 @@ enum req_op { /* reset all the zone present on the device */ REQ_OP_ZONE_RESET_ALL = (__force blk_opf_t)17, + /* copy offload dst and src operation */ + REQ_OP_COPY_SRC = (__force blk_opf_t)19, + REQ_OP_COPY_DST = (__force blk_opf_t)21, + /* Driver private requests */ REQ_OP_DRV_IN = (__force blk_opf_t)34, REQ_OP_DRV_OUT = (__force blk_opf_t)35, @@ -481,6 +485,12 @@ static inline bool op_is_write(blk_opf_t op) return !!(op & (__force blk_opf_t)1); } +static inline bool op_is_copy(blk_opf_t op) +{ + return ((op & REQ_OP_MASK) == REQ_OP_COPY_SRC || + (op & REQ_OP_MASK) == REQ_OP_COPY_DST); +} + /* * Check if the bio or request is one that needs special treatment in the * flush state machine. From patchwork Wed Sep 20 08:07:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392308 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 647B2CE79AD for ; Wed, 20 Sep 2023 08:58:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233938AbjITI6n (ORCPT ); Wed, 20 Sep 2023 04:58:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233941AbjITI6k (ORCPT ); Wed, 20 Sep 2023 04:58:40 -0400 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74EE1119 for ; Wed, 20 Sep 2023 01:58:22 -0700 (PDT) Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20230920085820epoutp02a40355151b0fe6b8988f5db0c12ad0cb~Gj7ExvD6k2774327743epoutp02B for ; Wed, 20 Sep 2023 08:58:20 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20230920085820epoutp02a40355151b0fe6b8988f5db0c12ad0cb~Gj7ExvD6k2774327743epoutp02B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200301; bh=Qu3cZ8sShKAAWOOA/tCP9AouB0CX13tqdGsd41wsXHM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GbcVDT53DYAdZ92CzvanyH/WT/nppiU9eIpkBrQaX8sC/9TTedfTrqH3AwC0nSJ7Y vaVA6YtuL79FWaJlxPHaNdbGdDRSPphLc+ViRdx5VcN+w413vfWnGvVoilv+QPKT1H YtVQRvdyXk3xSJm6gjqfOQE8RmdI44ZZfMHuqf04= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20230920085820epcas5p4fa78683738f322ddb817f1df58b6982a~Gj7D79qxP3151231512epcas5p4f; Wed, 20 Sep 2023 08:58:20 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.176]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4RrC9j6QMHz4x9Pt; Wed, 20 Sep 2023 08:58:17 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id FC.DA.09638.924BA056; Wed, 20 Sep 2023 17:58:17 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20230920081447epcas5p144e631c5b8c72acf64d38b04d6c2c925~GjVDAC7nW0572005720epcas5p1p; Wed, 20 Sep 2023 08:14:47 +0000 (GMT) Received: from epsmgmc1p1new.samsung.com (unknown [182.195.42.40]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20230920081447epsmtrp1c88fd41277906975f032cebc403bc6ca~GjVC_bZb12167621676epsmtrp1c; Wed, 20 Sep 2023 08:14:47 +0000 (GMT) X-AuditID: b6c32a4a-92df9700000025a6-6e-650ab429a537 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgmc1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 83.6C.08649.7F9AA056; Wed, 20 Sep 2023 17:14:47 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081444epsmtip1f712aa7ebd7aae27f666e8b72f0a70fe~GjU-ubyeY0186201862epsmtip1H; Wed, 20 Sep 2023 08:14:44 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 03/12] block: add copy offload support Date: Wed, 20 Sep 2023 13:37:40 +0530 Message-Id: <20230920080756.11919-4-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te1BUVRzHO/fu3t2lAS4P84AT4HWygHgssnhAUFHKOz0ZGvujrGWDG9Au u9veXUGaCiEoIAE1HFgIDYh4GBAPZ2EBBZINhNEGgUBRZHZT4pUQWoNIuyyU/33O9/f+nfnx cceTPFd+glzNqOQSGUXYcC70eHr5eDbbMP5LVUJU39+Lo/vLKxyUlr+Ko9qJPALN9CwCZLz0 JUAd88VcNHapFUPtZacwVF17GUOnukcAMg1rMdQx7o2+y6zgoPaOPg4aaish0NlKEw/ljOoI 9IPhMYZ+yzcBpDMeB+jCylkc1c0scNAv49vQ1VUDd78L3aqd4NFXb/3EoYcGNXRjTRZBN1V8 TuvHUgm6PPc0lz6RPk/Q903jHHqhc5igc5trAN105RN6qdGNbjTOYZF270hD4xlJLKPyYOQx itgEeVwY9epb4oNiUZC/0EcYjHZTHnJJIhNGRbwW6fNygsy8CMrjqESmMUuREpal/PaGqhQa NeMRr2DVYRSjjJUpA5W+rCSR1cjjfOWMOkTo7x8gMjtGS+Oz6w4pb+9LbhrRgVQwtSsbCPiQ DITTxh6uhR1JPYBj7W7ZwMbMiwC26yeB9fEAwN6Gu9hmxMzXuYTV0AFgweOODa8MDKbPmnjZ gM8nSG94ZY1v0Z3JVBw26MvXnXCyFIdNU4b1VE5kCGybXlkvziGfg+f/nuNZ2NasZy2nAUsi SPrBvNsOFllA7oFLFaMbLg6wr8jIsTBOusP0lmLckh+SxQJYvVzHtbYaAW/0b7btBP8wNPOs 7Aqn8zI3OAlWf1NFWIO/AFA7qgVWwz6Y0Z+HW5rASU9Y3+ZnlZ+FBf11mLWwHTyxYtzIbwt1 pZu8A56vP0dY2QWOPDy+wTQczPgZty47F8CqNbd84KF9Yh7tE/No/698DuA1wIVRsolxDCtS BsiZpP8+OUaR2AjWb8PrFR24M/mnbzfA+KAbQD5OOdsm7rRhHG1jJcdSGJVCrNLIGLYbiMz7 Pom7bolRmI9LrhYLA4P9A4OCggKDdwUJqa22MxnfxjqScRI1I2UYJaPajMP4AtdUzLn3DbvC l3CuYVCruznQ0leirCTsDlAthwuZ1rLDAVL6+QdpLyQbBkwT0U9562ZM28cXimb3Hxn7eLZV 2pXm7n5TwcTUhrcMdQ13OvV/JZIllHEpwtPebudHC7PhAW5bZ/U1Y1PlGrVojbHPzhrKXXz/ s/zySKctH0Qse9knsfnvRvUuvP1Mq/o6W/I0OR7/3sXSqgNR934UnKls2/Ni6D3uX9tDi/0S xDGuv2apRB9eji7sPBaylsUcPJpMnE55M0cebl9kKiX2wt15Odfn6i42iHd03SpYTY/+586N lNfv+n1vmnyERWUe+VRAZTx0+H3u0fyS6JrUIa2hSj+6bSB1+hrFYeMlQi9cxUr+BW8EKCWk BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA02Re0hTYRjG+c45nh2Xi+MU/MxYNbuIljYs+Miu0OUrgqLCQEJbelLTzXE2 S43KNro4y2tEbsa6iOEsLTXZNC1my9JCw7IUUqyJ1VBTG0XLVdOC/nt4fr/3+edlSHE7NY9J VWo4XilPl9JCqrFNKlnxrUrIrdT1SlBtxxMSTbjcFNIWTZOo+l0hjZxtkwA5Hp0DqGXM6IP6 HlkJ9OBGCYGqqu0EKrH1AjT82kCglv4IdP1sBYUetDyjUE9TOY1MlcMClP/GQqNb7R4CvS0a BsjiOA1Qo9tEohrnOIWe9oegrul2n43B2Gp4J8BdA/co3PMiE9eZ82hcX3EKN/fl0vhmQakP vqgbo/HEcD+Fx1tf07igwQxwfedxPFUnwXWOUWL33Djh2iQuPfUox0etPyhM0ddsUw1uyKrv tYBc8D5aD3wZyK6CzgsFtB4IGTHbDODEpbc+syAYVk4/JmdzAKzyjAhmJR0Bf3ZPEnrAMDQb ATt/Md4+kNWTMM86RnkPSPY2CRuuirw5gF0Dmz65Z0Ypdgm8/X1U4M2iP32eSwu8O5CNgoWD /t7al42BUxVvZhTxH6Wz2/RX94fPyhx/5xdA3X0jWQRYw3/I8B+6BggzCOZUakWyIlGmkim5 Y5FquUKdqUyOTMxQ1IGZX4eHWcCAyRNpAwQDbAAypDRQpFgq5MSiJHl2DsdnJPCZ6ZzaBkIY Shokkl0xJonZZLmGS+M4Fcf/owTjOy+XKNmkXWBuOOTU7T+xOt70qepkRvn8WhOzVVwQ88KY s1kk7pkT1vUjxq/tln9KWIdn15eoVoOxVZtduiNoIRs9pGvuupwpEfdZs0K2R2/U5o+fANER UvA9qeMjjH0+EDvfFbQXF8ske9yLXrZc31Ws1djLz6PEQFZkKnx4wBVq78Ej/k2+ZSFrD59y 8tzPOyMa14Rn4Toy4f6XnSOjCc2LlYWqpx9C7/L2scp0gdUTEFTstJ4sffixNzk+li0/GmVz Tx1vnHIsw34bnFrwypKmW2rOIJfn24/wfkMaghsP2/eKuXqmLxQbcvDolbmLur9+jiizC7fE rS/V8tVSSp0il4WTvFr+G73wPy1aAwAA X-CMS-MailID: 20230920081447epcas5p144e631c5b8c72acf64d38b04d6c2c925 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081447epcas5p144e631c5b8c72acf64d38b04d6c2c925 References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce blkdev_copy_offload to perform copy offload. Issue REQ_OP_COPY_SRC with source info along with taking a plug. This flows till request layer and waits for dst bio to arrive. Issue REQ_OP_COPY_DST with destination info and this bio reaches request layer and merges with src request. For any reason, if a request comes to the driver with only one of src/dst bio, we fail the copy offload. Larger copy will be divided, based on max_copy_sectors limit. Reviewed-by: Hannes Reinecke Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty --- block/blk-lib.c | 201 +++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 4 + 2 files changed, 205 insertions(+) diff --git a/block/blk-lib.c b/block/blk-lib.c index e59c3069e835..50d10fa3c4c5 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -10,6 +10,22 @@ #include "blk.h" +/* Keeps track of all outstanding copy IO */ +struct blkdev_copy_io { + atomic_t refcount; + ssize_t copied; + int status; + struct task_struct *waiter; + void (*endio)(void *private, int status, ssize_t copied); + void *private; +}; + +/* Keeps track of single outstanding copy offload IO */ +struct blkdev_copy_offload_io { + struct blkdev_copy_io *cio; + loff_t offset; +}; + static sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) { unsigned int discard_granularity = bdev_discard_granularity(bdev); @@ -115,6 +131,191 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector, } EXPORT_SYMBOL(blkdev_issue_discard); +static inline ssize_t blkdev_copy_sanity_check(struct block_device *bdev_in, + loff_t pos_in, + struct block_device *bdev_out, + loff_t pos_out, size_t len) +{ + unsigned int align = max(bdev_logical_block_size(bdev_out), + bdev_logical_block_size(bdev_in)) - 1; + + if ((pos_in & align) || (pos_out & align) || (len & align) || !len || + len >= BLK_COPY_MAX_BYTES) + return -EINVAL; + + return 0; +} + +static inline void blkdev_copy_endio(struct blkdev_copy_io *cio) +{ + if (cio->endio) { + cio->endio(cio->private, cio->status, cio->copied); + kfree(cio); + } else { + struct task_struct *waiter = cio->waiter; + + WRITE_ONCE(cio->waiter, NULL); + blk_wake_io_task(waiter); + } +} + +/* + * This must only be called once all bios have been issued so that the refcount + * can only decrease. This just waits for all bios to complete. + * Returns the length of bytes copied or error + */ +static ssize_t blkdev_copy_wait_io_completion(struct blkdev_copy_io *cio) +{ + ssize_t ret; + + for (;;) { + __set_current_state(TASK_UNINTERRUPTIBLE); + if (!READ_ONCE(cio->waiter)) + break; + blk_io_schedule(); + } + __set_current_state(TASK_RUNNING); + ret = cio->copied; + kfree(cio); + + return ret; +} + +static void blkdev_copy_offload_dst_endio(struct bio *bio) +{ + struct blkdev_copy_offload_io *offload_io = bio->bi_private; + struct blkdev_copy_io *cio = offload_io->cio; + + if (bio->bi_status) { + cio->copied = min_t(ssize_t, offload_io->offset, cio->copied); + if (!cio->status) + cio->status = blk_status_to_errno(bio->bi_status); + } + bio_put(bio); + + if (atomic_dec_and_test(&cio->refcount)) + blkdev_copy_endio(cio); +} + +/* + * @bdev: block device + * @pos_in: source offset + * @pos_out: destination offset + * @len: length in bytes to be copied + * @endio: endio function to be called on completion of copy operation, + * for synchronous operation this should be NULL + * @private: endio function will be called with this private data, + * for synchronous operation this should be NULL + * @gfp_mask: memory allocation flags (for bio_alloc) + * + * For synchronous operation returns the length of bytes copied or error + * For asynchronous operation returns -EIOCBQUEUED or error + * + * Description: + * Copy source offset to destination offset within block device, using + * device's native copy offload feature. + * We perform copy operation using 2 bio's. + * 1. We take a plug and send a REQ_OP_COPY_SRC bio along with source + * sector and length. Once this bio reaches request layer, we form a + * request and wait for dst bio to arrive. + * 2. We issue REQ_OP_COPY_DST bio along with destination sector, length. + * Once this bio reaches request layer and find a request with previously + * sent source info we merge the destination bio and return. + * 3. Release the plug and request is sent to driver + * This design works only for drivers with request queue. + */ +ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, + loff_t pos_out, size_t len, + void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp) +{ + struct blkdev_copy_io *cio; + struct blkdev_copy_offload_io *offload_io; + struct bio *src_bio, *dst_bio; + ssize_t rem, chunk, ret; + ssize_t max_copy_bytes = bdev_max_copy_sectors(bdev) << SECTOR_SHIFT; + struct blk_plug plug; + + if (!max_copy_bytes) + return -EOPNOTSUPP; + + ret = blkdev_copy_sanity_check(bdev, pos_in, bdev, pos_out, len); + if (ret) + return ret; + + cio = kzalloc(sizeof(*cio), GFP_KERNEL); + if (!cio) + return -ENOMEM; + atomic_set(&cio->refcount, 1); + cio->waiter = current; + cio->endio = endio; + cio->private = private; + + /* + * If there is a error, copied will be set to least successfully + * completed copied length + */ + cio->copied = len; + for (rem = len; rem > 0; rem -= chunk) { + chunk = min(rem, max_copy_bytes); + + offload_io = kzalloc(sizeof(*offload_io), GFP_KERNEL); + if (!offload_io) + goto err_free_cio; + offload_io->cio = cio; + /* + * For partial completion, we use offload_io->offset to truncate + * successful copy length + */ + offload_io->offset = len - rem; + + src_bio = bio_alloc(bdev, 0, REQ_OP_COPY_SRC, gfp); + if (!src_bio) + goto err_free_offload_io; + src_bio->bi_iter.bi_size = chunk; + src_bio->bi_iter.bi_sector = pos_in >> SECTOR_SHIFT; + + blk_start_plug(&plug); + dst_bio = blk_next_bio(src_bio, bdev, 0, REQ_OP_COPY_DST, gfp); + if (!dst_bio) + goto err_free_src_bio; + dst_bio->bi_iter.bi_size = chunk; + dst_bio->bi_iter.bi_sector = pos_out >> SECTOR_SHIFT; + dst_bio->bi_end_io = blkdev_copy_offload_dst_endio; + dst_bio->bi_private = offload_io; + + atomic_inc(&cio->refcount); + submit_bio(dst_bio); + blk_finish_plug(&plug); + pos_in += chunk; + pos_out += chunk; + } + + if (atomic_dec_and_test(&cio->refcount)) + blkdev_copy_endio(cio); + if (cio->endio) + return -EIOCBQUEUED; + + return blkdev_copy_wait_io_completion(cio); + +err_free_src_bio: + bio_put(src_bio); +err_free_offload_io: + kfree(offload_io); +err_free_cio: + cio->copied = min_t(ssize_t, cio->copied, (len - rem)); + cio->status = -ENOMEM; + if (rem == len) { + kfree(cio); + return cio->status; + } + if (cio->endio) + return cio->status; + + return blkdev_copy_wait_io_completion(cio); +} +EXPORT_SYMBOL_GPL(blkdev_copy_offload); + static int __blkdev_issue_write_zeroes(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned flags) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 7548f1685ee9..5405499bcf22 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1042,6 +1042,10 @@ int __blkdev_issue_discard(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop); int blkdev_issue_secure_erase(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp); +ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, + loff_t pos_out, size_t len, + void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp_mask); #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ From patchwork Wed Sep 20 08:07:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392330 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F8FECE79AB for ; Wed, 20 Sep 2023 08:59:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233954AbjITI7T (ORCPT ); Wed, 20 Sep 2023 04:59:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233887AbjITI6l (ORCPT ); Wed, 20 Sep 2023 04:58:41 -0400 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E6C1137 for ; Wed, 20 Sep 2023 01:58:25 -0700 (PDT) Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20230920085823epoutp03a184ba28151277cd91d9f9406c953044~Gj7HYc3lO1592315923epoutp03i for ; Wed, 20 Sep 2023 08:58:23 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20230920085823epoutp03a184ba28151277cd91d9f9406c953044~Gj7HYc3lO1592315923epoutp03i DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200303; bh=USZuQeTvMitlfy93ih628plbLc65lwCeOm5YsC+EdU0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sVfdM21jDAZhhsqHZ8zq7az9h/SpwJTRDfbOeDALfVKHR4PZTC9Be4EH9wEEoodDJ J1bWHY2zWUzmtGfBmTX7S12HEhlITDEJVLn0EU3+ozUlI7eZB5BGwebEkklYYdl/8N uVwmHmStuL596YiAzTtKUHmiy1XkNfgfu06iJxnE= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20230920085822epcas5p1f94e94baa1b8732aa448cf982905526d~Gj7GprOEC0802708027epcas5p1O; Wed, 20 Sep 2023 08:58:22 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.177]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4RrC9n144Cz4x9Q3; Wed, 20 Sep 2023 08:58:21 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 22.EA.09638.C24BA056; Wed, 20 Sep 2023 17:58:21 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20230920081458epcas5p3a3e12d8b5661b5d6f4420316630b02e1~GjVNKAH8J1823718237epcas5p3e; Wed, 20 Sep 2023 08:14:58 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20230920081458epsmtrp257233cb19de201dcd100c4f587068814~GjVNI2jX30164101641epsmtrp2y; Wed, 20 Sep 2023 08:14:58 +0000 (GMT) X-AuditID: b6c32a4a-92df9700000025a6-78-650ab42c5f42 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id F2.9E.08788.20AAA056; Wed, 20 Sep 2023 17:14:58 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081455epsmtip154f3f3e10d1122f2fe8c5d9c247e6cfb~GjVJ4fh3S0186201862epsmtip1O; Wed, 20 Sep 2023 08:14:54 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Vincent Fu , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 04/12] block: add emulation for copy Date: Wed, 20 Sep 2023 13:37:41 +0530 Message-Id: <20230920080756.11919-5-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Tf1BUVRTHue8tj4Vx7QkoFxppXVNDB1gQtouCOab0AscYoamxjF7sCxjY Zdvlh0QTGGACww9FIhcUVn4JGCghP10kCPnZLOGwGxAQuNsU1CKgZZHSsg/L/8753HPu937P ncPFbbOtnLiR0lhGLqWjBYQNp6nbZZera6MNI+yYxVD9wB0cLT5c4aDP8h7jqHYyl0Dz3UsA 6Ts/B0htLLJEY52tGKqu7cHQ+S4tQIZRJYbU43uQ6kw5B91S93PQ3bZiApVUGqxQlq6FQFW9 TzD0Q54BoBb9aYCaVkpwVDe/wEF948+je1lnAdI87rU86Ei1KietKM3UDQ5197s4qqEmg6C+ Lk+m2sdSCKosJ9+Syk41EtSiYZxDLXSMElROYw2glhucqQb971gQ70SUbwRDixk5n5GGxYgj peF+gsDg0FdDvUVCD1cPH/SygC+lJYyf4PDRIFf/yGjTEAT8eDo6zoSCaIVC4H7AVx4TF8vw I2IUsX4CRiaOlnnJ3BS0RBEnDXeTMrH7PIRCT29T4ftRESMXWzDZyKFT/6zex1LAQ+9MYM2F pBfsa23CMoEN15ZsBzDtGxXOJksAKqu+AmzyB4CPtEOmhGtu0eg+YrkawF/HlJZsko7Bur/u 4WtFBLkHDq5y17g9mYLD6+1l5ptwsgyH6qkSYk3cjkSwp62HsxZzyB1w9sEjM+eR+2Da6ATB qrnD3OlNa9ia3A+Xy3VWbMkm2H9Rb27FyRdg6s0i87MhWWYNF1Q5HNbcYTjUXESwsR2c6220 YmMnuGxUr/MEWH3hKsE2p5k865SAPXgFpg/kmt3gpAusb3Nn8VZYMFCHscIbYfaKHmM5D7Zc fhpvh9fqS9fvd4TaP0+ve6Fg6vWTa9iWzAGwoDYwD/CVz9hRPmNH+b9wKcBrgCMjU0jCGYW3 zFPKJPz3yWExkgZg3ovdAS1g5qf7bl0A44IuALm4wJ4n2WnD2PLEdOLHjDwmVB4XzSi6gLdp 3Odwp81hMabFksaGenj5CL1EIpGXz16Rh8CBN59+SWxLhtOxTBTDyBj50z6Ma+2Ugm3J+vBQ t9pYe8ni3V82Z2w7Wz+dOq7CnYtbNxgr0ex4/HCwJgHuKJxLLrxt99q1oJIQo3iqQqh+rnls +/GhU3E+NXPHfH6rnrmcdPwlQ+mRJeX3YR9EJTpGuwzLLEI6Fasi/7CAI1r7ManEQTPxorXW 3qtixVWgS6JvjPT3hTzRLZ+7KrR4HXv7QOGbJwe3OfpqxBkV/MlPeKrqzDlV4Vnf3s7Z80fn OrbSwY1XNhi+sN1ZP71L+/cilbT3258T7zTOE/Fch5n9+hL61oUz8ZUPnAdvX/myOd+vYMtE /o9OK5qAg+/4l97kit/TO6R6Vn26sUp17A3S8q3kFcNwYLGL+5TuhICjiKA9duNyBf0vqg9d LKAEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA02Rf0yMcRzHfb/P03PPHdeeruJbJjnKVlOdxHeJEeNZIczMDDl6VHNXuZMf +UOcXbpIQnYdYuXi2o4rUkd2LklIVqs4S4vOr5vqpGIpXLH577XX+/357LN9aEJUS/rSySl7 OUWKVCamBGRlrdhvDjQIuLDGbF9848kjAjsHhkl8NG+EwGUdpyjsqP0KcLclC+CaHp0bfmWp hvh6WR3E+dY2gO2thRDX2ILxFXUJie/VNJC4xXyBwkV6Ow/ntFdRuLR+FOKXeXaAq7qPAFw5 XERgo6OXxI9tU/G7nOMAN43Uuy3xYasLO3hs0xsTybY0prPlhmyKrSg5zN59lUmxxbln3NiT qh6KddptJNt7v5Vic28ZANtf7seWd3+Ba4WbBVEJnCx5H6cIXbxdkNSsrYJpzdEHfv7qg5lg IEIDaBox81BT+x4NENAi5i5A+eespAbw/3gfpB95SIyzJ7o++oE3XlJBVHIxk+capphg9PQX 7fJejIZA2dU9Y8MEYyJQ86P5LvZkMKoz1415kglAb799p1wsZCLRsdbX1PgRoehUp4dL85mF qL+knedi0Z/K0xdFvPG6B2rQdv9dPx2pbuuIPMAU/hcV/hddBtAAfLg0pTxRrpSkzU3h9oco pXJlekpiyM5UeTkY+3NQUBW4Z+gLsQJIAytANCH2EsoDBZxImCA9mMEpUuMV6TJOaQVTaVI8 RTjoOJkgYhKle7ndHJfGKf6lkOb7ZkJjwefwoQcrc4TuHtNO99Wv1Dom2mXq8+4a/vvU5NWS oe9tZt21GY6tk0ofms4Ndgzm2woO/XzuvWvLoC0mQGqOdJh2JMf0vMl16jpVxiO6G8XyyN7Z wKBesKDtcItz0ZUT0OTxoeuqUp+wjnPK7AMxk8LDNvmPBMfGxzYGWEIuwC/+SUMnuiRqL77b xMSGH6ROH1jqfjPes8XbdMlkz0rSRHQu9j499Gw9/PgpLLbggDovzrLOsmbFHX14JZ61MTrq WeV8MwyYE+2tndy8bcK+5cOrskLjjH5nrxa/1C6LuKmSdc14MnP7aEfUNEmj0Z6xdIlzw/61 4v6KhfKCOP8aMalMkkqCCIVS+hta4tVRVgMAAA== X-CMS-MailID: 20230920081458epcas5p3a3e12d8b5661b5d6f4420316630b02e1 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081458epcas5p3a3e12d8b5661b5d6f4420316630b02e1 References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org For the devices which does not support copy, copy emulation is added. It is required for in-kernel users like fabrics, where file descriptor is not available and hence they can't use copy_file_range. Copy-emulation is implemented by reading from source into memory and writing to the corresponding destination. At present in kernel user of emulation is fabrics. Signed-off-by: Nitesh Shetty Signed-off-by: Vincent Fu Signed-off-by: Anuj Gupta --- block/blk-lib.c | 223 +++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 4 + 2 files changed, 227 insertions(+) diff --git a/block/blk-lib.c b/block/blk-lib.c index 50d10fa3c4c5..da3594d25a3f 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -26,6 +26,20 @@ struct blkdev_copy_offload_io { loff_t offset; }; +/* Keeps track of single outstanding copy emulation IO */ +struct blkdev_copy_emulation_io { + struct blkdev_copy_io *cio; + struct work_struct emulation_work; + void *buf; + ssize_t buf_len; + loff_t pos_in; + loff_t pos_out; + ssize_t len; + struct block_device *bdev_in; + struct block_device *bdev_out; + gfp_t gfp; +}; + static sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) { unsigned int discard_granularity = bdev_discard_granularity(bdev); @@ -316,6 +330,215 @@ ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, } EXPORT_SYMBOL_GPL(blkdev_copy_offload); +static void *blkdev_copy_alloc_buf(ssize_t req_size, ssize_t *alloc_size, + gfp_t gfp) +{ + int min_size = PAGE_SIZE; + char *buf; + + while (req_size >= min_size) { + buf = kvmalloc(req_size, gfp); + if (buf) { + *alloc_size = req_size; + return buf; + } + req_size >>= 1; + } + + return NULL; +} + +static struct bio *bio_map_buf(void *data, unsigned int len, gfp_t gfp) +{ + unsigned long kaddr = (unsigned long)data; + unsigned long end = (kaddr + len + PAGE_SIZE - 1) >> PAGE_SHIFT; + unsigned long start = kaddr >> PAGE_SHIFT; + const int nr_pages = end - start; + bool is_vmalloc = is_vmalloc_addr(data); + struct page *page; + int offset, i; + struct bio *bio; + + bio = bio_kmalloc(nr_pages, gfp); + if (!bio) + return ERR_PTR(-ENOMEM); + bio_init(bio, NULL, bio->bi_inline_vecs, nr_pages, 0); + + if (is_vmalloc) { + flush_kernel_vmap_range(data, len); + bio->bi_private = data; + } + + offset = offset_in_page(kaddr); + for (i = 0; i < nr_pages; i++) { + unsigned int bytes = PAGE_SIZE - offset; + + if (len <= 0) + break; + + if (bytes > len) + bytes = len; + + if (!is_vmalloc) + page = virt_to_page(data); + else + page = vmalloc_to_page(data); + if (bio_add_page(bio, page, bytes, offset) < bytes) { + /* we don't support partial mappings */ + bio_uninit(bio); + kfree(bio); + return ERR_PTR(-EINVAL); + } + + data += bytes; + len -= bytes; + offset = 0; + } + + return bio; +} + +static void blkdev_copy_emulation_work(struct work_struct *work) +{ + struct blkdev_copy_emulation_io *emulation_io = container_of(work, + struct blkdev_copy_emulation_io, emulation_work); + struct blkdev_copy_io *cio = emulation_io->cio; + struct bio *read_bio, *write_bio; + loff_t pos_in = emulation_io->pos_in, pos_out = emulation_io->pos_out; + ssize_t rem, chunk; + int ret = 0; + + for (rem = emulation_io->len; rem > 0; rem -= chunk) { + chunk = min_t(int, emulation_io->buf_len, rem); + + read_bio = bio_map_buf(emulation_io->buf, + emulation_io->buf_len, + emulation_io->gfp); + if (IS_ERR(read_bio)) { + ret = PTR_ERR(read_bio); + break; + } + read_bio->bi_opf = REQ_OP_READ | REQ_SYNC; + bio_set_dev(read_bio, emulation_io->bdev_in); + read_bio->bi_iter.bi_sector = pos_in >> SECTOR_SHIFT; + read_bio->bi_iter.bi_size = chunk; + ret = submit_bio_wait(read_bio); + kfree(read_bio); + if (ret) + break; + + write_bio = bio_map_buf(emulation_io->buf, + emulation_io->buf_len, + emulation_io->gfp); + if (IS_ERR(write_bio)) { + ret = PTR_ERR(write_bio); + break; + } + write_bio->bi_opf = REQ_OP_WRITE | REQ_SYNC; + bio_set_dev(write_bio, emulation_io->bdev_out); + write_bio->bi_iter.bi_sector = pos_out >> SECTOR_SHIFT; + write_bio->bi_iter.bi_size = chunk; + ret = submit_bio_wait(write_bio); + kfree(write_bio); + if (ret) + break; + + pos_in += chunk; + pos_out += chunk; + } + cio->status = ret; + kvfree(emulation_io->buf); + kfree(emulation_io); + blkdev_copy_endio(cio); +} + +static inline ssize_t queue_max_hw_bytes(struct request_queue *q) +{ + return min_t(ssize_t, queue_max_hw_sectors(q) << SECTOR_SHIFT, + queue_max_segments(q) << PAGE_SHIFT); +} +/* + * @bdev_in: source block device + * @pos_in: source offset + * @bdev_out: destination block device + * @pos_out: destination offset + * @len: length in bytes to be copied + * @endio: endio function to be called on completion of copy operation, + * for synchronous operation this should be NULL + * @private: endio function will be called with this private data, + * for synchronous operation this should be NULL + * @gfp_mask: memory allocation flags (for bio_alloc) + * + * For synchronous operation returns the length of bytes copied or error + * For asynchronous operation returns -EIOCBQUEUED or error + * + * Description: + * If native copy offload feature is absent, caller can use this function + * to perform copy. + * We store information required to perform the copy along with temporary + * buffer allocation. We async punt copy emulation to a worker. And worker + * performs copy in 2 steps. + * 1. Read data from source to temporary buffer + * 2. Write data to destination from temporary buffer + */ +ssize_t blkdev_copy_emulation(struct block_device *bdev_in, loff_t pos_in, + struct block_device *bdev_out, loff_t pos_out, + size_t len, void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp) +{ + struct request_queue *in = bdev_get_queue(bdev_in); + struct request_queue *out = bdev_get_queue(bdev_out); + struct blkdev_copy_emulation_io *emulation_io; + struct blkdev_copy_io *cio; + ssize_t ret; + size_t max_hw_bytes = min(queue_max_hw_bytes(in), + queue_max_hw_bytes(out)); + + ret = blkdev_copy_sanity_check(bdev_in, pos_in, bdev_out, pos_out, len); + if (ret) + return ret; + + cio = kzalloc(sizeof(*cio), GFP_KERNEL); + if (!cio) + return -ENOMEM; + + cio->waiter = current; + cio->copied = len; + cio->endio = endio; + cio->private = private; + + emulation_io = kzalloc(sizeof(*emulation_io), gfp); + if (!emulation_io) + goto err_free_cio; + emulation_io->cio = cio; + INIT_WORK(&emulation_io->emulation_work, blkdev_copy_emulation_work); + emulation_io->pos_in = pos_in; + emulation_io->pos_out = pos_out; + emulation_io->len = len; + emulation_io->bdev_in = bdev_in; + emulation_io->bdev_out = bdev_out; + emulation_io->gfp = gfp; + + emulation_io->buf = blkdev_copy_alloc_buf(min(max_hw_bytes, len), + &emulation_io->buf_len, gfp); + if (!emulation_io->buf) + goto err_free_emulation_io; + + schedule_work(&emulation_io->emulation_work); + + if (cio->endio) + return -EIOCBQUEUED; + + return blkdev_copy_wait_io_completion(cio); + +err_free_emulation_io: + kfree(emulation_io); +err_free_cio: + kfree(cio); + return -ENOMEM; +} +EXPORT_SYMBOL_GPL(blkdev_copy_emulation); + static int __blkdev_issue_write_zeroes(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned flags) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 5405499bcf22..e0a832a1c3a7 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1046,6 +1046,10 @@ ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, loff_t pos_out, size_t len, void (*endio)(void *, int, ssize_t), void *private, gfp_t gfp_mask); +ssize_t blkdev_copy_emulation(struct block_device *bdev_in, loff_t pos_in, + struct block_device *bdev_out, loff_t pos_out, + size_t len, void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp); #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ From patchwork Wed Sep 20 08:07:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A986CE79AD for ; Wed, 20 Sep 2023 08:59:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234095AbjITI7Y (ORCPT ); Wed, 20 Sep 2023 04:59:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233915AbjITI6z (ORCPT ); Wed, 20 Sep 2023 04:58:55 -0400 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C610197 for ; Wed, 20 Sep 2023 01:58:29 -0700 (PDT) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20230920085827epoutp022a2fe8524574b82aa06abea39df4856e~Gj7K1ofj_2629026290epoutp02o for ; Wed, 20 Sep 2023 08:58:27 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20230920085827epoutp022a2fe8524574b82aa06abea39df4856e~Gj7K1ofj_2629026290epoutp02o DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200307; bh=WyEZwCEcMZEgd+uPDvb2WQHlAKDzDTA2b4v6jEJOw5k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PWFFbLhFiPWRJFp6j1CYQvkocziAXRaQ2Fap9szDajTbMXd7Tqk5UG1aJHVVBTH6F 7tuEkdCbhF9Oqh3+olTwQ6mIQ9VFSUAGww7YG6FMdXPVYyOJDvq5I+y4p6MQYWNrQy SqaXsNIn2hRjz655zFtYqwWHS4BefpgSZEoUk6L8= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20230920085826epcas5p2b0dc5c55c1e5a53f96a84d0cc4b708cb~Gj7KQdNb62273322733epcas5p2u; Wed, 20 Sep 2023 08:58:26 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.175]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4RrC9s0lJDz4x9Pv; Wed, 20 Sep 2023 08:58:25 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id 9D.D1.19094.034BA056; Wed, 20 Sep 2023 17:58:25 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20230920081508epcas5p4cf474394711300770d572af7d2fb621d~GjVW0BAaf3178931789epcas5p4f; Wed, 20 Sep 2023 08:15:08 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20230920081508epsmtrp12a183029ecd76fe9e5abb1613c009e80~GjVWuJnay2250022500epsmtrp1Q; Wed, 20 Sep 2023 08:15:08 +0000 (GMT) X-AuditID: b6c32a50-64fff70000004a96-a0-650ab4305868 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 9C.FE.08742.C0AAA056; Wed, 20 Sep 2023 17:15:08 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081505epsmtip1e9d81a65c92149340cdf3edfdf69aed3~GjVTepM8F3236832368epsmtip1e; Wed, 20 Sep 2023 08:15:05 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Anuj Gupta , Hannes Reinecke , Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 05/12] fs/read_write: Enable copy_file_range for block device. Date: Wed, 20 Sep 2023 13:37:42 +0530 Message-Id: <20230920080756.11919-6-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te1BUVRzHO/de7i7U0gVpPGDJtgUMb9YWPCBEJeSdZILGZrBmEFe4AbLc 3XaXgBpHkFeuI4tkD1dbCNd4yfsxqzwiNIElBhsGEAwtZ5cIDBQHreEVy4Xyv8/5/X7f8/v+ fmcOH3cs4bnwU1g1o2SlMhFpR7Rd8/T0FbfYMQEr3QDVm27g6OHiEoFOFK/gqGZSS6LZawsA mbsLAeqcO2+DxruvYKijvARDVTU/YaikZxQgy4gOQ50T3ui7AgOBOjr7CTR89QKJSr+38NCp MSOJKnpXMXSr2AKQ0ZwDUNtSKY7qZucJ1DexAw2t9Nq84Uxf0U3y6KE7jQQ9PJhON1WfJOlm w3G6fTybpC8WfWFDn86dI+mHlgmCnu8aIemilmpANw98Rj9q2kk3mf/CYuw/TA1NZqSJjFLI sAnyxBQ2KUy0/0D83vjAoACxrzgY7RYJWWkaEyaKiIrxfTtFtr4IkfATqSx9PRQjValE/q+H KuXpakaYLFepw0SMIlGmkCj8VNI0VTqb5Mcy6hBxQMCuwPXCw6nJazlTNoo5Qebcjd952eDm sxpgy4eUBN5eHcE0wI7vSHUA2HbyMWZNOFILAPZpnbnEYwCzp5uJLcU3SyYbLtEJ4ANNDskp 8jH4dzVfA/h8kvKGA2t8a40TlY3DhvaLwHrAKT0Op9bKgFWwjXofDtzV2liZoNzg8MmbGyyg QmD/rTzSehGk/KH2roM1bEvtgY8MYzyuxAH2nzNvGMIpV5jbeh7nzNXbQl15CscR0Fg3Azje Bmd6W3gcu8A/tQWbnAGrzlaSVm+QygNQN6bbFITDfJMWt3rAKU9Yf9WfC78EvzTVYVxfe3h6 yYxxcQE06rf4FXi5vozk2BmOPsnZZBo+KDcR3OKKAOwp7COKgVD31Dy6p+bR/d+6DODVwIVR qNKSmIRAhdiXZTL+e+UEeVoT2PgcXjFGUNOw4tcDMD7oAZCPi5wEae52jKMgUZr1KaOUxyvT ZYyqBwSuL/wM7vJCgnz9d7HqeLEkOEASFBQkCX4tSCzaLpjN/zbRkUqSqplUhlEwyi0dxrd1 ycZ8CgXbL987kn/E4PGb29GyvtjSuUHJfh+dPRm+WLD8ZuaZX30y34oenWLF4d6iobWvE9VO ml6J4pf2KnPl8/pp38Mau4qjGvaOxftQbeTiTF6uW7PGOK2asL0/uye21q373Yq1puW9X01m DF7qOtGxfM5BwVtdfO62nn2R9RAecB4VRv+htMzufJIlcT1LGYzP/HhQv5B1sMt93LNOFBxZ a4i0j3KKOmTq2J1mapeFlO+Ihj9Mvbpwf/SfKB5fLx55Jy608eX39ItszbGEIrdL+hLl8YXr RO2u7irtfGNr3rDXiEwe94E9ecz1858jQuovXL9n8PjIrzTu40r3hn37RmNbRYQqWSr2wpUq 6b9DTlutpQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA02Ra0hTYRjHe885OzvOVmdT6tXEaFGhS82QeK0QI4tDCdW6Et1GHix0a2yt vAReJpYr81KWunBdRN2sxEu6pXaZ5Vx2EdYqJ3ahuYJqZWKS5qwlQd9+z//3f54vD4ULbUQw dVh+lFXKpakikke0dolCI2Yaeeyy6pplqOFRN46GRycIlFs8iaP6wSISfer6DpDr3kmAOj06 Duq/Z8ZQx9VSDBnqH2Ko1PICoCFHJYY6nWJ0Jb+aQB2dNgLZb18ikb5miItOvzSRqNbqxdCr 4iGATK4cgFon9Di6+ekrgXqc89CzSSsnPogxVw5ymWevGwnG/kTNNBkLSKa5Ootp788mmWtn z3GYQo2HZIaHnATz9Y6DZM62GAHT3JvJjDSFMk2uL9jmWbt5q5PY1MPHWGVU3AHeoakcN0fh 4ad5ut9xs0Gfvxb4UZCOgeUTjzhawKOEdDuA4+fzwbQIgjWTD/BpDoAG7wfudEmDwev3dZgW UBRJi2HvFOXLA2ktDgvMHsI34PR1HD53VJC+7QBaAm1V/X8vEfQiaC/o4/iYT6+Etld5pO8Q pKNg0RuBL/ajV8GR6pdcHwv/VHr79NzpugDaKlyEj3F6PtTc0uHFgK78T1X+py4DzAiCWIVK lixTRSui5ezxSJVUplLLkyMPHpE1gb/fDg8zgTbjt0gLwChgAZDCRYF82WIeK+QnSdMzWOWR /Up1KquygHkUIZrLn/uxMElIJ0uPsiksq2CV/yxG+QVnY2Z9XFv5+5ZNzjH5xtwkt3edofSH WNPtSLS4vBLr7Yy37SWazlG1NtglTzHkloXGxOvSut3hwyFidcTOPQUJ/I4QLPbzeIk4f+Bc 0EF3BDF7oDaBOL18fU9LbciCMwlXSgIVBh32q9Fr1dsv5G3qWvPLWWG92IrfaEOX59/F6t9J 6p+atl/a0VF6IOZBWNTY07bwvC7CVNeQIfiorWvYcvJEIjnecmppoacu4X2hZGHY3bGfZfln NlydrH1cFbM1K2XFnG27GsvX+ic3CkoWSAfcsx6usBfZm8+fiD+WSMfu/hBXNUOaI4BrpzLF +3jmdIlXX5bpkAcM9MS9WCLfKyJUh6TR4bhSJf0NJJLZrFwDAAA= X-CMS-MailID: 20230920081508epcas5p4cf474394711300770d572af7d2fb621d X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081508epcas5p4cf474394711300770d572af7d2fb621d References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Anuj Gupta This is a prep patch. Allow copy_file_range to work for block devices. Relaxing generic_copy_file_checks allows us to reuse the existing infra, instead of adding a new user interface for block copy offload. Change generic_copy_file_checks to use ->f_mapping->host for both inode_in and inode_out. Allow block device in generic_file_rw_checks. Reviewed-by: Hannes Reinecke Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty --- fs/read_write.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/read_write.c b/fs/read_write.c index 4771701c896b..f0f52bf48f57 100644 --- a/fs/read_write.c +++ b/fs/read_write.c @@ -1405,8 +1405,8 @@ static int generic_copy_file_checks(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, size_t *req_count, unsigned int flags) { - struct inode *inode_in = file_inode(file_in); - struct inode *inode_out = file_inode(file_out); + struct inode *inode_in = file_in->f_mapping->host; + struct inode *inode_out = file_out->f_mapping->host; uint64_t count = *req_count; loff_t size_in; int ret; @@ -1708,7 +1708,9 @@ int generic_file_rw_checks(struct file *file_in, struct file *file_out) /* Don't copy dirs, pipes, sockets... */ if (S_ISDIR(inode_in->i_mode) || S_ISDIR(inode_out->i_mode)) return -EISDIR; - if (!S_ISREG(inode_in->i_mode) || !S_ISREG(inode_out->i_mode)) + if (!S_ISREG(inode_in->i_mode) && !S_ISBLK(inode_in->i_mode)) + return -EINVAL; + if ((inode_in->i_mode & S_IFMT) != (inode_out->i_mode & S_IFMT)) return -EINVAL; if (!(file_in->f_mode & FMODE_READ) || From patchwork Wed Sep 20 08:07:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392332 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06262CE79AD for ; Wed, 20 Sep 2023 08:59:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233910AbjITI72 (ORCPT ); Wed, 20 Sep 2023 04:59:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234012AbjITI64 (ORCPT ); Wed, 20 Sep 2023 04:58:56 -0400 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 693F01B2 for ; Wed, 20 Sep 2023 01:58:32 -0700 (PDT) Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20230920085830epoutp03b3918c91fc7d7ec61b56ec83a5f5aa5e~Gj7N14d0g1764817648epoutp030 for ; Wed, 20 Sep 2023 08:58:30 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20230920085830epoutp03b3918c91fc7d7ec61b56ec83a5f5aa5e~Gj7N14d0g1764817648epoutp030 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200310; bh=dWjB1X4Tkq4QWW6+lZPmoLyOTu3O/nojmE7xPmrQSyE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NVjsMIzWvxdI2pC+susV0lQgNsUvVzZ5neePmjnbm/HeuD9pwu7vQ6B2rtlGX7lja KqdA6OigmsfjJ7/5rFxRGlRBa6hUCV9BPYonkngFYBeRsre6xFqwBSvaq6z2xey/9e GlNrzpB9jAXU3zHtB6+5oSs6xO9kGy7SQy8g5KDc= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20230920085830epcas5p39ebd476d89a1e542215c8f140a2b0c3d~Gj7NSewOC0420204202epcas5p3J; Wed, 20 Sep 2023 08:58:30 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.174]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4RrC9w2LQZz4x9Ps; Wed, 20 Sep 2023 08:58:28 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 11.25.09635.434BA056; Wed, 20 Sep 2023 17:58:28 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20230920081519epcas5p24e047589278635b45aab3b260cb447f0~GjVgWL6Y43141931419epcas5p2E; Wed, 20 Sep 2023 08:15:19 +0000 (GMT) Received: from epsmgmc1p1new.samsung.com (unknown [182.195.42.40]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20230920081519epsmtrp1157ed79642ad55184f926482f8ada941~GjVgVHWKB2250122501epsmtrp1S; Wed, 20 Sep 2023 08:15:19 +0000 (GMT) X-AuditID: b6c32a4b-2f5ff700000025a3-96-650ab4348e01 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgmc1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id B9.6C.08649.61AAA056; Wed, 20 Sep 2023 17:15:19 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081515epsmtip15e8c3592b5843f13f663f32c55b96f38~GjVcov2vD0185801858epsmtip1K; Wed, 20 Sep 2023 08:15:15 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 06/12] fs, block: copy_file_range for def_blk_ops for direct block device Date: Wed, 20 Sep 2023 13:37:43 +0530 Message-Id: <20230920080756.11919-7-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te1BUVRzH59y73N1lAi6vOqDCzjJaQLC7uGyHl9pAcJNqaJpKS4IV7rAE +2B3YUtmEuRRorJAgrAoIg9FFJHnACspoOJCBROjPJJHstQkAyokikjFslj+9/l9z+87v8eZ Hwu3K2A6s+JlalopEydyCUtGa4+7u5ew2ZLmT4yxUX3fTRw9erzCQIfyVnF0YVxLoNmeBYCM 174FqHO+1AKNXmvH0JWKAgydv3ADQwXddwCaua3DUOeYJzqTXcVAVzoNDDTUcZJAp8/OMNGR 4TYCnev9G0MjeTMAtRnTAWpdOY2jS7MPGOjW2CY0sNprscuJateNM6mBiQYGNfRTMtVYe5ig mqoOUvrRNIKqzP3egjqWMU9Qj2bGGNSDH24TVG5zLaCa+lOpxUYXqtE4h0VYf5YQKKHFsbSS Q8ti5LHxsrggbvhHUcFRviK+wEvgh97icmRiKR3EDXkvwis0PnFtEVxOijgxeU2KEKtUXN6O QKU8WU1zJHKVOohLK2ITFUKFt0osVSXL4rxltNpfwOf7+K4lRidIau/+aqHQ23x1arWckQZ6 X8kBbBYkhVCftoLnAEuWHakHcOz5pIU5WABweb56I1gCsDU/H+QA1rpluijErHcCWKNd2LBn YfD63F9MUxJBesL+f1gm3YFMw+FlfSUwBThZhsOme72YKcmejIKFv+8x9cEgt0LDH+m4ia1I fzjd0b9RjAe1k7YmmU0GwMWqYaY5xRYaSowME+OkK8xoKV3vAZKVbLhwdYlhni0EfldSjJvZ Ht7vbWaa2Rn+qc3eYA08f7yGMJszAdQN64D5YSfM6tPipiZw0h3Wd/DM8hZY2HcJMxe2hsdW jJhZt4JtZS/YDV6sLyfM7ATvPEnfYAoa67I2NpoL4EjOFJEHOLqXBtK9NJDu/9LlAK8FTrRC JY2jVb6K7TJa898vx8iljWD9ODzC28C9qYfe3QBjgW4AWTjXwUq6zZK2s4oVf32AVsqjlMmJ tKob+K4tPB93doyRr12XTB0lEPrxhSKRSOi3XSTgvmY1m3Uq1o6ME6vpBJpW0MoXPozFdk7D Bve1H6oL19m+utvFkBrqb2jvyq7Wf/nBb0/984wHGmpSDjacYL9rM/8hMf72681yx4mdDzWN VqvB45MD6qYT07zcMnveYc+znptDihNW1T0B0Z8Epnc0JIqfdZXcP7Jv9Nk3PtscBzkdT6jI 7oqwG9G4nWSP2k+w43nSls9LU3dFLneBUr0mu9A9VGPj63LUKeOmz2ap9V6Qkbl3aqY0k/ej y1aNdDDS/839TfuLDbL8jxerKwN+GdW6LU1vkrf8fLemqE7iajgXuDzhluT6+GLCyPG5osPX K0S3tENi+6VaXpjjmcufBnuMp/ScfOr0ha7qalhYUgO/731r6p0Wd47DG7uXuQyVRCzwwJUq 8b+AeRKepQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA02ReUiTYRzHe9733bt3q+nrsnxmUDBoiab2UsjTbWD1/hESERXmNdqbmm6N vVpWUtqoaFauE/Oammk70JxmHhmi6TLtwDJ1pQZql2SH3UurZUH/ffhe/OBH4VIH4UslaJI5 nUaZJCfFRE2LfG6gj0XMLWpoxVHFnTYcvf/kItBh4wSOrP1ZJBpt+QDQcNMxgBrHcgWor6kO QzeKz2DIbG3F0JnmxwCNdOdgqNEZgIqOlhDoRmM7gR7W55HIVDoiRJk9tSQqc0xiqNc4AlDt cAZANS4TjspH3xLotnMOuj/hEITK2LqcfiF7f6CSYB/eTWHtluMkW1VyiG3oSyfZS6fOCtiT +jGSfT/iJNi3N7tJ9lS1BbBVHQfYcftc1j78BtvoESFeoeKSEvZwuuBVseJ4y9MnAm2DZ2r+ RCGRDhwzDICiIL0EDl0IMwAxJaUbAGw6UkQagOi3LoOlE7fwKZ4JzZMvhFMhPQadg3qBu0zS AbDjJ+XWvWkDDo/XjRHuAk7bcFidL3HzTDoKDunzMDcT9HzY/iLjz6iEXgaH6jvA1BHBMGvQ yy2L6OVwvKRH6Gbp70jHA5NwKu4F2y8O/52fB/XXcnEjoHP+s3L+swoBZgEyTsur49Q7GC2j 4fYG8Uo1n6KJC9qxW20Hf37t71cLBkyTQc0Ao0AzgBQu95aoFWJOKlEp9+3ndLtjdClJHN8M 5lCE3EfCZOeqpHScMplL5Dgtp/vnYpTINx2rD+njuxyLC6aPmzqfZtoW+0e2Zn63WTwXhBNY ZclBgXRFhk+sH+Md/GmQPMfXdO8UGz02L7PR2ert52tgttnOtDguZY0yvmXrtJ6JO/c5DV2l ZQVcNG/eNC1VFVn1aFcifP1MeC9tVlHymihwOXroa4HCY715g+vbwt7C2SEuhfm08bMt+0rX wKOLk37XvtxiUleBaDzDKzym+MlaRcR25aGr02NCr4dubTtKJq8u3Lb6hLVpLPFV2NewWCsT /pzq7+sJCLTUdWoDVRELY1Y6QqSilKVF+eUnCs6Naq6f1KW1pXW2WjfI7OZ3r2QvL1srMPVH kaGqWPRjyyZpr5zg45WMP67jlb8AEnlxkFoDAAA= X-CMS-MailID: 20230920081519epcas5p24e047589278635b45aab3b260cb447f0 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081519epcas5p24e047589278635b45aab3b260cb447f0 References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org For direct block device opened with O_DIRECT, use copy_file_range to issue device copy offload, or use generic_copy_file_range in case device copy offload capability is absent or the device files are not open with O_DIRECT. Reviewed-by: Hannes Reinecke Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty --- block/fops.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/block/fops.c b/block/fops.c index acff3d5d22d4..6aa537c0e24f 100644 --- a/block/fops.c +++ b/block/fops.c @@ -735,6 +735,30 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to) return ret; } +static ssize_t blkdev_copy_file_range(struct file *file_in, loff_t pos_in, + struct file *file_out, loff_t pos_out, + size_t len, unsigned int flags) +{ + struct block_device *in_bdev = I_BDEV(bdev_file_inode(file_in)); + struct block_device *out_bdev = I_BDEV(bdev_file_inode(file_out)); + ssize_t copied = 0; + + if ((in_bdev == out_bdev) && bdev_max_copy_sectors(in_bdev) && + (file_in->f_iocb_flags & IOCB_DIRECT) && + (file_out->f_iocb_flags & IOCB_DIRECT)) { + copied = blkdev_copy_offload(in_bdev, pos_in, pos_out, len, + NULL, NULL, GFP_KERNEL); + if (copied < 0) + copied = 0; + } else { + copied = generic_copy_file_range(file_in, pos_in + copied, + file_out, pos_out + copied, + len - copied, flags); + } + + return copied; +} + #define BLKDEV_FALLOC_FL_SUPPORTED \ (FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE | \ FALLOC_FL_ZERO_RANGE | FALLOC_FL_NO_HIDE_STALE) @@ -828,6 +852,7 @@ const struct file_operations def_blk_fops = { .splice_read = filemap_splice_read, .splice_write = iter_file_splice_write, .fallocate = blkdev_fallocate, + .copy_file_range = blkdev_copy_file_range, }; static __init int blkdev_init(void) From patchwork Wed Sep 20 08:07:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392333 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3E2ACE79AC for ; Wed, 20 Sep 2023 08:59:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233935AbjITI7x (ORCPT ); Wed, 20 Sep 2023 04:59:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53892 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234029AbjITI7U (ORCPT ); Wed, 20 Sep 2023 04:59:20 -0400 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A09CE5 for ; Wed, 20 Sep 2023 01:58:38 -0700 (PDT) Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20230920085836epoutp046fcad709d6af71ddcb7142f442290dfa~Gj7TJMHYe1275712757epoutp04J for ; Wed, 20 Sep 2023 08:58:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20230920085836epoutp046fcad709d6af71ddcb7142f442290dfa~Gj7TJMHYe1275712757epoutp04J DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200316; bh=9HUxsYJcUqY59rvCQaWh3ErxEvPd5vF6B6n+YdK8rBY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QdG9TGW7/66r/O3y46Ise0DXwmqgH/cZVABF/GpaQCO2UuUf2ZgBRCOoUBO8WQNYZ coUzOOXAs2atoKnm9wwHqTL+C4XdIrD3BCTH5cuDXbiOsudKgnxtjJxhvkPWKkFeq1 uijWZoG/nO2HxIcNTf9HlaHB1R+FSyNgf70LVUJc= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20230920085835epcas5p3ce375a9c404fc7a44189faa4cd55798f~Gj7ST4FA_1154311543epcas5p3U; Wed, 20 Sep 2023 08:58:35 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.175]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4RrCB0319Vz4x9Px; Wed, 20 Sep 2023 08:58:32 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 05.25.09635.834BA056; Wed, 20 Sep 2023 17:58:32 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20230920081529epcas5p33e1ff0e22817262b1cf749616a37d8a0~GjVp1XcJj1820618206epcas5p3e; Wed, 20 Sep 2023 08:15:29 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20230920081529epsmtrp1289506647955b558123e9cf03eee4f55~GjVp0LLws2250022500epsmtrp1f; Wed, 20 Sep 2023 08:15:29 +0000 (GMT) X-AuditID: b6c32a4b-563fd700000025a3-a1-650ab4385642 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 91.0F.08742.12AAA056; Wed, 20 Sep 2023 17:15:29 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081525epsmtip1442b86aed4352b80f58575bee6705cb5~GjVmWJFMM2849128491epsmtip1Y; Wed, 20 Sep 2023 08:15:25 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Kanchan Joshi , =?utf-8?q?Javier_Gonz=C3=A1lez?= , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 07/12] nvme: add copy offload support Date: Wed, 20 Sep 2023 13:37:44 +0530 Message-Id: <20230920080756.11919-8-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te1BUdRTH+d17ubuQ61yB6gdGLOtkAfFYefiDQMgYuYOMUj7KxglWuLIE +5jdJSmdEeQ1UDxEjFxeqzASjyARCFxwCBIUSEwCli2UYCGMEOQhNYTEskv53+d8zzm/8/jN YeMWRSwbdrRYwcjEglgeaU40djg4OKN6c8atrtoC1XZ34ujJ0gqBzuWs4qhqJJtE0x3zAOna 0gBqfVxgiobbmjHUciUXQxVVtzA0/usCC91amyFRbvsgQBMDSgy1ap3Q5dQyArW03iFQ/41C EpVcnWChz4eaSFTe9QxDmpwJgJp0iQA1rpTgqGZ6lkC3tdtR32qXacB2ulk5wqL7Hlwj6P4f 4+i6ynSSvl52llYPJ5B0adYFUzoz6TFJP5nQEvTszQGSzqqvBPT1ntP0Qt2rdJ1uBgvd+mGM r5ARRDIyLiOOkERGi6P8ePsPhb0T5unlxnfme6PdPK5YIGL8eIEhoc77omPXV8LjfiKIjVuX QgVyOc91j69MEqdguEKJXOHHY6SRsVIPqYtcIJLHiaNcxIzCh+/mtstzPTA8Rvjn8g9AWhgS X65bZCWA8j0ZwIwNKQ+oSr8MMoA524JSA5jWNosbjHkAR7p6jcZTAGcn1aabKYvLSyyDoxXA DI3GmJ+CweS/S9cNNpuknGDPGluvW1EJOPxWXboRhFNJBBysziL1T1lS3jBj7uIGE9RrsL0z faMEh/KBukvnTfUPQcoVZj/cppfNqLfgQtkQyxCyDd65pCP0jFN2MKmhADd0d98MzszYGTgQ 5nT8buzaEv7RVc8ysA18lJ1q5FOwIu9rUt8bpJIBVA4pgcHhD1O6s3F9DzjlAGtvuBpkW3ix uwYz1N0KM1d0mEHnwKbiTd4Bq2tVpIGt4eByopFpuDgzaVxpFoDFzffwHMBVPjeP8rl5lP+X VgG8ElgzUrkoipF7St3FzKn/vjlCIqoDG3fiuL8JjI3OubQDjA3aAWTjPCuOaKc5Y8GJFHz6 GSOThMniYhl5O/Bc3/d53ObFCMn6oYkVYXwPbzcPLy8vD293Lz7vZc50SlGkBRUlUDAxDCNl ZJt5GNvMJgHrKj4q9Iq/TV6N+iqocPS33viSXTMarvVu6iH/n7ECE9ekxPwXelX5B/y6O22m hnPf+9LvgZnOvY+zBUYMnRPHHMrPeNTQsNp74WZAgu3S8YMmc3NTWz7eN3F60EmbN58XwglK rguerRT7iw4nWx5pfNd1R+PA2yfevLuX0XxEHVEUHztz4plrjf1ZcOBkjuTMF6OvfPCTXcvP Afj4tVaNjzqootPqm7wB+6l70iCfKnlRXjjHxvb7KQvtG4oxOtBby51tGi8m7Ii7x0st+fap nJa+Rl8h25E20fq8LsrcGa566T5vrffp3l8mO3uE4+pg1fBf/keDu658d/DY4f73c096ppnw CLlQwHfEZXLBv98vK12wBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrJIsWRmVeSWpSXmKPExsWy7bCSnK7iKq5UgxU9vBbrTx1jtvj49TeL RdOEv8wWq+/2s1m8PvyJ0eLJgXZGi73vZrNa3Dywk8liz6JJTBYrVx9lsnh85zO7xdH/b9ks Jh26xmjx9OosJou9t7QtFrYtYbHYs/cki8XlXXPYLOYve8pu0X19B5vF8uP/mCxuTHjKaLHj SSOjxbbf85kt1r1+z2Jx4pa0xfm/x1kdpD12zrrL7nH+3kYWj8tnSz02repk89i8pN5j980G No/FfZNZPXqb37F5fHx6i8Xj/b6rbB59W1Yxemw+Xe3xeZOcx6Ynb5kC+KK4bFJSczLLUov0 7RK4Mt58P8JYMMenYvmTL+wNjMvtuhg5OSQETCS+fP/K3sXIxSEksJtR4vWU50wQCUmJZX+P MEPYwhIr/z2HKmpmktjy+hhQEQcHm4C2xOn/HCBxEYEuZonOne9YQBxmgcksEhsfnmME6RYW sJTo+jCVDcRmEVCVOHSskxXE5hWwkngycyIryCAJAX2J/vuCIGFOAWuJz0uus4PYQkAlpy/M Z4coF5Q4OfMJC0g5s4C6xPp5QiBhZgF5ieats5knMArOQlI1C6FqFpKqBYzMqxglUwuKc9Nz iw0LDPNSy/WKE3OLS/PS9ZLzczcxglODluYOxu2rPugdYmTiYDzEKMHBrCTCm6vGlSrEm5JY WZValB9fVJqTWnyIUZqDRUmcV/xFb4qQQHpiSWp2ampBahFMlomDU6qBqXXi5uLnitqKaxeq BVhXC7pvYU1wzeao7v25IeBz++TU5mrOJ8p1C+O95OY39eZZdvGr7F89Q0dlvf7C1y/dt2Tc ltHrbXzG82aS3xpNT49H9R3zOieFdUa7nUnbszK6ltN4U/7xb82rl3z0j+AVVJJzck34wPo1 Qm+tFmOC2vezPgzv09ZPeX6AtVy6uXoW40KjgqRPv9wkVt4yObK+4HfX9TsRXV9Fe48rr/o3 YXJopdL2iDIn6XWbtnx8dyC3VeRyrM8hjSafD48iUru+chi/veEQaDHXqK3pt3vX3RlJ7wPe ash95bH4+M/F8a1h+4RcSQsPy39uB/TTRWvFV9xRqm3qcvbdsdba2F+JpTgj0VCLuag4EQBw 0vYYfAMAAA== X-CMS-MailID: 20230920081529epcas5p33e1ff0e22817262b1cf749616a37d8a0 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081529epcas5p33e1ff0e22817262b1cf749616a37d8a0 References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Current design only supports single source range. We receive a request with REQ_OP_COPY_SRC. Parse this request which consists of src(1st) and dst(2nd) bios. Form a copy command (TP 4065) trace event support for nvme_copy_cmd. Set the device copy limits to queue limits. Reviewed-by: Hannes Reinecke Signed-off-by: Kanchan Joshi Signed-off-by: Nitesh Shetty Signed-off-by: Javier González Signed-off-by: Anuj Gupta --- drivers/nvme/host/constants.c | 1 + drivers/nvme/host/core.c | 79 +++++++++++++++++++++++++++++++++++ drivers/nvme/host/trace.c | 19 +++++++++ include/linux/blkdev.h | 1 + include/linux/nvme.h | 43 +++++++++++++++++-- 5 files changed, 140 insertions(+), 3 deletions(-) diff --git a/drivers/nvme/host/constants.c b/drivers/nvme/host/constants.c index 20f46c230885..2f504a2b1fe8 100644 --- a/drivers/nvme/host/constants.c +++ b/drivers/nvme/host/constants.c @@ -19,6 +19,7 @@ static const char * const nvme_ops[] = { [nvme_cmd_resv_report] = "Reservation Report", [nvme_cmd_resv_acquire] = "Reservation Acquire", [nvme_cmd_resv_release] = "Reservation Release", + [nvme_cmd_copy] = "Copy Offload", [nvme_cmd_zone_mgmt_send] = "Zone Management Send", [nvme_cmd_zone_mgmt_recv] = "Zone Management Receive", [nvme_cmd_zone_append] = "Zone Append", diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 21783aa2ee8e..4522c702610b 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -763,6 +763,63 @@ static inline void nvme_setup_flush(struct nvme_ns *ns, cmnd->common.nsid = cpu_to_le32(ns->head->ns_id); } +static inline blk_status_t nvme_setup_copy_offload(struct nvme_ns *ns, + struct request *req, + struct nvme_command *cmnd) +{ + struct nvme_copy_range *range = NULL; + struct bio *bio; + u64 dst_lba = 0, src_lba, n_lba; + u16 nr_range = 1, control = 0, seg = 1; + + if (blk_rq_nr_phys_segments(req) != BLK_COPY_MAX_SEGMENTS) + return BLK_STS_IOERR; + + /* + * First bio contains information about source and last bio contains + * information about destination. + */ + __rq_for_each_bio(bio, req) { + if (seg == blk_rq_nr_phys_segments(req)) { + dst_lba = nvme_sect_to_lba(ns, bio->bi_iter.bi_sector); + if (n_lba != bio->bi_iter.bi_size >> ns->lba_shift) + return BLK_STS_IOERR; + } else { + src_lba = nvme_sect_to_lba(ns, bio->bi_iter.bi_sector); + n_lba = bio->bi_iter.bi_size >> ns->lba_shift; + } + seg++; + } + + if (req->cmd_flags & REQ_FUA) + control |= NVME_RW_FUA; + + if (req->cmd_flags & REQ_FAILFAST_DEV) + control |= NVME_RW_LR; + + memset(cmnd, 0, sizeof(*cmnd)); + cmnd->copy.opcode = nvme_cmd_copy; + cmnd->copy.nsid = cpu_to_le32(ns->head->ns_id); + cmnd->copy.control = cpu_to_le16(control); + cmnd->copy.sdlba = cpu_to_le64(dst_lba); + cmnd->copy.nr_range = 0; + + range = kmalloc_array(nr_range, sizeof(*range), + GFP_ATOMIC | __GFP_NOWARN); + if (!range) + return BLK_STS_RESOURCE; + + range[0].slba = cpu_to_le64(src_lba); + range[0].nlb = cpu_to_le16(n_lba - 1); + + req->special_vec.bv_page = virt_to_page(range); + req->special_vec.bv_offset = offset_in_page(range); + req->special_vec.bv_len = sizeof(*range) * nr_range; + req->rq_flags |= RQF_SPECIAL_PAYLOAD; + + return BLK_STS_OK; +} + static blk_status_t nvme_setup_discard(struct nvme_ns *ns, struct request *req, struct nvme_command *cmnd) { @@ -1005,6 +1062,11 @@ blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req) case REQ_OP_ZONE_APPEND: ret = nvme_setup_rw(ns, req, cmd, nvme_cmd_zone_append); break; + case REQ_OP_COPY_SRC: + ret = nvme_setup_copy_offload(ns, req, cmd); + break; + case REQ_OP_COPY_DST: + return BLK_STS_IOERR; default: WARN_ON_ONCE(1); return BLK_STS_IOERR; @@ -1745,6 +1807,21 @@ static void nvme_config_discard(struct gendisk *disk, struct nvme_ns *ns) blk_queue_max_write_zeroes_sectors(queue, UINT_MAX); } +static void nvme_config_copy(struct gendisk *disk, struct nvme_ns *ns, + struct nvme_id_ns *id) +{ + struct nvme_ctrl *ctrl = ns->ctrl; + struct request_queue *q = disk->queue; + + if (!(ctrl->oncs & NVME_CTRL_ONCS_COPY)) { + blk_queue_max_copy_hw_sectors(q, 0); + return; + } + + blk_queue_max_copy_hw_sectors(q, nvme_lba_to_sect(ns, + le16_to_cpu(id->mssrl))); +} + static bool nvme_ns_ids_equal(struct nvme_ns_ids *a, struct nvme_ns_ids *b) { return uuid_equal(&a->uuid, &b->uuid) && @@ -1944,6 +2021,7 @@ static void nvme_update_disk_info(struct gendisk *disk, set_capacity_and_notify(disk, capacity); nvme_config_discard(disk, ns); + nvme_config_copy(disk, ns, id); blk_queue_max_write_zeroes_sectors(disk->queue, ns->ctrl->max_zeroes_sectors); } @@ -4654,6 +4732,7 @@ static inline void _nvme_check_size(void) BUILD_BUG_ON(sizeof(struct nvme_download_firmware) != 64); BUILD_BUG_ON(sizeof(struct nvme_format_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_dsm_cmd) != 64); + BUILD_BUG_ON(sizeof(struct nvme_copy_command) != 64); BUILD_BUG_ON(sizeof(struct nvme_write_zeroes_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_abort_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_get_log_page_command) != 64); diff --git a/drivers/nvme/host/trace.c b/drivers/nvme/host/trace.c index 1c36fcedea20..82c6aef77c31 100644 --- a/drivers/nvme/host/trace.c +++ b/drivers/nvme/host/trace.c @@ -150,6 +150,23 @@ static const char *nvme_trace_read_write(struct trace_seq *p, u8 *cdw10) return ret; } +static const char *nvme_trace_copy(struct trace_seq *p, u8 *cdw10) +{ + const char *ret = trace_seq_buffer_ptr(p); + u64 sdlba = get_unaligned_le64(cdw10); + u8 nr_range = get_unaligned_le16(cdw10 + 8); + u16 control = get_unaligned_le16(cdw10 + 10); + u32 dsmgmt = get_unaligned_le32(cdw10 + 12); + u32 reftag = get_unaligned_le32(cdw10 + 16); + + trace_seq_printf(p, + "sdlba=%llu, nr_range=%u, ctrl=0x%x, dsmgmt=%u, reftag=%u", + sdlba, nr_range, control, dsmgmt, reftag); + trace_seq_putc(p, 0); + + return ret; +} + static const char *nvme_trace_dsm(struct trace_seq *p, u8 *cdw10) { const char *ret = trace_seq_buffer_ptr(p); @@ -243,6 +260,8 @@ const char *nvme_trace_parse_nvm_cmd(struct trace_seq *p, return nvme_trace_zone_mgmt_send(p, cdw10); case nvme_cmd_zone_mgmt_recv: return nvme_trace_zone_mgmt_recv(p, cdw10); + case nvme_cmd_copy: + return nvme_trace_copy(p, cdw10); default: return nvme_trace_common(p, cdw10); } diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index e0a832a1c3a7..ce2009b693c8 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1226,6 +1226,7 @@ static inline unsigned int bdev_discard_granularity(struct block_device *bdev) /* maximum copy offload length, this is set to 128MB based on current testing */ #define BLK_COPY_MAX_BYTES (1 << 27) +#define BLK_COPY_MAX_SEGMENTS 2 static inline unsigned int bdev_max_copy_sectors(struct block_device *bdev) { diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 26dd3f859d9d..7744538c4ca4 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -337,7 +337,7 @@ struct nvme_id_ctrl { __u8 nvscc; __u8 nwpc; __le16 acwu; - __u8 rsvd534[2]; + __le16 ocfs; __le32 sgls; __le32 mnan; __u8 rsvd544[224]; @@ -365,6 +365,7 @@ enum { NVME_CTRL_ONCS_WRITE_ZEROES = 1 << 3, NVME_CTRL_ONCS_RESERVATIONS = 1 << 5, NVME_CTRL_ONCS_TIMESTAMP = 1 << 6, + NVME_CTRL_ONCS_COPY = 1 << 8, NVME_CTRL_VWC_PRESENT = 1 << 0, NVME_CTRL_OACS_SEC_SUPP = 1 << 0, NVME_CTRL_OACS_NS_MNGT_SUPP = 1 << 3, @@ -414,7 +415,10 @@ struct nvme_id_ns { __le16 npdg; __le16 npda; __le16 nows; - __u8 rsvd74[18]; + __le16 mssrl; + __le32 mcl; + __u8 msrc; + __u8 rsvd91[11]; __le32 anagrpid; __u8 rsvd96[3]; __u8 nsattr; @@ -831,6 +835,7 @@ enum nvme_opcode { nvme_cmd_resv_report = 0x0e, nvme_cmd_resv_acquire = 0x11, nvme_cmd_resv_release = 0x15, + nvme_cmd_copy = 0x19, nvme_cmd_zone_mgmt_send = 0x79, nvme_cmd_zone_mgmt_recv = 0x7a, nvme_cmd_zone_append = 0x7d, @@ -854,7 +859,8 @@ enum nvme_opcode { nvme_opcode_name(nvme_cmd_resv_release), \ nvme_opcode_name(nvme_cmd_zone_mgmt_send), \ nvme_opcode_name(nvme_cmd_zone_mgmt_recv), \ - nvme_opcode_name(nvme_cmd_zone_append)) + nvme_opcode_name(nvme_cmd_zone_append), \ + nvme_opcode_name(nvme_cmd_copy)) @@ -1031,6 +1037,36 @@ struct nvme_dsm_range { __le64 slba; }; +struct nvme_copy_command { + __u8 opcode; + __u8 flags; + __u16 command_id; + __le32 nsid; + __u64 rsvd2; + __le64 metadata; + union nvme_data_ptr dptr; + __le64 sdlba; + __u8 nr_range; + __u8 rsvd12; + __le16 control; + __le16 rsvd13; + __le16 dspec; + __le32 ilbrt; + __le16 lbat; + __le16 lbatm; +}; + +struct nvme_copy_range { + __le64 rsvd0; + __le64 slba; + __le16 nlb; + __le16 rsvd18; + __le32 rsvd20; + __le32 eilbrt; + __le16 elbat; + __le16 elbatm; +}; + struct nvme_write_zeroes_cmd { __u8 opcode; __u8 flags; @@ -1792,6 +1828,7 @@ struct nvme_command { struct nvme_download_firmware dlfw; struct nvme_format_cmd format; struct nvme_dsm_cmd dsm; + struct nvme_copy_command copy; struct nvme_write_zeroes_cmd write_zeroes; struct nvme_zone_mgmt_send_cmd zms; struct nvme_zone_mgmt_recv_cmd zmr; From patchwork Wed Sep 20 08:07:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392334 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78D6ACE79AE for ; Wed, 20 Sep 2023 08:59:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233967AbjITI7y (ORCPT ); Wed, 20 Sep 2023 04:59:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234037AbjITI7V (ORCPT ); Wed, 20 Sep 2023 04:59:21 -0400 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2764ED for ; Wed, 20 Sep 2023 01:58:39 -0700 (PDT) Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20230920085838epoutp0364a205f4a13059b6d4f404a995157576~Gj7Usbpap1764817648epoutp034 for ; Wed, 20 Sep 2023 08:58:38 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20230920085838epoutp0364a205f4a13059b6d4f404a995157576~Gj7Usbpap1764817648epoutp034 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200318; bh=ffw02p9pmXuydxofYTTqZSKySXsPh7qWn6KVyBPXmgo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qerXwVLYehJATb1o2MdytLOh6L2CWwaHbQ6B3RfoSoeGo9A6tiTbh8wkkXZR27KyM pGn5Z4mRbj4sB6WQWZhok+R87z6eq23YHZCI5Rt67JUFWzLNRSH61M83eSi6h4laM3 a0ZORpfSztTnhTQYyBKBBKJ3U4KJcp21tcybwbRQ= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20230920085837epcas5p17cf144a4b2b35fdf44e2ac3be7dcef1a~Gj7UHaeTi0342303423epcas5p1F; Wed, 20 Sep 2023 08:58:37 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.183]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4RrCB40W9dz4x9Q0; Wed, 20 Sep 2023 08:58:36 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id 71.F1.19094.B34BA056; Wed, 20 Sep 2023 17:58:35 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20230920081539epcas5p23035fe3e03eeb7848a40aff580e5cdbf~GjVy5NaBp1340213402epcas5p24; Wed, 20 Sep 2023 08:15:39 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20230920081539epsmtrp1ee11ca2f1b8c10ec528e1803f5e22428~GjVy4Lj-L2250122501epsmtrp1i; Wed, 20 Sep 2023 08:15:39 +0000 (GMT) X-AuditID: b6c32a50-64fff70000004a96-be-650ab43b700f Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 99.9E.08788.A2AAA056; Wed, 20 Sep 2023 17:15:38 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081535epsmtip1784cc8d37f19d0276b5712c51437ec8b~GjVvpi5ow3071830718epsmtip1U; Wed, 20 Sep 2023 08:15:35 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 08/12] nvmet: add copy command support for bdev and file ns Date: Wed, 20 Sep 2023 13:37:45 +0530 Message-Id: <20230920080756.11919-9-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TbUxTVxjeube0haTmyoc7dLo1ZXMTA7RSygFkGgG9DNxYTCSaLHiBO4qU tustOrcYYVAXNBYoYlyZ4ITJAAH5lI+CyscQEDohgDZjbKRsbAScsK0hiK704ua/533O85zn vO/Jy8fdC3lCfqpKR2tVlFLMdeO09Ozw9QtrcqMl7XY5qhv8HkdP/l7loC/y13BUPZXHRfM9 SwDZ7nwJUOdisQt6dKcNQ+ZrRgxVVvdhyNg9AdDsuAlDndad6Juz5Rxk7hzgoLH2r7mo9Pos D52fbOWiiv5nGHqYPwtQqy0LoJbVUhzVzj/moHvW15Blrd9lrzfZZprikZaf6jnk2HAG2VCV yyUby8+QHY8yuWSZodCFvJC9yCWfzFo55OOucS5paKoCZOPQ5+Ryw+tkg20Bi9t0NG23gqaS aa2IViWpk1NVKeHimEMJEQlBconUTxqCgsUiFZVOh4sjY+P89qcqHYMQi05QygwHFUcxjDjg 3d1adYaOFinUjC5cTGuSlRqZxp+h0pkMVYq/itaFSiWSXUEO4bE0RUtTG9DUHPjUOPcMzwQF YeeAKx8SMtg83IufA258d8IMoGW4GGOLJQB/We4FbPEPgD8+zXLI+E5LYccRlu8EsH2lfMOu x6C9psop4hI74dBz/jrvSWTi8GZHmfMmnCjBYeNMP7Ye7kF8CC8NjuPrmEO8BStHxpxYQITC 36yNG2kBMG968zrtSoTB5fJJHivZDAe+snHWMU68AbObi52PgES1K1y5UbfhjYSTrdFsnx7w j/4mHouF8Pe8sxv4JKy8+B2X9eYAaJo0AfZgD9QP5jnvwYkdsK49gKW3waLBWozN3QQvrNow lhfA1pIX2AfeqLvKZbE3nLBnbWAS/mpu47HDMgDYW7GI5wOR6aV+TC/1Y/o/+irAq4CQ1jDp KXRSkEbqp6JP/vfNSer0BuDcDt+4VlB9c82/G2B80A0gHxd7CtK3u9HugmTq1Ge0Vp2gzVDS TDcIcgy8ABd6Jakd66XSJUhlIRKZXC6XhQTKpeJXBfP6K8nuRAqlo9NoWkNrX/gwvqswE4v2 Ueq7+D+XhJx5c+a9xJVRKZoIcTn5CWbfQ031v11Wb/Ia2Dd8sKCHqX/wgdTSbY1bMG8p8jrh P2ZXLk7OJR4/HT9RKLKlHG2I+nY1sqb0cGLElNvxiPtbjXuvpwVfHjnm09WrPyUbvbvVP6rg 9EexzEHtO0OXbksKLxqPtJX69W7vEmxLak+0W/ZFFt+/p7gS4Ln08WiltT7eULHLRdFY25eL 3T50LdNQO/M0Cnoyf4UvWMzPedHNnQduxYize14Jnuvx2HI5dn+ALSdyBOv6U1yZMP4wzXBr vIh6oL7L+0Hf3DdgnNZa4mMCdaG5h2Pfz5kW+pLx8vbzmHd5XqooUMxhFJTUF9cy1L/CKy38 pgQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrOIsWRmVeSWpSXmKPExsWy7bCSnK7WKq5Ug6UvrC3WnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFnvfzWa1uHlgJ5PFnkWTmCxWrj7KZDHp0DVGi6dXZzFZ 7L2lbbGwbQmLxZ69J1ksLu+aw2Yxf9lTdovu6zvYLJYf/8dkcWPCU0aLHU8aGS22/Z7PbLHu 9XsWixO3pC3O/z3O6iDpsXPWXXaP8/c2snhcPlvqsWlVJ5vH5iX1HrtvNrB5LO6bzOrR2/yO zePj01ssHu/3XWXz6NuyitFj8+lqj8+b5Dw2PXnLFMAXxWWTkpqTWZZapG+XwJWxbctOxoK1 7hWTXvxjbmCcaN3FyMEhIWAiMXl3ZBcjF4eQwG5GiX0H/jN2MXICxSUllv09wgxhC0us/Pec HaKomUni/8YLbCDNbALaEqf/c4DERQS6mCU6d75jAWlgFljDLLFlLi+ILSzgL/H7VQs7iM0i oCqx8txlsKG8AlYSz29tZoY4Ql+i/74gSJhTwFri85LrYOVCQCWnL8xnhygXlDg58wnUeHmJ 5q2zmScwCsxCkpqFJLWAkWkVo2RqQXFuem6xYYFRXmq5XnFibnFpXrpecn7uJkZwXGtp7WDc s+qD3iFGJg7GQ4wSHMxKIry5alypQrwpiZVVqUX58UWlOanFhxilOViUxHm/ve5NERJITyxJ zU5NLUgtgskycXBKNTCdXLO5IzL8R+xOoeWTYqf3y1taay0Rb4xPL3Hd6vxpsljgTp3buy8y Mc/XnVkrsKOUefO8n/d+3NRRObAi9vdh7itHfqYeSzY6FinpZ1ajKL3qGcO6WdcLjA6r1mvc r37jfGkz781zl4//1/RYaeIZd8DoSqH5H3XW/0XnfGZYNDiUshvllcz97DLTZXVpYGHwLfNq /bmpYR6mB+R/dqk9yoltCbc6K2N98sVyvYWpOpZ2YVenOcvyXD307pP95a1fertj1Aok6ssE NhqwMO4o2Ry6Re6weQjjD9Pg0/9UK/JEr6RUGl+4bsXxfrOH6uTgb1t+Gz3TZwhjlrYM1Ds0 0cfufej1x8+//dXeKanEUpyRaKjFXFScCADPl5/zWgMAAA== X-CMS-MailID: 20230920081539epcas5p23035fe3e03eeb7848a40aff580e5cdbf X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081539epcas5p23035fe3e03eeb7848a40aff580e5cdbf References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add support for handling nvme_cmd_copy command on target. For bdev-ns if backing device supports copy offload we call device copy offload (blkdev_copy_offload). In case of absence of device copy offload capability, we use copy emulation (blkdev_copy_emulation) For file-ns we call vfs_copy_file_range to service our request. Currently target always shows copy capability by setting NVME_CTRL_ONCS_COPY in controller ONCS. loop target has copy support, which can be used to test copy offload. trace event support for nvme_cmd_copy. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta --- drivers/nvme/target/admin-cmd.c | 9 +++- drivers/nvme/target/io-cmd-bdev.c | 71 +++++++++++++++++++++++++++++++ drivers/nvme/target/io-cmd-file.c | 50 ++++++++++++++++++++++ drivers/nvme/target/nvmet.h | 1 + drivers/nvme/target/trace.c | 19 +++++++++ 5 files changed, 148 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c index 39cb570f833d..4e1a6ca09937 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -433,8 +433,7 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req) id->nn = cpu_to_le32(NVMET_MAX_NAMESPACES); id->mnan = cpu_to_le32(NVMET_MAX_NAMESPACES); id->oncs = cpu_to_le16(NVME_CTRL_ONCS_DSM | - NVME_CTRL_ONCS_WRITE_ZEROES); - + NVME_CTRL_ONCS_WRITE_ZEROES | NVME_CTRL_ONCS_COPY); /* XXX: don't report vwc if the underlying device is write through */ id->vwc = NVME_CTRL_VWC_PRESENT; @@ -536,6 +535,12 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req) if (req->ns->bdev) nvmet_bdev_set_limits(req->ns->bdev, id); + else { + id->msrc = (__force u8)to0based(BIO_MAX_VECS - 1); + id->mssrl = cpu_to_le16(BIO_MAX_VECS << + (PAGE_SHIFT - SECTOR_SHIFT)); + id->mcl = cpu_to_le32(le16_to_cpu(id->mssrl)); + } /* * We just provide a single LBA format that matches what the diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c index 468833675cc9..2d5cef6788be 100644 --- a/drivers/nvme/target/io-cmd-bdev.c +++ b/drivers/nvme/target/io-cmd-bdev.c @@ -46,6 +46,18 @@ void nvmet_bdev_set_limits(struct block_device *bdev, struct nvme_id_ns *id) id->npda = id->npdg; /* NOWS = Namespace Optimal Write Size */ id->nows = to0based(bdev_io_opt(bdev) / bdev_logical_block_size(bdev)); + + if (bdev_max_copy_sectors(bdev)) { + id->msrc = id->msrc; + id->mssrl = cpu_to_le16((bdev_max_copy_sectors(bdev) << + SECTOR_SHIFT) / bdev_logical_block_size(bdev)); + id->mcl = cpu_to_le32((__force u32)id->mssrl); + } else { + id->msrc = (__force u8)to0based(BIO_MAX_VECS - 1); + id->mssrl = cpu_to_le16((BIO_MAX_VECS << PAGE_SHIFT) / + bdev_logical_block_size(bdev)); + id->mcl = cpu_to_le32((__force u32)id->mssrl); + } } void nvmet_bdev_ns_disable(struct nvmet_ns *ns) @@ -449,6 +461,61 @@ static void nvmet_bdev_execute_write_zeroes(struct nvmet_req *req) } } +static void nvmet_bdev_copy_endio(void *private, int status, + ssize_t copied) +{ + struct nvmet_req *rq = (struct nvmet_req *)private; + u16 nvme_status; + + if (copied == rq->copy_len) + rq->cqe->result.u32 = cpu_to_le32(1); + else + rq->cqe->result.u32 = cpu_to_le32(0); + + nvme_status = errno_to_nvme_status(rq, status); + nvmet_req_complete(rq, nvme_status); +} + +/* + * At present we handle only one range entry, since copy offload is aligned with + * copy_file_range, only one entry is passed from block layer. + */ +static void nvmet_bdev_execute_copy(struct nvmet_req *rq) +{ + struct nvme_copy_range range; + struct nvme_command *cmd = rq->cmd; + ssize_t ret; + off_t dst, src; + + u16 status; + + status = nvmet_copy_from_sgl(rq, 0, &range, sizeof(range)); + if (status) + goto err_rq_complete; + + dst = le64_to_cpu(cmd->copy.sdlba) << rq->ns->blksize_shift; + src = le64_to_cpu(range.slba) << rq->ns->blksize_shift; + rq->copy_len = (range.nlb + 1) << rq->ns->blksize_shift; + + if (bdev_max_copy_sectors(rq->ns->bdev)) { + ret = blkdev_copy_offload(rq->ns->bdev, dst, src, rq->copy_len, + nvmet_bdev_copy_endio, + (void *)rq, GFP_KERNEL); + } else { + ret = blkdev_copy_emulation(rq->ns->bdev, dst, + rq->ns->bdev, src, rq->copy_len, + nvmet_bdev_copy_endio, + (void *)rq, GFP_KERNEL); + } + if (ret == -EIOCBQUEUED) + return; + + rq->cqe->result.u32 = cpu_to_le32(0); + status = blk_to_nvme_status(rq, ret); +err_rq_complete: + nvmet_req_complete(rq, status); +} + u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req) { switch (req->cmd->common.opcode) { @@ -467,6 +534,10 @@ u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req) case nvme_cmd_write_zeroes: req->execute = nvmet_bdev_execute_write_zeroes; return 0; + case nvme_cmd_copy: + req->execute = nvmet_bdev_execute_copy; + return 0; + default: return nvmet_report_invalid_opcode(req); } diff --git a/drivers/nvme/target/io-cmd-file.c b/drivers/nvme/target/io-cmd-file.c index 2d068439b129..4524cfffa4c6 100644 --- a/drivers/nvme/target/io-cmd-file.c +++ b/drivers/nvme/target/io-cmd-file.c @@ -322,6 +322,47 @@ static void nvmet_file_dsm_work(struct work_struct *w) } } +static void nvmet_file_copy_work(struct work_struct *w) +{ + struct nvmet_req *req = container_of(w, struct nvmet_req, f.work); + int nr_range = req->cmd->copy.nr_range + 1; + u16 status = 0; + int src, id; + ssize_t len, ret; + loff_t pos; + + pos = le64_to_cpu(req->cmd->copy.sdlba) << req->ns->blksize_shift; + if (unlikely(pos + req->transfer_len > req->ns->size)) { + nvmet_req_complete(req, errno_to_nvme_status(req, -ENOSPC)); + return; + } + + for (id = 0 ; id < nr_range; id++) { + struct nvme_copy_range range; + + status = nvmet_copy_from_sgl(req, id * sizeof(range), &range, + sizeof(range)); + if (status) + break; + + src = (le64_to_cpu(range.slba) << (req->ns->blksize_shift)); + len = (le16_to_cpu(range.nlb) + 1) << (req->ns->blksize_shift); + ret = vfs_copy_file_range(req->ns->file, src, req->ns->file, + pos, len, 0); + pos += ret; + if (ret != len) { + req->cqe->result.u32 = cpu_to_le32(id); + if (ret < 0) + status = errno_to_nvme_status(req, ret); + else + status = errno_to_nvme_status(req, -EIO); + break; + } + } + + nvmet_req_complete(req, status); +} + static void nvmet_file_execute_dsm(struct nvmet_req *req) { if (!nvmet_check_data_len_lte(req, nvmet_dsm_len(req))) @@ -330,6 +371,12 @@ static void nvmet_file_execute_dsm(struct nvmet_req *req) queue_work(nvmet_wq, &req->f.work); } +static void nvmet_file_execute_copy(struct nvmet_req *req) +{ + INIT_WORK(&req->f.work, nvmet_file_copy_work); + queue_work(nvmet_wq, &req->f.work); +} + static void nvmet_file_write_zeroes_work(struct work_struct *w) { struct nvmet_req *req = container_of(w, struct nvmet_req, f.work); @@ -376,6 +423,9 @@ u16 nvmet_file_parse_io_cmd(struct nvmet_req *req) case nvme_cmd_write_zeroes: req->execute = nvmet_file_execute_write_zeroes; return 0; + case nvme_cmd_copy: + req->execute = nvmet_file_execute_copy; + return 0; default: return nvmet_report_invalid_opcode(req); } diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h index 8cfd60f3b564..395f3af28413 100644 --- a/drivers/nvme/target/nvmet.h +++ b/drivers/nvme/target/nvmet.h @@ -393,6 +393,7 @@ struct nvmet_req { struct device *p2p_client; u16 error_loc; u64 error_slba; + size_t copy_len; }; #define NVMET_MAX_MPOOL_BVEC 16 diff --git a/drivers/nvme/target/trace.c b/drivers/nvme/target/trace.c index bff454d46255..551fdf029381 100644 --- a/drivers/nvme/target/trace.c +++ b/drivers/nvme/target/trace.c @@ -92,6 +92,23 @@ static const char *nvmet_trace_dsm(struct trace_seq *p, u8 *cdw10) return ret; } +static const char *nvmet_trace_copy(struct trace_seq *p, u8 *cdw10) +{ + const char *ret = trace_seq_buffer_ptr(p); + u64 sdlba = get_unaligned_le64(cdw10); + u8 nr_range = get_unaligned_le16(cdw10 + 8); + u16 control = get_unaligned_le16(cdw10 + 10); + u32 dsmgmt = get_unaligned_le32(cdw10 + 12); + u32 reftag = get_unaligned_le32(cdw10 + 16); + + trace_seq_printf(p, + "sdlba=%llu, nr_range=%u, ctrl=1x%x, dsmgmt=%u, reftag=%u", + sdlba, nr_range, control, dsmgmt, reftag); + trace_seq_putc(p, 0); + + return ret; +} + static const char *nvmet_trace_common(struct trace_seq *p, u8 *cdw10) { const char *ret = trace_seq_buffer_ptr(p); @@ -129,6 +146,8 @@ const char *nvmet_trace_parse_nvm_cmd(struct trace_seq *p, return nvmet_trace_read_write(p, cdw10); case nvme_cmd_dsm: return nvmet_trace_dsm(p, cdw10); + case nvme_cmd_copy: + return nvmet_trace_copy(p, cdw10); default: return nvmet_trace_common(p, cdw10); } From patchwork Wed Sep 20 08:07:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45D07CE79B2 for ; Wed, 20 Sep 2023 08:59:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233979AbjITI7z (ORCPT ); Wed, 20 Sep 2023 04:59:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234055AbjITI7W (ORCPT ); Wed, 20 Sep 2023 04:59:22 -0400 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1AE32CC4 for ; Wed, 20 Sep 2023 01:58:43 -0700 (PDT) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20230920085841epoutp0419f890c65abcdf9914009904efb0b69f~Gj7XrvKEj1265712657epoutp04U for ; Wed, 20 Sep 2023 08:58:41 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20230920085841epoutp0419f890c65abcdf9914009904efb0b69f~Gj7XrvKEj1265712657epoutp04U DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200321; bh=zohq6qFcyNQPlL9CW1gp72Nbdr99RwoaaLJ8/DxbinM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gjRORKPKWwdNEX125R3gRx6iZHENFQzfHk9BtRzQ75h7biE8KXgwu216qFOcN4vtp 9CqQ902AwVy9J2qRXnOfxDkVr1EneweJfV+YxNQf2mJvckwIScbtWUYR1rPj757rN8 IKzeCIXZE7QkuNuSpO6hKSCuNFkjKw/GOKR97ejA= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20230920085840epcas5p3beb33fd273b4b923788b1dad210b43b3~Gj7W4-4Zz1154211542epcas5p3e; Wed, 20 Sep 2023 08:58:40 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.176]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4RrCB66xRDz4x9Q9; Wed, 20 Sep 2023 08:58:38 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id 54.F1.19094.E34BA056; Wed, 20 Sep 2023 17:58:38 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20230920081548epcas5p14d9620a58744270e4f31a7cea4eec920~GjV7u96b42942529425epcas5p1j; Wed, 20 Sep 2023 08:15:48 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20230920081548epsmtrp2376c601f064748387c98462119b603fd~GjV7t1bN80262802628epsmtrp2U; Wed, 20 Sep 2023 08:15:48 +0000 (GMT) X-AuditID: b6c32a50-39fff70000004a96-c7-650ab43e328d Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 56.0F.08742.43AAA056; Wed, 20 Sep 2023 17:15:48 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081545epsmtip11c54e63139d09068832067b7cda16d8d~GjV4v7hjY0186201862epsmtip1g; Wed, 20 Sep 2023 08:15:45 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 09/12] dm: Add support for copy offload Date: Wed, 20 Sep 2023 13:37:46 +0530 Message-Id: <20230920080756.11919-10-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Tf1CTZRzned/t3eBYvQIeDyuTRtIJARtt86GA5CJ6T6vj8q7rvJR28ArE 2Hb7RYkZIr/EY5tYmZsgHRwgeIATaAJLghKHISE4YIF2OqxzByqkZYi0uVH+9/l8nu+vz/e5 LxsPKmdx2bkyNa2USaQ8IoDRPbg5Kia5M4DmG1xi1D58AUf37i8zUOusnkCuwUWAnP3lAFkX TEw03X8OQ6daf8JQ9YAdoLmrRgxZHdHo27IGBuqz2hhovOcEgU42zrHQ4UkLgZqGHmNoyjAH kMV5AKDu5ZM4anPdYaCLjufQ6MoQc2sodc44y6JGr51hUOMjGsrccoigzjZ8QfVOFxFUve4o k6o6uEBQ9+YcDOrO91cJStfZAqgl8wuU2TmPpXN25iXm0JIsWhlOyzLlWbmy7CTe9h0Zb2aI xHxBjCABbeGFyyT5dBIv9Z30mLRcqds6L1wrkWrcUrpEpeLFJScq5Ro1HZ4jV6mTeLQiS6oQ KmJVknyVRpYdK6PVrwn4/HiRO/DjvJyKywu44sewT0/fHMOLwK2QSuDPhqQQ3lw0MSpBADuI 7AOw97ti3EsWAXSV9PvIAwCr9GVgLaX+ktH3YAVw0mXCvKQUg4biGncUm02Q0fDSKtujh5BF OOzorQcegpPTGBw53415SgWTr8PqGjPuwQxyEyy12ZgezHHrh1ZLMU8hSMZB/fV1HtnfLS81 TLK8Ieug7biT4cE4uREe7DI9mQiSJn9o+MPC9I6aCvVlJsKLg+HtoU6WF3Ph0oLVpxfAU182 E97kEgCNk0afzzdg6bAe9wyBk5the0+cV94Avxpuw7yNn4FVy07Mq3OgpXYNR8DT7XW++mHQ /tcBH6ag/vwV37Z1AK40d7IMINz4lCHjU4aM/7euA3gL4NIKVX42nSlSCGJkdMF/H50pzzeD JxcRlW4BrR0rsQMAY4MBANk4L4STHxlAB3GyJJ/tpZXyDKVGSqsGgMi98SM4d32m3H1SMnWG QJjAF4rFYmHCq2IBL5TjKq3JCiKzJWo6j6YVtHItD2P7c4uwFwlp1bHlfYXFZ2+32WWRh1nV jyyz1ydCTrylOsPc3wAujD1Mfkm9u7Bx/ed/b3lIvB0dEVIxvi/l0YYH5Zf70V3/yOniXXVU 49Z5P9eex99ENJekOXfurdVpfjGPT6Tx/T6o3b87UTLbMq+Nabo2Y4vfKKXnqeP/3GrSKuX6 EW3Xxe2y+yVax5VuZsHyR4N+rbB2suUVaUWS6909Myk67ScVP3+t+XPGeoM71rfY9NvLO+zj gUdsol14qrrDVngj7Af7VCIdqHs2vtLataQ7Jvowc1teT9iKflNg6GhFeav9/cG6bVNRsbXP dx2V/7oaLnQkTvxuCSL9gg1xtCNh4m7KezyGKkciiMKVKsm/YnSQgZoEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA02Ra0hTcRjG+59zdnZczU5T2N8Mg4VCSaZ0+2Mp9aE8fZHKCLpArTyZ5Oba 1CysJqKlecMiaJqXWpabuVqpM1uJpnMus6g0l3ahbYrS0ixJp1tOCfr2vM/veZ/3w0vhgmZi OZUoTWHlUnGSiOQRDW2ioLUbNDw2vN2wAum6OnA0/ttFIO1gEYlG234CZGu5BJDRWcpB/S1N GKrRtmOopLUXIPt7FYaM1lBUlaMm0FOjmUBvn5SRqKLazkVX+gwkumtyY+hDsR0ggy0ToAZX BY7qRn8QqNMaiHpmTZxtQqZJNchlej49JJi33amMXpNLMo/UF5nmfiXJ3C68ymEKspwkM263 EsyPZ+9JpvCxBjAT+iBGb/uO7eYf5G2NZ5MS01j5uuijvJOXXzlx2YuA9Npvb3AlcPjnAR8K 0hvgbYsKzwM8SkA3A+jKe85ZAAGwevYFvqD9YI17iLsQysJgf3XHHKAokg6FFg/l9f3pPBzm NjkJ74DTdgyOT00C77YfvQWW3NTPNxF0MMw2m+cv8Of8XE825i2C9DpY9HmZ1/aZsyfUfVyv FtCR0PK6grsQXwbNN2yEV+P0SphVX4oXA1r1H1L9hyoBpgEBrEwhSZAoImQRUvZMmEIsUaRK E8KOJ0v0YP6/a1YbQKNmLKwVYBRoBZDCRf58SQiPFfDjxWfPsfLkI/LUJFbRCgIpQiTkC4cL 4gV0gjiFPcWyMlb+j2KUz3Il9mAix2+XK1KzNebCAXdy13SHNi0GO+bDGksTu4FHMirMeNOx qXe71JmTxrk/5jaFzCxh6PrUrj2O9NDfv7Cv6/dFTo1xSCz9cXRdzZdHlwe6PpuU11tehgt0 K0umzwV79sYYlSqHMmrHoaNZnYz12oW+WJG/aSg8LmT0mi528UB5pTm7Nj8luaJQ8t0t6R35 GWWrPFHf/mm3LqApKUNmbL8Xl88b/tIYNpj2x+Kr3n5w063IjQMjZewp+1DnZHTsDK/6252Z 4jKL53hBprVBiC1ShsvPb5ad1jYsDR5eX9DmUAb5uoodq6oO7NfWfqxrrIq+eqVw5zvd4YeT bkO5iFCcFEesweUK8V93BZtWTgMAAA== X-CMS-MailID: 20230920081548epcas5p14d9620a58744270e4f31a7cea4eec920 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081548epcas5p14d9620a58744270e4f31a7cea4eec920 References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Before enabling copy for dm target, check if underlying devices and dm target support copy. Avoid split happening inside dm target. Fail early if the request needs split, currently splitting copy request is not supported. Signed-off-by: Nitesh Shetty --- drivers/md/dm-table.c | 37 +++++++++++++++++++++++++++++++++++ drivers/md/dm.c | 7 +++++++ include/linux/device-mapper.h | 3 +++ 3 files changed, 47 insertions(+) diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index 37b48f63ae6a..8803c351624c 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -1878,6 +1878,38 @@ static bool dm_table_supports_nowait(struct dm_table *t) return true; } +static int device_not_copy_capable(struct dm_target *ti, struct dm_dev *dev, + sector_t start, sector_t len, void *data) +{ + struct request_queue *q = bdev_get_queue(dev->bdev); + + return !q->limits.max_copy_sectors; +} + +static bool dm_table_supports_copy(struct dm_table *t) +{ + struct dm_target *ti; + unsigned int i; + + for (i = 0; i < t->num_targets; i++) { + ti = dm_table_get_target(t, i); + + if (!ti->copy_offload_supported) + return false; + + /* + * target provides copy support (as implied by setting + * 'copy_offload_supported') + * and it relies on _all_ data devices having copy support. + */ + if (!ti->type->iterate_devices || + ti->type->iterate_devices(ti, device_not_copy_capable, NULL)) + return false; + } + + return true; +} + static int device_not_discard_capable(struct dm_target *ti, struct dm_dev *dev, sector_t start, sector_t len, void *data) { @@ -1960,6 +1992,11 @@ int dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, q->limits.discard_misaligned = 0; } + if (!dm_table_supports_copy(t)) { + q->limits.max_copy_sectors = 0; + q->limits.max_copy_hw_sectors = 0; + } + if (!dm_table_supports_secure_erase(t)) q->limits.max_secure_erase_sectors = 0; diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 64a1f306c96c..eca336487d44 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1714,6 +1714,13 @@ static blk_status_t __split_and_process_bio(struct clone_info *ci) if (unlikely(ci->is_abnormal_io)) return __process_abnormal_io(ci, ti); + if ((unlikely(op_is_copy(ci->bio->bi_opf)) && + max_io_len(ti, ci->sector) < ci->sector_count)) { + DMERR("Error, IO size(%u) > max target size(%llu)\n", + ci->sector_count, max_io_len(ti, ci->sector)); + return BLK_STS_IOERR; + } + /* * Only support bio polling for normal IO, and the target io is * exactly inside the dm_io instance (verified in dm_poll_dm_io) diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index 69d0435c7ebb..98db52d1c773 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -396,6 +396,9 @@ struct dm_target { * bio_set_dev(). NOTE: ideally a target should _not_ need this. */ bool needs_bio_set_dev:1; + + /* copy offload is supported */ + bool copy_offload_supported:1; }; void *dm_per_bio_data(struct bio *bio, size_t data_size); From patchwork Wed Sep 20 08:07:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392378 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C1A6CE79AF for ; Wed, 20 Sep 2023 09:20:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233976AbjITJUa (ORCPT ); Wed, 20 Sep 2023 05:20:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234067AbjITI7W (ORCPT ); Wed, 20 Sep 2023 04:59:22 -0400 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8ACC1CDB for ; Wed, 20 Sep 2023 01:58:47 -0700 (PDT) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20230920085846epoutp013b4d816eeb3cef27bf76fdb70fd979a4~Gj7cH05oA3066430664epoutp01y for ; Wed, 20 Sep 2023 08:58:46 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20230920085846epoutp013b4d816eeb3cef27bf76fdb70fd979a4~Gj7cH05oA3066430664epoutp01y DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200326; bh=W6Cvqjs8GJhwgBfxrn3JFatd+tYHBlqUZSV4Q7H1XFM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rhSdUafas7MKfZWxAVaZrdJW9z/GRnmgnd/JWKZX7QvgmLkXAe05qYbvlHt4R+pB+ 0i+0T8BYbS2yTscOHtdOn6l2w3tuMHybrBvQUsOzq1+C/knJhlJHg3uFtN4/ccB2GN 6wHDSM408sc+ZlqsHeNulT15Z127uJXCuu9jggPI= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20230920085845epcas5p396c04043dede83074b0dd1971a85635e~Gj7bg7d9g1154211542epcas5p3r; Wed, 20 Sep 2023 08:58:45 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.174]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4RrCBB2hH0z4x9Q6; Wed, 20 Sep 2023 08:58:42 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id F2.FA.09638.244BA056; Wed, 20 Sep 2023 17:58:42 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20230920081558epcas5p4d791ba4be2836264a6fb44e06e636d11~GjWEhY0AW3181231812epcas5p41; Wed, 20 Sep 2023 08:15:58 +0000 (GMT) Received: from epsmgmcp1.samsung.com (unknown [182.195.42.82]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20230920081558epsmtrp104f414d8759ea58100910ec961341dab~GjWEgcbpy2250122501epsmtrp14; Wed, 20 Sep 2023 08:15:58 +0000 (GMT) X-AuditID: b6c32a4a-6d5ff700000025a6-b1-650ab4426b85 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgmcp1.samsung.com (Symantec Messaging Gateway) with SMTP id DC.10.18916.D3AAA056; Wed, 20 Sep 2023 17:15:57 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081554epsmtip10685e21725660c9296a7bfa72629e389~GjWBgYFls2849128491epsmtip1e; Wed, 20 Sep 2023 08:15:54 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 10/12] dm: Enable copy offload for dm-linear target Date: Wed, 20 Sep 2023 13:37:47 +0530 Message-Id: <20230920080756.11919-11-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Tf1CTdRi/7/uOdxvn6HVAfF1SuI5qePyYMfySUPzw6L3wD87OK82gxd74 uR9uLJCua4DIyWQThMohCIImUxgBcRuIrREBI49LggGm6AGnRQgyOyMubWNo/vd5Ps/neT7f 5/new8K5WiaPlSXLo5UycS6f8GZ09wsEoQld3nSE/bYPMtl/wtH9v9YY6OINPYEW+lcAmrOW AdR3r9YLTVktGLp8tgpDLRcHMFRlmwBoftyAob7p7ajxaDMDXe4bZqCxntMEOnN+nom0DjOB vhl8hKHJE/MAmeeKAOpeO4OjtoUlBhqafgGN/jvoFQcpi+EGkxq9+S2DGruqpjqMxwiqs/kL qndKQ1BNupNeVEXJPYK6Pz/NoJaujBOUrssIqM6Rzyhnx4tUx9wiluJzICcmkxZLaGUQLUuX S7JkGbH85HfTEtNEURHCUGE02skPkomldCx/956U0KSsXNcO+EGfinPVLipFrFLxw9+MUcrV eXRQplyVF8unFZJcRaQiTCWWqtSyjDAZnfeGMCJih8gl/Cgn03rXiinavAq+vl4NNMDKKAds FiQjYcOXQ1g58GZxyV4AzdX1hDvBJVcAtIztfYrLzsWUA9Z6wZ3ZRI/eAuCKa5ueoBSDhpvd hFtEkNvhyGOWm/cjNThs720C7gAnVzFYcmEIuLv6km/D840DTDdmkMFw2fGA6S7mkLtg36VM j1k41M9sdivYLtbZ7FhXc8jNcPjU3PoAOPkSLPmuFne3h2QtG94p0m9Mthv+2FNOeLAv/GOw i+nBPPi7/ugGzoct1RcIT/ERAA0OA/Ak3oKldj3ufgROCqCpJ9xDB8IaexvmMfaBFWtzmIfn QHP9E/wyvGRq2PDdAiceFm1gCtZVOhmeZekAPNd+nDgBggzPDGR4ZiDD/9YNADeCLbRCJc2g VSLFDhmd//SP0+XSDrB+FSHvmMHtW8thNoCxgA1AFs7340hf8aa5HIn4cCGtlKcp1bm0ygZE rn1X4jz/dLnrrGR5acLI6IjIqKioyOjXo4T8AM5CaZ2ES2aI8+gcmlbQyid1GIvN02A7JyzH nFPD2f0HjbNZna8datE+TGo1Haqkry0WFsfsqcvZpEqXjgm2hZ9aUByPXjXe4sz6av8+sFys SxJkBxxM3dsS2Lg1ePGx7JNQTeimw5b8WX9YR1Rp4wTf7xMnjKtTy2qzavJnrkiSgu8GaCZD Om2G+LL4uNY/Nc5tkzqN9fMG3ujPPe0//HL1VeuumBl2irW0rlf0YOmapXWkouv6o/hehpX/ YZMkf7lKaTLzCoxKbs3JgXb5jMMemLxv3LTaL1u0J9RPsIF/9v7k92y/Zow0j49Nbi3/WJRe nKiTnNbZAhvS/3muwKn9Ch7R6EVFH7y/7OfY/3xhaj3x29kePkOVKRaG4EqV+D9/tfJVngQA AA== X-Brightmail-Tracker: H4sIAAAAAAAAA02Re0hTYRjG+c45nnNcrI7a6ktNa1HiJGtg9jEviAV9QlBJUBhkQw9abVM3 jW7UUkq2yFK72LosS6xmLbIsZ86Gl9SWaZqZihWorbKclyxCtDpK0H+/97nAAy9LetspX3a3 JpPXapQqKS2iHtVJA1ZGWUT86n5XBLr3/BmJRicmKVTWd5pGQ3VjAA04cgGyD1/yQN0OG4Gq rxcQ6HZZA4EKat8ANNhpIpC9JwQVnyihULW9mUIdVZdpZC4dZNDJrkoa3WycJtDbM4MAVQ4c A+jRpJlE1iE3hZp6/FDrVKNHDMQ2Ux+DW9/dp3BHSxYutxho/KDkKH7SrafxjbxCD3wqZ5jG o4M9FHbXdNI476EF4AfOQ3i8PACXD3wjNs9NEEUm86rd+3jtquhdolTHJweRbvXYX9R7FuiB gzICloVcGHT1rzMCT9abewxgmzNDYMgtgqVT9eQs+8Db0y7GCER/MzkEfGobBUKX5kKg8zcr 6PM5IwkNtmFKOEgul4TOL0OE0PbhNsDS4gZGYIpbDke6vjNCWcxFQPud1NkNq+Dp915CwvOv Ol7SxczuUUBnm3mGxZwXbL44QAlMcoEwp+ISeQZwpv8s03/WNUBYgIRP16lT1Enp8lCdUq3L 0qSEJqWpy8HMh2XxlaD03lRoLSBYUAsgS0rni9UrRLy3OFl54CCvTUvUZql4XS3wYynpQvEy lSHZm0tRZvJ7eT6d1/5zCdbTV0/oblmbcQM4srX9sWa9LMwn6XDGyA6v3pgyoNmW9+GLoabn fJpFuvOCbGnwj2pXvgLb50T3J1TYX/jO+fnUX1O0xc8cl3o9/H74gslEvL1JbmttiL5SfO3q 8u6NCrzWLbFuXpK9eFEQHxyxz72mcB4v2WR1mxm9KyrwZfhnkdwYFWhNKFydlz8RL3lR9Spu ewhj+CjpaHFGBPnHBtZlxW46Nx3a2uLQhEX+2lAzz1Xfe76t/n1jX5Ao5xZjatcZ4tCR3HjF 7+Cmg3L/6bfZmQsnrpirr1486TOmb3/zKjv+W9X6rsjXI/mqgAIUu+WwomjPV/9TNccrS+6W PfGTObRSSpeqlMtIrU75B4DsOJ1QAwAA X-CMS-MailID: 20230920081558epcas5p4d791ba4be2836264a6fb44e06e636d11 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081558epcas5p4d791ba4be2836264a6fb44e06e636d11 References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Setting copy_offload_supported flag to enable offload. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty --- drivers/md/dm-linear.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c index f4448d520ee9..1d1ee30bbefb 100644 --- a/drivers/md/dm-linear.c +++ b/drivers/md/dm-linear.c @@ -62,6 +62,7 @@ static int linear_ctr(struct dm_target *ti, unsigned int argc, char **argv) ti->num_discard_bios = 1; ti->num_secure_erase_bios = 1; ti->num_write_zeroes_bios = 1; + ti->copy_offload_supported = 1; ti->private = lc; return 0; From patchwork Wed Sep 20 08:07:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92554CE79AC for ; Wed, 20 Sep 2023 09:20:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233917AbjITJU3 (ORCPT ); Wed, 20 Sep 2023 05:20:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51924 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234109AbjITI7Y (ORCPT ); Wed, 20 Sep 2023 04:59:24 -0400 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F89BCF5 for ; Wed, 20 Sep 2023 01:58:49 -0700 (PDT) Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20230920085847epoutp023256d9dbe39bf91947d5d03d53f90219~Gj7d2Wgts2614126141epoutp02L for ; Wed, 20 Sep 2023 08:58:47 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20230920085847epoutp023256d9dbe39bf91947d5d03d53f90219~Gj7d2Wgts2614126141epoutp02L DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200327; bh=ddG+iMTBPykY7aem2/d5RFRU6pD1r9HGia7WdJCIOGI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DDz+UnGQ423tla/xax0Am7BkiBctpNH0PSSEIrrsqww+E9/PFRuFi4ucoK8X0Itr+ aQfB6ZkCwrcROpU4xr9V5jq1TmIPUv/C375JUC9DFR/iKdVFtwqKx7E9KshHRiC4t9 KcmQeS+1Hjn5GvnXIPd6DGcARzM4NDYuoqtfacB0= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20230920085847epcas5p2cc008ef9a9ded85cafc0c033339026f3~Gj7dRA2k72273522735epcas5p2I; Wed, 20 Sep 2023 08:58:47 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.178]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4RrCBF49fpz4x9Pw; Wed, 20 Sep 2023 08:58:45 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 38.FA.09638.544BA056; Wed, 20 Sep 2023 17:58:45 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20230920081607epcas5p3def8c43668f1bfb5555b65b93f2f7b6f~GjWNZ-Lby2994129941epcas5p3u; Wed, 20 Sep 2023 08:16:07 +0000 (GMT) Received: from epsmgmc1p1new.samsung.com (unknown [182.195.42.40]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20230920081607epsmtrp1edd5801b2581d477d5ff46bdb38677ce~GjWNY3xBc2250122501epsmtrp1K; Wed, 20 Sep 2023 08:16:07 +0000 (GMT) X-AuditID: b6c32a4a-6d5ff700000025a6-bc-650ab4458474 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgmc1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 15.7C.08649.74AAA056; Wed, 20 Sep 2023 17:16:07 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081604epsmtip16700d2985fe3f82725b430bdd8785f24~GjWKQRg_B3072130721epsmtip1b; Wed, 20 Sep 2023 08:16:04 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 11/12] null: Enable trace capability for null block Date: Wed, 20 Sep 2023 13:37:48 +0530 Message-Id: <20230920080756.11919-12-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Tf1BUVRTH5763vF2IxQdY3d0CaTF1ZVh+uNClJEtR36A4KE32w4IdeMMS y+7O/lDIZoCItTB+CCi0oKCgwFKA/HAWcBkDFBYzGo3fktosjYVCSEKyrcby1vK/z/nec+73 nnPncHCPIjafkyTX0Cq5RCYgXFgXeoQb/Le3utCBFeXeqHHgCo7mHlpZ6PMCG47qJ/MJNN3z ACDLpSMAmWbKnNDYpXYMXTxTiKG6+ssYKuweBmhqSI8h07gfOq2rZqGLJjML3egoJ1DFuSk2 OjpiJFBN32MMjRZMAWS0ZAJ0wVqBo4bpWRbqH38JDdr6nN7iUe36STY1+Mt5FnXjmpZqNnxF UC3V6VTnWAZBVeUVOVG5WTMENTc1zqJmu4YIKq/VAKiWq4ep+WZvqtlyH4t2+yB5s5SWJNAq H1oer0hIkieGC3bFxG6LDQkNDPIPCkOvCXzkkhQ6XBCxO9p/R5JseRACn4MSmXZZipao1YKA NzerFFoN7SNVqDXhAlqZIFOKlSK1JEWtlSeK5LTm9aDAwOCQ5cS4ZOlw6RG28ohn6s9N17EM UEzmAGcOJMVwpv28Uw5w4XiQnQDaquYIJngAYOn0GYwJFgDs/f4YyAGclZKOyo8Y3QTgQk+V Iykbg4MWi5M9iSD94NUnHLu+mszAYVNnFbAHOHkKhy2/9mF2c09yJzQ0mVeYRb4K5yf0hJ25 5Btw8K88nHELgPm33O2y87I8Xz3CZlLcofkbC8vOOLkGZrWV4fb7IVnjDO+UXMeY5iJg/+ka B3vCP/pa2Qzz4e/5OgcfgnXFtQRT/AWA+hE9YA62wOyB/JVH4KQQNnYEMLIXPD7QgDHGbjDX anHcz4XGU0/ZF37bWEkwzIPDi5kOpuDdo5OOYecBmDFmYBcAH/0zDemfaUj/v3UlwA2ARyvV KYm0OkQZLKcP/ffN8YqUZrCyHRsjjeDO7T9F3QDjgG4AObhgNTdlnQvtwU2QpH1KqxSxKq2M VneDkOWBH8P5z8crltdLrokNEocFikNDQ8Vhm0KDBC9yp7NPJniQiRINnUzTSlr1tA7jOPMz sB3vjZ6tSreqbm55/LXIVFPaO9q/P+pAu3IiymwW0HVrz5VLS16Im9kZF/kOfa2wnrfftfa3 smzuD97aYK15grfnk1znJwNeUbGKmwUvn1W4RSyBpeLm+8/ZiMsdqdIK8cO4koXZNi/d1hYZ VxdxbyjY2JFW+K6pR1hctHYiZdG2TmTb2rX+yqP4NN9b7vfW+1pjXqk9nL478sSHyul9bNft OfzOz/5RC122lW64u5dz0Dsq5m9hw8eGPfwF4/Evy3OTa1ofiXr7I/cdyAqrO/m262KTtZc4 keq5Spcq/Knlx6E1cZpcD/S+bqors5zjx/uuscFv0yC1qmc6/nZj29LeXQtQwFJLJUEbcZVa 8i9kO7jypgQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA02Rf0yMcRzHfZ/n6bnnzq49rsa3Us1trKFyYvsOYbP0mF/VmM0Yp56l6a5z 1wllIsRJEpkudqXLj4vk+nV3leVCVzeLJP3wK92PrSgUlRSd2Pz32uv9/nz22T4ULmggvKk4 aSIrl4rjhSSPqKwX+gWG63jsosudvuhe0xMcffk2RqDjWeM4Kn5znkR99V8BstWlA1Tbn+eG OuqMGKq5no2h28WPMZRtbgPI/lKNodrOBajglJZANbWNBHphukoizQ07B519ZSDRzYYJDLVn 2QEy2I4BVDmmwVFJ3wCBLJ0+qHm8wW21F2NUv+EwzW/vE8yLp0pGrztDMmXao0x1RyrJFGZe dGPOpfWTzBd7J8EMPHhJMpnlOsCUWZOZQb0fo7d9wiLct/NWxLDxcQdYefDK3by9bVfSObJ0 j4OtpS1YKrhEqwBFQXoJNOXvVAEeJaCrAbQ9zCRUgDvpveCN8Uf4FHvA2xNOzlQpDYMWSytw DZP0Amj9Rbm8J63C4Rlj/59hnL6Dw/JrfBd70OFQV9qIuZig58LBLjXpYj69HDYPZeJTRwTD 8+9muDR3Ug9qX3FcLKCXQeszDWeqPgM25tr+rveHaRV5eBag1f9F6v+ifIDpgBcrU0hiJdEi mUjKJgUpxBKFUhobFJ0g0YM/v54fYABvNRNBZoBRwAwghQs9+ZJ5PFbAjxEfOszKE3bJlfGs wgx8KEI4iy+6khcjoGPFiew+lpWx8n8pRnG9UzHj0qUdo17WVZXPHXPXBieUbMbyF7Ze7uo6 5fQeOpb7I3mkx+HvKFS8i4rkbJasKyf3bDLx2aEBx4Dzalz5oh291YU5Ye3mlkCQJA3Uy4ps rz/73Yo02bldVdy7vh9NllshEUPDPRGziwr7mvI2Coqiuk+023OaYhMIy+jiTRPQMuwTX6dy qC+Eumt1ffePmPcj/63bltQc9TOUKNcgG9Pzff/xXE0vZdxiELa4V3sWp21QwEMeOkPvwowP njdTYlQZVs7M070pBd1VI9N2VSTqwoadIl9qN54xWvqz4c77NbXLp6do80OjUo+0paucJwt8 1jMgJ6BHOWcLHSISEoq9YtF8XK4Q/wadwq4jWgMAAA== X-CMS-MailID: 20230920081607epcas5p3def8c43668f1bfb5555b65b93f2f7b6f X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081607epcas5p3def8c43668f1bfb5555b65b93f2f7b6f References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This is a prep patch to enable copy trace capability. At present only zoned null_block is using trace, so we decoupled trace and zoned dependency to make it usable in null_blk driver also. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta --- drivers/block/null_blk/Makefile | 2 -- drivers/block/null_blk/main.c | 3 +++ drivers/block/null_blk/trace.h | 2 ++ drivers/block/null_blk/zoned.c | 1 - 4 files changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/block/null_blk/Makefile b/drivers/block/null_blk/Makefile index 84c36e512ab8..672adcf0ad24 100644 --- a/drivers/block/null_blk/Makefile +++ b/drivers/block/null_blk/Makefile @@ -5,7 +5,5 @@ ccflags-y += -I$(src) obj-$(CONFIG_BLK_DEV_NULL_BLK) += null_blk.o null_blk-objs := main.o -ifeq ($(CONFIG_BLK_DEV_ZONED), y) null_blk-$(CONFIG_TRACING) += trace.o -endif null_blk-$(CONFIG_BLK_DEV_ZONED) += zoned.o diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index 968090935eb2..c56bef0edc5e 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -11,6 +11,9 @@ #include #include "null_blk.h" +#define CREATE_TRACE_POINTS +#include "trace.h" + #undef pr_fmt #define pr_fmt(fmt) "null_blk: " fmt diff --git a/drivers/block/null_blk/trace.h b/drivers/block/null_blk/trace.h index 6b2b370e786f..91446c34eac2 100644 --- a/drivers/block/null_blk/trace.h +++ b/drivers/block/null_blk/trace.h @@ -30,6 +30,7 @@ static inline void __assign_disk_name(char *name, struct gendisk *disk) } #endif +#ifdef CONFIG_BLK_DEV_ZONED TRACE_EVENT(nullb_zone_op, TP_PROTO(struct nullb_cmd *cmd, unsigned int zone_no, unsigned int zone_cond), @@ -67,6 +68,7 @@ TRACE_EVENT(nullb_report_zones, TP_printk("%s nr_zones=%u", __print_disk_name(__entry->disk), __entry->nr_zones) ); +#endif /* CONFIG_BLK_DEV_ZONED */ #endif /* _TRACE_NULLB_H */ diff --git a/drivers/block/null_blk/zoned.c b/drivers/block/null_blk/zoned.c index 55c5b48bc276..9694461a31a4 100644 --- a/drivers/block/null_blk/zoned.c +++ b/drivers/block/null_blk/zoned.c @@ -3,7 +3,6 @@ #include #include "null_blk.h" -#define CREATE_TRACE_POINTS #include "trace.h" #undef pr_fmt From patchwork Wed Sep 20 08:07:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13392336 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3E80CE79AB for ; Wed, 20 Sep 2023 09:00:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234092AbjITJAT (ORCPT ); Wed, 20 Sep 2023 05:00:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234145AbjITI7o (ORCPT ); Wed, 20 Sep 2023 04:59:44 -0400 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E395E45 for ; Wed, 20 Sep 2023 01:58:53 -0700 (PDT) Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20230920085851epoutp02d7b138118ed724384c76b68f54cc4dcd~Gj7hEaloP2614126141epoutp02N for ; Wed, 20 Sep 2023 08:58:51 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20230920085851epoutp02d7b138118ed724384c76b68f54cc4dcd~Gj7hEaloP2614126141epoutp02N DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1695200331; bh=OJzVFQCb3WofjpZljeXnGBqYi+B32Cvdl/hefyKURJs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nMyqAuA4UwDr+NaeiDWo7LxoZBIJK5AQIax509D3ocyxnzlfQkLGTFBteBUwP3BHJ JLpU68lG08awmjL87B9e4iKEQycySEZHVrUoKSTuacPDp1m0Fk5g8c2q9uvTcx6xqs rhJ6l1A0LHCf8m/Qqm/wwD1v3Dri2WOQrbrZod6E= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20230920085850epcas5p35a345b63acc2fae58848a393cf4bd6df~Gj7gJ5igH1154211542epcas5p30; Wed, 20 Sep 2023 08:58:50 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.176]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4RrCBK0m6Fz4x9Q0; Wed, 20 Sep 2023 08:58:49 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 77.A4.09949.844BA056; Wed, 20 Sep 2023 17:58:48 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20230920081617epcas5p4a0e3d23ed747115202c1d8765f6c5a6f~GjWWziplY2583825838epcas5p4J; Wed, 20 Sep 2023 08:16:17 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20230920081617epsmtrp120a0727694d9fc9bee7bd26f727ede2f~GjWWycog52250122501epsmtrp1Y; Wed, 20 Sep 2023 08:16:17 +0000 (GMT) X-AuditID: b6c32a49-bd9f8700000026dd-d3-650ab4487973 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id D0.1F.08742.15AAA056; Wed, 20 Sep 2023 17:16:17 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230920081613epsmtip13d39ccd65439ff7d195fb43480506a81~GjWTRj7Vl3236232362epsmtip1T; Wed, 20 Sep 2023 08:16:13 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Damien Le Moal , Anuj Gupta , Vincent Fu , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v16 12/12] null_blk: add support for copy offload Date: Wed, 20 Sep 2023 13:37:49 +0530 Message-Id: <20230920080756.11919-13-nj.shetty@samsung.com> X-Mailer: git-send-email 2.35.1.500.gb896f729e2 In-Reply-To: <20230920080756.11919-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Ta1BUZRie75yzZxcm8gg0fUAiLFMpxGUF1o9bajl6FEqK0hlthnbYEzDA 7s4uSJIjtzCCWAhippabweoKKCi3uK3tsBICOUQIBHJZEVACQUETQbRdD5T/nud5L8/7vt98 PNyylGvLi5TEMnKJKJpPmhMN+u3bXOk6c8ZjZdEHVXf9hqMHj1YJlJKzhqPK0WwSzeoXAZrU fQPQ6vUeHGnnCzhoSNeEodbSXAyVV7ZjKLdtAKCpfhWGtMMu6OfTagK1ajsJ1NdcSKKSc1Nc lDnYSCJNxzMM/ZUzBVDjZDJADaslOKqaXSDQtWE7dDszHaCetQ7Obju6STXKpXvGLhN03/U4 uqbiW5KuVSfSLUNJJF2mzOPQWanzJN2UNs6hH0wNE/TClX6SVtZVALq2+yt6qcaerpm8hwVv OhrlH8GIxIzcgZGEScWRkvAAfmBI6Puh3kIPgavAB+3kO0hEMUwAf29QsOu+yGjjXfgOx0XR cUYpWKRQ8N3f9ZdL42IZhwipIjaAz8jE0TIvmZtCFKOIk4S7SZhYX4GHxw5vY+LnURH5310j ZL37vpy+1MtJAlW+GcCMBykveLlkCDdhS6oFwPMTARnA3IgXASxNK+Gw5B8AlUuTxEZFo2oM YwNaAFVZjwmWpGFQr281Eh6PpFxg93OeSbemknB4qaUMmAhOLeOwrKODNLWyovbAi7pbmAkT 1JtwZuzGC92C8oPLd1KBqRGk3GH2+GaTbGaUl9SDXDZlM+z8iZ0Ip7bC1PoC3NQfUnozmH9V g7Gj7oV/rt0nWWwF/+6o47LYFs5kn17H8bD8h/MkW/y1cZ1BFWADu2BaVzZuGgKntsPqZndW 3gLzu6ow1vhVmLU6ue5lARuLN7ATvFB9Zt3XBg48Tl7HNNQUFwP2WkoAH+lq8RzgoHppIdVL C6n+tz4D8Apgw8gUMeGMwlsmkDDx/z1zmDSmBrz4LM4HGsGo4b5bG8B4oA1AHs63toh5y5yx tBCLTiQwcmmoPC6aUbQBb+PBv8dtXwuTGn+bJDZU4OXj4SUUCr18PIUC/usWs2lFYksqXBTL RDGMjJFv1GE8M9skzDPVYPe0O7t+VPqhzaGmvKKdJ32XHRVfbEtr073ykQS2GPymL3x2ldsy kccj54PrMroCyoPmlC6HDZ8eHTrneEypO1JkaNZGCgJ/cazDJs6OOWm1P17hjbwT5KLc9FBY OF/olq1Qa/LePpl+J/62YSWk7yy23Jo8EB6fVP8kjn+3wTox0KporX3wVMMM/+BWTjoR36k5 xbHkpjuvHHs4N6057OZ/8dbHfj79ubqbz9XmETx1xW6OaD/Xfi3lD3vVG7OR7Q1Pj//uQjcd WAgZ3HJXWXAj4cSvoZ/s8tw/MpIQOZSS+Sxw3LbtXsPcIfHURFF9b2J7n34Hs+cIevLezQ8O VjrxCUWESOCMyxWifwG0lAtetQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrOIsWRmVeSWpSXmKPExsWy7bCSnG7gKq5Ug8e75CzWnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFr/Pnme22PtuNqvFzQM7mSz2LJrEZLFy9VEmi0mHrjFa PL06i8li7y1ti4VtS1gs9uw9yWJxedccNov5y56yW3Rf38Fmsfz4PyaLGxOeMlrseNLIaLHt 93xmi3Wv37NYnLglbfG4u4PR4vzf46wO0h47Z91l9zh/byOLx+WzpR6bVnWyeWxeUu+x+2YD m8fivsmsHr3N79g8drbeZ/X4+PQWi8f7fVfZPPq2rGL02Hy62uPzJjmPTU/eMgXwR3HZpKTm ZJalFunbJXBlTO05wVJw0a3i2YaLrA2M66y6GDk5JARMJHbMuscEYgsJ7GaUWLOFCyIuKbHs 7xFmCFtYYuW/5+xdjFxANc1MEl/nH2fsYuTgYBPQljj9nwMkLiLQxSzRufMdC4jDLNDBInFh 2152kG5hAUeJtQcegm1gEVCVeHnvChuIzStgLfHjeTPYIAkBfYn++4IgYU6g8Ocl19khDrKS OH1hPjtEuaDEyZlPWEBsZgF5ieats5knMArMQpKahSS1gJFpFaNkakFxbnpusWGBYV5quV5x Ym5xaV66XnJ+7iZGcNxrae5g3L7qg94hRiYOxkOMEhzMSiK8uWpcqUK8KYmVValF+fFFpTmp xYcYpTlYlMR5xV/0pggJpCeWpGanphakFsFkmTg4pRqYqjzu799m2fxsmT+Xd6D8X6H/nMFz rn8qS1f0UFLgqru96Kpd8jNO34l1/MazsubJP8v+tqBxx8zDi1X3VeZJbeIP6vPdJDf98aY9 d+x+zZ0pohttnXDlz3QP6Sm95b9UpkXv1T6VYlEjdMSpv137r5z0Ag3hFVwlIiG9/ja7szLW /fpqaOPcMnnro57YZ6xS/2b6V0+7nZXd9apda0Hm9JfvHyWerTv/yPr7pJYK+yVhDWIZHXZ2 P1LPGgYfMSv6zCq9vXvpu+uyBdvvWmTdETg2odjCZJ6HakNo0uwi/pifRpGsns4529Nlf/20 uxgjnqZT8SVl6V5W6eNRzqwsbFsckr+2x24QtPeO5FRiKc5INNRiLipOBABtKl5UagMAAA== X-CMS-MailID: 20230920081617epcas5p4a0e3d23ed747115202c1d8765f6c5a6f X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230920081617epcas5p4a0e3d23ed747115202c1d8765f6c5a6f References: <20230920080756.11919-1-nj.shetty@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Implementation is based on existing read and write infrastructure. copy_max_bytes: A new configfs and module parameter is introduced, which can be used to set hardware/driver supported maximum copy limit. Only request based queue mode will support for copy offload. Added tracefs support to copy IO tracing. Reviewed-by: Hannes Reinecke Suggested-by: Damien Le Moal Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty Signed-off-by: Vincent Fu --- Documentation/block/null_blk.rst | 5 ++ drivers/block/null_blk/main.c | 97 ++++++++++++++++++++++++++++++- drivers/block/null_blk/null_blk.h | 1 + drivers/block/null_blk/trace.h | 23 ++++++++ 4 files changed, 123 insertions(+), 3 deletions(-) diff --git a/Documentation/block/null_blk.rst b/Documentation/block/null_blk.rst index 4dd78f24d10a..6153e02fcf13 100644 --- a/Documentation/block/null_blk.rst +++ b/Documentation/block/null_blk.rst @@ -149,3 +149,8 @@ zone_size=[MB]: Default: 256 zone_nr_conv=[nr_conv]: Default: 0 The number of conventional zones to create when block device is zoned. If zone_nr_conv >= nr_zones, it will be reduced to nr_zones - 1. + +copy_max_bytes=[size in bytes]: Default: COPY_MAX_BYTES + A module and configfs parameter which can be used to set hardware/driver + supported maximum copy offload limit. + COPY_MAX_BYTES(=128MB at present) is defined in fs.h diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index c56bef0edc5e..22361f4d5f71 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -160,6 +160,10 @@ static int g_max_sectors; module_param_named(max_sectors, g_max_sectors, int, 0444); MODULE_PARM_DESC(max_sectors, "Maximum size of a command (in 512B sectors)"); +static unsigned long g_copy_max_bytes = BLK_COPY_MAX_BYTES; +module_param_named(copy_max_bytes, g_copy_max_bytes, ulong, 0444); +MODULE_PARM_DESC(copy_max_bytes, "Maximum size of a copy command (in bytes)"); + static unsigned int nr_devices = 1; module_param(nr_devices, uint, 0444); MODULE_PARM_DESC(nr_devices, "Number of devices to register"); @@ -412,6 +416,7 @@ NULLB_DEVICE_ATTR(home_node, uint, NULL); NULLB_DEVICE_ATTR(queue_mode, uint, NULL); NULLB_DEVICE_ATTR(blocksize, uint, NULL); NULLB_DEVICE_ATTR(max_sectors, uint, NULL); +NULLB_DEVICE_ATTR(copy_max_bytes, uint, NULL); NULLB_DEVICE_ATTR(irqmode, uint, NULL); NULLB_DEVICE_ATTR(hw_queue_depth, uint, NULL); NULLB_DEVICE_ATTR(index, uint, NULL); @@ -553,6 +558,7 @@ static struct configfs_attribute *nullb_device_attrs[] = { &nullb_device_attr_queue_mode, &nullb_device_attr_blocksize, &nullb_device_attr_max_sectors, + &nullb_device_attr_copy_max_bytes, &nullb_device_attr_irqmode, &nullb_device_attr_hw_queue_depth, &nullb_device_attr_index, @@ -659,7 +665,8 @@ static ssize_t memb_group_features_show(struct config_item *item, char *page) "poll_queues,power,queue_mode,shared_tag_bitmap,size," "submit_queues,use_per_node_hctx,virt_boundary,zoned," "zone_capacity,zone_max_active,zone_max_open," - "zone_nr_conv,zone_offline,zone_readonly,zone_size\n"); + "zone_nr_conv,zone_offline,zone_readonly,zone_size," + "copy_max_bytes\n"); } CONFIGFS_ATTR_RO(memb_group_, features); @@ -725,6 +732,7 @@ static struct nullb_device *null_alloc_dev(void) dev->queue_mode = g_queue_mode; dev->blocksize = g_bs; dev->max_sectors = g_max_sectors; + dev->copy_max_bytes = g_copy_max_bytes; dev->irqmode = g_irqmode; dev->hw_queue_depth = g_hw_queue_depth; dev->blocking = g_blocking; @@ -1274,6 +1282,81 @@ static int null_transfer(struct nullb *nullb, struct page *page, return err; } +static inline int nullb_setup_copy(struct nullb *nullb, struct request *req, + bool is_fua) +{ + sector_t sector_in = 0, sector_out = 0; + loff_t offset_in, offset_out; + void *in, *out; + ssize_t chunk, rem = 0; + struct bio *bio; + struct nullb_page *t_page_in, *t_page_out; + u16 seg = 1; + int status = -EIO; + + if (blk_rq_nr_phys_segments(req) != BLK_COPY_MAX_SEGMENTS) + return status; + + /* + * First bio contains information about source and last bio contains + * information about destination. + */ + __rq_for_each_bio(bio, req) { + if (seg == blk_rq_nr_phys_segments(req)) { + sector_out = bio->bi_iter.bi_sector; + if (rem != bio->bi_iter.bi_size) + return status; + } else { + sector_in = bio->bi_iter.bi_sector; + rem = bio->bi_iter.bi_size; + } + seg++; + } + + trace_nullb_copy_op(req, sector_out << SECTOR_SHIFT, + sector_in << SECTOR_SHIFT, rem); + + spin_lock_irq(&nullb->lock); + while (rem > 0) { + chunk = min_t(size_t, nullb->dev->blocksize, rem); + offset_in = (sector_in & SECTOR_MASK) << SECTOR_SHIFT; + offset_out = (sector_out & SECTOR_MASK) << SECTOR_SHIFT; + + if (null_cache_active(nullb) && !is_fua) + null_make_cache_space(nullb, PAGE_SIZE); + + t_page_in = null_lookup_page(nullb, sector_in, false, + !null_cache_active(nullb)); + if (!t_page_in) + goto err; + t_page_out = null_insert_page(nullb, sector_out, + !null_cache_active(nullb) || + is_fua); + if (!t_page_out) + goto err; + + in = kmap_local_page(t_page_in->page); + out = kmap_local_page(t_page_out->page); + + memcpy(out + offset_out, in + offset_in, chunk); + kunmap_local(out); + kunmap_local(in); + __set_bit(sector_out & SECTOR_MASK, t_page_out->bitmap); + + if (is_fua) + null_free_sector(nullb, sector_out, true); + + rem -= chunk; + sector_in += chunk >> SECTOR_SHIFT; + sector_out += chunk >> SECTOR_SHIFT; + } + + status = 0; +err: + spin_unlock_irq(&nullb->lock); + return status; +} + static int null_handle_rq(struct nullb_cmd *cmd) { struct request *rq = cmd->rq; @@ -1283,13 +1366,16 @@ static int null_handle_rq(struct nullb_cmd *cmd) sector_t sector = blk_rq_pos(rq); struct req_iterator iter; struct bio_vec bvec; + bool fua = rq->cmd_flags & REQ_FUA; + + if (op_is_copy(req_op(rq))) + return nullb_setup_copy(nullb, rq, fua); spin_lock_irq(&nullb->lock); rq_for_each_segment(bvec, rq, iter) { len = bvec.bv_len; err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset, - op_is_write(req_op(rq)), sector, - rq->cmd_flags & REQ_FUA); + op_is_write(req_op(rq)), sector, fua); if (err) { spin_unlock_irq(&nullb->lock); return err; @@ -2053,6 +2139,9 @@ static int null_validate_conf(struct nullb_device *dev) return -EINVAL; } + if (dev->queue_mode == NULL_Q_BIO) + dev->copy_max_bytes = 0; + return 0; } @@ -2172,6 +2261,8 @@ static int null_add_dev(struct nullb_device *dev) dev->max_sectors = queue_max_hw_sectors(nullb->q); dev->max_sectors = min(dev->max_sectors, BLK_DEF_MAX_SECTORS); blk_queue_max_hw_sectors(nullb->q, dev->max_sectors); + blk_queue_max_copy_hw_sectors(nullb->q, + dev->copy_max_bytes >> SECTOR_SHIFT); if (dev->virt_boundary) blk_queue_virt_boundary(nullb->q, PAGE_SIZE - 1); diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h index 929f659dd255..e82e53a2e2df 100644 --- a/drivers/block/null_blk/null_blk.h +++ b/drivers/block/null_blk/null_blk.h @@ -107,6 +107,7 @@ struct nullb_device { unsigned int queue_mode; /* block interface */ unsigned int blocksize; /* block size */ unsigned int max_sectors; /* Max sectors per command */ + unsigned long copy_max_bytes; /* Max copy offload length in bytes */ unsigned int irqmode; /* IRQ completion handler */ unsigned int hw_queue_depth; /* queue depth */ unsigned int index; /* index of the disk, only valid with a disk */ diff --git a/drivers/block/null_blk/trace.h b/drivers/block/null_blk/trace.h index 91446c34eac2..2f2c1d1c2b48 100644 --- a/drivers/block/null_blk/trace.h +++ b/drivers/block/null_blk/trace.h @@ -70,6 +70,29 @@ TRACE_EVENT(nullb_report_zones, ); #endif /* CONFIG_BLK_DEV_ZONED */ +TRACE_EVENT(nullb_copy_op, + TP_PROTO(struct request *req, + sector_t dst, sector_t src, size_t len), + TP_ARGS(req, dst, src, len), + TP_STRUCT__entry( + __array(char, disk, DISK_NAME_LEN) + __field(enum req_op, op) + __field(sector_t, dst) + __field(sector_t, src) + __field(size_t, len) + ), + TP_fast_assign( + __entry->op = req_op(req); + __assign_disk_name(__entry->disk, req->q->disk); + __entry->dst = dst; + __entry->src = src; + __entry->len = len; + ), + TP_printk("%s req=%-15s: dst=%llu, src=%llu, len=%lu", + __print_disk_name(__entry->disk), + blk_op_str(__entry->op), + __entry->dst, __entry->src, __entry->len) +); #endif /* _TRACE_NULLB_H */ #undef TRACE_INCLUDE_PATH