From patchwork Mon May 20 10:20:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668226 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D5F0554735 for ; Mon, 20 May 2024 11:44:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.33 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205483; cv=none; b=mXBtFaxOYbUS9qcAtOCuZJGscM+YwuNYH9mgHsnhjh/465dFew66NUySXMM9sLbq2fwsqXzJaltJf+8mfThjIgPQYqAdbnDtRoytd7y86gOV3zYjrj/GWCeWYEALEwvRX/cpE8X89wV+ZN6un3FCju09SneIQq48U+l1PB9mH1g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205483; c=relaxed/simple; bh=14rJEDi2P1l+j3WYjAg8K5T48LOyGvVLKGZtIYKaQsE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=IaTQ2vBdO4GULneg5xOUcRqSMTXHEN1Dacm/DBeZNh/gfVuDZ9buRB2RtklbXyg3N/zTW35dDIIqjLlbVYKMtymbb/k3iayUYcI7nF92kgngMjcmC0bRRmSL7u1BRiG55+kVsXX+gcPzJjfcrbRQuF6r10gUH5Z4DmFklwcFJYk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=NhU9TtrB; arc=none smtp.client-ip=203.254.224.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="NhU9TtrB" Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20240520114439epoutp03a9f7c48493dc79d4e9e5ea8f17a20539~RL8p--PLH1758617586epoutp03F for ; Mon, 20 May 2024 11:44:39 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20240520114439epoutp03a9f7c48493dc79d4e9e5ea8f17a20539~RL8p--PLH1758617586epoutp03F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205479; bh=Tu6eLlHByn6DIRbEEkQ1IQdusiM8jiE1rtht0uezMEA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NhU9TtrBKlG7BggPdkqwn7vfYxZDWC1XmVfRDrB6Q+X6HRVvMu7gBhgHClGvpR/C5 8qdAR5VqlKqG+bK51Pu0oKGcS36a1fI44XCfw1dlTVS4+7uaE7acFhxS+BQfCGGBEc oReEHFa5FdYMeBuaELbxK6xJlMcDktbDczYoBan4= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20240520114439epcas5p25cd426d99f6b312791da4e80922a59b4~RL8pewmvc3086130861epcas5p2G; Mon, 20 May 2024 11:44:39 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.181]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4VjbMS3vXlz4x9Pv; Mon, 20 May 2024 11:44:36 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 8B.B9.09665.4A73B466; Mon, 20 May 2024 20:44:36 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20240520102830epcas5p27274901f3d0c2738c515709890b1dec4~RK6KR1F9d2367023670epcas5p20; Mon, 20 May 2024 10:28:30 +0000 (GMT) Received: from epsmgmcp1.samsung.com (unknown [182.195.42.82]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240520102830epsmtrp100741a1f1493d9edbca78a155f4b7777~RK6KQapIb2026620266epsmtrp1c; Mon, 20 May 2024 10:28:30 +0000 (GMT) X-AuditID: b6c32a4b-5cdff700000025c1-c2-664b37a43808 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmcp1.samsung.com (Symantec Messaging Gateway) with SMTP id 7B.F6.19234.DC52B466; Mon, 20 May 2024 19:28:30 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520102826epsmtip29f05b1f606c395534f9b241baf559c09~RK6Gcre1a2247522475epsmtip2f; Mon, 20 May 2024 10:28:25 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 01/12] block: Introduce queue limits and sysfs for copy-offload support Date: Mon, 20 May 2024 15:50:14 +0530 Message-Id: <20240520102033.9361-2-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0WTf0xTVxTHue89Hi0b2xuweGH+YDUuKIIUab0YUDJwPgcGNpNtWWLKCzwo obRNW34MF0cFRCSA6IBZVKAwBlRBKGOAoIUKDBxhyC8l0S0bBREQNFS2McZaitt/n/O953vO ybk5HNz5L9KdEy9VsQopI+GRjkSLcbend9WBsFjfa39A1DDQi6MzF9ZwpHtUQKI54wuAipf+ xNGUIRug1cEhHDX3PgaoXHuVQA8NbRjq0F7EUK2uB0OlJRkY6llfINHF7nGATGMaDHVOeqGK s1UE6ujsJ9BI+xUSlVWbHNB3ff9gqPDcGIZap9QAtayW4ah+bpFAP06+g4bW+uyDt9Ijo2H0 gBbSbZpHDvTQ40aCHhlMopvqckhaX/UV/UR/GdC3HqaTdGX+JXs6L+MZSbdl/WJPPzdNEvTi 7TGSzm+uA/RP5XcdIl0+TwgUs0wMq/BgpdGymHhpXBAv7IQoRCQQ+vK9+QHoAM9DyiSyQbzQ 8EjvD+Illg3xPJIZSZJFimSUSt6+Q4EKWZKK9RDLlKogHiuPkcj95T5KJlGZJI3zkbKqg3xf Xz+BJTEqQbxewsjVIanZ964Q6UCLzgMuB1L+sPNyLXEeOHKcqVsAzuTn2NuCFwC+vFkDrFkb we/XY1451id6CJveBmBnnYPNkIVBs7nYEnA4JOUF761zrLorpcNhrr5wowVO6XGoNhowq9uF Ognrf57YqERQu+By67K9lZ2oALi23Ivbuu2AupuGDeZSB6Hh+yVgLQSpSi5M7xq0tyWFQsP6 4qbBBT7ta3awsTucLTi7ySmw9usa0mbOBFAzoQG2h8Mwa6AAt46NU7thQ/s+m7wNFg3UbwyK U2/AvNUpzKY7wdZrr3gnvN5QTtrYDY6vqDeZhqUmI2ZbUR6A/cXbL4Dtmv87lANQB9xYuTIx jlUK5PulbMp/nxYtS2wCG0ewJ6wV/Pbrkk83wDigG0AOznN1amo+FuvsFMN8kcYqZCJFkoRV dgOBZYGFuPvb0TLLFUlVIr5/gK+/UCj0D9gv5PO2OM1lXY1xpuIYFZvAsnJW8cqHcbju6Vin dDh8Vj99u4Mb/MMIXeHV+HfUlx9O5Q0fStsxf/dGZsHJfP3zj3Wp03EzOaff7Cm1C+rLfYv0 YY6/5yMok98vuPRZPROak/1NiEJwR2PnShfa5UYmB7Ox76cc7XBTze/95LTfsMnca9ylVd9X 76WetuZ2pE1FSNUzp2pCTqUUbQlKjqu/Icr27KpJdZQoXPsd7yi05UXhxRK/9Blde4PXuWHX NPNH5urVErHIsDBIM9O5zMtKDXNE3JKYGRxV0hVx9JnnbNmDhXdXjSeIdTpiNL7x+Jn55gqT 2e71nOrDoy4P+NjWceS8jfvpsW+xVCJWOPIk0DNjZXzcVPvaSv5OHqEUM/w9uELJ/AtQakzp jQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWRfVDLcRzHfX+/3377NTd+tzg/1V2Zy5FTugtfyVPd6ctwzsMh1zH6SdrW bpOHPK0mFZclD93Sk0lUWLaalnK7tIpklHR1K47y2CP5g6zR4r/XfV7vz+f9x4fCBXcJDypG dohVyMQSIckjTI+F3gue+4r2L/xVgUP903ocJmU4cFjapSHh18ffALw69BOHPZYUAEebbTgs r+8GsECXS8AOixmD1bpMDBaXWjF4LUuNQauzn4SZta8B7G3LxmBN53x4/WwhAatrnhCwtSqH hPlFvVx4q2EMgxdT2zBY2ZMIoGk0H4f3vg4SsLHTE9ocDZxVXqj1lQg91THInN3FRbbu+wRq bY5HhpI0EhkLT6NPRi1ADztUJLpx4RIHpasHSGROfsNBw72dBBp81EaiC+UlAD0rqONuco/g hUSxkpjDrCJgxR7eAWeWWJ4YdjSlKYdQAR08B9wohg5inO1W4hzgUQL6AWDMV8ycCTGTKXLU 4RPszhSPfeROhNQYo8/p/CsoiqTnM01Oanw+jX6AM8Pq89j4Ak5bcebFHVeDO72LeV5U4zpK 0L7MSOWIi/n0UsYxUv+vwJspLbO42I0OZiwVQ2CcBX8zXe1DZAaYUgAmlYDprFwpjZbukwf6 K8VSZbws2n9fnNQAXB/121wJivQO/1qAUaAWMBQunMY3lK/dL+BHiY8lsIq43Yp4CausBZ4U IZzBny1JixLQ0eJDbCzLylnFf4tRbh4qTLZk74BM6x0W2uLH9Tkx43tUZF73sgiPF9ptUntw mSil+KXb5BaNZ8Vs9Op9VWugemzRWvuZ5LI5O/Q6ufF6bGgXubgl1Wu7hre5+f3nX3eFc4/I bFsNi6xSTeDRuMh2Tjq17lpEnz1yk92wIKga9I6usa9G2t/rwg17Cmd94aY1iUyS7ACfUwdv PlPJ1zR2qLac9A8RqRUDCbu2Lu5vb8TD21L5AZQynK8ZqqoeDPqx06aN3PCBl7E8olxPHvHq +ZGQOs8Z+w7c2Hg/PW9l7tnLPcEmQnI8Lh/3rZuTveX2Bzk2tS/kjGnwLRFwObaffx4mWYxZ hesdap8lxkStkFAeEAf64Qql+A/3YTbWQAMAAA== X-CMS-MailID: 20240520102830epcas5p27274901f3d0c2738c515709890b1dec4 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520102830epcas5p27274901f3d0c2738c515709890b1dec4 References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Add device limits as sysfs entries, - copy_max_bytes (RW) - copy_max_hw_bytes (RO) Above limits help to split the copy payload in block layer. copy_max_bytes: maximum total length of copy in single payload. copy_max_hw_bytes: Reflects the device supported maximum limit. Signed-off-by: Nitesh Shetty Signed-off-by: Kanchan Joshi Signed-off-by: Anuj Gupta Reviewed-by: Hannes Reinecke --- Documentation/ABI/stable/sysfs-block | 23 +++++++++++++++ block/blk-settings.c | 34 ++++++++++++++++++++-- block/blk-sysfs.c | 43 ++++++++++++++++++++++++++++ include/linux/blkdev.h | 14 +++++++++ 4 files changed, 112 insertions(+), 2 deletions(-) diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stable/sysfs-block index 831f19a32e08..52d8a253bf8e 100644 --- a/Documentation/ABI/stable/sysfs-block +++ b/Documentation/ABI/stable/sysfs-block @@ -165,6 +165,29 @@ Description: last zone of the device which may be smaller. +What: /sys/block//queue/copy_max_bytes +Date: May 2024 +Contact: linux-block@vger.kernel.org +Description: + [RW] This is the maximum number of bytes that the block layer + will allow for a copy request. This is always smaller or + equal to the maximum size allowed by the hardware, indicated by + 'copy_max_hw_bytes'. An attempt to set a value higher than + 'copy_max_hw_bytes' will truncate this to 'copy_max_hw_bytes'. + Writing '0' to this file will disable offloading copies for this + device, instead copy is done via emulation. + + +What: /sys/block//queue/copy_max_hw_bytes +Date: May 2024 +Contact: linux-block@vger.kernel.org +Description: + [RO] This is the maximum number of bytes that the hardware + will allow for single data copy request. + A value of 0 means that the device does not support + copy offload. + + What: /sys/block//queue/crypto/ Date: February 2022 Contact: linux-block@vger.kernel.org diff --git a/block/blk-settings.c b/block/blk-settings.c index a7fe8e90240a..67010ed82422 100644 --- a/block/blk-settings.c +++ b/block/blk-settings.c @@ -52,6 +52,9 @@ void blk_set_stacking_limits(struct queue_limits *lim) lim->max_write_zeroes_sectors = UINT_MAX; lim->max_zone_append_sectors = UINT_MAX; lim->max_user_discard_sectors = UINT_MAX; + lim->max_copy_hw_sectors = UINT_MAX; + lim->max_copy_sectors = UINT_MAX; + lim->max_user_copy_sectors = UINT_MAX; } EXPORT_SYMBOL(blk_set_stacking_limits); @@ -219,6 +222,9 @@ static int blk_validate_limits(struct queue_limits *lim) lim->misaligned = 0; } + lim->max_copy_sectors = + min(lim->max_copy_hw_sectors, lim->max_user_copy_sectors); + return blk_validate_zoned_limits(lim); } @@ -231,10 +237,11 @@ int blk_set_default_limits(struct queue_limits *lim) { /* * Most defaults are set by capping the bounds in blk_validate_limits, - * but max_user_discard_sectors is special and needs an explicit - * initialization to the max value here. + * but max_user_discard_sectors and max_user_copy_sectors are special + * and needs an explicit initialization to the max value here. */ lim->max_user_discard_sectors = UINT_MAX; + lim->max_user_copy_sectors = UINT_MAX; return blk_validate_limits(lim); } @@ -316,6 +323,25 @@ void blk_queue_max_discard_sectors(struct request_queue *q, } EXPORT_SYMBOL(blk_queue_max_discard_sectors); +/* + * blk_queue_max_copy_hw_sectors - set max sectors for a single copy payload + * @q: the request queue for the device + * @max_copy_sectors: maximum number of sectors to copy + */ +void blk_queue_max_copy_hw_sectors(struct request_queue *q, + unsigned int max_copy_sectors) +{ + struct queue_limits *lim = &q->limits; + + if (max_copy_sectors > (BLK_COPY_MAX_BYTES >> SECTOR_SHIFT)) + max_copy_sectors = BLK_COPY_MAX_BYTES >> SECTOR_SHIFT; + + lim->max_copy_hw_sectors = max_copy_sectors; + lim->max_copy_sectors = + min(max_copy_sectors, lim->max_user_copy_sectors); +} +EXPORT_SYMBOL_GPL(blk_queue_max_copy_hw_sectors); + /** * blk_queue_max_secure_erase_sectors - set max sectors for a secure erase * @q: the request queue for the device @@ -633,6 +659,10 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b, t->max_segment_size = min_not_zero(t->max_segment_size, b->max_segment_size); + t->max_copy_sectors = min(t->max_copy_sectors, b->max_copy_sectors); + t->max_copy_hw_sectors = min(t->max_copy_hw_sectors, + b->max_copy_hw_sectors); + t->misaligned |= b->misaligned; alignment = queue_limit_alignment_offset(b, start); diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index f0f9314ab65c..805c2b6b0393 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -205,6 +205,44 @@ static ssize_t queue_discard_zeroes_data_show(struct request_queue *q, char *pag return queue_var_show(0, page); } +static ssize_t queue_copy_hw_max_show(struct request_queue *q, char *page) +{ + return sprintf(page, "%llu\n", (unsigned long long) + q->limits.max_copy_hw_sectors << SECTOR_SHIFT); +} + +static ssize_t queue_copy_max_show(struct request_queue *q, char *page) +{ + return sprintf(page, "%llu\n", (unsigned long long) + q->limits.max_copy_sectors << SECTOR_SHIFT); +} + +static ssize_t queue_copy_max_store(struct request_queue *q, const char *page, + size_t count) +{ + unsigned long max_copy_bytes; + struct queue_limits lim; + ssize_t ret; + int err; + + ret = queue_var_store(&max_copy_bytes, page, count); + if (ret < 0) + return ret; + + if (max_copy_bytes & (queue_logical_block_size(q) - 1)) + return -EINVAL; + + blk_mq_freeze_queue(q); + lim = queue_limits_start_update(q); + lim.max_user_copy_sectors = max_copy_bytes >> SECTOR_SHIFT; + err = queue_limits_commit_update(q, &lim); + blk_mq_unfreeze_queue(q); + + if (err) + return err; + return count; +} + static ssize_t queue_write_same_max_show(struct request_queue *q, char *page) { return queue_var_show(0, page); @@ -505,6 +543,9 @@ QUEUE_RO_ENTRY(queue_nr_zones, "nr_zones"); QUEUE_RO_ENTRY(queue_max_open_zones, "max_open_zones"); QUEUE_RO_ENTRY(queue_max_active_zones, "max_active_zones"); +QUEUE_RO_ENTRY(queue_copy_hw_max, "copy_max_hw_bytes"); +QUEUE_RW_ENTRY(queue_copy_max, "copy_max_bytes"); + QUEUE_RW_ENTRY(queue_nomerges, "nomerges"); QUEUE_RW_ENTRY(queue_rq_affinity, "rq_affinity"); QUEUE_RW_ENTRY(queue_poll, "io_poll"); @@ -618,6 +659,8 @@ static struct attribute *queue_attrs[] = { &queue_discard_max_entry.attr, &queue_discard_max_hw_entry.attr, &queue_discard_zeroes_data_entry.attr, + &queue_copy_hw_max_entry.attr, + &queue_copy_max_entry.attr, &queue_write_same_max_entry.attr, &queue_write_zeroes_max_entry.attr, &queue_zone_append_max_entry.attr, diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index aefdda9f4ec7..109d9f905c3c 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -309,6 +309,10 @@ struct queue_limits { unsigned int discard_alignment; unsigned int zone_write_granularity; + unsigned int max_copy_hw_sectors; + unsigned int max_copy_sectors; + unsigned int max_user_copy_sectors; + unsigned short max_segments; unsigned short max_integrity_segments; unsigned short max_discard_segments; @@ -933,6 +937,8 @@ void blk_queue_max_secure_erase_sectors(struct request_queue *q, unsigned int max_sectors); extern void blk_queue_max_discard_sectors(struct request_queue *q, unsigned int max_discard_sectors); +extern void blk_queue_max_copy_hw_sectors(struct request_queue *q, + unsigned int max_copy_sectors); extern void blk_queue_max_write_zeroes_sectors(struct request_queue *q, unsigned int max_write_same_sectors); extern void blk_queue_logical_block_size(struct request_queue *, unsigned int); @@ -1271,6 +1277,14 @@ static inline unsigned int bdev_discard_granularity(struct block_device *bdev) return bdev_get_queue(bdev)->limits.discard_granularity; } +/* maximum copy offload length, this is set to 128MB based on current testing */ +#define BLK_COPY_MAX_BYTES (1 << 27) + +static inline unsigned int bdev_max_copy_sectors(struct block_device *bdev) +{ + return bdev_get_queue(bdev)->limits.max_copy_sectors; +} + static inline unsigned int bdev_max_secure_erase_sectors(struct block_device *bdev) { From patchwork Mon May 20 10:20:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668227 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61F4E5674A for ; Mon, 20 May 2024 11:44:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.25 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205487; cv=none; b=spZnhWdmYeK9FdpktBKY/S+HK9Aevl3BivUq+8ytYyUe///EtohDd7ePBTNBZbL6sl8uk4dcY+KmLdJv64mh15S3saubsXZEUWG+VbcuNj1LKk/vvVuj3/3hhw7cHhhvM/w0VLedbLqrmcAjFb0vJ0nO+PhDs0UGoidKIAYastM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205487; c=relaxed/simple; bh=tDW90go0cwdQKfVxfCFMOlHa/JVIItF8C/bqx5wNnLI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=OAzIkPDAkpi7JWPS8d0T3YVm5iHT1QQxTTPCJ3WXvI8DQkjhsSKNJWpjlsMU8akrwSQG/cFeYmP9+FtrKxRwSYt5I8yy4VqHljVTJjzvXG+Ge0zVr8DEq7v9NVPzhK2hFYTce8vE4vfiITFK7xqx6hjGHqNfg4twODw/ejZhm8g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=dGTUL8SA; arc=none smtp.client-ip=203.254.224.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="dGTUL8SA" Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20240520114443epoutp02291491c7c4becbec07110ca5306f2746~RL8tf7zPL0849508495epoutp02K for ; Mon, 20 May 2024 11:44:43 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20240520114443epoutp02291491c7c4becbec07110ca5306f2746~RL8tf7zPL0849508495epoutp02K DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205483; bh=WdNRoHumSn7w5xCV6vWF2e9p3PFPIhHw8dbDxe51WtM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dGTUL8SAv4pKuChohpMYoF8K6ueKbJBgmpfB5gaO+qz6ioQ9ErXBWpPozzuv+JJFE BExOpSu6s2FQE7pcqqt/pMs3vBfsv2RRyn+iIo2VAM2XU4zJM5BmnXAF8d/E4kbRfJ SmmvErZc+7qggGSnwlmpQDB5UFAMexQVlBBvDFWE= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20240520114443epcas5p21e69d571d5ffd31eeb5915946f52636d~RL8s-8eDJ2455024550epcas5p2a; Mon, 20 May 2024 11:44:43 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.179]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4VjbMX64nSz4x9Px; Mon, 20 May 2024 11:44:40 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 8D.B9.09665.8A73B466; Mon, 20 May 2024 20:44:40 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20240520102842epcas5p4949334c2587a15b8adab2c913daa622f~RK6VX63oy1370113701epcas5p4D; Mon, 20 May 2024 10:28:42 +0000 (GMT) Received: from epsmgmcp1.samsung.com (unknown [182.195.42.82]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240520102842epsmtrp123a9d33826cf03d228d42e765b92b7cd~RK6VWvDE92026720267epsmtrp1k; Mon, 20 May 2024 10:28:42 +0000 (GMT) X-AuditID: b6c32a4b-5cdff700000025c1-c8-664b37a8b833 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmcp1.samsung.com (Symantec Messaging Gateway) with SMTP id 2F.F6.19234.9D52B466; Mon, 20 May 2024 19:28:41 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520102838epsmtip23bacddee0f9b0fe75b4460140bd89c31~RK6RkNFC32248422484epsmtip2h; Mon, 20 May 2024 10:28:37 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 02/12] Add infrastructure for copy offload in block and request layer. Date: Mon, 20 May 2024 15:50:15 +0530 Message-Id: <20240520102033.9361-3-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0WTe1DUVRTH5/5+uz8Wm83fAI13KWRddUYgcFcXuLzKmQh+BhWjThmlyw78 ZFn21T6SbCYgxHjIU4kgDAJUXkLyfgizLgG5uAEiONDwSGEiyCDKMUKgXRbsv88593zP99xz 57Jwu2eEIytGoaXVCrGMR+xgNHe7uLhXeIec4Q/fd0N1xl4cfZG9hqPqiSwCLXQvA/TV0gqO ZvRfArRqGsBRY+8kQCWlVxhoTN+GoVuluRiqrO7B0Df5SRjq2XhMoFzDKECzI4UY6hx3Q99d KGegW513GGi4vYhAxddmbdD1vnUM5aSMYKh1JhGg5tViHNUuLDLQj+Mvo4G1PuaRV6jh+yGU sRRSbYUTNtTA5E0GNWzSUfVVqQTVUB5PzTUUAKpjLIGgyjIvMamMpD8Iqi15ikn9OTvOoBa7 Rggqs7EKUHdLfrAJsw+P9ZfQ4ihazaUVkcqoGEV0AC/kuOgNkacXX+Au8EHePK5CLKcDeIGh Ye5BMTLzhnjcT8QynTkVJtZoeAdf81crdVqaK1FqtAE8WhUlUwlVHhqxXKNTRHsoaK2vgM8/ 5GkujIiV/PQkgaka9ItbmyuzSQDF/DRgy4KkEF7KvYilgR0sO7IDwJWGdWANlgFczB5hPA/S jUawLTHeXN46aAPQ1Fe7JUnG4OJUs7kZi0WQbrB/g2XJO5DVOExvyNlU4GQDDhO79ZillT35 Efz7RibDwgxyP0wZmmZamE36wPKrVzGrnTOs/l6PW9iW9IX6pqVNN0iW2cKkiaFNN0gGwpqx d6z19nC+r9HGyo7wt6wLW3wWVl6uIKza8wAWPijcus/rMNmYhVv64KQLrGs/aE07wTxj7eYM OPkizFid2ZqHDVu/3ea9sKauhLAyB44+TSSs41Dw5xpn61IyABy8Nw+ywe7C/x1KAKgCHFql kUfTGk/VYQV99vmzRSrl9WDzG7iGtIKH00seBoCxgAFAFs5zYNc3Hj1jx44Sf3qOVitFap2M 1hiAp3l/ObjjS5FK8z9SaEUCoQ9f6OXlJfQ57CXg7WIvJF+JsiOjxVo6lqZVtHpbh7FsHROw t6ci4kNTNe8KQ+MFn3Nk+5t8bx/1+/XEnqn4oQ+m5V2nbthXd382750rvnagPNUfq5rboz/2 JC0vOCMymr8aLgraZ+qv4EqwjYK4ghFN0nH93fzguNFgP2m3K+cfKbf+Q5+eRx6Xu9rX73w9 8IJ00DTIX/IokBo2yiZ1TeyEex+fKPslzllSQRU/hoy63oX5994cv35kbxOV2Yc/zZ/eGejw r3TnSuAjzsz5ftmBIhmvyHeX0/hbu1umZ6oS09mmxdvBp+dMD+LFz+Q1ne/3h7W0QKdXpaOO zJRjQThz3ynRuYeVER0nf28qyVZQpw0cj6SI/lGXQ9zwkxU5vInAv/KceAyNRCxwxdUa8X9o dnqfjwQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWRfyyUcRzHfZ/nueceNvZwrG/YbCfWKLH1xzf6yXa+ptJstWKViydZjtsd rbRKzon0+4eKynXOCUl+5EepLpJLDk1qhNrq6kqF22xd3Mmx/nvt/f589n5vb4Z0q6E8mZS0 DE6WJk4V0k5UU4fQZ+WQX/T+4M/PwlBt90sS5Vywkqh69DyNxjvMABVNWkj0RXcKoBlDH4ka X44BpFLfotCQrpVAbepLBKqs7iRQyTUFgTrnftHoUvs7gIyDxQR6MhyI7uRpKNT25BWFBh7d pFGp1shHFV02Al3MHyRQy5eTADXNlJLo/vgEhfTDXqjP2sXb6I0H3kbjbjXErcWjfNw3Vkfh AUMmrq8qoHGD5gQ2NdwA+PFQNo3Lzl3m4bOK3zRuVX7k4SnjMIUnng7S+FxjFcA9qhf8bYI4 p7VJXGrKIU62an2C04He6WyetD/ssNVUxs8GpcGngSMD2dWwu85M2dmNbQbwflXYor4Uaq0v yEUWwErbN/5p4DR/oyBgrkE1bzAMzQbC13OMXXdnm0k4pSgk7A8k20nC/nvIzgI2DjbP6fl2 plg/mP/mE8/OzuwaqCkvJxYDfGD1A91CmCMbCnUPJ8FioTVw9P0kfQG4qIBDFfDgpHJJsiRR GhIkF0vkmWnJQYnpknqwsGhAbAvQ1lqD2gHBgHYAGVLo7lzfGLXfzTlJfCSLk6XvlWWmcvJ2 4MVQwiXOvqkFSW5ssjiDO8hxUk723yUYR89sglHCmgGlb+Q60Qrz526y0rvNQWor8z9m+Ohh idp8d/mJlO+F2/e1EK3m5yP7toQTu76eD1jVI5r1NNmOb9OV4MikfLQlOGjPslimIv3oj6yi qGc7O3LVk+DaCPNz01vDXHD0xYTrj21bE24nxl4pbLvC27DMJ5wa913i3UTWlJpmNYHFu0WW lcbcaXWTBahcjONdfyp2/Y266Z+RmDGVWS5KFqD4iB/SCcUOulHiLvKTaJV5dVzgVYUlnvrQ McuFxxxT8+g9rp3KOHzdOqJxfXhQn/NIYHaJKD5shTHCX/HMA1OC65kYD22OefrkEWmIS62+ qMHwabO+N/SqkJIfEIcEkDK5+B8289o3QAMAAA== X-CMS-MailID: 20240520102842epcas5p4949334c2587a15b8adab2c913daa622f X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520102842epcas5p4949334c2587a15b8adab2c913daa622f References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: We add two new opcode REQ_OP_COPY_DST, REQ_OP_COPY_SRC. Since copy is a composite operation involving src and dst sectors/lba, each needs to be represented by a separate bio to make it compatible with device mapper. We expect caller to take a plug and send bio with destination information, followed by bio with source information. Once the dst bio arrives we form a request and wait for source bio. Upon arrival of source bio we merge these two bio's and send corresponding request down to device driver. Merging non copy offload bio is avoided by checking for copy specific opcodes in merge function. Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta --- block/blk-core.c | 7 +++++++ block/blk-merge.c | 41 +++++++++++++++++++++++++++++++++++++++ block/blk.h | 16 +++++++++++++++ block/elevator.h | 1 + include/linux/bio.h | 6 +----- include/linux/blk_types.h | 10 ++++++++++ 6 files changed, 76 insertions(+), 5 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index ea44b13af9af..f18ee5f709c0 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -122,6 +122,8 @@ static const char *const blk_op_name[] = { REQ_OP_NAME(ZONE_FINISH), REQ_OP_NAME(ZONE_APPEND), REQ_OP_NAME(WRITE_ZEROES), + REQ_OP_NAME(COPY_SRC), + REQ_OP_NAME(COPY_DST), REQ_OP_NAME(DRV_IN), REQ_OP_NAME(DRV_OUT), }; @@ -838,6 +840,11 @@ void submit_bio_noacct(struct bio *bio) * requests. */ fallthrough; + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: + if (!q->limits.max_copy_sectors) + goto not_supported; + break; default: goto not_supported; } diff --git a/block/blk-merge.c b/block/blk-merge.c index 8534c35e0497..f8dc48a03379 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -154,6 +154,20 @@ static struct bio *bio_split_write_zeroes(struct bio *bio, return bio_split(bio, lim->max_write_zeroes_sectors, GFP_NOIO, bs); } +static struct bio *bio_split_copy(struct bio *bio, + const struct queue_limits *lim, + unsigned int *nsegs) +{ + *nsegs = 1; + if (bio_sectors(bio) <= lim->max_copy_sectors) + return NULL; + /* + * We don't support splitting for a copy bio. End it with EIO if + * splitting is required and return an error pointer. + */ + return ERR_PTR(-EIO); +} + /* * Return the maximum number of sectors from the start of a bio that may be * submitted as a single request to a block device. If enough sectors remain, @@ -362,6 +376,12 @@ struct bio *__bio_split_to_limits(struct bio *bio, case REQ_OP_WRITE_ZEROES: split = bio_split_write_zeroes(bio, lim, nr_segs, bs); break; + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: + split = bio_split_copy(bio, lim, nr_segs); + if (IS_ERR(split)) + return NULL; + break; default: split = bio_split_rw(bio, lim, nr_segs, bs, get_max_io_size(bio, lim) << SECTOR_SHIFT); @@ -925,6 +945,9 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio) if (!rq_mergeable(rq) || !bio_mergeable(bio)) return false; + if (blk_copy_offload_mergable(rq, bio)) + return true; + if (req_op(rq) != bio_op(bio)) return false; @@ -958,6 +981,8 @@ enum elv_merge blk_try_merge(struct request *rq, struct bio *bio) { if (blk_discard_mergable(rq)) return ELEVATOR_DISCARD_MERGE; + else if (blk_copy_offload_mergable(rq, bio)) + return ELEVATOR_COPY_OFFLOAD_MERGE; else if (blk_rq_pos(rq) + blk_rq_sectors(rq) == bio->bi_iter.bi_sector) return ELEVATOR_BACK_MERGE; else if (blk_rq_pos(rq) - bio_sectors(bio) == bio->bi_iter.bi_sector) @@ -1065,6 +1090,20 @@ static enum bio_merge_status bio_attempt_discard_merge(struct request_queue *q, return BIO_MERGE_FAILED; } +static enum bio_merge_status bio_attempt_copy_offload_merge(struct request *req, + struct bio *bio) +{ + if (req->__data_len != bio->bi_iter.bi_size) + return BIO_MERGE_FAILED; + + req->biotail->bi_next = bio; + req->biotail = bio; + req->nr_phys_segments++; + req->__data_len += bio->bi_iter.bi_size; + + return BIO_MERGE_OK; +} + static enum bio_merge_status blk_attempt_bio_merge(struct request_queue *q, struct request *rq, struct bio *bio, @@ -1085,6 +1124,8 @@ static enum bio_merge_status blk_attempt_bio_merge(struct request_queue *q, break; case ELEVATOR_DISCARD_MERGE: return bio_attempt_discard_merge(q, rq, bio); + case ELEVATOR_COPY_OFFLOAD_MERGE: + return bio_attempt_copy_offload_merge(rq, bio); default: return BIO_MERGE_NONE; } diff --git a/block/blk.h b/block/blk.h index 189bc25beb50..6528a2779b84 100644 --- a/block/blk.h +++ b/block/blk.h @@ -174,6 +174,20 @@ static inline bool blk_discard_mergable(struct request *req) return false; } +/* + * Copy offload sends a pair of bio with REQ_OP_COPY_DST and REQ_OP_COPY_SRC + * operation by taking a plug. + * Initially DST bio is sent which forms a request and + * waits for SRC bio to arrive. Once SRC bio arrives + * we merge it and send request down to driver. + */ +static inline bool blk_copy_offload_mergable(struct request *req, + struct bio *bio) +{ + return (req_op(req) == REQ_OP_COPY_DST && + bio_op(bio) == REQ_OP_COPY_SRC); +} + static inline unsigned int blk_rq_get_max_segments(struct request *rq) { if (req_op(rq) == REQ_OP_DISCARD) @@ -323,6 +337,8 @@ static inline bool bio_may_exceed_limits(struct bio *bio, case REQ_OP_DISCARD: case REQ_OP_SECURE_ERASE: case REQ_OP_WRITE_ZEROES: + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: return true; /* non-trivial splitting decisions */ default: break; diff --git a/block/elevator.h b/block/elevator.h index e9a050a96e53..c7a45c1f4156 100644 --- a/block/elevator.h +++ b/block/elevator.h @@ -18,6 +18,7 @@ enum elv_merge { ELEVATOR_FRONT_MERGE = 1, ELEVATOR_BACK_MERGE = 2, ELEVATOR_DISCARD_MERGE = 3, + ELEVATOR_COPY_OFFLOAD_MERGE = 4, }; struct blk_mq_alloc_data; diff --git a/include/linux/bio.h b/include/linux/bio.h index d5379548d684..528ef22dd65b 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -53,11 +53,7 @@ static inline unsigned int bio_max_segs(unsigned int nr_segs) */ static inline bool bio_has_data(struct bio *bio) { - if (bio && - bio->bi_iter.bi_size && - bio_op(bio) != REQ_OP_DISCARD && - bio_op(bio) != REQ_OP_SECURE_ERASE && - bio_op(bio) != REQ_OP_WRITE_ZEROES) + if (bio && (bio_op(bio) == REQ_OP_READ || bio_op(bio) == REQ_OP_WRITE)) return true; return false; diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 781c4500491b..7f692bade271 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -342,6 +342,10 @@ enum req_op { /* reset all the zone present on the device */ REQ_OP_ZONE_RESET_ALL = (__force blk_opf_t)15, + /* copy offload src and dst operation */ + REQ_OP_COPY_SRC = (__force blk_opf_t)18, + REQ_OP_COPY_DST = (__force blk_opf_t)19, + /* Driver private requests */ REQ_OP_DRV_IN = (__force blk_opf_t)34, REQ_OP_DRV_OUT = (__force blk_opf_t)35, @@ -430,6 +434,12 @@ static inline bool op_is_write(blk_opf_t op) return !!(op & (__force blk_opf_t)1); } +static inline bool op_is_copy(blk_opf_t op) +{ + return ((op & REQ_OP_MASK) == REQ_OP_COPY_SRC || + (op & REQ_OP_MASK) == REQ_OP_COPY_DST); +} + /* * Check if the bio or request is one that needs special treatment in the * flush state machine. From patchwork Mon May 20 10:20:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668228 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B947E53814 for ; Mon, 20 May 2024 11:44:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.24 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205492; cv=none; b=FpL4XV7Bo0mcyOMq0MM/FDUEOMzKvTgKuQJUsbuDPyXVCLPbfr5FBpWyyM4sQuU+YI/z0s9RL1r/3unF2qq1oN8dFz8NjMtKcxL9Ni3VTY5MhEidas3H9m5xZZQPLaDVIwSDLO2CzAtYNdDPzZjFGesEeTSdctY9CaJow43/8cU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205492; c=relaxed/simple; bh=Rie2jTnrYiolLjNDyBVH3DLqZyqJ10GciO4Xq0mERrk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=YIOcYSUArRCxIFWsUTpeX29VG1FiI6VT3Q0WV9TEEONOJZ0zqlgXK4V8e0NHEdSM3V8FD/EW9NjBOiVlyCVrLBESwlQ55ck5puIWUxVGEnlE4KRGlPV3Wc/+OGpszvA6gBEHDTvnz0Rk38N/gfRI/jF8777juhIs5E6PnAOL2Ig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=Bb8/hktb; arc=none smtp.client-ip=203.254.224.24 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="Bb8/hktb" Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20240520114448epoutp015e6c111b829ca408a6f3dff2cbe67bab~RL8xv2wlJ1820118201epoutp01Y for ; Mon, 20 May 2024 11:44:48 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20240520114448epoutp015e6c111b829ca408a6f3dff2cbe67bab~RL8xv2wlJ1820118201epoutp01Y DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205488; bh=eUSQE9j14Vy0JyXIjPMM6UBiUlBOzDa6HuAmKoMKPTo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Bb8/hktb261THpw8MLGi7N2jNmVryNqYHBr19cFEqUskiy9j08PRAhvc8P7u/L+w8 iKs6OqyULqAqQCFRwcVBllh8F/7o9AboXlScWyNo7yqDhLRe7Tz+2/5HUPMFCt7zg1 hq15yqUlf39tfRZoQr5bmY2pTdIBBwP6WdOkchco= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20240520114447epcas5p2125bb61b9966a69fc16f04429520654e~RL8xSeJwP2455024550epcas5p2h; Mon, 20 May 2024 11:44:47 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.180]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4VjbMd5QQ1z4x9Py; Mon, 20 May 2024 11:44:45 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 10.C9.09665.DA73B466; Mon, 20 May 2024 20:44:45 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20240520102853epcas5p42d635d6712b8876ea22a45d730cb1378~RK6gNvh6Z1363313633epcas5p4m; Mon, 20 May 2024 10:28:53 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20240520102853epsmtrp2cd54f4a807d23e90c255b4f734b418f1~RK6gLXkS02133221332epsmtrp2E; Mon, 20 May 2024 10:28:53 +0000 (GMT) X-AuditID: b6c32a4b-829fa700000025c1-d1-664b37ad2147 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 51.68.08924.5E52B466; Mon, 20 May 2024 19:28:53 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520102849epsmtip2a94a8a0cb33c589bd82bf0075fa82437~RK6cdEmpV2182421824epsmtip2G; Mon, 20 May 2024 10:28:49 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 03/12] block: add copy offload support Date: Mon, 20 May 2024 15:50:16 +0530 Message-Id: <20240520102033.9361-4-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0WTe0xTVxzHPfdebgtJ4QZcPDB0WIQJDKRS8GBAzER2kbkxYFmyZGOVXguj tE1b5sAtvGQis7wmspXnkGzjZZVWR8tDUh51KJIJSCCDIcJ0Q4FBgmPAWEtx++/z+57v75Hf yY+NO66RLuwkiZKRSwRiLmlH3Oz22u/bfCjqtP9qpSfS9PfhKLtoA0eNE4UkmuteAujy4iqO ZrrOA7Q2MIgjXd8kQDW1lQQa69JjqL22BEP1jb0YKi/LwVDv5jMSlRgfADQ7osZQx7gP+vaL OgK1d/xEoCFDBYmqv5tloe9N/2CoOG8EQ60zWQDdXKvG0dW5BQLdHn8ZDW6YbI660kPDUXR/ LaT16gkWPTh5naCHBlLploYLJK2ty6CfaL8BdNtYJklfKfjKhlblzJO0PvdXG/rP2XGCXugc IekCXQOg79b0sKKd3k8OSWQEQkbuxkgSpMIkiSiUGxUbfyw+MMif58sLRoe4bhJBChPKDX8z 2jciSWzeENftE4E41SxFCxQK7oEjIXJpqpJxS5QqlKFcRiYUy/gyP4UgRZEqEflJGOVhnr// wUCz8aPkxL56LUu2EvbpZF8OKxM8C8gHbDak+LDMZJ8P7NiOVBuAs+0mPB/YmoMlALPHX7Py CoClF2UWtvjV5RukNaEDwIdP+raDXAxevtIJLFVJygfe2WRb9J1UIw6/1BYTlgCntDjM6u7C LKWcqMPw7+VmloUJygPOVxRsMYcKhpUlOtza7hXYeK1ri23N/q4bi8BSCFIaW/hcpyespnC4 +rRrm53gHyYdy8oucHm+g7TyGVh/6QfSmnwOQPWoGlgfwmBufyFuGRunvKDGcMAq74al/Ve3 BsUpe6ham8GsOge2Vr1gd9ikqdmu7wwfPM/aZhqqfx4A1rWoAFxeL7UpAnvU/7eoAaABODMy RYqIUQTKAiTMmf9+LUGa0gK2rsA7qhVMTy36GQHGBkYA2Th3J6dFF3nakSMUpKUzcmm8PFXM KIwg0LzBYtzlpQSp+YwkyngeP9ifHxQUxA8OCOJxd3HmciuFjpRIoGSSGUbGyF/kYWxbl0xs utNwazSupNHgMX1f6JDfGfnGjc3hW/rXVy8+feRxnI8u5MbF2KUZCx+fLXStMLjsTf+Q9/ny DjeQkeR8W2jDWqEXfnOb3zfDU+Q05sneKri+Y6mqdn5on1dnlEaqG26/f4KtCPrAxsQvX78X erK0+S/xu62xsaJfRq+FcvcaT/nzy1q7I72PLO15x/Nx9sc1rktl7zXF+Cy6O0Zcyus5Gc7f bLE7tcoxzWXFOfzo/aj3WNH5ez0e6WFjdUdVv7u/rRcxCUbPCNPutP3rGSHrd+395D0P7bnO U8Rnu74+odp00LU1vWrojzMc1N4RsqrFWk3KWtXElMpxprD57LmlGF5CEZdQJAp43rhcIfgX cQNgCY4EAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrEIsWRmVeSWpSXmKPExsWy7bCSvO5TVe80g+4DGhbrTx1jtmia8JfZ YvXdfjaL14c/MVpM+/CT2eLJgXZGi99nzzNbbDl2j9FiwaK5LBY3D+xkstizaBKTxcrVR5ks Zk9vZrI4+v8tm8WkQ9cYLZ5encVksfeWtsXCtiUsFnv2nmSxuLxrDpvF/GVP2S2WH//HZDGx 4yqTxY4njYwW237PZ7ZY9/o9i8WJW9IW5/8eZ3WQ8bh8xdvj1CIJj52z7rJ7nL+3kcXj8tlS j02rOtk8Ni+p93ixeSajx+6bDWwei/sms3r0Nr9j89jZep/V4+PTWywe7/ddZfPo27KK0ePM giPsAcJRXDYpqTmZZalF+nYJXBnHVm5mL/hmX3HvWDN7A+Nb4y5GTg4JAROJWbP/snUxcnEI CexmlNi0ZhorREJSYtnfI8wQtrDEyn/P2SGKmpkk7m64wNjFyMHBJqAtcfo/B0hcRGA7s8TH 5m4mkAZmgaPMEhfWWIDYwgJWEr8+r2UHsVkEVCXezekDs3kFLCXmTtoCtUBeYvWGA2A2J1D9 ga0fGEFsIaCau9c/sE1g5FvAyLCKUTK1oDg3PbfYsMAwL7Vcrzgxt7g0L10vOT93EyM4WrU0 dzBuX/VB7xAjEwfjIUYJDmYlEd5NWzzThHhTEiurUovy44tKc1KLDzFKc7AoifOKv+hNERJI TyxJzU5NLUgtgskycXBKNTAZ+Is9mJUb80trr1Xig3VrM+O1u+JWWP63/yi1L13uwm/3Dsvr vsWdohv7+3rqwx4USR1e2//U426uwYfnK81fcXepb7PuVVkwfXm+2YMGvSMSa1xsgg/8TFhx vfTsUtulb902rfba+kdtMqvMWRPuW3pcHVe+xP5I2SE47T27f6DDAp6ER8X3JnMu/HH0Yvhd 68wz7TNqO63q6i/4uX2Z7Ho2ZP2HjEA29Rmve6+y6iR9li7cukrOXZKXa4/WRn6mriU35j9M vbSo/uo62ag56sdkgxmtfkkZyzp7NIRsWzrbUbPKkiUl7crNZH8+mwtXe9/Zaa+W+1TDd1e3 8n599MzetY86ooqWaAk1xymxFGckGmoxFxUnAgCpfyHoRQMAAA== X-CMS-MailID: 20240520102853epcas5p42d635d6712b8876ea22a45d730cb1378 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520102853epcas5p42d635d6712b8876ea22a45d730cb1378 References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Introduce blkdev_copy_offload to perform copy offload. Issue REQ_OP_COPY_DST with destination info along with taking a plug. This flows till request layer and waits for src bio to arrive. Issue REQ_OP_COPY_SRC with source info and this bio reaches request layer and merges with dst request. For any reason, if a request comes to the driver with only one of src/dst bio, we fail the copy offload. Larger copy will be divided, based on max_copy_sectors limit. Reviewed-by: Hannes Reinecke Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty --- block/blk-lib.c | 204 +++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 4 + 2 files changed, 208 insertions(+) diff --git a/block/blk-lib.c b/block/blk-lib.c index 442da9dad042..e83461abb581 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -10,6 +10,22 @@ #include "blk.h" +/* Keeps track of all outstanding copy IO */ +struct blkdev_copy_io { + atomic_t refcount; + ssize_t copied; + int status; + struct task_struct *waiter; + void (*endio)(void *private, int status, ssize_t copied); + void *private; +}; + +/* Keeps track of single outstanding copy offload IO */ +struct blkdev_copy_offload_io { + struct blkdev_copy_io *cio; + loff_t offset; +}; + static sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) { unsigned int discard_granularity = bdev_discard_granularity(bdev); @@ -103,6 +119,194 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector, } EXPORT_SYMBOL(blkdev_issue_discard); +static inline ssize_t blkdev_copy_sanity_check(struct block_device *bdev_in, + loff_t pos_in, + struct block_device *bdev_out, + loff_t pos_out, size_t len) +{ + unsigned int align = max(bdev_logical_block_size(bdev_out), + bdev_logical_block_size(bdev_in)) - 1; + + if ((pos_in & align) || (pos_out & align) || (len & align) || !len || + len >= BLK_COPY_MAX_BYTES) + return -EINVAL; + + return 0; +} + +static inline void blkdev_copy_endio(struct blkdev_copy_io *cio) +{ + if (cio->endio) { + cio->endio(cio->private, cio->status, cio->copied); + kfree(cio); + } else { + struct task_struct *waiter = cio->waiter; + + WRITE_ONCE(cio->waiter, NULL); + blk_wake_io_task(waiter); + } +} + +/* + * This must only be called once all bios have been issued so that the refcount + * can only decrease. This just waits for all bios to complete. + * Returns the length of bytes copied or error + */ +static ssize_t blkdev_copy_wait_for_completion_io(struct blkdev_copy_io *cio) +{ + ssize_t ret; + + for (;;) { + __set_current_state(TASK_UNINTERRUPTIBLE); + if (!READ_ONCE(cio->waiter)) + break; + blk_io_schedule(); + } + __set_current_state(TASK_RUNNING); + ret = cio->copied; + kfree(cio); + + return ret; +} + +static void blkdev_copy_offload_src_endio(struct bio *bio) +{ + struct blkdev_copy_offload_io *offload_io = bio->bi_private; + struct blkdev_copy_io *cio = offload_io->cio; + + if (bio->bi_status) { + cio->copied = min_t(ssize_t, offload_io->offset, cio->copied); + if (!cio->status) + cio->status = blk_status_to_errno(bio->bi_status); + } + bio_put(bio); + kfree(offload_io); + + if (atomic_dec_and_test(&cio->refcount)) + blkdev_copy_endio(cio); +} + +/* + * @bdev: block device + * @pos_in: source offset + * @pos_out: destination offset + * @len: length in bytes to be copied + * @endio: endio function to be called on completion of copy operation, + * for synchronous operation this should be NULL + * @private: endio function will be called with this private data, + * for synchronous operation this should be NULL + * @gfp_mask: memory allocation flags (for bio_alloc) + * + * For synchronous operation returns the length of bytes copied or error + * For asynchronous operation returns -EIOCBQUEUED or error + * + * Description: + * Copy source offset to destination offset within block device, using + * device's native copy offload feature. + * We perform copy operation using 2 bio's. + * 1. We take a plug and send a REQ_OP_COPY_DST bio along with destination + * sector and length. Once this bio reaches request layer, we form a + * request and wait for dst bio to arrive. + * 2. We issue REQ_OP_COPY_SRC bio along with source sector, length. + * Once this bio reaches request layer and find a request with previously + * sent destination info we merge the source bio and return. + * 3. Release the plug and request is sent to driver + * This design works only for drivers with request queue. + */ +ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, + loff_t pos_out, size_t len, + void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp) +{ + struct blkdev_copy_io *cio; + struct blkdev_copy_offload_io *offload_io; + struct bio *src_bio, *dst_bio; + size_t rem, chunk; + size_t max_copy_bytes = bdev_max_copy_sectors(bdev) << SECTOR_SHIFT; + ssize_t ret; + struct blk_plug plug; + + if (!max_copy_bytes) + return -EOPNOTSUPP; + + ret = blkdev_copy_sanity_check(bdev, pos_in, bdev, pos_out, len); + if (ret) + return ret; + + cio = kzalloc(sizeof(*cio), gfp); + if (!cio) + return -ENOMEM; + atomic_set(&cio->refcount, 1); + cio->waiter = current; + cio->endio = endio; + cio->private = private; + + /* + * If there is a error, copied will be set to least successfully + * completed copied length + */ + cio->copied = len; + for (rem = len; rem > 0; rem -= chunk) { + chunk = min(rem, max_copy_bytes); + + offload_io = kzalloc(sizeof(*offload_io), gfp); + if (!offload_io) + goto err_free_cio; + offload_io->cio = cio; + /* + * For partial completion, we use offload_io->offset to truncate + * successful copy length + */ + offload_io->offset = len - rem; + + dst_bio = bio_alloc(bdev, 0, REQ_OP_COPY_DST, gfp); + if (!dst_bio) + goto err_free_offload_io; + dst_bio->bi_iter.bi_size = chunk; + dst_bio->bi_iter.bi_sector = pos_out >> SECTOR_SHIFT; + + blk_start_plug(&plug); + src_bio = blk_next_bio(dst_bio, bdev, 0, REQ_OP_COPY_SRC, gfp); + if (!src_bio) + goto err_free_dst_bio; + src_bio->bi_iter.bi_size = chunk; + src_bio->bi_iter.bi_sector = pos_in >> SECTOR_SHIFT; + src_bio->bi_end_io = blkdev_copy_offload_src_endio; + src_bio->bi_private = offload_io; + + atomic_inc(&cio->refcount); + submit_bio(src_bio); + blk_finish_plug(&plug); + pos_in += chunk; + pos_out += chunk; + } + + if (atomic_dec_and_test(&cio->refcount)) + blkdev_copy_endio(cio); + if (endio) + return -EIOCBQUEUED; + + return blkdev_copy_wait_for_completion_io(cio); + +err_free_dst_bio: + bio_put(dst_bio); +err_free_offload_io: + kfree(offload_io); +err_free_cio: + cio->copied = min_t(ssize_t, cio->copied, (len - rem)); + cio->status = -ENOMEM; + if (rem == len) { + ret = cio->status; + kfree(cio); + return ret; + } + if (cio->endio) + return cio->status; + + return blkdev_copy_wait_for_completion_io(cio); +} +EXPORT_SYMBOL_GPL(blkdev_copy_offload); + static int __blkdev_issue_write_zeroes(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned flags) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 109d9f905c3c..3b88dcd5c433 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1079,6 +1079,10 @@ int __blkdev_issue_discard(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop); int blkdev_issue_secure_erase(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp); +ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, + loff_t pos_out, size_t len, + void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp_mask); #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ From patchwork Mon May 20 10:20:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668229 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D9BF96BFA2 for ; Mon, 20 May 2024 11:44:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.24 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205500; cv=none; b=RV+6HQmlaV6PPiQSx3Nbx94WOZty8qhGVOWrm0cPD92yZnh8RMkSB9MoNCn1LVitB8sbuf9ctP78SbIWbKATuA49o8tPGIzmf7P2Owuhrd6Eeg7GekRHv1YTTjMQOPgsCbySvLijMKO4YpKNgi643Zf8Q75AgBLRl9Zj9GOjmXQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205500; c=relaxed/simple; bh=dDlxmNUmdyaVy+5Tx3fIDMiF0+e6QmCyIAL4BMRQDco=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=ibhc1FcoWJzJhDanW8to/SaOi8gZ9/8nTNVTZAYRkcYuMMO6QYzA6oo0rTkp1jmdLMOds/Mj6IOhpAWyHC3HV/JjoQOkFgPbOYYacHZdrN7mWxoZgMG08qil1QGPWsuqCLGCYVBn9ypOGSvpxYC+OI9j8akxLBhBBGcefNdhoyU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=bN89vSds; arc=none smtp.client-ip=203.254.224.24 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="bN89vSds" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20240520114457epoutp015d866fa8693f258d3c3f161f10330570~RL86Iy1Pa1820118201epoutp01b for ; Mon, 20 May 2024 11:44:57 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20240520114457epoutp015d866fa8693f258d3c3f161f10330570~RL86Iy1Pa1820118201epoutp01b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205497; bh=KzugG4K2NaDtQ5yBa1aGRu+hsulTFMGc6SCKJgEm9mk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bN89vSdsi736X9E0FW+BxlXh0yG33o0Kjnj/pAP9JDHXfKkv9QM46WuSqDQ+KVqle nwJdtIDIo5M0uF9+rHEZyq96uZTDOsvQiwYw2tZ997GLlewkqjaFEkvjC31fghFD60 YIyeVi7CxswNpBzHM7PQr8b6rJOLB8IFrrxnIVJA= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20240520114456epcas5p212c90913d64d86a15d7b279f82a18c9c~RL85TzU9v3085730857epcas5p20; Mon, 20 May 2024 11:44:56 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.182]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4VjbMp1ljMz4x9Py; Mon, 20 May 2024 11:44:54 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 2B.0D.09666.5B73B466; Mon, 20 May 2024 20:44:54 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20240520102906epcas5p15b5a0b3c8edd0bf3073030a792a328bb~RK6rliQlI1728917289epcas5p1S; Mon, 20 May 2024 10:29:05 +0000 (GMT) Received: from epsmgmc1p1new.samsung.com (unknown [182.195.42.40]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240520102905epsmtrp169417753e2cc00699cbe79d756354e11~RK6rkZBPI2026720267epsmtrp1N; Mon, 20 May 2024 10:29:05 +0000 (GMT) X-AuditID: b6c32a49-cefff700000025c2-cd-664b37b55210 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmc1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 77.ED.09238.1F52B466; Mon, 20 May 2024 19:29:05 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520102901epsmtip201169637c5bcedb66ae04a739ad6f033~RK6nifmXN2248422484epsmtip2r; Mon, 20 May 2024 10:29:01 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , Vincent Fu , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 04/12] block: add emulation for copy Date: Mon, 20 May 2024 15:50:17 +0530 Message-Id: <20240520102033.9361-5-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0VTa1BUZRie75zl7C7N1hmw/CAxZrUpcVhYYpcPAm3KaQ4ukxSjSVF4Yg+X AZZlzy5KTggSCihyUWlYRIFdQMARheUm1yBAIaQRWQTjki5ORMitLIeAWBbr3/M+7/s+zzfP Ny8Pt1kl7HkRCjWjUtBRQsKaU9e5y8m5zkMW6mo04qiqtxtHJ7NWcFQ5lkmgmc5FgHLnn+PI 1H4aoOX+ARwZuscBKiwu4KCR9kYMNRfnYKi8sgtD+d8lY6hrbZZAOR1GgKaGtBhqGd2Nik7p Oai55Q4HDd66RKArpVNcVNaziqHs1CEMNZiSAKpbvoKj6zNzHHR79HX0+EwqQAMrPVbvOVCD 92VUbzGkGrVjXGpg/CaHGuzXUNUVaQRVoz9B/VqTB6imkUSC0p07b0VlJD8lqMaUCStqYWqU Q821DhHUOUMFoH4s/IHrb/tZpHc4Q8sZlSOjCImRRyjCfISygOAPgiVSV7Gz2BN5CB0VdDTj I9zn5+/8YUTUekxCxzg6SrNO+dMsK3TZ462K0agZx/AYVu0jZJTyKKW7UsTS0axGESZSMGov saurm2R98Ehk+MjaU66y7f1jpnIjlggeSdIBjwdJd1jb4pEO+DwbsgnABzo3C14EsPqRKh1Y r+NnAObrfsHMDfN8fcmfmKXRAuDShB5YihQMZj38h2tWJcjdsG+NZ+a3kJU4PFOTzTEXOPkA hyU3nmxI2ZIIdszc5Zoxh3wT5jX1cczLAtIT1ppEFrc3YOWNdtyM+aQXbK+d3zCDZA0fDrYa uZahfXDlScXm82zhbz2GTd4eTmee2sRHYfmFq4Rl+VsAtcNaYGnshSm9mbjZGCd3wapbLhba AV7svb6hiZMvw4xl06a+ADZcfoF3wGtVhYQF20HjX0mEJVIKZl/73BJKBoDTv18lssB27f8O hQBUADtGyUaHMaxEKVYwR//7s5CY6GqwcQhOvg1gbHJe1AEwHugAkIcLtwiqDb6hNgI5Hf81 o4oJVmmiGLYDSNbzy8btXw2JWb8khTpY7O7p6i6VSt0935GKhVsFMykFchsyjFYzkQyjZFQv 9jAe3z4RI6V763fqfzYkjOZLzrddiMOUP/3R9LZtwr1tewIDRZdU0ysB45/IpoNc5lr4XiV5 /ICBDC8Hh7bjEbGdXQUTD+OdEtUviXxzXIqGMr5vmlxYTom1HtavNuvEs4ElxwSta1MBduN3 RAdH/Ny6D+XKmZBXRlWX61NPpHseOIt9k/spW1fAStLKLvbNBx901nCXSotiQ5srg2Sl7yZ/ dWhYP1v4d8OzgcOvpd1L3H7XLv7+W/Uf2ZTUV9BJQXELO76Yxrr03atU4ultj4t2Fm/V1g32 2cj6/W8mHQn1T+g9PvdxB82pLZu0p9y+HNAZorwHTx7YvxStM5gO6wP1i7dpP/v9z4UcNpwW O+Eqlv4XIAOT35EEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWRe0hTYRjG+845nh2F5WmTPHZRWdnNnC2KPqPMQvSEIgVRdiFbelylTttm WRKay9RJakrpzJhNuzjLdFszL5lMl6Wphak5mEE5DEydhdmFaTrpv4f39zzv88JLoJx6bAVx VixjJGJhPA93wQxtPC+/KZ+w2C05k1vg085XKMwosKOw2pKPw7G27wDetv1G4UhrFoB/u3tR qH81DGC5+i4Gh1obENisLkRgVbUJgXeK5Qg0zY3jsNA4AKC1vxSBL8y+8N71Sgw2v3iDwb7G MhyqHlhZ8GHHLAJvZvcj8PnIVQANf1UorBmbxOBr80r4JTcbwF57h1PQarrvQxjdqabohlIL i+4drsPovu5kWqvJwWldZRr9VacEdNNQOk5X5BU50TfkEzjdkPnJiZ6ymjF6sqUfp/P0GkC/ LW9nHeAec9kVw8SfvcBI/ANPuZwZmptgJb3clzJSNYCkg8/bFcCZoMhtVP39aUQBXAgO2QQo U98PsAg8qAf2dnRRc6mq2VHWokmOUJ8fTcwDgsBJX6prjliYu5H1KDUlz3VsQslvKNU984G1 kOaSkDKO9Tg0RvpQyqYubCHMJgOoZyP8xQIvqrq21VHmTO6kWp/ZHEdw5i2WQRteAJaWgyUa 4MEkSRNECdGCJIGYuciXChOkyWIRPzoxQQscv9204TkYVs3yjQAhgBFQBMpzY2v1+2M57Bjh pcuMJDFKkhzPSI1gJYHx3NmCkjsxHFIklDFxDJPESP5ThHBekY6I/Jf5ufF+iCLOKVIEj82D T9ALJ9qe/JEpS9DmK3KD5+8Dobe7Zq/8Gvw4A6a577P7f2X15DWfCNDVueamDmTLjv8s1uxy PvnOpI9TedQdjlOvMm5rV4dLOu2l+uOna26Vb3RrKQu3r/UvCjDnrw/2xL4dsZqITK2Bq9AJ 2gonePXLx1VLfN3zD8q4NQfZhzuepnZc21FVu9lHNnqE0CpSNtqie89vPvrOe2uAa8ia5aGa KLNYlujXYtGLigfylcesnl45JXt7ggMjdjbakoPWGOAhDNmd1phyPuv7eOOnHTPL1iuu+nEg Oo1Gev/RWU7tyYjih9QaRpWlWesiK3iY9IxQsAmVSIX/ACP3pNxKAwAA X-CMS-MailID: 20240520102906epcas5p15b5a0b3c8edd0bf3073030a792a328bb X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520102906epcas5p15b5a0b3c8edd0bf3073030a792a328bb References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: For the devices which does not support copy, copy emulation is added. It is required for in-kernel users like fabrics, where file descriptor is not available and hence they can't use copy_file_range. Copy-emulation is implemented by reading from source into memory and writing to the corresponding destination. At present in kernel user of emulation is fabrics. Signed-off-by: Nitesh Shetty Signed-off-by: Vincent Fu Signed-off-by: Anuj Gupta --- block/blk-lib.c | 223 +++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 4 + 2 files changed, 227 insertions(+) diff --git a/block/blk-lib.c b/block/blk-lib.c index e83461abb581..e0dd17f42c8e 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -26,6 +26,20 @@ struct blkdev_copy_offload_io { loff_t offset; }; +/* Keeps track of single outstanding copy emulation IO */ +struct blkdev_copy_emulation_io { + struct blkdev_copy_io *cio; + struct work_struct emulation_work; + void *buf; + ssize_t buf_len; + loff_t pos_in; + loff_t pos_out; + ssize_t len; + struct block_device *bdev_in; + struct block_device *bdev_out; + gfp_t gfp; +}; + static sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) { unsigned int discard_granularity = bdev_discard_granularity(bdev); @@ -307,6 +321,215 @@ ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, } EXPORT_SYMBOL_GPL(blkdev_copy_offload); +static void *blkdev_copy_alloc_buf(ssize_t req_size, ssize_t *alloc_size, + gfp_t gfp) +{ + int min_size = PAGE_SIZE; + char *buf; + + while (req_size >= min_size) { + buf = kvmalloc(req_size, gfp); + if (buf) { + *alloc_size = req_size; + return buf; + } + req_size >>= 1; + } + + return NULL; +} + +static struct bio *bio_map_buf(void *data, unsigned int len, gfp_t gfp) +{ + unsigned long kaddr = (unsigned long)data; + unsigned long end = (kaddr + len + PAGE_SIZE - 1) >> PAGE_SHIFT; + unsigned long start = kaddr >> PAGE_SHIFT; + const int nr_pages = end - start; + bool is_vmalloc = is_vmalloc_addr(data); + struct page *page; + int offset, i; + struct bio *bio; + + bio = bio_kmalloc(nr_pages, gfp); + if (!bio) + return ERR_PTR(-ENOMEM); + bio_init(bio, NULL, bio->bi_inline_vecs, nr_pages, 0); + + if (is_vmalloc) { + flush_kernel_vmap_range(data, len); + bio->bi_private = data; + } + + offset = offset_in_page(kaddr); + for (i = 0; i < nr_pages; i++) { + unsigned int bytes = PAGE_SIZE - offset; + + if (len <= 0) + break; + + if (bytes > len) + bytes = len; + + if (!is_vmalloc) + page = virt_to_page(data); + else + page = vmalloc_to_page(data); + if (bio_add_page(bio, page, bytes, offset) < bytes) { + /* we don't support partial mappings */ + bio_uninit(bio); + kfree(bio); + return ERR_PTR(-EINVAL); + } + + data += bytes; + len -= bytes; + offset = 0; + } + + return bio; +} + +static void blkdev_copy_emulation_work(struct work_struct *work) +{ + struct blkdev_copy_emulation_io *emulation_io = container_of(work, + struct blkdev_copy_emulation_io, emulation_work); + struct blkdev_copy_io *cio = emulation_io->cio; + struct bio *read_bio, *write_bio; + loff_t pos_in = emulation_io->pos_in, pos_out = emulation_io->pos_out; + ssize_t rem, chunk; + int ret = 0; + + for (rem = emulation_io->len; rem > 0; rem -= chunk) { + chunk = min_t(int, emulation_io->buf_len, rem); + + read_bio = bio_map_buf(emulation_io->buf, + emulation_io->buf_len, + emulation_io->gfp); + if (IS_ERR(read_bio)) { + ret = PTR_ERR(read_bio); + break; + } + read_bio->bi_opf = REQ_OP_READ | REQ_SYNC; + bio_set_dev(read_bio, emulation_io->bdev_in); + read_bio->bi_iter.bi_sector = pos_in >> SECTOR_SHIFT; + read_bio->bi_iter.bi_size = chunk; + ret = submit_bio_wait(read_bio); + kfree(read_bio); + if (ret) + break; + + write_bio = bio_map_buf(emulation_io->buf, + emulation_io->buf_len, + emulation_io->gfp); + if (IS_ERR(write_bio)) { + ret = PTR_ERR(write_bio); + break; + } + write_bio->bi_opf = REQ_OP_WRITE | REQ_SYNC; + bio_set_dev(write_bio, emulation_io->bdev_out); + write_bio->bi_iter.bi_sector = pos_out >> SECTOR_SHIFT; + write_bio->bi_iter.bi_size = chunk; + ret = submit_bio_wait(write_bio); + kfree(write_bio); + if (ret) + break; + + pos_in += chunk; + pos_out += chunk; + } + cio->status = ret; + kvfree(emulation_io->buf); + kfree(emulation_io); + blkdev_copy_endio(cio); +} + +static inline ssize_t queue_max_hw_bytes(struct request_queue *q) +{ + return min_t(ssize_t, queue_max_hw_sectors(q) << SECTOR_SHIFT, + queue_max_segments(q) << PAGE_SHIFT); +} +/* + * @bdev_in: source block device + * @pos_in: source offset + * @bdev_out: destination block device + * @pos_out: destination offset + * @len: length in bytes to be copied + * @endio: endio function to be called on completion of copy operation, + * for synchronous operation this should be NULL + * @private: endio function will be called with this private data, + * for synchronous operation this should be NULL + * @gfp_mask: memory allocation flags (for bio_alloc) + * + * For synchronous operation returns the length of bytes copied or error + * For asynchronous operation returns -EIOCBQUEUED or error + * + * Description: + * If native copy offload feature is absent, caller can use this function + * to perform copy. + * We store information required to perform the copy along with temporary + * buffer allocation. We async punt copy emulation to a worker. And worker + * performs copy in 2 steps. + * 1. Read data from source to temporary buffer + * 2. Write data to destination from temporary buffer + */ +ssize_t blkdev_copy_emulation(struct block_device *bdev_in, loff_t pos_in, + struct block_device *bdev_out, loff_t pos_out, + size_t len, void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp) +{ + struct request_queue *in = bdev_get_queue(bdev_in); + struct request_queue *out = bdev_get_queue(bdev_out); + struct blkdev_copy_emulation_io *emulation_io; + struct blkdev_copy_io *cio; + ssize_t ret; + size_t max_hw_bytes = min(queue_max_hw_bytes(in), + queue_max_hw_bytes(out)); + + ret = blkdev_copy_sanity_check(bdev_in, pos_in, bdev_out, pos_out, len); + if (ret) + return ret; + + cio = kzalloc(sizeof(*cio), gfp); + if (!cio) + return -ENOMEM; + + cio->waiter = current; + cio->copied = len; + cio->endio = endio; + cio->private = private; + + emulation_io = kzalloc(sizeof(*emulation_io), gfp); + if (!emulation_io) + goto err_free_cio; + emulation_io->cio = cio; + INIT_WORK(&emulation_io->emulation_work, blkdev_copy_emulation_work); + emulation_io->pos_in = pos_in; + emulation_io->pos_out = pos_out; + emulation_io->len = len; + emulation_io->bdev_in = bdev_in; + emulation_io->bdev_out = bdev_out; + emulation_io->gfp = gfp; + + emulation_io->buf = blkdev_copy_alloc_buf(min(max_hw_bytes, len), + &emulation_io->buf_len, gfp); + if (!emulation_io->buf) + goto err_free_emulation_io; + + schedule_work(&emulation_io->emulation_work); + + if (cio->endio) + return -EIOCBQUEUED; + + return blkdev_copy_wait_for_completion_io(cio); + +err_free_emulation_io: + kfree(emulation_io); +err_free_cio: + kfree(cio); + return -ENOMEM; +} +EXPORT_SYMBOL_GPL(blkdev_copy_emulation); + static int __blkdev_issue_write_zeroes(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned flags) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 3b88dcd5c433..8b1edb46880a 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1083,6 +1083,10 @@ ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, loff_t pos_out, size_t len, void (*endio)(void *, int, ssize_t), void *private, gfp_t gfp_mask); +ssize_t blkdev_copy_emulation(struct block_device *bdev_in, loff_t pos_in, + struct block_device *bdev_out, loff_t pos_out, + size_t len, void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp); #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ From patchwork Mon May 20 10:20:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668230 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFBC250286 for ; Mon, 20 May 2024 11:45:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205506; cv=none; b=aANI/LTrzACSndOGQsjNvqMmPQG8rmDE9QNNXsE9IX5A6mT4hBeam9D8shHP/KwDnjqGD8ffCxtLcdqsn2TRIRvP79Q6NUNCA+j51wIpcSUMull2h5b5prRR6bsNNOBNb7ZDJIRhKUlVLiW0Jm3i8yxBrDTy0vw0yJmfhpePxtE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205506; c=relaxed/simple; bh=Ac3WHWfXlPQZczxqpVamDMZW8w7I6olApg/rcA7vauo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=C4p7DDCkTkKC3Y4kLduGm2aq418DvQA/9nYE0DNHvY8tBNuxX+864cRzcL+rzEXjSoPzTNrMpnsj65mi6T856qToxXYaZFqQt2/HJWz82v3gjMrn7ZDBp7ZDqkJGK9KqKX/drp8+2Tub6jeWK+uW5XV8aaIwoRVU/3E5YLVamFk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=Qx88QH5I; arc=none smtp.client-ip=203.254.224.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="Qx88QH5I" Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20240520114502epoutp04123289423c8c45a522e91570ace1d64f~RL8-kZEWE2442124421epoutp04O for ; Mon, 20 May 2024 11:45:02 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20240520114502epoutp04123289423c8c45a522e91570ace1d64f~RL8-kZEWE2442124421epoutp04O DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205502; bh=eFjYOy/rO8gowEbK0HM6LksYvSQzjlGqTWi9wTV85D4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Qx88QH5I4iNQymT5c/u9o9/+xCJN3AHSZanjySqJrfMRkVdVmxdHSEAYxxHk1ellp v4Wp9QWBTVjtJKybh5Dac/dUYKRH9OhT0enW6oPwb+WbARKwZYDrWxv4Yhyo6mZag4 /9lUEKF/EjNEjh8WFB2Gm2g9NcHSpAyMTXMCgd8A= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20240520114502epcas5p314d3ac8847fbfa8f00e6469d2bee365a~RL8_9Pvzi2210822108epcas5p39; Mon, 20 May 2024 11:45:02 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.183]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4VjbMw3W1Yz4x9Pv; Mon, 20 May 2024 11:45:00 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id DA.60.19431.CB73B466; Mon, 20 May 2024 20:45:00 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20240520102917epcas5p1bda532309b9174bf2702081f6f58daf7~RK62SXh_40443504435epcas5p1G; Mon, 20 May 2024 10:29:17 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20240520102917epsmtrp2ac75c71748c9cbc2118a64cd866875eb~RK62RQdSo2153521535epsmtrp2y; Mon, 20 May 2024 10:29:17 +0000 (GMT) X-AuditID: b6c32a50-f57ff70000004be7-6c-664b37bccb97 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 88.08.08390.DF52B466; Mon, 20 May 2024 19:29:17 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520102913epsmtip223ed300d14416990c75ecc6ce4fcc8f9~RK6yqRUzB2247922479epsmtip2r; Mon, 20 May 2024 10:29:13 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 05/12] fs/read_write: Enable copy_file_range for block device. Date: Mon, 20 May 2024 15:50:18 +0530 Message-Id: <20240520102033.9361-6-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0WTe0xTVxzHd+693Ba2uiugO5bBWDeUx4pUaDkQUDfMcgkYWbZsy5aIDb08 Umi7PkQXpjCsAzZegm7W8RCMDhCKtCBP0xUFZXQwsTAg4DSQuTFRgWEYqa6lsP33+b2+319+ J4eNu9tILjtNpmaUMnE6j3Qj2noDAvjd4XHJIbm3IdIP9OHoyxIbjhqmikk017sA0JnHKzia MX0F0KplCEfGvmmAqmsqCDRu6sBQd80pDNU13MDQuW9zMXTj+UMSnTKPAjRr1WGoZyIInT95 gUDdPbcINNL5PYmqLs6y0KX+ZxgqzbNiqH0mB6C21SocNc09ItDNCS80ZOt32fsqPXInjh6o gXSHbopFD01fIegRi4Zuqc8nacOF4/QDw1lAd41nk3RtUZkLXZg7T9Id2rsu9JPZCYJ+dM1K 0kXGekAPVl9nJXh8Io1KZcQSRunLyJLkkjRZSjQv7v3EmEShKETAF0SgcJ6vTJzBRPP2xSfw 301Lt1+I53tYnK6xpxLEKhVv5+4opVyjZnxT5Sp1NI9RSNIVYYpglThDpZGlBMsYdaQgJGSX 0N54SJr6TdcwqZjmHOm9VM7KBqYXC4ArG1JhcLXMABzsTnUD2DkZWgDc7LwA4L1RI8sZLAP4 130da2Oi5Oc/gLPQA+Bk+RLpDLQY/HH4HFEA2GySCoI/PWc78p5UAw6/NpQSjgCnDDjM6TVh DikP6gNovKldkyUoP9hsnScczKEiYPn4b4TT7jXY0GzCHexKRUJT6+M1a0jpXeFISRHmbNoH ny0+Wd/PA/7Zb1xnLlyc7yGdnAnryn8gncMnANSN6YCzsAdqB4pxx9o4FQD1nTudaW94eqBp TR+nNsHC1Zl1Lw5sr9zgN+BlffW6/jY4+jSHdMhAiobmk0HOqxQC2PNgAi8BPrr/HaoBqAdc RqHKSGGShAoBX8Zk/vduSfKMFrD2DwIT2kFDsy3YDDA2MAPIxnmenBZjbLI7RyI++jmjlCcq NemMygyE9guW4twtSXL7R5KpEwVhESFhIpEoLCJUJOC9wpnTVkjcqRSxmpEyjIJRbsxhbFdu NuYj4lNT5yMrNscenVvU+ilW4g+8/VFa4+5+js9ojM3itd3rrUNRd49ZGlv43MG+1mBv/kXX oluueZVHVre+9+lTi9vvxwOmriY27cU3e6SvdFpYmk3jxbePkVLPLjL2gHD5O2nQvUz4kr/M 8OFCSlRevodHbtq4JX/5H+HD7vjqN3eo/Fn79ebexCydaWz48PXGusHgg9lbAoShHN93CuWR ki+yahVjry/oZ1tVSyrp8o7kybPXlqRtv4BdZzK6/MNXKz/ev52dhTXRk1dzrgwXZ/Xesf56 2eQzX7OnzG/AxI8uq5JYt82f2LpQmzUZeFpk+/uzF7wPvpwQo1+pmEwOu+/GI1SpYkEgrlSJ /wXV832lkAQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrEIsWRmVeSWpSXmKPExsWy7bCSvO5fVe80g/c9YhbrTx1jtmia8JfZ YvXdfjaL14c/MVpM+/CT2eLJgXZGi99nzzNbbDl2j9FiwaK5LBY3D+xkstizaBKTxcrVR5ks Zk9vZrI4+v8tm8WkQ9cYLZ5encVksfeWtsXCtiUsFnv2nmSxuLxrDpvF/GVP2S2WH//HZDGx 4yqTxY4njYwW237PZ7ZY9/o9i8WJW9IW5/8eZ3WQ8bh8xdvj1CIJj52z7rJ7nL+3kcXj8tlS j02rOtk8Ni+p93ixeSajx+6bDWwei/sms3r0Nr9j89jZep/V4+PTWywe7/ddZfPo27KK0ePM giPsAcJRXDYpqTmZZalF+nYJXBk9uy+wFdzjrTi8fAp7A+MB7i5GTg4JAROJCedeMnYxcnEI CexmlOg9upUFIiEpsezvEWYIW1hi5b/n7BBFzUwS65f0MXUxcnCwCWhLnP7PARIXEdjOLPGx uZsJpIFZ4CizxIU1FiC2sECQxLcND8HiLAKqEhuuvgNbwCtgKTHl5gOoZfISqzccAFvGKWAl cWDrB0YQWwio5u71D2wTGPkWMDKsYpRMLSjOTc8tNiwwykst1ytOzC0uzUvXS87P3cQIjlYt rR2Me1Z90DvEyMTBeIhRgoNZSYR30xbPNCHelMTKqtSi/Pii0pzU4kOM0hwsSuK83173pggJ pCeWpGanphakFsFkmTg4pRqYChxP7/f+uftI0t5Nne+Yjsf+/bWlkkF02Xebdg/mJyIs01Pn NJ24wXCoZaV05awODn9f41kLmOLdamMerBO0Ot9kO2v+ua0rrupMXLQsirvoSVnC2tmeVyJN jJ/52O97YLP9z4l1ta5H7PcXT+2wz2zPNTkkWZU4j7eiVjv+aeOGZT9/s7hl3v21JnA1N9sG uaqe2R7yd5hcp82cpFx7asqVSTKJN0627/j5zebNWfUHhU7lyiELrhdZ7z+6Qqv57Aa/rnl/ k2M8xZM3bi3ZwvHpsbmRsvNMv4uBj7PX/lbTvyBj+WjxD/3gZ/lvf0YWFRvcrxf8p/dx2mcJ Oe+YKtZ16kxqQcc+uYiUTfPPV2Ipzkg01GIuKk4EAIU1DcFFAwAA X-CMS-MailID: 20240520102917epcas5p1bda532309b9174bf2702081f6f58daf7 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520102917epcas5p1bda532309b9174bf2702081f6f58daf7 References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: From: Anuj Gupta This is a prep patch. Allow copy_file_range to work for block devices. Relaxing generic_copy_file_checks allows us to reuse the existing infra, instead of adding a new user interface for block copy offload. Change generic_copy_file_checks to use ->f_mapping->host for both inode_in and inode_out. Allow block device in generic_file_rw_checks. Reviewed-by: Hannes Reinecke Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty Reviewed-by: Hannes Reinecke --- fs/read_write.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/read_write.c b/fs/read_write.c index ef6339391351..31645ca5ed58 100644 --- a/fs/read_write.c +++ b/fs/read_write.c @@ -1413,8 +1413,8 @@ static int generic_copy_file_checks(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, size_t *req_count, unsigned int flags) { - struct inode *inode_in = file_inode(file_in); - struct inode *inode_out = file_inode(file_out); + struct inode *inode_in = file_in->f_mapping->host; + struct inode *inode_out = file_out->f_mapping->host; uint64_t count = *req_count; loff_t size_in; int ret; @@ -1726,7 +1726,9 @@ int generic_file_rw_checks(struct file *file_in, struct file *file_out) /* Don't copy dirs, pipes, sockets... */ if (S_ISDIR(inode_in->i_mode) || S_ISDIR(inode_out->i_mode)) return -EISDIR; - if (!S_ISREG(inode_in->i_mode) || !S_ISREG(inode_out->i_mode)) + if (!S_ISREG(inode_in->i_mode) && !S_ISBLK(inode_in->i_mode)) + return -EINVAL; + if ((inode_in->i_mode & S_IFMT) != (inode_out->i_mode & S_IFMT)) return -EINVAL; if (!(file_in->f_mode & FMODE_READ) || From patchwork Mon May 20 10:20:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668231 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE2736E5EC for ; Mon, 20 May 2024 11:45:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.24 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205511; cv=none; b=JBImQfybemCxxbLhHUxx1v8tsDue0clHBMOLpFx6Xsn5pLK2c0roXZL4e0sEBIoeI+/RmB8d7XotbGI86mfUO8gRsssVNLBkzUPMmgqMLRdL8L/T/u8XI/HdYWNMrZVwgdsIqJMXZFlLZJrwLYv7tzDn7VBIFSaDzGmPTAZzC2A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205511; c=relaxed/simple; bh=e665OnEyWefogk0vb4VFDyJsEhhglmgHT8IJCAx1QrY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=AKc3Lbow4KSvcA6anGEmX1ZM8PR8xz/CeaM4J1Hr3WRiriwEMWKMwuQTM00QmjKDLbNZnpmg4KaLwrnrePOsn+/l3lVcoybBm5DEax5MAWaPMIea7u8Ei0/hAdQ39g2lMQdh0ouXLjsOXWHaTl2xNOK7Q2pcqH66ck6nQSMw45E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=nV3cn/Hr; arc=none smtp.client-ip=203.254.224.24 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="nV3cn/Hr" Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20240520114507epoutp0157f629a6712b94eacddd46b025db41e2~RL9D9dKcT1906819068epoutp01v for ; Mon, 20 May 2024 11:45:07 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20240520114507epoutp0157f629a6712b94eacddd46b025db41e2~RL9D9dKcT1906819068epoutp01v DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205507; bh=/5WrYlM6k89NsKl1z4Z8uUwH+ZTcI7GeL+q1n6MWvbc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nV3cn/HrIJ9I28gSoSLooLCP96sKlzwp3oQUuSxAUMkW3pUZbQiY3K53BzdnJw0qS /53m/nQf/aAhBuC9Cn0FI5AeiKIJynqjphP/MppiyBqSa1soBCVMmeNdu8lOrQyGOr RJ8ctPUyLjUJLFJnHyu1rv3yJzh6O8cfAy+odbsI= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20240520114507epcas5p1770da6e88bb8e69b58441ec32768b20f~RL9DbPS8Z0060100601epcas5p1H; Mon, 20 May 2024 11:45:07 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.178]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4VjbN12cCRz4x9Pv; Mon, 20 May 2024 11:45:05 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 96.1D.09666.1C73B466; Mon, 20 May 2024 20:45:05 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20240520102929epcas5p2f4456f6fa0005d90769615eb2c2bf273~RK7BClKh10987709877epcas5p2U; Mon, 20 May 2024 10:29:29 +0000 (GMT) Received: from epsmgmc1p1new.samsung.com (unknown [182.195.42.40]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20240520102929epsmtrp2668c16e2b9ae57c2df89353a51211e4a~RK7BBRsWd2153521535epsmtrp2_; Mon, 20 May 2024 10:29:29 +0000 (GMT) X-AuditID: b6c32a49-f53fa700000025c2-e8-664b37c1a522 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmc1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 2B.ED.09238.8062B466; Mon, 20 May 2024 19:29:28 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520102925epsmtip265107529beb8c915e4bcd516677cf197~RK69azK9t2204722047epsmtip2F; Mon, 20 May 2024 10:29:25 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 06/12] fs, block: copy_file_range for def_blk_ops for direct block device Date: Mon, 20 May 2024 15:50:19 +0530 Message-Id: <20240520102033.9361-7-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0VTa1ATVxT27oZNgEZXou2VguDaygjDIwp4QwFRnHYRZ0ot/VFtJ+6Q5SGQ pEkQ25mOQQVbNCLIwyY80lTlEQcqAQoqjwaBilIYMDhgeXQGUEsllFqwZdQCCe2/7/vO+c45 99w5PNzlJeHKS5KqWIWUSaEIJ05D+3Zv3x93RccHlGWvRTXdnTg6eeEFjowjOQSabp8DqHD2 bxxNtJ0BaLGnF0d1naMA6Q0lHDTU1oShW4Y8DFUaOzCkKzqFoY5XTwmUZx4EaNKixVDzsA/6 NusyB91qvsNBAzeKCVR2dZKLyrteYij3KwuGGicyAGpYLMNR9bSVg34afhP1vuhyiHCjB+5H 090GSDdpR7h07+h1Dj3Qk0bXVn1N0KbLJ+jHpm8AfXNITdDfnb/oQGtOzRB0U+aYA/3H5DCH trZYCPp8XRWg7+lvc2MEh5JDE1lGwio8WWmcTJIkTQijoj8UR4qDggOEvkIR2kV5SplUNoza dyDG992klKUNUZ7HmJS0JSmGUSop//BQhSxNxXomypSqMIqVS1LkgXI/JZOqTJMm+ElZVYgw IGBH0FLikeTEh6ZxB/lz8ri+os9BDX7lZwNHHiQDYeuQBssGTjwX8iaA5boCO5kDcGiqgbCR eQC1RY3EqmVBv2ppBrC54SzXRjIxOFg8gGcDHo8gfeDdV7xlfQNpxOFZUy5nmeCkCYcZ7W3Y cikBKYYd9VfBMuaQb0Nd37OVFnxSBMcWzFxbOw9o/L4NX8aOZAhsq58Fy4UgWeMIe07qcFvS Pniue8Y+nwD+1lVnN7vCP2ea7Xo6rMyvIGzm00sPeqAFtsBumNmdszI2Tm6HNTf8bbI7LOiu XhkUJ9dCzeIEZtP5sLF0FW+F12r09vqb4OBChh3TcOSH+/YdaQCcrSgGF8Bm7f8t9ABUgU2s XJmawCqD5EIpm/7fx8XJUmvByiF4RzWCkfFZPzPAeMAMIA+nNvBr66LiXfgS5vMvWIVMrEhL YZVmELS0wVzcdWOcbOmSpCqxMFAUEBgcHBwo2hkspN7gT2eWSFzIBEbFJrOsnFWs+jCeo6sa 0136R5PoFRQOuGeUhX3b7hxY51Ry+Hb7Xe5bUu/Twwcp8bZzzS2DRfOt+PpI1xPHiAdNzua2 NQzT2TLlrr40FRjy3kGYWfp0yNmtNKml0nrUas7r13LnrbWWEWFV4bghyqvB55Nn4dU6Z49P 1a1Gw+MCQavgSkT95N7dXk8Ytz3Ov1CqR/uffPl+fno5ZTniYazweb5HeP3K+EzkRo/cwxGC xtC/Io6OEdr9e48Lr92Dhi1qrohoycr/2V22tXP09d/nPrJoumNjM3YeKvCx9mz+OL00bfyd zILXCqxVa/zFZpF6bcOA5IPFHXGmLE5qf0fsw5z18i0h1vj+R67Cz9YNUhxlIiP0xhVK5l9E 6d4ekQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrEIsWRmVeSWpSXmKPExsWy7bCSvC6HmneawZrz5hbrTx1jtmia8JfZ YvXdfjaL14c/MVpM+/CT2eLJgXZGi99nzzNbbDl2j9FiwaK5LBY3D+xkstizaBKTxcrVR5ks Zk9vZrI4+v8tm8WkQ9cYLZ5encVksfeWtsXCtiUsFnv2nmSxuLxrDpvF/GVP2S2WH//HZDGx 4yqTxY4njYwW237PZ7ZY9/o9i8WJW9IW5/8eZ3WQ8bh8xdvj1CIJj52z7rJ7nL+3kcXj8tlS j02rOtk8Ni+p93ixeSajx+6bDWwei/sms3r0Nr9j89jZep/V4+PTWywe7/ddZfPo27KK0ePM giPsAcJRXDYpqTmZZalF+nYJXBm3Nz9gLfghULFgxQXWBsaHvF2MnBwSAiYS3xf0MnUxcnEI CexmlPh9YhMrREJSYtnfI8wQtrDEyn/P2SGKmpkkVj0/CuRwcLAJaEuc/s8BEhcR2M4s8bG5 mwmkgVngKLPEhTUWILawQKxEz79WsEEsAqoSsy98YQOxeQUsJe5/P8QOsUBeYvWGA2A1nAJW Ege2fmAEsYWAau5e/8A2gZFvASPDKkbJ1ILi3PTcZMMCw7zUcr3ixNzi0rx0veT83E2M4GjV 0tjBeG/+P71DjEwcjIcYJTiYlUR4N23xTBPiTUmsrEotyo8vKs1JLT7EKM3BoiTOazhjdoqQ QHpiSWp2ampBahFMlomDU6qBKSotOnXuR/VM58hf25ILLxm5WxavPLVTc2mSdC975OM9UYmV Rdf89l48HlNgvNOw2m+LUtPHs9+1ju81+aG4v3b577Qfqw+1aHhdbnzdcKBy5ocMhoi5m39c eqCcOLF2ytTrD8xrNWvnyAtuXZrVPbU69VFrXWVoiM/8eyGbFz4TXLkoakG2nXZY4v7ma09U xdby9J70DAo21a9NDl7xOeHvU8G5fP7fa2buMZ8dO+v3d1fpPq9AydCOKN+PRnvXaMVf2nVk Z0tbqcvKty+6sp+H2HsoaedHxZ9zmbjr+3GpNBWWE+ftTZZebsvaZ2WbKLrb7Nibt4dmsTCG Xi4UjG5c8vnYg0T5ecKfn4mfVGIpzkg01GIuKk4EANH/NelFAwAA X-CMS-MailID: 20240520102929epcas5p2f4456f6fa0005d90769615eb2c2bf273 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520102929epcas5p2f4456f6fa0005d90769615eb2c2bf273 References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: For direct block device opened with O_DIRECT, use blkdev_copy_offload to issue device copy offload, or use splice_copy_file_range in case device copy offload capability is absent or the device files are not open with O_DIRECT. Reviewed-by: Hannes Reinecke Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty --- block/fops.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/block/fops.c b/block/fops.c index 376265935714..5a4bba4f43aa 100644 --- a/block/fops.c +++ b/block/fops.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "blk.h" static inline struct inode *bdev_file_inode(struct file *file) @@ -754,6 +755,30 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to) return ret; } +static ssize_t blkdev_copy_file_range(struct file *file_in, loff_t pos_in, + struct file *file_out, loff_t pos_out, + size_t len, unsigned int flags) +{ + struct block_device *in_bdev = I_BDEV(bdev_file_inode(file_in)); + struct block_device *out_bdev = I_BDEV(bdev_file_inode(file_out)); + ssize_t copied = 0; + + if ((in_bdev == out_bdev) && bdev_max_copy_sectors(in_bdev) && + (file_in->f_iocb_flags & IOCB_DIRECT) && + (file_out->f_iocb_flags & IOCB_DIRECT)) { + copied = blkdev_copy_offload(in_bdev, pos_in, pos_out, len, + NULL, NULL, GFP_KERNEL); + if (copied < 0) + copied = 0; + } else { + copied = splice_copy_file_range(file_in, pos_in + copied, + file_out, pos_out + copied, + len - copied); + } + + return copied; +} + #define BLKDEV_FALLOC_FL_SUPPORTED \ (FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE | \ FALLOC_FL_ZERO_RANGE | FALLOC_FL_NO_HIDE_STALE) @@ -859,6 +884,7 @@ const struct file_operations def_blk_fops = { .splice_write = iter_file_splice_write, .fallocate = blkdev_fallocate, .fop_flags = FOP_BUFFER_RASYNC, + .copy_file_range = blkdev_copy_file_range, }; static __init int blkdev_init(void) From patchwork Mon May 20 10:20:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668232 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E88255E74 for ; Mon, 20 May 2024 11:45:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205516; cv=none; b=RPkG332iyqv7uqzH/ht6wjKsVlu7qAKQ0YPDAKVgkeJnqjlM1kM78XtjMBR93bnG8+PDef49KrOQ1RBZvXIxeiyrdXGeX/QfoPiyCm2gNkOK4G81BgHggd5dodaoa8J9DSzJuK3adbwjyn0T+h0EZ1kcTfbT67Yh9+irhqI7LC4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205516; c=relaxed/simple; bh=zbdY5tHhklQFu9SYQEcT9ClPqLcCjca9x8/uc+/Cb44=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:MIME-Version: Content-Type:References; b=WxdePxiYI968N/USqe7WxI5r62kqzcZVw38e/aoWoTurb5UcyZnVRb/5vC1OX0J9AQ5+K1uU0R0VneHRGKTrMoIgmSh0tuISe11uCdKpL526Fd48m+9tcN22FBOr9z9eQVhqmUYXpN7vjBCPNMHI4mLpDmdpAkIDCPWFZquxPbc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=a9+LN0ba; arc=none smtp.client-ip=203.254.224.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="a9+LN0ba" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20240520114512epoutp0438e6ca81302d7b22513f76e4f4669bef~RL9IG5JQI2508225082epoutp04- for ; Mon, 20 May 2024 11:45:12 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20240520114512epoutp0438e6ca81302d7b22513f76e4f4669bef~RL9IG5JQI2508225082epoutp04- DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205512; bh=UmOrteyCtGQQIJEluJXtswKB0vgH3ahj62ckBz59El4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a9+LN0baY/ri82hR+zhLMzjX49of98wRHdDkBMY8mQhuTYgBcMvczhPgcTv1wT5Jm Il6pApyJ6tooctWR3yW6xJz8sRRcX9B5jSQlYdOzi9upplyMqoyJT5gQf00TbFr2oe 69aqQ7lZ02z3OBQtp1Nv5jsqWveVZjx4VUR2Ni4U= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20240520114511epcas5p2d4891faf656d4cb96755fcae21b857e3~RL9HoqAXU2455024550epcas5p2d; Mon, 20 May 2024 11:45:11 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.180]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4VjbN63Tn8z4x9Pt; Mon, 20 May 2024 11:45:10 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 1D.E9.09665.6C73B466; Mon, 20 May 2024 20:45:10 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20240520102940epcas5p2b5f38ceabe94bed3905fb386a0d65ec7~RK7L_xfHU0987709877epcas5p2p; Mon, 20 May 2024 10:29:40 +0000 (GMT) Received: from epsmgmcp1.samsung.com (unknown [182.195.42.82]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240520102940epsmtrp1fa8f72a8e159b4bf0fd8e9abb46225c8~RK7L9y_hp2026620266epsmtrp1d; Mon, 20 May 2024 10:29:40 +0000 (GMT) X-AuditID: b6c32a4b-829fa700000025c1-08-664b37c6c0cd Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmcp1.samsung.com (Symantec Messaging Gateway) with SMTP id 6A.07.19234.4162B466; Mon, 20 May 2024 19:29:40 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520102936epsmtip2263bba809d8095e622cf6fc493b8814e~RK7IMFrVF2247422474epsmtip26; Mon, 20 May 2024 10:29:36 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , =?utf-8?q?Javier_Gonz=C3=A1lez?= , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 07/12] nvme: add copy offload support Date: Mon, 20 May 2024 15:50:20 +0530 Message-Id: <20240520102033.9361-8-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Te0xTdxjd797LbWErq0XxJ8aNlWwoBKTK48dLnWPkOhbCZrJkLlvp6BUY pW3aIqhZRGgBcTydI5QJFRCUjoe8xhuCvESBCcIGAjpekyLCZMtmEFhLi/O/852ck5PzffmY OKeSYcMMFytomVgg4pIWRO2tfXudujwCTrrkdlih8t4uHMWlr+FIO5FGooVbzwD6Yfk5jmba EgFa7RvAUXXXJECa/CsEGm2rx1BTfiaGbmg7MZSTFY+h6fEVBurcWCRRZvsIQLPDagw1jzmi qwmFBGpqvk2goYYfSZRXNMtAxd3rGMpIGsZQ3cx5gGpX83BUtrBEoJ6x3WhgrdvsyB5q6H4A 1ZsPqXr1BIMamLxJUEN9UVRlyQWSqio8Rz2uygZU42gsSRWkXjKjUuKfklS96qEZ9efsGEEt tQyTVGp1CaDuajoYQVYnInzCaIGQltnS4hCJMFwc6ssNOM7/gO/m7sJz4nkiD66tWBBJ+3L9 Pg5y8g8X6dfEtT0lEEXpqSCBXM7df8hHJolS0LZhErnCl0tLhSKpq9RZLoiUR4lDncW0wovn 4nLATS8Mjgh7PNcHpOOBMe2l9UQsSDqSDMyZkO0KX/ytJZOBBZPDbgTwctwMZhyeAdg5nmX2 cogdTQdbFk1xh5kBc9j1ACpHnYwiFQZLH/XjyYDJJNmO8M4G08BvZ2txeLEqgzAMOHsFh/0P Hm66rdiecC67BDdggv0uXLw5u5nA0vOauN9JY9rbUFvRtqkxZ3vBtpplk2YbvJ09QxgwrtfE 1+TghgDInjOHTeuzDKPZD/aUFuBGbAV13dUm3gbOpyWYcDS88f110mhWAqj+VW3qeRiqetM2 6+DsfbC8Yb+R3gMv95ZhxmBLmLI6gxl5FqzL3cJ28KdyjanALjjyz3kTpqBW2QmM60oB8O76 d0Q6sFW/Ukj9SiH1/9EagJeAXbRUHhlKy92kB8V09Ms7h0giK8Hm8zgE1IGpR8vO7QBjgnYA mTh3O6uy+thJDksoOH2Glkn4sigRLW8HbvqNZ+A2O0Ik+u8TK/g8V08XV3d3d1fPg+487k7W guqKkMMOFSjoCJqW0rItH8Y0t4nFMg4rszIDClpf35bYItJZEv9e/WxaZ/mJKNghq/Qv6YN5 VV/eabDTLvvMga+WKnL6p775emJe3VB6kcii7GMS1ppTP38r79OzpE/F1B/Buh2rbkTkE3uH 5LkinwbNRKz/XA7nNR4W4aTzslvoDPwFOb4TcWltsKb420Tq3CLrvVMtY7XZX6gHZGXRUSte gYW5+EZP45tP5semjw4cH96Qvr9SsFvosbzmzT974SN+ebhnbpFFzF5f/zfiGhZajw57nYCH 7nz5gvYeFFuPTAYKwnXOjd5PHQd/vn/tmPLetQ+Tkp5ft5a22f3W+kwyqLK2t87pDuxNs/S6 Vx7P9FNqCWdzpZBLyMMEPAdcJhf8B28CVN3FBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrBIsWRmVeSWpSXmKPExsWy7bCSvK6ImneawcqNGhbrTx1jtmia8JfZ YvXdfjaL14c/MVpM+/CT2eLJgXZGi99nzzNbbDl2j9FiwaK5LBY3D+xkstizaBKTxcrVR5ks Zk9vZrJ4fOczu8XR/2/ZLCYdusZo8fTqLCaLvbe0LRa2LWGx2LP3JIvF5V1z2CzmL3vKbrH8 +D8mi4kdV5ksdjxpZLTY9ns+s8W61+9ZLE7ckrY4//c4q4Osx+Ur3h6nFkl47Jx1l93j/L2N LB6Xz5Z6bFrVyeaxeUm9x4vNMxk9dt9sYPNY3DeZ1aO3+R2bx87W+6weH5/eYvF4v+8qm0ff llWMHmcWHGEPEI7isklJzcksSy3St0vgynjx7CxjwR2/ikNrd7I0MHY4dDFyckgImEgsWH6E tYuRi0NIYDujxLHTH1ghEpISy/4eYYawhSVW/nvODmILCTQzSSw9JtLFyMHBJqAtcfo/B0iv iMB2ZomPzd1MIDXMAq0sEnefVoHYwgKWEs9mrgKbwyKgKvF241NGEJsXKL6g6SEbxHx5idUb DoDVcApYSRzY+oERYpelxN3rH9gg6gUlTs58wgKyl1lAXWL9PCGIVfISzVtnM09gFJyFpGoW QtUsJFULGJlXMYqmFhTnpucmFxjqFSfmFpfmpesl5+duYgQnDq2gHYzL1v/VO8TIxMF4iFGC g1lJhHfTFs80Id6UxMqq1KL8+KLSnNTiQ4zSHCxK4rzKOZ0pQgLpiSWp2ampBalFMFkmDk6p BqaYZ994TkRXuVxYfvjynCzdL48TxBVNL9puKpSNOyGozWtRfdfFzcP87Z/9J40E3kj93Hn9 ysTe4Bfze5/vmxtpwPm+e1eKnGsltwa3yoSfcxRN1BpDDp+QTpcsuiZR9GL1Dk69ecF1HAw8 t0/M/LCv+9Iu1gY2taPrdD9w2Vb67rgQG38zar/Cee7vyZsTm1Sja2YxNK3s/rHsw5vLLa09 AW1VK7+on4n6onH7p+06px2tfru/qBgtKv1c8M7RmPfyHJ2DFzqkF9/RP3CCqzKgvuhM9PGD ASZbWxlS/+xx0p7KeHB5W3H56RzDVRtsXEJvTlljoXP66nava1cnHbBfFeIXzfXhwM8dDbWV u64psRRnJBpqMRcVJwIAO0Fg4YsDAAA= X-CMS-MailID: 20240520102940epcas5p2b5f38ceabe94bed3905fb386a0d65ec7 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520102940epcas5p2b5f38ceabe94bed3905fb386a0d65ec7 References: <20240520102033.9361-1-nj.shetty@samsung.com> Current design only supports single source range. We receive a request with REQ_OP_COPY_DST. Parse this request which consists of dst(1st) and src(2nd) bios. Form a copy command (TP 4065) trace event support for nvme_copy_cmd. Set the device copy limits to queue limits. Reviewed-by: Hannes Reinecke Signed-off-by: Kanchan Joshi Signed-off-by: Nitesh Shetty Signed-off-by: Javier González Signed-off-by: Anuj Gupta --- drivers/nvme/host/constants.c | 1 + drivers/nvme/host/core.c | 81 ++++++++++++++++++++++++++++++++++- drivers/nvme/host/trace.c | 19 ++++++++ include/linux/blkdev.h | 1 + include/linux/nvme.h | 43 +++++++++++++++++-- 5 files changed, 141 insertions(+), 4 deletions(-) diff --git a/drivers/nvme/host/constants.c b/drivers/nvme/host/constants.c index 6f2ebb5fcdb0..01b02e76e070 100644 --- a/drivers/nvme/host/constants.c +++ b/drivers/nvme/host/constants.c @@ -19,6 +19,7 @@ static const char * const nvme_ops[] = { [nvme_cmd_resv_report] = "Reservation Report", [nvme_cmd_resv_acquire] = "Reservation Acquire", [nvme_cmd_resv_release] = "Reservation Release", + [nvme_cmd_copy] = "Copy Offload", [nvme_cmd_zone_mgmt_send] = "Zone Management Send", [nvme_cmd_zone_mgmt_recv] = "Zone Management Receive", [nvme_cmd_zone_append] = "Zone Append", diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 954f850f113a..238905f9fff6 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -795,6 +795,66 @@ static inline void nvme_setup_flush(struct nvme_ns *ns, cmnd->common.nsid = cpu_to_le32(ns->head->ns_id); } +static inline blk_status_t nvme_setup_copy_offload(struct nvme_ns *ns, + struct request *req, + struct nvme_command *cmnd) +{ + struct nvme_copy_range *range = NULL; + struct bio *bio; + u64 dst_lba = 0, src_lba = 0, n_lba = 0; + u16 nr_range = 1, control = 0, seg = 1; + + if (blk_rq_nr_phys_segments(req) != BLK_COPY_MAX_SEGMENTS) + return BLK_STS_IOERR; + + /* + * First bio contains information about destination and last bio + * contains information about source. + */ + __rq_for_each_bio(bio, req) { + if (seg == blk_rq_nr_phys_segments(req)) { + src_lba = nvme_sect_to_lba(ns->head, + bio->bi_iter.bi_sector); + if (n_lba != + bio->bi_iter.bi_size >> ns->head->lba_shift) + return BLK_STS_IOERR; + } else { + dst_lba = nvme_sect_to_lba(ns->head, + bio->bi_iter.bi_sector); + n_lba = bio->bi_iter.bi_size >> ns->head->lba_shift; + } + seg++; + } + + if (req->cmd_flags & REQ_FUA) + control |= NVME_RW_FUA; + + if (req->cmd_flags & REQ_FAILFAST_DEV) + control |= NVME_RW_LR; + + memset(cmnd, 0, sizeof(*cmnd)); + cmnd->copy.opcode = nvme_cmd_copy; + cmnd->copy.nsid = cpu_to_le32(ns->head->ns_id); + cmnd->copy.control = cpu_to_le16(control); + cmnd->copy.sdlba = cpu_to_le64(dst_lba); + cmnd->copy.nr_range = 0; + + range = kmalloc_array(nr_range, sizeof(*range), + GFP_ATOMIC | __GFP_NOWARN); + if (!range) + return BLK_STS_RESOURCE; + + range[0].slba = cpu_to_le64(src_lba); + range[0].nlb = cpu_to_le16(n_lba - 1); + + req->special_vec.bv_page = virt_to_page(range); + req->special_vec.bv_offset = offset_in_page(range); + req->special_vec.bv_len = sizeof(*range) * nr_range; + req->rq_flags |= RQF_SPECIAL_PAYLOAD; + + return BLK_STS_OK; +} + static blk_status_t nvme_setup_discard(struct nvme_ns *ns, struct request *req, struct nvme_command *cmnd) { @@ -1041,6 +1101,11 @@ blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req) case REQ_OP_ZONE_APPEND: ret = nvme_setup_rw(ns, req, cmd, nvme_cmd_zone_append); break; + case REQ_OP_COPY_DST: + ret = nvme_setup_copy_offload(ns, req, cmd); + break; + case REQ_OP_COPY_SRC: + return BLK_STS_IOERR; default: WARN_ON_ONCE(1); return BLK_STS_IOERR; @@ -1218,7 +1283,7 @@ EXPORT_SYMBOL_NS_GPL(nvme_passthru_end, NVME_TARGET_PASSTHRU); /* * Recommended frequency for KATO commands per NVMe 1.4 section 7.12.1: - * + * * The host should send Keep Alive commands at half of the Keep Alive Timeout * accounting for transport roundtrip times [..]. */ @@ -1802,6 +1867,18 @@ static void nvme_config_discard(struct nvme_ns *ns, struct queue_limits *lim) lim->max_discard_segments = NVME_DSM_MAX_RANGES; } +static void nvme_config_copy(struct nvme_ns *ns, struct nvme_id_ns *id, + struct queue_limits *lim) +{ + struct nvme_ctrl *ctrl = ns->ctrl; + + if (ctrl->oncs & NVME_CTRL_ONCS_COPY) + lim->max_copy_hw_sectors = + nvme_lba_to_sect(ns->head, id->mssrl); + else + lim->max_copy_hw_sectors = 0; +} + static bool nvme_ns_ids_equal(struct nvme_ns_ids *a, struct nvme_ns_ids *b) { return uuid_equal(&a->uuid, &b->uuid) && @@ -2098,6 +2175,7 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns, if (!nvme_update_disk_info(ns, id, &lim)) capacity = 0; nvme_config_discard(ns, &lim); + nvme_config_copy(ns, id, &lim); if (IS_ENABLED(CONFIG_BLK_DEV_ZONED) && ns->head->ids.csi == NVME_CSI_ZNS) nvme_update_zone_info(ns, &lim, &zi); @@ -4833,6 +4911,7 @@ static inline void _nvme_check_size(void) BUILD_BUG_ON(sizeof(struct nvme_download_firmware) != 64); BUILD_BUG_ON(sizeof(struct nvme_format_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_dsm_cmd) != 64); + BUILD_BUG_ON(sizeof(struct nvme_copy_command) != 64); BUILD_BUG_ON(sizeof(struct nvme_write_zeroes_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_abort_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_get_log_page_command) != 64); diff --git a/drivers/nvme/host/trace.c b/drivers/nvme/host/trace.c index 0288315f0050..dfc97fff886b 100644 --- a/drivers/nvme/host/trace.c +++ b/drivers/nvme/host/trace.c @@ -153,6 +153,23 @@ static const char *nvme_trace_read_write(struct trace_seq *p, u8 *cdw10) return ret; } +static const char *nvme_trace_copy(struct trace_seq *p, u8 *cdw10) +{ + const char *ret = trace_seq_buffer_ptr(p); + u64 sdlba = get_unaligned_le64(cdw10); + u8 nr_range = get_unaligned_le16(cdw10 + 8); + u16 control = get_unaligned_le16(cdw10 + 10); + u32 dsmgmt = get_unaligned_le32(cdw10 + 12); + u32 reftag = get_unaligned_le32(cdw10 + 16); + + trace_seq_printf(p, + "sdlba=%llu, nr_range=%u, ctrl=0x%x, dsmgmt=%u, reftag=%u", + sdlba, nr_range, control, dsmgmt, reftag); + trace_seq_putc(p, 0); + + return ret; +} + static const char *nvme_trace_dsm(struct trace_seq *p, u8 *cdw10) { const char *ret = trace_seq_buffer_ptr(p); @@ -340,6 +357,8 @@ const char *nvme_trace_parse_nvm_cmd(struct trace_seq *p, return nvme_trace_resv_rel(p, cdw10); case nvme_cmd_resv_report: return nvme_trace_resv_report(p, cdw10); + case nvme_cmd_copy: + return nvme_trace_copy(p, cdw10); default: return nvme_trace_common(p, cdw10); } diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 8b1edb46880a..1c5974bb23d5 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1287,6 +1287,7 @@ static inline unsigned int bdev_discard_granularity(struct block_device *bdev) /* maximum copy offload length, this is set to 128MB based on current testing */ #define BLK_COPY_MAX_BYTES (1 << 27) +#define BLK_COPY_MAX_SEGMENTS 2 static inline unsigned int bdev_max_copy_sectors(struct block_device *bdev) { diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 425573202295..5275a0962a02 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -341,7 +341,7 @@ struct nvme_id_ctrl { __u8 nvscc; __u8 nwpc; __le16 acwu; - __u8 rsvd534[2]; + __le16 ocfs; __le32 sgls; __le32 mnan; __u8 rsvd544[224]; @@ -369,6 +369,7 @@ enum { NVME_CTRL_ONCS_WRITE_ZEROES = 1 << 3, NVME_CTRL_ONCS_RESERVATIONS = 1 << 5, NVME_CTRL_ONCS_TIMESTAMP = 1 << 6, + NVME_CTRL_ONCS_COPY = 1 << 8, NVME_CTRL_VWC_PRESENT = 1 << 0, NVME_CTRL_OACS_SEC_SUPP = 1 << 0, NVME_CTRL_OACS_NS_MNGT_SUPP = 1 << 3, @@ -418,7 +419,10 @@ struct nvme_id_ns { __le16 npdg; __le16 npda; __le16 nows; - __u8 rsvd74[18]; + __le16 mssrl; + __le32 mcl; + __u8 msrc; + __u8 rsvd91[11]; __le32 anagrpid; __u8 rsvd96[3]; __u8 nsattr; @@ -830,6 +834,7 @@ enum nvme_opcode { nvme_cmd_resv_report = 0x0e, nvme_cmd_resv_acquire = 0x11, nvme_cmd_resv_release = 0x15, + nvme_cmd_copy = 0x19, nvme_cmd_zone_mgmt_send = 0x79, nvme_cmd_zone_mgmt_recv = 0x7a, nvme_cmd_zone_append = 0x7d, @@ -853,7 +858,8 @@ enum nvme_opcode { nvme_opcode_name(nvme_cmd_resv_release), \ nvme_opcode_name(nvme_cmd_zone_mgmt_send), \ nvme_opcode_name(nvme_cmd_zone_mgmt_recv), \ - nvme_opcode_name(nvme_cmd_zone_append)) + nvme_opcode_name(nvme_cmd_zone_append), \ + nvme_opcode_name(nvme_cmd_copy)) @@ -1030,6 +1036,36 @@ struct nvme_dsm_range { __le64 slba; }; +struct nvme_copy_command { + __u8 opcode; + __u8 flags; + __u16 command_id; + __le32 nsid; + __u64 rsvd2; + __le64 metadata; + union nvme_data_ptr dptr; + __le64 sdlba; + __u8 nr_range; + __u8 rsvd12; + __le16 control; + __le16 rsvd13; + __le16 dspec; + __le32 ilbrt; + __le16 lbat; + __le16 lbatm; +}; + +struct nvme_copy_range { + __le64 rsvd0; + __le64 slba; + __le16 nlb; + __le16 rsvd18; + __le32 rsvd20; + __le32 eilbrt; + __le16 elbat; + __le16 elbatm; +}; + struct nvme_write_zeroes_cmd { __u8 opcode; __u8 flags; @@ -1794,6 +1830,7 @@ struct nvme_command { struct nvme_download_firmware dlfw; struct nvme_format_cmd format; struct nvme_dsm_cmd dsm; + struct nvme_copy_command copy; struct nvme_write_zeroes_cmd write_zeroes; struct nvme_zone_mgmt_send_cmd zms; struct nvme_zone_mgmt_recv_cmd zmr; From patchwork Mon May 20 10:20:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668233 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A5ECC74402 for ; Mon, 20 May 2024 11:45:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205521; cv=none; b=SKO7dSIITp5DAjBX47Ockppk3emmxdqotvVLvBIuo13/+BZV68T/c+fU0kwKliR+lMVXAoBAAvBa/e9wBxtxVBFCH4i9kQvoskYIcpnxGUiS2ndTIhcQQyhv9oWdDqTN3iGOQiHeYt6uxolDbyrMAugUzbwmjEiqE4dvPl8vT3g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205521; c=relaxed/simple; bh=t2a79m2cBKYe7ckC30tK8h2A2PYq9LqU0AmskqnjpnY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=UXA2DQN/AA+tAQEqxZQLRgKV0r37UAmhQ8si3DSBPnJm/RdbGiQoQXJLjcqgCJILUiRkBJE/SLdoCnpx/7f0kAklpHkdT/pMMGygJwbyCZATK46nZBqvqERXyeh0pZxFHEy9xN2CSy6sdnptL+GoEL3KLoY6TUuXiEH+LmU2ryA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=RNIegZEI; arc=none smtp.client-ip=203.254.224.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="RNIegZEI" Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20240520114517epoutp0458695f258ed78b1b1438954de54690d1~RL9NBK9Ps2442124421epoutp04Z for ; Mon, 20 May 2024 11:45:17 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20240520114517epoutp0458695f258ed78b1b1438954de54690d1~RL9NBK9Ps2442124421epoutp04Z DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205517; bh=Mcb2IBWLoPjGcsDETrvIJf7i7Jctj2/f3RwhTuyWe0c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RNIegZEIDx4uC0K8syJwoZGloBs/VkI7BW6GKGpQ1NbQ9AvW6N2U8beZ0BREaPorY Inaxid+pkuv/9PNvtbkwBxh97ZfNTbcbQPgo3dqmPCOSvKu4o9zm6Z4VZz1fAvICiN Fmjsd/lcGYBNGcpjxqkUcsAut6ljA77QLY/YmM6c= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20240520114516epcas5p32f3cc7423f12163798604815ec0b0b8b~RL9Mk1FXB0631306313epcas5p39; Mon, 20 May 2024 11:45:16 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.179]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4VjbNC3zNnz4x9Q1; Mon, 20 May 2024 11:45:15 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id EB.2D.09666.BC73B466; Mon, 20 May 2024 20:45:15 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20240520102952epcas5p18716d203d1810c38397e7fcc9a26922a~RK7XIxgM10053100531epcas5p1j; Mon, 20 May 2024 10:29:52 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240520102952epsmtrp152974acae286b3bea846aa78500e713e~RK7XHeFoA2103421034epsmtrp1F; Mon, 20 May 2024 10:29:52 +0000 (GMT) X-AuditID: b6c32a49-f53fa700000025c2-fe-664b37cbe5c4 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id EE.08.08390.0262B466; Mon, 20 May 2024 19:29:52 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520102948epsmtip2b62fa5b9c2a0bb655ad341d83b0431dd~RK7TX7MfH2248422484epsmtip2y; Mon, 20 May 2024 10:29:48 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 08/12] nvmet: add copy command support for bdev and file ns Date: Mon, 20 May 2024 15:50:21 +0530 Message-Id: <20240520102033.9361-9-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0WTe1BUdRTH+927e1nIhSvI9ONhwVpTgOCu8fhBkoUOXoemQbCY6Q9phSsQ u3e3XR5GjvIQeTg8S4jlvWgmmzwXBBFZQKB4xBRvAmQEJpOAdamgCIllsf77fM/vfM85c35z OLj5BmHNiWCiaBkjFPEIE1Zjp4Ojc5+H3zn+er8Nqu7txlFi9iaOVNNZBFrs1AGUp/0LR/Oa FIA2BgZxpO6eAahMWcxCE5pmDN1T5mLolqoLQ4X5SRjq2loiUG7HKEALIwoMtU46ofIr11no Xuv3LDR0t4hApV8vGKGbPc8wlJM6gqGm+QSAGjdKcVS1uMJC303aoMHNHvY7ttTQsB/Vq4RU s2LaiBqcqWVRQwPRVF1lGkHVX79EPa4vAFTLRDxBVWR+waYykpYJqjn5IZt6ujDJolbujxBU proSUP1lD4z8LT6KPBJOC0NpmR3NhEhCI5gwb55fYPCxYDd3vsBZ4Ik8eHaMUEx7846/5+/s GyHa3hDPLkYoit4O+Qvlct6ht4/IJNFRtF24RB7lzaOloSKpq9RFLhTLo5kwF4aO8hLw+Yfd thM/jgwf3lrDpeMnzq+rRkE8aHkrHRhzIOkKdakz7HRgwjEnWwD8p/E+bhA6APvU60YG8SeA V2bntgVnx5LfedoQbwVQVf7brj0ZgwMjW0CfRJBOsG+Lo4/vI1U4vFqfw9ILnKzHYUKnBtM3 tyBPwZLlHwg9s8jXYGX6ZZaeuaQnfNZUyjYM+ApU1WhwPRuTXlDToAX6QpBUGcO18lXMkHQc pv+UuWuwgE961EYGtoa/Zl3Z5Vh468tvCIP5MoCKMQUwPByFyb1ZuH5snHSA1XcPGcL74bXe qp36OGkKMzbmd3txYVPJcz4Av60uIwxsBUfXEnaZgimP2tj6kuZkBoBTIdngZcX/DcoAqARW tFQuDqPlblIBQ8f+92shEnEd2LkCx5NNYHpW69IBMA7oAJCD8/Zx69Qnz5lzQ4WfxdEySbAs WkTLO4Db9v5ycGvLEMn2GTFRwQJXT76ru7u7q+eb7gLeS9zF5OJQczJMGEVH0rSUlj33YRxj 63iMef3MRMXE+3tFDnKT2gcBGUvXVD6WsjzrBB/PHz20eIOLvTJw/AOg/CPxRlVSe3tB0NED 9qkBF8KHg87WSMVTpsV/Gz+6c+bUxlIprt1/evJJz+/dpqTtrC5w1nZMdpGY+tlxb0MF8i20 cPj085tOt098xe996nPwFybMjbmg3rzzyeO0Vdke3UBKpde0VvbGXJaS/65P4ZBvnstDxUFN S1/J7Wx/yxi6Y63/hlkCNzF8I/t8ROS639yLVodr0mKVK/iYbYfZ1dXqV1+4aMU2O/Zh3CXn xi7LtgD7riSxPGbQrCiCtkktyIhjj48EtZ1NrV3O1frXt+cXOXo37hGl60QcHkseLhQ44jK5 8F9ogvJnjgQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWRa0hTYRzGe885Ox4ni8Mm+Walue6K2sQPb5ZlRXrCoEIpiEKXHk1yaltq RhdtXRfaMlBSS5tmObNSl21typza1sWsvJSmBqXM0Lx3MXHWrL79+P2fh+fDn8L5lYQrFZdw lJUmiOOFJJeoaRC6ey9dGRqztjdjOXrw/CmOzihncFTec4VEgw3jAOWMTuGoz3gBoOnmFhxp nvYCVKS6QaBOow5DBlU2hsrKmzCUnyvHUNPsVxJlmzoA6m/Pw1Btlxe6db6EQIbaZwRqfVJA osLSfgd0x2zD0NWL7RjS9mUAVDNdiKP7gyMEsnQtQi0zZk7QYqa1LZR5roKMLq/HgWnprSSY 1uZkpkp9iWSqS04zA9XXAaPvTCeZ4qxrHCZTPkwyunMfOcxYfxfBjNS1k0yWRg2Yl0WNDrsE +7gbotn4uBRW6rsxknuobfYHnvQ+5NjP8g6QDvTrFYCiIO0PcxvCFYBL8Wk9gLe1Bo4COP7x C2HpTCP+lwWwzGZ1+BuSY1Bu0XDsZZL2gi9mKbt3ph/jcEx+GbMXcLoJh6/vITsL6J1w6lXd nCfoFVCtOEvYmUevgzZt4b8xd1j+0Dg35kgHQOOjUWBn/p9Mz7tRUgnmF4F5arCQTZJJYiUy UZJfApvqIxNLZMkJsT5RiZIqMPdVT08tMKhHfUwAo4AJQAoXOvOqNNtj+LxocdpxVpoYIU2O Z2UmsIgihC6874OZ0Xw6VnyUPcyySaz0/xWjHF3TsYqTfn7e1t0ZBokmc+O2NamRHVFXB+on Z5YoAypC6z99cVe6Bntd8v4VxT+t8unecd514vCzLb1uTgFh+RPRmbO6BYtbd1TtaeaXUQey 7xaMYCIPs0biWPdm3tcpgUe3OPhNV9ve8dc3fC2/Tui0w4FXehqDAicfFWQt2e+fb/UIq+9+ XyzqzPsxNLnbxfxt4GZIrKVQ4RSUETGU4tZm/bAyuA6vlhgnL37yn6j0srwtKuMeMU3jr7rN p2SJOdP6arf7LmlC/X6R52bxMt5nv7i7OXC1wlmxtdmpIqU2ZJPAVvxNFTJ1iz44sWrdNYzl eWyKSFWl5T20KUOtNeGsb4SQkB0SizxxqUz8G/MofhNEAwAA X-CMS-MailID: 20240520102952epcas5p18716d203d1810c38397e7fcc9a26922a X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520102952epcas5p18716d203d1810c38397e7fcc9a26922a References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Add support for handling nvme_cmd_copy command on target. For bdev-ns if backing device supports copy offload we call device copy offload (blkdev_copy_offload). In case of absence of device copy offload capability, we use copy emulation (blkdev_copy_emulation) For file-ns we call vfs_copy_file_range to service our request. Currently target always shows copy capability by setting NVME_CTRL_ONCS_COPY in controller ONCS. loop target has copy support, which can be used to test copy offload. trace event support for nvme_cmd_copy. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta --- drivers/nvme/target/admin-cmd.c | 9 +++- drivers/nvme/target/io-cmd-bdev.c | 71 +++++++++++++++++++++++++++++++ drivers/nvme/target/io-cmd-file.c | 50 ++++++++++++++++++++++ drivers/nvme/target/nvmet.h | 1 + drivers/nvme/target/trace.c | 19 +++++++++ 5 files changed, 148 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c index f5b7054a4a05..59021552084f 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -433,8 +433,7 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req) id->nn = cpu_to_le32(NVMET_MAX_NAMESPACES); id->mnan = cpu_to_le32(NVMET_MAX_NAMESPACES); id->oncs = cpu_to_le16(NVME_CTRL_ONCS_DSM | - NVME_CTRL_ONCS_WRITE_ZEROES); - + NVME_CTRL_ONCS_WRITE_ZEROES | NVME_CTRL_ONCS_COPY); /* XXX: don't report vwc if the underlying device is write through */ id->vwc = NVME_CTRL_VWC_PRESENT; @@ -536,6 +535,12 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req) if (req->ns->bdev) nvmet_bdev_set_limits(req->ns->bdev, id); + else { + id->msrc = (__force u8)to0based(BIO_MAX_VECS - 1); + id->mssrl = cpu_to_le16(BIO_MAX_VECS << + (PAGE_SHIFT - SECTOR_SHIFT)); + id->mcl = cpu_to_le32(le16_to_cpu(id->mssrl)); + } /* * We just provide a single LBA format that matches what the diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c index 6426aac2634a..4c2c1784fec9 100644 --- a/drivers/nvme/target/io-cmd-bdev.c +++ b/drivers/nvme/target/io-cmd-bdev.c @@ -46,6 +46,18 @@ void nvmet_bdev_set_limits(struct block_device *bdev, struct nvme_id_ns *id) id->npda = id->npdg; /* NOWS = Namespace Optimal Write Size */ id->nows = to0based(bdev_io_opt(bdev) / bdev_logical_block_size(bdev)); + + if (bdev_max_copy_sectors(bdev)) { + id->msrc = id->msrc; + id->mssrl = cpu_to_le16((bdev_max_copy_sectors(bdev) << + SECTOR_SHIFT) / bdev_logical_block_size(bdev)); + id->mcl = cpu_to_le32((__force u32)id->mssrl); + } else { + id->msrc = (__force u8)to0based(BIO_MAX_VECS - 1); + id->mssrl = cpu_to_le16((BIO_MAX_VECS << PAGE_SHIFT) / + bdev_logical_block_size(bdev)); + id->mcl = cpu_to_le32((__force u32)id->mssrl); + } } void nvmet_bdev_ns_disable(struct nvmet_ns *ns) @@ -451,6 +463,61 @@ static void nvmet_bdev_execute_write_zeroes(struct nvmet_req *req) } } +static void nvmet_bdev_copy_endio(void *private, int status, + ssize_t copied) +{ + struct nvmet_req *rq = (struct nvmet_req *)private; + u16 nvme_status; + + if (copied == rq->copy_len) + rq->cqe->result.u32 = cpu_to_le32(1); + else + rq->cqe->result.u32 = cpu_to_le32(0); + + nvme_status = errno_to_nvme_status(rq, status); + nvmet_req_complete(rq, nvme_status); +} + +/* + * At present we handle only one range entry, since copy offload is aligned with + * copy_file_range, only one entry is passed from block layer. + */ +static void nvmet_bdev_execute_copy(struct nvmet_req *rq) +{ + struct nvme_copy_range range; + struct nvme_command *cmd = rq->cmd; + ssize_t ret; + off_t dst, src; + + u16 status; + + status = nvmet_copy_from_sgl(rq, 0, &range, sizeof(range)); + if (status) + goto err_rq_complete; + + dst = le64_to_cpu(cmd->copy.sdlba) << rq->ns->blksize_shift; + src = le64_to_cpu(range.slba) << rq->ns->blksize_shift; + rq->copy_len = ((__force size_t)range.nlb + 1) << rq->ns->blksize_shift; + + if (bdev_max_copy_sectors(rq->ns->bdev)) { + ret = blkdev_copy_offload(rq->ns->bdev, dst, src, rq->copy_len, + nvmet_bdev_copy_endio, + (void *)rq, GFP_KERNEL); + } else { + ret = blkdev_copy_emulation(rq->ns->bdev, dst, + rq->ns->bdev, src, rq->copy_len, + nvmet_bdev_copy_endio, + (void *)rq, GFP_KERNEL); + } + if (ret == -EIOCBQUEUED) + return; + + rq->cqe->result.u32 = cpu_to_le32(0); + status = errno_to_nvme_status(rq, ret); +err_rq_complete: + nvmet_req_complete(rq, status); +} + u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req) { switch (req->cmd->common.opcode) { @@ -469,6 +536,10 @@ u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req) case nvme_cmd_write_zeroes: req->execute = nvmet_bdev_execute_write_zeroes; return 0; + case nvme_cmd_copy: + req->execute = nvmet_bdev_execute_copy; + return 0; + default: return nvmet_report_invalid_opcode(req); } diff --git a/drivers/nvme/target/io-cmd-file.c b/drivers/nvme/target/io-cmd-file.c index 2d068439b129..0a8337596f0c 100644 --- a/drivers/nvme/target/io-cmd-file.c +++ b/drivers/nvme/target/io-cmd-file.c @@ -322,6 +322,47 @@ static void nvmet_file_dsm_work(struct work_struct *w) } } +static void nvmet_file_copy_work(struct work_struct *w) +{ + struct nvmet_req *req = container_of(w, struct nvmet_req, f.work); + int nr_range = req->cmd->copy.nr_range + 1; + u16 status = 0; + int src, id; + ssize_t len, ret; + loff_t pos; + + pos = le64_to_cpu(req->cmd->copy.sdlba) << req->ns->blksize_shift; + if (unlikely(pos + req->transfer_len > req->ns->size)) { + nvmet_req_complete(req, errno_to_nvme_status(req, -ENOSPC)); + return; + } + + for (id = 0 ; id < nr_range; id++) { + struct nvme_copy_range range; + + status = nvmet_copy_from_sgl(req, id * sizeof(range), &range, + sizeof(range)); + if (status) + break; + + src = (le64_to_cpu(range.slba) << (req->ns->blksize_shift)); + len = (le16_to_cpu(range.nlb) + 1) << (req->ns->blksize_shift); + ret = vfs_copy_file_range(req->ns->file, src, req->ns->file, + pos, len, COPY_FILE_SPLICE); + pos += ret; + if (ret != len) { + req->cqe->result.u32 = cpu_to_le32(id); + if (ret < 0) + status = errno_to_nvme_status(req, ret); + else + status = errno_to_nvme_status(req, -EIO); + break; + } + } + + nvmet_req_complete(req, status); +} + static void nvmet_file_execute_dsm(struct nvmet_req *req) { if (!nvmet_check_data_len_lte(req, nvmet_dsm_len(req))) @@ -330,6 +371,12 @@ static void nvmet_file_execute_dsm(struct nvmet_req *req) queue_work(nvmet_wq, &req->f.work); } +static void nvmet_file_execute_copy(struct nvmet_req *req) +{ + INIT_WORK(&req->f.work, nvmet_file_copy_work); + queue_work(nvmet_wq, &req->f.work); +} + static void nvmet_file_write_zeroes_work(struct work_struct *w) { struct nvmet_req *req = container_of(w, struct nvmet_req, f.work); @@ -376,6 +423,9 @@ u16 nvmet_file_parse_io_cmd(struct nvmet_req *req) case nvme_cmd_write_zeroes: req->execute = nvmet_file_execute_write_zeroes; return 0; + case nvme_cmd_copy: + req->execute = nvmet_file_execute_copy; + return 0; default: return nvmet_report_invalid_opcode(req); } diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h index 2f22b07eab29..480e4ffa01fd 100644 --- a/drivers/nvme/target/nvmet.h +++ b/drivers/nvme/target/nvmet.h @@ -406,6 +406,7 @@ struct nvmet_req { struct device *p2p_client; u16 error_loc; u64 error_slba; + size_t copy_len; }; #define NVMET_MAX_MPOOL_BVEC 16 diff --git a/drivers/nvme/target/trace.c b/drivers/nvme/target/trace.c index 8d1806a82887..8be6f77ebfd8 100644 --- a/drivers/nvme/target/trace.c +++ b/drivers/nvme/target/trace.c @@ -92,6 +92,23 @@ static const char *nvmet_trace_dsm(struct trace_seq *p, u8 *cdw10) return ret; } +static const char *nvmet_trace_copy(struct trace_seq *p, u8 *cdw10) +{ + const char *ret = trace_seq_buffer_ptr(p); + u64 sdlba = get_unaligned_le64(cdw10); + u8 nr_range = get_unaligned_le16(cdw10 + 8); + u16 control = get_unaligned_le16(cdw10 + 10); + u32 dsmgmt = get_unaligned_le32(cdw10 + 12); + u32 reftag = get_unaligned_le32(cdw10 + 16); + + trace_seq_printf(p, + "sdlba=%llu, nr_range=%u, ctrl=1x%x, dsmgmt=%u, reftag=%u", + sdlba, nr_range, control, dsmgmt, reftag); + trace_seq_putc(p, 0); + + return ret; +} + static const char *nvmet_trace_common(struct trace_seq *p, u8 *cdw10) { const char *ret = trace_seq_buffer_ptr(p); @@ -195,6 +212,8 @@ const char *nvmet_trace_parse_nvm_cmd(struct trace_seq *p, return nvmet_trace_zone_mgmt_send(p, cdw10); case nvme_cmd_zone_mgmt_recv: return nvmet_trace_zone_mgmt_recv(p, cdw10); + case nvme_cmd_copy: + return nvmet_trace_copy(p, cdw10); default: return nvmet_trace_common(p, cdw10); } From patchwork Mon May 20 10:20:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668234 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E685C74E11 for ; Mon, 20 May 2024 11:45:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.33 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205525; cv=none; b=PECO1YJWnRoSXjL/iDSLPRcB8oCCVsZK/xeY7bSD6R/9Y2z0ulm+hlQ8H3A4uj1DveNfon1zrShLH7OOfvs8Hv+eixfyur+PXciH5hGU856skEEGbMnQkMM7HBXsfgiGFtvF4RanVE5Auuh2dfhZDJJYRcSp0q8bO6nmwU5Ip94= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205525; c=relaxed/simple; bh=j3MX9uE/to6vHmgEXOSBH9WWWHt04pmktDS08ErKkqk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=fkkx2ML/kM3peyfIbi3RlSiGJNhUUty309fwP/FmV47zHuNeBjSh/cNOc+4myEZMwFQRiwqZKZ4qzU5IwRc6Wjxop7ienVdAsA3/Ja9cqJR3P5Nqpex6/kyfDDJ64lWlYBQuhzmV3BpuSM65scIQkkcvn3nxb46nTdUvM1F7M+Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=HDoz4gJw; arc=none smtp.client-ip=203.254.224.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="HDoz4gJw" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20240520114522epoutp0379d56e5f0a6814411125292aebdd3970~RL9Ri8P3n1758617586epoutp03g for ; Mon, 20 May 2024 11:45:22 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20240520114522epoutp0379d56e5f0a6814411125292aebdd3970~RL9Ri8P3n1758617586epoutp03g DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205522; bh=k3ZzCJqgdNN/MTkGUl0/6z5ZTfmIadxi7gg3Kolrnkk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HDoz4gJws/cBVq8ODqtGB+kpjj9iMTiZhBNzcNn5uOZ+M1WXy85aCppjWBbQDTV3R W53Fe9K8UszdxqG2p5t4d10QhDuH2qWfVf7XbngwZI5qlNUmKJgMwx7Ve1zrv1gXJ8 gFz/KrepcbBLAqTgMkV+1IkQ4KlfmYmoTwqsfvBw= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20240520114521epcas5p2e48ea1278eee17e99710573981dce0eb~RL9RCh5a93085730857epcas5p21; Mon, 20 May 2024 11:45:21 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.175]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4VjbNJ2y4Tz4x9Pw; Mon, 20 May 2024 11:45:20 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 6E.3D.09666.0D73B466; Mon, 20 May 2024 20:45:20 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20240520103004epcas5p4a18f3f6ba0f218d57b0ab4bb84c6ff18~RK7iKHGbB1831418314epcas5p4t; Mon, 20 May 2024 10:30:04 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240520103004epsmtrp1decb440a9d79f03d4fd2c7882c043635~RK7iJLyfK2107921079epsmtrp1C; Mon, 20 May 2024 10:30:04 +0000 (GMT) X-AuditID: b6c32a49-f53fa700000025c2-0a-664b37d00754 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id BB.78.08924.C262B466; Mon, 20 May 2024 19:30:04 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520103000epsmtip2f410f05b749878c57352c028ac2515be~RK7ee83R-2248422484epsmtip27; Mon, 20 May 2024 10:30:00 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 09/12] dm: Add support for copy offload Date: Mon, 20 May 2024 15:50:22 +0530 Message-Id: <20240520102033.9361-10-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0WTe1BUdRTH+927e1mYFi8LxQ8kwutAPAJ3E9Yf8hCL6hI9aGpsNBvagcsj 2MfsQ0v/kFVW5C0SYasEkQjsEpsLuDykdhZ1DQUCXByZwJogAqYkNHvwatmF+u9zvr/zPefM +c3h4LwVwpeTLVEycokolyLcWJf7QkLCv9uVnMG3NZLI0H8dR8dPr+BIP1FOoPm+RYA+Wfgb R1PmAoCWBoZw1H59EqC6+hoWumvuwtCV+jMYatZfw9C56hMYurb2K4HOWMYAmrZpMdQ7HoY+ P3mBha70fstCo93nCVR7cdoFNVpXMVRxyoahzik1QJeXanHUOn+fhW6Mb0VDK1Z2gh89ejuZ 7q+HdJd2woUemrzEokcHVLRRV0jQbReO0b+0fQronrt5BP1FWSWbLj3xG0F3ae6x6d+nx1n0 /a9tBF3WrgP0rbqrLimeB3JisxhROiMPYCRp0vRsSWYclfxW6gupUUK+IFwQjXZRARKRmImj El9NCX8pO9e+ISrgkChXZZdSRAoFtSM+Vi5VKZmALKlCGUcxsvRcWaQsQiESK1SSzAgJo9wt 4POfi7Invp+T9b3JgslafD4smjlL5IFBryLgyoFkJPzm7CRWBNw4PLIHQE1DAXAGiwBaH7aw 17N45CMAbff4m44iUzHLmdQL4NypbsIZaDA4Ot9ir8XhEGQYvLnGWde9SD0Oi9sqHA6cbMOh us+MrZfyJGNgY0GTyzqzyED4z4NaRzsuuRvWVH+JO9s9DfVfmR3satfNHQuO+SCpd4XLD2dZ zqRE2FDa6OJkTzhnbd9gXzhbfnKDD8Pmj5sIpzkfQO0dLXA+7IGa/nJ8fWycDIGG7h1O+SlY 1d/qGBQn3WHp0hTm1Lmw87NN3g5bDHWEk33g2J/qDaah7Xgb7lxLKYCWBRN2Gvhr/29RB4AO +DAyhTiTUUTJBBLm8H//liYVG4HjDkKTOsHEDwsRFoBxgAVADk55cY3tSRk8brrooyOMXJoq V+UyCguIsm+wAvd9Ik1qPySJMlUQGc2PFAqFkdE7hQLKmzuvqUnnkZkiJZPDMDJGvunDOK6+ eVhHziFpye2RhqLpB52VqyOFxuDAxAK9lPOu+sAbkxc91gRPHozdHphfGhAUL37Ns+Qx94L9 V88Fyl9u8riVkBK8NHRMcr5ha4IkzLSnlTXg5r+vNzh2xHsbNyhPJwsTWh839mz5QEWNQZ3k 9RdXfNTjw8+E/Nzh/07ywdW0mKFLRzUF7PIMwT4wbHWVepu8qqCaKSne+dPe6ecX3fvcawVH t43tF+e9YghuNmZUP0JjTaq/orp959OCBusNicva2diyH99kU6l/+HncmBk3sZPXhldW3zuy 7J+/Jc7LL6m6a7CqkMAGpJhauSqa4719M97sUY7daRmc0XWMGmKeDZ2spFiKLJEgFJcrRP8C 7o32NJAEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWRe0xSYRjG+c45HA4YdkK3jsXWYF01SddlX1ktV21fc+uy5rTWlqRHuwAx ULO2SrMsSc3sYlmkmdNAuwheILMxvHQzKQlNy3Iu0uYs7DKTZhe0/vvteZ73ff54KFxkImZQ e1TJrEYlV0hJAVHXJJ0VunBuVGJYjkUM7zxpxeGx/HEcVvaeIeFQ0xcAL3rGcPjedhLAn88c OKxpfQtgSamegN02KwbvlxZg0FDZgsErhZkYbPk9TMICeyeAblcRBht7QuD1rDIC3m98TEDn vaskLC5382DFw18YPHvKhUHL+wwA634W4/D20GcCPuqZCR3jD7lrxMj5Mgo9KWWQtaiXhxxv qwnkfJaCTMZsEpnLjqJB82WAGrrTSXQj7xwX5WZ+IpH1xDsuGnH3EOjzAxeJ8mqMALWVNPM2 B2wXrExgFXtSWc2i1XGC3W/q7Zi6KihNN3CJTAftgTrApxh6CaOrP03ogIAS0Q2AcbjzeJNG EFM+3oxPcgBj+DXAmwxlYkzFiyFSByiKpEOYp78pnx5I1+PMSOZpzHeA0y0487wK+jiAjmAq Tt6ceErQcxjv12Kuj4X0CkZfeOtfwSym8q5tgvl/dVutB/hYRC9ners8ZD7wLwEcIwhi1Vpl klIbrg5XsQdkWrlSm6JKksXvV5rAxKzBCyyg3uiR2QFGATtgKFwaKDTVbEgUCRPkBw+xmv07 NSkKVmsHMylCOl04fTA3QUQnyZPZfSyrZjX/XYziz0jH8BD9q1pLR2Qv/0dauSRVFepvLi9c uCUlTLH2RsGhpVaJX3RJarcjXTmmaUWHw3OPRls/btjWNM3/msP7wS08ggbmfT/sp29T7x0O W7BRlLV9IO77wa5VCtvZK/P4GdFe7dIGu8FrHqmEzsFTkbGzY72r+4xTN6WdcfKMzYteRCgk ncP9MaY2xZocR+j6rbJ4Y36EuT1qyrn2iqeN4g+Jb/ra8j3i+ETxGPVNnD3auTfmoysrdZ3h h+yVX0t/8LTargLOeZdkyxT2rjVuccdOTkxdZHZuFWdQX3vMmIzdeT3/nqVz9HiGY7RYQgv6 cqovcJfRBGdH9c0lPbv6DR1SQrtbHh6Ma7TyP0LTr7VFAwAA X-CMS-MailID: 20240520103004epcas5p4a18f3f6ba0f218d57b0ab4bb84c6ff18 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520103004epcas5p4a18f3f6ba0f218d57b0ab4bb84c6ff18 References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Before enabling copy for dm target, check if underlying devices and dm target support copy. Avoid split happening inside dm target. Fail early if the request needs split, currently splitting copy request is not supported. Signed-off-by: Nitesh Shetty --- drivers/md/dm-table.c | 37 +++++++++++++++++++++++++++++++++++ drivers/md/dm.c | 7 +++++++ include/linux/device-mapper.h | 3 +++ 3 files changed, 47 insertions(+) diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index cc66a27c363a..d58c67ecd794 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -1899,6 +1899,38 @@ static bool dm_table_supports_nowait(struct dm_table *t) return true; } +static int device_not_copy_capable(struct dm_target *ti, struct dm_dev *dev, + sector_t start, sector_t len, void *data) +{ + struct request_queue *q = bdev_get_queue(dev->bdev); + + return !q->limits.max_copy_sectors; +} + +static bool dm_table_supports_copy(struct dm_table *t) +{ + struct dm_target *ti; + unsigned int i; + + for (i = 0; i < t->num_targets; i++) { + ti = dm_table_get_target(t, i); + + if (!ti->copy_offload_supported) + return false; + + /* + * target provides copy support (as implied by setting + * 'copy_offload_supported') + * and it relies on _all_ data devices having copy support. + */ + if (!ti->type->iterate_devices || + ti->type->iterate_devices(ti, device_not_copy_capable, NULL)) + return false; + } + + return true; +} + static int device_not_discard_capable(struct dm_target *ti, struct dm_dev *dev, sector_t start, sector_t len, void *data) { @@ -1975,6 +2007,11 @@ int dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, limits->discard_misaligned = 0; } + if (!dm_table_supports_copy(t)) { + limits->max_copy_sectors = 0; + limits->max_copy_hw_sectors = 0; + } + if (!dm_table_supports_write_zeroes(t)) limits->max_write_zeroes_sectors = 0; diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 597dd7a25823..070b41b83a97 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1717,6 +1717,13 @@ static blk_status_t __split_and_process_bio(struct clone_info *ci) if (unlikely(ci->is_abnormal_io)) return __process_abnormal_io(ci, ti); + if ((unlikely(op_is_copy(ci->bio->bi_opf)) && + max_io_len(ti, ci->sector) < ci->sector_count)) { + DMERR("Error, IO size(%u) > max target size(%llu)\n", + ci->sector_count, max_io_len(ti, ci->sector)); + return BLK_STS_IOERR; + } + /* * Only support bio polling for normal IO, and the target io is * exactly inside the dm_io instance (verified in dm_poll_dm_io) diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index 82b2195efaca..6868941bc7d9 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -397,6 +397,9 @@ struct dm_target { * bio_set_dev(). NOTE: ideally a target should _not_ need this. */ bool needs_bio_set_dev:1; + + /* copy offload is supported */ + bool copy_offload_supported:1; }; void *dm_per_bio_data(struct bio *bio, size_t data_size); From patchwork Mon May 20 10:20:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668235 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C03CA7603F for ; Mon, 20 May 2024 11:45:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.24 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205530; cv=none; b=Tqaqqn5XGM+0AXkxMT+6Epag5GlUzX30Z78I+7DUVh21YpvLvntQN/U6tSDPJAfnPW8nCGoqTXjbbkLgi9W6aAm3VC+wB5BsrqkhRrwY8ke+FExfxBXq2pWr0+/ONav5/z2O4ktvXRH9b87JDw7RMyAhl/fVaEOAclFoNWjD2iY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205530; c=relaxed/simple; bh=WbpYhsR869XNt+4RG+0RSqan1Uuxl6RdgoojeeVxnzQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=Hk6gHkJIrJ/Fuzi1C8ErX2lHwgCFVFn5VSeiK6/rHAjrgMc6Hx6H+DHC4dFf+rKm7nfU9bTKyEq41ljywB/dm6WILoECFFrly8KvdBt53xxggAylUx+qJJ+/PWPaCp2Mu9TKThoag+NuX1EgjI7nZZVe0RvM/3rynxBc/I+NEU0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=OtDYruN7; arc=none smtp.client-ip=203.254.224.24 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="OtDYruN7" Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20240520114527epoutp01a8931a9011c915c8cb5aa8da08839243~RL9V_1y6Q1937919379epoutp01M for ; Mon, 20 May 2024 11:45:27 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20240520114527epoutp01a8931a9011c915c8cb5aa8da08839243~RL9V_1y6Q1937919379epoutp01M DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205527; bh=0x72vxug1sX4QFwBsS/SRZ9PFW1UoWmnX4+Nx/d4D4A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OtDYruN7a+CXF1yrOw7nRlb/Znsu5nQpn9bKaQuqoGkWKgKpgJ/zoT5lwFFkEwQPT Q8C6jrLqbrFE8g0HC3SrDJFvKXBYyg/j86kFNng86FgDYsUfPyDtAWB8yLWkhEhrug 2Qn4NypVNHh1LCthgLtFQ7BqnH4Ra6od6SMuRvgg= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20240520114526epcas5p4b6cc85b882bf26f20984f380972d0676~RL9Viu-MD2411824118epcas5p4X; Mon, 20 May 2024 11:45:26 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.176]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4VjbNN63L9z4x9Pw; Mon, 20 May 2024 11:45:24 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id C3.80.19431.4D73B466; Mon, 20 May 2024 20:45:24 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20240520103016epcas5p31b9a0f3637959626d49763609ebda6ef~RK7tKInxL0624906249epcas5p3_; Mon, 20 May 2024 10:30:16 +0000 (GMT) Received: from epsmgmc1p1new.samsung.com (unknown [182.195.42.40]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240520103016epsmtrp16603da3cc6bfd283061b5195588c54c3~RK7tInXSj2227122271epsmtrp13; Mon, 20 May 2024 10:30:16 +0000 (GMT) X-AuditID: b6c32a50-ccbff70000004be7-ad-664b37d4050f Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmc1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 38.0E.09238.8362B466; Mon, 20 May 2024 19:30:16 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520103012epsmtip2bd5849ea7ca94911fcfed243fd15de18~RK7pfge6T2119121191epsmtip2V; Mon, 20 May 2024 10:30:12 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 10/12] dm: Enable copy offload for dm-linear target Date: Mon, 20 May 2024 15:50:23 +0530 Message-Id: <20240520102033.9361-11-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0VTe1BUZRSf797l7kKgdxbUL6qFWUsFBXaTxwdJGJreghp6wcA0wA5cAYHd nX1I7lQuS2CAgEBQ8pCXlbLCEgvy3HUHWDCMgeJRbkGRMIEmGE46SkC77FL//c7vnN/5ne98 c1g4e51wZaUIZbREKEjjEg6Ma/0e+7wmAsJO8Nr/9kSa4UEcqc6v4Ug9XUSgu/0rAJXff4yj OcNZgFZHRnHUNjgDUG19NQPdMnRhqLe+BENX1EYMVX6ehSHjxj0ClfRNATQ/WYEhnWk/qsu5 xEC9um8ZaLy7ikA1X80z0ddD6xgq/nQSQ51zmQBdW63BUfPdZQa6YXoGja4N2R1+lhqfCKOG 6yHVVTHNpEZnvmFQ4yNyqrUxl6C0l85QC9oLgOq5pSSohsJSO6oga4mgurJ/taP+mjcxqGX9 JEEVtjUC6rvaAWaEc0zqoWRakEhL3GlhgigxRZgUzA17J+5InJ8/j+/FD0QBXHehIJ0O5h4N j/A6lpJm3hDX/ZQgTW6mIgRSKdfn5UMSkVxGuyeLpLJgLi1OTBP7ir2lgnSpXJjkLaRlQXwe 70U/c2F8avLKdC8hvmj3wcONIkwJWhh5wJ4FSV+oW69k5gEHFpvsBbC0/k/MGqwAWNabT1iD hwA+yKrDtiRN5UqmBbNJHYD9F4TWomwMzjwascsDLBZB7oc3N1gW3oVU4zBfW8ywBDipxWFm v2GzkzN5HM4va3ELZpAvQPXA2OZQTmQQVBp/Ylrd3KC6xbBZY2/mDe33gaURJNX2sKCx0zbS UaiZ19te5AzvDLXZxK5wsSjHhjPglc8uE1bxJwBW/FgBrIkQmD1chFvGxkkPqOn2sdLPwbLh 5s3+OLkNFqzO2bycYOfFLbwbXtXUElb8NJx6lGnDFFSpxmyLLACwvayHOA84Ff9b1ALQCFxp sTQ9iU7wE/O9hHTGfx+XIEpvBZuH4BnRCdQta959AGOBPgBZONfFqbXttRNsp0TBaQUtEcVJ 5Gm0tA/4mVdYjLvuSBCZL0koi+P7BvJ8/f39fQMP+vO5u5zuZlcnsskkgYxOpWkxLdnSYSx7 VyXG4jSrHGs4DkfWyonoaNW9DmSadtRXZT52K/jCVCbXd/iYDsYvrSTM5oJQYPwh6P3rHhk/ P+92jmO8XbW9bOG2bncUu2pv24GoV31AR8+eur7p4IkQQ0yeQfFxQ6xSn+8YcyyUrYiPPRlw vdQY2f7UEyLnw6bjydzs2OruhtxtjlJhqcZh3yu1JRxF5R9Tql2ePYdnlG+3V78RvpM/c6aQ 84uqZPa3yyHRAxHq4g4x/1TUu4oDuqSTD7ThbxV1fO8y+9Ge02fr/1krf2+wGpe9PlSsSFcs Nj1ZL3Gv3BlgdLwaGb1j++96zmJ87kt7/aril6JvDt8IYdyJHFv4khflrXkzlMuQJgv4nrhE KvgXJ30+SZEEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWRf0yMcRzHfZ/n6bmnm8vjMj2ptJ2xVhwJ+4aZH5mv3RhtamPDqcdpupy7 TpQfl+jI1AkhUlKZrri60HHldl0uKUVFpTsbneRHd2GmpUs/5r/X3p/35/N+bx8K51cQs6i4 hERWniCOF5Bc4mGdIHABnCfau+jvFx683/gMhyc1IzjU2rJI+LXuB4A5riEc9prUAA43t+Cw 6pkdwILCPAJ2mQwYNBZmY/Cuth6D16+kYbB+9DsJs81vAHR05GKwpjsE3kovIqCx5jkB2x7f IGF+iYMD71jdGLxwpgOD1b2pAD4czsfhva9OAjZ0+8GWEavHan/U1i5CjYUMMuTaOKjFXkGg tmYlqiw9SyJ90Qn0WX8NoCddKhLdzrzogc6nDZDIcPq9Bxp0dBPIWdtBosyqUoCaCiycLd7b uStj2fi4Q6x84ard3H0/bEZSdtPj8O/RLEwFdEQG8KQYeglTnqPijDOffgKYHsueSd2XKRmx 4JPszdx19415uGOeNIz59scIMgBFkXQI82KUGtdn0I9wZjDtHDa+gNP1ONNaBsfZm97AOJz6 iUMEPZfRWlongnn0ckZV38mZDAhktDrThMdzTDc9cIHJQuGM7a2L1ACvAjClFPiyMoVUIo0J lYUmsElChViqUCZIhDEHpJVg4qvBQdXAnu8WmgFGATNgKFwwg1dZtXEvnxcrPpLMyg/skivj WYUZ+FGEwIcXevV6LJ+WiBPZ/SwrY+X/pxjlOUuFyWTBAT/bp+W5+iusW2vDkoTla2yasgXJ kbr5r/xGg6KUznWPwaaPyqU9pssPyPWqkEIfNJQS3nWDn14wzzKc8fzYGihVD9jDNhsNYfaf 0RpHlKusc/HFaMtiw0zieLGuP+XOnKR3L0dq4i75XliemrtSKJ3u/3Gt1dyX7KmPvhp4qke3 o1wToIzcMySqeEH37vLKiEzUVzc221Wl5D02NfjlXHXTsk+ZFkm69YgzXHv+VH+ITvRhxTYv fYSgoTb+ZnaT+7c979fOQa5w062Bg6+1U9Ut2a/Uziz37JiGlC9LWger/Tc+Lfa5ciZiVcmQ NegpKD68pabxxFFJX+rJOgGh2CcODcblCvE/Zzp3o0QDAAA= X-CMS-MailID: 20240520103016epcas5p31b9a0f3637959626d49763609ebda6ef X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520103016epcas5p31b9a0f3637959626d49763609ebda6ef References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Setting copy_offload_supported flag to enable offload. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty --- drivers/md/dm-linear.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c index 2d3e186ca87e..cfec2fac28e1 100644 --- a/drivers/md/dm-linear.c +++ b/drivers/md/dm-linear.c @@ -62,6 +62,7 @@ static int linear_ctr(struct dm_target *ti, unsigned int argc, char **argv) ti->num_discard_bios = 1; ti->num_secure_erase_bios = 1; ti->num_write_zeroes_bios = 1; + ti->copy_offload_supported = 1; ti->private = lc; return 0; From patchwork Mon May 20 10:20:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668236 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA783770E6 for ; Mon, 20 May 2024 11:45:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205536; cv=none; b=YEx1eXWtahK/+kj2LdDF2bepwBWGRDxzvfmIAZLr3MmV3ncmM09itrEB9UHyllitvw5c0Cy/6X4JkHHEj+OXb0u2RwViJBJosnnIHhCP0AobpUA9gi07OHhHfcyEebOa06rAWJYrm/9ZDEFk43O0QQATOHDL0Is4skx3ckdoGRU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205536; c=relaxed/simple; bh=cAuZ44MnfXumddyGNuhSAEOj8RADCvGvGuZ/6MB2xIQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=OCRPYfvuTRJaoliGItpndt17w+thG0Qke2Yd03v8MJETY5gHvnDw+NI8Vu91drqqhqhs8SUhqiCl7AWi6+MFKz+/EfPL0xRpPAb0FTZxYKWZNcyd249zxt+SM7M7t0Qbm94qi9bWUfBbxdW9PEN/yJ1Z5vyr7ca43Cx3HVudrdk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=vbItXjMb; arc=none smtp.client-ip=203.254.224.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="vbItXjMb" Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20240520114533epoutp043492c3ffbdde0f40176e168c6e935195~RL9bwHFF-2508025080epoutp04P for ; Mon, 20 May 2024 11:45:33 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20240520114533epoutp043492c3ffbdde0f40176e168c6e935195~RL9bwHFF-2508025080epoutp04P DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205533; bh=bSZUkbC/ZQpCoDZSpk6xHyDNFo0X+YS5ZTMJ5s/Gxic=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vbItXjMbSQNjCPdJ6bCxcJgvxWcRhv6Q2KjgWtfraq6Df6HoplhOcqKsxEkpUpLg4 xkhWvzhHhLJsjDuj2AtKMvdPSmMXsHEYI7wGNAHrXNKnGvxwwMtClCmI06X8fXYb8P pVwf63mK3N2zXX9r+E1RZJMqrxQIt59rIFc1olcQ= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20240520114532epcas5p3f34132c23d11b79ee9843c7df5910da8~RL9bL1Wrp0451804518epcas5p3n; Mon, 20 May 2024 11:45:32 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.178]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4VjbNV3HLrz4x9Pp; Mon, 20 May 2024 11:45:30 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id F5.80.19431.AD73B466; Mon, 20 May 2024 20:45:30 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20240520103027epcas5p4789defe8ab3bff23bd2abcf019689fa2~RK737w-oH1831418314epcas5p46; Mon, 20 May 2024 10:30:27 +0000 (GMT) Received: from epsmgmcp1.samsung.com (unknown [182.195.42.82]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240520103027epsmtrp1946a4d61e2ae40463c15065f4c80c842~RK735Xxv22229722297epsmtrp1e; Mon, 20 May 2024 10:30:27 +0000 (GMT) X-AuditID: b6c32a50-ccbff70000004be7-b5-664b37dab6c6 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmcp1.samsung.com (Symantec Messaging Gateway) with SMTP id A3.37.19234.3462B466; Mon, 20 May 2024 19:30:27 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520103024epsmtip228edfc3ec310b9b2361e5df416cb9fa3~RK70S6Ven2512325123epsmtip2H; Mon, 20 May 2024 10:30:23 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 11/12] null: Enable trace capability for null block Date: Mon, 20 May 2024 15:50:24 +0530 Message-Id: <20240520102033.9361-12-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0WTe1BUdRTH+91797LLuM1tgennGkqLTAHDY21ZfjggFD6ugQPlP5kVbssV CNhd9i6RphNPSeTpo3IDIUB5LAnyCghs2w0QhGhal4RBSWItJAGxxogB22XR/vuc7+98zzlz fnO4uGCFFHITFBpGrZAliUhHot3o+bLPeGDEYf+TCwLUONiHo8ziFRzpbhWRaNa4CNBnC0s4 mtbnArQ8PIKj1r7bAFVUlhFoTN+Joe7K0xiq0/Vi6MvPszDU+/g+iU4bRgGymLUY6hn3Rl+d qCZQd88AgUxdpSQqv2RxQDX9qxgq+dSMoY7pDIDal8txdHl2nkDXxjehkZV+TtgLtOlGBD1Y CelO7S0HeuT2FYI2DafSzfUnSbql+hP6j5bzgP52LJ2kqwrPcOiCrDmS7syZ5NAPLOMEPX/V TNKFrfWAHqr4wSHa6e3E4HhGFsuo3RiFXBmboIgLEUXsjwmPCZD6i33EQShQ5KaQJTMhop2R 0T67E5KsGxK5fShLSrVK0TKWFfntCFYrUzWMW7yS1YSIGFVskkqi8mVlyWyqIs5XwWi2i/39 twVYEw8lxo8ZHnJUjwQfme4a8HSQTeUBHhdSEtiW8Q0nDzhyBVQ3gAOWK6Q9WARwPlOPPQ2y akdBHuCuWf7+zdeud1qTqvvWHTkYrM7WO9iSSMobXn/MtenOlA6Hp1pKCFuAUy04zDDayvK4 TtQeePF6DbAxQXnA1eFujs3Mp7ZDc228fb4tUNekx23Ms8r6tgVgqwMpHQ/W3Wzg2JN2wgbt EGFnJ3ivv9XBzkI4U3RindNg3dla0m7OBlD7ixbYH0JhzmARbmuMU56wscvPLrvCc4OX1+bE qWdhwfI0Ztf5sOPCE3aHDY0VpJ03wtFHGetMw6Xp3PWlFgB4wTyGF4PN2v9bVABQD4SMik2O Y+QBKrGPgkl7+m9yZXIzWLsDr+gOoGta8TUAjAsMAHJxkTO/uXXvYQE/VnbkKKNWxqhTkxjW AAKsGyzBhS5ypfWQFJoYsSTIXyKVSiVBr0jFouf5szllsQIqTqZhEhlGxaif+DAuT5iOtd7Y 3Wt86YF8lzIz8GvzRKRlZulAd8lscPieyhdfy//LZavu9QIX+mOj8k2yh1foqTs6V/PdzGbL u6vytMr7UdBVvhi98nDhfCgMuFvt+GPYF2drXALYgT+f+Scv1Oxt6j53c9NV15S9x76vcXdf nLpokjKX3tuaNDdTqO06FTZ1POXQ8bKffEIG8pQupbvyJ6vKO7F9Byu3qX9/66Db8juR4WXX +qv2HxsiDDuIRUcjD5tqINK6nO9E5Cee2RdxoO3Vsoktsg1YbhFKidyg9KKhX6jHv79KIL/q iHsP0ZQZymJRQz9jRjZ/WWt8Lqr4jcmJO70baz+4V/q+sH3UwxxlUYkINl4m9sLVrOw/6406 /JAEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrDIsWRmVeSWpSXmKPExsWy7bCSvK6zmneawYwXxhbrTx1jtmia8JfZ YvXdfjaL14c/MVpM+/CT2eLJgXZGi99nzzNbbDl2j9FiwaK5LBY3D+xkstizaBKTxcrVR5ks Zk9vZrI4+v8tm8WkQ9cYLZ5encVksfeWtsXCtiUsFnv2nmSxuLxrDpvF/GVP2S2WH//HZDGx 4yqTxY4njYwW237PZ7ZY9/o9i8WJW9IW5/8eZ3WQ8bh8xdvj1CIJj52z7rJ7nL+3kcXj8tlS j02rOtk8Ni+p93ixeSajx+6bDWwei/sms3r0Nr9j89jZep/V4+PTWywe7/ddZfPo27KK0ePM giPsAcJRXDYpqTmZZalF+nYJXBk3D31mLfguVHH52SHmBsYWgS5GDg4JAROJr4/1uhi5OIQE tjNKXDjwgKWLkRMoLimx7O8RZghbWGLlv+fsEEXNTBJXTtxmB2lmE9CWOP2fAyQuIrCdWeJj czcTSAOzwFFmiQtrLEBsYQF3iaWnlzOC2CwCqhL/zu5hBenlFbCSuLoiA2K+vMTqDQfAdnEC hQ9s/QBWLiRgKXH3+ge2CYx8CxgZVjGKphYU56bnJhcY6hUn5haX5qXrJefnbmIER6hW0A7G Zev/6h1iZOJgPMQowcGsJMK7aYtnmhBvSmJlVWpRfnxRaU5q8SFGaQ4WJXFe5ZzOFCGB9MSS 1OzU1ILUIpgsEwenVANT0ObT0idPVk6euerWpKztq3Kn3opyVyhcEp0r37H+2FutzrkVfRMe F3Hs35r56Xbc0dV/e7lYp37jaZYQuBefyBZy/N/HvmxhCYNAptbnu49zWqhI3grw41av1quK zla02//S6bXwNStvZymvuD7z8zLdu4/5Z8U8Lvi+ti/0f7fY2cQXSoYd12U2Py0zWrxL0cVT MbRqoqNOaDNj3Fs94XXT+k09wmxLLRaLH/0w8zDzYu0DWQ233HhP7vqhdkLvtoF/otGSF0rJ E1YbaHdvsTUxf/zzvq3VDsWa/b63p2WuUVmZsfTxXdltBiFne9z3b12Se8z2Z4tE45+KEw/5 oq5WsfXdf+hiWyZ3+aMSS3FGoqEWc1FxIgBOVbFnPwMAAA== X-CMS-MailID: 20240520103027epcas5p4789defe8ab3bff23bd2abcf019689fa2 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520103027epcas5p4789defe8ab3bff23bd2abcf019689fa2 References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: This is a prep patch to enable copy trace capability. At present only zoned null_block is using trace, so we decoupled trace and zoned dependency to make it usable in null_blk driver also. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta Reviewed-by: Bart Van Assche --- drivers/block/null_blk/Makefile | 2 -- drivers/block/null_blk/main.c | 3 +++ drivers/block/null_blk/trace.h | 2 ++ drivers/block/null_blk/zoned.c | 1 - 4 files changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/block/null_blk/Makefile b/drivers/block/null_blk/Makefile index 84c36e512ab8..672adcf0ad24 100644 --- a/drivers/block/null_blk/Makefile +++ b/drivers/block/null_blk/Makefile @@ -5,7 +5,5 @@ ccflags-y += -I$(src) obj-$(CONFIG_BLK_DEV_NULL_BLK) += null_blk.o null_blk-objs := main.o -ifeq ($(CONFIG_BLK_DEV_ZONED), y) null_blk-$(CONFIG_TRACING) += trace.o -endif null_blk-$(CONFIG_BLK_DEV_ZONED) += zoned.o diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index 5d56ad4ce01a..b33b9ebfebd2 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -11,6 +11,9 @@ #include #include "null_blk.h" +#define CREATE_TRACE_POINTS +#include "trace.h" + #undef pr_fmt #define pr_fmt(fmt) "null_blk: " fmt diff --git a/drivers/block/null_blk/trace.h b/drivers/block/null_blk/trace.h index 82b8f6a5e5f0..f9eadac6b22f 100644 --- a/drivers/block/null_blk/trace.h +++ b/drivers/block/null_blk/trace.h @@ -30,6 +30,7 @@ static inline void __assign_disk_name(char *name, struct gendisk *disk) } #endif +#ifdef CONFIG_BLK_DEV_ZONED TRACE_EVENT(nullb_zone_op, TP_PROTO(struct nullb_cmd *cmd, unsigned int zone_no, unsigned int zone_cond), @@ -73,6 +74,7 @@ TRACE_EVENT(nullb_report_zones, TP_printk("%s nr_zones=%u", __print_disk_name(__entry->disk), __entry->nr_zones) ); +#endif /* CONFIG_BLK_DEV_ZONED */ #endif /* _TRACE_NULLB_H */ diff --git a/drivers/block/null_blk/zoned.c b/drivers/block/null_blk/zoned.c index 5b5a63adacc1..47470a732d21 100644 --- a/drivers/block/null_blk/zoned.c +++ b/drivers/block/null_blk/zoned.c @@ -3,7 +3,6 @@ #include #include "null_blk.h" -#define CREATE_TRACE_POINTS #include "trace.h" #undef pr_fmt From patchwork Mon May 20 10:20:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 13668237 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1EB5770F7 for ; Mon, 20 May 2024 11:45:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205540; cv=none; b=pNMEIp815ZqEt4EDBRnlVq5K4rPx+6YnR0sRx8n24GHeiV+Gcs+fLTAilRjWsPcW2ViYwm5lE3ytxr3A1DP4f2dpaqwR42Hl/5pV5GnpnoelPdIzFYHb0OWGNO811cpP65iH7GHjKEhwKIviSk8GQYiSIGVXzqyx6qZiem5/a84= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716205540; c=relaxed/simple; bh=WxBOT7xRxNJe08AQ2ltG9cdASh90iKIABHzWDmS2oBg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:Content-Type: References; b=pNEcvY1jlRUtSEDQ1cXJ3H9deapNiPX0lxHdkxFWCb/zo0YCrPAewDh4e7pZOF6RIOtZqcz2Ag6MJ3eHe5o6P25s8sHKWoipPDf/WsDe63WpRQXcs0KAnlOrNNYEfktBJqgv0tb2EjzYsMxeNwkvNbb3I30MHXsAgP05hUvsLHI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=R4BBaoiC; arc=none smtp.client-ip=203.254.224.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="R4BBaoiC" Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20240520114537epoutp042e18c14239a98386a0346a81f17b4fe2~RL9fnjkHI2445324453epoutp04W for ; Mon, 20 May 2024 11:45:37 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20240520114537epoutp042e18c14239a98386a0346a81f17b4fe2~RL9fnjkHI2445324453epoutp04W DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1716205537; bh=684w/N8yQzC6Tn8Ze9K1madA6uBopMwCq6oINoaC0Mk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R4BBaoiCg7eO3NbkyQRj20qNXHD/Cb4YTJ/QwhNs+nRp6U6K3WTtYgLBhYAvh8rB3 f+jEdjjnoc9gPqhk0Ol9g6yQeLiyQypbwdyXqovOgDrQNzp1X8hbYdZTg/wtua3cMI dA3op2tJo3arCHZNMIzTy1/nPpg1yhhsgOAWX/7E= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20240520114536epcas5p4b47a99e3eb8f8fbcfb7db37da5dd4812~RL9e0A5aO0859008590epcas5p4U; Mon, 20 May 2024 11:45:36 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.177]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4VjbNb0ddLz4x9Px; Mon, 20 May 2024 11:45:35 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id BD.92.09688.ED73B466; Mon, 20 May 2024 20:45:34 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20240520103039epcas5p4373f7234162a32222ac225b976ae30ce~RK8DBSwIz0626706267epcas5p4I; Mon, 20 May 2024 10:30:39 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240520103039epsmtrp14db3a1d303b0f5c58bde00b466dac9a4~RK8DAL2vU2229722297epsmtrp10; Mon, 20 May 2024 10:30:39 +0000 (GMT) X-AuditID: b6c32a4a-5dbff700000025d8-99-664b37de4891 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id E1.98.08924.F462B466; Mon, 20 May 2024 19:30:39 +0900 (KST) Received: from green245.sa.corp.samsungelectronics.net (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240520103035epsmtip2856ca038c4c00631139b63e93b5062cd~RK7-I3dqL2512325123epsmtip2W; Mon, 20 May 2024 10:30:35 +0000 (GMT) From: Nitesh Shetty To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner , Jan Kara Cc: martin.petersen@oracle.com, bvanassche@acm.org, david@fromorbit.com, hare@suse.de, damien.lemoal@opensource.wdc.com, anuj20.g@samsung.com, joshi.k@samsung.com, nitheshshetty@gmail.com, gost.dev@samsung.com, Nitesh Shetty , Vincent Fu , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dm-devel@lists.linux.dev, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v20 12/12] null_blk: add support for copy offload Date: Mon, 20 May 2024 15:50:25 +0530 Message-Id: <20240520102033.9361-13-nj.shetty@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520102033.9361-1-nj.shetty@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA0WTe0xbVRzHc+693LZLOm4KwzOQScqmAuFRoHDYYJsM9TI0IfE5NKuFXh4D Su1j6GLkWcZjjJcoMjaQwdioAQblOZhdEdCypRpegwkDeYjCBEnEDbJhS0H/+/y+5/f45nfy Y+O8LdKeHSdVMnKpOIFP7iHael1c3Kf8w6K9crOFqNHQj6P0wqc40kwWkGipdw2gL1ef4GhO dx6gzXtGHGn7pwCqqr5MoHFdJ4a6q4sxdEPTh6FLX2VgqG/rEYmK9aMAzY+UY6hnwg19k1VD oO6eHwk01FVBospr8yxUN/AMQ0XZIxjqmEsDqG2zEkcNSysE+mHCAc3mZQNkfDpgddyRHhoO ow3VkO4sn2TRxqmbBD10T0U31+eQdEtNCr3Y8jWgb42nkvTViyVWdH7GnyTdqX5oRf81P0HQ K7dHSPqith7Qd6u+Z4XbRMQHxjJiCSN3YqRRSZI4aUwQP+wt0QmR0M9L4C4IQP58J6k4kQni h7wR7v5aXIJpTXyns+IElUkKFysUfM+jgfIklZJxik1SKIP4jEySIPOVeSjEiQqVNMZDyigP C7y8vIWmxI/iY1e0PJk29JOM5cRUYAjMBRw2pHxhW+kCyAV72DzqFoBZv03i5gcetQZgzqqr hdcBXGt32S242jRAWgp6AByu2cAtgRqD6soNUys2m6Tc4OAW26zbUhoc5rUUEeYAp+7jsLZp ATO3sqFegQ8yNdtMUIegdmYFmJlLHYZ/N/wELONegJom3bYljknXta5ue4VUCwfWLmpIS1II zLyfhlvYBv4xoGVZ2B7+XpC1w8nwxhfXSUtxJoDlY+U7E45BtaEAN9vGKRfY2OVpkR1hqaFh 2xxO7YX5m3OYRefCjiu77Ay/baza8bAfjv6TtsM0XO6f31lqPoATHbWsQnCg/P8RVQDUg/2M TJEYwyiEMm8pk/zfp0UlJTaD7UtwPdkBZqZXPfQAYwM9gGycb8tt1oZG87gS8afnGHmSSK5K YBR6IDRtsAi33xeVZDolqVIk8A3w8vXz8/MN8PET8J/jLqkvS3hUjFjJxDOMjJHv1mFsjn0q lndIVTI/E3z7+uk+weeVGcKOI1YuR96PjL8zbq3/eeim9dpYybtB6cqi/IC9dRvD1h5nqkPO vrQuupJ+p+Cd41sO3DMHZh92X2oZ7o0qGwvOcj517rFBqPcMXBibD/2VmE5+m/wO9q5/QJzI Xe9RsQ1l0+LPCPC87jxayXnyavqHj91TrgkmT98tNLz8XoORqhgU1VWwjM4X9ultPB2Xwod6 ph4VWHGrTvFLM2YjotR2i7YPAsftfFrLWW3L6byI13X+bTORrDKvwlSHN40c6Wjws+J2/8Gj v6Smt5707/KIPZZie3CQtIlp93nRvvWgMjI6T2W3nspTLdQ3uElmHD725vAJRaxY4IrLFeJ/ AV/DHzSSBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrGIsWRmVeSWpSXmKPExsWy7bCSvK6/mneaweR36hbrTx1jtmia8JfZ YvXdfjaL14c/MVpM+/CT2eLJgXZGi99nzzNbbDl2j9FiwaK5LBY3D+xkstizaBKTxcrVR5ks Zk9vZrI4+v8tm8WkQ9cYLZ5encVksfeWtsXCtiUsFnv2nmSxuLxrDpvF/GVP2S2WH//HZDGx 4yqTxY4njYwW237PZ7ZY9/o9i8WJW9IWj7s7GC3O/z3O6iDrcfmKt8epRRIeO2fdZfc4f28j i8fls6Uem1Z1snlsXlLv8WLzTEaP3Tcb2DwW901m9ehtfsfmsbP1PqvHx6e3WDze77vK5tG3 ZRWjx5kFR9gDhKO4bFJSczLLUov07RK4Mt5vESrY4lnR/Ca3gfGUTRcjJ4eEgInE4g3H2boY uTiEBHYzSvya+I0RIiEpsezvEWYIW1hi5b/n7BBFzUwSHet2AyU4ONgEtCVO/+cAiYsIbGeW +NjczQTiMAu8YZY4+/0KO0i3sICjxO2W1UwgNouAqsSWh+/BNvAKWEl8XXcBapu8xOoNB8C2 cQLFD2z9ABYXErCUuHv9A9sERr4FjAyrGCVTC4pz03OLDQsM81LL9YoTc4tL89L1kvNzNzGC o1ZLcwfj9lUf9A4xMnEwHmKU4GBWEuHdtMUzTYg3JbGyKrUoP76oNCe1+BCjNAeLkjiv+Ive FCGB9MSS1OzU1ILUIpgsEwenVANT6rmL7QlpPx+HFgq+znct9/FxVXpo2GH/6XbL8eLqxZmf Qu74B8ndFq18KeznE71uAvOkjU3eezQcpvEoaTVYX2Z+//BhSmPi9ezZMwUmqxmKWTsyqm8M ecAY5d53OWeK/EyPmxZVtfmfE4y3hC2O1RHvVvglo3Pam0viP99izlvTv4T/0nlVtP20WdeD 508bRC2P/hHgedaWYqDaFeF36OWR3bvnLVxnf1L0uPAP7tzjG0vWzhY4Ua3LFlibX6sby7S2 bZJZEY9c9ebYRxv4elm6ykuKm05tvuOq0vvBZu8miUkZq8RCt717E+yjKSed7XDl0DqWjH6b lIVPGc/9nuDKNdXh6dYze150xCmxFGckGmoxFxUnAgBu3X+ISQMAAA== X-CMS-MailID: 20240520103039epcas5p4373f7234162a32222ac225b976ae30ce X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240520103039epcas5p4373f7234162a32222ac225b976ae30ce References: <20240520102033.9361-1-nj.shetty@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Implementation is based on existing read and write infrastructure. copy_max_bytes: A new configfs and module parameter is introduced, which can be used to set hardware/driver supported maximum copy limit. Only request based queue mode will support for copy offload. Added tracefs support to copy IO tracing. Reviewed-by: Hannes Reinecke Suggested-by: Damien Le Moal Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty Signed-off-by: Vincent Fu --- Documentation/block/null_blk.rst | 5 ++ drivers/block/null_blk/main.c | 102 +++++++++++++++++++++++++++++- drivers/block/null_blk/null_blk.h | 1 + drivers/block/null_blk/trace.h | 23 +++++++ 4 files changed, 128 insertions(+), 3 deletions(-) diff --git a/Documentation/block/null_blk.rst b/Documentation/block/null_blk.rst index 4dd78f24d10a..6153e02fcf13 100644 --- a/Documentation/block/null_blk.rst +++ b/Documentation/block/null_blk.rst @@ -149,3 +149,8 @@ zone_size=[MB]: Default: 256 zone_nr_conv=[nr_conv]: Default: 0 The number of conventional zones to create when block device is zoned. If zone_nr_conv >= nr_zones, it will be reduced to nr_zones - 1. + +copy_max_bytes=[size in bytes]: Default: COPY_MAX_BYTES + A module and configfs parameter which can be used to set hardware/driver + supported maximum copy offload limit. + COPY_MAX_BYTES(=128MB at present) is defined in fs.h diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index b33b9ebfebd2..dcfbd5275414 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -172,6 +172,10 @@ static int g_max_sectors; module_param_named(max_sectors, g_max_sectors, int, 0444); MODULE_PARM_DESC(max_sectors, "Maximum size of a command (in 512B sectors)"); +static unsigned long g_copy_max_bytes = BLK_COPY_MAX_BYTES; +module_param_named(copy_max_bytes, g_copy_max_bytes, ulong, 0444); +MODULE_PARM_DESC(copy_max_bytes, "Maximum size of a copy command (in bytes)"); + static unsigned int nr_devices = 1; module_param(nr_devices, uint, 0444); MODULE_PARM_DESC(nr_devices, "Number of devices to register"); @@ -433,6 +437,7 @@ NULLB_DEVICE_ATTR(home_node, uint, NULL); NULLB_DEVICE_ATTR(queue_mode, uint, NULL); NULLB_DEVICE_ATTR(blocksize, uint, NULL); NULLB_DEVICE_ATTR(max_sectors, uint, NULL); +NULLB_DEVICE_ATTR(copy_max_bytes, uint, NULL); NULLB_DEVICE_ATTR(irqmode, uint, NULL); NULLB_DEVICE_ATTR(hw_queue_depth, uint, NULL); NULLB_DEVICE_ATTR(index, uint, NULL); @@ -577,6 +582,7 @@ static struct configfs_attribute *nullb_device_attrs[] = { &nullb_device_attr_queue_mode, &nullb_device_attr_blocksize, &nullb_device_attr_max_sectors, + &nullb_device_attr_copy_max_bytes, &nullb_device_attr_irqmode, &nullb_device_attr_hw_queue_depth, &nullb_device_attr_index, @@ -687,7 +693,7 @@ static ssize_t memb_group_features_show(struct config_item *item, char *page) "shared_tags,size,submit_queues,use_per_node_hctx," "virt_boundary,zoned,zone_capacity,zone_max_active," "zone_max_open,zone_nr_conv,zone_offline,zone_readonly," - "zone_size,zone_append_max_sectors\n"); + "zone_size,zone_append_max_sectors,copy_max_bytes\n"); } CONFIGFS_ATTR_RO(memb_group_, features); @@ -753,6 +759,7 @@ static struct nullb_device *null_alloc_dev(void) dev->queue_mode = g_queue_mode; dev->blocksize = g_bs; dev->max_sectors = g_max_sectors; + dev->copy_max_bytes = g_copy_max_bytes; dev->irqmode = g_irqmode; dev->hw_queue_depth = g_hw_queue_depth; dev->blocking = g_blocking; @@ -1221,6 +1228,81 @@ static int null_transfer(struct nullb *nullb, struct page *page, return err; } +static inline int nullb_setup_copy(struct nullb *nullb, struct request *req, + bool is_fua) +{ + sector_t sector_in = 0, sector_out = 0; + loff_t offset_in, offset_out; + void *in, *out; + ssize_t chunk, rem = 0; + struct bio *bio; + struct nullb_page *t_page_in, *t_page_out; + u16 seg = 1; + int status = -EIO; + + if (blk_rq_nr_phys_segments(req) != BLK_COPY_MAX_SEGMENTS) + return status; + + /* + * First bio contains information about destination and last bio + * contains information about source. + */ + __rq_for_each_bio(bio, req) { + if (seg == blk_rq_nr_phys_segments(req)) { + sector_in = bio->bi_iter.bi_sector; + if (rem != bio->bi_iter.bi_size) + return status; + } else { + sector_out = bio->bi_iter.bi_sector; + rem = bio->bi_iter.bi_size; + } + seg++; + } + + trace_nullb_copy_op(req, sector_out << SECTOR_SHIFT, + sector_in << SECTOR_SHIFT, rem); + + spin_lock_irq(&nullb->lock); + while (rem > 0) { + chunk = min_t(size_t, nullb->dev->blocksize, rem); + offset_in = (sector_in & SECTOR_MASK) << SECTOR_SHIFT; + offset_out = (sector_out & SECTOR_MASK) << SECTOR_SHIFT; + + if (null_cache_active(nullb) && !is_fua) + null_make_cache_space(nullb, PAGE_SIZE); + + t_page_in = null_lookup_page(nullb, sector_in, false, + !null_cache_active(nullb)); + if (!t_page_in) + goto err; + t_page_out = null_insert_page(nullb, sector_out, + !null_cache_active(nullb) || + is_fua); + if (!t_page_out) + goto err; + + in = kmap_local_page(t_page_in->page); + out = kmap_local_page(t_page_out->page); + + memcpy(out + offset_out, in + offset_in, chunk); + kunmap_local(out); + kunmap_local(in); + __set_bit(sector_out & SECTOR_MASK, t_page_out->bitmap); + + if (is_fua) + null_free_sector(nullb, sector_out, true); + + rem -= chunk; + sector_in += chunk >> SECTOR_SHIFT; + sector_out += chunk >> SECTOR_SHIFT; + } + + status = 0; +err: + spin_unlock_irq(&nullb->lock); + return status; +} + static blk_status_t null_handle_rq(struct nullb_cmd *cmd) { struct request *rq = blk_mq_rq_from_pdu(cmd); @@ -1230,13 +1312,16 @@ static blk_status_t null_handle_rq(struct nullb_cmd *cmd) sector_t sector = blk_rq_pos(rq); struct req_iterator iter; struct bio_vec bvec; + bool fua = rq->cmd_flags & REQ_FUA; + + if (op_is_copy(req_op(rq))) + return nullb_setup_copy(nullb, rq, fua); spin_lock_irq(&nullb->lock); rq_for_each_segment(bvec, rq, iter) { len = bvec.bv_len; err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset, - op_is_write(req_op(rq)), sector, - rq->cmd_flags & REQ_FUA); + op_is_write(req_op(rq)), sector, fua); if (err) break; sector += len >> SECTOR_SHIFT; @@ -1721,6 +1806,12 @@ static void null_config_discard(struct nullb *nullb, struct queue_limits *lim) lim->max_hw_discard_sectors = UINT_MAX >> 9; } +static void null_config_copy(struct nullb *nullb, struct queue_limits *lim) +{ + lim->max_copy_hw_sectors = nullb->dev->copy_max_bytes >> SECTOR_SHIFT; + lim->max_copy_sectors = nullb->dev->copy_max_bytes >> SECTOR_SHIFT; +} + static const struct block_device_operations null_ops = { .owner = THIS_MODULE, .report_zones = null_report_zones, @@ -1843,6 +1934,9 @@ static int null_validate_conf(struct nullb_device *dev) return -EINVAL; } + if (dev->queue_mode == NULL_Q_BIO) + dev->copy_max_bytes = 0; + return 0; } @@ -1909,6 +2003,8 @@ static int null_add_dev(struct nullb_device *dev) if (dev->virt_boundary) lim.virt_boundary_mask = PAGE_SIZE - 1; null_config_discard(nullb, &lim); + null_config_copy(nullb, &lim); + if (dev->zoned) { rv = null_init_zoned_dev(dev, &lim); if (rv) diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h index 3234e6c85eed..c588729c17bd 100644 --- a/drivers/block/null_blk/null_blk.h +++ b/drivers/block/null_blk/null_blk.h @@ -91,6 +91,7 @@ struct nullb_device { unsigned int queue_mode; /* block interface */ unsigned int blocksize; /* block size */ unsigned int max_sectors; /* Max sectors per command */ + unsigned long copy_max_bytes; /* Max copy offload length in bytes */ unsigned int irqmode; /* IRQ completion handler */ unsigned int hw_queue_depth; /* queue depth */ unsigned int index; /* index of the disk, only valid with a disk */ diff --git a/drivers/block/null_blk/trace.h b/drivers/block/null_blk/trace.h index f9eadac6b22f..cda1a2249978 100644 --- a/drivers/block/null_blk/trace.h +++ b/drivers/block/null_blk/trace.h @@ -76,6 +76,29 @@ TRACE_EVENT(nullb_report_zones, ); #endif /* CONFIG_BLK_DEV_ZONED */ +TRACE_EVENT(nullb_copy_op, + TP_PROTO(struct request *req, + sector_t dst, sector_t src, size_t len), + TP_ARGS(req, dst, src, len), + TP_STRUCT__entry( + __array(char, disk, DISK_NAME_LEN) + __field(enum req_op, op) + __field(sector_t, dst) + __field(sector_t, src) + __field(size_t, len) + ), + TP_fast_assign( + __entry->op = req_op(req); + __assign_disk_name(__entry->disk, req->q->disk); + __entry->dst = dst; + __entry->src = src; + __entry->len = len; + ), + TP_printk("%s req=%-15s: dst=%llu, src=%llu, len=%lu", + __print_disk_name(__entry->disk), + blk_op_str(__entry->op), + __entry->dst, __entry->src, __entry->len) +); #endif /* _TRACE_NULLB_H */ #undef TRACE_INCLUDE_PATH