| Message ID | 20201201053949.143175-1-selvakuma.s1@samsung.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | add simple copy support | expand |
+ Mike and DM list On 2020/12/01 16:12, SelvaKumar S wrote: > This patchset tries to add support for TP4065a ("Simple Copy Command"), > v2020.05.04 ("Ratified") > > The Specification can be found in following link. > https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip > > This is an RFC. Looking forward for any feedbacks or other alternate > designs for plumbing simple copy to IO stack. > > Simple copy command is a copy offloading operation and is used to copy > multiple contiguous ranges (source_ranges) of LBA's to a single destination > LBA within the device reducing traffic between host and device. > > This implementation accepts destination, no of sources and arrays of > source ranges from application and attach it as payload to the bio and > submits to the device. > > Following limits are added to queue limits and are exposed in sysfs > to userspace > - *max_copy_sectors* limits the sum of all source_range length > - *max_copy_nr_ranges* limits the number of source ranges > - *max_copy_range_sectors* limit the maximum number of sectors > that can constitute a single source range. This is interesting. I think there are several possible use in the kernel in various components: FS (btrfs rebalance, f2fs GC, liklely others) and DM at the very least. However, your patches add support only for NVMe devices that have native support for simple copy, leaving all other block devices out. That seriously limits the use cases and also does not make this solution attractive since any use of it would need to be conditional on the underlying drive capabilities. That means more code for the file systems or device mapper developers and maintainers, not less. To avoid this, I would suggest that this code be extended to add emulation for drives that do not implement simple copy natively. This would allow this interface to work on any block device, including SAS & SATA HDDs and RAID arrays. The emulation part of this copy service could I think be based on dm-kcopyd. See include/linux/dm-kcopyd.h for the interface. The current dm-kcopyd interface takes one source and multiple destination, the reverse of simple copy. But it would be fairly straightforward to also allow multiple sources and one destination. Simple copy native support would accelerate this case, everything else using the regular BIO read+write interface. Moving dm-kcopyd from DM infrastructure into the block layer as a set a generic block device sector copy service would allow its use in more places. And SCSI XCOPY could also be integrated in there as a different drive native support command. > > > SelvaKumar S (2): > block: add simple copy support > nvme: add simple copy support > > block/blk-core.c | 104 +++++++++++++++++++++++++++++++--- > block/blk-lib.c | 116 ++++++++++++++++++++++++++++++++++++++ > block/blk-merge.c | 2 + > block/blk-settings.c | 11 ++++ > block/blk-sysfs.c | 23 ++++++++ > block/blk-zoned.c | 1 + > block/bounce.c | 1 + > block/ioctl.c | 43 ++++++++++++++ > drivers/nvme/host/core.c | 91 ++++++++++++++++++++++++++++++ > drivers/nvme/host/nvme.h | 4 ++ > include/linux/bio.h | 1 + > include/linux/blk_types.h | 7 +++ > include/linux/blkdev.h | 15 +++++ > include/linux/nvme.h | 45 +++++++++++++-- > include/uapi/linux/fs.h | 21 +++++++ > 15 files changed, 473 insertions(+), 12 deletions(-) >
On 01.12.2020 11:22, Damien Le Moal wrote: >+ Mike and DM list > >On 2020/12/01 16:12, SelvaKumar S wrote: >> This patchset tries to add support for TP4065a ("Simple Copy Command"), >> v2020.05.04 ("Ratified") >> >> The Specification can be found in following link. >> https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip >> >> This is an RFC. Looking forward for any feedbacks or other alternate >> designs for plumbing simple copy to IO stack. >> >> Simple copy command is a copy offloading operation and is used to copy >> multiple contiguous ranges (source_ranges) of LBA's to a single destination >> LBA within the device reducing traffic between host and device. >> >> This implementation accepts destination, no of sources and arrays of >> source ranges from application and attach it as payload to the bio and >> submits to the device. >> >> Following limits are added to queue limits and are exposed in sysfs >> to userspace >> - *max_copy_sectors* limits the sum of all source_range length >> - *max_copy_nr_ranges* limits the number of source ranges >> - *max_copy_range_sectors* limit the maximum number of sectors >> that can constitute a single source range. > >This is interesting. I think there are several possible use in the kernel in >various components: FS (btrfs rebalance, f2fs GC, liklely others) and DM at the >very least. Totally agree. We have more patches for simple copy, among others work on F2FS that leverages for GC. We wanted to start with a simple patchset enabling IOCTL to be an easy to review start. The rest of the patches will come. >However, your patches add support only for NVMe devices that have native support >for simple copy, leaving all other block devices out. That seriously limits the >use cases and also does not make this solution attractive since any use of it >would need to be conditional on the underlying drive capabilities. That means >more code for the file systems or device mapper developers and maintainers, not >less. Makes sense. > >To avoid this, I would suggest that this code be extended to add emulation for >drives that do not implement simple copy natively. This would allow this >interface to work on any block device, including SAS & SATA HDDs and RAID arrays. > >The emulation part of this copy service could I think be based on dm-kcopyd. See >include/linux/dm-kcopyd.h for the interface. The current dm-kcopyd interface >takes one source and multiple destination, the reverse of simple copy. But it >would be fairly straightforward to also allow multiple sources and one >destination. Simple copy native support would accelerate this case, everything >else using the regular BIO read+write interface. Moving dm-kcopyd from DM >infrastructure into the block layer as a set a generic block device sector copy >service would allow its use in more places. And SCSI XCOPY could also be >integrated in there as a different drive native support command. Let us look into this. It makes sense. It seems fair to start support bottom up from NVMe and then extend to block layer and F2FS (potentially others). At this stage, we were assuming that all the work that people like Martin, Bart and other have been doing through the years on XCOPY could be integrated. Emulation can be a part of this. How can we move forward to do this in stages?
This patchset tries to add support for TP4065a ("Simple Copy Command"), v2020.05.04 ("Ratified") The Specification can be found in following link. https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip This is an RFC. Looking forward for any feedbacks or other alternate designs for plumbing simple copy to IO stack. Simple copy command is a copy offloading operation and is used to copy multiple contiguous ranges (source_ranges) of LBA's to a single destination LBA within the device reducing traffic between host and device. This implementation accepts destination, no of sources and arrays of source ranges from application and attach it as payload to the bio and submits to the device. Following limits are added to queue limits and are exposed in sysfs to userspace - *max_copy_sectors* limits the sum of all source_range length - *max_copy_nr_ranges* limits the number of source ranges - *max_copy_range_sectors* limit the maximum number of sectors that can constitute a single source range. SelvaKumar S (2): block: add simple copy support nvme: add simple copy support block/blk-core.c | 104 +++++++++++++++++++++++++++++++--- block/blk-lib.c | 116 ++++++++++++++++++++++++++++++++++++++ block/blk-merge.c | 2 + block/blk-settings.c | 11 ++++ block/blk-sysfs.c | 23 ++++++++ block/blk-zoned.c | 1 + block/bounce.c | 1 + block/ioctl.c | 43 ++++++++++++++ drivers/nvme/host/core.c | 91 ++++++++++++++++++++++++++++++ drivers/nvme/host/nvme.h | 4 ++ include/linux/bio.h | 1 + include/linux/blk_types.h | 7 +++ include/linux/blkdev.h | 15 +++++ include/linux/nvme.h | 45 +++++++++++++-- include/uapi/linux/fs.h | 21 +++++++ 15 files changed, 473 insertions(+), 12 deletions(-)