| Message ID | 20210207170924.2933035-6-ruansy.fnst@cn.fujitsu.com (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Headers | show |
| Series | fsdax,xfs: Add reflink&dedupe support for fsdax | expand |
On Mon, Feb 08, 2021 at 01:09:22AM +0800, Shiyang Ruan wrote: > With dax we cannot deal with readpage() etc. So, we create a > funciton callback to perform the file data comparison and pass s/funciton/function/g > +#define MIN(a, b) (((a) < (b)) ? (a) : (b)) This should use the existing min or min_t helpers. > int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in, > struct file *file_out, loff_t pos_out, > - loff_t *len, unsigned int remap_flags) > + loff_t *len, unsigned int remap_flags, > + compare_range_t compare_range_fn) Can we keep generic_remap_file_range_prep as-is, and add a new dax_remap_file_range_prep, both sharing a low-level __generic_remap_file_range_prep implementation? And for that implementation I'd also go for classic if/else instead of the function pointer. > +extern int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff, > + struct inode *dest, loff_t destoff, > + loff_t len, bool *is_same); no need for the extern.
On 2021/2/8 下午11:19, Christoph Hellwig wrote: > On Mon, Feb 08, 2021 at 01:09:22AM +0800, Shiyang Ruan wrote: >> With dax we cannot deal with readpage() etc. So, we create a >> funciton callback to perform the file data comparison and pass > > s/funciton/function/g > >> +#define MIN(a, b) (((a) < (b)) ? (a) : (b)) > > This should use the existing min or min_t helpers. > > >> int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in, >> struct file *file_out, loff_t pos_out, >> - loff_t *len, unsigned int remap_flags) >> + loff_t *len, unsigned int remap_flags, >> + compare_range_t compare_range_fn) > > Can we keep generic_remap_file_range_prep as-is, and add a new > dax_remap_file_range_prep, both sharing a low-level > __generic_remap_file_range_prep implementation? And for that > implementation I'd also go for classic if/else instead of the function > pointer. The dax dedupe comparison need the iomap_ops pointer as argument, so my understanding is that we don't modify the argument list of generic_remap_file_range_prep(), but move its code into __generic_remap_file_range_prep() whose argument list can be modified to accepts the iomap_ops pointer. Then it looks like this: ``` int dax_remap_file_range_prep(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, loff_t *len, unsigned int remap_flags, const struct iomap_ops *ops) { return __generic_remap_file_range_prep(file_in, pos_in, file_out, pos_out, len, remap_flags, ops); } EXPORT_SYMBOL(dax_remap_file_range_prep); int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, loff_t *len, unsigned int remap_flags) { return __generic_remap_file_range_prep(file_in, pos_in, file_out, pos_out, len, remap_flags, NULL); } EXPORT_SYMBOL(generic_remap_file_range_prep); ``` Am i right? -- Thanks, Ruan Shiyang. > >> +extern int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff, >> + struct inode *dest, loff_t destoff, >> + loff_t len, bool *is_same); > > no need for the extern. > >
On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote: > The dax dedupe comparison need the iomap_ops pointer as argument, so my > understanding is that we don't modify the argument list of > generic_remap_file_range_prep(), but move its code into > __generic_remap_file_range_prep() whose argument list can be modified to > accepts the iomap_ops pointer. Then it looks like this: I'd say just add the iomap_ops pointer to generic_remap_file_range_prep and do away with the extra wrappers. We only have three callers anyway.
On 2021/2/9 下午5:34, Christoph Hellwig wrote: > On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote: >> The dax dedupe comparison need the iomap_ops pointer as argument, so my >> understanding is that we don't modify the argument list of >> generic_remap_file_range_prep(), but move its code into >> __generic_remap_file_range_prep() whose argument list can be modified to >> accepts the iomap_ops pointer. Then it looks like this: > > I'd say just add the iomap_ops pointer to > generic_remap_file_range_prep and do away with the extra wrappers. We > only have three callers anyway. OK. -- Thanks, Ruan Shiyang. > >
On Tue, Feb 09, 2021 at 05:46:13PM +0800, Ruan Shiyang wrote: > > > On 2021/2/9 下午5:34, Christoph Hellwig wrote: >> On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote: >>> The dax dedupe comparison need the iomap_ops pointer as argument, so my >>> understanding is that we don't modify the argument list of >>> generic_remap_file_range_prep(), but move its code into >>> __generic_remap_file_range_prep() whose argument list can be modified to >>> accepts the iomap_ops pointer. Then it looks like this: >> >> I'd say just add the iomap_ops pointer to >> generic_remap_file_range_prep and do away with the extra wrappers. We >> only have three callers anyway. > > OK. So looking at this again I think your proposal actaully is better, given that the iomap variant is still DAX specific. Sorry for the noise. Also I think dax_file_range_compare should use iomap_apply instead of open coding it.
On 2021/2/10 下午9:19, Christoph Hellwig wrote: > On Tue, Feb 09, 2021 at 05:46:13PM +0800, Ruan Shiyang wrote: >> >> >> On 2021/2/9 下午5:34, Christoph Hellwig wrote: >>> On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote: >>>> The dax dedupe comparison need the iomap_ops pointer as argument, so my >>>> understanding is that we don't modify the argument list of >>>> generic_remap_file_range_prep(), but move its code into >>>> __generic_remap_file_range_prep() whose argument list can be modified to >>>> accepts the iomap_ops pointer. Then it looks like this: >>> >>> I'd say just add the iomap_ops pointer to >>> generic_remap_file_range_prep and do away with the extra wrappers. We >>> only have three callers anyway. >> >> OK. > > So looking at this again I think your proposal actaully is better, > given that the iomap variant is still DAX specific. Sorry for > the noise. > > Also I think dax_file_range_compare should use iomap_apply instead > of open coding it. > There are two files, which are not reflinked, need to be direct_access() here. The iomap_apply() can handle one file each time. So, it seems that iomap_apply() is not suitable for this case... The pseudo code of this process is as follows: srclen = ops->begin(&srcmap) destlen = ops->begin(&destmap) direct_access(&srcmap, &saddr) direct_access(&destmap, &daddr) same = memcpy(saddr, daddr, min(srclen,destlen)) ops->end(&destmap) ops->end(&srcmap) I think a nested call like this is necessary. That's why I use the open code way. -- Thanks, Ruan Shiyang. >
On Wed, Feb 17, 2021 at 11:24:18AM +0800, Ruan Shiyang wrote: > > > On 2021/2/10 下午9:19, Christoph Hellwig wrote: > > On Tue, Feb 09, 2021 at 05:46:13PM +0800, Ruan Shiyang wrote: > > > > > > > > > On 2021/2/9 下午5:34, Christoph Hellwig wrote: > > > > On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote: > > > > > The dax dedupe comparison need the iomap_ops pointer as argument, so my > > > > > understanding is that we don't modify the argument list of > > > > > generic_remap_file_range_prep(), but move its code into > > > > > __generic_remap_file_range_prep() whose argument list can be modified to > > > > > accepts the iomap_ops pointer. Then it looks like this: > > > > > > > > I'd say just add the iomap_ops pointer to > > > > generic_remap_file_range_prep and do away with the extra wrappers. We > > > > only have three callers anyway. > > > > > > OK. > > > > So looking at this again I think your proposal actaully is better, > > given that the iomap variant is still DAX specific. Sorry for > > the noise. > > > > Also I think dax_file_range_compare should use iomap_apply instead > > of open coding it. > > > > There are two files, which are not reflinked, need to be direct_access() > here. The iomap_apply() can handle one file each time. So, it seems that > iomap_apply() is not suitable for this case... > > > The pseudo code of this process is as follows: > > srclen = ops->begin(&srcmap) > destlen = ops->begin(&destmap) > > direct_access(&srcmap, &saddr) > direct_access(&destmap, &daddr) > > same = memcpy(saddr, daddr, min(srclen,destlen)) > > ops->end(&destmap) > ops->end(&srcmap) > > I think a nested call like this is necessary. That's why I use the open > code way. This might be a good place to implement an iomap_apply2() loop that actually /does/ walk all the extents of file1 and file2. There's now two users of this idiom. (Possibly structured as a "get next mappings from both" generator function like Matthew Wilcox keeps asking for. :)) --D > > -- > Thanks, > Ruan Shiyang. > > > >
On Thu, Feb 18, 2021 at 08:20:18AM -0800, Darrick J. Wong wrote: > > I think a nested call like this is necessary. That's why I use the open > > code way. > > This might be a good place to implement an iomap_apply2() loop that > actually /does/ walk all the extents of file1 and file2. There's now > two users of this idiom. Why do we need a special helper for that? > (Possibly structured as a "get next mappings from both" generator > function like Matthew Wilcox keeps asking for. :)) OTOH this might be a good first use for that.
diff --git a/fs/btrfs/reflink.c b/fs/btrfs/reflink.c index 99aa87c08912..efe094f7f748 100644 --- a/fs/btrfs/reflink.c +++ b/fs/btrfs/reflink.c @@ -783,7 +783,8 @@ static int btrfs_remap_file_range_prep(struct file *file_in, loff_t pos_in, return ret; return generic_remap_file_range_prep(file_in, pos_in, file_out, pos_out, - len, remap_flags); + len, remap_flags, + vfs_dedupe_file_range_compare); } loff_t btrfs_remap_file_range(struct file *src_file, loff_t off, diff --git a/fs/dax.c b/fs/dax.c index 29698a3d2e37..6e9391845057 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -30,6 +30,8 @@ #define CREATE_TRACE_POINTS #include <trace/events/fs_dax.h> +#define MIN(a, b) (((a) < (b)) ? (a) : (b)) + static inline unsigned int pe_order(enum page_entry_size pe_size) { if (pe_size == PE_SIZE_PTE) @@ -1827,3 +1829,68 @@ vm_fault_t dax_finish_sync_fault(struct vm_fault *vmf, return dax_insert_pfn_mkwrite(vmf, pfn, order); } EXPORT_SYMBOL_GPL(dax_finish_sync_fault); + +int dax_file_range_compare(struct inode *src, loff_t srcoff, struct inode *dest, + loff_t destoff, loff_t len, bool *is_same, + const struct iomap_ops *ops) +{ + void *saddr, *daddr; + struct iomap smap = { 0 }; + struct iomap dmap = { 0 }; + bool same = false; + loff_t cmp_len; + int id, ret = 0; + + id = dax_read_lock(); + while (len) { + ret = ops->iomap_begin(src, srcoff, len, 0, &smap, NULL); + if (ret < 0) + goto out_src; + cmp_len = MIN(len, smap.offset + smap.length - srcoff); + + ret = ops->iomap_begin(dest, destoff, cmp_len, 0, &dmap, NULL); + if (ret < 0) + goto out_dest; + cmp_len = MIN(cmp_len, dmap.offset + dmap.length - destoff); + + if (smap.type == IOMAP_HOLE && dmap.type == IOMAP_HOLE) + goto next; + + if (smap.type == IOMAP_HOLE || dmap.type == IOMAP_HOLE) { + same = false; + goto next; + } + + ret = dax_iomap_direct_access(&smap, srcoff, + ALIGN(srcoff + cmp_len, PAGE_SIZE), &saddr, NULL); + if (ret < 0) + goto out_dest; + + ret = dax_iomap_direct_access(&dmap, destoff, + ALIGN(destoff + cmp_len, PAGE_SIZE), &daddr, NULL); + if (ret < 0) + goto out_dest; + + same = !memcmp(saddr, daddr, cmp_len); + if (!same) + break; +next: + len -= cmp_len; + srcoff += cmp_len; + destoff += cmp_len; +out_dest: + if (ops->iomap_end) + ops->iomap_end(dest, destoff, cmp_len, 0, 0, &dmap); +out_src: + if (ops->iomap_end) + ops->iomap_end(src, srcoff, len, 0, 0, &smap); + + if (ret < 0) + goto out; + } + *is_same = same; +out: + dax_read_unlock(id); + return ret; +} +EXPORT_SYMBOL_GPL(dax_file_range_compare); diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c index 85979e2214b3..9d101f129d16 100644 --- a/fs/ocfs2/file.c +++ b/fs/ocfs2/file.c @@ -2591,7 +2591,7 @@ static loff_t ocfs2_remap_file_range(struct file *file_in, loff_t pos_in, goto out_unlock; ret = generic_remap_file_range_prep(file_in, pos_in, file_out, pos_out, - &len, remap_flags); + &len, remap_flags, vfs_dedupe_file_range_compare); if (ret < 0 || len == 0) goto out_unlock; diff --git a/fs/remap_range.c b/fs/remap_range.c index e6099beefa97..fc64f7e2af5a 100644 --- a/fs/remap_range.c +++ b/fs/remap_range.c @@ -199,9 +199,9 @@ static void vfs_unlock_two_pages(struct page *page1, struct page *page2) * Compare extents of two files to see if they are the same. * Caller must have locked both inodes to prevent write races. */ -static int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff, - struct inode *dest, loff_t destoff, - loff_t len, bool *is_same) +int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff, + struct inode *dest, loff_t destoff, + loff_t len, bool *is_same) { loff_t src_poff; loff_t dest_poff; @@ -280,6 +280,7 @@ static int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff, out_error: return error; } +EXPORT_SYMBOL(vfs_dedupe_file_range_compare); /* * Check that the two inodes are eligible for cloning, the ranges make @@ -291,7 +292,8 @@ static int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff, */ int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, - loff_t *len, unsigned int remap_flags) + loff_t *len, unsigned int remap_flags, + compare_range_t compare_range_fn) { struct inode *inode_in = file_inode(file_in); struct inode *inode_out = file_inode(file_out); @@ -351,8 +353,8 @@ int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in, if (remap_flags & REMAP_FILE_DEDUP) { bool is_same = false; - ret = vfs_dedupe_file_range_compare(inode_in, pos_in, - inode_out, pos_out, *len, &is_same); + ret = compare_range_fn(inode_in, pos_in, inode_out, pos_out, + *len, &is_same); if (ret) return ret; if (!is_same) diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c index 6fa05fb78189..26708c01fd78 100644 --- a/fs/xfs/xfs_reflink.c +++ b/fs/xfs/xfs_reflink.c @@ -1309,7 +1309,7 @@ xfs_reflink_remap_prep( goto out_unlock; ret = generic_remap_file_range_prep(file_in, pos_in, file_out, pos_out, - len, remap_flags); + len, remap_flags, vfs_dedupe_file_range_compare); if (ret || *len == 0) goto out_unlock; diff --git a/include/linux/dax.h b/include/linux/dax.h index b52f084aa643..183e084c2ac7 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -238,6 +238,11 @@ int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index); int dax_invalidate_mapping_entry_sync(struct address_space *mapping, pgoff_t index); s64 dax_iomap_zero(loff_t pos, u64 length, struct iomap *iomap); +int dax_file_range_compare(struct inode *src, loff_t srcoff, + struct inode *dest, loff_t destoff, + loff_t len, bool *is_same, + const struct iomap_ops *ops); + static inline bool dax_mapping(struct address_space *mapping) { return mapping->host && IS_DAX(mapping->host); diff --git a/include/linux/fs.h b/include/linux/fs.h index 8667d0cdc71e..159ec755e47b 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1912,13 +1912,20 @@ extern ssize_t vfs_read(struct file *, char __user *, size_t, loff_t *); extern ssize_t vfs_write(struct file *, const char __user *, size_t, loff_t *); extern ssize_t vfs_copy_file_range(struct file *, loff_t , struct file *, loff_t, size_t, unsigned int); +extern int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff, + struct inode *dest, loff_t destoff, + loff_t len, bool *is_same); +typedef int (*compare_range_t)(struct inode *src, loff_t srcpos, + struct inode *dest, loff_t destpos, + loff_t len, bool *is_same); extern ssize_t generic_copy_file_range(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, size_t len, unsigned int flags); extern int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, loff_t *count, - unsigned int remap_flags); + unsigned int remap_flags, + compare_range_t compare_range_fn); extern loff_t do_clone_file_range(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, loff_t len, unsigned int remap_flags);