diff mbox series

[5/7] fsdax: Dedup file range to use a compare function

Message ID 20210207170924.2933035-6-ruansy.fnst@cn.fujitsu.com (mailing list archive)
State New
Headers show
Series fsdax,xfs: Add reflink&dedupe support for fsdax | expand

Commit Message

Ruan Shiyang Feb. 7, 2021, 5:09 p.m. UTC
With dax we cannot deal with readpage() etc. So, we create a
funciton callback to perform the file data comparison and pass
it to generic_remap_file_range_prep() so it can use iomap-based
functions.

Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>
---
 fs/btrfs/reflink.c   |  3 +-
 fs/dax.c             | 67 ++++++++++++++++++++++++++++++++++++++++++++
 fs/ocfs2/file.c      |  2 +-
 fs/remap_range.c     | 14 +++++----
 fs/xfs/xfs_reflink.c |  2 +-
 include/linux/dax.h  |  5 ++++
 include/linux/fs.h   |  9 +++++-
 7 files changed, 92 insertions(+), 10 deletions(-)

Comments

Christoph Hellwig Feb. 8, 2021, 3:19 p.m. UTC | #1
On Mon, Feb 08, 2021 at 01:09:22AM +0800, Shiyang Ruan wrote:
> With dax we cannot deal with readpage() etc. So, we create a
> funciton callback to perform the file data comparison and pass

s/funciton/function/g

> +#define MIN(a, b) (((a) < (b)) ? (a) : (b))

This should use the existing min or min_t helpers.


>  int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in,
>  				  struct file *file_out, loff_t pos_out,
> -				  loff_t *len, unsigned int remap_flags)
> +				  loff_t *len, unsigned int remap_flags,
> +				  compare_range_t compare_range_fn)

Can we keep generic_remap_file_range_prep as-is, and add a new
dax_remap_file_range_prep, both sharing a low-level
__generic_remap_file_range_prep implementation?  And for that
implementation I'd also go for classic if/else instead of the function
pointer.

> +extern int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
> +					 struct inode *dest, loff_t destoff,
> +					 loff_t len, bool *is_same);

no need for the extern.
Ruan Shiyang Feb. 9, 2021, 9:15 a.m. UTC | #2
On 2021/2/8 下午11:19, Christoph Hellwig wrote:
> On Mon, Feb 08, 2021 at 01:09:22AM +0800, Shiyang Ruan wrote:
>> With dax we cannot deal with readpage() etc. So, we create a
>> funciton callback to perform the file data comparison and pass
> 
> s/funciton/function/g
> 
>> +#define MIN(a, b) (((a) < (b)) ? (a) : (b))
> 
> This should use the existing min or min_t helpers.
> 
> 
>>   int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in,
>>   				  struct file *file_out, loff_t pos_out,
>> -				  loff_t *len, unsigned int remap_flags)
>> +				  loff_t *len, unsigned int remap_flags,
>> +				  compare_range_t compare_range_fn)
> 
> Can we keep generic_remap_file_range_prep as-is, and add a new
> dax_remap_file_range_prep, both sharing a low-level
> __generic_remap_file_range_prep implementation?  And for that
> implementation I'd also go for classic if/else instead of the function
> pointer.

The dax dedupe comparison need the iomap_ops pointer as argument, so my 
understanding is that we don't modify the argument list of 
generic_remap_file_range_prep(), but move its code into 
__generic_remap_file_range_prep() whose argument list can be modified to 
accepts the iomap_ops pointer.  Then it looks like this:

```
int dax_remap_file_range_prep(struct file *file_in, loff_t pos_in,
				  struct file *file_out, loff_t pos_out,
				  loff_t *len, unsigned int remap_flags,
				  const struct iomap_ops *ops)
{
	return __generic_remap_file_range_prep(file_in, pos_in, file_out,
			pos_out, len, remap_flags, ops);
}
EXPORT_SYMBOL(dax_remap_file_range_prep);

int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in,
				  struct file *file_out, loff_t pos_out,
				  loff_t *len, unsigned int remap_flags)
{
	return __generic_remap_file_range_prep(file_in, pos_in, file_out,
			pos_out, len, remap_flags, NULL);
}
EXPORT_SYMBOL(generic_remap_file_range_prep);
```

Am i right?


--
Thanks,
Ruan Shiyang.

> 
>> +extern int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
>> +					 struct inode *dest, loff_t destoff,
>> +					 loff_t len, bool *is_same);
> 
> no need for the extern.
> 
>
Christoph Hellwig Feb. 9, 2021, 9:34 a.m. UTC | #3
On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote:
> The dax dedupe comparison need the iomap_ops pointer as argument, so my 
> understanding is that we don't modify the argument list of 
> generic_remap_file_range_prep(), but move its code into 
> __generic_remap_file_range_prep() whose argument list can be modified to 
> accepts the iomap_ops pointer.  Then it looks like this:

I'd say just add the iomap_ops pointer to
generic_remap_file_range_prep and do away with the extra wrappers.  We
only have three callers anyway.
Ruan Shiyang Feb. 9, 2021, 9:46 a.m. UTC | #4
On 2021/2/9 下午5:34, Christoph Hellwig wrote:
> On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote:
>> The dax dedupe comparison need the iomap_ops pointer as argument, so my
>> understanding is that we don't modify the argument list of
>> generic_remap_file_range_prep(), but move its code into
>> __generic_remap_file_range_prep() whose argument list can be modified to
>> accepts the iomap_ops pointer.  Then it looks like this:
> 
> I'd say just add the iomap_ops pointer to
> generic_remap_file_range_prep and do away with the extra wrappers.  We
> only have three callers anyway.

OK.


--
Thanks,
Ruan Shiyang.
> 
>
Christoph Hellwig Feb. 10, 2021, 1:19 p.m. UTC | #5
On Tue, Feb 09, 2021 at 05:46:13PM +0800, Ruan Shiyang wrote:
>
>
> On 2021/2/9 下午5:34, Christoph Hellwig wrote:
>> On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote:
>>> The dax dedupe comparison need the iomap_ops pointer as argument, so my
>>> understanding is that we don't modify the argument list of
>>> generic_remap_file_range_prep(), but move its code into
>>> __generic_remap_file_range_prep() whose argument list can be modified to
>>> accepts the iomap_ops pointer.  Then it looks like this:
>>
>> I'd say just add the iomap_ops pointer to
>> generic_remap_file_range_prep and do away with the extra wrappers.  We
>> only have three callers anyway.
>
> OK.

So looking at this again I think your proposal actaully is better,
given that the iomap variant is still DAX specific.  Sorry for
the noise.

Also I think dax_file_range_compare should use iomap_apply instead
of open coding it.
Ruan Shiyang Feb. 17, 2021, 3:24 a.m. UTC | #6
On 2021/2/10 下午9:19, Christoph Hellwig wrote:
> On Tue, Feb 09, 2021 at 05:46:13PM +0800, Ruan Shiyang wrote:
>>
>>
>> On 2021/2/9 下午5:34, Christoph Hellwig wrote:
>>> On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote:
>>>> The dax dedupe comparison need the iomap_ops pointer as argument, so my
>>>> understanding is that we don't modify the argument list of
>>>> generic_remap_file_range_prep(), but move its code into
>>>> __generic_remap_file_range_prep() whose argument list can be modified to
>>>> accepts the iomap_ops pointer.  Then it looks like this:
>>>
>>> I'd say just add the iomap_ops pointer to
>>> generic_remap_file_range_prep and do away with the extra wrappers.  We
>>> only have three callers anyway.
>>
>> OK.
> 
> So looking at this again I think your proposal actaully is better,
> given that the iomap variant is still DAX specific.  Sorry for
> the noise.
> 
> Also I think dax_file_range_compare should use iomap_apply instead
> of open coding it.
> 

There are two files, which are not reflinked, need to be direct_access() 
here.  The iomap_apply() can handle one file each time.  So, it seems 
that iomap_apply() is not suitable for this case...


The pseudo code of this process is as follows:

   srclen = ops->begin(&srcmap)
   destlen = ops->begin(&destmap)

   direct_access(&srcmap, &saddr)
   direct_access(&destmap, &daddr)

   same = memcpy(saddr, daddr, min(srclen,destlen))

   ops->end(&destmap)
   ops->end(&srcmap)

I think a nested call like this is necessary.  That's why I use the open 
code way.


--
Thanks,
Ruan Shiyang.
>
Darrick J. Wong Feb. 18, 2021, 4:20 p.m. UTC | #7
On Wed, Feb 17, 2021 at 11:24:18AM +0800, Ruan Shiyang wrote:
> 
> 
> On 2021/2/10 下午9:19, Christoph Hellwig wrote:
> > On Tue, Feb 09, 2021 at 05:46:13PM +0800, Ruan Shiyang wrote:
> > > 
> > > 
> > > On 2021/2/9 下午5:34, Christoph Hellwig wrote:
> > > > On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote:
> > > > > The dax dedupe comparison need the iomap_ops pointer as argument, so my
> > > > > understanding is that we don't modify the argument list of
> > > > > generic_remap_file_range_prep(), but move its code into
> > > > > __generic_remap_file_range_prep() whose argument list can be modified to
> > > > > accepts the iomap_ops pointer.  Then it looks like this:
> > > > 
> > > > I'd say just add the iomap_ops pointer to
> > > > generic_remap_file_range_prep and do away with the extra wrappers.  We
> > > > only have three callers anyway.
> > > 
> > > OK.
> > 
> > So looking at this again I think your proposal actaully is better,
> > given that the iomap variant is still DAX specific.  Sorry for
> > the noise.
> > 
> > Also I think dax_file_range_compare should use iomap_apply instead
> > of open coding it.
> > 
> 
> There are two files, which are not reflinked, need to be direct_access()
> here.  The iomap_apply() can handle one file each time.  So, it seems that
> iomap_apply() is not suitable for this case...
> 
> 
> The pseudo code of this process is as follows:
> 
>   srclen = ops->begin(&srcmap)
>   destlen = ops->begin(&destmap)
> 
>   direct_access(&srcmap, &saddr)
>   direct_access(&destmap, &daddr)
> 
>   same = memcpy(saddr, daddr, min(srclen,destlen))
> 
>   ops->end(&destmap)
>   ops->end(&srcmap)
> 
> I think a nested call like this is necessary.  That's why I use the open
> code way.

This might be a good place to implement an iomap_apply2() loop that
actually /does/ walk all the extents of file1 and file2.  There's now
two users of this idiom.

(Possibly structured as a "get next mappings from both" generator
function like Matthew Wilcox keeps asking for. :))

--D

> 
> --
> Thanks,
> Ruan Shiyang.
> > 
> 
>
Christoph Hellwig Feb. 25, 2021, 7:35 a.m. UTC | #8
On Thu, Feb 18, 2021 at 08:20:18AM -0800, Darrick J. Wong wrote:
> > I think a nested call like this is necessary.  That's why I use the open
> > code way.
> 
> This might be a good place to implement an iomap_apply2() loop that
> actually /does/ walk all the extents of file1 and file2.  There's now
> two users of this idiom.

Why do we need a special helper for that?

> (Possibly structured as a "get next mappings from both" generator
> function like Matthew Wilcox keeps asking for. :))

OTOH this might be a good first use for that.
diff mbox series

Patch

diff --git a/fs/btrfs/reflink.c b/fs/btrfs/reflink.c
index 99aa87c08912..efe094f7f748 100644
--- a/fs/btrfs/reflink.c
+++ b/fs/btrfs/reflink.c
@@ -783,7 +783,8 @@  static int btrfs_remap_file_range_prep(struct file *file_in, loff_t pos_in,
 		return ret;
 
 	return generic_remap_file_range_prep(file_in, pos_in, file_out, pos_out,
-					    len, remap_flags);
+					    len, remap_flags,
+					    vfs_dedupe_file_range_compare);
 }
 
 loff_t btrfs_remap_file_range(struct file *src_file, loff_t off,
diff --git a/fs/dax.c b/fs/dax.c
index 29698a3d2e37..6e9391845057 100644
--- a/fs/dax.c
+++ b/fs/dax.c
@@ -30,6 +30,8 @@ 
 #define CREATE_TRACE_POINTS
 #include <trace/events/fs_dax.h>
 
+#define MIN(a, b) (((a) < (b)) ? (a) : (b))
+
 static inline unsigned int pe_order(enum page_entry_size pe_size)
 {
 	if (pe_size == PE_SIZE_PTE)
@@ -1827,3 +1829,68 @@  vm_fault_t dax_finish_sync_fault(struct vm_fault *vmf,
 	return dax_insert_pfn_mkwrite(vmf, pfn, order);
 }
 EXPORT_SYMBOL_GPL(dax_finish_sync_fault);
+
+int dax_file_range_compare(struct inode *src, loff_t srcoff, struct inode *dest,
+		loff_t destoff, loff_t len, bool *is_same,
+		const struct iomap_ops *ops)
+{
+	void *saddr, *daddr;
+	struct iomap smap = { 0 };
+	struct iomap dmap = { 0 };
+	bool same = false;
+	loff_t cmp_len;
+	int id, ret = 0;
+
+	id = dax_read_lock();
+	while (len) {
+		ret = ops->iomap_begin(src, srcoff, len, 0, &smap, NULL);
+		if (ret < 0)
+			goto out_src;
+		cmp_len = MIN(len, smap.offset + smap.length - srcoff);
+
+		ret = ops->iomap_begin(dest, destoff, cmp_len, 0, &dmap, NULL);
+		if (ret < 0)
+			goto out_dest;
+		cmp_len = MIN(cmp_len, dmap.offset + dmap.length - destoff);
+
+		if (smap.type == IOMAP_HOLE && dmap.type == IOMAP_HOLE)
+			goto next;
+
+		if (smap.type == IOMAP_HOLE || dmap.type == IOMAP_HOLE) {
+			same = false;
+			goto next;
+		}
+
+		ret = dax_iomap_direct_access(&smap, srcoff,
+			  ALIGN(srcoff + cmp_len, PAGE_SIZE), &saddr, NULL);
+		if (ret < 0)
+			goto out_dest;
+
+		ret = dax_iomap_direct_access(&dmap, destoff,
+			  ALIGN(destoff + cmp_len, PAGE_SIZE), &daddr, NULL);
+		if (ret < 0)
+			goto out_dest;
+
+		same = !memcmp(saddr, daddr, cmp_len);
+		if (!same)
+			break;
+next:
+		len -= cmp_len;
+		srcoff += cmp_len;
+		destoff += cmp_len;
+out_dest:
+		if (ops->iomap_end)
+			ops->iomap_end(dest, destoff, cmp_len, 0, 0, &dmap);
+out_src:
+		if (ops->iomap_end)
+			ops->iomap_end(src, srcoff, len, 0, 0, &smap);
+
+		if (ret < 0)
+			goto out;
+	}
+	*is_same = same;
+out:
+	dax_read_unlock(id);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(dax_file_range_compare);
diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
index 85979e2214b3..9d101f129d16 100644
--- a/fs/ocfs2/file.c
+++ b/fs/ocfs2/file.c
@@ -2591,7 +2591,7 @@  static loff_t ocfs2_remap_file_range(struct file *file_in, loff_t pos_in,
 		goto out_unlock;
 
 	ret = generic_remap_file_range_prep(file_in, pos_in, file_out, pos_out,
-			&len, remap_flags);
+			&len, remap_flags, vfs_dedupe_file_range_compare);
 	if (ret < 0 || len == 0)
 		goto out_unlock;
 
diff --git a/fs/remap_range.c b/fs/remap_range.c
index e6099beefa97..fc64f7e2af5a 100644
--- a/fs/remap_range.c
+++ b/fs/remap_range.c
@@ -199,9 +199,9 @@  static void vfs_unlock_two_pages(struct page *page1, struct page *page2)
  * Compare extents of two files to see if they are the same.
  * Caller must have locked both inodes to prevent write races.
  */
-static int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
-					 struct inode *dest, loff_t destoff,
-					 loff_t len, bool *is_same)
+int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
+				  struct inode *dest, loff_t destoff,
+				  loff_t len, bool *is_same)
 {
 	loff_t src_poff;
 	loff_t dest_poff;
@@ -280,6 +280,7 @@  static int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
 out_error:
 	return error;
 }
+EXPORT_SYMBOL(vfs_dedupe_file_range_compare);
 
 /*
  * Check that the two inodes are eligible for cloning, the ranges make
@@ -291,7 +292,8 @@  static int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
  */
 int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in,
 				  struct file *file_out, loff_t pos_out,
-				  loff_t *len, unsigned int remap_flags)
+				  loff_t *len, unsigned int remap_flags,
+				  compare_range_t compare_range_fn)
 {
 	struct inode *inode_in = file_inode(file_in);
 	struct inode *inode_out = file_inode(file_out);
@@ -351,8 +353,8 @@  int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in,
 	if (remap_flags & REMAP_FILE_DEDUP) {
 		bool		is_same = false;
 
-		ret = vfs_dedupe_file_range_compare(inode_in, pos_in,
-				inode_out, pos_out, *len, &is_same);
+		ret = compare_range_fn(inode_in, pos_in, inode_out, pos_out,
+				       *len, &is_same);
 		if (ret)
 			return ret;
 		if (!is_same)
diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c
index 6fa05fb78189..26708c01fd78 100644
--- a/fs/xfs/xfs_reflink.c
+++ b/fs/xfs/xfs_reflink.c
@@ -1309,7 +1309,7 @@  xfs_reflink_remap_prep(
 		goto out_unlock;
 
 	ret = generic_remap_file_range_prep(file_in, pos_in, file_out, pos_out,
-			len, remap_flags);
+			len, remap_flags, vfs_dedupe_file_range_compare);
 	if (ret || *len == 0)
 		goto out_unlock;
 
diff --git a/include/linux/dax.h b/include/linux/dax.h
index b52f084aa643..183e084c2ac7 100644
--- a/include/linux/dax.h
+++ b/include/linux/dax.h
@@ -238,6 +238,11 @@  int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index);
 int dax_invalidate_mapping_entry_sync(struct address_space *mapping,
 				      pgoff_t index);
 s64 dax_iomap_zero(loff_t pos, u64 length, struct iomap *iomap);
+int dax_file_range_compare(struct inode *src, loff_t srcoff,
+			   struct inode *dest, loff_t destoff,
+			   loff_t len, bool *is_same,
+			   const struct iomap_ops *ops);
+
 static inline bool dax_mapping(struct address_space *mapping)
 {
 	return mapping->host && IS_DAX(mapping->host);
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 8667d0cdc71e..159ec755e47b 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1912,13 +1912,20 @@  extern ssize_t vfs_read(struct file *, char __user *, size_t, loff_t *);
 extern ssize_t vfs_write(struct file *, const char __user *, size_t, loff_t *);
 extern ssize_t vfs_copy_file_range(struct file *, loff_t , struct file *,
 				   loff_t, size_t, unsigned int);
+extern int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
+					 struct inode *dest, loff_t destoff,
+					 loff_t len, bool *is_same);
+typedef int (*compare_range_t)(struct inode *src, loff_t srcpos,
+			       struct inode *dest, loff_t destpos,
+			       loff_t len, bool *is_same);
 extern ssize_t generic_copy_file_range(struct file *file_in, loff_t pos_in,
 				       struct file *file_out, loff_t pos_out,
 				       size_t len, unsigned int flags);
 extern int generic_remap_file_range_prep(struct file *file_in, loff_t pos_in,
 					 struct file *file_out, loff_t pos_out,
 					 loff_t *count,
-					 unsigned int remap_flags);
+					 unsigned int remap_flags,
+					 compare_range_t compare_range_fn);
 extern loff_t do_clone_file_range(struct file *file_in, loff_t pos_in,
 				  struct file *file_out, loff_t pos_out,
 				  loff_t len, unsigned int remap_flags);