Incremental btrfs receive in opposite direction fails
diff mbox

Message ID 1TpCWu-000Fz2-6t@internal.tormail.org
State New, archived
Headers show

Commit Message

junlion@tormail.org Dec. 30, 2012, 6:40 a.m. UTC
On 2012-12-29 15:00 +0200, Alex Lyakas wrote:
> There is no special repo, but you may want, in addition to the patch
> you mentioned, apply this one as well:
> https://patchwork.kernel.org/patch/1604391/

Very useful! Somehow a few lines got wrapped though.

> > Is there a way for me to directly change the received_uuid of
> > /mnt/bak/.snap to make it identical to the UUID of /.snap? This looks
> > like the easiest way if I only need to do it once.
> There is no implemented way, but since you debugged this far, you can
> put up some code that sends BTRFS_IOC_SET_RECEIVED_SUBVOL, which is
> the one setting the received_uuid (and some other small stuff, please
> check the kernel code for it).

Ah, BTRFS_IOC_SET_RECEIVED_SUBVOL. Thanks for pointing me in the right
direction. After some hacking, it worked without eating my data... These
are the two commands to run before the failing one:

btrfs subvolume snapshot /mnt/bak/.snap /mnt/bak/.snap-mangled
uu / .snap /mnt/bak/.snap-mangled

... where uu is my crude little received_uuid and stransid display and
adjustment tool. I've attached its source.

Would be cool if someone modified btrfs send/receive to handle this use
case automatically, but it's too difficult for me.

Comments

Jan Schmidt Jan. 2, 2013, 4:57 p.m. UTC | #1
Hi,

I admit I haven't completely understood what you're trying to achieve. You can
only receive an incremental stream if the internal (!) file system state on the
receiver's side is the same as on the sender's. Thus I don't see where setting a
uuid to trick btrfs receive could do you any good.

On Sun, December 30, 2012 at 07:40 (+0100), junlion wrote:
> On 2012-12-29 15:00 +0200, Alex Lyakas wrote:
>> There is no special repo, but you may want, in addition to the patch
>> you mentioned, apply this one as well:
>> https://patchwork.kernel.org/patch/1604391/
> 
> Very useful! Somehow a few lines got wrapped though.
> 
>>> Is there a way for me to directly change the received_uuid of
>>> /mnt/bak/.snap to make it identical to the UUID of /.snap? This looks
>>> like the easiest way if I only need to do it once.
>> There is no implemented way, but since you debugged this far, you can
>> put up some code that sends BTRFS_IOC_SET_RECEIVED_SUBVOL, which is
>> the one setting the received_uuid (and some other small stuff, please
>> check the kernel code for it).
> 
> Ah, BTRFS_IOC_SET_RECEIVED_SUBVOL. Thanks for pointing me in the right
> direction. After some hacking, it worked without eating my data... These
> are the two commands to run before the failing one:
> 
> btrfs subvolume snapshot /mnt/bak/.snap /mnt/bak/.snap-mangled
> uu / .snap /mnt/bak/.snap-mangled
> 
> ... where uu is my crude little received_uuid and stransid display and
> adjustment tool. I've attached its source.

Still in doubt, I'd like to recommend checking with Arne's fssum tool if both
file systems are really the same now. You can fetch and build from his repository at

	git://git.kernel.org/pub/scm/linux/kernel/git/arne/far-progs.git

As a general rule, please send patches inline for people to quote them when
commenting.

> Would be cool if someone modified btrfs send/receive to handle this use
> case automatically, but it's too difficult for me.

We could completely drop the check leading to the "could not find parent
subvolume" error message, if someone explained why removing it was sensible.

Looking forward to your explanation!
-Jan
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
junlion@tormail.org Jan. 2, 2013, 8:56 p.m. UTC | #2
Hi,

> I admit I haven't completely understood what you're trying to achieve. You can
> only receive an incremental stream if the internal (!) file system state on the
> receiver's side is the same as on the sender's.

Understood, and the internal states are identical, except for the
snapshot metadata.

This executable example that shows the problem:


    # Create two throwaway filesystems /tmp/img1 and /tmp/img2
    # and mount them at /tmp/mp1 and /tmp/mp2
    for i in 1 2; do
        truncate -s 1G /tmp/img$i
        loop=`losetup -f --show /tmp/img$i`
        mkfs.btrfs "$loop"
        losetup -d "$loop"
        mkdir /tmp/mp$i
        mount /tmp/img$i /tmp/mp$i
    done

    # Create a subvol on the first fs, populate it
    btrfs sub create /tmp/mp1/foo
    echo 1 >/tmp/mp1/foo/1

    # Make an ro snapshot of the subvol, transfer it to the 2nd fs
    btrfs sub snap -r /tmp/mp1/foo /tmp/mp1/foo.1
    btrfs send /tmp/mp1/foo.1 | btrfs receive -v /tmp/mp2

    # Make an rw snapshot of the transferred snapshot on the 2nd fs,
    # add something to this rw snapshot
    btrfs sub snap /tmp/mp2/foo.1 /tmp/mp2/foo
    echo 2 >/tmp/mp2/foo/2

    # Make an ro snapshot of the added-to snapshot
    btrfs sub snap -r /tmp/mp2/foo /tmp/mp2/foo.2

    # Now try to transfer it back in the other direction
    btrfs send -p /tmp/mp2/foo.1 /tmp/mp2/foo.2 |
    btrfs receive -v /tmp/mp1
    # "ERROR: could not find parent subvolume"
    # even though both fs have the same foo.1


As I understand it, if "clone" is a clone/parent source, btrfs send
transmits
    clone.uuid
    clone.ctransid
and btrfs receive searches for a subvol "sub" with
    sub.received_uuid == clone.uuid and sub.stransid == clone.ctransid.

But because the direction of the transfer has changed, it can't find
anything. It would work if btrfs send was modified to *additionally*
transmit
    clone.received_uuid
    clone.stransid
and btrfs receive to do a *fallback* search for a subvol with
    sub.uuid == clone.received_uuid and sub.ctransid == clone.stransid

However, since I'm really not familiar with btrfs code, it was easier
for me to just make a rw snapshot of mp1/foo.1 and modify its metadata:

    # can't modify the info about a ro snapshot
    btrfs sub snap /tmp/mp1/foo.1 /tmp/mp1/foo.1rw

    # mp1/foo.1rw.received_uuid := mp2/foo.1.uuid
    # mp1/foo.1rw.stransid      := mp2/foo.1.ctransid
    uu /tmp/mp2 foo.1 /tmp/mp1/foo.1rw

    # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    # !! Now we've made it look as if mp1/foo.1 was received from !!
    # !! mp2/foo.1 when actually it was the other way around.     !!
    # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    
    # Let's retry the original problematic command:
    btrfs send -p /tmp/mp2/foo.1 /tmp/mp2/foo.2 |
    btrfs receive -v /tmp/mp1

    # It works fine! Hurray!
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
junlion@tormail.org Jan. 2, 2013, 10:19 p.m. UTC | #3
>     # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>     # !! Now we've made it look as if mp1/foo.1 was received from !!
>     # !! mp2/foo.1 when actually it was the other way around.     !!
>     # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Better just ignore what I wrote there, it's not completely correct.
(Because we do not modify mp1/foo.1 but snapshot it to mp1/foo.1rw
and modify that one.)
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch
diff mbox

diff --git a/cmds-receive.c b/cmds-receive.c
index a8be6fa..40d2e48 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -52,11 +52,13 @@  static int g_verbose = 0;
 struct btrfs_receive
 {
 	int mnt_fd;
+	int dest_dir_fd;
 
 	int write_fd;
 	char *write_path;
 
 	char *root_path;
+	char *dest_dir_path; /* relative to root_path */
 	char *full_subvol_path;
 
 	struct subvol_info *cur_subvol;
@@ -150,8 +152,11 @@  static int process_subvol(const char *path, const u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;
 
-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);
 
 	fprintf(stderr, "At subvol %s\n", path);
 
@@ -167,7 +172,7 @@  static int process_subvol(const char *path, const u8 *uuid, u64 ctransid,
 
 	memset(&args_v1, 0, sizeof(args_v1));
 	strcpy(args_v1.name, path);
-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
 	if (ret < 0) {
 		ret = -errno;
 		fprintf(stderr, "ERROR: creating subvolume %s failed. "
@@ -195,8 +200,11 @@  static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;
 
-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);
 
 	fprintf(stderr, "At snapshot %s\n", path);
 
@@ -243,7 +251,7 @@  static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,
 		goto out;
 	}
 
-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
 	close(args_v2.fd);
 	if (ret < 0) {
 		ret = -errno;
@@ -790,17 +798,48 @@  struct btrfs_send_ops send_ops = {
 int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd)
 {
 	int ret;
+	char *dest_dir_full_path;
 	int end = 0;
 
-	r->root_path = strdup(tomnt);
-	r->mnt_fd = open(tomnt, O_RDONLY | O_NOATIME);
+	dest_dir_full_path = realpath(tomnt, NULL);
+	if (!dest_dir_full_path) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: realpath(%s) failed. %s\n", tomnt,
+				strerror(-ret));
+		goto out;
+	}
+	r->dest_dir_fd = open(dest_dir_full_path, O_RDONLY | O_NOATIME);
+	if (r->dest_dir_fd < 0) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: failed to open destination directory %s. %s\n",
+			    dest_dir_full_path, strerror(-ret));
+		goto out;
+	}
+
+	ret = find_mount_root(dest_dir_full_path, &r->root_path);
+	if (ret < 0) {
+		ret = -EINVAL;
+		fprintf(stderr, "ERROR: failed to determine mount point "
+				"for %s\n", dest_dir_full_path);
+		goto out;
+	}
+	r->mnt_fd = open(r->root_path, O_RDONLY | O_NOATIME);
 	if (r->mnt_fd < 0) {
 		ret = -errno;
-		fprintf(stderr, "ERROR: failed to open %s. %s\n", tomnt,
+		fprintf(stderr, "ERROR: failed to open %s. %s\n", r->root_path,
 				strerror(-ret));
 		goto out;
 	}
 
+	/*
+	 * find_mount_root returns a root_path that is a subpath of
+	 * dest_dir_full_path. Now get the other part of root_path,
+	 * which is the destination dir relative to root_path.
+	 */
+	r->dest_dir_path = dest_dir_full_path + strlen(r->root_path);
+	if (r->dest_dir_path[0] == '/')
+		r->dest_dir_path++;
+
 	ret = subvol_uuid_search_init(r->mnt_fd, &r->sus);
 	if (ret < 0)
 		return ret;
diff --git a/cmds-send.c b/cmds-send.c
index 9b47e70..c408bc7 100644
--- a/cmds-send.c
+++ b/cmds-send.c
@@ -81,6 +81,14 @@  int find_mount_root(const char *path, char **mount_root)
 		}
 	}
 
+	if (!longest_match) {
+		fprintf(stderr, "ERROR: Failed to find mount root for path %s.\n",
+			    path);
+		fprintf(stderr, "Please make sure that you have a valid \
+			/etc/mtab file.\n");
+		return -ENOENT;
+	}
+
 	*mount_root = realpath(longest_match, NULL);
 	free(longest_match);
 
diff --git a/send-utils.h b/send-utils.h
index da407eb..a3e038b 100644
--- a/send-utils.h
+++ b/send-utils.h
@@ -65,5 +65,6 @@  void subvol_uuid_search_add(struct subvol_uuid_search *s,
 char *path_cat(const char *p1, const char *p2);
 char *path_cat3(const char *p1, const char *p2, const char *p3);
 
+int find_mount_root(const char *path, char **mount_root);
 
 #endif /* SEND_UTILS_H_ */