From patchwork Thu Apr 29 09:06:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 12230841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1B67C433B4 for ; Thu, 29 Apr 2021 09:07:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6F74E61441 for ; Thu, 29 Apr 2021 09:07:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233500AbhD2JHx (ORCPT ); Thu, 29 Apr 2021 05:07:53 -0400 Received: from mx2.suse.de ([195.135.220.15]:41356 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232734AbhD2JHv (ORCPT ); Thu, 29 Apr 2021 05:07:51 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1619687224; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/RoVjMJByJv/KChToVgXY+iQb7XvM269HatuufiwXlY=; b=YktXvNBXRHFPPyt03iKe4k1ReQyIfuOYPZN1+T3wfzxFmvV+0r6c01j48HyFGDWRPIkH6i BotCdnn4D9KOvTYy6Bhxv1mqpYazm1ZwKJze4sfz2fTW25qJblztdVoP0kyud01T3gMJpL k00wZfvifjQdrNrMM4Zkf2EQ8Sfr37s= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id C6BFCAF83; Thu, 29 Apr 2021 09:07:04 +0000 (UTC) From: Qu Wenruo To: linux-btrfs@vger.kernel.org Cc: Su Yue Subject: [PATCH v2 1/3] btrfs-progs: image: remove the dead stat() call Date: Thu, 29 Apr 2021 17:06:56 +0800 Message-Id: <20210429090658.245238-2-wqu@suse.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210429090658.245238-1-wqu@suse.com> References: <20210429090658.245238-1-wqu@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org In restore_metadump(), we call stat() but never uses the result get. This call site is left by some code refactor, as the stat() call is now moved into fixup_device_size(). So we can safely remove the call. Signed-off-by: Qu Wenruo Reviewed-by: Su Yue --- image/main.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/image/main.c b/image/main.c index 48070e52c21f..24393188e5e3 100644 --- a/image/main.c +++ b/image/main.c @@ -2690,7 +2690,6 @@ static int restore_metadump(const char *input, FILE *out, int old_restore, if (!ret && !multi_devices && !old_restore && btrfs_super_num_devices(mdrestore.original_super) != 1) { struct btrfs_root *root; - struct stat st; root = open_ctree_fd(fileno(out), target, 0, OPEN_CTREE_PARTIAL | @@ -2703,13 +2702,6 @@ static int restore_metadump(const char *input, FILE *out, int old_restore, } info = root->fs_info; - if (stat(target, &st)) { - error("stat %s failed: %m", target); - close_ctree(info->chunk_root); - free(cluster); - return 1; - } - ret = fixup_chunks_and_devices(info, &mdrestore, fileno(out)); close_ctree(info->chunk_root); if (ret) From patchwork Thu Apr 29 09:06:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 12230843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CAC1C433B4 for ; Thu, 29 Apr 2021 09:07:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E4ED261449 for ; Thu, 29 Apr 2021 09:07:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232734AbhD2JHz (ORCPT ); Thu, 29 Apr 2021 05:07:55 -0400 Received: from mx2.suse.de ([195.135.220.15]:41374 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240062AbhD2JHy (ORCPT ); Thu, 29 Apr 2021 05:07:54 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1619687227; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cQ2ngzZaVaRwNLM06AwozGCjJj+HKkzIZX1xPnT9UBg=; b=MfMnqYcc5sEhpNIBmcDTicJgbi3mr7Xds+7waZTeWwxRIbGywh04eV47+oKnc9fffS7Ofx TGULoWz9D5Y8MAD8WBBA88zBEO+JeedKDZMawAAo9tGh1EoJcXGn/ihYNUMXMIA9QhAklv AFlFQbscUIkW6r6bM5YOso8ru8GTjn4= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 736FEAE58; Thu, 29 Apr 2021 09:07:07 +0000 (UTC) From: Qu Wenruo To: linux-btrfs@vger.kernel.org Cc: Nikolay Borisov , Su Yue Subject: [PATCH v2 2/3] btrfs-progs: image: enlarge the output file if no tree modification is needed for restore Date: Thu, 29 Apr 2021 17:06:57 +0800 Message-Id: <20210429090658.245238-3-wqu@suse.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210429090658.245238-1-wqu@suse.com> References: <20210429090658.245238-1-wqu@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org [BUG] If restoring dumpped image into a new file, under most cases kernel will reject it: # mkfs.btrfs -f /dev/test/test # btrfs-image /dev/test/test /tmp/dump # btrfs-image -r /tmp/dump ~/test.img # mount ~/test.img /mnt/btrfs mount: /mnt/btrfs: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error. # dmesg -t | tail -n 7 loop0: detected capacity change from 10592 to 0 BTRFS info (device loop0): disk space caching is enabled BTRFS info (device loop0): has skinny extents BTRFS info (device loop0): flagging fs with big metadata feature BTRFS error (device loop0): device total_bytes should be at most 5423104 but found 10737418240 BTRFS error (device loop0): failed to read chunk tree: -22 BTRFS error (device loop0): open_ctree failed [CAUSE] When btrfs-image restores an image into a file, and the source image contains only single device, then we don't need to modify the chunk/device tree, as we can reuse the existing chunk/dev tree without any problem. This also means, for such restore, we also won't do any target file enlarge. This behavior itself is fine, as at that time, kernel won't check if the device is smaller than the device size recorded in device tree. But later kernel commit 3a160a933111 ("btrfs: drop never met disk total bytes check in verify_one_dev_extent") introduces new check on device size at mount time, rejecting any loop file which is smaller than the original device size. [FIX] Do extra file enlarge for single device restore if the restored file is smaller than the device size. Reported-by: Nikolay Borisov Signed-off-by: Qu Wenruo Reviewed-by: Su Yue --- image/main.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/image/main.c b/image/main.c index 24393188e5e3..188f387b3354 100644 --- a/image/main.c +++ b/image/main.c @@ -2706,6 +2706,49 @@ static int restore_metadump(const char *input, FILE *out, int old_restore, close_ctree(info->chunk_root); if (ret) goto out; + } else { + struct btrfs_root *root; + struct stat st; + u64 dev_size; + + if (!info) { + root = open_ctree_fd(fileno(out), target, 0, 0); + if (!root) { + error("open ctree failed in %s", target); + ret = -EIO; + goto out; + } + + info = root->fs_info; + + dev_size = btrfs_stack_device_total_bytes( + &info->super_copy->dev_item); + close_ctree(root); + info = NULL; + } else { + dev_size = btrfs_stack_device_total_bytes( + &info->super_copy->dev_item); + } + + /* + * We don't need extra tree modification, but if the output is + * a file, we need to enlarge the output file so that + * newer kernel won't report error. + */ + ret = fstat(fileno(out), &st); + if (ret < 0) { + error("failed to stat result image: %m"); + ret = -errno; + goto out; + } + if (S_ISREG(st.st_mode) && st.st_size < dev_size) { + ret = ftruncate64(fileno(out), dev_size); + if (ret < 0) { + error("failed to enlarge result image: %m"); + ret = -errno; + goto out; + } + } } out: mdrestore_destroy(&mdrestore, num_threads); From patchwork Thu Apr 29 09:06:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 12230845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1319C433B4 for ; Thu, 29 Apr 2021 09:07:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6D1886144E for ; Thu, 29 Apr 2021 09:07:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239984AbhD2JH6 (ORCPT ); Thu, 29 Apr 2021 05:07:58 -0400 Received: from mx2.suse.de ([195.135.220.15]:41394 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240148AbhD2JH4 (ORCPT ); Thu, 29 Apr 2021 05:07:56 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1619687229; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JGMFhLDvA8Q3WfafxCnO2/WWDNBUw2lX4Hai6PgD1Ug=; b=kZviSmJjDqnDdtT+JTayT3g9n8o9hRBSnRQa+byIGan99F8AlYho1De/hzTSYpVYl3J0A/ hAasCr9rhxSwA925CiLtzQOsf9ADqclmdDFkUudlNXQjkCHNSyaureT0JjFbMRL6vsfBSt ohG3StRGNjvd2TXM+DgF8PtFqe/l/cw= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 606B0AF83 for ; Thu, 29 Apr 2021 09:07:09 +0000 (UTC) From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH v2 3/3] btrfs-progs: misc-tests: add test to ensure the restored image can be mounted Date: Thu, 29 Apr 2021 17:06:58 +0800 Message-Id: <20210429090658.245238-4-wqu@suse.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210429090658.245238-1-wqu@suse.com> References: <20210429090658.245238-1-wqu@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This new test case is to make sure the restored image file has been properly enlarged so that newer kernel won't complain. Signed-off-by: Qu Wenruo --- .../048-image-restore-mount/test.sh | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100755 tests/misc-tests/048-image-restore-mount/test.sh diff --git a/tests/misc-tests/048-image-restore-mount/test.sh b/tests/misc-tests/048-image-restore-mount/test.sh new file mode 100755 index 000000000000..b61b7a7188cf --- /dev/null +++ b/tests/misc-tests/048-image-restore-mount/test.sh @@ -0,0 +1,20 @@ +#!/bin/bash +# Verify that the restored image of an empty btrfs filesystem can still be +# mounted + +source "$TEST_TOP/common" + +check_prereq btrfs-image +check_prereq mkfs.btrfs +check_prereq btrfs + +tmp=$(mktemp -d --tmpdir btrfs-progs-image.XXXXXXXX) +prepare_test_dev + +run_check_mkfs_test_dev +run_check "$TOP/btrfs-image" "$TEST_DEV" "$tmp/dump" +run_check "$TOP/btrfs-image" -r "$tmp/dump" "$tmp/restored" + +run_check $SUDO_HELPER mount -t btrfs -o loop "$tmp/restored" "$TEST_MNT" +umount "$TEST_MNT" &> /dev/null +rm -rf -- "$tmp"