From patchwork Fri Aug 22 06:43:45 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gui Hecheng X-Patchwork-Id: 4761361 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 4837F9F383 for ; Fri, 22 Aug 2014 06:44:31 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 8164C2018A for ; Fri, 22 Aug 2014 06:44:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id ACE682017A for ; Fri, 22 Aug 2014 06:44:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755649AbaHVGoX (ORCPT ); Fri, 22 Aug 2014 02:44:23 -0400 Received: from cn.fujitsu.com ([59.151.112.132]:29581 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1752937AbaHVGoW (ORCPT ); Fri, 22 Aug 2014 02:44:22 -0400 X-IronPort-AV: E=Sophos;i="5.04,378,1406563200"; d="scan'208";a="34933751" Received: from localhost (HELO edo.cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 22 Aug 2014 14:41:06 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (localhost.localdomain [127.0.0.1]) by edo.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id s7M6hvPZ016518; Fri, 22 Aug 2014 14:43:57 +0800 Received: from [10.167.226.111] (10.167.226.111) by G08CNEXCHPEKD01.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server id 14.3.181.6; Fri, 22 Aug 2014 14:44:00 +0800 Message-ID: <1408689825.22226.14.camel@localhost.localdomain> Subject: Re: btrfs restore memory corruption (bug: 82701) From: Gui Hecheng To: Marc Dietrich CC: linux-btrfs Date: Fri, 22 Aug 2014 14:43:45 +0800 In-Reply-To: <2196812.uedtk6DxDd@fb07-iapwap2> References: <2058629.ulFxBAG3Lx@fb07-iapwap2> <1408614736.11298.6.camel@localhost.localdomain> <2196812.uedtk6DxDd@fb07-iapwap2> X-Mailer: Evolution 3.6.4 (3.6.4-3.fc18) MIME-Version: 1.0 X-Originating-IP: [10.167.226.111] Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, 2014-08-21 at 16:19 +0200, Marc Dietrich wrote: > Am Donnerstag, 21. August 2014, 17:52:16 schrieb Gui Hecheng: > > On Mon, 2014-08-18 at 11:25 +0200, Marc Dietrich wrote: > > > Hi, > > > > > > I did a checkout of the latest btrfs progs to repair my damaged > > > filesystem. > > > Running btrfs restore gives me several failed to inflate: -6 and crashes > > > with some memory corruption. I ran it again with valgrind and got: > > > > > > valgrind --log-file=x2 -v --leak-check=yes btrfs restore /dev/sda9 > > > /mnt/backup > > > > > > ==8528== Memcheck, a memory error detector > > > ==8528== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al. > > > ==8528== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info > > > ==8528== Command: btrfs restore /dev/sda9 /mnt/backup > > > ==8528== Parent PID: 8453 > > > ==8528== > > > ==8528== Syscall param pwrite64(buf) points to uninitialised byte(s) > > > ==8528== at 0x59BE3C3: __pwrite_nocancel (in /lib64/libpthread-2.18.so) > > > ==8528== by 0x41F22F: search_dir (cmds-restore.c:392) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== Address 0x66956a0 is 7,056 bytes inside a block of size 8,192 > > > alloc'd > > > ==8528== at 0x4C277AB: malloc (in > > > /usr/lib64/valgrind/vgpreload_memcheck- amd64-linux.so) > > > ==8528== by 0x41EEAD: search_dir (cmds-restore.c:316) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > > -------------------[snip]--------------------------------- > > > > > ==8528== Invalid read of size 1 > > > ==8528== at 0x4C2BF15: memcpy@@GLIBC_2.14 (in > > > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > > > ==8528== by 0x43818F: read_extent_buffer (string3.h:51) > > > ==8528== by 0x41EC66: search_dir (cmds-restore.c:233) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== Address 0x684c186 is 1,110 bytes inside a block of size 4,224 > > > free'd ==8528== at 0x4C28ADC: free (in > > > /usr/lib64/valgrind/vgpreload_memcheck- amd64-linux.so) > > > ==8528== by 0x437895: free_extent_buffer (extent_io.c:618) > > > ==8528== by 0x41E053: next_leaf (cmds-restore.c:202) > > > ==8528== by 0x41E50F: search_dir (cmds-restore.c:731) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== > > > ==8528== Invalid read of size 8 > > > ==8528== at 0x4C2BF40: memcpy@@GLIBC_2.14 (in > > > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > > > ==8528== by 0x43818F: read_extent_buffer (string3.h:51) > > > ==8528== by 0x41EC66: search_dir (cmds-restore.c:233) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== Address 0x684c178 is 1,096 bytes inside a block of size 4,224 > > > free'd ==8528== at 0x4C28ADC: free (in > > > /usr/lib64/valgrind/vgpreload_memcheck- amd64-linux.so) > > > ==8528== by 0x437895: free_extent_buffer (extent_io.c:618) > > > ==8528== by 0x41E053: next_leaf (cmds-restore.c:202) > > > ==8528== by 0x41E50F: search_dir (cmds-restore.c:731) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== > > > ==8528== Invalid read of size 8 > > > ==8528== at 0x4C2BF52: memcpy@@GLIBC_2.14 (in > > > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > > > ==8528== by 0x43818F: read_extent_buffer (string3.h:51) > > > ==8528== by 0x41EC66: search_dir (cmds-restore.c:233) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== Address 0x684c168 is 1,080 bytes inside a block of size 4,224 > > > free'd ==8528== at 0x4C28ADC: free (in > > > /usr/lib64/valgrind/vgpreload_memcheck- amd64-linux.so) > > > ==8528== by 0x437895: free_extent_buffer (extent_io.c:618) > > > ==8528== by 0x41E053: next_leaf (cmds-restore.c:202) > > > ==8528== by 0x41E50F: search_dir (cmds-restore.c:731) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== > > > ==8528== Invalid read of size 1 > > > ==8528== at 0x4C2BFE4: memcpy@@GLIBC_2.14 (in > > > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > > > ==8528== by 0x43818F: read_extent_buffer (string3.h:51) > > > ==8528== by 0x41EC66: search_dir (cmds-restore.c:233) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== Address 0x6a385f8 is 2,680 bytes inside a block of size 4,224 > > > free'd ==8528== at 0x4C28ADC: free (in > > > /usr/lib64/valgrind/vgpreload_memcheck- amd64-linux.so) > > > ==8528== by 0x437895: free_extent_buffer (extent_io.c:618) > > > ==8528== by 0x41E053: next_leaf (cmds-restore.c:202) > > > ==8528== by 0x41E50F: search_dir (cmds-restore.c:731) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== > > > > ---------------------------------------------------------- > > For the above piece, > > maybe you would like to try if the following helps or not: > > > > diff --git a/cmds-restore.c b/cmds-restore.c > > index 239ea6c..dde7de8 100644 > > --- a/cmds-restore.c > > +++ b/cmds-restore.c > > @@ -182,6 +182,7 @@ again: > > c = path->nodes[level]; > > if (slot >= btrfs_header_nritems(c)) { > > level++; > > + offset = 1; > > if (level == BTRFS_MAX_LEVEL) > > return 1; > > continue; > > > > it doesn't seems to go the right way when entering the next level, > > it should starts at the first slot at least. > > Can't tell if it's the right thing to do, but at least I haven't seen *this* > leak message for a while now. I think it works a bit, because it shows that you've passed all the stuff with inline extents. > Additionally, I get many of these (unrelated) leaks now: For the leak below... I've no idea why the @decompress_lzo() is not statisfied with @inbuf with the exact size of the disk bytes. Or maybe the compressed data had just sufferred damages... BTW, when you wrote your data, did that kernel has the following commit for btrfs? commit: 59516f6017c589e7316418fda6128ba8f829a77f If *NO*, then you may try the following and see if it makes any difference: --------------------------------------------------------- @@ -376,7 +376,7 @@ again: goto out; } - ret = decompress(inbuf, outbuf, disk_size, &ram_size, compress); + ret = decompress(inbuf, outbuf, num_bytes, &ram_size, compress); if (ret) { num_copies = btrfs_num_copies(&root->fs_info->mapping_tree, bytenr, length); ------------------------------------------------------------------------ *NOTE*: the above is just a trial, it is actually not proper, but please don't worry, it does no harm. -Gui > ==3007== Invalid read of size 1 > ==3007== at 0x57A11B1: lzo1x_decompress_safe (in > /usr/lib64/liblzo2.so.2.0.0) > ==3007== by 0x41E2C4: decompress (cmds-restore.c:122) > ==3007== by 0x41F19D: search_dir (cmds-restore.c:378) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== Address 0x6887774 is 4 bytes after a block of size 4,096 alloc'd > ==3007== at 0x4C277AB: malloc (in /usr/lib64/valgrind/vgpreload_memcheck- > amd64-linux.so) > ==3007== by 0x41EE61: search_dir (cmds-restore.c:309) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > ==3007== by 0x41F8D7: search_dir (cmds-restore.c:895) > > Thanks so far! > > Marc > > > > > ==8528== Invalid read of size 2 > > > ==8528== at 0x4C2BFA0: memcpy@@GLIBC_2.14 (in > > > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > > > ==8528== by 0x43818F: read_extent_buffer (string3.h:51) > > > ==8528== by 0x41EC66: search_dir (cmds-restore.c:233) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== Address 0x6b0bfb8 is 632 bytes inside a block of size 4,224 > > > free'd ==8528== at 0x4C28ADC: free (in > > > /usr/lib64/valgrind/vgpreload_memcheck- amd64-linux.so) > > > ==8528== by 0x437895: free_extent_buffer (extent_io.c:618) > > > ==8528== by 0x4261CA: btrfs_release_path (ctree.c:61) > > > ==8528== by 0x426212: btrfs_free_path (ctree.c:51) > > > ==8528== by 0x41F93B: search_dir (cmds-restore.c:911) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== > > > ==8528== Invalid read of size 2 > > > ==8528== at 0x4C2BFB3: memcpy@@GLIBC_2.14 (in > > > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > > > ==8528== by 0x43818F: read_extent_buffer (string3.h:51) > > > ==8528== by 0x41EC66: search_dir (cmds-restore.c:233) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x4204B8: cmd_restore (cmds-restore.c:1284) > > > ==8528== by 0x4043FE: main (btrfs.c:286) > > > ==8528== Address 0x6b0bfb4 is 628 bytes inside a block of size 4,224 > > > free'd ==8528== at 0x4C28ADC: free (in > > > /usr/lib64/valgrind/vgpreload_memcheck- amd64-linux.so) > > > ==8528== by 0x437895: free_extent_buffer (extent_io.c:618) > > > ==8528== by 0x4261CA: btrfs_release_path (ctree.c:61) > > > ==8528== by 0x426212: btrfs_free_path (ctree.c:51) > > > ==8528== by 0x41F93B: search_dir (cmds-restore.c:911) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== by 0x41F8D0: search_dir (cmds-restore.c:895) > > > ==8528== > > > ==8528== > > > ==8528== HEAP SUMMARY: > > > ==8528== in use at exit: 0 bytes in 0 blocks > > > ==8528== total heap usage: 260,452 allocs, 260,452 frees, 278,189,550 > > > bytes allocated > > > ==8528== > > > ==8528== All heap blocks were freed -- no leaks are possible > > > ==8528== > > > ==8528== For counts of detected and suppressed errors, rerun with: -v > > > ==8528== Use --track-origins=yes to see where uninitialised values come > > > from ==8528== ERROR SUMMARY: 16597 errors from 7 contexts (suppressed: 2 > > > from 2) > > > > > > see: https://bugzilla.kernel.org/show_bug.cgi?id=82701 > > > > > > Marc > > > > > > p.s. > > > > > > I wonder if this list should be autosubscribed to btrfs related bugs > > > > > > -- > > > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > > > the body of a message to majordomo@vger.kernel.org > > > More majordomo info at http://vger.kernel.org/majordomo-info.html --- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/cmds-restore.c b/cmds-restore.c index dde7de8..ae1ea72 100644 --- a/cmds-restore.c +++ b/cmds-restore.c @@ -297,7 +297,7 @@ static int copy_one_extent(struct btrfs_root *root, int fd, ram_size = btrfs_file_extent_ram_bytes(leaf, fi); offset = btrfs_file_extent_offset(leaf, fi); num_bytes = btrfs_file_extent_num_bytes(leaf, fi); - size_left = disk_size; + size_left = num_bytes; if (compress == BTRFS_COMPRESS_NONE) bytenr += offset;