From patchwork Wed Aug 26 19:33:38 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timofey Titovets X-Patchwork-Id: 7079711 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 967FABEEC1 for ; Wed, 26 Aug 2015 19:34:29 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 0D6A8208E2 for ; Wed, 26 Aug 2015 19:34:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 47839208E7 for ; Wed, 26 Aug 2015 19:34:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753468AbbHZTeW (ORCPT ); Wed, 26 Aug 2015 15:34:22 -0400 Received: from mail-la0-f54.google.com ([209.85.215.54]:34245 "EHLO mail-la0-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751647AbbHZTeU (ORCPT ); Wed, 26 Aug 2015 15:34:20 -0400 Received: by laba3 with SMTP id a3so126697781lab.1 for ; Wed, 26 Aug 2015 12:34:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=7Os7FhljWCh1UDCL4kGqbhKM4sF3qc2UsVEfJxxeaRo=; b=Ou+ek2bP+Z80IIKUYk7iPuTGBTLdcGrg9auc9XSBAfQ7bjgMEHzKQqOUYpHuuN3ujQ O4UfnaygjcFP0hLIkxyOX86hntq/UWHRuUDKu/4n2aFUqMFDECKXOjPDe36/L0ieXt7M t4+u12fBLDEXf6TGmBIm6F+r8xbjwCgc+BBAhTRpgiaWUevi+q+43uyh65HAwiXM+2Tn 3YaRAsA5B5wGEn4/EMx58xJrzke1iCzZGBwZA6uUn1cWYy8+/9/BBoBIsA+Zq9bzB47b wLIiUHkVHx1VzVQlF0/96OffKTBFo8EEXlu58zEz8f1VIMFYJpS3qVjNQdyW13JGP/Jv xxlg== X-Received: by 10.112.235.130 with SMTP id um2mr117895lbc.72.1440617658658; Wed, 26 Aug 2015 12:34:18 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.73.136 with HTTP; Wed, 26 Aug 2015 12:33:38 -0700 (PDT) From: Timofey Titovets Date: Wed, 26 Aug 2015 22:33:38 +0300 Message-ID: Subject: Btrfs duperemove corrupt data while dedup To: linux-btrfs Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Spam-Status: No, score=-8.2 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, T_DKIM_INVALID, T_TVD_MIME_EPI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hello guys, i like btrfs, and i want put it in production soon, one of the feature that i want use, is a deduplication. i frequently testing duperemove on btrfs and already see this problem before. i know what btrfs before, change mtime while deduping, but after dedup fixes from Mark (https://github.com/markfasheh), i've try to get checksums. As i know duperemove use kernel ioctl for deduping, i.e. it's not a duperemove issue, kernel must keep data consistent. File system is fresh and btrfs check not show any metadata corruption. Github issue: https://github.com/markfasheh/duperemove/issues/91 System info: $ uname -a Linux titovetst-beplan 4.2.0-rc8-next-20150825-0959-ARCH #1 SMP Wed Aug 26 10:27:18 MSK 2015 x86_64 GNU/Linux Mount options: rw,relatime,compress=lzo,space_cache,subvolid=257,subvol=/@home Okay, how i find it: md5sum_recursive(){ find $@ -type f -exec md5sum {} \; } cp -av --reflink=always ~/ ~/ md5sum_recursive ~/ > ~/dedup.before duperemove -vhrdb 8k ~/ md5sum_recursive ~/ > ~/dedup.after Only dependencies what i've find it is what smallest block -> more corruptions and vise versa, i.e. more data deduped -> more corrupted. Smart of the disk, it's not looks, like damaged. (attach) What i can provide to help fix this issue? If it's needed, i can recompile kernel with some parameters if it can help, of course. Thanks. diff -up ~/dedup.before ~/dedup.after what i've got (full diff in attach): --- /home/nefelim4ag/dedup.after 2015-08-26 21:36:55.773452558 +0300 +++ /home/nefelim4ag/dedup.before 2015-08-26 21:21:01.203600761 +0300 @@ -25139,9 +25139,9 @@ caf9d41036e46b85d90a9541e8bc9ce1 /home/ .... -0ccbc9c81a51f59dcf2ac0d102de37cb /home/nefelim4ag/L4D2/left4dead2/pak01_003.vpk +e665b502ee977dc1c619ecbd415c91b8 /home/nefelim4ag/L4D2/left4dead2/pak01_000.vpk .... Files sizes not changed and it's > 1MB. Every time i've get a random data corruption.