From patchwork Thu Jan 12 13:57:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amir Goldstein X-Patchwork-Id: 9513239 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5926C60710 for ; Thu, 12 Jan 2017 13:58:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31FFB2868C for ; Thu, 12 Jan 2017 13:58:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 26986286C4; Thu, 12 Jan 2017 13:58:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID, T_TVD_MIME_EPI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54875286B9 for ; Thu, 12 Jan 2017 13:58:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750799AbdALN6N (ORCPT ); Thu, 12 Jan 2017 08:58:13 -0500 Received: from mail-oi0-f65.google.com ([209.85.218.65]:36407 "EHLO mail-oi0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750769AbdALN6E (ORCPT ); Thu, 12 Jan 2017 08:58:04 -0500 Received: by mail-oi0-f65.google.com with SMTP id u143so2895638oif.3 for ; Thu, 12 Jan 2017 05:57:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Xma6x9Fc8qcdu6Q5VN7NirqfTyZKT5Cmpjbpq2QgWUI=; b=TH/LIYrB9ORJKIul1mVjmHm1bSbHxuTcT471NdlxjjByZwHqsiehXDcMkreSZ9CGh9 xXrLqpqra0o2D7ezzq2l0B8365Goi86DN8gfMbsk4dV+FSokzSZwkzawRS4EayUf5d4D C2Hky6NjV22Ot+mlQoNi8OMYqubl4GwUiZTh4VqKrPP46IRYggDGf5i02GLd7H8+qsoB T1y+FxJKb1tSDEd2RvRUERRWm7piXas/l8WQridfu7xrT8H5Ss72B4uSWxgmhHKu/rZG 7tu0F1ASBCoZLd5WoxicHrcjFbv6t4ZahiKDcoFCmqzD1zhiUuLpehMHrmBvS0xB66Ra MkKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Xma6x9Fc8qcdu6Q5VN7NirqfTyZKT5Cmpjbpq2QgWUI=; b=iBmFviJk9mFo5qDameqTDcEGfFCGVt7v0BRdWs9gIpWSbrnpmbdhtKURNXP2LTRaiV p0e56hg0ahX93lUdVdB9QZ4+MB8JEHw15NAb69MhMEz8axeIUQe3+oJ3+qFgUcynMj91 nvrfQ51QLiUykuh7rmcEzAFNuYr6gzIMGnQlOJgSkSt6Q3DQchNPQVFWhym/M4NeXFYK 7uOvL8r6jw+W/EkxnE/OvV0KR93uz4KaxnlfVLoK3poz5DM+gSkdPzBH3NdDEsCclBDH nQwE+OXYHRqo4jVaRkuNikD4IfjoECXcbfW5pCbbb7RjS3//UCQKlXH5+edYnJ4p60tI JxGg== X-Gm-Message-State: AIkVDXIXFUVcs78F8rpfWLOvhNrg+S/MwvIpETpoGntD2Im0y0zxfd4wNZJPRTa2UC8S0rMZGAulhaKhzJd5xQ== X-Received: by 10.202.79.150 with SMTP id d144mr7093629oib.149.1484229442112; Thu, 12 Jan 2017 05:57:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.182.61.14 with HTTP; Thu, 12 Jan 2017 05:57:21 -0800 (PST) In-Reply-To: <20170111173227.GS14038@birch.djwong.org> References: <20170110194249.GK14038@birch.djwong.org> <20170111173227.GS14038@birch.djwong.org> From: Amir Goldstein Date: Thu, 12 Jan 2017 15:57:21 +0200 Message-ID: Subject: Re: [RFC PATCH] xfs_db: sanitize geometry on load To: "Darrick J. Wong" Cc: Eric Sandeen , linux-xfs@vger.kernel.org, Eryu Guan Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Wed, Jan 11, 2017 at 7:32 PM, Darrick J. Wong wrote: > On Wed, Jan 11, 2017 at 12:01:22PM +0200, Amir Goldstein wrote: ... >> Now I am hitting these xfs_db crashes during xfs/1316, which are apparently not >> related to OOM killer. I have seen them last run as well but dmesg is quiet now. >> >> xfs/1316 *** Error in `/usr/sbin/xfs_db': free(): invalid >> pointer: 0x00007f9dbf036b78 *** >> ======= Backtrace: ========= >> /lib/x86_64-linux-gnu/libc.so.6(+0x77725)[0x7f9dbecea725] >> /lib/x86_64-linux-gnu/libc.so.6(+0x7ff4a)[0x7f9dbecf2f4a] >> /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f9dbecf6abc] >> /usr/sbin/xfs_db[0x414961] >> /usr/sbin/xfs_db[0x4154de] >> /usr/sbin/xfs_db[0x420d38] >> /usr/sbin/xfs_db[0x420926] >> /usr/sbin/xfs_db[0x405125] >> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f9dbec93830] >> /usr/sbin/xfs_db[0x405179] > > Ok well I definitely don't see /this/ happening. I gather you built > xfsprogs with the insane geometry patch; if so, against what git commit? > And, did the binary get installed as /usr/sbin/xfs_db, or is this just > the system xfs_db? > The story with xfs/1316 is that _scratch_xfs_get_metadata_field is broken for some reason related to feeding stdin to xfs_db and the following line ends up with empty inode_ver: inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "inode ${inum}") and it's all down hill from there I fixed _scratch_xfs_get_metadata_field and friends to use xfs_db -c (see attached fuzzy.patch) and then I got this assertion when running xfs/1316: (attached full results and dmesg upto the assert) Jan 12 15:40:23 c800p-fstests kernel: [338446.829922] XFS: Assertion failed: i + num_recs <= room, file: /home/amir/src/linux/fs/xfs/libxfs/xfs_bmap.c, line: 1331 Jan 12 15:40:23 c800p-fstests kernel: [338446.829948] ------------[ cut here ]------------ Jan 12 15:40:23 c800p-fstests kernel: [338446.829978] kernel BUG at /home/amir/src/linux/fs/xfs/xfs_message.c:113! Jan 12 15:40:23 c800p-fstests kernel: [338446.830006] invalid opcode: 0000 [#1] SMP Jan 12 15:40:23 c800p-fstests kernel: [338446.830023] Modules linked in: btrfs xor raid6_pq ufs qnx4 hfsplus hfs minix ntfs msdos jfs xfs dm_thin_pool dm_persistent_data dm_bio_prison dm_snapshot dm_bufio dm_flakey libcrc32c pl2303 usbserial snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core intel_rapl snd_pcm x86_pkg_temp_thermal intel_powerclamp coretemp snd_hwdep crct10dif_pclmul crc32_pclmul snd_seq_midi aesni_intel snd_seq_midi_event aes_x86_64 snd_rawmidi crypto_simd cryptd snd_seq glue_helper snd_timer snd_seq_device serio_raw snd soundcore mei_me lpc_ich ie31200_edac shpchp mei edac_core mac_hid parport_pc ppdev lp parport autofs4 i915 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm psmouse ahci libahci r8169 mii video [last unloaded: xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.830311] CPU: 2 PID: 3784 Comm: xfs_scrub Not tainted 4.10.0-rc2+ #14 Jan 12 15:40:23 c800p-fstests kernel: [338446.830338] Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./B75N, BIOS F3 10/23/2012 Jan 12 15:40:23 c800p-fstests kernel: [338446.830374] task: ffff8e2d4bd8cf80 task.stack: ffffa1334dac8000 Jan 12 15:40:23 c800p-fstests kernel: [338446.830447] RIP: 0010:assfail+0x20/0x30 [xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.830466] RSP: 0018:ffffa1334dacb838 EFLAGS: 00010246 Jan 12 15:40:23 c800p-fstests kernel: [338446.830489] RAX: 0000000000000000 RBX: ffff8e2ac1bac000 RCX: 0000000000000000 Jan 12 15:40:23 c800p-fstests kernel: [338446.830518] RDX: 00000000ffffffc0 RSI: 000000000000000a RDI: ffffffffc0854a65 Jan 12 15:40:23 c800p-fstests kernel: [338446.830545] RBP: ffffa1334dacb838 R08: 0000000000000000 R09: 0000000000000000 Jan 12 15:40:23 c800p-fstests kernel: [338446.830573] R10: 000000000000000a R11: f000000000000000 R12: 0000000000000082 Jan 12 15:40:23 c800p-fstests kernel: [338446.830602] R13: 000000000000007f R14: ffff8e2b57031488 R15: ffff8e2a7e34f838 Jan 12 15:40:23 c800p-fstests kernel: [338446.830630] FS: 00007fcb89489700(0000) GS:ffff8e2d4e800000(0000) knlGS:0000000000000000 Jan 12 15:40:23 c800p-fstests kernel: [338446.830662] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 12 15:40:23 c800p-fstests kernel: [338446.830686] CR2: 00007fcb8bdfac60 CR3: 0000000276993000 CR4: 00000000001406e0 Jan 12 15:40:23 c800p-fstests kernel: [338446.830714] Call Trace: Jan 12 15:40:23 c800p-fstests kernel: [338446.830756] xfs_bmap_read_extents+0x403/0x500 [xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.830810] xfs_iread_extents+0x99/0x180 [xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.830862] xfs_bmap_count_blocks+0x1f7/0x240 [xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.830921] xfs_scrub_inode+0x717/0x12e0 [xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.830978] ? __xfs_scrub_setup_inode+0x2c0/0x2c0 [xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.831035] xfs_scrub_metadata+0x381/0xa20 [xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.831090] ? xfs_scrub_setup_inode+0x20/0x20 [xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.831147] xfs_ioc_scrub_metadata+0x6d/0xd0 [xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.831201] xfs_file_ioctl+0xa3e/0xe20 [xfs] Jan 12 15:40:23 c800p-fstests kernel: [338446.831222] ? __lock_acquire+0x27f/0x14b0 Jan 12 15:40:23 c800p-fstests kernel: [338446.831242] ? mntput_no_expire+0x5/0x3f0 Jan 12 15:40:23 c800p-fstests kernel: [338446.831260] ? sched_clock+0x9/0x10 Jan 12 15:40:23 c800p-fstests kernel: [338446.831277] ? sched_clock_cpu+0xa1/0xb0 Jan 12 15:40:23 c800p-fstests kernel: [338446.831295] ? __fget+0xf5/0x200 Jan 12 15:40:23 c800p-fstests kernel: [338446.831311] do_vfs_ioctl+0xa4/0x710 Jan 12 15:40:23 c800p-fstests kernel: [338446.831328] ? __fget+0x114/0x200 Jan 12 15:40:23 c800p-fstests kernel: [338446.831344] ? __fget+0x5/0x200 Jan 12 15:40:23 c800p-fstests kernel: [338446.831360] SyS_ioctl+0x79/0x90 Jan 12 15:40:23 c800p-fstests kernel: [338446.831377] entry_SYSCALL_64_fastpath+0x23/0xc6 Jan 12 15:40:23 c800p-fstests kernel: [338446.831397] RIP: 0033:0x7fcb8bd89687 Jan 12 15:40:23 c800p-fstests kernel: [338446.831413] RSP: 002b:00007fcb89486538 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 Jan 12 15:40:23 c800p-fstests kernel: [338446.831443] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcb8bd89687 Jan 12 15:40:23 c800p-fstests kernel: [338446.831471] RDX: 00007fcb89486670 RSI: 00000000c040583c RDI: 0000000000000003 Jan 12 15:40:23 c800p-fstests kernel: [338446.831499] RBP: 0000000001a2c2a0 R08: 0000000000000000 R09: 0000000000000010 Jan 12 15:40:23 c800p-fstests kernel: [338446.832426] R10: 0000000000000006 R11: 0000000000000246 R12: 00007fff8f118e10 Jan 12 15:40:23 c800p-fstests kernel: [338446.833353] R13: 0000000000000000 R14: 00007fcb89486700 R15: 00007fcb894869f0 Jan 12 15:40:23 c800p-fstests kernel: [338446.834281] Code: 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 f1 41 89 d0 48 c7 c6 00 3b 86 c0 48 89 fa 31 ff 48 89 e5 e8 b0 f8 ff ff <0f> 0b 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 Jan 12 15:40:23 c800p-fstests kernel: [338446.836229] RIP: assfail+0x20/0x30 [xfs] RSP: ffffa1334dacb838 Jan 12 15:40:23 c800p-fstests kernel: [338446.840623] ---[ end trace a5349c4247bd4fe3 ]--- diff --git a/common/fuzzy b/common/fuzzy index d970710..fdd45c8 100644 --- a/common/fuzzy +++ b/common/fuzzy @@ -116,10 +116,11 @@ _scratch_xfs_list_metadata_fields() { return; fi - (for arg in "$@"; do - echo "${arg}" + local cmds=() + for arg in "$@"; do + cmds+=("-c" "${arg}") done - echo "print") | _scratch_xfs_db | __filter_xfs_db_print_fields "${filter}" + _scratch_xfs_db "${cmds[@]}" -c print | __filter_xfs_db_print_fields "${filter}" } # Get a metadata field @@ -130,10 +131,11 @@ _scratch_xfs_get_metadata_field() { shift grep_key="$(echo "${key}" | tr '[]()' '....')" - (for arg in "$@"; do - echo "${arg}" + local cmds=() + for arg in "$@"; do + cmds+=("-c" "${arg}") done - echo "print ${key}") | _scratch_xfs_db | grep "^${grep_key}" | \ + _scratch_xfs_db "${cmds[@]}" -c "print ${key}" | grep "^${grep_key}" | \ sed -e 's/^.* = //g' } @@ -145,10 +147,11 @@ _scratch_xfs_set_metadata_field() { key="$1" value="$2" shift; shift - (for arg in "$@"; do - echo "${arg}" + local cmds=() + for arg in "$@"; do + cmds+=("-c" "${arg}") done - echo "write -d ${key} ${value}") | _scratch_xfs_db -x + _scratch_xfs_db -x "${cmds[@]}" -c "write -d ${key} ${value}" echo } @@ -167,10 +170,11 @@ _scratch_xfs_fuzz_metadata_field() { fuzz_arg="-d" fi oldval="$(_scratch_xfs_get_metadata_field "${key}" "$@")" - (for arg in "$@"; do - echo "${arg}" + local cmds=() + for arg in "$@"; do + cmds+=("-c" "${arg}") done - echo "fuzz ${fuzz_arg} ${key} ${value}") | _scratch_xfs_db -x + _scratch_xfs_db -x "${cmds[@]}" -c "fuzz ${fuzz_arg} ${key} ${value}" echo newval="$(_scratch_xfs_get_metadata_field "${key}" "$@" 2> /dev/null)" if [ "${oldval}" = "${newval}" ]; then