From patchwork Sat Feb 5 14:28:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ritesh Harjani X-Patchwork-Id: 12736078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46259C433EF for ; Sat, 5 Feb 2022 14:29:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1380130AbiBEO3R (ORCPT ); Sat, 5 Feb 2022 09:29:17 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:2336 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1380122AbiBEO3Q (ORCPT ); Sat, 5 Feb 2022 09:29:16 -0500 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 215BOEsg025853; Sat, 5 Feb 2022 14:29:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : mime-version; s=pp1; bh=oybUHuEDFVN53eQO0IGUWJxdwIqWJ4mony3xODNm0Qs=; b=XfuVeCatkAg8VoQLJxZl8R2KS1KlECXkaitVLfcbv/OUyTsv/wXeotGg+xil6oEDLPYc 627GvTqv8b+ew/uKt3CL6e35yGFT/RUIb6qsk+9HQ/NenxihRXDn/OCapuMXmy3311CT ujN7HQHbEyq/b6XSIUtacTRkkIVFyqpR905AWryb1vZ+SJhRVcrfhSiYs18O+dQy/Bc1 1Sb03ItRzACySDxIKC1QKEcf6KseKB7/+YfhSQgSszkMMPsJvX4kSQ1utmEaDykotlq/ jKWgS2344zgJnGfN34SmKA9XaC5po4x3QudPzk4S2omNOCcwiJ+SI0QlY7cFv5/IRPp+ Yw== Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com [159.122.73.70]) by mx0a-001b2d01.pphosted.com with ESMTP id 3e1j7mexrj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 05 Feb 2022 14:29:16 +0000 Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1]) by ppma01fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 215ERmx3022922; Sat, 5 Feb 2022 14:29:14 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma01fra.de.ibm.com with ESMTP id 3e1gv8t2pe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 05 Feb 2022 14:29:14 +0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 215ET8sZ48628094 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 5 Feb 2022 14:29:08 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 695544C040; Sat, 5 Feb 2022 14:29:08 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1C1344C046; Sat, 5 Feb 2022 14:29:07 +0000 (GMT) Received: from localhost (unknown [9.43.12.205]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Sat, 5 Feb 2022 14:29:06 +0000 (GMT) From: Ritesh Harjani To: fstests Cc: linux-ext4@vger.kernel.org, Ritesh Harjani Subject: [PATCHv1 1/4] ext4/056: Add fast_commit regression test causing data abort exception Date: Sat, 5 Feb 2022 19:58:51 +0530 Message-Id: <78fa98fd44ad52e3c89bee87f7261d1eb5cfe045.1644070604.git.riteshh@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: qv5m-Pe8l-C1rkh_mDX7KI8g7Hu4-dWS X-Proofpoint-GUID: qv5m-Pe8l-C1rkh_mDX7KI8g7Hu4-dWS X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-05_10,2022-02-03_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 bulkscore=0 phishscore=0 impostorscore=0 malwarescore=0 clxscore=1015 priorityscore=1501 spamscore=0 adultscore=0 suspectscore=0 lowpriorityscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202050095 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org This adds a targeted regression test which could cause data abort exception to hit on latest kernel with fast_commit enabled. ext4_mb_mark_bb() does not takes care of block boundary overflow of a given block group while doing set/clear in buffer_head bitmap during fast_commit recovery (after a sudden shutdown). Signed-off-by: Ritesh Harjani --- tests/ext4/056 | 101 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/056.out | 3 ++ 2 files changed, 104 insertions(+) create mode 100755 tests/ext4/056 create mode 100644 tests/ext4/056.out diff --git a/tests/ext4/056 b/tests/ext4/056 new file mode 100755 index 00000000..588e8af8 --- /dev/null +++ b/tests/ext4/056 @@ -0,0 +1,101 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 IBM Corporation. All Rights Reserved. +# +# FS QA Test 056 +# +# regression testing to test kernel crash with fast_commit feature. +# (based on example from tests/generic/468) +# commit: https://patchwork.ozlabs.org/patch/1586856 +# + +. ./common/preamble +_begin_fstest auto quick log + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -r -f $tmp.* + _scratch_unmount > /dev/null 2>&1 +} + +# Import common functions. +. ./common/filter + +# real QA test starts here + +# Modify as appropriate. +_supported_fs ext4 +_require_scratch +_require_scratch_shutdown +_require_scratch_ext4_feature "fast_commit" + +testfile1=$SCRATCH_MNT/testfile1 +testfile2=$SCRATCH_MNT/testfile2 +blocksize=4096 + +$MKFS_EXT4_PROG -F -b $blocksize -O fast_commit $SCRATCH_DEV 5G >> $seqres.full 2>&1 || + _notrun "Require 5G disk space" + +_scratch_mount >> $seqres.full 2>&1 + +check_stat_before_after() +{ + before="$1" + after="$2" + + if [ "$before" != "$after" ]; then + echo "Before: $before" + echo "After : $after" + fi + echo "Before: $before" >> $seqres.full + echo "After : $after" >> $seqres.full +} + +# +# Idea is to create a file of 5M & write 5M of data followed by fsync. +# Then falloc extra blocks (with -k) such that the blocks extend the block group +# boundary considerably, followed by fsync, followed by shutdown. +# This will trigger fast_commit recovery and will help test kernel crash w/o +# fix. +# +fact=10 +echo "Test-1: fsync shutdown recovery test" +$XFS_IO_PROG -f -c "truncate $((5*1024*1024))" \ + -c "pwrite 0 $((5*1024*1024))" \ + -c "fsync" \ + -c "falloc -k $((5*1024*1024)) $((32768*$blocksize*$fact))" \ + $testfile1 >> $seqres.full 2>&1 + +stat_opt='-c "b: %b s: %s a: %x m: %y c: %z"' +before=$(stat "$stat_opt" "$testfile1") + +$XFS_IO_PROG -c "fsync" $testfile1 +_scratch_shutdown -v >> $seqres.full 2>&1 +_scratch_cycle_mount >> $seqres.full 2>&1 + +after=$(stat "$stat_opt" $testfile1) + +check_stat_before_after "$before" "$after" + +echo "Test-2: fdatasync shutdown recovery test" +$XFS_IO_PROG -f -c "truncate $((5*1024*1024))" \ + -c "pwrite 0 $((5*1024*1024))" \ + -c "fsync" \ + -c "falloc -k $((5*1024*1024)) $((32768*$blocksize*$fact))" \ + $testfile2 >> $seqres.full 2>&1 + +stat_opt='-c "b: %b s: %s"' +before=$(stat "$stat_opt" $testfile2) + +$XFS_IO_PROG -c "fdatasync" $testfile2 +_scratch_shutdown -v >> $seqres.full 2>&1 +_scratch_cycle_mount >> $seqres.full 2>&1 + +after=$(stat "$stat_opt" "$testfile2") +check_stat_before_after "$before" "$after" + +# success, all done +status=0 +exit diff --git a/tests/ext4/056.out b/tests/ext4/056.out new file mode 100644 index 00000000..0a793a4c --- /dev/null +++ b/tests/ext4/056.out @@ -0,0 +1,3 @@ +QA output created by 056 +Test-1: fsync shutdown recovery test +Test-2: fdatasync shutdown recovery test From patchwork Sat Feb 5 14:28:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ritesh Harjani X-Patchwork-Id: 12736079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BD96C433EF for ; Sat, 5 Feb 2022 14:29:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345823AbiBEO3V (ORCPT ); Sat, 5 Feb 2022 09:29:21 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:31102 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355500AbiBEO3T (ORCPT ); Sat, 5 Feb 2022 09:29:19 -0500 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21583TgV002439; Sat, 5 Feb 2022 14:29:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=EvIJ6Q1EY4tw45dTRGd8+HjE4HztvK3unbqkRLOKWTg=; b=dpEovuU42nbvK8SOpfvXmZ3ZNQh6ONhj8TLws4oLQIMKwGhZG/QB3TD2bxwmFqROl5Xz vM5VaXdf2qSvWDhrWUuJgK8hBjwG12uG8BN0CRkslW6PeVZS/UKA2udenGk3R90aTWtW eXsUxEghGxM+1CQvjuDgbzBGd/vPaMgkRNKW7Q1+3eWTnitvgApiNR5bg2cZtMqAhMEL rvFnX/5ttFwcjnlzNlXS8tVY86HmwRhtyw2zHiTwePGbE5vXwGHiQWzntbUqxkJWHslo mPfiW5k9k0RXY5AnqFD+zWd15hWnlZBg1FmFbxNADlzVeNlgoDRoUWMGlv5LLTD38o0z 6Q== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3e1ekuhsjw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 05 Feb 2022 14:29:19 +0000 Received: from m0098393.ppops.net (m0098393.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 215EFWP0009297; Sat, 5 Feb 2022 14:29:19 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 3e1ekuhsjj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 05 Feb 2022 14:29:18 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 215ERalN032508; Sat, 5 Feb 2022 14:29:16 GMT Received: from b06avi18626390.portsmouth.uk.ibm.com (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192]) by ppma04ams.nl.ibm.com with ESMTP id 3e1gv8j7u7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 05 Feb 2022 14:29:16 +0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 215EJGXE49480110 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 5 Feb 2022 14:19:16 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AFFC2AE053; Sat, 5 Feb 2022 14:29:14 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9EAE8AE04D; Sat, 5 Feb 2022 14:29:13 +0000 (GMT) Received: from localhost (unknown [9.43.12.205]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Sat, 5 Feb 2022 14:29:12 +0000 (GMT) From: Ritesh Harjani To: fstests Cc: linux-ext4@vger.kernel.org, Ritesh Harjani Subject: [PATCHv1 2/4] common/punch: Add block_size argument to _filter_fiemap_** Date: Sat, 5 Feb 2022 19:58:52 +0530 Message-Id: <6db398df705564ba2776d2eb15a7ceec1b236d3b.1644070604.git.riteshh@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: SrAHkeWkpE5wFPqidE43kdjQBa_5SfN9 X-Proofpoint-ORIG-GUID: GEuguBBkNyhroD_3qks_KzejtmJVA5t4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-05_10,2022-02-03_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 bulkscore=0 mlxlogscore=999 clxscore=1015 lowpriorityscore=0 malwarescore=0 suspectscore=0 phishscore=0 adultscore=0 impostorscore=0 priorityscore=1501 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202050095 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Ritesh Harjani Add block_size paramter to _filter_fiemap_flags() and _filter_hole_fiemap(). This is used in next patches Also this fixes some of the end of line whitespace issues while we are at it. Signed-off-by: Ritesh Harjani --- common/punch | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/common/punch b/common/punch index b6b8a0b9..f99e21ad 100644 --- a/common/punch +++ b/common/punch @@ -109,6 +109,7 @@ _filter_fiemap() _filter_fiemap_flags() { + block_size=$1 $AWK_PROG ' $3 ~ /hole/ { print $1, $2, $3; @@ -135,23 +136,24 @@ _filter_fiemap_flags() } print $1, $2, flag_str }' | - _coalesce_extents + _coalesce_extents $block_size } -# Filters fiemap output to only print the +# Filters fiemap output to only print the # file offset column and whether or not # it is an extent or a hole _filter_hole_fiemap() { + block_size=$1 $AWK_PROG ' $3 ~ /hole/ { - print $1, $2, $3; + print $1, $2, $3; next; - } + } $5 ~ /0x[[:xdigit:]]+/ { print $1, $2, "extent"; }' | - _coalesce_extents + _coalesce_extents $block_size } # 10000 Unwritten preallocated extent From patchwork Sat Feb 5 14:28:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ritesh Harjani X-Patchwork-Id: 12736080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99F40C433F5 for ; Sat, 5 Feb 2022 14:29:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1380136AbiBEO3Z (ORCPT ); Sat, 5 Feb 2022 09:29:25 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:47362 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355500AbiBEO3X (ORCPT ); Sat, 5 Feb 2022 09:29:23 -0500 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 215BS84M026407; Sat, 5 Feb 2022 14:29:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=DOdkmKm1Y2btrP4rCP5FQCDxOI86sx/OHN8djhv4Gv8=; b=thMbseC/YZ2Z0qdLaisV/0T7XQZ5pVe10E7hwf7pgw19gp41Q6hma9wZ0tLzAMCe8o6V 7GeB2UZnqwBfDW2F71FGocdNqqSz4BytgU6yKUPIH2D6dAV2NFYkRADgRsCHgMeYDOYp nM1kgfruGn0ylJZuPWyqf6/3TuemPqO5CBYhEnbG2fo1FuzoZ0D6htKpxO2aSs68TiPB GAFw6RbMpCxA9dMkEvPlme1bHSuwdPeh8e3vba/68Ng6JSgiy4bNt6PycIFoShTdd8KF CltXePMloTPt8i+X+8z4VxVpVeXbIm2D9/cjS3BVdaKGr/DhDzxZTdVtKu7O3QSzl+8m ZA== Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com [159.122.73.70]) by mx0a-001b2d01.pphosted.com with ESMTP id 3e1j7mexse-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 05 Feb 2022 14:29:23 +0000 Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1]) by ppma01fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 215ERk2P022884; Sat, 5 Feb 2022 14:29:20 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma01fra.de.ibm.com with ESMTP id 3e1gv8t2pn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 05 Feb 2022 14:29:20 +0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 215ETHqk47382786 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 5 Feb 2022 14:29:17 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7EE7842042; Sat, 5 Feb 2022 14:29:17 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5C7034203F; Sat, 5 Feb 2022 14:29:16 +0000 (GMT) Received: from localhost (unknown [9.43.12.205]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Sat, 5 Feb 2022 14:29:16 +0000 (GMT) From: Ritesh Harjani To: fstests Cc: linux-ext4@vger.kernel.org, Ritesh Harjani Subject: [PATCHv1 3/4] ext4/057: Add crash test to check unwritten extents tracking with fast_commit Date: Sat, 5 Feb 2022 19:58:53 +0530 Message-Id: <0c28ae0676bf76d02d2b0edf092f55cb6d049737.1644070604.git.riteshh@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: Hv93-zQ7okUoipHulrx6GII7AewXWRjT X-Proofpoint-GUID: Hv93-zQ7okUoipHulrx6GII7AewXWRjT X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-05_10,2022-02-03_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 bulkscore=0 phishscore=0 impostorscore=0 malwarescore=0 clxscore=1015 priorityscore=1501 spamscore=0 adultscore=0 suspectscore=0 lowpriorityscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202050095 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org fast_commit in certain cases as discussed in the corresponding kernel commit may miss to track the unwritten extents. This patch add a test case around that (based on test case shared in kernel commit) 5e4d0eba1ccaf19f ext4: fix fast commit may miss tracking range for FALLOC_FL_ZERO_RANGE Signed-off-by: Ritesh Harjani --- tests/ext4/057 | 62 ++++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/057.out | 6 +++++ 2 files changed, 68 insertions(+) create mode 100755 tests/ext4/057 create mode 100644 tests/ext4/057.out diff --git a/tests/ext4/057 b/tests/ext4/057 new file mode 100755 index 00000000..9d008b65 --- /dev/null +++ b/tests/ext4/057 @@ -0,0 +1,62 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 IBM. All Rights Reserved. +# +# FS QA Test 057 +# +# Test for ext4 fast_commit crash recovery test fixed by below kernel commit. +# +# commit 5e4d0eba1ccaf19f +# ext4: fix fast commit may miss tracking range for FALLOC_FL_ZERO_RANGE +# +. ./common/preamble +_begin_fstest auto quick log shutdown + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -r -f $tmp.* +} + +# Import common functions. +. ./common/filter +. ./common/punch + +# real QA test starts here + +# Modify as appropriate. +_supported_fs ext4 +_require_scratch +_require_scratch_ext4_feature "fast_commit" + +t1=$SCRATCH_MNT/t1 + +$MKFS_EXT4_PROG -F -O 64bit,fast_commit $SCRATCH_DEV 512m >> $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +bs=$(_get_block_size $SCRATCH_MNT) + +# create and write data to t1 +$XFS_IO_PROG -f -c "pwrite 0 $((100*$bs))" $t1 | _filter_xfs_io + +# fsync t1 +$XFS_IO_PROG -c "fsync" $t1 + +# fzero certain range in between +$XFS_IO_PROG -c "fzero -k $((40*$bs)) $((20*$bs))" $t1 | _filter_xfs_io + +# fsync t1 +$XFS_IO_PROG -c "fsync" $t1 + +# shutdown FS now for replay of FC to kick in next mount +_scratch_shutdown -v >> $seqres.full 2>&1 + +_scratch_cycle_mount + +# check fiemap reported is valid or not +$XFS_IO_PROG -c "fiemap -v" $t1 | _filter_fiemap_flags $bs + +# success, all done +status=0 +exit diff --git a/tests/ext4/057.out b/tests/ext4/057.out new file mode 100644 index 00000000..fc905de6 --- /dev/null +++ b/tests/ext4/057.out @@ -0,0 +1,6 @@ +QA output created by 057 +wrote 409600/409600 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +0: [0..39]: none +1: [40..59]: unwritten +2: [60..99]: nonelast From patchwork Sat Feb 5 14:28:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ritesh Harjani X-Patchwork-Id: 12736081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6ADBC433FE for ; Sat, 5 Feb 2022 14:29:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355500AbiBEO32 (ORCPT ); Sat, 5 Feb 2022 09:29:28 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:13740 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1380134AbiBEO31 (ORCPT ); Sat, 5 Feb 2022 09:29:27 -0500 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 215DQNjl021802; Sat, 5 Feb 2022 14:29:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=foYX60cfFPgyXzRx97URzzhS+rlh07CUJw4hGmMnd6A=; b=jb3INu2TzCAjMK+zN2htAx75UCsWp02GGogPTb+VC5NtXg6OqVcVFPEux+LT/AadDbA6 kiZ56UT7H06yJHT0M5lM3akP+0aCj538iPuPlOpv5qRYpR6FM9Mt1eCLDsDhhnzz7p05 MuyQYPZvbqYOtshLSX50edjA7wb90b1Lt408hKnXCLhg+oD1RGkNHmMAHf7CcSNLxJ+9 M0aclwGcB+eiM11AK0bbWao09bJ8hvqXmhj2QfRWhDhI2/DAzyRgGACHS7dt2BP1Qz14 KtLztjY4WzNGrUwiBzTD8sLVmwBJs5ilN2iSlSRvn1lX69q8wMYwEWBYq7nKnhNUuwyl Iw== Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0b-001b2d01.pphosted.com with ESMTP id 3e1en89tec-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 05 Feb 2022 14:29:26 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 215ERalO032508; Sat, 5 Feb 2022 14:29:25 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma04ams.nl.ibm.com with ESMTP id 3e1gv8j7uf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 05 Feb 2022 14:29:25 +0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 215ETLUq44892462 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 5 Feb 2022 14:29:21 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D48415204F; Sat, 5 Feb 2022 14:29:21 +0000 (GMT) Received: from localhost (unknown [9.43.12.205]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id F251A5204E; Sat, 5 Feb 2022 14:29:20 +0000 (GMT) From: Ritesh Harjani To: fstests Cc: linux-ext4@vger.kernel.org, Ritesh Harjani Subject: [PATCHv1 4/4] ext4/058: Add shutdown recovery test with fast_commit Date: Sat, 5 Feb 2022 19:58:54 +0530 Message-Id: <6f55546f021fac9b35ae07d4323f4aba00baad95.1644070604.git.riteshh@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: iFg6-LsGGO1xqf6iGh6AJ6J9w_1HSL_K X-Proofpoint-ORIG-GUID: iFg6-LsGGO1xqf6iGh6AJ6J9w_1HSL_K X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-05_10,2022-02-03_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 phishscore=0 malwarescore=0 bulkscore=0 mlxscore=0 adultscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 priorityscore=1501 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202050095 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org In certain cases fast_commit may not delete the range during replay phase (after sudden FS shutdown) due to some operations which depends on inode->i_size (which during replay of an inode with fast_commit could be 0 for sometime). This fstest tests such scenario with fast_commit in ext4. This test case is based on the test case shared via Xin Yin. Signed-off-by: Ritesh Harjani --- tests/ext4/058 | 66 ++++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/058.out | 7 +++++ 2 files changed, 73 insertions(+) create mode 100755 tests/ext4/058 create mode 100644 tests/ext4/058.out diff --git a/tests/ext4/058 b/tests/ext4/058 new file mode 100755 index 00000000..4d70f483 --- /dev/null +++ b/tests/ext4/058 @@ -0,0 +1,66 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 IBM. All Rights Reserved. +# +# FS QA Test 058 +# +# Tests fast_commit feature of ext4 as fixed in below commit +# 0b5b5a62b945a141: ext4: use ext4_ext_remove_space() for fast commit replay delete range +# (Based on test case shared by Xin Yin ) +# +. ./common/preamble +_begin_fstest auto shutdown quick log recoveryloop + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -r -f $tmp.* + _scratch_unmount > /dev/null 2>&1 +} + +# Import common functions. +. ./common/filter +. ./common/punch + +# real QA test starts here + +# Modify as appropriate. +_supported_fs ext4 +_require_scratch +_require_scratch_ext4_feature "fast_commit" + +t1=$SCRATCH_MNT/foo +t2=$SCRATCH_MNT/bar + +$MKFS_EXT4_PROG -F -O 64bit,fast_commit $SCRATCH_DEV 512m >> $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +bs=$(_get_block_size $SCRATCH_MNT) + +# create and write data to t1 +$XFS_IO_PROG -f -c "pwrite 0 $((100*$bs))" $t1 | _filter_xfs_io + +# fzero certain range in between with -k +$XFS_IO_PROG -c "fzero -k $((40*$bs)) $((20*$bs))" $t1 | _filter_xfs_io + +# create and fsync a new file t2 +$XFS_IO_PROG -f -c "fsync" $t2 + +# fpunch within the i_size of a file +$XFS_IO_PROG -c "fpunch $((30*$bs)) $((20*$bs))" $t1 + +# fsync t1 to trigger fast_commit operation +$XFS_IO_PROG -c "fsync" $t1 + +# shutdown FS now for replay of FC to kick in next mount +_scratch_shutdown -v >> $seqres.full 2>&1 + +_scratch_cycle_mount + +# check fiemap reported is valid or not +$XFS_IO_PROG -c "fiemap -v" $t1 | _filter_fiemap_flags $bs + +# success, all done +status=0 +exit diff --git a/tests/ext4/058.out b/tests/ext4/058.out new file mode 100644 index 00000000..98359e01 --- /dev/null +++ b/tests/ext4/058.out @@ -0,0 +1,7 @@ +QA output created by 058 +wrote 409600/409600 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +0: [0..29]: none +1: [30..49]: hole +2: [50..59]: unwritten +3: [60..99]: nonelast