From patchwork Tue Oct 8 01:03:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 11178663 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4FB4E1864 for ; Tue, 8 Oct 2019 01:03:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2E70121835 for ; Tue, 8 Oct 2019 01:03:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="e7WRy+jq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729327AbfJHBDP (ORCPT ); Mon, 7 Oct 2019 21:03:15 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:57750 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728983AbfJHBDP (ORCPT ); Mon, 7 Oct 2019 21:03:15 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x980xNV5190794; Tue, 8 Oct 2019 01:03:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2019-08-05; bh=bF4y32Oh22uUHynBlfNeyYOAi2KI9yl111Hjs4N6mRE=; b=e7WRy+jqHuj9dYTnzI/SbajaxaqfHDaeKNDpOX1o4aBBdysn0o50RngQNI67ibR3CK8b YnZqjKGHqNKsfnCQ9jw0M+6+w2WToCLoK1q5llJk7jTkWnzci6c4FMgY+92KVxBvUpo+ y3RzAThJZBM635FLonB2ptUuNzXvJW/3d6og/ebEzNIrCuyqUwO6L4i3nkcMedu+MJbw WCNDbhnQc8e1LOMXNkwC7ZUxizNe86HoNZ5eq01Ni5deapeUIdmixkkX3PNUwbrEu51X 1cPKr/XO2hUWOWeEWQYEDOtIpCMSLqZSGzAnSPsBrWkupRbuLBmO4VuR5MEFOnKsdOZy +g== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2120.oracle.com with ESMTP id 2vek4qa4fp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 08 Oct 2019 01:03:13 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x980wZ0A025750; Tue, 8 Oct 2019 01:03:13 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserp3020.oracle.com with ESMTP id 2vgef9mw9h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 08 Oct 2019 01:03:13 +0000 Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x9813C9R010277; Tue, 8 Oct 2019 01:03:12 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 07 Oct 2019 18:03:12 -0700 Subject: [PATCH 1/4] xfs/196: check for delalloc blocks after pwrite From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Mon, 07 Oct 2019 18:03:11 -0700 Message-ID: <157049659135.2397321.4055705884999858018.stgit@magnolia> In-Reply-To: <157049658503.2397321.13914737091290093511.stgit@magnolia> References: <157049658503.2397321.13914737091290093511.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9403 signatures=668684 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910080009 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9403 signatures=668684 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910080009 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong This test depends on the pwrite creating delalloc blocks, which doesn't happen if the scratch fs is mounted in dax mode (or has an extent size hint applied). Therefore, check for delalloc blocks and _notrun if we didn't get any. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- tests/xfs/196 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/xfs/196 b/tests/xfs/196 index 5dc28670..406146c5 100755 --- a/tests/xfs/196 +++ b/tests/xfs/196 @@ -50,6 +50,8 @@ bytes=$((64 * 1024)) # create sequential delayed allocation $XFS_IO_PROG -f -c "pwrite 0 $bytes" $file >> $seqres.full 2>&1 +$XFS_IO_PROG -c "bmap -elpv" $file | grep -q delalloc || \ + _notrun "Unable to create delayed allocations" # Enable write drops. All buffered writes are dropped from this point on. _scratch_inject_error "drop_writes" 1 From patchwork Tue Oct 8 01:03:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 11178667 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3120B1864 for ; Tue, 8 Oct 2019 01:03:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0F8B321721 for ; Tue, 8 Oct 2019 01:03:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="PFRduCy4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729375AbfJHBDW (ORCPT ); Mon, 7 Oct 2019 21:03:22 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:57848 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728983AbfJHBDW (ORCPT ); Mon, 7 Oct 2019 21:03:22 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x980xNir190797; Tue, 8 Oct 2019 01:03:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2019-08-05; bh=/P4HlNzecQYTk7oidI45U4323OgpjZLmg406TAvtVBU=; b=PFRduCy4HK6MwCBv+arM7B7xRAMxidDHycKxW2qJ6sVsZjSBgmsj1l8R1wlvifTzorzk eR4Uabkkwdt5j1wSaU5u8ZZYjQ5ynsjr4XX4bGJIP7SUnpgzG2dZ7fk85/K3Q80yPgF1 fHkvnfj6zCQBc/pQ077Y4hS92wjOj4NWaoJi+86H+Ns27Is7Jhn6FwgAI1QrWCRInP+L nrurs7XyB1fo0zT6MrdZ4ieOlFF4Xp7t4H/Pg+ncngSBEUpBIOq/ktO4Ojs9006XT1/n ArEKn6LtlpuGnShY8S/GnZHaCWvFf8ZPx+jBzGIj2TYOD2t5IaCpyWRrhIzIX5K+xEGS Yw== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 2vek4qa4fw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 08 Oct 2019 01:03:20 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x980wKDw112805; Tue, 8 Oct 2019 01:03:19 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3030.oracle.com with ESMTP id 2vgeuwnbpq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 08 Oct 2019 01:03:19 +0000 Received: from abhmp0003.oracle.com (abhmp0003.oracle.com [141.146.116.9]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x9813IMi003854; Tue, 8 Oct 2019 01:03:18 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 07 Oct 2019 18:03:18 -0700 Subject: [PATCH 2/4] xfs/{088, 089, 091}: redirect stderr when writing to corrupt fs From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Mon, 07 Oct 2019 18:03:17 -0700 Message-ID: <157049659754.2397321.4949328613812405852.stgit@magnolia> In-Reply-To: <157049658503.2397321.13914737091290093511.stgit@magnolia> References: <157049658503.2397321.13914737091290093511.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9403 signatures=668684 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910080009 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9403 signatures=668684 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910080009 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong These tests primarily check that writes to a corrupt fs don't take down the system, and that running repair will fix them. Therefore, redirect stderr to seqres.full so that we don't fail these tests in DAX mode. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- tests/xfs/088 | 2 +- tests/xfs/089 | 2 +- tests/xfs/091 | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/xfs/088 b/tests/xfs/088 index 74b45163..d8ca877a 100755 --- a/tests/xfs/088 +++ b/tests/xfs/088 @@ -80,7 +80,7 @@ echo "+ mount image && modify files" if _try_scratch_mount >> $seqres.full 2>&1; then for x in `seq 1 64`; do - $XFS_IO_PROG -f -c "pwrite -S 0x62 0 ${blksz}" "${TESTFILE}.${x}" >> $seqres.full + $XFS_IO_PROG -f -c "pwrite -S 0x62 0 ${blksz}" "${TESTFILE}.${x}" >> $seqres.full 2>> $seqres.full done umount "${SCRATCH_MNT}" fi diff --git a/tests/xfs/089 b/tests/xfs/089 index bcbc6363..ad980769 100755 --- a/tests/xfs/089 +++ b/tests/xfs/089 @@ -80,7 +80,7 @@ echo "+ mount image && modify files" if _try_scratch_mount >> $seqres.full 2>&1; then for x in `seq 1 64`; do - $XFS_IO_PROG -f -c "pwrite -S 0x62 0 ${blksz}" "${TESTFILE}.${x}" >> $seqres.full + $XFS_IO_PROG -f -c "pwrite -S 0x62 0 ${blksz}" "${TESTFILE}.${x}" >> $seqres.full 2>> $seqres.full done umount "${SCRATCH_MNT}" fi diff --git a/tests/xfs/091 b/tests/xfs/091 index be56d8ae..37c07a52 100755 --- a/tests/xfs/091 +++ b/tests/xfs/091 @@ -80,7 +80,7 @@ echo "+ mount image && modify files" if _try_scratch_mount >> $seqres.full 2>&1; then for x in `seq 1 64`; do - $XFS_IO_PROG -f -c "pwrite -S 0x62 0 ${blksz}" "${TESTFILE}.${x}" >> $seqres.full + $XFS_IO_PROG -f -c "pwrite -S 0x62 0 ${blksz}" "${TESTFILE}.${x}" >> $seqres.full 2>> $seqres.full done umount "${SCRATCH_MNT}" fi From patchwork Tue Oct 8 01:03:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 11178671 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 40CA71864 for ; Tue, 8 Oct 2019 01:03:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1EB7A21721 for ; Tue, 8 Oct 2019 01:03:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="R20yYBEX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729389AbfJHBDa (ORCPT ); Mon, 7 Oct 2019 21:03:30 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:54688 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728983AbfJHBDa (ORCPT ); Mon, 7 Oct 2019 21:03:30 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x980wsr2035959; Tue, 8 Oct 2019 01:03:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2019-08-05; bh=c6vWHuXEDZChkRQPIhyMpVUSekMEbG5U0+AIqLGiR0k=; b=R20yYBEXn/BNvPcNDjS2UCf4IsRwhfJCMptqWJUorhqe6hxNlI5jmSKk3m9HPBhpIxC1 AH8rKJ/cEBieZbHuU987oh276H08mjWJ1WmXpMaogGwMJRruRtywC4ZSQ+h9et7duSS/ /6OWnWDHZIB/T6ZKb8QastoPf2L9dzDk+dIUpogmOJ183fZ4Mpn71TXcAKewyKietVNc iSp7etOSsVhx57czxHRDxMWgAq8a5QioA43/I7yIX5eNt19pcVtE/LO1/2WbIX3bhXPw EwVL0nKfAqTZS3a8rCY5i0PXK8Z+QM6DPhX+KOknulzF/GA3vHT1BeeBhGY0Pd8fQUl7 VQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2130.oracle.com with ESMTP id 2vejkua6am-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 08 Oct 2019 01:03:28 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x980wD3U007437; Tue, 8 Oct 2019 01:03:27 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3030.oracle.com with ESMTP id 2vg1yuy2sm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 08 Oct 2019 01:03:27 +0000 Received: from abhmp0014.oracle.com (abhmp0014.oracle.com [141.146.116.20]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x9813OtL011164; Tue, 8 Oct 2019 01:03:25 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 07 Oct 2019 18:03:24 -0700 Subject: [PATCH 3/4] xfs/263: use _scratch_mkfs_xfs instead of open-coded mkfs call From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Mon, 07 Oct 2019 18:03:23 -0700 Message-ID: <157049660366.2397321.3207595496710777905.stgit@magnolia> In-Reply-To: <157049658503.2397321.13914737091290093511.stgit@magnolia> References: <157049658503.2397321.13914737091290093511.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9403 signatures=668684 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=989 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910080009 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9403 signatures=668684 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910080009 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Fix this test to use _scratch_mkfs_xfs instead of the open-coded mkfs call. This is needed to make the test succeed when XFS DAX is enabled and mkfs enables reflink by default. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- tests/xfs/263 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/xfs/263 b/tests/xfs/263 index 75477937..578f9ee7 100755 --- a/tests/xfs/263 +++ b/tests/xfs/263 @@ -75,11 +75,11 @@ function test_all_state() echo "==== NO CRC ====" # Control size to control inode numbers -$MKFS_XFS_PROG -f -m crc=0 -n ftype=0 -d size=512m $SCRATCH_DEV >>$seqres.full +_scratch_mkfs_xfs "-m crc=0 -n ftype=0 -d size=512m" >> $seqres.full test_all_state echo "==== CRC ====" -$MKFS_XFS_PROG -f -m crc=1 -d size=512m $SCRATCH_DEV >>$seqres.full +_scratch_mkfs_xfs "-m crc=1 -d size=512m" >>$seqres.full test_all_state status=0 From patchwork Tue Oct 8 01:03:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 11178675 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE8071864 for ; Tue, 8 Oct 2019 01:03:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9ECF321721 for ; Tue, 8 Oct 2019 01:03:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="YJBhpsa/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729413AbfJHBDf (ORCPT ); Mon, 7 Oct 2019 21:03:35 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:58044 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728983AbfJHBDf (ORCPT ); Mon, 7 Oct 2019 21:03:35 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x980xM21190786; Tue, 8 Oct 2019 01:03:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2019-08-05; bh=gHQY9KWNJbUZfPmD+jF+V2FHrj0k0SEVSRngRx/Odvg=; b=YJBhpsa/ZLFrO/S9dJihonGuqvwpJsODNGO83b/LtwbcWxpq2Umqpsuv0MXu/cFJyzH5 FsJVwtNLmASI74LfANSTw4KlqbT1E6hCbGk98+iyZDrvGwcyAS/dGBxkXD8YZor5vN90 qRwkIj8ZkHVc2iRxKqBtzpqLL1CkgIIFS0+QNGzsBN21BWUocvLhoW1SdnIjRVUM07FL TXbydBqwahaNOhXM+noawh0RpOWv/gfJ8Zb6tW0nGwn7/XWGs2KqUVAueCSYOaxdtr5f pRjU41gUEc+BexBKTGReUzCz+Rlyw3Xli0DIC1tryckSVEvgDYyfYt1XWx2jlfiUZ76U aw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 2vek4qa4gd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 08 Oct 2019 01:03:32 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x980wD2j007450; Tue, 8 Oct 2019 01:03:32 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3030.oracle.com with ESMTP id 2vg1yuy2wg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 08 Oct 2019 01:03:32 +0000 Received: from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x9813V9h011192; Tue, 8 Oct 2019 01:03:31 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 08 Oct 2019 01:03:30 +0000 Subject: [PATCH 4/4] populate: punch files after writing to fragment free space properly From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Mon, 07 Oct 2019 18:03:29 -0700 Message-ID: <157049660991.2397321.6295105033631507023.stgit@magnolia> In-Reply-To: <157049658503.2397321.13914737091290093511.stgit@magnolia> References: <157049658503.2397321.13914737091290093511.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9403 signatures=668684 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910080009 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9403 signatures=668684 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910080009 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong The filesystem population code frequently allocates a large file and punches out every other block ("swiss-cheese files") in an attempt to cause the creation of a lot of metadata to fill out the btrees. This pattern, however, has a subtle bug if the writes to the swiss-cheese file are not allocated in batches and we're trying to fragment the free space records in order to achieve a certain metadata btree shape. This is exactly what happens on a DAX filesystem, since we no longer have the page cache to stage delalloc writes. Each xfs_io pwrite call to the multi-megabyte swiss-chese file turns into multiple 4k pwrites, which means that file data blocks are allocated 4k at a time. This can be fatal to our goal of fragmenting the free space btrees because the allocator sees a 4k allocation request and uses 4k blocks from the fragmented parts of the free space to satisfy the "small" request. When this happens, the XFS populate function cannot fill out the free space btree to sufficient height and tests fail. (In regular delalloc mode we'd cache all those small write() in memory and try for a single large allocation, which we'd generally get.) To fix this, we need to force the filesystem to allocate all blocks before freeing any blocks. Split the creation of swiss-cheese files into two parts: (a) writing data to the file to force allocation, and (b) punching the holes to fragment free space. It's a little hokey for helpers to be modifying variables in the caller's scope, but there's not really a better way to do that in bash. This bug affects only XFS but we convert the one ext4 usage anyway. Signed-off-by: Darrick J. Wong --- common/populate | 54 ++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 42 insertions(+), 12 deletions(-) diff --git a/common/populate b/common/populate index 7403dec3..4aab0274 100644 --- a/common/populate +++ b/common/populate @@ -120,6 +120,32 @@ _populate_xfs_qmount_option() fi } +# Set up a file that we'll later use to fragment metadata and free space. +# Our strategy here is to force the fs to allocate large contiguous extents +# to a file and then punch every other block to force the fs to suffer the +# worst allocation outcome possible. +# +# NOTE: In order to prevent *subsequent* allocations from using the holes we +# punch, we must store the relevant filenames for later. This function +# deliberately adds each file name to the @punch_files array, which must be +# declared by the caller and will be picked up by __force_fragmentation. +function __setup_fragmentation() { + local sz="$1" + local fname="$2" + + $XFS_IO_PROG -f -c "pwrite -S 0x62 -W -b 1m 0 $sz" "${fname}" + punch_files+=("${fname}") +} + +# Actually punch holes in the file. Call this /after/ you're done calling +# __setup_fragmentation. +__force_fragmentation() { + for file in "${punch_files[@]}"; do + ./src/punch-alternating "${file}" + done + punch_files=() +} + # Populate an XFS on the scratch device with (we hope) all known # types of metadata block _scratch_xfs_populate() { @@ -132,6 +158,8 @@ _scratch_xfs_populate() { esac done + local punch_files=() + _populate_xfs_qmount_option _scratch_mount blksz="$(stat -f -c '%s' "${SCRATCH_MNT}")" @@ -161,8 +189,7 @@ _scratch_xfs_populate() { # - FMT_BTREE echo "+ btree extents file" nr="$((blksz * 2 / 16))" - $XFS_IO_PROG -f -c "pwrite -S 0x62 0 $((blksz * nr))" "${SCRATCH_MNT}/S_IFREG.FMT_BTREE" - ./src/punch-alternating "${SCRATCH_MNT}/S_IFREG.FMT_BTREE" + __setup_fragmentation $((blksz * nr)) "${SCRATCH_MNT}/S_IFREG.FMT_BTREE" # Directories # - INLINE @@ -257,8 +284,7 @@ _scratch_xfs_populate() { # Free space btree echo "+ freesp btree" nr="$((blksz * 2 / 8))" - $XFS_IO_PROG -f -c "pwrite -S 0x62 0 $((blksz * nr))" "${SCRATCH_MNT}/BNOBT" - ./src/punch-alternating "${SCRATCH_MNT}/BNOBT" + __setup_fragmentation $((blksz * nr)) "${SCRATCH_MNT}/BNOBT" # Inode btree echo "+ inobt btree" @@ -280,8 +306,7 @@ _scratch_xfs_populate() { if [ $is_rmapbt -gt 0 ]; then echo "+ rmapbt btree" nr="$((blksz * 2 / 24))" - $XFS_IO_PROG -f -c "pwrite -S 0x62 0 $((blksz * nr))" "${SCRATCH_MNT}/RMAPBT" - ./src/punch-alternating "${SCRATCH_MNT}/RMAPBT" + __setup_fragmentation $((blksz * nr)) "${SCRATCH_MNT}/RMAPBT" fi # Realtime Reverse-mapping btree @@ -289,8 +314,7 @@ _scratch_xfs_populate() { if [ $is_rmapbt -gt 0 ] && [ $is_rt -gt 0 ]; then echo "+ rtrmapbt btree" nr="$((blksz * 2 / 32))" - $XFS_IO_PROG -f -R -c "pwrite -S 0x62 0 $((blksz * nr))" "${SCRATCH_MNT}/RTRMAPBT" - ./src/punch-alternating "${SCRATCH_MNT}/RTRMAPBT" + __setup_fragmentation $((blksz * nr)) "${SCRATCH_MNT}/RTRMAPBT" fi # Reference-count btree @@ -298,15 +322,17 @@ _scratch_xfs_populate() { if [ $is_reflink -gt 0 ]; then echo "+ reflink btree" nr="$((blksz * 2 / 12))" - $XFS_IO_PROG -f -c "pwrite -S 0x62 0 $((blksz * nr))" "${SCRATCH_MNT}/REFCOUNTBT" + __setup_fragmentation $((blksz * nr)) "${SCRATCH_MNT}/REFCOUNTBT" cp --reflink=always "${SCRATCH_MNT}/REFCOUNTBT" "${SCRATCH_MNT}/REFCOUNTBT2" - ./src/punch-alternating "${SCRATCH_MNT}/REFCOUNTBT" fi # Copy some real files (xfs tests, I guess...) echo "+ real files" test $fill -ne 0 && __populate_fill_fs "${SCRATCH_MNT}" 5 + # Make sure we get all the fragmentation we asked for + __force_fragmentation + umount "${SCRATCH_MNT}" } @@ -322,6 +348,8 @@ _scratch_ext4_populate() { esac done + local punch_files=() + _scratch_mount blksz="$(stat -f -c '%s' "${SCRATCH_MNT}")" dblksz="${blksz}" @@ -342,8 +370,7 @@ _scratch_ext4_populate() { # - FMT_ETREE echo "+ extent tree file" nr="$((blksz * 2 / 12))" - $XFS_IO_PROG -f -c "pwrite -S 0x62 0 $((blksz * nr))" "${SCRATCH_MNT}/S_IFREG.FMT_ETREE" - ./src/punch-alternating "${SCRATCH_MNT}/S_IFREG.FMT_ETREE" + __setup_fragmentation $((blksz * nr)) "${SCRATCH_MNT}/S_IFREG.FMT_ETREE" # Directories # - INLINE @@ -406,6 +433,9 @@ _scratch_ext4_populate() { echo "+ real files" test $fill -ne 0 && __populate_fill_fs "${SCRATCH_MNT}" 5 + # Make sure we get all the fragmentation we asked for + __force_fragmentation + umount "${SCRATCH_MNT}" }