From patchwork Thu Sep 22 05:54:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 12984600 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3291C6FA8B for ; Thu, 22 Sep 2022 05:55:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229603AbiIVFzm (ORCPT ); Thu, 22 Sep 2022 01:55:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229904AbiIVFzk (ORCPT ); Thu, 22 Sep 2022 01:55:40 -0400 Received: from esa1.hgst.iphmx.com (esa1.hgst.iphmx.com [68.232.141.245]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE902B4420; Wed, 21 Sep 2022 22:55:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1663826139; x=1695362139; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fuX+N/jPRiLEpVvBWbWnHHkQ24UIw9RfyIEX4XwB7qo=; b=jbagDYYDrOoRmvFVLNOHqZng9KmfZF0t7LX4JRY4TNv1twxLIeMq8w+p 3d80Jc7Hh1A6Ou2uJwHjv3EUccEAVW4dInUAYVrmhhL+nnmCmVAvc6COI KDnMpFig/AnUyguwGq+Hj7sIZZJeKiiy/56JqKfPLFztXAnfmoVRM3ATl Pu1cgd5mTC/Aay6cVBKh+zU14XmtdvyNNUkY6bgCE5d7EEKkPp9GdR1Oi j5zbohlvBLg58LECwNCh/fZ6FdrIkaVXoClwVSyWGaH+b4W9nqVtUGiWi re5Djuwrtz8fJrIlwtg5ZsS84O8uIj0PdFncvcbkPuebiWnPxgcUvoWZS g==; X-IronPort-AV: E=Sophos;i="5.93,335,1654531200"; d="scan'208";a="324089196" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 22 Sep 2022 13:55:39 +0800 IronPort-SDR: Mi1gDXmM48GU1xSIZFx0yYUgCnVIggG1JAS2YBA3MdCxS/T79Yg+Nk1ZfZoM/0Wey5JWVOp/Le N74rE5oJXS1T/wKtxrbUBUdC4ktR5KiDAL7uN1LvU6vvnEI0Pn4hR5ppINgJmJfmp5Z78aCG+q JBeGS2R/zSqUXO6wOennC2t1W+FfMp211SKv3gNuUWhbT4rooAqyQtA1qcB7gw65WKIaQHsGLk WTAeE8BjVbqiSPmDva86hJ6Otd6u+4gm4SvlQiB1n0zfKuKnDitkZZcvvALnEVDe8hFeG+EHJ6 AcyvFO1gG/LGUzITLTGvHl4E Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 21 Sep 2022 22:10:13 -0700 IronPort-SDR: j2idZ8wu9C8ck77ehJDkMTUjYtjy6/ipMS3OlGFq8AL+MvqmrXTQME/Zb6bOJO+UqGgfv57QBX sH1NKKwEMxh9arSw1QpCzlTleaE3H+8U2TFaQk6K4OxJLw/yyHZIsYFEUGw7Ia3mSyJy9EYekS InyCAEv73naKXMhI370SK0KqeTz6q1c/71kS0wQzpz68MWnGrDJYIbcAkuCq8VCpNzdMiVy2GE uwzjf3rHAeyW7u8fnDDwoMLzNV4MOK+ngJ47xq2oST4ERV0jOV/sA/Vot1i11+kRave6qMYxHK rrQ= WDCIronportException: Internal Received: from ghkxqv2.ad.shared (HELO naota-xeon.wdc.com) ([10.225.49.86]) by uls-op-cesaip01.wdc.com with ESMTP; 21 Sep 2022 22:55:39 -0700 From: Naohiro Aota To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Naohiro Aota Subject: [PATCH 1/2] common: introduce zone_capacity() to return a zone capacity Date: Thu, 22 Sep 2022 14:54:58 +0900 Message-Id: <97ede9bba67f0848fc0b706d757170d7dfacb7fd.1663825728.git.naohiro.aota@wdc.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org Introduce _zone_capacity() to return a zone capacity of the given address in the given device (optional). Rewrite btrfs/237 with it. Signed-off-by: Naohiro Aota --- common/zbd | 17 +++++++++++++++++ tests/btrfs/237 | 8 ++------ 2 files changed, 19 insertions(+), 6 deletions(-) create mode 100644 common/zbd diff --git a/common/zbd b/common/zbd new file mode 100644 index 000000000000..329bb7be6b7b --- /dev/null +++ b/common/zbd @@ -0,0 +1,17 @@ +# +# Common zoned block device specific functions +# + +. common/filter + +_zone_capacity() { + local phy=$1 + local dev=$2 + + [ -z "$dev" ] && dev=$SCRATCH_DEV + + size=$($BLKZONE_PROG report -o $phy -l 1 $dev |\ + _filter_blkzone_report |\ + grep -Po "cap 0x[[:xdigit:]]+" | cut -d ' ' -f 2) + echo $((size << 9)) +} diff --git a/tests/btrfs/237 b/tests/btrfs/237 index bc6522e2200a..101094b5ce70 100755 --- a/tests/btrfs/237 +++ b/tests/btrfs/237 @@ -13,7 +13,7 @@ _begin_fstest auto quick zone balance # Import common functions. -. ./common/filter +. ./common/zbd # real QA test starts here @@ -56,11 +56,7 @@ fi start_data_bg_phy=$(get_data_bg_physical) start_data_bg_phy=$((start_data_bg_phy >> 9)) - -size=$($BLKZONE_PROG report -o $start_data_bg_phy -l 1 $SCRATCH_DEV |\ - _filter_blkzone_report |\ - grep -Po "cap 0x[[:xdigit:]]+" | cut -d ' ' -f 2) -size=$((size << 9)) +size=$(_zone_capacity $start_data_bg_phy) reclaim_threshold=75 echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold From patchwork Thu Sep 22 05:54:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Aota X-Patchwork-Id: 12984601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C2E5C6FA82 for ; Thu, 22 Sep 2022 05:55:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229797AbiIVFzr (ORCPT ); Thu, 22 Sep 2022 01:55:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229734AbiIVFzp (ORCPT ); Thu, 22 Sep 2022 01:55:45 -0400 Received: from esa1.hgst.iphmx.com (esa1.hgst.iphmx.com [68.232.141.245]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 089D1B4422; Wed, 21 Sep 2022 22:55:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1663826141; x=1695362141; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fIYnDrM5zolSy4IWbZU17+zV+GczOAiQsjZcy2FcsWg=; b=HnARpTIjSuIFtp7qvB6g0O+shKphQUi7XdFsQ7ULDrwDbbSF0dnwm+M+ MSl25rcvOiqgh2BH4fd3ui+C9igZS4qz4wWLULcTGcQyGlBYUEtrFFWcD YzW5RCYtz1CY7kMEGd/Nph5y+mmgv65mXCrJ9WxRMwIuTTaoYubWGL/9X PNpjWVhT+iwNnBEMM7DDxC/FsuPx8ljPEva3E1Ytz2cAhxZCyM/xoncCJ 1AkMmfFEmF/opk2UgpQuVW0AJWBcl3VCr38mezgaEZl3udqF6TGVgVu7A uYbD77QtGnQCfAo3nvpaadEJihkA0+YDCs/sZWjV16dvLRZHFRRyarAaE g==; X-IronPort-AV: E=Sophos;i="5.93,335,1654531200"; d="scan'208";a="324089197" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 22 Sep 2022 13:55:41 +0800 IronPort-SDR: F4+jEYCSoL0+u1GPp4+glXd2sdtYmuYV5fhUVPVfP/76ERUBqy2ht8w1BHPZ03oLslCDa4yyyw lnihUCxV8/xRqaFZMxAZkf3bFtuDFS+SyJWd+cpXOj1r7eaHEhsH5lT8MbJWZmXCLnAOq1yAmw NSDaZ7xqb8XMv2MpiUzSsltN6t4ERS6DNSqKMV2Buv9uhE2OkmdjQ4CwhD0SNQDy+EwCQHSAna wYPQRopjEcjZSLoiL7BXfZFRSgiKRmrch2QB4uj/LO81icSaxPwFzaMpFx9nit74dx7sERWyQA IXDIvcikHUM6y6HNltU14yJr Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 21 Sep 2022 22:10:15 -0700 IronPort-SDR: XxwYQiJv/j9u6BtrArOeh9D63r6S9Uru5JmM6WxdgIAWHSlV+qBCEBQN5VFfCB8Y96vqQE/1YA 9hoCwJQVpAkq7b6/0zNozWTQAjHVxyNXiJmdrxGkQk7BXBm3GQlVmmzp/bn8p9qnCtY34SydJf L2v7DWaDRJ8LPZRm5vtz5YkYinZ8jL50cTXxVZYqpEgNUPFzqyomyu16OUTYzJMPcpy+MGXzFu 7j1+58roLeD6Se4oBAooQ3KUMh5/7Q+4k6qMHJ3tHzT8Xshe6yVaXRl7qzK4Y3jZXC5eyHD9NR Xw8= WDCIronportException: Internal Received: from ghkxqv2.ad.shared (HELO naota-xeon.wdc.com) ([10.225.49.86]) by uls-op-cesaip01.wdc.com with ESMTP; 21 Sep 2022 22:55:41 -0700 From: Naohiro Aota To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Naohiro Aota Subject: [PATCH 2/2] btrfs: test active zone tracking Date: Thu, 22 Sep 2022 14:54:59 +0900 Message-Id: <7e30a693ce98783d68bc70d07c185bffc693a4d1.1663825728.git.naohiro.aota@wdc.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org A ZNS device limits the number of active zones, which is the number of zones can be written at the same time. To deal with the limit, btrfs's zoned mode tracks which zone (corresponds to a block group on the SINGLE profile) is active, and finish a zone if necessary. This test checks if the active zone tracking and the finishing of zones works properly. First, it fills zones mostly. And, run some data/metadata stress workload to force btrfs to use a new zone. This test fails on an older kernel (e.g, 5.18.2) like below. btrfs/292 [failed, exit status 1]- output mismatch (see /host/btrfs/292.out.bad) --- tests/btrfs/292.out 2022-09-15 07:52:18.000000000 +0000 +++ /host/btrfs/292.out.bad 2022-09-15 07:59:14.290967793 +0000 @@ -1,2 +1,5 @@ QA output created by 292 -Silence is golden +stress_data_bgs failed +stress_data_bgs_2 failed +failed: '/bin/btrfs subvolume snapshot /mnt/scratch /mnt/scratch/snap825' +(see /host/btrfs/292.full for details) ... (Run 'diff -u /var/lib/xfstests/tests/btrfs/292.out /host/btrfs/292.out.bad' to see the entire diff) The failure is fixed with a series "btrfs: zoned: fix active zone tracking issues" [1] (upstream commits from 65ea1b66482f ("block: add bdev_max_segments() helper") to 2ce543f47843 ("btrfs: zoned: wait until zone is finished when allocation didn't progress")). [1] https://lore.kernel.org/linux-btrfs/cover.1657321126.git.naohiro.aota@wdc.com/ Signed-off-by: Naohiro Aota --- common/zbd | 12 ++++ tests/btrfs/292 | 137 ++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/292.out | 2 + 3 files changed, 151 insertions(+) create mode 100755 tests/btrfs/292 create mode 100644 tests/btrfs/292.out diff --git a/common/zbd b/common/zbd index 329bb7be6b7b..19a59fd27e69 100644 --- a/common/zbd +++ b/common/zbd @@ -2,8 +2,20 @@ # Common zoned block device specific functions # +. common/rc . common/filter +_require_limited_active_zones() { + local dev=$1 + local sysfs=$(_sysfs_dev ${dev}) + local attr="${sysfs}/queue/max_active_zones" + + [ -e "${attr}" ] || _notrun "cannot find queue/max_active_zones. Maybe non-zoned device?" + if [ $(cat "${attr}") == 0 ]; then + _notrun "this test requires limited active zones" + fi +} + _zone_capacity() { local phy=$1 local dev=$2 diff --git a/tests/btrfs/292 b/tests/btrfs/292 new file mode 100755 index 000000000000..30c027c4ba5f --- /dev/null +++ b/tests/btrfs/292 @@ -0,0 +1,137 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Western Digital Corporation. All Rights Reserved. +# +# FS QA Test 292 +# +# Test that an active zone is properly reclaimed to allow the further +# allocations, even if the active zones are mostly filled. +# +. ./common/preamble +_begin_fstest auto quick snapshot zone + +# Import common functions. +. ./common/btrfs +. ./common/zbd + +# real QA test starts here + +_supported_fs btrfs +_require_scratch +_require_zoned_device "$SCRATCH_DEV" +_require_limited_active_zones "$SCRATCH_DEV" + +_require_command "$BLKZONE_PROG" blkzone +_require_fio +_require_btrfs_command inspect-internal dump-tree + +# This test requires specific data space usage, skip if we have compression +# enabled. +_require_no_compress + +max_active=$(cat $(_sysfs_dev ${SCRATCH_DEV})/queue/max_active_zones) + +# Fill the zones leaving the last 1MB +fill_active_zones() { + # Asuumes we have the same capacity between zones. + local capacity=$(_zone_capacity 0) + local fill_size=$((capacity - 1024 * 1024)) + + for x in $(seq ${max_active}); do + dd if=/dev/zero of=${SCRATCH_MNT}/fill$(printf "%02d" $x) bs=${fill_size} \ + count=1 oflag=direct 2>/dev/null + $BTRFS_UTIL_PROG filesystem sync ${SCRATCH_MNT} + + local nactive=$($BLKZONE_PROG report ${SCRATCH_DEV} | grep oi | wc -l) + if [[ ${nactive} == ${max_active} ]]; then + break + fi + done + + echo "max active zones: ${max_active}" >> $seqres.full + $BLKZONE_PROG report ${SCRATCH_DEV} | grep oi | cat -n >> $seqres.full +} + +workout() { + local func="$1" + + _scratch_mkfs >/dev/null 2>&1 + _scratch_mount + + fill_active_zones + eval "$func" + local ret=$? + + _scratch_unmount + _check_btrfs_filesystem ${SCRATCH_DEV} + + return $ret +} + +stress_data_bgs() { + # This dd fails with ENOSPC, which should not :( + dd if=/dev/zero of=${SCRATCH_MNT}/large bs=64M count=1 oflag=sync \ + >>$seqres.full 2>&1 +} + +stress_data_bgs_2() { + # This dd fails with ENOSPC, which should not :( + dd if=/dev/zero of=${SCRATCH_MNT}/large bs=64M count=10 conv=fsync \ + >>$seqres.full 2>&1 & + local pid1=$! + + dd if=/dev/zero of=${SCRATCH_MNT}/large2 bs=64M count=10 conv=fsync \ + >>$seqres.full 2>&1 & + local pid2=$! + + wait $pid1; local ret1=$? + wait $pid2; local ret2=$? + + if [ $ret1 -ne 0 -o $ret2 -ne 0 ]; then + return 1 + fi + return 0 +} + +get_meta_bgs() { + $BTRFS_UTIL_PROG inspect-internal dump-tree -t EXTENT ${SCRATCH_DEV} | + grep BLOCK_GROUP -A 1 |grep -B1 'METADATA|' | + grep -oP '\(\d+ BLOCK_GROUP_ITEM \d+\)' +} + +# This test case does not return the result because +# _run_btrfs_util_prog will call _fail() in the error case anyway. +stress_metadata_bgs() { + local metabgs=$(get_meta_bgs) + local count=0 + + while : ; do + _run_btrfs_util_prog subvolume snapshot ${SCRATCH_MNT} ${SCRATCH_MNT}/snap$i + _run_btrfs_util_prog filesystem sync ${SCRATCH_MNT} + cur_metabgs=$(get_meta_bgs) + if [[ "${cur_metabgs}" != "${metabgs}" ]]; then + break + fi + i=$((i + 1)) + done +} + +WORKS=( + stress_data_bgs + stress_data_bgs_2 + stress_metadata_bgs +) + +status=0 +for work in "${WORKS[@]}"; do + if ! workout "${work}"; then + echo "${work} failed" + status=1 + fi +done + +# success, all done +if [ $status -eq 0 ]; then + echo "Silence is golden" +fi +exit diff --git a/tests/btrfs/292.out b/tests/btrfs/292.out new file mode 100644 index 000000000000..627309d3fbd2 --- /dev/null +++ b/tests/btrfs/292.out @@ -0,0 +1,2 @@ +QA output created by 292 +Silence is golden