From patchwork Wed Jan 15 05:51:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 13939858 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 540FC22DFBA for ; Wed, 15 Jan 2025 06:03:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736920987; cv=none; b=VxYQ1DHsQScyZUUvjMhK/wK31ZKQVVmkJvP94SFQscmNqJqBL1cnaboOvFQ3G0lIsvgzoNsuNX2LCWIsscKg/g23WvCCkd3sKsPeYFCvQx1RLKGdRItYVZi3IcZV9kZMKIWbHizLjh8ejlvC1mVruNeySW71WJGv1oeEQeQf6ZY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736920987; c=relaxed/simple; bh=H8p6u0E+Ozv4HyVMTmrHKteOF6IlF+1nwDowI5wjGqs=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=btmoNvhnfoBntzfeCpndF8oe+J4AJTSo8O+8avPs+Q/DVwZrS8T9Hz0U5DxmHszHOLLAmqqUHEdWEmoIgaCe/v10mA1xt0sV6qmP3qBvHJj8FVoR3ZWJZod+fhZOrhOILcNAb2gMDcbqzI3rJYzKcpO01VE98hWCSYG7cqA3ZYQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fromorbit.com; spf=pass smtp.mailfrom=fromorbit.com; dkim=pass (2048-bit key) header.d=fromorbit-com.20230601.gappssmtp.com header.i=@fromorbit-com.20230601.gappssmtp.com header.b=GJqN/eND; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fromorbit.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fromorbit.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fromorbit-com.20230601.gappssmtp.com header.i=@fromorbit-com.20230601.gappssmtp.com header.b="GJqN/eND" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-21680814d42so93887115ad.2 for ; Tue, 14 Jan 2025 22:03:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20230601.gappssmtp.com; s=20230601; t=1736920985; x=1737525785; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=jZ518HjSKouwTknISSUOgzctwWvRO5yFxjmqsxs/VPU=; b=GJqN/eNDLu6NftoIORoD17L0LPJmBjYqgpw5kf97DEeKGcvjj5nE6Y1qTZeBm1YMzt Gvnz8TZ/kP8QRUqfiJGwav+S6DiCVS8OOpsE2hLiCAekrlFq3QicYbmFwHiZsjvm2el3 RqwI2b0Gl4HDWGqmYBHm7+KfL9Z1We5l6Oj5bQEHfaJJruo6Jj10WtJyPjOBmF+YOKuf JVr9ORIo2soc6SiuWQ9vtea2CMfgj40kuLFdbXP2R3h3iyxPBtJAFssGS9XQfzxOXZzX i7MD1e6W4IhGurVXZgerdWuPbO+WrtU3bNG/taHoaXj400I9NAqDQrDlSM7L2FEn5P/y F+lQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736920985; x=1737525785; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jZ518HjSKouwTknISSUOgzctwWvRO5yFxjmqsxs/VPU=; b=hY5UuFKRwW/FngqUMRIDnOFq3y54h/qK0KvnlrYVMkh90ewFW4Qsc7XuAOqn3l7Ell KVuj03k9JMT2zO/xg2nqN/At2Ro/1X8xsysAMVGNxOy/zhw/bt05aGRsocscgW6eEhGF NVoGkGBFS7Gxcq5ZGGq/X6yeKt8WGojBLaoBFFCg9VQXnP3t2CMqL/nZAXzeM8qUeN54 UYoPMh3nq92BTsoP9POY1dcA/1hOT8SJZ5bO0fjFBoNY5fZ3Pd4auJ6bpSfw8v1dAomI UZyRGjeEb8yaOnjQt7ZLNvZthCdEg9Hh04x2YWuvWVsULGIWpKscVrBnzZKGaPRRYmvA Mbyg== X-Gm-Message-State: AOJu0YwQyDYrrXKHjXmNE4DLTssFruQ61iQqN3y+ZahUWVC4Ur/z4/qM 8KezpFQdnxieDLzSgf91Uj/nltxpMxHtqauWporTjdxn8uooY/Z5n1UB1m1+jB9qyw36DQPdUJM y X-Gm-Gg: ASbGncshrr9VGe+4fVAYMWv6yoOHKdXyL+g8MavRFSb+7S8zo6Cvwd7mMpkPsVxPX3v pz/nkz2L2YNxFCrOei9JiNPb09pCpHO+jg/q9W3lWHl05KjzOL2GMlOazykkxAHnIPeECmf8Zf5 v2piQ5jChpdsaDm/qYJE8h4wy96j6OsmnHrtdisFTwHrycdfwi7XU4jU3Ax2v1YfU0R8AWBRZh4 MyFz8T+sq8hNl9TqTkMbuH9EqfPWM6ZoMRTG9hSLNQEAqqUxxmhdDhv/vOpPJ/AedU3Bb8fazIC pJFraI/i2+4Fw+lInwPUjpjkfx8RrEc= X-Google-Smtp-Source: AGHT+IGG0h0fLggOIMIXJpi/LTjLSzOBOPu1wxCu4xk0xZfj8MgKQl4S6WnxoC7Q6wMMZTm78CENWw== X-Received: by 2002:a17:902:d485:b0:216:32c4:f7f5 with SMTP id d9443c01a7336-21a83f54d55mr405464985ad.19.1736920985613; Tue, 14 Jan 2025 22:03:05 -0800 (PST) Received: from dread.disaster.area (pa49-186-89-135.pa.vic.optusnet.com.au. [49.186.89.135]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-a319a125fe4sm7794144a12.46.2025.01.14.22.03.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jan 2025 22:03:03 -0800 (PST) Received: from [192.168.253.23] (helo=devoid.disaster.area) by dread.disaster.area with esmtp (Exim 4.98) (envelope-from ) id 1tXwUW-0000000641V-1Zf2 for fstests@vger.kernel.org; Wed, 15 Jan 2025 17:03:00 +1100 Received: from dave by devoid.disaster.area with local (Exim 4.98) (envelope-from ) id 1tXwUW-0000000GabK-2QXl for fstests@vger.kernel.org; Wed, 15 Jan 2025 17:03:00 +1100 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 1/5] check-parallel: adjust concurrency according to CPU count Date: Wed, 15 Jan 2025 16:51:12 +1100 Message-ID: <20250115060258.3951185-2-david@fromorbit.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115060258.3951185-1-david@fromorbit.com> References: <20250115060258.3951185-1-david@fromorbit.com> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Dave Chinner COncurrency is currently hard coded at 64 worker threads. THis is too many for small CPU count machines; the idea is to create a sustained load of roughly one test per CPU as they are mostly single threaded/single process tests. The number "64" was chosen because I've been developing this functionality on a 64p VM. Rather than hard coding the concurrency, probe the number of CPUs available and create that many running contexts. Signed-off-by: Dave Chinner --- check-parallel | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/check-parallel b/check-parallel index c85437252..d34c73f66 100755 --- a/check-parallel +++ b/check-parallel @@ -13,7 +13,7 @@ export SRC_DIR="tests" basedir=$1 shift check_args="$*" -runners=64 +runners=$(getconf _NPROCESSORS_CONF) runner_list=() runtimes=() From patchwork Wed Jan 15 05:51:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 13939860 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D5CDA22E419 for ; Wed, 15 Jan 2025 06:03:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736920988; cv=none; b=e6zDSnjOjH018BFNqZJ/JG/EOLWkdI7AUHg3HVStRhVl54iTpOKfWxSOBAzS+d66bGJNnU9zs+nRJzFelCwYB7x0nHVtvbT7ZMHfVCIlLicAnj0afmaU7Cp+pa0SimqaIQhdFaoWooZGiRyFIlbxRH0wUt5WEP6BOJkho8akpjs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736920988; c=relaxed/simple; bh=qK0xXXy6mlcIhjTu+n2GKINd1vjba9DuKS7Bh1j9DLw=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XkpHGvTIBzzvdFisv+6S+hpOzh2DEpMiiVn1+7ImYwEmvl0KjGCifYGWXRLrZ4XWdJdrD/84DCX4tc53ps5r+6ac64iX4trGjQJggWsln75RWzM8XeGOYXHinLb1h9MP8fcxc2pwjvj+bLqo7o/1wnQAP1iFx1lHpq/QBKuvNGY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fromorbit.com; spf=pass smtp.mailfrom=fromorbit.com; dkim=pass (2048-bit key) header.d=fromorbit-com.20230601.gappssmtp.com header.i=@fromorbit-com.20230601.gappssmtp.com header.b=bhoLyPT6; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fromorbit.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fromorbit.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fromorbit-com.20230601.gappssmtp.com header.i=@fromorbit-com.20230601.gappssmtp.com header.b="bhoLyPT6" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2166651f752so142562645ad.3 for ; Tue, 14 Jan 2025 22:03:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20230601.gappssmtp.com; s=20230601; t=1736920985; x=1737525785; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=SFk1FWouxMjeQQYaCjTdkGov0cv5f8Ey4Cb2GqdivKo=; b=bhoLyPT6d3NQVINIv37Ie8K7eysF0QbPaR5wbw60BvLL901e0mcEvqfDc/ehnSZ7Va nuQf0eH6CE2VASoH+WJ7ya2DEtM15wKlQc7IYdsu0BqmeUDNsriAueyqDt7wb+TWtNkj hB+UBTUF1TKS6R5MvvKXnFmZip+3QJt6gryP7nJ6WH9EOAB2YbNVpIQKCUTMM4iDuSZA x9YaKohxAuYNkZKgrmlyUoPcpSSzyPsyNgXWuAMYrO9KaXYnErpftzKfNkuuZX3MMc6k cA0v75fsVJQmH+W6im6aO2zwaAF0t5LxodkIhs50U66TG9xcS57yDOB2oeVpqlQmyFXG qkFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736920985; x=1737525785; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SFk1FWouxMjeQQYaCjTdkGov0cv5f8Ey4Cb2GqdivKo=; b=GisXcSHY/b7b2HDhNpcLdX5kZLQgqoVOU6z2+QHG0itoWBpqVAHBgUHmVB/bWkUQQB mctMATeiR78Qc2tMv94rLtTkeJ7fYfMsKmN7jql14lInkG5W5zhrd9waQRbDvu0mYJCB C8MFiBM2DJRyV4sdE+JBNc7ImXE63Jr0IbWTSAPiFrdMeuTLZGFnnAk/Mgh/IXJNZ0OQ njcSLzfdFCp2TgLs7lpDmQgKzgxCyBXYo8uN6xna6TEZTu1sxT0m04pXJIxy5YKR6yKW lKnjigFDrVRFuGbKoinNWSlE8vLaEMJJYcsmSy5y/I6aswYS0HFjAGPEhZKwtRI5GCQY M45w== X-Gm-Message-State: AOJu0YyFEjxckb3SkgUKjVTb/BD5ZAq25iTABFp89bFdLdgFJt7fo3w7 GRPEUZsCRTE8Q/Aye99mthWKMiYQuyXiEv3cXRM0DAC3NQ58yKY01ZdPWZwy71M7l6+f4YWkOhP q X-Gm-Gg: ASbGncs4MS4IhCGdA7fyKe/Ufmyp+8trJItBmV7yh4LiDJNgE4P1+ApTXi8xfh6pmY/ Xco1ijXDs8DDVNSDwReKL/6JUZx6/w1R7LcjpUalvUyn0YwhmCVVfV+aSuxGqeUzBRmSrErkR9a ppngwX9GxVvG6BE3fRqpRFoBT0sBNF3Ovre4np52gLQxVP5PAsI4HQ5ojO8tEgBZSlwvNLEYVkU Y5CPf0yKLa1s4QYcGTJX3ZyY8ZoFzrc6gOyzcXN4CwztOmcXpgTRrIP9Bfn57lmIy7Yw8tZvS6M 2K8JeR09TJ4zyIEDfGaM8bz5Cvi3R0g= X-Google-Smtp-Source: AGHT+IEeJ6FbPThuSNvTvCkzCH9j0TuoKc6RZuow/bI8KaxgGRUo18olOKaDpDE/HOpQVYM1MSSGiQ== X-Received: by 2002:a17:902:f681:b0:215:385e:921c with SMTP id d9443c01a7336-21a8400b31amr355588425ad.51.1736920984873; Tue, 14 Jan 2025 22:03:04 -0800 (PST) Received: from dread.disaster.area (pa49-186-89-135.pa.vic.optusnet.com.au. [49.186.89.135]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21a9f10e0cfsm74886195ad.43.2025.01.14.22.03.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jan 2025 22:03:03 -0800 (PST) Received: from [192.168.253.23] (helo=devoid.disaster.area) by dread.disaster.area with esmtp (Exim 4.98) (envelope-from ) id 1tXwUW-0000000641Y-1fRS for fstests@vger.kernel.org; Wed, 15 Jan 2025 17:03:00 +1100 Received: from dave by devoid.disaster.area with local (Exim 4.98) (envelope-from ) id 1tXwUW-0000000GabO-2Ycm for fstests@vger.kernel.org; Wed, 15 Jan 2025 17:03:00 +1100 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 2/5] check: factor out test list building code Date: Wed, 15 Jan 2025 16:51:13 +1100 Message-ID: <20250115060258.3951185-3-david@fromorbit.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115060258.3951185-1-david@fromorbit.com> References: <20250115060258.3951185-1-david@fromorbit.com> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Dave Chinner factor out all the test list parsing and building code to common/test_list so that it can be used by both check and check-parallel. Signed-off-by: Dave Chinner --- check | 270 +++---------------------------------------- common/report | 2 +- common/test_list | 295 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 314 insertions(+), 253 deletions(-) create mode 100644 common/test_list diff --git a/check b/check index 607d2456e..4dc266dcf 100755 --- a/check +++ b/check @@ -15,19 +15,13 @@ notrun=() interrupt=true diff="diff -u" showme=false -have_test_arg=false -randomize=false -exact_order=false export here=`pwd` -xfile="" -subdir_xfile="" brief_test_summary=false do_report=false DUMP_OUTPUT=false iterations=1 istop=false loop_on_fail=0 -exclude_tests=() # This is a global variable used to pass test failure text to reporting gunk _err_msg="" @@ -49,8 +43,9 @@ timestamp=${TIMESTAMP:=false} rm -f $tmp.list $tmp.tmp $tmp.grep $here/$iam.out $tmp.report.* $tmp.arglist -SRC_GROUPS="generic" -export SRC_DIR="tests" +# We need to include the test list processing first as argument parsing +# requires test list parsing and setup. +. ./common/test_list usage() { @@ -124,153 +119,12 @@ examples: exit 1 } -get_sub_group_list() -{ - local d=$1 - local grp=$2 - - test -s "$SRC_DIR/$d/group.list" || return 1 - - local grpl=$(sed -n < $SRC_DIR/$d/group.list \ - -e 's/#.*//' \ - -e 's/$/ /' \ - -e "s;^\($VALID_TEST_NAME\).* $grp .*;$SRC_DIR/$d/\1;p") - echo $grpl -} - -get_group_list() -{ - local grp=$1 - local grpl="" - local sub=$(dirname $grp) - local fsgroup="$FSTYP" - - if [ -n "$sub" -a "$sub" != "." -a -d "$SRC_DIR/$sub" ]; then - # group is given as / (e.g. xfs/quick) - grp=$(basename $grp) - get_sub_group_list $sub $grp - return - fi - - if [ "$FSTYP" = ext2 -o "$FSTYP" = ext3 ]; then - fsgroup=ext4 - fi - for d in $SRC_GROUPS $fsgroup; do - if ! test -d "$SRC_DIR/$d" ; then - continue - fi - grpl="$grpl $(get_sub_group_list $d $grp)" - done - echo $grpl -} - -# Find all tests, excluding files that are test metadata such as group files. -# It matches test names against $VALID_TEST_NAME defined in common/rc -get_all_tests() -{ - touch $tmp.list - for d in $SRC_GROUPS $FSTYP; do - if ! test -d "$SRC_DIR/$d" ; then - continue - fi - ls $SRC_DIR/$d/* | \ - grep -v "\..*" | \ - grep "^$SRC_DIR/$d/$VALID_TEST_NAME"| \ - grep -v "group\|Makefile" >> $tmp.list 2>/dev/null - done -} - -# takes the list of tests to run in $tmp.list, and removes the tests passed to -# the function from that list. -trim_test_list() -{ - local test_list="$*" - - rm -f $tmp.grep - local numsed=0 - for t in $test_list - do - if [ $numsed -gt 100 ]; then - grep -v -f $tmp.grep <$tmp.list >$tmp.tmp - mv $tmp.tmp $tmp.list - numsed=0 - rm -f $tmp.grep - fi - echo "^$t\$" >>$tmp.grep - numsed=`expr $numsed + 1` - done - grep -v -f $tmp.grep <$tmp.list >$tmp.tmp - mv $tmp.tmp $tmp.list - rm -f $tmp.grep -} - _timestamp() { local now=`date "+%T"` echo -n " [$now]" } -_prepare_test_list() -{ - unset list - # Tests specified on the command line - if [ -s $tmp.arglist ]; then - cat $tmp.arglist > $tmp.list - else - touch $tmp.list - fi - - # Specified groups to include - # Note that the CLI processing adds a leading space to the first group - # parameter, so we have to catch that here checking for "all" - if ! $have_test_arg && [ "$GROUP_LIST" == " all" ]; then - # no test numbers, do everything - get_all_tests - else - for group in $GROUP_LIST; do - list=$(get_group_list $group) - if [ -z "$list" ]; then - echo "Group \"$group\" is empty or not defined?" - exit 1 - fi - - for t in $list; do - grep -s "^$t\$" $tmp.list >/dev/null || \ - echo "$t" >>$tmp.list - done - done - fi - - # Specified groups to exclude - for xgroup in $XGROUP_LIST; do - list=$(get_group_list $xgroup) - if [ -z "$list" ]; then - echo "Group \"$xgroup\" is empty or not defined?" - continue - fi - - trim_test_list $list - done - - # sort the list of tests into numeric order unless we're running tests - # in the exact order specified - if ! $exact_order; then - if $randomize; then - if type shuf >& /dev/null; then - sorter="shuf" - else - sorter="awk -v seed=$RANDOM -f randomize.awk" - fi - else - sorter="cat" - fi - list=`sort -n $tmp.list | uniq | $sorter` - else - list=`cat $tmp.list` - fi - rm -f $tmp.list -} - # Process command arguments first. while [ $# -gt 0 ]; do case "$1" in @@ -285,48 +139,20 @@ while [ $# -gt 0 ]; do export OVERLAY=true ;; - -g) group=$2 ; shift ; - GROUP_LIST="$GROUP_LIST ${group//,/ }" - ;; + -g) _tl_setup_group $2 ; shift ;; + -e) _tl_setup_exclude_tests $2 ; shift ;; + -E) _tl_setup_exclude_file $2 ; shift ;; + -x) _tl_setup_exclude_group $2; shift ;; + -X) _tl_setup_exclude_subdir $2; shift ;; + -r) _tl_setup_randomise ;; + --exact-order) _tl_setup_ordered ;; - -x) xgroup=$2 ; shift ; - XGROUP_LIST="$XGROUP_LIST ${xgroup//,/ }" - ;; - - -X) subdir_xfile=$2; shift ; - ;; - -e) - xfile=$2; shift ; - readarray -t -O "${#exclude_tests[@]}" exclude_tests < \ - <(echo "$xfile" | tr ', ' '\n\n') - ;; - - -E) xfile=$2; shift ; - if [ -f $xfile ]; then - readarray -t -O ${#exclude_tests[@]} exclude_tests < \ - <(sed "s/#.*$//" $xfile) - fi - ;; -s) RUN_SECTION="$RUN_SECTION $2"; shift ;; -S) EXCLUDE_SECTION="$EXCLUDE_SECTION $2"; shift ;; -l) diff="diff" ;; -udiff) diff="$diff -u" ;; -n) showme=true ;; - -r) - if $exact_order; then - echo "Cannot specify -r and --exact-order." - exit 1 - fi - randomize=true - ;; - --exact-order) - if $randomize; then - echo "Cannnot specify --exact-order and -r." - exit 1 - fi - exact_order=true - ;; -i) iterations=$2; shift ;; -I) iterations=$2; istop=true; shift ;; -T) timestamp=true ;; @@ -344,13 +170,13 @@ while [ $# -gt 0 ]; do -*) usage ;; *) # not an argument, we've got tests now. - have_test_arg=true ;; + _tl_setup_cli $* esac # if we've found a test specification, the break out of the processing # loop before we shift the arguments so that this is the first argument # that we process in the test arg loop below. - if $have_test_arg; then + if $_tl_have_test_args; then break; fi @@ -388,51 +214,6 @@ if [ -n "$FUZZ_REWRITE_DURATION" ]; then fi fi -if [ -n "$subdir_xfile" ]; then - for d in $SRC_GROUPS $FSTYP; do - [ -f $SRC_DIR/$d/$subdir_xfile ] || continue - for f in `sed "s/#.*$//" $SRC_DIR/$d/$subdir_xfile`; do - exclude_tests+=($d/$f) - done - done -fi - -# Process tests from command line now. -if $have_test_arg; then - while [ $# -gt 0 ]; do - case "$1" in - -*) echo "Arguments before tests, please!" - status=1 - exit $status - ;; - *) # Expand test pattern (e.g. xfs/???, *fs/001) - list=$(cd $SRC_DIR; echo $1) - for t in $list; do - t=${t#$SRC_DIR/} - test_dir=${t%%/*} - test_name=${t##*/} - group_file=$SRC_DIR/$test_dir/group.list - - if grep -Eq "^$test_name" $group_file; then - # in group file ... OK - echo $SRC_DIR/$test_dir/$test_name \ - >>$tmp.arglist - else - # oops - echo "$t - unknown test, ignored" - fi - done - ;; - esac - - shift - done -elif [ -z "$GROUP_LIST" ]; then - # default group list is the auto group. If any other group or test is - # specified, we use that instead. - GROUP_LIST="auto" -fi - if [ `id -u` -ne 0 ] then echo "check: QA must be run as root" @@ -593,21 +374,6 @@ _check_filesystems() return $ret } -_expunge_test() -{ - local TEST_ID="$1" - - for f in "${exclude_tests[@]}"; do - # $f may contain traling spaces and comments - local id_regex="^${TEST_ID}\b" - if [[ "$f" =~ ${id_regex} ]]; then - echo " [expunged]" - return 0 - fi - done - return 1 -} - # retain files which would be overwritten in subsequent reruns of the same test _stash_fail_loop_files() { local seq_prefix="${REPORT_DIR}/${1}" @@ -714,7 +480,7 @@ _run_seq() { } _detect_kmemleak -_prepare_test_list +_tl_prepare_test_list fstests_start_time="$(date +"%F %T")" if $OPTIONS_HAVE_SECTIONS; then @@ -794,7 +560,7 @@ function run_section() # TEST_DEV could be changed, source common/rc again with # correct FSTYP to get FSTYP specific configs, e.g. common/xfs . common/rc - _prepare_test_list + _tl_prepare_test_list elif [ "$OLD_TEST_FS_MOUNT_OPTS" != "$TEST_FS_MOUNT_OPTS" ]; then _test_unmount 2> /dev/null if ! _test_mount @@ -859,7 +625,7 @@ function run_section() loop_status=() # track rerun-on-failure state local tc_status ix - local -a _list=( $list ) + local -a _list=( $_tl_tests ) for ((ix = 0; ix < ${#_list[*]}; !${#loop_status[*]} && ix++)); do seq="${_list[$ix]}" @@ -880,7 +646,7 @@ function run_section() # the filename for the test and the name output are different. # we don't include the tests/ directory in the name output. - export seqnum=${seq#$SRC_DIR/} + export seqnum=$(_tl_strip_src_dir $seq) group=${seqnum%%/*} if $OPTIONS_HAVE_SECTIONS; then REPORT_DIR="$RESULT_BASE/$section" @@ -902,7 +668,7 @@ function run_section() echo -n "$seqnum" if $showme; then - if _expunge_test $seqnum; then + if _tl_expunge_test $seqnum; then tc_status="expunge" else echo @@ -928,7 +694,7 @@ function run_section() rm -f $seqres.out.bad $seqres.hints # check if we really should run it - if _expunge_test $seqnum; then + if _tl_expunge_test $seqnum; then tc_status="expunge" _stash_test_status "$seqnum" "$tc_status" continue diff --git a/common/report b/common/report index 7128bbeba..5697d2540 100644 --- a/common/report +++ b/common/report @@ -196,7 +196,7 @@ _xunit_make_testcase_report() echo -e "\t\t" >> $report ;; "fail") - local out_src="${SRC_DIR}/${test_name}.out" + local out_src="${_tl_src_dir}/${test_name}.out" local full_file="${REPORT_DIR}/${test_name}.full" local dmesg_file="${REPORT_DIR}/${test_name}.dmesg" local outbad_file="${REPORT_DIR}/${test_name}.out.bad" diff --git a/common/test_list b/common/test_list new file mode 100644 index 000000000..2432be6f7 --- /dev/null +++ b/common/test_list @@ -0,0 +1,295 @@ +##/bin/bash +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2000-2002,2006 Silicon Graphics, Inc. All Rights Reserved. +# Copyright (c) 2024 Red Hat, Inc. All Rights Reserved. +# +# Test list parsing and building functions +# +# Note: this file must stand alone and not be dependent on any other includes, +# most especially common/rc and common/config. This is because we have to +# include this file before option parsing, whilst the rc/config includes need to +# be included -after- option parsing. +# +# Any function or variable that is public should have a "_tl_" prefix. + +export _tl_src_dir="tests" + +_SRC_GROUPS="generic" +_GROUP_LIST= +_XGROUP_LIST= +_tl_exact_order=false +_tl_randomise=false +_tl_have_test_args=false +_tl_file="$tmp.test_list" +_tl_exclude_tests=() +_tl_tests= + +_tl_strip_src_dir() +{ + local test="$1" + + echo ${test#$_tl_src_dir/} +} + +get_sub_group_list() +{ + local d=$1 + local grp=$2 + + test -s "$_tl_src_dir/$d/group.list" || return 1 + + local grpl=$(sed -n < $_tl_src_dir/$d/group.list \ + -e 's/#.*//' \ + -e 's/$/ /' \ + -e "s;^\($VALID_TEST_NAME\).* $grp .*;$_tl_src_dir/$d/\1;p") + echo $grpl +} + +get_group_list() +{ + local grp=$1 + local grpl="" + local sub=$(dirname $grp) + local fsgroup="$FSTYP" + + if [ -n "$sub" -a "$sub" != "." -a -d "$_tl_src_dir/$sub" ]; then + # group is given as / (e.g. xfs/quick) + grp=$(basename $grp) + get_sub_group_list $sub $grp + return + fi + + if [ "$FSTYP" = ext2 -o "$FSTYP" = ext3 ]; then + fsgroup=ext4 + fi + for d in $_SRC_GROUPS $fsgroup; do + if ! test -d "$_tl_src_dir/$d" ; then + continue + fi + grpl="$grpl $(get_sub_group_list $d $grp)" + done + echo $grpl +} + +# Find all tests, excluding files that are test metadata such as group files. +# It matches test names against $VALID_TEST_NAME defined in common/rc +get_all_tests() +{ + touch $tmp.list + for d in $_SRC_GROUPS $FSTYP; do + if ! test -d "$_tl_src_dir/$d" ; then + continue + fi + ls $_tl_src_dir/$d/* | \ + grep -v "\..*" | \ + grep "^$_tl_src_dir/$d/$VALID_TEST_NAME"| \ + grep -v "group\|Makefile" >> $tmp.list 2>/dev/null + done +} + +# takes the list of tests to run in $tmp.list, and removes the tests passed to +# the function from that list. +trim_test_list() +{ + local test_list="$*" + + rm -f $tmp.grep + local numsed=0 + for t in $test_list + do + if [ $numsed -gt 100 ]; then + grep -v -f $tmp.grep <$tmp.list >$tmp.tmp + mv $tmp.tmp $tmp.list + numsed=0 + rm -f $tmp.grep + fi + echo "^$t\$" >>$tmp.grep + numsed=`expr $numsed + 1` + done + grep -v -f $tmp.grep <$tmp.list >$tmp.tmp + mv $tmp.tmp $tmp.list + rm -f $tmp.grep +} + +_tl_prepare_test_list() +{ + unset _tl_tests + # Tests specified on the command line + if [ -s $_tl_file ]; then + cat $_tl_file > $tmp.list + else + touch $tmp.list + fi + + # Specified groups to include + # Note that the CLI processing adds a leading space to the first group + # parameter, so we have to catch that here checking for "all" + if ! $_tl_have_test_args && [ "$_GROUP_LIST" == " all" ]; then + # no test numbers, do everything + get_all_tests + else + for group in $_GROUP_LIST; do + list=$(get_group_list $group) + if [ -z "$list" ]; then + echo "Group \"$group\" is empty or not defined?" + exit 1 + fi + + for t in $list; do + grep -s "^$t\$" $tmp.list >/dev/null || \ + echo "$t" >>$tmp.list + done + done + fi + + # Specified groups to exclude + for xgroup in $_XGROUP_LIST; do + list=$(get_group_list $xgroup) + if [ -z "$list" ]; then + echo "Group \"$xgroup\" is empty or not defined?" + continue + fi + + trim_test_list $list + done + + # sort the list of tests into numeric order unless we're running tests + # in the exact order specified + if ! $_tl_exact_order; then + if $_tl_randomise; then + if type shuf >& /dev/null; then + sorter="shuf" + else + sorter="awk -v seed=$RANDOM -f randomize.awk" + fi + else + sorter="cat" + fi + _tl_tests=`sort -n $tmp.list | uniq | $sorter` + else + _tl_tests=`cat $tmp.list` + fi + rm -f $tmp.list +} + +_tl_expunge_test() +{ + local TEST_ID="$1" + + for f in "${_tl_exclude_tests[@]}"; do + # $f may contain traling spaces and comments + local id_regex="^${TEST_ID}\b" + if [[ "$f" =~ ${id_regex} ]]; then + echo " [expunged]" + return 0 + fi + done + return 1 +} + +_tl_setup_exclude_tests() +{ + local list="$1" + + readarray -t -O "${#_tl_exclude_tests[@]}" _tl_exclude_tests < \ + <(echo "$list" | tr ', ' '\n\n') +} + +_tl_setup_exclude_file() +{ + local xfile="$1" + + if [ -f $xfile ]; then + readarray -t -O ${#_tl_exclude_tests[@]} _tl_exclude_tests < \ + <(sed "s/#.*$//" $xfile) + fi +} + +_tl_setup_exclude_subdir() +{ + local xfile="$1" + local d + local f + + [ -z "$xfile" ] && return + + for d in $_SRC_GROUPS $FSTYP; do + [ -f $_tl_src_dir/$d/$xfile ] || continue + for f in `sed "s/#.*$//" $_tl_src_dir/$d/$xfile`; do + _tl_exclude_tests+=($d/$f) + done + done +} + +_tl_setup_exclude_group() +{ + local xgroup="$1" + + _XGROUP_LIST="$_XGROUP_LIST ${xgroup//,/ }" +} + +_tl_setup_group() +{ + local group="$1" + + _GROUP_LIST="$_GROUP_LIST ${group//,/ }" +} + +_tl_setup_randomise() +{ + if $_tl_exact_order; then + echo "Cannot specify -r and --exact-order." + exit 1 + fi + _tl_randomise=true +} + +_tl_setup_ordered() +{ + if $_tl_randomise; then + echo "Cannnot specify --exact-order and -r." + exit 1 + fi + _tl_exact_order=true +} + +_tl_setup_cli() +{ + while [ $# -gt 0 ]; do + case "$1" in + -*) echo "Arguments before tests, please!" + status=1 + exit $status + ;; + *) # Expand test pattern (e.g. xfs/???, *fs/001) + local list=$(cd $_tl_src_dir; echo $1) + local t + + for t in $list; do + t=${t#$_tl_src_dir/} + local test_dir=${t%%/*} + local test_name=${t##*/} + local group_file=$_tl_src_dir/$test_dir/group.list + + if grep -Eq "^$test_name" $group_file; then + # in group file ... OK + echo $_tl_src_dir/$test_dir/$test_name \ + >> $_tl_file + _tl_have_test_args=true + else + # oops + echo "$t - unknown test, ignored" + fi + done + ;; + esac + + shift + done + + if ! $_tl_have_test_args && [ -z "$_GROUP_LIST" ]; then + # default group list is the auto group. If any other group or + # test is specified, we use that instead. + _GROUP_LIST="auto" + fi +} From patchwork Wed Jan 15 05:51:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 13939857 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 585542054E9 for ; Wed, 15 Jan 2025 06:03:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736920987; cv=none; b=V/dGcVJdfuCLIs8vfr9CbK3xFLrmdlIGNdB8NJunZWqDEMA+MsXGxXFbQ4sQezc0SNPUHpN2Lm+80kXfR+iM5jCK2cDnhATfvcNZTqI/gCqELMlBJCtt0b9vMyC5z0kvUWSnEVfEesl2kPD0Rn/rwX083SK2TyR+4/l3NoRSYkE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736920987; c=relaxed/simple; bh=Rs2CL2oGTEJ4im+8mN6kltlq5ox4e5BdJiatg9nxtn8=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=A2T0EpQ5kwRJtAqEjUSb5dhM4dd01gOpUVBA0m2Gpd3Mb9AiIf+VLMLJQ+naKHM9wF/HMdlpO333KKFToOfXi3qftSosiRTOYHW9te7XagfqcTcOkJPiVWJlh4TCUe3I4itlzqrPBd5njsetEAlEhCbl1d5EEejfbuVp+n/GhFM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fromorbit.com; spf=pass smtp.mailfrom=fromorbit.com; dkim=pass (2048-bit key) header.d=fromorbit-com.20230601.gappssmtp.com header.i=@fromorbit-com.20230601.gappssmtp.com header.b=Vm8gE2XO; arc=none smtp.client-ip=209.85.216.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fromorbit.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fromorbit.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fromorbit-com.20230601.gappssmtp.com header.i=@fromorbit-com.20230601.gappssmtp.com header.b="Vm8gE2XO" Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2ef70c7efa5so8651999a91.2 for ; Tue, 14 Jan 2025 22:03:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20230601.gappssmtp.com; s=20230601; t=1736920984; x=1737525784; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=s/VIejK/47Hv2yqRjB4Op7RT5n+QkZx2ByFzp9lIxHg=; b=Vm8gE2XO/9dROFyefj02LqH63bTVPZAUoZ2x0hxq3osjgLOIMmyZB1Da0pPhebiwWE pxrlsS38kLqCrYjfdaYqjx2LTLsMQuDROSYZd31dE+dSEmMoI4lJfqrxg4olQo8/Jo9I uslwctUoET3cepHazt51XZYkfRbzVRTb1rH1WOKPWssOAMWT/G/fWxWXl3Ow/SxkjWWf xYlyPF+yPoJj0Gzkf0kePjuuChdIrebajTlG3qNFkWerRu4EHinUuRClEb+mLk4p9a8F 7jY2TLJuzYoIz/mvoe8b3xr2l9vPZJWzZi6No6VuXyBYL7mMgD9oaRfn3HYzg0I34RR0 quaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736920984; x=1737525784; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s/VIejK/47Hv2yqRjB4Op7RT5n+QkZx2ByFzp9lIxHg=; b=jx24PVHPP9IfJY525VTsXZHd/lP9/Mv9jPh9Ql/2G25NYtXm6YnkTAU26WNEelVdPi XKGCbvcRh1aCOcSWpMMLhr5SVRbMBwhbB6Uw7e647jVA1LXvVIwg3xqp9PNMvSU1ZGXi ZZAPNOtZnN33IH4vNmmvkrbMPz5xL1FbmI1Ss56PWX2NjEHwfGg/uWMsdCdSujWfHjth lhQY1CBxkr3jtOdNZx7SiWhebgJpcbDFCYMsa5S04v8ODP/J7qxXWJATEb6c0nmqDpsn DZr/Q8Ib+P6JCv5nJAL1lZ9Y1JaZSaG958dGesVdAenfnLOScEs3MJZAlMvs2vEYsR3V IyGg== X-Gm-Message-State: AOJu0YzPfCaPuqP6tk4xa2cBS3ayddd8Q/4bYLEvC9HDEHOuRq+HUndG ZKxsmgAZUsTr7wCX+NRMpP0HRePR9EnFN2UeJDLxaUcq42wGROEAv9FTeQ1MC2pV/TPaly2/TWa 5 X-Gm-Gg: ASbGncvZ4LZ+NlpIsEJNfsYVtzhD8jRYZuCiDAOgX0z5q/9S1s09MKhd4SXJhWkcAp7 Jz1910YjwnvRWruioYvSM02q6r/MxQUtVPONQqtPkPOWMWdqu1UIMbpbc27K3HXH3Yikhb+LwtN Ys3I2uSXtydYtqbZqO8uJBHR5X9IfUzr0OssShkqQCLOGlIJcaxnVTjoW5YFZNC+8H+u2y/HyXT EnBovEqqBqFaO3Ebqiq3JU9yz3k+wQGwXVXqjweuL34Q8qDnITGuaWXCvL+sqnBLbap2m4RslSK wnu3v2f0sUNsIhm+vH7/4JF6HbKmbH4= X-Google-Smtp-Source: AGHT+IHZwUvntRo747RCyUhvPsBXBw/LP5MdUBSCWGR8S6qE7jjia0ro7h1J3qNFhCKoHlP4BvbCjg== X-Received: by 2002:a17:90b:4d05:b0:2ee:f80c:6884 with SMTP id 98e67ed59e1d1-2f548f426ccmr42464663a91.33.1736920984434; Tue, 14 Jan 2025 22:03:04 -0800 (PST) Received: from dread.disaster.area (pa49-186-89-135.pa.vic.optusnet.com.au. [49.186.89.135]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21a9f152c6asm74777825ad.104.2025.01.14.22.03.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jan 2025 22:03:03 -0800 (PST) Received: from [192.168.253.23] (helo=devoid.disaster.area) by dread.disaster.area with esmtp (Exim 4.98) (envelope-from ) id 1tXwUW-0000000641c-1oVF for fstests@vger.kernel.org; Wed, 15 Jan 2025 17:03:00 +1100 Received: from dave by devoid.disaster.area with local (Exim 4.98) (envelope-from ) id 1tXwUW-0000000GabS-2iah for fstests@vger.kernel.org; Wed, 15 Jan 2025 17:03:00 +1100 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 3/5] check-parallel: use common group list parsing code Date: Wed, 15 Jan 2025 16:51:14 +1100 Message-ID: <20250115060258.3951185-4-david@fromorbit.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115060258.3951185-1-david@fromorbit.com> References: <20250115060258.3951185-1-david@fromorbit.com> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Dave Chinner Build the test list directly from command line prompts, rather than hard coding the tests and using the check infrastructure to filter that list. We still pass exact test lists to check to execute the tests that each runner needs to execute, but all other test list commands are no longer passed to check. As a result of this change, check-parallel no longer passes unknown CLI parameters through to the internal check invocations. At this point, the only non test-list related option is config file section selection; more of the check options will be brought across as needed in future patches. Signed-off-by: Dave Chinner --- check-parallel | 153 ++++++++++++++++++++++++++++++++++++++++------- common/test_list | 7 +++ 2 files changed, 139 insertions(+), 21 deletions(-) diff --git a/check-parallel b/check-parallel index d34c73f66..a341c9c2e 100755 --- a/check-parallel +++ b/check-parallel @@ -9,18 +9,115 @@ # for them and runs the test in the background. When it completes, it tears down # the loop devices. -export SRC_DIR="tests" -basedir=$1 -shift -check_args="$*" +basedir="" runners=$(getconf _NPROCESSORS_CONF) runner_list=() runtimes=() +show_test_list= +run_section="" +tmp=/tmp/check-parallel.$$ -# tests in auto group -test_list=$(awk '/^[0-9].*auto/ { print "generic/" $1 }' tests/generic/group.list) -test_list+=$(awk '/^[0-9].*auto/ { print "xfs/" $1 }' tests/xfs/group.list) +export FSTYP=xfs + +# We need to include the test list processing first as argument parsing +# requires test list parsing and setup. +. ./common/test_names +. ./common/test_list + +usage() +{ + echo "Usage: $0 [options] [testlist]"' + +check options + -D Directory to run in + -n Output test list, do not run tests + -r randomize test order + --exact-order run tests in the exact order specified + -s section run only specified section from config file + +testlist options + -g group[,group...] include tests from these groups + -x group[,group...] exclude tests from these groups + -X exclude_file exclude individual tests + -e testlist exclude a specific list of tests + -E external_file exclude individual tests + [testlist] include tests matching names in testlist + +testlist argument is a list of tests in the form of /. + + is a directory under tests that contains a group file, +with a list of the names of the tests in that directory. + + may be either a specific test file name (e.g. xfs/001) or +a test file name match pattern (e.g. xfs/*). + +group argument is either a name of a tests group to collect from all +the test dirs (e.g. quick) or a name of a tests group to collect from +a specific tests dir in the form of / (e.g. xfs/quick). +If you want to run all the tests in the test suite, use "-g all" to specify all +groups. + +exclude_file argument refers to a name of a file inside each test directory. +for every test dir where this file is found, the listed test names are +excluded from the list of tests to run from that test dir. + +external_file argument is a path to a single file containing a list of tests +to exclude in the form of /. + +examples: + check-parallel -D /mnt xfs/001 + check-parallel -D /mnt -g quick + check-parallel -D /mnt -g xfs/quick + check-parallel -D /mnt -x stress xfs/* + check-parallel -D /mnt -X .exclude -g auto + check-parallel -D /mnt -E ~/.xfstests.exclude +' + exit 1 +} + +# Process command arguments first. +while [ $# -gt 0 ]; do + case "$1" in + -\? | -h | --help) usage ;; + + -D) basedir=$2; shift ;; + -g) _tl_setup_group $2 ; shift ;; + -e) _tl_setup_exclude_tests $2 ; shift ;; + -E) _tl_setup_exclude_file $2 ; shift ;; + -x) _tl_setup_exclude_group $2; shift ;; + -X) _tl_setup_exclude_subdir $2; shift ;; + -r) _tl_setup_randomise ;; + --exact-order) _tl_setup_ordered ;; + -n) show_test_list="yes" ;; + + -s) run_section="$run_section -s $2"; shift ;; + + -*) usage ;; + *) # not an argument, we've got tests now. + _tl_setup_cli $* + esac + + # if we've found a test specification, the break out of the processing + # loop before we shift the arguments so that this is the first argument + # that we process in the test arg loop below. + if $_tl_have_test_args; then + break; + fi + + shift +done + +if [ ! -d "$basedir" ]; then + echo "Invalid basedir specification" + usage +fi +if [ -d "$basedir/runner-0/" ]; then + prev_results=`ls -tr $basedir/runner-0/ | grep results | tail -1` +fi + +_tl_prepare_test_list +_tl_strip_test_list # grab all previously run tests and order them from highest runtime to lowest # We are going to try to run the longer tests first, hopefully so we can avoid @@ -30,25 +127,23 @@ test_list+=$(awk '/^[0-9].*auto/ { print "xfs/" $1 }' tests/xfs/group.list) # # If we have tests in the test list that don't have runtimes recorded, then # append them to be run last. - -build_runner_list() +time_order_test_list() { local runtimes local run_list=() - local prev_results=`ls -tr $basedir/runner-0/ | grep results | tail -1` runtimes=$(cat $basedir/*/$prev_results/check.time | sort -k 2 -nr | cut -d " " -f 1) # Iterate the timed list first. For every timed list entry that # is found in the test_list, add it to the local runner list. local -a _list=( $runtimes ) - local -a _tlist=( $test_list ) + local -a _tlist=( $_tl_tests ) local rx=0 local ix local jx #set -x for ((ix = 0; ix < ${#_list[*]}; ix++)); do - echo $test_list | grep -q ${_list[$ix]} + echo $_tl_tests | grep -q ${_list[$ix]} if [ $? == 0 ]; then # add the test to the new run list and remove # it from the remaining test list. @@ -60,20 +155,21 @@ build_runner_list() # The final test list is all the time ordered tests followed by # all the tests we didn't find time records for. - test_list="${run_list[*]} ${_tlist[*]}" + _tl_tests="${run_list[*]} ${_tlist[*]}" } -if [ -f $basedir/runner-0/results/check.time ]; then - build_runner_list +if ! $_tl_randomise -a ! $_tl_exact_order; then + if [ -f $basedir/runner-0/$prev_results/check.time ]; then + time_order_test_list + fi fi # split the list amongst N runners - split_runner_list() { local ix local rx - local -a _list=( $test_list ) + local -a _list=( $_tl_tests ) for ((ix = 0; ix < ${#_list[*]}; ix++)); do seq="${_list[$ix]}" rx=$((ix % $runners)) @@ -137,7 +233,7 @@ runner_go() # Run the tests in it's own mount namespace, as per the comment below # that precedes making the basedir a private mount. - ./src/nsexec -m ./check $check_args -x unreliable_in_parallel --exact-order ${runner_list[$id]} > $me/log 2>&1 + ./src/nsexec -m ./check $run_section -x unreliable_in_parallel --exact-order ${runner_list[$id]} > $me/log 2>&1 wait sleep 1 @@ -165,6 +261,13 @@ cleanup() trap "cleanup; exit" HUP INT QUIT TERM +split_runner_list +if [ -n "$show_test_list" ]; then + echo Time ordered test list: + echo $_tl_tests + echo +fi + # Each parallel test runner needs to only see it's own mount points. If we # leave the basedir as shared, then all tests see all mounts and then we get @@ -178,15 +281,23 @@ trap "cleanup; exit" HUP INT QUIT TERM # in it's own mount namespace so that they cannot see mounts that other tests # are performing. mount --make-private $basedir -split_runner_list + now=`date +%Y-%m-%d-%H:%M:%S` for ((i = 0; i < $runners; i++)); do - runner_go $i $now & + if [ -n "$show_test_list" ]; then + echo "Runner $i: ${runner_list[$i]}" + else + runner_go $i $now & + fi done; wait +if [ -n "$show_test_list" ]; then + exit 0 +fi + echo -n "Tests run: " grep Ran /mnt/xfs/*/log | sed -e 's,^.*:,,' -e 's, ,\n,g' | sort | uniq | wc -l @@ -195,7 +306,7 @@ grep Failures: $basedir/*/log | uniq | sed -e "s/^.*Failures://" -e "s,\([0-9]\) echo echo Ten slowest tests - runtime in seconds: -cat $basedir/*/results/check.time | sort -k 2 -nr | head -10 +cat $basedir/*/results-$now/check.time | sort -k 2 -nr | head -10 echo echo Cleanup on Aisle 5? diff --git a/common/test_list b/common/test_list index 2432be6f7..2b3ae9fbf 100644 --- a/common/test_list +++ b/common/test_list @@ -24,6 +24,7 @@ _tl_file="$tmp.test_list" _tl_exclude_tests=() _tl_tests= +# strip 'tests\' prefix from the provided test name _tl_strip_src_dir() { local test="$1" @@ -31,6 +32,12 @@ _tl_strip_src_dir() echo ${test#$_tl_src_dir/} } +# strip 'tests\' prefix from all the tests in the test list +_tl_strip_test_list() +{ + _tl_tests=$(echo $_tl_tests | sed -e "s/$_tl_src_dir\///g") +} + get_sub_group_list() { local d=$1 From patchwork Wed Jan 15 05:51:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 13939859 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC9121F9F5C for ; Wed, 15 Jan 2025 06:03:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736920988; cv=none; b=dPNME9LOw7GKQpjt7LMj1KRIdY2X2Q6v5bC9iCs+FgRmjL7+WKlTuNIPJKqbqa+wZGzfs58wVXlqBHNmLkGVVCeyhspS/YXv6zGcom35umSIcGRIHeW884Grh3s6wlGNgcebNxRnS3OX7l2i+9XG+9KzTVdfq6poO8f5Lak62wQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736920988; c=relaxed/simple; bh=70aX2grdn7mhJu+t9ymdcv9dvII1AWUBMNOgdPqHuQI=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OW4fjuCOpL2fx4j6lLHJAqMSyHn2AcwiMHsPuPj+ODsN4HR4DNBTfaBRyJylRc0YysIjdIsa/QY8bY8q28riKW6ohvD8MopmBsq6AR8fjGs3gNKmSIb4PWMu/jegiNHCestw6OiZssCjAboVsbUYepF1vGjZPSA6gp9eT5Meu/Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fromorbit.com; spf=pass smtp.mailfrom=fromorbit.com; dkim=pass (2048-bit key) header.d=fromorbit-com.20230601.gappssmtp.com header.i=@fromorbit-com.20230601.gappssmtp.com header.b=AvqCQ/3v; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fromorbit.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fromorbit.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fromorbit-com.20230601.gappssmtp.com header.i=@fromorbit-com.20230601.gappssmtp.com header.b="AvqCQ/3v" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-218c8aca5f1so136970875ad.0 for ; Tue, 14 Jan 2025 22:03:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20230601.gappssmtp.com; s=20230601; t=1736920985; x=1737525785; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=pfNZ2QuURZIoZqHQzg7cCMYCy3ZgmB4MlCZfMTYK5jY=; b=AvqCQ/3vFGmeyX3dISw//NFkf79xnH9nyKNWHNXoulNKO7xH+1Xy8kQEftZXqbXbuh rfoDxSt/VMaWGhd9UsplGd5R5xwtJcLYbTz+7EmpuJkrnmcAKLl5pef1YE4wS1ae3NBT uUkeNkKqyyDIgkum1dkmj4wAG4GV8NYpT1/x7py5uzLiqG+GM180/9mTx3rrXrAAFvOx PulOMNACBef6aj5qJdC4xbRmqDmsMVYEDd1a+fPkkIzhr7d3CvHKBh62cqL0NRaFOffo LZLEF1KnvXI6+f2w1V0DRV+SRv7yA5qsuXTZvLB7FLrmWOLsobHtkqy8hK1cjxvkHf3K 1+Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736920985; x=1737525785; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pfNZ2QuURZIoZqHQzg7cCMYCy3ZgmB4MlCZfMTYK5jY=; b=B+zjcrltfjVVKTyjK8ORPb/6/Gdk5Lq6poh7XYMt+qGuf5z2sDf5PK+VepHPXBn0Qc G/LKxE+/ggm5f35vhKu5G2WYAg6L0iKLqu1OW87f9D+H4SLZVVfB9JmkcgEoBZ4YsA1Q 3IzAprnd3LLST3NN2BuTUqR8n5Fk99x9eePl7UuRIm9HZgLEHNjGfY4gfyH+hK5eHlmM pQ7SVPoEofOSo2ie/3WG7a5atlKSx5elfEEs4wgxfF6gK5864aGh+K02vV4SSamTvn6v LALqSDKrgwi6NIl87Fcn3FWuAPI3T0xvG8d10v0wTX3sOsRv5tUyxeJKtiGjnjoUkdro E+sw== X-Gm-Message-State: AOJu0Yyu2SaeaX9eq3z4KzkD6NseF8mlTWelhGgZUgBBKcu9YirqzGgU xARGDjpVHwSAI5wSdCOlRGD9FoqPRf27/jghyFQhJSji9/4SLRPPvOe00zgOPyc8PrweYzYwu4U P X-Gm-Gg: ASbGnctndgx1KIjhLjo1OuuY5dkI907EqscY8MtuVWwyWiqFcFeQgbLrnU0oqlhXv+u Wt9r/Nquodw9wK0qYq9SzukliEaYe9whZH6Y6jhC4p7sQmFBA8vAuAwPxJr1u/ezA7E7tCaTmNX Gc0x/sJiJNBalhZCQssDa5yF4Qg+X+AkoPlsFC836IwfPGAXXmqOuo7gmIcF1LW5yHHgguFrfCE X1CuDbWYmX6Z2Pkb9uttCLhf6WUKAsQ05Sl/VJK5mdBSNwDYHImN67f0IN1oL+QtlW54BypUH22 0dxsTCXtVj789oZXkokOBGxwdctQ17k= X-Google-Smtp-Source: AGHT+IFnnF89oPAIKI03sym3L10IADQsfgNQOI7i0htwSVAtWbMMhATCaDl6sPyxYLokSEW5yiADuw== X-Received: by 2002:a05:6a20:3d86:b0:1e0:c5d2:f215 with SMTP id adf61e73a8af0-1e88d18b424mr9578958637.12.1736920985244; Tue, 14 Jan 2025 22:03:05 -0800 (PST) Received: from dread.disaster.area (pa49-186-89-135.pa.vic.optusnet.com.au. [49.186.89.135]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-a31d5047e01sm9099263a12.54.2025.01.14.22.03.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jan 2025 22:03:03 -0800 (PST) Received: from [192.168.253.23] (helo=devoid.disaster.area) by dread.disaster.area with esmtp (Exim 4.98) (envelope-from ) id 1tXwUW-0000000641f-21rJ for fstests@vger.kernel.org; Wed, 15 Jan 2025 17:03:00 +1100 Received: from dave by devoid.disaster.area with local (Exim 4.98) (envelope-from ) id 1tXwUW-0000000GabX-2rPP for fstests@vger.kernel.org; Wed, 15 Jan 2025 17:03:00 +1100 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 4/5] check-parallel: add logwrite device support Date: Wed, 15 Jan 2025 16:51:15 +1100 Message-ID: <20250115060258.3951185-5-david@fromorbit.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115060258.3951185-1-david@fromorbit.com> References: <20250115060258.3951185-1-david@fromorbit.com> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Dave Chinner It is just another loop device per test runner.... Signed-off-by: Dave Chinner --- check-parallel | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/check-parallel b/check-parallel index a341c9c2e..498b53c5a 100755 --- a/check-parallel +++ b/check-parallel @@ -208,12 +208,14 @@ runner_go() local me=$basedir/runner-$id local _test=$me/test.img local _scratch=$me/scratch.img + local _logwrites=$me/logwrites.img local _results=$me/results-$2 mkdir -p $me xfs_io -f -c 'truncate 2g' $_test xfs_io -f -c 'truncate 8g' $_scratch + xfs_io -f -c 'truncate 1g' $_logwrites mkfs.xfs -f $_test > /dev/null 2>&1 @@ -221,6 +223,7 @@ runner_go() export TEST_DIR=$me/test export SCRATCH_DEV=$(_create_loop_device $_scratch) export SCRATCH_MNT=$me/scratch + export LOGWRITES_DEV=$(_create_loop_device $_logwrites) export FSTYP=xfs export RESULT_BASE=$_results @@ -241,6 +244,7 @@ runner_go() umount -R $SCRATCH_MNT 2> /dev/null _destroy_loop_device $TEST_DEV _destroy_loop_device $SCRATCH_DEV + _destroy_loop_device $LOGWRITES_DEV grep -q Failures: $me/log if [ $? -eq 0 ]; then From patchwork Wed Jan 15 05:51:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 13939856 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA9D01F9F5C for ; Wed, 15 Jan 2025 06:03:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736920986; cv=none; b=WvMNEiTRqUdz+1OaZA+p2VPAi7zzoSEXm/+IuwB4tphS8JEff3dTLuTocYI1Hf7HQ8GuA4RW326NE01E601asil43wFD9JlK6tm52BB5N23xCtW2/0ZM4I9bPa36QMMjfmvtlpyIgZLrKnMKSDTU1xLsMvhuevegkQQet1t8ly4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736920986; c=relaxed/simple; bh=NQ1NVX7mDRpk3eFImqWLbKttBReOmTgfmygnEDXVKcU=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HJhE3+jCoWl4TjLtbE3m1PKnBgN91UA0rUr8BlQuua3BlOvlFn1SNxgXp8tblZdy6C/7W6p/jSHRKgn59dIVd5frAhTQxkkj9mhMZ2rWOW/H4LIYBOyyYUp/yVc/yRl3nRvT1UHpPjzA4ctSDJ+ppt26TlBcaGEBzKrgOTfAxA8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fromorbit.com; spf=pass smtp.mailfrom=fromorbit.com; dkim=pass (2048-bit key) header.d=fromorbit-com.20230601.gappssmtp.com header.i=@fromorbit-com.20230601.gappssmtp.com header.b=Z4CKFn4a; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fromorbit.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fromorbit.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fromorbit-com.20230601.gappssmtp.com header.i=@fromorbit-com.20230601.gappssmtp.com header.b="Z4CKFn4a" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-21670dce0a7so137226475ad.1 for ; Tue, 14 Jan 2025 22:03:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20230601.gappssmtp.com; s=20230601; t=1736920984; x=1737525784; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=DDyM9C+bh4pg44LGWqVZr4fOonPK+G5ritIIjqyIkgo=; b=Z4CKFn4aq7qacoSwSbx7DhUKO5/cx7w35Q12NhzkPzSH/u6FbrOcs+B5x8EVCIgFbW WpIzf1P6VGlT6ywachZFYbzNiEiansJnEhvbMDR5L5fcbPVct0RnzjRsg2kEOrrQKXDR +JxOUb+ZYMYYnfB5UfaqphjbCtUVx3qrVfLEQ3FyaMgrGxkEGUtZtxlfb24bJTH7c2d/ BddRnp7Z9LWMZLJbH1J3v75pCrl4OckaRwe44Dh/+A+BefyITWpKBsikogqp8nOwe/N1 1FaMjZKHFM0HQG5AwyMVFa/xAUpukrN+/jkNOLeHJNGjl5FV23qGy6uZIXGemxsMrPac nB6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736920984; x=1737525784; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DDyM9C+bh4pg44LGWqVZr4fOonPK+G5ritIIjqyIkgo=; b=WzVNgrXH9S2fYg0W6piOcoPw26f7z0yPPVnbhx8n2FMQbjGAq+Yt+/HHpkcQQ7MgmC D+M+rfe8YEwKagGAnzOviUjB7U2evnsjn0d+9at5ItSWhC1zYDnJ+BC3oYc2fL9s47Gz zw071YAZqnLudOzp6VQYSMag58jQNtJviSh/fvC7AG9VZa4HdUHbw03BBzK3fKM3EUCZ qgaA2OE/eaNU6hk1yMtEZ8RbooWnrI0g0vMuKYXZL9dlodv1YpJiT58KFhx2PITbmWki CU7ibTnmU0pGQApTdNEzmxX70kOKS1+N5xvyKI6SLlBogz9PJ2f9i1sIoOSKPfP18k05 qiwA== X-Gm-Message-State: AOJu0YzWtEDxTdkd1WPj04MPNIJ/cuuDYwR80fqHaTNXSqQk35q/mLRO e8ZQxFDQXZDQy1sl7ma2gdCFbMnczeqPHlrAo0XMo1E9/INEm5EhF3/h9JGuU7lRlnd1CUCoDpm + X-Gm-Gg: ASbGncvLZpkDJYHKAJnn8YbkO44N7WpS35bAx2swnahbuUkxoG2sfP1KPikTchQs1r0 Ef6xvzMM5Oo8k5EAg2j+IA99KYsLIlcP0ZgeFTEGGKtNN0/NpY6cgbnpB77b9CGSK2nIksQy1F8 Dm/oQCPKpO8/HL3szdBKS/26GeoVFl89XDTO+ZsRsqOKSsCXYJQLlHVayyDYjgIb4fK+1Jxp4fM ZtM26HLSLFqohd+yLqTbDU3iBSRQsLqojk+gNYf7m1EeuGlWwhVl1vNaTXIqUauQeMqp1EbDgHa Te2lHY5u3cMdB8d4W3zUrGP7LHXxZ74= X-Google-Smtp-Source: AGHT+IEQzF6KjrbH9bb7HxEdgA+Kg03emAnemZVSnpBNFX+gk6mEpz8MEUTmGXbqgcmv02n/wn33ew== X-Received: by 2002:a17:902:d481:b0:215:b01a:627f with SMTP id d9443c01a7336-21a83f48e86mr442042485ad.4.1736920983986; Tue, 14 Jan 2025 22:03:03 -0800 (PST) Received: from dread.disaster.area (pa49-186-89-135.pa.vic.optusnet.com.au. [49.186.89.135]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21a9f10e2basm75162815ad.19.2025.01.14.22.03.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jan 2025 22:03:03 -0800 (PST) Received: from [192.168.253.23] (helo=devoid.disaster.area) by dread.disaster.area with esmtp (Exim 4.98) (envelope-from ) id 1tXwUW-0000000641i-29Ga for fstests@vger.kernel.org; Wed, 15 Jan 2025 17:03:00 +1100 Received: from dave by devoid.disaster.area with local (Exim 4.98) (envelope-from ) id 1tXwUW-0000000Gabd-31Ig for fstests@vger.kernel.org; Wed, 15 Jan 2025 17:03:00 +1100 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 5/5] check-parallel: allow FSTYP selection from the CLI Date: Wed, 15 Jan 2025 16:51:16 +1100 Message-ID: <20250115060258.3951185-6-david@fromorbit.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115060258.3951185-1-david@fromorbit.com> References: <20250115060258.3951185-1-david@fromorbit.com> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Dave Chinner Add a CLI option to specify the initial FSTYP to test. If this is not specified the the default of "xfs" will be used. This option is different to the way check has FSTYP specified as check-parallel has no infrastructure to support non block device based filesystems and hence we have to reject virtual or network based filesysetms are this point in time. This patch only implements default mkfs parameter support for the test device. Config sections can be used to override this as check will then format the test device when the section that defines non-default test device mkfs options is selected. Signed-off-by: Dave Chinner --- check-parallel | 44 +++++++++++++++++++++++++++++++++++++------- 1 file changed, 37 insertions(+), 7 deletions(-) diff --git a/check-parallel b/check-parallel index 498b53c5a..38fa4f7f9 100755 --- a/check-parallel +++ b/check-parallel @@ -18,7 +18,7 @@ run_section="" tmp=/tmp/check-parallel.$$ -export FSTYP=xfs +FSTYP= # We need to include the test list processing first as argument parsing # requires test list parsing and setup. @@ -35,6 +35,7 @@ check options -r randomize test order --exact-order run tests in the exact order specified -s section run only specified section from config file + -f specify the filesystem type to test testlist options -g group[,group...] include tests from these groups @@ -66,16 +67,39 @@ external_file argument is a path to a single file containing a list of tests to exclude in the form of /. examples: - check-parallel -D /mnt xfs/001 - check-parallel -D /mnt -g quick + check-parallel -f xfs -D /mnt xfs/001 + check-parallel -f ext4 -D /mnt -g quick check-parallel -D /mnt -g xfs/quick check-parallel -D /mnt -x stress xfs/* - check-parallel -D /mnt -X .exclude -g auto - check-parallel -D /mnt -E ~/.xfstests.exclude + check-parallel -f btrfs -D /mnt -X .exclude -g auto + check-parallel -f udf -D /mnt -E ~/.xfstests.exclude ' exit 1 } +# Only support block device based filesystems with generic mkfs support +# at the moment. +is_supported_fstype() +{ + local fstype=$1 + + case $fstype in + xfs) ;; + ext2|ext3|ext4) ;; + udf) ;; + jfs) ;; + f2fs) ;; + btrfs) ;; + bcachefs) ;; + gfs2) ;; + ocfs2) ;; + *) + echo "unsupported FSTYPE: $fstype" + usage + ;; + esac +} + # Process command arguments first. while [ $# -gt 0 ]; do case "$1" in @@ -91,6 +115,8 @@ while [ $# -gt 0 ]; do --exact-order) _tl_setup_ordered ;; -n) show_test_list="yes" ;; + -f) is_supported_fstype $2 ; FSTYP=$2; shift ;; + -s) run_section="$run_section -s $2"; shift ;; -*) usage ;; @@ -108,6 +134,8 @@ while [ $# -gt 0 ]; do shift done +export FSTYP=${FSTYP:=xfs} + if [ ! -d "$basedir" ]; then echo "Invalid basedir specification" usage @@ -217,8 +245,6 @@ runner_go() xfs_io -f -c 'truncate 8g' $_scratch xfs_io -f -c 'truncate 1g' $_logwrites - mkfs.xfs -f $_test > /dev/null 2>&1 - export TEST_DEV=$(_create_loop_device $_test) export TEST_DIR=$me/test export SCRATCH_DEV=$(_create_loop_device $_scratch) @@ -232,6 +258,10 @@ runner_go() mkdir -p $RESULT_BASE rm -f $RESULT_BASE/check.* + # Only supports default mkfs parameters right now + wipefs $TEST_DEV > /dev/null 2>&1 + yes | mkfs -t $FSTYP $TEST_DEV > /dev/null 2>&1 + # export DUMP_CORRUPT_FS=1 # Run the tests in it's own mount namespace, as per the comment below