From patchwork Fri Jan 10 09:10:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nirjhar Roy (IBM)" X-Patchwork-Id: 13934010 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E213B1ADFE4; Fri, 10 Jan 2025 09:12:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736500333; cv=none; b=nalTxnwipPzt6HN3tCS53uKXJaZN0BnAWtSbAPZ+Hupjg3KVkmcqU+EEjc6XrCmQx4jxb6LHSikVcUJxFuRz9MPjB1sTeRYAi3qTYwIERUmiquNMoanmVNypHkaLKoh17GQTjIllu/TYEKWW6su1LIDiUJoJOnB8qShJaO1XwUE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736500333; c=relaxed/simple; bh=SguCOjP6z/+FgHUYSN3G1shA7kbiBLimwKmytp5P90g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eZHEjJPo3EEnew/fHWu/ukB+8ylj3BavhNYN0dUVEfwM1W2VDMJUlOPc9SmDvU/93yU20qSooMv3MOSxgT75jBPu1Wp2sqrdZyA70SqlyHCN6m+4lEy9L4kKn/QhwVqU6yU4IwHMcIdgEfvxGWaolyR4KUXSkrCj02R8MN6tfrQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GDyU3h8g; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GDyU3h8g" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-2163b0c09afso30816525ad.0; Fri, 10 Jan 2025 01:12:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736500331; x=1737105131; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mtH7dHpk2xqNXJcdg5awcc4LiW3ChxFkh539kBF/I5I=; b=GDyU3h8gcsJY+fnOM0uvy3KCf3B+YLOWvJQ/H74p3Cj2SgHhlLZVXKKjCT/PzrwArn YeUweuQnY49PBT6oQqxnBkuOS6i9e8AeyBRnyp/jcuFpmZlWswSalKQFLqeHkh3NBlUo d3TAgfce8drGvg1wa/2Nq3/198IUnZvMG5d0259EgBlQmrebsQIDpZVFIucsIgMJnoC3 hkYQd6X7oOfSEmImzZ6FtTrMTS5hYh2KxEEPRSmVmS2UUUmq2+pCMv0v4Rn2N3BUoUXt N/R8Ci0BWKneiIDlFBbbs03Kbv1AMYi8qxzXz/6OHYUE6UgfsVNu3jYSFPLbf8VycOfR LQBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736500331; x=1737105131; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mtH7dHpk2xqNXJcdg5awcc4LiW3ChxFkh539kBF/I5I=; b=AVuugZhLQQeihGESbcEfScevnQXtiHT90DNg7e3l3K41BNnFYxi9sr5YKa3yzou7U2 1aq2AiXaTQLwYYGLM1vYqofWt3gCZMDfmvOStIQCi6ka3lc4tQ8Euil7YXTgCguGo8KK iDQmlBXhXEf/Zj+DcyBEZLhk+Qs+f9WblMxFqyzeQDbaiLqBPWk3gCSNqf4B97DpW+mt qOedmachsnjObyZ5LR5gJdUYvJQQ5UxH+Otgv9KGLaR1snJ1ODCXkdGzWk4IBg/0m2UP nzJDzTc0IXNbvz2WiXYy3IUll/yDD0Kexiq1doTlUIJs2UK6/6NesujEPm22OrUzCGyJ qLzQ== X-Forwarded-Encrypted: i=1; AJvYcCWiG5iDJPWdvhuwiQ6fwttLWWoPazu4Ya2MLIIMvIrHXfuh5KfZNBQ2N3u603o1Ss1H/xnlpS3vuVU=@vger.kernel.org X-Gm-Message-State: AOJu0Yxn+1Uj+KGXJQ81kPVYBSRiVk28s8sUYCuIH7st2kAbIY3JpuZd kB36TAgGMu5lDhvDLMnNEf2dg3xmAIOsO6DXDkru8MWJvKwh59aYiJOJCApX X-Gm-Gg: ASbGncs2NOX2SqtXro1h2npzgDTydikv+Fboaba88oj4Ko0vnwc3HjNJ63ja0Xu0s5q wLOitVuu6YjrtiDbkDkBnmwhLiAK1K9pHtyhA7xes7o4BK4q73eO8ZJ+N68JnJkQEMQAOmGxyt9 wtomhCakXjmDKlRmdbhuTLc0qHtqZiqPk5dq+tvANtdvoTZ8C/E+Et/TDVoiDLRk66OWTeLw381 7Nufx3zf7M/bZkxIuFrueXQrApursqB3p8xtmmuLa/DZwdbaudeQhbAIJGd X-Google-Smtp-Source: AGHT+IFIrz3yln/sn6M4mR7oJa+B/nO6UiFxlTqaczM/WleL0UpwLrpkuywYRnyGnr91zqs+fdKwlA== X-Received: by 2002:a17:902:ecc5:b0:215:7dbf:f3de with SMTP id d9443c01a7336-21a83f5e4e5mr149319395ad.28.1736500330913; Fri, 10 Jan 2025 01:12:10 -0800 (PST) Received: from citest-1.. ([49.205.38.219]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21a9f219a94sm10166485ad.129.2025.01.10.01.12.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jan 2025 01:12:10 -0800 (PST) From: "Nirjhar Roy (IBM)" To: fstests@vger.kernel.org Cc: linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, ritesh.list@gmail.com, ojaswin@linux.ibm.com, djwong@kernel.org, zlang@kernel.org, nirjhar.roy.lists@gmail.com Subject: [RFC 1/5] tests/selftest: Add a new pseudo flaky test. Date: Fri, 10 Jan 2025 09:10:25 +0000 Message-Id: <03ba6c154c9e2cf3d68131b3af2ca12b96663d25.1736496620.git.nirjhar.roy.lists@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This test is to simulate the behavior of a flaky test. This will be required when we will make some modifications to the pass/fail metric calculation of the test infrastructure where we will need a test with non-zero pass and non-zero failure rate. Reviewed-by: Ritesh Harjani (IBM) Signed-off-by: Nirjhar Roy (IBM) --- tests/selftest/007 | 21 +++++++++++++++++++++ tests/selftest/007.out | 2 ++ 2 files changed, 23 insertions(+) create mode 100755 tests/selftest/007 create mode 100644 tests/selftest/007.out diff --git a/tests/selftest/007 b/tests/selftest/007 new file mode 100755 index 00000000..f100ec5f --- /dev/null +++ b/tests/selftest/007 @@ -0,0 +1,21 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024 IBM Corporation. All Rights Reserved. +# Author: Nirjhar Roy (IBM) +# +# FS QA Test 007 +# +# This test is to simulate the behavior of a flakey test. +# + +. ./common/preamble +_begin_fstest selftest + +if (($RANDOM % 2)); then + echo "Silence is golden" +else + echo "Silence is flakey" +fi + +status=0 +exit diff --git a/tests/selftest/007.out b/tests/selftest/007.out new file mode 100644 index 00000000..fd3590e6 --- /dev/null +++ b/tests/selftest/007.out @@ -0,0 +1,2 @@ +QA output created by 007 +Silence is golden From patchwork Fri Jan 10 09:10:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nirjhar Roy (IBM)" X-Patchwork-Id: 13934011 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BE3D209F4B; Fri, 10 Jan 2025 09:12:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736500351; cv=none; b=uffEMJJaMTae9JxCoDJH5tDQ7aNSj5ARpJeD1j52dvRE6L+iJeZAFj0KnFrU1eATe3bCCd/jqaAqPWe/uvmw5BrVMx8X2w00Ucss57VRIaXZs79HHA/9/cL2b6th6bik13lJ1Y03zYNdv7Yg8o//kCvUDJ+azHFedYkbUP7AMdE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736500351; c=relaxed/simple; bh=pLyCWYOzkv9DIEhBJmj6yPaQqZsogfCxBLYjsKjq/zc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rrQ4BsF2MXa7BEtWcBAXqUqtSvXneBoE30ekYDiDjzipQUP3btf7rGeEik1zhIR+dNSOwF5/8WxQaKIep0ORHsuXXTOiCwtLzz8T4qPRlrXgMzzC9TIGuckkdC7J6Je+dsscAynsbXRHYnq749+zMvpWKsZSVcD4gXIF+YIH0GA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XjxirZBV; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XjxirZBV" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-21670dce0a7so37231965ad.1; Fri, 10 Jan 2025 01:12:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736500349; x=1737105149; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dZmCRYH+zmel61PZtTGTzmCyDMxfmGVR2XKn2zcnKqI=; b=XjxirZBVw1KhlLWggwgFEb4T/amTvjT9LXJKAjhZd/3iYQNIa0ApEf3dCbCOfGFWZe dPYe4m4suptylVN3wgGzc5eITmjiqloJSfyaE4+EwRXLZLN+hFcVpg1yrKW+VPBuRVsK M7HscurCbAdLbpYJ2mr0aZbjbjbYly8qs3yOX5Y5ji47V7j3NZpRwxdJaAAAbtiJXAmE 4yT3YRRNY4PyNaUJCN66VbM5deM9PVsxDX0P28Sve2tAMZc5mxBB9HXfF1C5SnqvQ3dT CK7jv2pDirRWhoBstGe3qQPoeG5GtW+03YoVR+Cs7//qYDUV7YZ8uKGYQz1LREC9TLm6 M8lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736500349; x=1737105149; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dZmCRYH+zmel61PZtTGTzmCyDMxfmGVR2XKn2zcnKqI=; b=VcM50OCCmccU24dhI5uBXEpU2Xt8G4tn615T0M5am9ai4ntz4PjlL0FpwP/6eF3cuk 8X+VkQv7k1LnyQvLYhkYGXP6MSQwfaYPRBtF6unIQWw3FNYvXluwmcwDiH58Z1CLy6S4 ZUaoE60P+Xz1wsLGp+xJTjwGMko+8Zy5LIzZZmpN5KTb91hyyQSPhlFGgIGrVSGCUaNA qVkSZwnohVg0g5aCZhD2dStlbpWiprrNIFHwznqCB9C+xu1yPZgQEXl5Aumjzgi/WdFu 4nNpO+zZM+0scl9KSNJCXUhg/5ubuNsU88J7H379wdWB8BldomHFZ0wdT/w3UaX4C2z0 YFjA== X-Forwarded-Encrypted: i=1; AJvYcCUlS3lnwA388h581WYuF5Mmrr1n5w97WIk7GcNN6Foen+EcN3d7uvOBM1h1lItTUsU3MQCaFb8mxZk=@vger.kernel.org X-Gm-Message-State: AOJu0YzR97YxROcvYKqDlvGkBlq3wQmyxC7X+tz8ZIbEHk/UPUZ0u0AJ 8n7ysRL8jknOOa9oW8oCghB1wba8TUsljM/8xtBcbQ2mZAsm0+mYIMywEwIh X-Gm-Gg: ASbGncvFw0JrMU+WuppEHAp00zsVNThdX6dGdYgxi3t57yK9unMbbTjNVFnbHYojqJ5 oDbFvu2qHVoZI7VWsxPLXAOanIhIgzfh+Xlpfjmpagv7n8eC5qVDb2DMKWRHqvRUdLUvZaksUx/ seouGiPLc2dWB2OS8xtZFaDokzsk5tnIHKWRrkVKmbhBm8LSvnN7hVBL95Y4MsHAdWGmRAjhdbb HRJIWgwZEFul8+20J0V3ck8EURbl8JKewMdYkcdBCyN/kQpVYGHc7tbTxD2 X-Google-Smtp-Source: AGHT+IHEabX6sx3xiDacZPapqA2JUDOh6nxGbTfqkIlolXAJ/ApKLa2RXafYfx/1t4yr4UWjSLXibQ== X-Received: by 2002:a17:903:230b:b0:20c:6399:d637 with SMTP id d9443c01a7336-21a83ffbea3mr178475105ad.40.1736500348613; Fri, 10 Jan 2025 01:12:28 -0800 (PST) Received: from citest-1.. ([49.205.38.219]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21a9f219a94sm10166485ad.129.2025.01.10.01.12.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jan 2025 01:12:28 -0800 (PST) From: "Nirjhar Roy (IBM)" To: fstests@vger.kernel.org Cc: linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, ritesh.list@gmail.com, ojaswin@linux.ibm.com, djwong@kernel.org, zlang@kernel.org, nirjhar.roy.lists@gmail.com Subject: [RFC 2/5] check: Add -q option to support unconditional looping. Date: Fri, 10 Jan 2025 09:10:26 +0000 Message-Id: <1826e6084fd71e3e9755b1d2750876eb5f0e1161.1736496620.git.nirjhar.roy.lists@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch adds -q option through which one can run a given test times unconditionally. It also prints pass/fail metrics at the end. The advantage of this over -L and -i/-I is that: a. -L will not re-run a flakey test if the test passes for the first time. b. -I/-i sets up devices during each iteration and hence slower. Note -q will override -L . Also rename _stash_fail_loop_files() to _stash_loop_files() because this function will now be used even when the test doesn't fail (i.e. when ran with -q ). Suggested-by: Ritesh Harjani (IBM) Signed-off-by: Nirjhar Roy (IBM) Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) --- check | 46 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 36 insertions(+), 10 deletions(-) diff --git a/check b/check index 607d2456..e8ac0ce9 100755 --- a/check +++ b/check @@ -11,6 +11,7 @@ needsum=true try=() sum_bad=0 bad=() +is_bad_test=false notrun=() interrupt=true diff="diff -u" @@ -27,6 +28,7 @@ DUMP_OUTPUT=false iterations=1 istop=false loop_on_fail=0 +loop_unconditional=0 exclude_tests=() # This is a global variable used to pass test failure text to reporting gunk @@ -83,6 +85,7 @@ check options -s section run only specified section from config file -S section exclude the specified section from the config file -L loop tests times following a failure, measuring aggregate pass/fail metrics + -q loop tests times irrespective of a pass or a failure, measuring aggregate pass/fail metrics testlist options -g group[,group...] include tests from these groups @@ -341,7 +344,9 @@ while [ $# -gt 0 ]; do -L) [[ $2 =~ ^[0-9]+$ ]] || usage loop_on_fail=$2; shift ;; - + -q) [[ $2 =~ ^[0-9]+$ ]] || usage + loop_unconditional=$(($2 - 1)); shift + ;; -*) usage ;; *) # not an argument, we've got tests now. have_test_arg=true ;; @@ -357,6 +362,11 @@ while [ $# -gt 0 ]; do shift done +# -q overrides -L +if [ "$loop_unconditional" -gt 0 ]; then + loop_on_fail=0 +fi + # we need common/rc, that also sources common/config. We need to source it # after processing args, overlay needs FSTYP set before sourcing common/config if ! . ./common/rc; then @@ -609,7 +619,7 @@ _expunge_test() } # retain files which would be overwritten in subsequent reruns of the same test -_stash_fail_loop_files() { +_stash_loop_files() { local seq_prefix="${REPORT_DIR}/${1}" local cp_suffix="$2" @@ -633,10 +643,18 @@ _stash_test_status() { fi if ((${#loop_status[*]} > 0)); then - # continuing or completing rerun-on-failure loop - _stash_fail_loop_files "$test_seq" ".rerun${#loop_status[*]}" + # continuing or completing rerun loop + _stash_loop_files "$test_seq" ".rerun${#loop_status[*]}" loop_status+=("$test_status") - if ((${#loop_status[*]} > loop_on_fail)); then + + # only stash @bad result for initial failure in loop + if [[ "$test_status" == "fail" ]] && ! $is_bad_test; then + bad+=("$test_seq") + is_bad_test=true + fi + + if ((loop_on_fail && ${#loop_status[*]} > loop_on_fail)) || \ + ((loop_unconditional && ${#loop_status[*]} > loop_unconditional)); then printf "%s aggregate results across %d runs: " \ "$test_seq" "${#loop_status[*]}" awk "BEGIN { @@ -650,23 +668,30 @@ _stash_test_status() { }' echo loop_status=() + is_bad_test=false fi - return # only stash @bad result for initial failure in loop + return fi case "$test_status" in fail) - if ((loop_on_fail > 0)); then - # initial failure, start rerun-on-failure loop - _stash_fail_loop_files "$test_seq" ".rerun0" + # re-run if either of the loop argument is set + if ((loop_on_fail > 0)) || ((loop_unconditional > 0)); then + _stash_loop_files "$test_seq" ".rerun0" loop_status+=("$test_status") fi bad+=("$test_seq") + is_bad_test=true ;; list|notrun) notrun+=("$test_seq") ;; pass|expunge) + # re-run if loop_unconditional argument is set + if ((loop_unconditional > 0)); then + _stash_loop_files "$test_seq" ".rerun0" + loop_status+=("$test_status") + fi ;; *) echo "Unexpected test $test_seq status: $test_status" @@ -857,7 +882,8 @@ function run_section() seqres="$check" _check_test_fs - loop_status=() # track rerun-on-failure state + is_bad_test=false + loop_status=() # track loop rerun state local tc_status ix local -a _list=( $list ) for ((ix = 0; ix < ${#_list[*]}; !${#loop_status[*]} && ix++)); do From patchwork Fri Jan 10 09:10:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nirjhar Roy (IBM)" X-Patchwork-Id: 13934012 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 63AF51ADFE4; Fri, 10 Jan 2025 09:12:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.195 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736500378; cv=none; b=mHVcWM632dRM4XSwrYheYcdn1M+9vEDiygOPUmzlGOf6BAS4fThQcM5QTdHY8kgCG4VsU7RI2FS7PAGFJN6qdZADR8fYcmciqNUbul6rQddxsMm45v2JbS4tOLKGI+W/sBtQGN//WefK6ArSfv63C6QHrRUlgVkdFItTQLP5jPs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736500378; c=relaxed/simple; bh=UfuCQdTf8KFSIx0HUKuDNJx7sXmm6WeKrstDgVeYf3w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TVKh1Huf6gsLUczwpCpDHODgJIRqvAkf+u7GInDfwxFBYfHu4hFySgKDXbR+gRbVnuhoK1uAcH7JJkwaW3GAovOlwQNDR7mSE+6zDIeAfF9xmRQLh7ehO4IB7i3DBOY3L3uN4eH77ZMUkDQpy+ASuNJzeOeVlTBDpv0xaqTleFA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MhDCFGyA; arc=none smtp.client-ip=209.85.214.195 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MhDCFGyA" Received: by mail-pl1-f195.google.com with SMTP id d9443c01a7336-2156e078563so26055615ad.2; Fri, 10 Jan 2025 01:12:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736500375; x=1737105175; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MNHVd5f+RmGJ130wS048pYq3Wu/sA+aJSBfgORf1PRM=; b=MhDCFGyA5EZFwDrgxQHTC6G8wh6ljzKCcY2OYmS9RpBP7eD0y3ySvj/g2x95J1qdJp 9GftbC4JwDmnwowfpqXBETY43bzrVG8THHsDNdeeDubAo4cJZimoY2hogFZTZRXzNSyX hE0nNU+4OXxo/VxsMgjzD/ugMJt8SmUC2oSMp+Nxwr/neMcMaQQmS2LNwZTuDKIOMSui Ixlro5xnFFFh1fZujtMOFd1O0toBsRVSlBaS96VrogKEI+0tCreD12TMVyFccSJJM6qz Qe6uq9BCC/ELL8szUfR3vAV+lUn+2LeAEoTMPr1RoltdG9eBsENRrDC7lBqqjWOnj5CR H39w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736500375; x=1737105175; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MNHVd5f+RmGJ130wS048pYq3Wu/sA+aJSBfgORf1PRM=; b=vfrOwl6s1bAny/sN1GcoL6IsaMhAh5F/rRRXOhKfhBbFdHqeXe8g9NYZOMcdPSfFbj kDz1moyQWlSlNh1IkblUKuZ/Xqv7HC263UXgLOh92euf68mBX6ICJsbbMYLZ7j5GPB+7 c1fkWy1Pua7jNdYYmA8nftvKyN3XOtUN3AqFiR+JhKyCUiYC5NLBr/QpIGFoaFuD8+gH jhCVjA7G1Q1RHlvtVNHE8UUWRmkEx83xBKiefA8zVgTBL40Tij/oI6B1GLUNHltzU0fo FkpOlYnF/OKwDFCVNSknk23RITQHfNmmL+ALEh1G8yq7SB7oVv/SraPXoK/EdjWCJP7c XK6w== X-Forwarded-Encrypted: i=1; AJvYcCW7mU+Z8WYui5y8T/QdLNZEeVJv5edw6Slx5/aW78zOVBqxu7FFxhGlHs9wSbQVNNx3+/CyoxKs/Mc=@vger.kernel.org X-Gm-Message-State: AOJu0YxoKycNe8g7TqUuS0RxKcRuRsYqwBkKxBiBRThpBndMa0/M1Ood lRcVD/tw23FG60tfXoSMJXBwY2izt5UM7XJyj1PHThxNb0WqumfImD9QeGIIyPg= X-Gm-Gg: ASbGncseOANPFW/DYSUSqtUckEX/r/mVIU963Tl7OQlkaqMdTAB/iCZTbnNPfD5zpBA BJTQiUBXiqZLMoM6IXCODD5Dix0aAvxK49aMDCSiyux5YrKeP88T2Iu8AcT1vJInY9kCTdfZupf 3NkkjN7dYs9oRLcUiGDPlvMpTZplQFYOLRBnFZXsIcu5apYCiYHdP+8b8R/49LuZkS0CSjoDbPe 6Q29KGCO2h/Cc90EyoF7/X6arpKih1IEeVzAsFbpaGmLcQqWEiEQGIaSO8A X-Google-Smtp-Source: AGHT+IGTz8lGbytPSR6x1DXX/SDtRvQlD82ShBxJ6P4sfSUQfoU939GTq2584BeKyM61xd8YITJrFQ== X-Received: by 2002:a17:902:d50d:b0:216:2abc:194f with SMTP id d9443c01a7336-21a83fb1563mr149881585ad.40.1736500375196; Fri, 10 Jan 2025 01:12:55 -0800 (PST) Received: from citest-1.. ([49.205.38.219]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21a9f219a94sm10166485ad.129.2025.01.10.01.12.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jan 2025 01:12:54 -0800 (PST) From: "Nirjhar Roy (IBM)" To: fstests@vger.kernel.org Cc: linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, ritesh.list@gmail.com, ojaswin@linux.ibm.com, djwong@kernel.org, zlang@kernel.org, nirjhar.roy.lists@gmail.com Subject: [RFC 3/5] check: Improve pass/fail metrics and section config output Date: Fri, 10 Jan 2025 09:10:27 +0000 Message-Id: <05768096e43da32f50ef337c0676d06fb8c9454e.1736496620.git.nirjhar.roy.lists@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch improves the output by: - Adding pass/fail aggregate results in the stdout for each section's summary. Also adds the location of the xunit xml files in the section's summary. - Adds section config details into the result's global log file. e.g. $ ./check -q 2 -R xunit-quiet selftest/001 SECTION -- s1 ========================= selftest/001 aggregate results across 2 runs: pass=2 (100.0%) Ran: selftest/001 Passed all 1 tests Xunit report: /home/ubuntu/xfstests/results//s1/result.xml SECTION -- s2 ========================= selftest/001 aggregate results across 2 runs: pass=2 (100.0%) Ran: selftest/001 Passed all 1 tests Xunit report: /home/ubuntu/xfstests/results//s2/result.xml Output in results/check.log file Kernel version: 6.13.0-rc1-00182-gb8f52214c61a-dirty Wed Jan 8 11:23:57 UTC 2025 SECTION -- s1 ========================= FSTYP -- ext4 PLATFORM -- Linux/x86_64 citest-1 6.13.0-rc1-00182-gb8f52214c61a-dirty #5 SMP PREEMPT_DYNAMIC Wed Dec 18 14:03:34 IST 2024 MKFS_OPTIONS -- -F /dev/loop1 MOUNT_OPTIONS -- -o acl,user_xattr /dev/loop1 /mnt1/scratch selftest/001 aggregate results across 2 runs: pass=2 (100.0%) Ran: selftest/001 Passed all 1 tests Xunit report: /home/ubuntu/xfstests/results//s1/result.xml Kernel version: 6.13.0-rc1-00182-gb8f52214c61a-dirty Wed Jan 8 11:24:02 UTC 2025 SECTION -- s2 ========================= FSTYP -- ext4 PLATFORM -- Linux/x86_64 citest-1 6.13.0-rc1-00182-gb8f52214c61a-dirty #5 SMP PREEMPT_DYNAMIC Wed Dec 18 14:03:34 IST 2024 MKFS_OPTIONS -- -F /dev/loop1 MOUNT_OPTIONS -- -o acl,user_xattr /dev/loop1 /mnt1/scratch selftest/001 aggregate results across 2 runs: pass=2 (100.0%) Ran: selftest/001 Passed all 1 tests Xunit report: /home/ubuntu/xfstests/results//s2/result.xml Suggested-by: Ritesh Harjani Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Signed-off-by: Nirjhar Roy (IBM) --- check | 67 ++++++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 46 insertions(+), 21 deletions(-) diff --git a/check b/check index e8ac0ce9..f7028836 100755 --- a/check +++ b/check @@ -468,6 +468,25 @@ if [ -n "$REPORT_GCOV" ]; then _gcov_check_report_gcov fi +_print_list() +{ + local -n list=$1 + local item + for item in "${list[@]}"; do + echo "$item" + done +} + +_display_test_configuration() +{ + echo "FSTYP -- `_full_fstyp_details`" + echo "PLATFORM -- `_full_platform_details`" + if [ ! -z "$SCRATCH_DEV" ]; then + echo "MKFS_OPTIONS -- `_scratch_mkfs_options`" + echo "MOUNT_OPTIONS -- `_scratch_mount_options`" + fi +} + _wrapup() { seq="check.$$" @@ -505,11 +524,19 @@ _wrapup() echo "SECTION -- $section" >>$tmp.summary echo "=========================" >>$tmp.summary + + _global_log "SECTION -- $section" + _global_log "=========================" >>$tmp.summary + _global_log "$(_display_test_configuration)" if ((${#try[*]} > 0)); then + local test_aggr_stats=$(_print_list loop_test_stats_per_section) if [ $brief_test_summary == "false" ]; then + [[ -n "$test_aggr_stats" ]] && echo "$test_aggr_stats" >> \ + $tmp.summary echo "Ran: ${try[*]}" echo "Ran: ${try[*]}" >>$tmp.summary fi + _global_log "$test_aggr_stats" _global_log "Ran: ${try[*]}" fi @@ -539,12 +566,15 @@ _wrapup() _global_log "Passed all ${#try[*]} tests" echo "Passed all ${#try[*]} tests" >>$tmp.summary fi - echo "" >>$tmp.summary if $do_report; then _make_section_report "$section" "${#try[*]}" \ "${#bad[*]}" "${#notrun[*]}" \ "$((sect_stop - sect_start))" + local out_fn="$REPORT_DIR/result.xml" + echo "Xunit report: $out_fn" >> $tmp.summary + _global_log "Xunit report: $out_fn" fi + echo "" >>$tmp.summary # Generate code coverage report if [ -n "$REPORT_GCOV" ]; then @@ -655,18 +685,19 @@ _stash_test_status() { if ((loop_on_fail && ${#loop_status[*]} > loop_on_fail)) || \ ((loop_unconditional && ${#loop_status[*]} > loop_unconditional)); then - printf "%s aggregate results across %d runs: " \ - "$test_seq" "${#loop_status[*]}" - awk "BEGIN { - n=split(\"${loop_status[*]}\", arr);"' - for (i = 1; i <= n; i++) - stats[arr[i]]++; - for (x in stats) - printf("%s=%d (%.1f%%)", - (i-- > n ? x : ", " x), - stats[x], 100 * stats[x] / n); - }' - echo + local test_stats=$(printf "%s aggregate results across %d runs: " \ + "$test_seq" "${#loop_status[*]}") + test_stats+=$(awk "BEGIN { + n=split(\"${loop_status[*]}\", arr);"' + for (i = 1; i <= n; i++) + stats[arr[i]]++; + for (x in stats) + printf("%s=%d (%.1f%%)", + (i-- > n ? x : ", " x), + stats[x], 100 * stats[x] / n); + }') + echo "$test_stats" + loop_test_stats_per_section+=("$test_stats") loop_status=() is_bad_test=false fi @@ -839,14 +870,7 @@ function run_section() rm -f $check.full [ -f $check.time ] || touch $check.time - - # print out our test configuration - echo "FSTYP -- `_full_fstyp_details`" - echo "PLATFORM -- `_full_platform_details`" - if [ ! -z "$SCRATCH_DEV" ]; then - echo "MKFS_OPTIONS -- `_scratch_mkfs_options`" - echo "MOUNT_OPTIONS -- `_scratch_mount_options`" - fi + _display_test_configuration echo test -n "$REPORT_GCOV" && _gcov_reset needwrap=true @@ -884,6 +908,7 @@ function run_section() is_bad_test=false loop_status=() # track loop rerun state + loop_test_stats_per_section=() # store loop test statistics per section local tc_status ix local -a _list=( $list ) for ((ix = 0; ix < ${#_list[*]}; !${#loop_status[*]} && ix++)); do From patchwork Fri Jan 10 09:10:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Nirjhar Roy (IBM)" X-Patchwork-Id: 13934013 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DA1B206F06; Fri, 10 Jan 2025 09:13:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736500393; cv=none; b=aGVJidKKOt0Q1hdhsKTVfLU713AScGg+ZVPtX1XG2a3GKdEa0MWkTi/rk6d23A+I7K6WCBHF5MiJmEMeUrBlf/F1witBuk1theP8JXWHpwXWOC4QtPE4E9IjZPAG4ytoFZL//z/NDtjfXSAt8E7hzISyMu938YulvnbXu5a3eNc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736500393; c=relaxed/simple; bh=osXmSc+d9s0VDc8vmJPxoNfJLMIqnBEOpIPI8GJnhCg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=r6Z+ieDbJsoJWOph5E4n+Ss4Oku1knejd7/omzpfYMX3C9VneaSrZSqbYG2yhjV44lH46p6lShnQPyocBNAAiELTew8ODhvE95MW/iwr5w9FGRB1bFgA5X8MFRsfiJTyYrijpx337w8IZpRIyCVGyLONOvxEqmo3J02Vs69bBG0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=j4HIUt1M; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="j4HIUt1M" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-21661be2c2dso28181035ad.1; Fri, 10 Jan 2025 01:13:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736500390; x=1737105190; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Nwfekoxfp+MbUnjT7fxgtz18HcGaexSsgzbOvFkAHUw=; b=j4HIUt1MT0f5Y+FS4GTW+WQ1zYGzE/qmTTQzn1raeDjx/VOd2reJWgHq8uaG5iBVVH /l+GpfEgojkFdPPrTTiULGNAMOO+i1l+3LYnnYblLL6XAD5X8wgnbjQ0xe4LupAwDvle zFTs7UMMXf6cmTWqLbk1v59U7B9eQLugVGy4Yq8nJvWgiQWG4n7pWS5ckKyfiHmS6IAz qm1gp7rFKwLCYlS94j8A3sIbnr4wjHh40tH4v8XYWKDzKGyKdglIoMLjgbZaaR7mfXC+ 1/kzJPD/azWs6ZxQA65MTEilbvI31no/K/UdC4u2C+2oKkMg0qWy+6855zQVqeJpnqL4 3G7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736500390; x=1737105190; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Nwfekoxfp+MbUnjT7fxgtz18HcGaexSsgzbOvFkAHUw=; b=V2fzXyKHz3us9a1K/rQgvO5d4WD1Sv4Vh+kSQMtbN50l4sppv+nqU7wmPlJJfarY1R 5ZEhrS3b9Z7QgcAfTy4x7pLm/2Mzm+JzMH4oesufKLNFPwFGDHfEp7hAhekG5H627YwR Df2BuZsLMsblq9jZU1BAjDCX6B2PSrsHXCM5DtoWkLMbgaFi+WlkhG2V+IZ3rsdSNrTR ZtPz5LoPBZuf7BqATUBvbL73DpwLOpOx/C4AHC21T6CI6EA2WIz0Dd0AhUooAFvp01QW YyU687O3FRDp5k9B52R/NSQqhr0i903uhIaWO5kG54vDyg8d+PlPX5oykstkhFjuyJsB 67tA== X-Forwarded-Encrypted: i=1; AJvYcCXpHNU5kzQg+1yPVWS1CQ0Xs0uVs7bC5g8Jj5g+FSS+v7xc3n3ho0zmEESi/xrhaXluvQYePzzQ6Kk=@vger.kernel.org X-Gm-Message-State: AOJu0YyGpugPCKZRv6jR2fvv5auMDt4dBoNaOzutCf9kh98ak4oXMDcy CNbD5YOrXrvkTY7J8LdbW/DfJTMmREieEhQHlbazOHAEfk6N9GLdImTwtrwm X-Gm-Gg: ASbGncuXFRdjuGfvccpFv5OVviXWk1Mcap+Bj8PELXiwM05aPhQzDB0B3l3QhMZlXai 23Gv3WydCpMrNarY7D0WCPZLSI2iKTBcMUm6KsnzTU552rT4hpSdow/EokRbxYwhiQrR78lP/8a d/Lfy2MVdNpZJwdVyMvW1eheogAMsYqvj2CXVCoQ0Ppkn1qy9GCwi/y8+y54W+NER4b1txZsvIX JQDa164cq1j40IOVeBD2d5Y99B4Gh6s/c/TsUPRSgrB74dr7LYfrg+qD4/8 X-Google-Smtp-Source: AGHT+IEUV/d+GJdKrslMUpeAxaK60BkEdx3rsMi9BlIZv1o1JTeUFabz1qArW9QesRohebfoq/1zbA== X-Received: by 2002:a17:902:cccb:b0:216:6769:9eca with SMTP id d9443c01a7336-21a83fd35bdmr147260335ad.37.1736500389931; Fri, 10 Jan 2025 01:13:09 -0800 (PST) Received: from citest-1.. ([49.205.38.219]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21a9f219a94sm10166485ad.129.2025.01.10.01.13.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jan 2025 01:13:09 -0800 (PST) From: "Nirjhar Roy (IBM)" To: fstests@vger.kernel.org Cc: linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, ritesh.list@gmail.com, ojaswin@linux.ibm.com, djwong@kernel.org, zlang@kernel.org, nirjhar.roy.lists@gmail.com Subject: [RFC 4/5] check,common/config: Add support for central fsconfig Date: Fri, 10 Jan 2025 09:10:28 +0000 Message-Id: <9a6764237b900f40e563d8dee2853f1430245b74.1736496620.git.nirjhar.roy.lists@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This adds support to pick and use any existing FS config from configs//. e.g. configs/xfs/1k configs/xfs/4k configs/ext4/4k configs/ext4/64k This should help us maintain and test different fs test configurations from a central place. We also hope that this will be useful for both developers and testers to look into what is being actively maintained and tested by FS Maintainers. When we will have fsconfigs set, then will be another subdirectory created in results/
. For example let's look at the following: The directory tree structure on running sudo ./check -q 2 -R xunit-quiet -c xfs/4k,configs/xfs/1k selftest/001 selftest/007 $ tree results/ results/ ├── check.full ├── check.log ├── check.time ├── xfs_1 │ └── xfs │ ├── 1k │ │ ├── check.log │ │ ├── check.time │ │ ├── result.xml │ │ └── selftest │ │ ├── 001.full │ │ ├── 001.full.rerun0 │ │ ├── 001.full.rerun1 │ │ ├── 007.full │ │ ├── 007.full.rerun0 │ │ ├── 007.full.rerun1 │ │ ├── 007.out.bad │ │ ├── 007.out.bad.rerun0 │ │ ├── 007.out.bad.rerun1 │ │ └── dmesg_filter │ └── 4k │ ├── check.log │ ├── check.time │ ├── result.xml │ └── selftest │ ├── 001.full │ ├── 001.full.rerun0 │ ├── 001.full.rerun1 │ ├── 007.full │ ├── 007.full.rerun0 │ ├── 007.full.rerun1 │ ├── 007.out.bad │ ├── 007.out.bad.rerun0 │ ├── 007.out.bad.rerun1 │ └── dmesg_filter └── xfs_2 └── xfs ├── 1k │ ├── check.log │ ├── check.time │ ├── result.xml │ └── selftest │ ├── 001.full │ ├── 001.full.rerun0 │ ├── 001.full.rerun1 │ ├── 007.full │ ├── 007.full.rerun0 │ ├── 007.full.rerun1 │ └── dmesg_filter └── 4k ├── check.log ├── check.time ├── result.xml └── selftest ├── 001.full ├── 001.full.rerun0 ├── 001.full.rerun1 ├── 007.full ├── 007.full.rerun0 ├── 007.full.rerun1 ├── 007.out.bad ├── 007.out.bad.rerun1 └── dmesg_filter 12 directories, 51 files Contents of one of the xml/xunit files (results/xfs_1/xfs/4k/result.xml): (Please note that FS_CONFIG_OPTION is a new property that is added in this patch) ... ... Suggested-by: Ritesh Harjani Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Signed-off-by: Nirjhar Roy (IBM) --- README.config-sections | 12 +++++++ check | 82 +++++++++++++++++++++++++++++++++--------- common/config | 51 ++++++++++++++++++++++++-- common/report | 2 +- 4 files changed, 128 insertions(+), 19 deletions(-) diff --git a/README.config-sections b/README.config-sections index a42d9d7b..0511b707 100644 --- a/README.config-sections +++ b/README.config-sections @@ -108,10 +108,22 @@ MKFS_OPTIONS="-q -F -b1024" [ext4_nojournal] MKFS_OPTIONS="-q -F -b4096 -O ^has_journal" +# we could also do like: FS_CONFIG_OPTION=ext4/1k +[ext4_1k_bs_fsconfig] +FS_CONFIG_OPTION=configs/ext4/1k + +# we could also do like: FS_CONFIG_OPTION=ext4/nojournal +[ext4_nojournal_fsconfig] +FS_CONFIG_OPTION=configs/ext4/nojournal + [xfs_filesystem] MKFS_OPTIONS="-f" FSTYP=xfs +# we could also do like: FS_CONFIG_OPTION=xfs/4k +[xfs_4k_bs_fsconfig] +FS_CONFIG_OPTION=configs/xfs/4k + [ext3_filesystem] FSTYP=ext3 MOUNT_OPTIONS="-o noatime" diff --git a/check b/check index f7028836..90aa94ba 100755 --- a/check +++ b/check @@ -80,6 +80,7 @@ check options -I iterate the test list times, but stops iterating further in case of any test failure -d dump test output to stdout -b brief test summary + -c use fs configs from configs// e.g. configs/xfs/4k -R fmt[,fmt] generate report in formats specified. Supported formats: xunit, xunit-quiet --large-fs optimise scratch device for large filesystems -s section run only specified section from config file @@ -116,6 +117,14 @@ excluded from the list of tests to run from that test dir. external_file argument is a path to a single file containing a list of tests to exclude in the form of /. +-c can be used in various formats as mentioned below. We can also set the +FS_CONFIG_OPTION environment variable in the local.config file and get the +same functionality. e.g. + 1) FS_CONFIG_OPTION=xfs/4k + 2) FS_CONFIG_OPTION=configs/xfs/4k +Please note that cmdline will take precedence if FS_CONFIG_OPTION is +passed in both local.config file and via cmdline. + examples: check xfs/001 check -g quick @@ -123,6 +132,9 @@ examples: check -x stress xfs/* check -X .exclude -g auto check -E ~/.xfstests.exclude + check -c xfs/4k,ext4/1k + check -c xfs/4k -c configs/ext4/1k + check -c $(echo configs/xfs/* | tr ' ' ',') -q 5 tests/selftest/007 ' exit 1 } @@ -312,6 +324,8 @@ while [ $# -gt 0 ]; do ;; -s) RUN_SECTION="$RUN_SECTION $2"; shift ;; -S) EXCLUDE_SECTION="$EXCLUDE_SECTION $2"; shift ;; + -c) CMD_FS_CONFIG_OPTION="$CMD_FS_CONFIG_OPTION$2,"; + export OPTIONS_HAVE_FSCONFIGS=true; shift ;; -l) diff="diff" ;; -udiff) diff="$diff -u" ;; @@ -458,7 +472,7 @@ _wipe_counters() _global_log() { echo "$1" >> $check.log - if $OPTIONS_HAVE_SECTIONS; then + if options_have_sections_or_fs_configs; then echo "$1" >> ${REPORT_DIR}/check.log fi } @@ -513,7 +527,7 @@ _wrapup() }' \ | sort -n >$tmp.out mv $tmp.out $check.time - if $OPTIONS_HAVE_SECTIONS; then + if options_have_sections_or_fs_configs; then cp $check.time ${REPORT_DIR}/check.time fi fi @@ -524,9 +538,14 @@ _wrapup() echo "SECTION -- $section" >>$tmp.summary echo "=========================" >>$tmp.summary - _global_log "SECTION -- $section" _global_log "=========================" >>$tmp.summary + + if [[ -n $FS_CONFIG_OPTION ]]; then + echo "FSCONFIG -- $FS_CONFIG_OPTION" >> $tmp.summary + _global_log "FSCONFIG -- $FS_CONFIG_OPTION" + fi + _global_log "$(_display_test_configuration)" if ((${#try[*]} > 0)); then local test_aggr_stats=$(_print_list loop_test_stats_per_section) @@ -541,7 +560,7 @@ _wrapup() fi $interrupt && echo "Interrupted!" | tee -a $check.log - if $OPTIONS_HAVE_SECTIONS; then + if options_have_sections_or_fs_configs; then $interrupt && echo "Interrupted!" | tee -a \ ${REPORT_DIR}/check.log fi @@ -592,7 +611,7 @@ _wrapup() sum_bad=`expr $sum_bad + ${#bad[*]}` _wipe_counters - if ! $OPTIONS_HAVE_SECTIONS; then + if ! options_have_sections_or_fs_configs; then rm -f $tmp.* fi } @@ -773,7 +792,7 @@ _detect_kmemleak _prepare_test_list fstests_start_time="$(date +"%F %T")" -if $OPTIONS_HAVE_SECTIONS; then +if options_have_sections_or_fs_configs; then trap "_summary; exit \$status" 0 1 2 3 15 else trap "_wrapup; exit \$status" 0 1 2 3 15 @@ -782,6 +801,7 @@ fi function run_section() { local section=$1 skip + local fsconfig=$2 OLD_FSTYP=$FSTYP OLD_TEST_FS_MOUNT_OPTS=$TEST_FS_MOUNT_OPTS @@ -814,7 +834,10 @@ function run_section() fi fi - get_next_config $section + # FS_CONFIG_OPTION (if provided) will only be set after get_next_config + # has run which will export the FS_CONFIG_OPTION, so that we can use it + # later + get_next_config $section $fsconfig _canonicalize_devices mkdir -p $RESULT_BASE @@ -828,6 +851,10 @@ function run_section() echo "SECTION -- $section" fi + if $OPTIONS_HAVE_FSCONFIGS; then + echo "FSCONFIG -- $FS_CONFIG_OPTION" + fi + sect_start=`_wallclock` if $RECREATE_TEST_DEV || [ "$OLD_FSTYP" != "$FSTYP" ]; then echo "RECREATING -- $FSTYP on $TEST_DEV" @@ -849,7 +876,12 @@ function run_section() # TEST_DEV has been recreated, previous FSTYP derived from # TEST_DEV could be changed, source common/rc again with # correct FSTYP to get FSTYP specific configs, e.g. common/xfs + # sourcing common/rc can re-set FS_CONFIG_OPTION to the value in + # local.config file(if present). So store and restore the value of + # FS_CONFIG_OPTION which is there currently at this point. + local fs_config_tmp="$FS_CONFIG_OPTION" . common/rc + export FS_CONFIG_OPTION="$fs_config_tmp" _prepare_test_list elif [ "$OLD_TEST_FS_MOUNT_OPTS" != "$TEST_FS_MOUNT_OPTS" ]; then _test_unmount 2> /dev/null @@ -933,10 +965,14 @@ function run_section() # we don't include the tests/ directory in the name output. export seqnum=${seq#$SRC_DIR/} group=${seqnum%%/*} + + REPORT_DIR="$RESULT_BASE" if $OPTIONS_HAVE_SECTIONS; then - REPORT_DIR="$RESULT_BASE/$section" - else - REPORT_DIR="$RESULT_BASE" + REPORT_DIR="$REPORT_DIR/$section" + fi + if $OPTIONS_HAVE_FSCONFIGS; then + local fsconfig_subdir="${FS_CONFIG_OPTION//configs/''}" + REPORT_DIR="$REPORT_DIR/$fsconfig_subdir" fi export RESULT_DIR="$REPORT_DIR/$group" seqres="$REPORT_DIR/$seqnum" @@ -1151,14 +1187,28 @@ function run_section() _scratch_unmount 2> /dev/null } +prepare_fs_config_list() +{ + IFS=',' read -r -a CMD_FS_CONFIG_OPTION_LIST <<< "$CMD_FS_CONFIG_OPTION" + CMD_FS_OPTION_LIST_LEN=${#CMD_FS_CONFIG_OPTION_LIST[@]} + if [ $CMD_FS_OPTION_LIST_LEN -eq 0 ]; then + # Create an empty fsconfig "" to pass to run_section in iters loop + CMD_FS_CONFIG_OPTION_LIST=("") + fi +} + +prepare_fs_config_list + for ((iters = 0; iters < $iterations; iters++)) do for section in $HOST_OPTIONS_SECTIONS; do - run_section $section - if [ "$sum_bad" != 0 ] && [ "$istop" = true ]; then - interrupt=false - status=`expr $sum_bad != 0` - exit - fi + for fsconfig in "${CMD_FS_CONFIG_OPTION_LIST[@]}"; do + run_section $section $fsconfig + if [ "$sum_bad" != 0 ] && [ "$istop" = true ]; then + interrupt=false + status=`expr $sum_bad != 0` + exit + fi + done done done diff --git a/common/config b/common/config index 5f86fc2c..901dc1e8 100644 --- a/common/config +++ b/common/config @@ -343,6 +343,11 @@ if [ -x /usr/sbin/selinuxenabled ] && /usr/sbin/selinuxenabled; then export SELINUX_MOUNT_OPTIONS fi +options_have_sections_or_fs_configs() +{ + $OPTIONS_HAVE_SECTIONS || $OPTIONS_HAVE_FSCONFIGS +} + _common_mount_opts() { case $FSTYP in @@ -783,8 +788,32 @@ parse_config_section() { | sed -n -e "/^\[$SECTION\]/,/^\s*\[/{/^[^#].*\=.*/p;}"` } +parse_fs_config_option() { + local fs_config_option + if ! $OPTIONS_HAVE_FSCONFIGS; then + return 0 + fi + + if [ -f "$1" ]; then + fs_config_option=$1 + elif [ -f configs/"$1" ]; then + fs_config_option=configs/"$1" + else + echo "Invalid FS_CONFIG_OPTION $1" + exit 1 + fi + + eval `sed -e 's/[[:space:]]*\=[[:space:]]*/=/g' \ + -e 's/#.*$//' \ + -e 's/[[:space:]]*$//' \ + -e 's/^[[:space:]]*//' \ + -e "s/^\([^=]*\)=\"\?'\?\([^\"']*\)\"\?'\?$/export \1=\"\2\"/" \ + < $fs_config_option` +} + get_next_config() { - if [ ! -z "$CONFIG_INCLUDED" ] && ! $OPTIONS_HAVE_SECTIONS; then + local cmd_fs_config_option=$2 + if [[ $CONFIG_INCLUDED && ! options_have_sections_or_fs_configs ]]; then return 0 fi @@ -817,6 +846,23 @@ get_next_config() { fi parse_config_section $1 + + # FS_CONFIG_OPTION can be set either via cmdline -c xfs/4k (through + # CMD_FS_CONFIG_OPTION) or can be set in section config file + # FS_CONFIG_OPTION=xfs/4k. + # Please note that FS_CONFIG_OPTION in the local.config file can be + # overridden by the value passed through the command line. + if [[ -n "$cmd_fs_config_option" ]]; then + export FS_CONFIG_OPTION=$cmd_fs_config_option + fi + + if [[ -n "$FS_CONFIG_OPTION" ]]; then + export OPTIONS_HAVE_FSCONFIGS=true + parse_fs_config_option $FS_CONFIG_OPTION + else + export OPTIONS_HAVE_FSCONFIGS=false + fi + if [ ! -z "$OLD_FSTYP" ] && [ $OLD_FSTYP != $FSTYP ]; then [ -z "$MOUNT_OPTIONS" ] && _mount_opts [ -z "$TEST_FS_MOUNT_OPTS" ] && _test_mount_opts @@ -900,7 +946,8 @@ get_next_config() { } if [ -z "$CONFIG_INCLUDED" ]; then - get_next_config `echo $HOST_OPTIONS_SECTIONS | cut -f1 -d" "` + get_next_config `echo $HOST_OPTIONS_SECTIONS | cut -f1 -d" "` \ + `echo $CMD_FS_CONFIG_OPTION | cut -f1 -d","` export CONFIG_INCLUDED=true # Autodetect fs type based on what's on $TEST_DEV unless it's been set diff --git a/common/report b/common/report index 7128bbeb..8723c510 100644 --- a/common/report +++ b/common/report @@ -12,7 +12,7 @@ REPORT_ENV_LIST=("SECTION" "FSTYP" "PLATFORM" "MKFS_OPTIONS" "MOUNT_OPTIONS" \ # Variables that are captured in the report /if/ they are set. REPORT_ENV_LIST_OPT=("TAPE_DEV" "RMT_TAPE_DEV" "FSSTRESS_AVOID" "FSX_AVOID" "KCONFIG_PATH" "PERF_CONFIGNAME" "MIN_FSSIZE" - "IDMAPPED_MOUNTS") + "IDMAPPED_MOUNTS" "FS_CONFIG_OPTION") encode_xml() { From patchwork Fri Jan 10 09:10:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nirjhar Roy (IBM)" X-Patchwork-Id: 13934014 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A07BE206F06; Fri, 10 Jan 2025 09:13:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736500412; cv=none; b=XeC+eTvLjXaZH49VQ3LtfpvYrNUHXEvXOlTom+Krn24wq2F6Jdt/btsLqvvjqmLkSuN21y3jXrLIqpiN3+Ei4nimSSAnvDRXfxhlaRKjDx/FLo0vVkz/h08Xz2ASLXrhDPR04CKN6d7EpXbxZFGk/LI5JNvMHOnYYVXCaMyjq3o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736500412; c=relaxed/simple; bh=SRbCW5elgWrfQrsMHXEyXZfnCTiDhxeyV6LBA3Y2WKU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QG2wsRCxu9uuhSy5eWwgkdECuRhyrDgN7yCHeSIbnSk41BK6zDoA55s+ahRFpwnHncnJOdS+3K8RWjIctODZWyLShH4MjxmMb7eMLcej2JgxPIm0ctLIpsLp4jMHVrty1nWRkE3KeWN3YZ5JJKlEZToXztEr5LARnoPR1etTaKs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QeBNd6jD; arc=none smtp.client-ip=209.85.216.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QeBNd6jD" Received: by mail-pj1-f49.google.com with SMTP id 98e67ed59e1d1-2ee989553c1so2956368a91.3; Fri, 10 Jan 2025 01:13:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736500409; x=1737105209; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0eMn14kRYlF/w5tvKcVfVLe+UkdswT1vD8flcxY22Ew=; b=QeBNd6jDx3pUC/0M2rwcLJg16QV7TEmmS4A79PIYKdZSU92tQRrAlY0fsIOyvzCUsF BJSJUKMQS/a/H9HyoQGTt3scf7sjAbSvr95YCW5aU8g0hzsywP5M5zNIGIGyGgWhlq55 3fIYZ10A18KUGrTVezQr9vh+8iYgy07x7XbDtjkOdCQBLWaIiPar6hKeO/6DFu0GPmkE 8MEKzw2Zi/slw+UMZmfaGbXlLHpNcv3GnzmIQQg17X/yv3qN+qf1iecmT2v0v9a4LD5T pV0USuv/Gbh1owL1V1HQH5CGg/2K1jUTxQh8RqKz3VbB6I3FKz+o7pNlVw1giBPQqeeF ITSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736500409; x=1737105209; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0eMn14kRYlF/w5tvKcVfVLe+UkdswT1vD8flcxY22Ew=; b=egFa5SlfScSYzx5oKoPzNSZgK0+fnd9xOi1HESG4OuvCltrD9eIcg3HiVyKqDpe/tW X09q7sh5CdGIA8cy7EqDQFK0BVU8vhAbKGeqxHVhybZZJ4lIs2suiJEcSuqJPt6S6UvP vBrT0iM8oC6thUQTUiw22iqLF3OnjUiUYqSjtikdIc22l9HalADECiseMflNBlZmJL7+ StFyR8oY68wj6nk6OA9HiCHFty/H0om+G49XbK2j8vr3Cu8FoW9WZkXvwed86w8K2PEw frFG9vGMO467HNLvxSrhgZyqKJqkI5m4f7FrgiWkDarSni1Vx6WwlaFcGruQlzYqv9RA uQAg== X-Forwarded-Encrypted: i=1; AJvYcCX5cOwkiL2Gp7DFGtJuVOu4dJZZM5DyWGYKuisQG0637Qh4+3lxc0ME5BENBUgrd5KYHtOjXLk6haU=@vger.kernel.org X-Gm-Message-State: AOJu0YzOwOaK/xM4TOVTzsihv1zsCDjqQ49jcOibfXY1a4V8jL8TnVqg mc2EjZFus69iwNZDmAzPY3s+YjWgqvAKp+ny6QIpdfdIY+okrXh778lZHbYn X-Gm-Gg: ASbGncuxgYvxBWyNDzA+46mymord4hHbSlDURQdmW5gfoONVSeBPcTXCaXxnTZ8IQRY FHy5Lq9Q3KJqgww0ta/luQ6Y4bZCi2A7E7Hy9UGzHLu12/ludExTcfEL752shAM8PBiwNHjcF8e PIWfx7MPsIcis8DLumndhLAL9+CDyPi7cx1EKutFnkAZBep489p00cwjc+yMj9HTEKIJ1nIs5gb VRyiHy0tZINJM2oj0CvOkcPbB6a3nhBq3/woXPdVS5FWhWnSHTM5+5a23qM X-Google-Smtp-Source: AGHT+IHdxYNRKqI5sJMPe7bnhy+5VJJDr8pnOhgoaKyf6j477Yi5CYZ3esqn9S9ENXAtHVVy4xhIgQ== X-Received: by 2002:a17:90b:3d50:b0:2ea:9ccb:d1f4 with SMTP id 98e67ed59e1d1-2f548e3c0demr17106561a91.0.1736500409491; Fri, 10 Jan 2025 01:13:29 -0800 (PST) Received: from citest-1.. ([49.205.38.219]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21a9f219a94sm10166485ad.129.2025.01.10.01.13.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jan 2025 01:13:29 -0800 (PST) From: "Nirjhar Roy (IBM)" To: fstests@vger.kernel.org Cc: linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, ritesh.list@gmail.com, ojaswin@linux.ibm.com, djwong@kernel.org, zlang@kernel.org, nirjhar.roy.lists@gmail.com Subject: [RFC 5/5] configs: Add a couple of xfs and ext4 configuration templates. Date: Fri, 10 Jan 2025 09:10:29 +0000 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit adds a couple of simple configuration templates. Developers should add more templates based on the tests they add. Reviewed-by: Ritesh Harjani (IBM) Signed-off-by: Nirjhar Roy (IBM) --- configs/ext4/1k | 3 +++ configs/ext4/4k | 3 +++ configs/ext4/64k | 3 +++ configs/xfs/1k | 3 +++ configs/xfs/4k | 3 +++ configs/xfs/64k | 3 +++ configs/xfs/adv | 3 +++ configs/xfs/quota | 3 +++ 8 files changed, 24 insertions(+) create mode 100644 configs/ext4/1k create mode 100644 configs/ext4/4k create mode 100644 configs/ext4/64k create mode 100644 configs/xfs/1k create mode 100644 configs/xfs/4k create mode 100644 configs/xfs/64k create mode 100644 configs/xfs/adv create mode 100644 configs/xfs/quota diff --git a/configs/ext4/1k b/configs/ext4/1k new file mode 100644 index 00000000..6f16fc44 --- /dev/null +++ b/configs/ext4/1k @@ -0,0 +1,3 @@ +FSTYP=ext4 +MKFS_OPTIONS="-F -b 1024" +MOUNT_OPTIONS="-o block_validity " diff --git a/configs/ext4/4k b/configs/ext4/4k new file mode 100644 index 00000000..32f02ea9 --- /dev/null +++ b/configs/ext4/4k @@ -0,0 +1,3 @@ +FSTYP=ext4 +MKFS_OPTIONS="-F -b 4096" +MOUNT_OPTIONS="-o block_validity " diff --git a/configs/ext4/64k b/configs/ext4/64k new file mode 100644 index 00000000..6c50b839 --- /dev/null +++ b/configs/ext4/64k @@ -0,0 +1,3 @@ +FSTYP=ext4 +MKFS_OPTIONS="-F -b 65536" +MOUNT_OPTIONS="-o block_validity " diff --git a/configs/xfs/1k b/configs/xfs/1k new file mode 100644 index 00000000..ab74d3a7 --- /dev/null +++ b/configs/xfs/1k @@ -0,0 +1,3 @@ +FSTYP=xfs +MKFS_OPTIONS="-b size=1024" +MOUNT_OPTIONS=" " diff --git a/configs/xfs/4k b/configs/xfs/4k new file mode 100644 index 00000000..9a390404 --- /dev/null +++ b/configs/xfs/4k @@ -0,0 +1,3 @@ +FSTYP=xfs +MKFS_OPTIONS="-b size=4096" +MOUNT_OPTIONS=" " diff --git a/configs/xfs/64k b/configs/xfs/64k new file mode 100644 index 00000000..ddcd7977 --- /dev/null +++ b/configs/xfs/64k @@ -0,0 +1,3 @@ +FSTYP=xfs +MKFS_OPTIONS="-b size=65536" +MOUNT_OPTIONS=" " diff --git a/configs/xfs/adv b/configs/xfs/adv new file mode 100644 index 00000000..f1b2ec2e --- /dev/null +++ b/configs/xfs/adv @@ -0,0 +1,3 @@ +FSTYP=xfs +MKFS_OPTIONS="-m inobtcount=1,bigtime=1,rmapbt=1" +MOUNT_OPTIONS=" " diff --git a/configs/xfs/quota b/configs/xfs/quota new file mode 100644 index 00000000..a7f0c7e7 --- /dev/null +++ b/configs/xfs/quota @@ -0,0 +1,3 @@ +FSTYP=xfs +MKFS_OPTIONS="-f " +MOUNT_OPTIONS="-o usrquota,grpquota,prjquota"