diff mbox series

nvme: test nvmet-wq sysfs interface

Message ID 20241104192907.21358-1-kch@nvidia.com (mailing list archive)
State New
Headers show
Series nvme: test nvmet-wq sysfs interface | expand

Commit Message

Chaitanya Kulkarni Nov. 4, 2024, 7:29 p.m. UTC 
Add a test that randomly sets the cpumask from available CPUs for
the nvmet-wq while running the fio workload. This patch has been
tested on nvme-loop and nvme-tcp transport.

Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
 tests/nvme/055     | 99 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/055.out |  3 ++
 2 files changed, 102 insertions(+)
 create mode 100755 tests/nvme/055
 create mode 100644 tests/nvme/055.out

Comments

Daniel Wagner Nov. 5, 2024, 7:02 a.m. UTC | #1 
On Mon, Nov 04, 2024 at 11:29:07AM GMT, Chaitanya Kulkarni wrote:
> +# Test nvmet-wq cpumask sysfs attribute with NVMe-oF and fio workload

What do you want to test here? What's the objective? I don't see any
block layer related parts or do I miss something?
Chaitanya Kulkarni Nov. 6, 2024, 3:46 a.m. UTC | #2 
On 11/4/24 23:02, Daniel Wagner wrote:
> On Mon, Nov 04, 2024 at 11:29:07AM GMT, Chaitanya Kulkarni wrote:
>> +# Test nvmet-wq cpumask sysfs attribute with NVMe-oF and fio workload 
> What do you want to test here? What's the objective? I don't see any 
> block layer related parts or do I miss something? 

For NVMeOF target we have exported nvmet-wq to userspace with
recent patch. This behavior is new and I don't think there is any test
exists that runs the fio workload while changing the CPUMASK randomly
to ensure the stability for nvmet-wq when application is using the block
layer. Hence we need the test for latest patch we have added to the 6.13.

-ck
Daniel Wagner Nov. 6, 2024, 12:13 p.m. UTC | #3 
On Wed, Nov 06, 2024 at 03:46:26AM GMT, Chaitanya Kulkarni wrote:
> On 11/4/24 23:02, Daniel Wagner wrote:
> > On Mon, Nov 04, 2024 at 11:29:07AM GMT, Chaitanya Kulkarni wrote:
> >> +# Test nvmet-wq cpumask sysfs attribute with NVMe-oF and fio workload 
> > What do you want to test here? What's the objective? I don't see any 
> > block layer related parts or do I miss something? 
> 
> For NVMeOF target we have exported nvmet-wq to userspace with
> recent patch. This behavior is new and I don't think there is any test
> exists that runs the fio workload while changing the CPUMASK randomly
> to ensure the stability for nvmet-wq when application is using the block
> layer. Hence we need the test for latest patch we have added to the
> 6.13.

Though this is not a performance measurement just a functional testing if
the scheduler works. I don't think we should add such tests
because it will add to the overall runtime for little benefit. I am
pretty sure there already tests (e.g. kunit) which do more elaborate
scheduler tests.
diff mbox series

Patch

diff --git a/tests/nvme/055 b/tests/nvme/055
new file mode 100755
index 0000000..9fe27a3
--- /dev/null
+++ b/tests/nvme/055
@@ -0,0 +1,99 @@ 
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+# Copyright (C) 2024 Chaitanya Kulkarni
+#
+# Test nvmet-wq cpumask sysfs attribute with NVMe-oF and fio workload
+#
+
+. tests/nvme/rc
+
+DESCRIPTION="Test nvmet-wq cpumask sysfs attribute with fio on NVMe-oF device"
+TIMED=1
+
+requires() {
+	_nvme_requires
+	 _have_fio && _have_loop
+	_require_nvme_trtype_is_fabrics
+}
+
+cleanup_setup() {
+	_nvme_disconnect_subsys
+	_nvmet_target_cleanup
+}
+
+test() {
+	local cpumask_path="/sys/devices/virtual/workqueue/nvmet-wq/cpumask"
+	local original_cpumask
+	local min_cpus
+	local max_cpus
+	local numbers
+	local idx
+	local ns
+
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+	_nvmet_target_setup
+	_nvme_connect_subsys
+
+	if [ ! -f "$cpumask_path" ]; then
+		SKIP_REASONS+=("nvmet_wq cpumask sysfs attribute not found.")
+		cleanup_setup
+		return 1
+	fi
+
+	ns=$(_find_nvme_ns "${def_subsys_uuid}")
+
+	original_cpumask=$(cat "$cpumask_path")
+
+	num_cpus=$(nproc)
+	max_cpus=$(( num_cpus < 20 ? num_cpus : 20 ))
+	min_cpus=0
+	#shellcheck disable=SC2207
+	numbers=($(seq $min_cpus $max_cpus))
+
+	_run_fio_rand_io --filename="/dev/${ns}" --time_based --runtime=130s \
+			 --iodepth=8 --size="${NVME_IMG_SIZE}" &> "$FULL" &
+
+	# Let the fio settle down else we will break in the loop for fio check
+	sleep 1
+	for ((i = 0; i < max_cpus; i++)); do
+		if ! pgrep -x fio &> /dev/null ; then
+			break
+		fi
+
+		if [[ ${#numbers[@]} -eq 0 ]]; then
+			break
+		fi
+
+		idx=$((RANDOM % ${#numbers[@]}))
+
+		#shellcheck disable=SC2004
+		cpu_mask=$(printf "%X" $((1 << ${numbers[idx]})))
+		echo "$cpu_mask" > "$cpumask_path"
+		if [[ $(cat "$cpumask_path") =~ ^[0,]*${cpu_mask}\n$ ]]; then
+			echo "Test Failed: cpumask was not set correctly "
+			echo "Expected ${cpu_mask} found $(cat "$cpumask_path")"
+			cleanup_setup
+			return 1
+		fi
+		sleep 3
+		# Remove the selected number
+		numbers=("${numbers[@]:0:$idx}" "${numbers[@]:$((idx + 1))}")
+	done
+
+	killall fio &> /dev/null
+
+	# Restore original cpumask
+	echo "$original_cpumask" > "$cpumask_path"
+	restored_cpumask=$(cat "$cpumask_path")
+
+	if [[ "$restored_cpumask" != "$original_cpumask" ]]; then
+		echo "Failed to restore original cpumask."
+		cleanup_setup
+		return 1
+	fi
+
+	cleanup_setup
+	echo "Test complete"
+}
diff --git a/tests/nvme/055.out b/tests/nvme/055.out
new file mode 100644
index 0000000..427dfee
--- /dev/null
+++ b/tests/nvme/055.out
@@ -0,0 +1,3 @@ 
+Running nvme/055
+disconnected 1 controller(s)
+Test complete