diff mbox series

[3/3] selftests: ALSA: Add test for the 'valsa' driver

Message ID 20230513202037.158777-3-ivan.orlov0322@gmail.com (mailing list archive)
State New, archived
Headers show
Series [1/3] docs: admin-guide: add valsa driver documentation | expand

Commit Message

Ivan Orlov May 13, 2023, 8:20 p.m. UTC
This test covers the new Virtual ALSA driver, including the capturing,
playback and ioctl redefinition functionalities. This test is also helpful
as an usage example of the 'valsa' driver.

We have a lot of different virtual media drivers, which can be used for
testing of the userspace applications and media subsystem middle layer.
However, all of them are aimed at testing the video functionality and
simulating the video devices. For audio devices we have only snd-dummy
module, which is good in simulating the correct behavior of an ALSA device.
I decided to write a tool, which would help to test the userspace ALSA
programs (and the PCM middle layer as well) under unusual circumstances
to figure out how they would behave. So I came up with this Virtual ALSA
Driver.

This new Virtual ALSA Driver has several features which can be useful
during the userspace ALSA applications testing/fuzzing, or testing/fuzzing
of the PCM middle layer. Not all of them can be implemented using the
existing virtual drivers (like dummy or loopback). Here is what can this
driver do:

- Simulate both capture and playback processes
- Generate random or pattern-based capture data
- Check the playback stream for containing the looped pattern
- Inject delays into the playback and capturing processes
- Inject errors during the PCM callbacks

Also, this driver can check the playback stream for containing the
predefined pattern, which is used in the corresponding selftest to check
the PCM middle layer data transferring functionality. Additionally, this
driver redefines the default RESET ioctl, and the selftest covers this PCM
API functionality as well.

After all, I think this driver would be a good start, and I believe in the
future we will see more virtual sound drivers.

Signed-off-by: Ivan Orlov <ivan.orlov0322@gmail.com>
---
 tools/testing/selftests/alsa/valsa-test.sh | 55 ++++++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100755 tools/testing/selftests/alsa/valsa-test.sh

Comments

Ivan Orlov May 15, 2023, 5:58 a.m. UTC | #1
On 5/15/23 05:28, Mark Brown wrote:
> On Sun, May 14, 2023 at 12:20:37AM +0400, Ivan Orlov wrote:
> 
>> +uid=$(id -u)
>> +if [ $uid -ne 0 ]; then
>> +	echo "$0: Must be run as root"
>> +	exit 1
>> +fi
> 
> It is not an error to run the selftest as a non-root user, the test
> should be skipped.

Alright, thanks!
>> +modprobe snd-valsa
> 
> We don't check if the module was already loaded.
> 
>> +if [ ! -e /sys/kernel/debug/valsa/pc_test ]; then
>> +	mount -t debugfs none /sys/kernel/debug
>> +
>> +	if [ ! -e /sys/kernel/debug/valsa/pc_test ]; then
>> +		echo "$0: Error mounting debugfs"
>> +		exit 4
>> +	fi
>> +fi
> 
> This will unconditionally attempt to mount debugfs in a standard
> location and won't clean up after itself, if the system didn't have
> debugfs mounted for some reason then this will leave it sitting there.

Yes, I agree... I'll remove this in the future versions, I think the 
approach with skipping in case of missing debugfs would be better.

By the way, I used the 'fpa' test as an example, and it looks like it 
should be fixed as well...

> Would it not be better to have a C program that actually calls the ioctl
> rather than a custom debugfs thing that may or may not be wired up to do
> the same thing as an ioctl would?  It seems like other than whatever
> this ioctl test actually does this is all just a simplified version of
> the existing pcm-test.

Well, the idea was to test the playback buffer consistency - the driver 
has the functionality of testing the playback buffer for containing the 
looped pattern (and if the buffer doesn't contain the looped pattern the 
test fails). So, firstly we get the capture buffer with this pattern 
(via arecord), and then send it to the driver for the test (via aplay).

The pcm-test (as I understand) performs only time checks, not the checks 
of the data played/captured, and I think it is pointless to test the 
time again. But I agree, that C test implementation would be better - 
using this approach I can perform additional buffer checks and cover 
more functionality of the driver.

Thank you very much for the review!

Kind regards,
Ivan Orlov.
diff mbox series

Patch

diff --git a/tools/testing/selftests/alsa/valsa-test.sh b/tools/testing/selftests/alsa/valsa-test.sh
new file mode 100755
index 000000000000..872dfac138c2
--- /dev/null
+++ b/tools/testing/selftests/alsa/valsa-test.sh
@@ -0,0 +1,55 @@ 
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Test for the 'valsa' virtual driver. It tests capture and playback functionalities,
+# as well as the 'reset' ioctl redefinition.
+
+uid=$(id -u)
+if [ $uid -ne 0 ]; then
+	echo "$0: Must be run as root"
+	exit 1
+fi
+
+if ! which modprobe > /dev/null 2>&1; then
+	echo "$0: You need modprobe installed"
+        exit 4
+fi
+
+if ! modinfo snd-valsa > /dev/null 2>&1; then
+	echo "$0: You must have the following enabled in your kernel:"
+	echo "CONFIG_SND_ALSAV=m"
+	exit 4
+fi
+
+modprobe snd-valsa
+
+if [ ! -e /sys/kernel/debug/valsa/pc_test ]; then
+	mount -t debugfs none /sys/kernel/debug
+
+	if [ ! -e /sys/kernel/debug/valsa/pc_test ]; then
+		echo "$0: Error mounting debugfs"
+		exit 4
+	fi
+fi
+
+success="1"
+arecord -D hw:CARD=valsa,DEV=0 -c 1 --buffer-size=8192 -f S16_LE -r 8000 --duration=4 out.wav
+
+if [[ $(< /sys/kernel/debug/valsa/ioctl_test) == "$success" ]]; then
+	echo "$0: ioctl test: success"
+else
+	echo "$0: ioctl test: fail"
+fi
+
+aplay -D hw:CARD=valsa,DEV=0 -c 1 --buffer-size=8192 -f S16_LE -r 8000 out.wav
+
+if [[ $(< /sys/kernel/debug/valsa/pc_test) == "$success" ]]; then
+	echo "$0: playback test: success"
+else
+	echo "$0: playback test: fail"
+fi
+
+rm -rf out.wav
+if ! rmmod snd-valsa > /dev/null 2>&1; then
+	echo "$0: Can't remove 'valsa' module: disable PulseAudio and unload it manually"
+fi