From patchwork Thu Apr 25 22:23:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13643998 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 26581155A34; Thu, 25 Apr 2024 22:23:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714083825; cv=none; b=qmeM+gTVk0x5VMO4eV6bjYlaKq5SYWi3TeI4EHzfwUyUcGUlwzSmW6DVU9OZ9MizNQu+Eh6QbfDliL8foVz/t50ogtPWXjiBNErw8doCEop0pb2p42vZKoeBt/ct82O8lJ1Zpt+LHet2/UhJD45SWRXWUyVA5mEXfi61646J0wA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714083825; c=relaxed/simple; bh=DUJp5cHn4WtHeGghigmtRvGEuyz77UPiJusThdZtrc4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MJPa5Kc76bZkHUF9iYvvM3NXEOg3MjQV77jf4NJZx6OSQmnDuCmt9ItnZCdXLjHaKgeK/JtnGvICL5H8dg6Y4bcTljC/LO6z+ecoUAMAv+lrUFH/cCJs9gWCAawUbwXotW6LPkBCk19ZqyrjwpkvAVrAP7J4QZIWr/aGCZDTVzw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HYuKnfot; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HYuKnfot" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5FCE5C2BD10; Thu, 25 Apr 2024 22:23:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1714083824; bh=DUJp5cHn4WtHeGghigmtRvGEuyz77UPiJusThdZtrc4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HYuKnfotilGuOZSLxnq3jwKQuUbT5c4JIiOpCkHIEx4hx0O7hRdnVRJeS705gceDr 8E26OqMBvxH7cHZVjODclBM2r/qzWaLy8MS66hQAP5YTWoy1VpLIq+CX1iXPoza9Rn +v2lMl+p3JR7HuHOoQzz/3NVGxjGyxaVPcvI4kDVoTza0Hf9p0mivj8oSA5/sR4/dh +jsdOVGZsYEHKH4U+4ExQrTDRsH7WmNGJpwn92xeNlCBf85mTAdf3zsDj7RbswOT0f MJa2ai/3JeejcGHo4KJDbXA+GPOfRQzNDXfKRYAnOrmSezmjtpi9OsAmJlduP4+Iq4 vVpWG4beIXouQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, linux-kselftest@vger.kernel.org, willemdebruijn.kernel@gmail.com, Jakub Kicinski , Willem de Bruijn Subject: [PATCH net-next v2 1/3] selftests: drv-net: extend the README with more info and example Date: Thu, 25 Apr 2024 15:23:39 -0700 Message-ID: <20240425222341.309778-2-kuba@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240425222341.309778-1-kuba@kernel.org> References: <20240425222341.309778-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Add more info to the README. It's also now copied to GitHub for increased visibility: https://github.com/linux-netdev/nipa/wiki/Running-driver-tests Reviewed-by: Willem de Bruijn Signed-off-by: Jakub Kicinski --- .../testing/selftests/drivers/net/README.rst | 97 ++++++++++++++++--- 1 file changed, 85 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/drivers/net/README.rst b/tools/testing/selftests/drivers/net/README.rst index 0cbab33dad1f..3b6a29e6564b 100644 --- a/tools/testing/selftests/drivers/net/README.rst +++ b/tools/testing/selftests/drivers/net/README.rst @@ -1,18 +1,42 @@ -Running tests -============= +.. SPDX-License-Identifier: GPL-2.0 -Tests are executed within kselftest framework like any other tests. -By default tests execute against software drivers such as netdevsim. -All tests must support running against a real device (SW-only tests -should instead be placed in net/ or drivers/net/netdevsim, HW-only -tests in drivers/net/hw). +Running driver tests +==================== -Set appropriate variables to point the tests at a real device. +Networking driver tests are executed within kselftest framework like any +other tests. They support testing both real device drivers and emulated / +software drivers (latter mostly to test the core parts of the stack). + +SW mode +~~~~~~~ + +By default, when no extra parameters are set or exported, tests execute +against software drivers such as netdevsim. No extra preparation is required +the software devices are created and destroyed as part of the test. +In this mode the tests are indistinguishable from other selftests and +(for example) can be run under ``virtme-ng`` like the core networking selftests. + +HW mode +~~~~~~~ + +Executing tests against a real device requires external preparation. +The netdevice against which tests will be run must exist, be running +(in UP state) and be configured with an IP address. + +Refer to list of :ref:`Variables` later in this file to set up running +the tests against a real device. + +Both modes required +~~~~~~~~~~~~~~~~~~~ + +All tests in drivers/net must support running both against a software device +and a real device. SW-only tests should instead be placed in net/ or +drivers/net/netdevsim, HW-only tests in drivers/net/hw. Variables ========= -Variables can be set in the environment or by creating a net.config +The variables can be set in the environment or by creating a net.config file in the same directory as this README file. Example:: $ NETIF=eth0 ./some_test.sh @@ -23,9 +47,9 @@ Variables can be set in the environment or by creating a net.config # Variable set in a file NETIF=eth0 -Please note that the config parser is very simple, if there are -any non-alphanumeric characters in the value it needs to be in -double quotes. +Local test (which don't require endpoint for sending / receiving traffic) +need only the ``NETIF`` variable. Remaining variables define the endpoint +and communication method. NETIF ~~~~~ @@ -61,3 +85,52 @@ Arguments used to construct the communication channel. for netns - name of the "remote" namespace for ssh - name/address of the remote host + +Example +======= + +Build the selftests:: + + # make -C tools/testing/selftests/ TARGETS="drivers/net drivers/net/hw" + +"Install" the tests and copy them over to the target machine:: + + # make -C tools/testing/selftests/ TARGETS="drivers/net drivers/net/hw" \ + install INSTALL_PATH=/tmp/ksft-net-drv + + # rsync -ra --delete /tmp/ksft-net-drv root@192.168.1.1:/root/ + +On the target machine, running the tests will use netdevsim by default:: + + [/root] # ./ksft-net-drv/run_kselftest.sh -t drivers/net:ping.py + TAP version 13 + 1..1 + # timeout set to 45 + # selftests: drivers/net: ping.py + # KTAP version 1 + # 1..3 + # ok 1 ping.test_v4 + # ok 2 ping.test_v6 + # ok 3 ping.test_tcp + # # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0 + ok 1 selftests: drivers/net: ping.py + +Create a config with remote info:: + + [/root] # cat > ./ksft-net-drv/drivers/net/net.config < X-Patchwork-Id: 13643999 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E8E1156226; Thu, 25 Apr 2024 22:23:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714083825; cv=none; b=e9MPLspgDV2+jMIjJvUT1F8L8yc1eYffLib6ChZT8xIHLAfYLS2hn9L/9hkMUHOHTqfLg2Z6Pcjec2tLA+RD5V1H6x594gTLQGT/M3ZlBnBmZr43cdeFprK0MlDkBJT7ZDSMjiVjteI9L+9CuCkVUAIyQMOkN3fnB0EbR+Hnafc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714083825; c=relaxed/simple; bh=gMri0/2HbfqDkBXyRafH3fexR2k3G9Jim8h87K7dtds=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Yo5/VpvGvbKnO9IyuDNpIT8Hs8MFkuW3V0Y4mzzDhVqtCa5yBe4XNTsugzel9Sdi01AhM2ehc4g+4sYqs9lpU3d4gqmxIBL8evnPAtVoNTETSbhexReG4SJa1Xopi9iKh1TeAA41ah+WyiqwF+HV/RsMOlcZt6Ydq3LmcuEWNbU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RnJc7q9a; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RnJc7q9a" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E9376C2BD11; Thu, 25 Apr 2024 22:23:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1714083825; bh=gMri0/2HbfqDkBXyRafH3fexR2k3G9Jim8h87K7dtds=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RnJc7q9aZlMGQK76SksSegiMjPGaW4kfzq8yMJVlIxfGvPKQvNcMx4H1W//0D/1Fb xWed616phaOXHDaDvghnpz3XUpvP4/eMkYaS2YiVCvGFDlhYx/hCSznjN5ZCAMUeN0 LuaekwLGk0sNYwCXHLlP+lkLcUbc3kzS1Lmxp/q2QaDLpZcdwYUXQvhAE1R5JSQ95W 5LKGcm3XDkmoKq6y1YsMR49xF89Fy6u9/KGeCd4Xb2ghwlVIo68jzZ324Rlqn0/4/c RC4a9jypDLwCI82CV/TySYTW1QM7zm7vX5ZIb/sIaa/D3mppNn3nSJsdgSbqcktZAs QNKAkZK+Zuw2A== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, linux-kselftest@vger.kernel.org, willemdebruijn.kernel@gmail.com, Jakub Kicinski , Willem de Bruijn Subject: [PATCH net-next v2 2/3] selftests: drv-net: reimplement the config parser Date: Thu, 25 Apr 2024 15:23:40 -0700 Message-ID: <20240425222341.309778-3-kuba@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240425222341.309778-1-kuba@kernel.org> References: <20240425222341.309778-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org The shell lexer is not helping much, do very basic parsing manually. Reviewed-by: Willem de Bruijn Signed-off-by: Jakub Kicinski --- v2: - use split() to split the line on = --- .../selftests/drivers/net/lib/py/env.py | 26 ++++++++++--------- 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/drivers/net/lib/py/env.py b/tools/testing/selftests/drivers/net/lib/py/env.py index a3db1bb1afeb..6f57bd5c0ed7 100644 --- a/tools/testing/selftests/drivers/net/lib/py/env.py +++ b/tools/testing/selftests/drivers/net/lib/py/env.py @@ -1,7 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 import os -import shlex from pathlib import Path from lib.py import KsftSkipEx from lib.py import cmd, ip @@ -16,17 +15,20 @@ from .remote import Remote if not (src_dir / "net.config").exists(): return env - lexer = shlex.shlex(open((src_dir / "net.config").as_posix(), 'r').read()) - k = None - for token in lexer: - if k is None: - k = token - env[k] = "" - elif token == "=": - pass - else: - env[k] = token - k = None + with open((src_dir / "net.config").as_posix(), 'r') as fp: + for line in fp.readlines(): + full_file = line + # Strip comments + pos = line.find("#") + if pos >= 0: + line = line[:pos] + line = line.strip() + if not line: + continue + pair = line.split('=', maxsplit=1) + if len(pair) != 2: + raise Exception("Can't parse configuration line:", full_file) + env[pair[0]] = pair[1] return env From patchwork Thu Apr 25 22:23:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13644000 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E45D15665C; Thu, 25 Apr 2024 22:23:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714083826; cv=none; b=JJO0lC6Wvv5/xGzU0fVoMbSLXDGKetn+iN3OM0hMara8sAUwW5qd3pbKYmYZEZd91Iqtt9TeRMq12ZTZUsDycHiLPT6hQ3ItVyFCwHdbtcZVTbjJymNTKt65FZIbNR4Sm4luNwGFxDiTiAcRFLgBisW+BTwTqm2tQ8GjsclG6Gc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714083826; c=relaxed/simple; bh=yIhLkmj1LbR7a0JWHfGMmeLybevbTRq425qq1r8AFhM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IIwO/ZzjEjW3DRHgjako9auirfIuvgTqp/+IxiZ1mcA1qgV245lKChdJ0ck03xtyKIh2qTpgM0b7seaC0k8HN7lB0hazVNL08FvW60kPYEVXbwfqkZ4jNyicLXHn/nw4xO0ZbYSnLTwmut9RFPANANC8L/RbreTyDGNbKQCm0ig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UFKD8Pmv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UFKD8Pmv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7EBEFC2BD10; Thu, 25 Apr 2024 22:23:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1714083825; bh=yIhLkmj1LbR7a0JWHfGMmeLybevbTRq425qq1r8AFhM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UFKD8PmvKpzHiBk7nC7VRU9zo0Bxkcq3uCO5uXOoXieGeRKrjwfhbjF3T6kAIDDDt jbFZ07tUWtb2PI8z3XtYjYzau3QHKA6N6vjR6XD+AsCwKOjFtz37pgnyMtHxLJraSa 79f/bcLtbQSr5ZNq79/8YzppsHldMgz6ZO/tpERqAFYxefoxF55EGjYIvWAzNkT46Z K+QRD269E2NbWtXOPMSd6pgBbwLcV2FwFXGCGk8Uxb/W1QSAdpa+zVTqHIkj8dPca2 fx2PpH+KOb4K21u9G6TeJ/b+X71dzI1BA8abCxTno+6yxzZVgJfu/CsL/S1WD4PeFL 0rO/pp24SQqUQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, linux-kselftest@vger.kernel.org, willemdebruijn.kernel@gmail.com, Jakub Kicinski , Willem de Bruijn Subject: [PATCH net-next v2 3/3] selftests: drv-net: validate the environment Date: Thu, 25 Apr 2024 15:23:41 -0700 Message-ID: <20240425222341.309778-4-kuba@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240425222341.309778-1-kuba@kernel.org> References: <20240425222341.309778-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Throw a slightly more helpful exception when env variables are partially populated. Prior to this change we'd get a dictionary key exception somewhere later on. Reviewed-by: Willem de Bruijn Signed-off-by: Jakub Kicinski --- v2: - check "both or none" for addresses --- .../selftests/drivers/net/lib/py/env.py | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/tools/testing/selftests/drivers/net/lib/py/env.py b/tools/testing/selftests/drivers/net/lib/py/env.py index 6f57bd5c0ed7..e2ab637e56dc 100644 --- a/tools/testing/selftests/drivers/net/lib/py/env.py +++ b/tools/testing/selftests/drivers/net/lib/py/env.py @@ -88,6 +88,7 @@ from .remote import Remote self._ns_peer = None if "NETIF" in self.env: + self._check_env() self.dev = ip("link show dev " + self.env['NETIF'], json=True)[0] self.v4 = self.env.get("LOCAL_V4") @@ -143,6 +144,30 @@ from .remote import Remote ip(f"-6 addr add dev {self._ns_peer.nsims[0].ifname} {self.nsim_v6_pfx}2/64 nodad", ns=self._netns) ip(f" link set dev {self._ns_peer.nsims[0].ifname} up", ns=self._netns) + def _check_env(self): + vars_needed = [ + ["LOCAL_V4", "LOCAL_V6"], + ["REMOTE_V4", "REMOTE_V6"], + ["REMOTE_TYPE"], + ["REMOTE_ARGS"] + ] + missing = [] + + for choice in vars_needed: + for entry in choice: + if entry in self.env: + break + else: + missing.append(choice) + # Make sure v4 / v6 configs are symmetric + if ("LOCAL_V6" in self.env) != ("REMOTE_V6" in self.env): + missing.append(["LOCAL_V6", "REMOTE_V6"]) + if ("LOCAL_V4" in self.env) != ("REMOTE_V4" in self.env): + missing.append(["LOCAL_V4", "REMOTE_V4"]) + if missing: + raise Exception("Invalid environment, missing configuration:", missing, + "Please see tools/testing/selftests/drivers/net/README.rst") + def __enter__(self): return self