diff mbox series

[v3,4/9] kunit: Add ability to filter attributes

Message ID 20230725212522.1622716-5-rmoar@google.com (mailing list archive)
State Mainlined
Commit 529534e8cba3e60f843a682e2a3149612b30d608
Headers show
Series kunit: Add test attributes API | expand

Commit Message

Rae Moar July 25, 2023, 9:25 p.m. UTC
Add filtering of test attributes. Users can filter tests using the
module_param called "filter".

Filters are imputed in the format: <attribute_name><operation><value>

Example: kunit.filter="speed>slow"

Operations include: >, <, >=, <=, !=, and =. These operations will act the
same for attributes of the same type but may not between types.

Note multiple filters can be inputted by separating them with a comma.
Example: kunit.filter="speed=slow, module!=example"

Since both suites and test cases can have attributes, there may be
conflicts. The process of filtering follows these rules:
- Filtering always operates at a per-test level.
- If a test has an attribute set, then the test's value is filtered on.
- Otherwise, the value falls back to the suite's value.
- If neither are set, the attribute has a global "default" value, which
  is used.

Filtered tests will not be run or show in output. The tests can instead be
skipped using the configurable option "kunit.filter_action=skip".

Note the default settings for running tests remains unfiltered.

Finally, add "filter" methods for the speed and module attributes to parse
and compare attribute values.

Note this filtering functionality will be added to kunit.py in the next
patch.

Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Rae Moar <rmoar@google.com>
---

Changes since v2:
- Remove extra line.
Changes since v1:
- Fix compile warning of use of uninitialized variable
Changes since RFC v2:
- Change to output only one error before exiting.
Changes since RFC v1:
- Change method for inputting filters to allow for spaces in filtering
  values
- Add option to skip filtered tests instead of not run or show them with
  the- –filter_skip flag

 include/kunit/attributes.h |  31 +++++
 lib/kunit/attributes.c     | 271 +++++++++++++++++++++++++++++++++++++
 lib/kunit/executor.c       |  93 ++++++++++---
 lib/kunit/executor_test.c  |  12 +-
 lib/kunit/test.c           |  10 +-
 5 files changed, 389 insertions(+), 28 deletions(-)

Comments

Guenter Roeck Aug. 29, 2023, 10:46 p.m. UTC | #1
Hi,

On Tue, Jul 25, 2023 at 09:25:15PM +0000, Rae Moar wrote:
> Add filtering of test attributes. Users can filter tests using the
> module_param called "filter".
> 
> Filters are imputed in the format: <attribute_name><operation><value>
> 
> Example: kunit.filter="speed>slow"
> 
> Operations include: >, <, >=, <=, !=, and =. These operations will act the
> same for attributes of the same type but may not between types.
> 
> Note multiple filters can be inputted by separating them with a comma.
> Example: kunit.filter="speed=slow, module!=example"
> 
> Since both suites and test cases can have attributes, there may be
> conflicts. The process of filtering follows these rules:
> - Filtering always operates at a per-test level.
> - If a test has an attribute set, then the test's value is filtered on.
> - Otherwise, the value falls back to the suite's value.
> - If neither are set, the attribute has a global "default" value, which
>   is used.
> 
> Filtered tests will not be run or show in output. The tests can instead be
> skipped using the configurable option "kunit.filter_action=skip".
> 
> Note the default settings for running tests remains unfiltered.
> 
> Finally, add "filter" methods for the speed and module attributes to parse
> and compare attribute values.
> 
> Note this filtering functionality will be added to kunit.py in the next
> patch.
> 
> Reviewed-by: David Gow <davidgow@google.com>
> Signed-off-by: Rae Moar <rmoar@google.com>

This patch causes all my s390 boot tests to crash.

[   17.009188]     # module: kunit
[   17.009239]     1..8
[   17.014967]     ok 1 parse_filter_test
[   17.020755]     ok 2 filter_suites_test
[   17.026350]     ok 3 filter_suites_test_glob_test
[   17.033102]     ok 4 filter_suites_to_empty_test
[   17.035422] Unable to handle kernel pointer dereference in virtual kernel address space
[   17.035546] Failing address: 0000000001501000 TEID: 0000000001501407
[   17.035617] Fault in home space mode while using kernel ASCE.
[   17.035833] AS:0000000002164007 R3:000000001f71c007 S:000000001f711000 P:000000000150131f
[   17.036313] Oops: 0004 ilc:2 [#1] SMP
[   17.036851] Modules linked in:
[   17.037026] CPU: 0 PID: 193 Comm: kunit_try_catch Tainted: G                 N 6.5.0-01207-g1c59d383390f #1
[   17.037106] Hardware name: QEMU 8561 QEMU (KVM/Linux)
[   17.037199] Krnl PSW : 0704e00180000000 0000000000950e18 (kunit_next_attr_filter+0xc8/0x1c8)
[   17.037451]            R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3
[   17.037531] Krnl GPRS: 0000037f00000000 000000000150120a 000000000000003e 000000000000003e
[   17.037583]            0000038000197e04 0000038000197c04 0000038000197df0 000003800000000a
[   17.037629]            0000038000000005 0000000001501203 00000000015011fe 0000037f0000000c
[   17.037675]            0000000005404100 0000038000197e08 0000000000950dda 0000038000197cd8
[   17.038176] Krnl Code: 0000000000950e0a: b9180098		agfr	%r9,%r8
[   17.038176]            0000000000950e0e: a7080000		lhi	%r0,0
[   17.038176]           #0000000000950e12: e32090000090	llgc	%r2,0(%r9)
[   17.038176]           >0000000000950e18: 92009000		mvi	0(%r9),0
[   17.038176]            0000000000950e1c: b904003a		lgr	%r3,%r10
[   17.038176]            0000000000950e20: c418007bd73c	lgrl	%r1,00000000018cbc98
[   17.038176]            0000000000950e26: b25d0013		clst	%r1,%r3
[   17.038176]            0000000000950e2a: a714fffe		brc	1,0000000000950e26
[   17.038625] Call Trace:
[   17.038679]  [<0000000000950e18>] kunit_next_attr_filter+0xc8/0x1c8
[   17.038749]  [<000000000094e5f2>] parse_filter_attr_test+0x92/0x360
[   17.038796]  [<000000000094e39a>] kunit_generic_run_threadfn_adapter+0x32/0x48
[   17.038840]  [<0000000000186702>] kthread+0x142/0x148
[   17.038884]  [<0000000000103138>] __ret_from_fork+0x40/0x58
[   17.038927]  [<000000000102df9a>] ret_from_fork+0xa/0x30
[   17.038984] INFO: lockdep is turned off.
[   17.039027] Last Breaking-Event-Address:
[   17.039062]  [<0000000000950da4>] kunit_next_attr_filter+0x54/0x1c8
[   17.039303] Kernel panic - not syncing: Fatal exception: panic_on_oops

Bisect log is attached.

Guenter

---
# bad: [1c59d383390f970b891b503b7f79b63a02db2ec5] Merge tag 'linux-kselftest-nolibc-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
# good: [2dde18cd1d8fac735875f2e4987f11817cc0bc2c] Linux 6.5
git bisect start '1c59d383390f' 'v6.5'
# good: [42a7f6e3ffe06308c1ec43a7dac39a27de101574] Merge tag 'x86_microcode_for_v6.6_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 42a7f6e3ffe06308c1ec43a7dac39a27de101574
# good: [330235e87410349042468b52baff02af7cb7d331] Merge tag 'acpi-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
git bisect good 330235e87410349042468b52baff02af7cb7d331
# good: [5a31cc7297072a7266a910ca5266b640d27803b4] Merge tag 'linux-kselftest-next-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
git bisect good 5a31cc7297072a7266a910ca5266b640d27803b4
# good: [eea70cdac61087fdb7a1c1e6f306c0bf38ec39d4] tools/nolibc: riscv: shrink _start with _start_c
git bisect good eea70cdac61087fdb7a1c1e6f306c0bf38ec39d4
# good: [faeb4e09fe77262f9a6b2f9f874eec0b6850721e] selftests/nolibc: add test support for ppc64
git bisect good faeb4e09fe77262f9a6b2f9f874eec0b6850721e
# bad: [76066f93f1df27657eb937b7c9c091e3a6abf4db] kunit: add tests for filtering attributes
git bisect bad 76066f93f1df27657eb937b7c9c091e3a6abf4db
# good: [64bd4641310c41a1ecf07c13c67bc0ed61045dfd] MAINTAINERS: add Rust KUnit files to the KUnit entry
git bisect good 64bd4641310c41a1ecf07c13c67bc0ed61045dfd
# bad: [529534e8cba3e60f843a682e2a3149612b30d608] kunit: Add ability to filter attributes
git bisect bad 529534e8cba3e60f843a682e2a3149612b30d608
# good: [02c2d0c2a84172c3c7ec0229c61db55d23dd4730] kunit: Add speed attribute
git bisect good 02c2d0c2a84172c3c7ec0229c61db55d23dd4730
# good: [a00a72709175a4d53096301a8792b8171d1223e5] kunit: Add module attribute
git bisect good a00a72709175a4d53096301a8792b8171d1223e5
# first bad commit: [529534e8cba3e60f843a682e2a3149612b30d608] kunit: Add ability to filter attributes
David Gow Aug. 30, 2023, 12:24 a.m. UTC | #2
On Wed, 30 Aug 2023 at 06:46, Guenter Roeck <linux@roeck-us.net> wrote:
>
> Hi,
>
> On Tue, Jul 25, 2023 at 09:25:15PM +0000, Rae Moar wrote:
> > Add filtering of test attributes. Users can filter tests using the
> > module_param called "filter".
> >
> > Filters are imputed in the format: <attribute_name><operation><value>
> >
> > Example: kunit.filter="speed>slow"
> >
> > Operations include: >, <, >=, <=, !=, and =. These operations will act the
> > same for attributes of the same type but may not between types.
> >
> > Note multiple filters can be inputted by separating them with a comma.
> > Example: kunit.filter="speed=slow, module!=example"
> >
> > Since both suites and test cases can have attributes, there may be
> > conflicts. The process of filtering follows these rules:
> > - Filtering always operates at a per-test level.
> > - If a test has an attribute set, then the test's value is filtered on.
> > - Otherwise, the value falls back to the suite's value.
> > - If neither are set, the attribute has a global "default" value, which
> >   is used.
> >
> > Filtered tests will not be run or show in output. The tests can instead be
> > skipped using the configurable option "kunit.filter_action=skip".
> >
> > Note the default settings for running tests remains unfiltered.
> >
> > Finally, add "filter" methods for the speed and module attributes to parse
> > and compare attribute values.
> >
> > Note this filtering functionality will be added to kunit.py in the next
> > patch.
> >
> > Reviewed-by: David Gow <davidgow@google.com>
> > Signed-off-by: Rae Moar <rmoar@google.com>
>
> This patch causes all my s390 boot tests to crash.
>

Thanks: I can reproduce this here.

Looks like this is an issue with string constants in the tests not
being writable. (I'd've thought my testing with KASAN would pick up on
this, but I guess string literals are in a writable segment on
x86_64?)

Regardless, using char[] for the filters seems to work here. I've sent
out a patch to do so:
https://lore.kernel.org/linux-kselftest/20230830002116.3768675-1-davidgow@google.com/

That fixes the issue with the stacktrack below, but if you bisected it
to this patch, maybe there's another issue. The tests weren't
introduced until 76066f93f1df ("kunit: add tests for filtering
attributes"). I don't get any crashes with this patch applied, though,
so let me know if something's still broken.

Cheers,
-- David


> [   17.009188]     # module: kunit
> [   17.009239]     1..8
> [   17.014967]     ok 1 parse_filter_test
> [   17.020755]     ok 2 filter_suites_test
> [   17.026350]     ok 3 filter_suites_test_glob_test
> [   17.033102]     ok 4 filter_suites_to_empty_test
> [   17.035422] Unable to handle kernel pointer dereference in virtual kernel address space
> [   17.035546] Failing address: 0000000001501000 TEID: 0000000001501407
> [   17.035617] Fault in home space mode while using kernel ASCE.
> [   17.035833] AS:0000000002164007 R3:000000001f71c007 S:000000001f711000 P:000000000150131f
> [   17.036313] Oops: 0004 ilc:2 [#1] SMP
> [   17.036851] Modules linked in:
> [   17.037026] CPU: 0 PID: 193 Comm: kunit_try_catch Tainted: G                 N 6.5.0-01207-g1c59d383390f #1
> [   17.037106] Hardware name: QEMU 8561 QEMU (KVM/Linux)
> [   17.037199] Krnl PSW : 0704e00180000000 0000000000950e18 (kunit_next_attr_filter+0xc8/0x1c8)
> [   17.037451]            R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3
> [   17.037531] Krnl GPRS: 0000037f00000000 000000000150120a 000000000000003e 000000000000003e
> [   17.037583]            0000038000197e04 0000038000197c04 0000038000197df0 000003800000000a
> [   17.037629]            0000038000000005 0000000001501203 00000000015011fe 0000037f0000000c
> [   17.037675]            0000000005404100 0000038000197e08 0000000000950dda 0000038000197cd8
> [   17.038176] Krnl Code: 0000000000950e0a: b9180098            agfr    %r9,%r8
> [   17.038176]            0000000000950e0e: a7080000            lhi     %r0,0
> [   17.038176]           #0000000000950e12: e32090000090        llgc    %r2,0(%r9)
> [   17.038176]           >0000000000950e18: 92009000            mvi     0(%r9),0
> [   17.038176]            0000000000950e1c: b904003a            lgr     %r3,%r10
> [   17.038176]            0000000000950e20: c418007bd73c        lgrl    %r1,00000000018cbc98
> [   17.038176]            0000000000950e26: b25d0013            clst    %r1,%r3
> [   17.038176]            0000000000950e2a: a714fffe            brc     1,0000000000950e26
> [   17.038625] Call Trace:
> [   17.038679]  [<0000000000950e18>] kunit_next_attr_filter+0xc8/0x1c8
> [   17.038749]  [<000000000094e5f2>] parse_filter_attr_test+0x92/0x360
> [   17.038796]  [<000000000094e39a>] kunit_generic_run_threadfn_adapter+0x32/0x48
> [   17.038840]  [<0000000000186702>] kthread+0x142/0x148
> [   17.038884]  [<0000000000103138>] __ret_from_fork+0x40/0x58
> [   17.038927]  [<000000000102df9a>] ret_from_fork+0xa/0x30
> [   17.038984] INFO: lockdep is turned off.
> [   17.039027] Last Breaking-Event-Address:
> [   17.039062]  [<0000000000950da4>] kunit_next_attr_filter+0x54/0x1c8
> [   17.039303] Kernel panic - not syncing: Fatal exception: panic_on_oops
>
> Bisect log is attached.
>
> Guenter
>
> ---
> # bad: [1c59d383390f970b891b503b7f79b63a02db2ec5] Merge tag 'linux-kselftest-nolibc-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
> # good: [2dde18cd1d8fac735875f2e4987f11817cc0bc2c] Linux 6.5
> git bisect start '1c59d383390f' 'v6.5'
> # good: [42a7f6e3ffe06308c1ec43a7dac39a27de101574] Merge tag 'x86_microcode_for_v6.6_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
> git bisect good 42a7f6e3ffe06308c1ec43a7dac39a27de101574
> # good: [330235e87410349042468b52baff02af7cb7d331] Merge tag 'acpi-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
> git bisect good 330235e87410349042468b52baff02af7cb7d331
> # good: [5a31cc7297072a7266a910ca5266b640d27803b4] Merge tag 'linux-kselftest-next-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
> git bisect good 5a31cc7297072a7266a910ca5266b640d27803b4
> # good: [eea70cdac61087fdb7a1c1e6f306c0bf38ec39d4] tools/nolibc: riscv: shrink _start with _start_c
> git bisect good eea70cdac61087fdb7a1c1e6f306c0bf38ec39d4
> # good: [faeb4e09fe77262f9a6b2f9f874eec0b6850721e] selftests/nolibc: add test support for ppc64
> git bisect good faeb4e09fe77262f9a6b2f9f874eec0b6850721e
> # bad: [76066f93f1df27657eb937b7c9c091e3a6abf4db] kunit: add tests for filtering attributes
> git bisect bad 76066f93f1df27657eb937b7c9c091e3a6abf4db
> # good: [64bd4641310c41a1ecf07c13c67bc0ed61045dfd] MAINTAINERS: add Rust KUnit files to the KUnit entry
> git bisect good 64bd4641310c41a1ecf07c13c67bc0ed61045dfd
> # bad: [529534e8cba3e60f843a682e2a3149612b30d608] kunit: Add ability to filter attributes
> git bisect bad 529534e8cba3e60f843a682e2a3149612b30d608
> # good: [02c2d0c2a84172c3c7ec0229c61db55d23dd4730] kunit: Add speed attribute
> git bisect good 02c2d0c2a84172c3c7ec0229c61db55d23dd4730
> # good: [a00a72709175a4d53096301a8792b8171d1223e5] kunit: Add module attribute
> git bisect good a00a72709175a4d53096301a8792b8171d1223e5
> # first bad commit: [529534e8cba3e60f843a682e2a3149612b30d608] kunit: Add ability to filter attributes
Guenter Roeck Aug. 30, 2023, 1:48 a.m. UTC | #3
On 8/29/23 17:24, David Gow wrote:
> On Wed, 30 Aug 2023 at 06:46, Guenter Roeck <linux@roeck-us.net> wrote:
>>
>> Hi,
>>
>> On Tue, Jul 25, 2023 at 09:25:15PM +0000, Rae Moar wrote:
>>> Add filtering of test attributes. Users can filter tests using the
>>> module_param called "filter".
>>>
>>> Filters are imputed in the format: <attribute_name><operation><value>
>>>
>>> Example: kunit.filter="speed>slow"
>>>
>>> Operations include: >, <, >=, <=, !=, and =. These operations will act the
>>> same for attributes of the same type but may not between types.
>>>
>>> Note multiple filters can be inputted by separating them with a comma.
>>> Example: kunit.filter="speed=slow, module!=example"
>>>
>>> Since both suites and test cases can have attributes, there may be
>>> conflicts. The process of filtering follows these rules:
>>> - Filtering always operates at a per-test level.
>>> - If a test has an attribute set, then the test's value is filtered on.
>>> - Otherwise, the value falls back to the suite's value.
>>> - If neither are set, the attribute has a global "default" value, which
>>>    is used.
>>>
>>> Filtered tests will not be run or show in output. The tests can instead be
>>> skipped using the configurable option "kunit.filter_action=skip".
>>>
>>> Note the default settings for running tests remains unfiltered.
>>>
>>> Finally, add "filter" methods for the speed and module attributes to parse
>>> and compare attribute values.
>>>
>>> Note this filtering functionality will be added to kunit.py in the next
>>> patch.
>>>
>>> Reviewed-by: David Gow <davidgow@google.com>
>>> Signed-off-by: Rae Moar <rmoar@google.com>
>>
>> This patch causes all my s390 boot tests to crash.
>>
> 
> Thanks: I can reproduce this here.
> 
> Looks like this is an issue with string constants in the tests not
> being writable. (I'd've thought my testing with KASAN would pick up on
> this, but I guess string literals are in a writable segment on
> x86_64?)
> 
> Regardless, using char[] for the filters seems to work here. I've sent
> out a patch to do so:
> https://lore.kernel.org/linux-kselftest/20230830002116.3768675-1-davidgow@google.com/
> 
> That fixes the issue with the stacktrack below, but if you bisected it
> to this patch, maybe there's another issue. The tests weren't
> introduced until 76066f93f1df ("kunit: add tests for filtering
> attributes"). I don't get any crashes with this patch applied, though,
> so let me know if something's still broken.
> 

Your patch works just fine. Maybe bisect acted up ? I don't recall seeing
this happen, but who knows.

Thanks a lot for the quick fix!

Guenter
diff mbox series

Patch

diff --git a/include/kunit/attributes.h b/include/kunit/attributes.h
index 9fcd184cce36..bc76a0b786d2 100644
--- a/include/kunit/attributes.h
+++ b/include/kunit/attributes.h
@@ -9,6 +9,20 @@ 
 #ifndef _KUNIT_ATTRIBUTES_H
 #define _KUNIT_ATTRIBUTES_H
 
+/*
+ * struct kunit_attr_filter - representation of attributes filter with the
+ * attribute object and string input
+ */
+struct kunit_attr_filter {
+	struct kunit_attr *attr;
+	char *input;
+};
+
+/*
+ * Returns the name of the filter's attribute.
+ */
+const char *kunit_attr_filter_name(struct kunit_attr_filter filter);
+
 /*
  * Print all test attributes for a test case or suite.
  * Output format for test cases: "# <test_name>.<attribute>: <value>"
@@ -16,4 +30,21 @@ 
  */
 void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level);
 
+/*
+ * Returns the number of fitlers in input.
+ */
+int kunit_get_filter_count(char *input);
+
+/*
+ * Parse attributes filter input and return an objects containing the
+ * attribute object and the string input of the next filter.
+ */
+struct kunit_attr_filter kunit_next_attr_filter(char **filters, int *err);
+
+/*
+ * Returns a copy of the suite containing only tests that pass the filter.
+ */
+struct kunit_suite *kunit_filter_attr_tests(const struct kunit_suite *const suite,
+		struct kunit_attr_filter filter, char *action, int *err);
+
 #endif /* _KUNIT_ATTRIBUTES_H */
diff --git a/lib/kunit/attributes.c b/lib/kunit/attributes.c
index 9dce4f4d726c..d37c40c0ce4f 100644
--- a/lib/kunit/attributes.c
+++ b/lib/kunit/attributes.c
@@ -67,6 +67,104 @@  static const char *attr_string_to_string(void *attr, bool *to_free)
 	return (char *) attr;
 }
 
+/* Filter Methods */
+
+static const char op_list[] = "<>!=";
+
+/*
+ * Returns whether the inputted integer value matches the filter given
+ * by the operation string and inputted integer.
+ */
+static int int_filter(long val, const char *op, int input, int *err)
+{
+	if (!strncmp(op, "<=", 2))
+		return (val <= input);
+	else if (!strncmp(op, ">=", 2))
+		return (val >= input);
+	else if (!strncmp(op, "!=", 2))
+		return (val != input);
+	else if (!strncmp(op, ">", 1))
+		return (val > input);
+	else if (!strncmp(op, "<", 1))
+		return (val < input);
+	else if (!strncmp(op, "=", 1))
+		return (val == input);
+	*err = -EINVAL;
+	pr_err("kunit executor: invalid filter operation: %s\n", op);
+	return false;
+}
+
+/*
+ * Returns whether the inputted enum value "attr" matches the filter given
+ * by the input string. Note: the str_list includes the corresponding string
+ * list to the enum values.
+ */
+static int attr_enum_filter(void *attr, const char *input, int *err,
+		const char * const str_list[], int max)
+{
+	int i, j, input_int;
+	long test_val = (long)attr;
+	const char *input_val = NULL;
+
+	for (i = 0; input[i]; i++) {
+		if (!strchr(op_list, input[i])) {
+			input_val = input + i;
+			break;
+		}
+	}
+
+	if (!input_val) {
+		*err = -EINVAL;
+		pr_err("kunit executor: filter value not found: %s\n", input);
+		return false;
+	}
+
+	for (j = 0; j <= max; j++) {
+		if (!strcmp(input_val, str_list[j]))
+			input_int = j;
+	}
+
+	if (!input_int) {
+		*err = -EINVAL;
+		pr_err("kunit executor: invalid filter input: %s\n", input);
+		return false;
+	}
+
+	return int_filter(test_val, input, input_int, err);
+}
+
+static int attr_speed_filter(void *attr, const char *input, int *err)
+{
+	return attr_enum_filter(attr, input, err, speed_str_list, KUNIT_SPEED_MAX);
+}
+
+/*
+ * Returns whether the inputted string value (attr) matches the filter given
+ * by the input string.
+ */
+static int attr_string_filter(void *attr, const char *input, int *err)
+{
+	char *str = attr;
+
+	if (!strncmp(input, "<", 1)) {
+		*err = -EINVAL;
+		pr_err("kunit executor: invalid filter input: %s\n", input);
+		return false;
+	} else if (!strncmp(input, ">", 1)) {
+		*err = -EINVAL;
+		pr_err("kunit executor: invalid filter input: %s\n", input);
+		return false;
+	} else if (!strncmp(input, "!=", 2)) {
+		return (strcmp(input + 2, str) != 0);
+	} else if (!strncmp(input, "=", 1)) {
+		return (strcmp(input + 1, str) == 0);
+	}
+	*err = -EINVAL;
+	pr_err("kunit executor: invalid filter operation: %s\n", input);
+	return false;
+}
+
+
 /* Get Attribute Methods */
 
 static void *attr_speed_get(void *test_or_suite, bool is_test)
@@ -99,6 +197,7 @@  static struct kunit_attr kunit_attr_list[] = {
 		.name = "speed",
 		.get_attr = attr_speed_get,
 		.to_string = attr_speed_to_string,
+		.filter = attr_speed_filter,
 		.attr_default = (void *)KUNIT_SPEED_NORMAL,
 		.print = PRINT_ALWAYS,
 	},
@@ -106,6 +205,7 @@  static struct kunit_attr kunit_attr_list[] = {
 		.name = "module",
 		.get_attr = attr_module_get,
 		.to_string = attr_string_to_string,
+		.filter = attr_string_filter,
 		.attr_default = (void *)"",
 		.print = PRINT_SUITE,
 	}
@@ -113,6 +213,11 @@  static struct kunit_attr kunit_attr_list[] = {
 
 /* Helper Functions to Access Attributes */
 
+const char *kunit_attr_filter_name(struct kunit_attr_filter filter)
+{
+	return filter.attr->name;
+}
+
 void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level)
 {
 	int i;
@@ -145,3 +250,169 @@  void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level
 		}
 	}
 }
+
+/* Helper Functions to Filter Attributes */
+
+int kunit_get_filter_count(char *input)
+{
+	int i, comma_index, count = 0;
+
+	for (i = 0; input[i]; i++) {
+		if (input[i] == ',') {
+			if ((i - comma_index) > 1)
+				count++;
+			comma_index = i;
+		}
+	}
+	if ((i - comma_index) > 0)
+		count++;
+	return count;
+}
+
+struct kunit_attr_filter kunit_next_attr_filter(char **filters, int *err)
+{
+	struct kunit_attr_filter filter = {};
+	int i, j, comma_index, new_start_index;
+	int op_index = -1, attr_index = -1;
+	char op;
+	char *input = *filters;
+
+	/* Parse input until operation */
+	for (i = 0; input[i]; i++) {
+		if (op_index < 0 && strchr(op_list, input[i])) {
+			op_index = i;
+		} else if (!comma_index && input[i] == ',') {
+			comma_index = i;
+		} else if (comma_index && input[i] != ' ') {
+			new_start_index = i;
+			break;
+		}
+	}
+
+	if (op_index <= 0) {
+		*err = -EINVAL;
+		pr_err("kunit executor: filter operation not found: %s\n", input);
+		return filter;
+	}
+
+	/* Temporarily set operator to \0 character. */
+	op = input[op_index];
+	input[op_index] = '\0';
+
+	/* Find associated kunit_attr object */
+	for (j = 0; j < ARRAY_SIZE(kunit_attr_list); j++) {
+		if (!strcmp(input, kunit_attr_list[j].name)) {
+			attr_index = j;
+			break;
+		}
+	}
+
+	input[op_index] = op;
+
+	if (attr_index < 0) {
+		*err = -EINVAL;
+		pr_err("kunit executor: attribute not found: %s\n", input);
+	} else {
+		filter.attr = &kunit_attr_list[attr_index];
+	}
+
+	if (comma_index) {
+		input[comma_index] = '\0';
+		filter.input = input + op_index;
+		input = input + new_start_index;
+	} else {
+		filter.input = input + op_index;
+		input = NULL;
+	}
+
+	*filters = input;
+
+	return filter;
+}
+
+struct kunit_suite *kunit_filter_attr_tests(const struct kunit_suite *const suite,
+		struct kunit_attr_filter filter, char *action, int *err)
+{
+	int n = 0;
+	struct kunit_case *filtered, *test_case;
+	struct kunit_suite *copy;
+	void *suite_val, *test_val;
+	bool suite_result, test_result, default_result, result;
+
+	/* Allocate memory for new copy of suite and list of test cases */
+	copy = kmemdup(suite, sizeof(*copy), GFP_KERNEL);
+	if (!copy)
+		return ERR_PTR(-ENOMEM);
+
+	kunit_suite_for_each_test_case(suite, test_case) { n++; }
+
+	filtered = kcalloc(n + 1, sizeof(*filtered), GFP_KERNEL);
+	if (!filtered) {
+		kfree(copy);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	n = 0;
+
+	/* Save filtering result on default value */
+	default_result = filter.attr->filter(filter.attr->attr_default, filter.input, err);
+	if (*err) {
+		kfree(copy);
+		kfree(filtered);
+		return NULL;
+	}
+
+	/* Save suite attribute value and filtering result on that value */
+	suite_val = filter.attr->get_attr((void *)suite, false);
+	suite_result = filter.attr->filter(suite_val, filter.input, err);
+	if (*err) {
+		kfree(copy);
+		kfree(filtered);
+		return NULL;
+	}
+
+	/* For each test case, save test case if passes filtering. */
+	kunit_suite_for_each_test_case(suite, test_case) {
+		test_val = filter.attr->get_attr((void *) test_case, true);
+		test_result = filter.attr->filter(filter.attr->get_attr(test_case, true),
+				filter.input, err);
+		if (*err) {
+			kfree(copy);
+			kfree(filtered);
+			return NULL;
+		}
+
+		/*
+		 * If attribute value of test case is set, filter on that value.
+		 * If not, filter on suite value if set. If not, filter on
+		 * default value.
+		 */
+		result = false;
+		if (test_val) {
+			if (test_result)
+				result = true;
+		} else if (suite_val) {
+			if (suite_result)
+				result = true;
+		} else if (default_result) {
+			result = true;
+		}
+
+		if (result) {
+			filtered[n++] = *test_case;
+		} else if (action && strcmp(action, "skip") == 0) {
+			test_case->status = KUNIT_SKIPPED;
+			filtered[n++] = *test_case;
+		}
+	}
+
+	if (n == 0) {
+		kfree(copy);
+		kfree(filtered);
+		return NULL;
+	}
+
+	copy->test_cases = filtered;
+
+	return copy;
+}
diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c
index 12e38a48a5cc..483f7b7873a7 100644
--- a/lib/kunit/executor.c
+++ b/lib/kunit/executor.c
@@ -17,6 +17,8 @@  extern struct kunit_suite * const __kunit_suites_end[];
 
 static char *filter_glob_param;
 static char *action_param;
+static char *filter_param;
+static char *filter_action_param;
 
 module_param_named(filter_glob, filter_glob_param, charp, 0);
 MODULE_PARM_DESC(filter_glob,
@@ -27,15 +29,23 @@  MODULE_PARM_DESC(action,
 		 "<none>: run the tests like normal\n"
 		 "'list' to list test names instead of running them.\n"
 		 "'list_attr' to list test names and attributes instead of running them.\n");
+module_param_named(filter, filter_param, charp, 0);
+MODULE_PARM_DESC(filter,
+		"Filter which KUnit test suites/tests run at boot-time using attributes, e.g. speed>slow");
+module_param_named(filter_action, filter_action_param, charp, 0);
+MODULE_PARM_DESC(filter_action,
+		"Changes behavior of filtered tests using attributes, valid values are:\n"
+		"<none>: do not run filtered tests as normal\n"
+		"'skip': skip all filtered tests instead so tests will appear in output\n");
 
 /* glob_match() needs NULL terminated strings, so we need a copy of filter_glob_param. */
-struct kunit_test_filter {
+struct kunit_glob_filter {
 	char *suite_glob;
 	char *test_glob;
 };
 
 /* Split "suite_glob.test_glob" into two. Assumes filter_glob is not empty. */
-static void kunit_parse_filter_glob(struct kunit_test_filter *parsed,
+static void kunit_parse_glob_filter(struct kunit_glob_filter *parsed,
 				    const char *filter_glob)
 {
 	const int len = strlen(filter_glob);
@@ -57,7 +67,7 @@  static void kunit_parse_filter_glob(struct kunit_test_filter *parsed,
 
 /* Create a copy of suite with only tests that match test_glob. */
 static struct kunit_suite *
-kunit_filter_tests(const struct kunit_suite *const suite, const char *test_glob)
+kunit_filter_glob_tests(const struct kunit_suite *const suite, const char *test_glob)
 {
 	int n = 0;
 	struct kunit_case *filtered, *test_case;
@@ -111,12 +121,15 @@  static void kunit_free_suite_set(struct suite_set suite_set)
 
 static struct suite_set kunit_filter_suites(const struct suite_set *suite_set,
 					    const char *filter_glob,
+						char *filters,
+						char *filter_action,
 					    int *err)
 {
-	int i;
-	struct kunit_suite **copy, *filtered_suite;
+	int i, j, k, filter_count;
+	struct kunit_suite **copy, *filtered_suite, *new_filtered_suite;
 	struct suite_set filtered;
-	struct kunit_test_filter filter;
+	struct kunit_glob_filter parsed_glob;
+	struct kunit_attr_filter *parsed_filters;
 
 	const size_t max = suite_set->end - suite_set->start;
 
@@ -127,17 +140,52 @@  static struct suite_set kunit_filter_suites(const struct suite_set *suite_set,
 		return filtered;
 	}
 
-	kunit_parse_filter_glob(&filter, filter_glob);
+	if (filter_glob)
+		kunit_parse_glob_filter(&parsed_glob, filter_glob);
 
-	for (i = 0; &suite_set->start[i] != suite_set->end; i++) {
-		if (!glob_match(filter.suite_glob, suite_set->start[i]->name))
-			continue;
-
-		filtered_suite = kunit_filter_tests(suite_set->start[i], filter.test_glob);
-		if (IS_ERR(filtered_suite)) {
-			*err = PTR_ERR(filtered_suite);
+	/* Parse attribute filters */
+	if (filters) {
+		filter_count = kunit_get_filter_count(filters);
+		parsed_filters = kcalloc(filter_count + 1, sizeof(*parsed_filters), GFP_KERNEL);
+		for (j = 0; j < filter_count; j++)
+			parsed_filters[j] = kunit_next_attr_filter(&filters, err);
+		if (*err)
 			return filtered;
+	}
+
+	for (i = 0; &suite_set->start[i] != suite_set->end; i++) {
+		filtered_suite = suite_set->start[i];
+		if (filter_glob) {
+			if (!glob_match(parsed_glob.suite_glob, filtered_suite->name))
+				continue;
+			filtered_suite = kunit_filter_glob_tests(filtered_suite,
+					parsed_glob.test_glob);
+			if (IS_ERR(filtered_suite)) {
+				*err = PTR_ERR(filtered_suite);
+				return filtered;
+			}
 		}
+		if (filter_count) {
+			for (k = 0; k < filter_count; k++) {
+				new_filtered_suite = kunit_filter_attr_tests(filtered_suite,
+						parsed_filters[k], filter_action, err);
+
+				/* Free previous copy of suite */
+				if (k > 0 || filter_glob)
+					kfree(filtered_suite);
+				filtered_suite = new_filtered_suite;
+
+				if (*err)
+					return filtered;
+				if (IS_ERR(filtered_suite)) {
+					*err = PTR_ERR(filtered_suite);
+					return filtered;
+				}
+				if (!filtered_suite)
+					break;
+			}
+		}
+
 		if (!filtered_suite)
 			continue;
 
@@ -145,8 +193,14 @@  static struct suite_set kunit_filter_suites(const struct suite_set *suite_set,
 	}
 	filtered.end = copy;
 
-	kfree(filter.suite_glob);
-	kfree(filter.test_glob);
+	if (filter_glob) {
+		kfree(parsed_glob.suite_glob);
+		kfree(parsed_glob.test_glob);
+	}
+
+	if (filter_count)
+		kfree(parsed_filters);
+
 	return filtered;
 }
 
@@ -206,8 +260,9 @@  int kunit_run_all_tests(void)
 		goto out;
 	}
 
-	if (filter_glob_param) {
-		suite_set = kunit_filter_suites(&suite_set, filter_glob_param, &err);
+	if (filter_glob_param || filter_param) {
+		suite_set = kunit_filter_suites(&suite_set, filter_glob_param,
+				filter_param, filter_action_param, &err);
 		if (err) {
 			pr_err("kunit executor: error filtering suites: %d\n", err);
 			goto out;
@@ -223,7 +278,7 @@  int kunit_run_all_tests(void)
 	else
 		pr_err("kunit executor: unknown action '%s'\n", action_param);
 
-	if (filter_glob_param) { /* a copy was made of each suite */
+	if (filter_glob_param || filter_param) { /* a copy was made of each suite */
 		kunit_free_suite_set(suite_set);
 	}
 
diff --git a/lib/kunit/executor_test.c b/lib/kunit/executor_test.c
index ce6749af374d..d7ab069324b5 100644
--- a/lib/kunit/executor_test.c
+++ b/lib/kunit/executor_test.c
@@ -24,15 +24,15 @@  static struct kunit_case dummy_test_cases[] = {
 
 static void parse_filter_test(struct kunit *test)
 {
-	struct kunit_test_filter filter = {NULL, NULL};
+	struct kunit_glob_filter filter = {NULL, NULL};
 
-	kunit_parse_filter_glob(&filter, "suite");
+	kunit_parse_glob_filter(&filter, "suite");
 	KUNIT_EXPECT_STREQ(test, filter.suite_glob, "suite");
 	KUNIT_EXPECT_FALSE(test, filter.test_glob);
 	kfree(filter.suite_glob);
 	kfree(filter.test_glob);
 
-	kunit_parse_filter_glob(&filter, "suite.test");
+	kunit_parse_glob_filter(&filter, "suite.test");
 	KUNIT_EXPECT_STREQ(test, filter.suite_glob, "suite");
 	KUNIT_EXPECT_STREQ(test, filter.test_glob, "test");
 	kfree(filter.suite_glob);
@@ -50,7 +50,7 @@  static void filter_suites_test(struct kunit *test)
 	subsuite[1] = alloc_fake_suite(test, "suite2", dummy_test_cases);
 
 	/* Want: suite1, suite2, NULL -> suite2, NULL */
-	got = kunit_filter_suites(&suite_set, "suite2", &err);
+	got = kunit_filter_suites(&suite_set, "suite2", NULL, NULL, &err);
 	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start);
 	KUNIT_ASSERT_EQ(test, err, 0);
 	kfree_at_end(test, got.start);
@@ -74,7 +74,7 @@  static void filter_suites_test_glob_test(struct kunit *test)
 	subsuite[1] = alloc_fake_suite(test, "suite2", dummy_test_cases);
 
 	/* Want: suite1, suite2, NULL -> suite2 (just test1), NULL */
-	got = kunit_filter_suites(&suite_set, "suite2.test2", &err);
+	got = kunit_filter_suites(&suite_set, "suite2.test2", NULL, NULL, &err);
 	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start);
 	KUNIT_ASSERT_EQ(test, err, 0);
 	kfree_at_end(test, got.start);
@@ -100,7 +100,7 @@  static void filter_suites_to_empty_test(struct kunit *test)
 	subsuite[0] = alloc_fake_suite(test, "suite1", dummy_test_cases);
 	subsuite[1] = alloc_fake_suite(test, "suite2", dummy_test_cases);
 
-	got = kunit_filter_suites(&suite_set, "not_found", &err);
+	got = kunit_filter_suites(&suite_set, "not_found", NULL, NULL, &err);
 	KUNIT_ASSERT_EQ(test, err, 0);
 	kfree_at_end(test, got.start); /* just in case */
 
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index 9ee55139ecd1..cb9797fa6303 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -613,18 +613,22 @@  int kunit_run_tests(struct kunit_suite *suite)
 	kunit_suite_for_each_test_case(suite, test_case) {
 		struct kunit test = { .param_value = NULL, .param_index = 0 };
 		struct kunit_result_stats param_stats = { 0 };
-		test_case->status = KUNIT_SKIPPED;
 
 		kunit_init_test(&test, test_case->name, test_case->log);
-
-		if (!test_case->generate_params) {
+		if (test_case->status == KUNIT_SKIPPED) {
+			/* Test marked as skip */
+			test.status = KUNIT_SKIPPED;
+			kunit_update_stats(&param_stats, test.status);
+		} else if (!test_case->generate_params) {
 			/* Non-parameterised test. */
+			test_case->status = KUNIT_SKIPPED;
 			kunit_run_case_catch_errors(suite, test_case, &test);
 			kunit_update_stats(&param_stats, test.status);
 		} else {
 			/* Get initial param. */
 			param_desc[0] = '\0';
 			test.param_value = test_case->generate_params(NULL, param_desc);
+			test_case->status = KUNIT_SKIPPED;
 			kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
 				  "KTAP version 1\n");
 			kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT