| Message ID | 20190406214915.16914-1-pvorel@suse.cz (mailing list archive) |
|---|---|
| Headers | show |
| Series | Kselftest shell (or even C) API | expand |
Hi! > It's just a small example how things could be. Draft, not meant to be > really merged. But instead of defining shell library (with more useful > helpers), I'd rather adopt LTP shell [1] and C [2] API to kselftest. > LTP API [1] is more like a framework, easy to use with a lot of helpers > making tests 1) small, concentrating on the problem itself 2) have > unique output. API is well documented [3] [4], it's creator Cyril Hrubis > made it after years experience of handling (at the time) quite bad > quality LTP code. Rewriting LTP tests to use this API improved tests a > lot (less buggy, easier to read). > > Some examples of advantages of LTP API: > * SAFE_*() macros for C, which handles errors inside a library > * unified messages, unified test status, unified way to exit testing due > missing functionality, at the end of testing there is summary of passed, > failed and skipped tests > * many prepared functionality for both C and shell > * handling threads, parent-child synchronization > * setup and cleanup functions > * "flags" for defining requirements or certain functionality (need root, temporary > directory, ...) > * and many other I guess that I can help to create a library with a subset of LTP C API that could be used to implement C tests if that is something that has a good chance to get adopted.
Hi Petr, Shuah, On Sat, 2019-04-06 at 23:49 +0200, Petr Vorel wrote: > Hi, > > this is a draft trying to define some API in order to remove some > redundancy from kselftest shell scripts. Existing kselftest.h already > defines some sort of API for C, there is none for shell. Shuah, when the tests were in the selftests/ima directory I was planning on including them in my pull request; and then they moved to selftests/kexec. As they were still IMA related, I was still shepherding them and planned on including them in my pull request. (Is this Okay? Your Review/Ack would be much appreciated.) This patch set, however, introduces a set of "common" set of kselftest functions. Originally, you suggested deferring defining a set of "common" kselftests functions to prevent delaying upstreaming the tests. With these patches, that time is here. How do you want to handle this? Thanks, Mimi > > It's just a small example how things could be. Draft, not meant to be > really merged. But instead of defining shell library (with more useful > helpers), I'd rather adopt LTP shell [1] and C [2] API to kselftest. > LTP API [1] is more like a framework, easy to use with a lot of helpers > making tests 1) small, concentrating on the problem itself 2) have > unique output. API is well documented [3] [4], it's creator Cyril Hrubis > made it after years experience of handling (at the time) quite bad > quality LTP code. Rewriting LTP tests to use this API improved tests a > lot (less buggy, easier to read). > > Some examples of advantages of LTP API: > * SAFE_*() macros for C, which handles errors inside a library > * unified messages, unified test status, unified way to exit testing due > missing functionality, at the end of testing there is summary of passed, > failed and skipped tests > * many prepared functionality for both C and shell > * handling threads, parent-child synchronization > * setup and cleanup functions > * "flags" for defining requirements or certain functionality (need root, temporary > directory, ...) > * and many other > > kselftest and LTP has a bit different goals and approach. Probably > not all of LTP API is needed atm, but I guess it's at least worth of > thinking to adopt it. > > There are of course other options: reinvent a wheel or left kselftest > code in a state it is now (code quality varies, some of the code is > really messy, buggy, not even compile). > > [1] https://github.com/linux-test-project/ltp/blob/master/testcases/lib/tst_test.sh > [2] https://github.com/linux-test-project/ltp/tree/master/lib > [3] https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#22-writing-a-test-in-c > [4] https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#23-writing-a-testcase-in-shell > > Petr Vorel (2): > selftests: Start shell API > selftest/kexec: Use kselftest shell API > > .../selftests/kexec/kexec_common_lib.sh | 74 +++++-------------- > .../selftests/kexec/test_kexec_file_load.sh | 53 ++++++------- > .../selftests/kexec/test_kexec_load.sh | 20 ++--- > tools/testing/selftests/kselftest.sh | 53 +++++++++++++ > 4 files changed, 105 insertions(+), 95 deletions(-) > mode change 100755 => 100644 tools/testing/selftests/kexec/kexec_common_lib.sh > create mode 100644 tools/testing/selftests/kselftest.sh >
Hi Mimi, Shuah, > Hi Petr, Shuah, > On Sat, 2019-04-06 at 23:49 +0200, Petr Vorel wrote: > > Hi, > > this is a draft trying to define some API in order to remove some > > redundancy from kselftest shell scripts. Existing kselftest.h already > > defines some sort of API for C, there is none for shell. > Shuah, when the tests were in the selftests/ima directory I was > planning on including them in my pull request; and then they moved to > selftests/kexec. As they were still IMA related, I was still > shepherding them and planned on including them in my pull request. (Is > this Okay? Your Review/Ack would be much appreciated.) This patch > set, however, introduces a set of "common" set of kselftest functions. > Originally, you suggested deferring defining a set of "common" > kselftests functions to prevent delaying upstreaming the tests. With > these patches, that time is here. How do you want to handle this? I agree with separation of common kselftests functions / proper API effort. kexec tests are ready and IMHO should not be delayed with this effort. "common functions" proposed by this patchset are more for to start a discussion about it, what I brought doesn't help much. Proper design takes some time. > Thanks, > Mimi Kind regards, Petr
Hi, > I guess that I can help to create a library with a subset of LTP C API > that could be used to implement C tests if that is something that has a > good chance to get adopted. Great! I'd take shell part. So, it's up to Shuah I guess. Kind regards, Petr