[v2,linux-kselftest-test,0/3] kunit: support building core/tests as modules

Message

Alan Maguire Oct. 8, 2019, 2:55 p.m. UTC

The current kunit execution model is to provide base kunit functionality
and tests built-in to the kernel.  The aim of this series is to allow
building kunit itself and tests as modules.  This in turn allows a
simple form of selective execution; load the module you wish to test.
In doing so, kunit itself (if also built as a module) will be loaded as
an implicit dependency.

Because this requires a core API modification - if a module delivers
multiple suites, they must be declared with the kunit_test_suites()
macro - we're proposing this patch as a candidate to be applied to the
test tree before too many kunit consumers appear.  We attempt to deal
with existing consumers in patch 1.

Changes since v1:

- sent correct patch set; apologies, previous patch set was built
  prior to kunit move to lib/ and should be ignored.

Patch 1 consists changes needed to support loading tests as modules.
Patch 2 allows kunit itself to be loaded as a module.
Patch 3 documents module support.

Alan Maguire (3):
  kunit: allow kunit tests to be loaded as a module
  kunit: allow kunit to be loaded as a module
  kunit: update documentation to describe module-based build

 Documentation/dev-tools/kunit/faq.rst   |  3 ++-
 Documentation/dev-tools/kunit/index.rst |  3 +++
 Documentation/dev-tools/kunit/usage.rst | 16 ++++++++++++++++
 include/kunit/test.h                    | 30 +++++++++++++++++++++++-------
 kernel/sysctl-test.c                    |  6 +++++-
 lib/Kconfig.debug                       |  4 ++--
 lib/kunit/Kconfig                       |  6 +++---
 lib/kunit/Makefile                      |  4 +++-
 lib/kunit/assert.c                      |  8 ++++++++
 lib/kunit/example-test.c                |  6 +++++-
 lib/kunit/string-stream-test.c          |  9 +++++++--
 lib/kunit/string-stream.c               |  7 +++++++
 lib/kunit/test-test.c                   |  8 ++++++--
 lib/kunit/test.c                        | 12 ++++++++++++
 lib/kunit/try-catch.c                   |  8 ++++++--
 15 files changed, 108 insertions(+), 22 deletions(-)

Comments

Brendan Higgins Oct. 8, 2019, 9 p.m. UTC | #1

On Tue, Oct 08, 2019 at 03:55:43PM +0100, Alan Maguire wrote:
> The current kunit execution model is to provide base kunit functionality
> and tests built-in to the kernel.  The aim of this series is to allow
> building kunit itself and tests as modules.  This in turn allows a

Cool! I had plans for supporting this eventually, so I am more than
happy to accept support for this!

> simple form of selective execution; load the module you wish to test.
> In doing so, kunit itself (if also built as a module) will be loaded as
> an implicit dependency.

Seems like a reasonable initial approach. I had some plans for a
centralized test executor, but I don't think that this should be a
problem.

> Because this requires a core API modification - if a module delivers
> multiple suites, they must be declared with the kunit_test_suites()
> macro - we're proposing this patch as a candidate to be applied to the
> test tree before too many kunit consumers appear.  We attempt to deal
> with existing consumers in patch 1.

Makese sense.

Luis Chamberlain Oct. 14, 2019, 9:20 a.m. UTC | #2

On Tue, Oct 08, 2019 at 03:55:43PM +0100, Alan Maguire wrote:
> The current kunit execution model is to provide base kunit functionality
> and tests built-in to the kernel.  The aim of this series is to allow
> building kunit itself and tests as modules.  This in turn allows a
> simple form of selective execution; load the module you wish to test.
> In doing so, kunit itself (if also built as a module) will be loaded as
> an implicit dependency.
> 
> Because this requires a core API modification - if a module delivers
> multiple suites, they must be declared with the kunit_test_suites()
> macro - we're proposing this patch as a candidate to be applied to the
> test tree before too many kunit consumers appear.  We attempt to deal
> with existing consumers in patch 1.

This is neat and makes sense to me. However the ordering of the patches
seems odd. If modules depend on kunit module, then shouldn't that go
first? Ie, we want this to be bisectable in proper order.

  Luis

Alan Maguire Oct. 14, 2019, 2:02 p.m. UTC | #3

On Mon, 14 Oct 2019, Luis Chamberlain wrote:

> On Tue, Oct 08, 2019 at 03:55:43PM +0100, Alan Maguire wrote:
> > The current kunit execution model is to provide base kunit functionality
> > and tests built-in to the kernel.  The aim of this series is to allow
> > building kunit itself and tests as modules.  This in turn allows a
> > simple form of selective execution; load the module you wish to test.
> > In doing so, kunit itself (if also built as a module) will be loaded as
> > an implicit dependency.
> > 
> > Because this requires a core API modification - if a module delivers
> > multiple suites, they must be declared with the kunit_test_suites()
> > macro - we're proposing this patch as a candidate to be applied to the
> > test tree before too many kunit consumers appear.  We attempt to deal
> > with existing consumers in patch 1.
> 
> This is neat and makes sense to me.

Thanks for taking a look!

> However the ordering of the patches
> seems odd. If modules depend on kunit module, then shouldn't that go
> first? Ie, we want this to be bisectable in proper order.
> 

The reasoning here is it seemed a more likely scenario that users mught  
build kunit built-in (CONFIG_KUNIT=y) along with test suites built as 
modules (CONFIG_KUNIT_TEST=m). So the intermediate state after patch 2 - 
tests buildable as modules while kunit is still built-in-only - made more 
sense to me as something users might do in practice so that's why I 
ordered things that way.  I'm working on a new revision of the patchset
though, so if you feel strongly about this shout and I'll try and accommodate
the alternative ordering.

Thanks!

Alan  

>   Luis
>

Luis Chamberlain Oct. 16, 2019, 12:47 p.m. UTC | #4

On Mon, Oct 14, 2019 at 03:02:03PM +0100, Alan Maguire wrote:
> 
> 
> On Mon, 14 Oct 2019, Luis Chamberlain wrote:
> 
> > On Tue, Oct 08, 2019 at 03:55:43PM +0100, Alan Maguire wrote:
> > > The current kunit execution model is to provide base kunit functionality
> > > and tests built-in to the kernel.  The aim of this series is to allow
> > > building kunit itself and tests as modules.  This in turn allows a
> > > simple form of selective execution; load the module you wish to test.
> > > In doing so, kunit itself (if also built as a module) will be loaded as
> > > an implicit dependency.
> > > 
> > > Because this requires a core API modification - if a module delivers
> > > multiple suites, they must be declared with the kunit_test_suites()
> > > macro - we're proposing this patch as a candidate to be applied to the
> > > test tree before too many kunit consumers appear.  We attempt to deal
> > > with existing consumers in patch 1.
> > 
> > This is neat and makes sense to me.
> 
> Thanks for taking a look!
> 
> > However the ordering of the patches
> > seems odd. If modules depend on kunit module, then shouldn't that go
> > first? Ie, we want this to be bisectable in proper order.
> > 
> 
> The reasoning here is it seemed a more likely scenario that users mught  
> build kunit built-in (CONFIG_KUNIT=y) along with test suites built as 
> modules (CONFIG_KUNIT_TEST=m). So the intermediate state after patch 2 - 
> tests buildable as modules while kunit is still built-in-only - made more 
> sense to me as something users might do in practice so that's why I 
> ordered things that way.  I'm working on a new revision of the patchset
> though, so if you feel strongly about this shout and I'll try and accommodate
> the alternative ordering.

No, that makes sense. All good.

  Luis