[v2,3/4] kbuild: support header-test-pattern-y
diff mbox series

Message ID 20190627014617.600-4-yamada.masahiro@socionext.com
State New
Headers show
Series
  • Compile-test UAPI and kernel headers
Related show

Commit Message

Masahiro Yamada June 27, 2019, 1:46 a.m. UTC
In my view, most of headers can be self-contained. So, it would be
tedious to add every header to header-test-y explicitly. We usually
end up with "all headers with some exceptions".

There are two types in exceptions:

[1] headers that are never compiled as standalone units

  For examples, include/linux/compiler-gcc.h is not intended to be
  included directly. We should always exclude such ones.

[2] headers that are conditionally compiled as standalone units

  Some headers can be compiled only for particular architectures.
  For example, include/linux/arm-cci.h can be compiled only for
  arm/arm64 because it requires <asm/arm-cci.h> to exist.
  Clang can compile include/soc/nps/mtm.h only for arc because
  it contains an arch-specific register in inline assembler.

For [2], we can write Makefile like this:

  header-test-$(CONFIG_ARM) += linux/arm-cci.h

The new syntax header-test-pattern-y will be useful to specify
"the rest".

The typical usage is like this:

  header-test-pattern-y += */*.h

This adds all the headers in sub-directories to the test coverage,
but headers added to header-test- are excluded. In this regards,
header-test-pattern-y behaves like a weaker variant of header-test-y.

Caveat:
The patterns in header-test-pattern-y are prefixed with $(srctree)/$(src)/
but not $(objtree)/$(obj)/. Stale generated patterns are often left over.
For example, you will have ones when you traverse the git history for
'git bisect' without cleaning. If a wildcard is used for generated
headers, it may match to stale headers.

If you really want to compile-test generated headers, I recommend to
add them to header-test-y explicitly. One pitfall is $(srctree)/$(src)/
and $(objtree)/$(obj)/ point to the same directory for in-tree building.
So, header-test-pattern-y should be used with care. It can potentially
match to generated headers, which may be stale and fail to compile.

Caveat2:
You could use wildcard for header-test-. For example,

  header-test- += asm-generic/%

... will exclude headers in asm-generic directory. Unfortunately, the
wildcard character is '%' instead of '*' because this is evaluated by
$(filter-out ...) whereas header-test-pattern-y is evaluated by
$(wildcard ...). This is a kludge, but seems useful in some places...

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v2:
  - New patch

 Documentation/kbuild/makefiles.txt | 10 ++++++++++
 scripts/Makefile.lib               | 10 ++++++++++
 2 files changed, 20 insertions(+)

Comments

Jani Nikula June 27, 2019, 11:28 a.m. UTC | #1
On Thu, 27 Jun 2019, Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
> In my view, most of headers can be self-contained. So, it would be
> tedious to add every header to header-test-y explicitly. We usually
> end up with "all headers with some exceptions".
>
> There are two types in exceptions:
>
> [1] headers that are never compiled as standalone units
>
>   For examples, include/linux/compiler-gcc.h is not intended to be
>   included directly. We should always exclude such ones.
>
> [2] headers that are conditionally compiled as standalone units
>
>   Some headers can be compiled only for particular architectures.
>   For example, include/linux/arm-cci.h can be compiled only for
>   arm/arm64 because it requires <asm/arm-cci.h> to exist.
>   Clang can compile include/soc/nps/mtm.h only for arc because
>   it contains an arch-specific register in inline assembler.
>
> For [2], we can write Makefile like this:
>
>   header-test-$(CONFIG_ARM) += linux/arm-cci.h
>
> The new syntax header-test-pattern-y will be useful to specify
> "the rest".
>
> The typical usage is like this:
>
>   header-test-pattern-y += */*.h
>
> This adds all the headers in sub-directories to the test coverage,
> but headers added to header-test- are excluded. In this regards,
> header-test-pattern-y behaves like a weaker variant of header-test-y.
>
> Caveat:
> The patterns in header-test-pattern-y are prefixed with $(srctree)/$(src)/
> but not $(objtree)/$(obj)/. Stale generated patterns are often left over.
> For example, you will have ones when you traverse the git history for
> 'git bisect' without cleaning. If a wildcard is used for generated
> headers, it may match to stale headers.
>
> If you really want to compile-test generated headers, I recommend to
> add them to header-test-y explicitly. One pitfall is $(srctree)/$(src)/
> and $(objtree)/$(obj)/ point to the same directory for in-tree building.
> So, header-test-pattern-y should be used with care. It can potentially
> match to generated headers, which may be stale and fail to compile.
>
> Caveat2:
> You could use wildcard for header-test-. For example,
>
>   header-test- += asm-generic/%
>
> ... will exclude headers in asm-generic directory. Unfortunately, the
> wildcard character is '%' instead of '*' because this is evaluated by
> $(filter-out ...) whereas header-test-pattern-y is evaluated by
> $(wildcard ...). This is a kludge, but seems useful in some places...
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Awesome! This will let us get rid of our local $(wildcard) hacks.

Tested-by: Jani Nikula <jani.nikula@intel.com>

> ---
>
> Changes in v2:
>   - New patch
>
>  Documentation/kbuild/makefiles.txt | 10 ++++++++++
>  scripts/Makefile.lib               | 10 ++++++++++
>  2 files changed, 20 insertions(+)
>
> diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
> index 5080fec34609..b817e6cefb77 100644
> --- a/Documentation/kbuild/makefiles.txt
> +++ b/Documentation/kbuild/makefiles.txt
> @@ -1025,6 +1025,16 @@ When kbuild executes, the following steps are followed (roughly):
>  	i.e. compilable as standalone units. If CONFIG_HEADER_TEST is enabled,
>  	this builds them as part of extra-y.
>  
> +    header-test-pattern-y
> +
> +	This works as a weaker version of header-test-y, and accepts wildcard
> +	patterns. The typical usage is:
> +
> +		  header-test-pattern-y += *.h
> +
> +	This specifies all the files that matches to '*.h' in the current
> +	directory, but the files in 'header-test-' are excluded.
> +
>  --- 6.7 Commands useful for building a boot image
>  
>  	Kbuild provides a few macros that are useful when building a
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 55ae1ec65342..54444933bbab 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -67,6 +67,16 @@ extra-$(CONFIG_OF_ALL_DTBS) += $(patsubst %.dtb,%.dt.yaml, $(dtb-))
>  endif
>  
>  # Test self-contained headers
> +
> +# Wildcard searches in $(srctree)/$(src)/, but not in $(objtree)/$(obj)/.
> +# Stale generated headers are often left over, so wildcard matching should
> +# be avoided. Please notice $(srctree)/$(src)/ and $(objtree)/$(obj) point
> +# to the same location for in-tree building.
> +header-test-y	+= $(filter-out $(header-test-), \
> +		$(patsubst $(srctree)/$(src)/%, %, \
> +		$(wildcard $(addprefix $(srctree)/$(src)/, \
> +		$(header-test-pattern-y)))))
> +
>  extra-$(CONFIG_HEADER_TEST) += $(addsuffix .s, $(header-test-y))
>  
>  # Add subdir path

Patch
diff mbox series

diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
index 5080fec34609..b817e6cefb77 100644
--- a/Documentation/kbuild/makefiles.txt
+++ b/Documentation/kbuild/makefiles.txt
@@ -1025,6 +1025,16 @@  When kbuild executes, the following steps are followed (roughly):
 	i.e. compilable as standalone units. If CONFIG_HEADER_TEST is enabled,
 	this builds them as part of extra-y.
 
+    header-test-pattern-y
+
+	This works as a weaker version of header-test-y, and accepts wildcard
+	patterns. The typical usage is:
+
+		  header-test-pattern-y += *.h
+
+	This specifies all the files that matches to '*.h' in the current
+	directory, but the files in 'header-test-' are excluded.
+
 --- 6.7 Commands useful for building a boot image
 
 	Kbuild provides a few macros that are useful when building a
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 55ae1ec65342..54444933bbab 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -67,6 +67,16 @@  extra-$(CONFIG_OF_ALL_DTBS) += $(patsubst %.dtb,%.dt.yaml, $(dtb-))
 endif
 
 # Test self-contained headers
+
+# Wildcard searches in $(srctree)/$(src)/, but not in $(objtree)/$(obj)/.
+# Stale generated headers are often left over, so wildcard matching should
+# be avoided. Please notice $(srctree)/$(src)/ and $(objtree)/$(obj) point
+# to the same location for in-tree building.
+header-test-y	+= $(filter-out $(header-test-), \
+		$(patsubst $(srctree)/$(src)/%, %, \
+		$(wildcard $(addprefix $(srctree)/$(src)/, \
+		$(header-test-pattern-y)))))
+
 extra-$(CONFIG_HEADER_TEST) += $(addsuffix .s, $(header-test-y))
 
 # Add subdir path