[RFC,01/19] kbuild: Fixes to rules for host-cshlib and host-cxxshlib
diff mbox series

Message ID be2c361eac49ded2848b2a555b75e30cc3c24e71.1565676440.git-series.knut.omang@oracle.com
State New
Headers show
Series
  • Integration of Kernel Test Framework (KTF) into the kernel tree
Related show

Commit Message

Knut Omang Aug. 13, 2019, 6:09 a.m. UTC
C++ libraries interfacing to C APIs might sometimes need some glue
logic more easily written in C.
Allow a C++ library to also contain 0 or more C objects.

Also fix rules for both C and C++ shared libraries:
- C++ shared libraries depended on .c instead of .cc files
- Rules were referenced as -objs instead of the intended
  -cobjs and -cxxobjs following the pattern from hostprogs*.

Signed-off-by: Knut Omang <knut.omang@oracle.com>
---
 scripts/Makefile.host | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

Comments

Masahiro Yamada Aug. 13, 2019, 2:01 p.m. UTC | #1
On Tue, Aug 13, 2019 at 3:13 PM Knut Omang <knut.omang@oracle.com> wrote:
>
> C++ libraries interfacing to C APIs might sometimes need some glue
> logic more easily written in C.
> Allow a C++ library to also contain 0 or more C objects.
>
> Also fix rules for both C and C++ shared libraries:
> - C++ shared libraries depended on .c instead of .cc files
> - Rules were referenced as -objs instead of the intended
>   -cobjs and -cxxobjs following the pattern from hostprogs*.
>
> Signed-off-by: Knut Omang <knut.omang@oracle.com>


How is this patch related to the rest of this series?


This patch breaks GCC-plugins.
Did you really compile-test this patch before the submission?



--
Best Regards

Masahiro Yamada
Knut Omang Aug. 13, 2019, 4:19 p.m. UTC | #2
On Tue, 2019-08-13 at 23:01 +0900, Masahiro Yamada wrote:
> On Tue, Aug 13, 2019 at 3:13 PM Knut Omang <knut.omang@oracle.com> wrote:
> >
> > C++ libraries interfacing to C APIs might sometimes need some glue
> > logic more easily written in C.
> > Allow a C++ library to also contain 0 or more C objects.
> >
> > Also fix rules for both C and C++ shared libraries:
> > - C++ shared libraries depended on .c instead of .cc files
> > - Rules were referenced as -objs instead of the intended
> >   -cobjs and -cxxobjs following the pattern from hostprogs*.
> >
> > Signed-off-by: Knut Omang <knut.omang@oracle.com>
> 
> 
> How is this patch related to the rest of this series?

This is just my (likely naive) way I to get what I had working 
using autotools in the Github version of KTF) translated into something 
comparable using kbuild only. We need to build a shared library consisting 
of a few C++ files and a very simple C file, and a couple of simple binaries, 
and the rule in there does seem to take .c files and subject them to the 
C++ compiler, which makes this difficult to achieve?

> This patch breaks GCC-plugins.
> Did you really compile-test this patch before the submission?

Sorry for my ignorance here:
I ran through the kernel build and installed the resulting kernel 
on a VM that I used to test this, if that's what you are asking 
about?

Do I need some unusual .config options or run a special make target 
to trigger the problem you see?

I used a recent Fedora config with default values for new options,
and ran the normal default make target (also with O=) and make selftests 
to test the patch itself.

Thanks,
Knut

> --
> Best Regards
> 
> Masahiro Yamada
Masahiro Yamada Aug. 14, 2019, 2:02 a.m. UTC | #3
Hi Knut,

On Wed, Aug 14, 2019 at 1:19 AM Knut Omang <knut.omang@oracle.com> wrote:
>
> On Tue, 2019-08-13 at 23:01 +0900, Masahiro Yamada wrote:
> > On Tue, Aug 13, 2019 at 3:13 PM Knut Omang <knut.omang@oracle.com> wrote:
> > >
> > > C++ libraries interfacing to C APIs might sometimes need some glue
> > > logic more easily written in C.
> > > Allow a C++ library to also contain 0 or more C objects.
> > >
> > > Also fix rules for both C and C++ shared libraries:
> > > - C++ shared libraries depended on .c instead of .cc files
> > > - Rules were referenced as -objs instead of the intended
> > >   -cobjs and -cxxobjs following the pattern from hostprogs*.
> > >
> > > Signed-off-by: Knut Omang <knut.omang@oracle.com>
> >
> >
> > How is this patch related to the rest of this series?
>
> This is just my (likely naive) way I to get what I had working
> using autotools in the Github version of KTF) translated into something
> comparable using kbuild only. We need to build a shared library consisting
> of a few C++ files and a very simple C file, and a couple of simple binaries,
> and the rule in there does seem to take .c files and subject them to the
> C++ compiler, which makes this difficult to achieve?


Looking at the diff stat of the cover-letter,
the rest of this patch series is touching only
Documentation/ and tools/testing/kselftests/.

So, this one is unused by the rest of the changes, isn't it?
Am I missing something?



> > This patch breaks GCC-plugins.
> > Did you really compile-test this patch before the submission?
>
> Sorry for my ignorance here:
> I ran through the kernel build and installed the resulting kernel
> on a VM that I used to test this, if that's what you are asking
> about?
>
> Do I need some unusual .config options or run a special make target
> to trigger the problem you see?
>
> I used a recent Fedora config with default values for new options,
> and ran the normal default make target (also with O=) and make selftests
> to test the patch itself.


I just built allmodconfig for arm.

(The 0-day bot tests allmodconfig for most of architectures,
so you may receive error reports anyway.)


With your patch, I got the following:


masahiro@grover:~/ref/linux$ make  ARCH=arm
CROSS_COMPILE=arm-linux-gnueabihf-  allmodconfig all
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.h
  HOSTCC  scripts/kconfig/lexer.lex.o
  YACC    scripts/kconfig/parser.tab.c
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTLD  scripts/kconfig/conf
scripts/kconfig/conf  --allmodconfig Kconfig
#
# configuration written to .config
#
  SYSHDR  arch/arm/include/generated/uapi/asm/unistd-common.h
  SYSHDR  arch/arm/include/generated/uapi/asm/unistd-oabi.h
  SYSHDR  arch/arm/include/generated/uapi/asm/unistd-eabi.h
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.h
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  YACC    scripts/dtc/dtc-parser.tab.c
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CC      scripts/gcc-plugins/latent_entropy_plugin.o
cc1: error: cannot load plugin ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so
   ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: cannot open
shared object file: No such file or directory
cc1: error: cannot load plugin ./scripts/gcc-plugins/structleak_plugin.so
   ./scripts/gcc-plugins/structleak_plugin.so: cannot open shared
object file: No such file or directory
cc1: error: cannot load plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
   ./scripts/gcc-plugins/latent_entropy_plugin.so: cannot open shared
object file: No such file or directory
cc1: error: cannot load plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
   ./scripts/gcc-plugins/randomize_layout_plugin.so: cannot open
shared object file: No such file or directory
make[3]: *** [scripts/Makefile.build;281:
scripts/gcc-plugins/latent_entropy_plugin.o] Error 1
make[2]: *** [scripts/Makefile.build;497: scripts/gcc-plugins] Error 2
make[1]: *** [Makefile;1097: scripts] Error 2
make: *** [Makefile;330: __build_one_by_one] Error 2
Knut Omang Aug. 14, 2019, 5:50 a.m. UTC | #4
On Wed, 2019-08-14 at 11:02 +0900, Masahiro Yamada wrote:
> Hi Knut,
> 
> On Wed, Aug 14, 2019 at 1:19 AM Knut Omang <knut.omang@oracle.com> wrote:
> > On Tue, 2019-08-13 at 23:01 +0900, Masahiro Yamada wrote:
> > > On Tue, Aug 13, 2019 at 3:13 PM Knut Omang <knut.omang@oracle.com> wrote:
> > > > C++ libraries interfacing to C APIs might sometimes need some glue
> > > > logic more easily written in C.
> > > > Allow a C++ library to also contain 0 or more C objects.
> > > > 
> > > > Also fix rules for both C and C++ shared libraries:
> > > > - C++ shared libraries depended on .c instead of .cc files
> > > > - Rules were referenced as -objs instead of the intended
> > > >   -cobjs and -cxxobjs following the pattern from hostprogs*.
> > > > 
> > > > Signed-off-by: Knut Omang <knut.omang@oracle.com>
> > > 
> > > How is this patch related to the rest of this series?
> > 
> > This is just my (likely naive) way I to get what I had working
> > using autotools in the Github version of KTF) translated into something
> > comparable using kbuild only. We need to build a shared library consisting
> > of a few C++ files and a very simple C file, and a couple of simple binaries,
> > and the rule in there does seem to take .c files and subject them to the
> > C++ compiler, which makes this difficult to achieve?
> 
> Looking at the diff stat of the cover-letter,
> the rest of this patch series is touching only
> Documentation/ and tools/testing/kselftests/.
> 
> So, this one is unused by the rest of the changes, isn't it?
> Am I missing something?

What I am trying to do is to build kernel **and** user mode components 
under tools/testing/selftests/ktf and to keep the kernel test code out 
of the tree. The only way I was able to do this was by means of the module build,
eg. 'make TARGETS="ktf" kselftest' will cause a toplevel 'make M=tools/testing/selftests/ktf'
which is what builds the lot. One can argue that this is a misuse of that mechanism,
but as I argue in the cover letter, I think there is a need to make it possible 
to have kernel code that is not going to be part of the installed kernel 
somewhere else than in the normal build path of the kernel, which is what 
I am trying to achieve.

The toplevel Makefile logic for this is in patch 17 of the series in 
tools/testing/selftests/ktf/Makefile
and for ktf/kernel/Makefile and ktf/lib/Makefile and ktf/user/Makefile 
in patches 2, 12 and 16 respectively.

It is a bit of a hack still, since the hostcxxprogs-y and hostcxxlib-y
targets in ktf/user and ktf/lib do not get invoked "automatically" 
from __build like the obj-m targets. I haven't been able to figure out 
yet how to achieve that, and was hoping I could get help on it :-)

Thanks!
Knut

> > > This patch breaks GCC-plugins.
> > > Did you really compile-test this patch before the submission?
> > 
> > Sorry for my ignorance here:
> > I ran through the kernel build and installed the resulting kernel
> > on a VM that I used to test this, if that's what you are asking
> > about?
> > 
> > Do I need some unusual .config options or run a special make target
> > to trigger the problem you see?
> > 
> > I used a recent Fedora config with default values for new options,
> > and ran the normal default make target (also with O=) and make selftests
> > to test the patch itself.
> 
> I just built allmodconfig for arm.
> 
> (The 0-day bot tests allmodconfig for most of architectures,
> so you may receive error reports anyway.)
> 
> 
> With your patch, I got the following:
> 
> 
> masahiro@grover:~/ref/linux$ make  ARCH=arm
> CROSS_COMPILE=arm-linux-gnueabihf-  allmodconfig all
>   HOSTCC  scripts/basic/fixdep
>   HOSTCC  scripts/kconfig/conf.o
>   HOSTCC  scripts/kconfig/confdata.o
>   HOSTCC  scripts/kconfig/expr.o
>   LEX     scripts/kconfig/lexer.lex.c
>   YACC    scripts/kconfig/parser.tab.h
>   HOSTCC  scripts/kconfig/lexer.lex.o
>   YACC    scripts/kconfig/parser.tab.c
>   HOSTCC  scripts/kconfig/parser.tab.o
>   HOSTCC  scripts/kconfig/preprocess.o
>   HOSTCC  scripts/kconfig/symbol.o
>   HOSTLD  scripts/kconfig/conf
> scripts/kconfig/conf  --allmodconfig Kconfig
> #
> # configuration written to .config
> #
>   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-common.h
>   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-oabi.h
>   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-eabi.h
>   HOSTCC  scripts/dtc/dtc.o
>   HOSTCC  scripts/dtc/flattree.o
>   HOSTCC  scripts/dtc/fstree.o
>   HOSTCC  scripts/dtc/data.o
>   HOSTCC  scripts/dtc/livetree.o
>   HOSTCC  scripts/dtc/treesource.o
>   HOSTCC  scripts/dtc/srcpos.o
>   HOSTCC  scripts/dtc/checks.o
>   HOSTCC  scripts/dtc/util.o
>   LEX     scripts/dtc/dtc-lexer.lex.c
>   YACC    scripts/dtc/dtc-parser.tab.h
>   HOSTCC  scripts/dtc/dtc-lexer.lex.o
>   YACC    scripts/dtc/dtc-parser.tab.c
>   HOSTCC  scripts/dtc/dtc-parser.tab.o
>   HOSTCC  scripts/dtc/yamltree.o
>   HOSTLD  scripts/dtc/dtc
>   CC      scripts/gcc-plugins/latent_entropy_plugin.o
> cc1: error: cannot load plugin ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so
>    ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: cannot open
> shared object file: No such file or directory
> cc1: error: cannot load plugin ./scripts/gcc-plugins/structleak_plugin.so
>    ./scripts/gcc-plugins/structleak_plugin.so: cannot open shared
> object file: No such file or directory
> cc1: error: cannot load plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
>    ./scripts/gcc-plugins/latent_entropy_plugin.so: cannot open shared
> object file: No such file or directory
> cc1: error: cannot load plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
>    ./scripts/gcc-plugins/randomize_layout_plugin.so: cannot open
> shared object file: No such file or directory
> make[3]: *** [scripts/Makefile.build;281:
> scripts/gcc-plugins/latent_entropy_plugin.o] Error 1
> make[2]: *** [scripts/Makefile.build;497: scripts/gcc-plugins] Error 2
> make[1]: *** [Makefile;1097: scripts] Error 2
> make: *** [Makefile;330: __build_one_by_one] Error 2
> 
> 
> 
> 
> 
> 
>
Knut Omang Aug. 14, 2019, 5:52 a.m. UTC | #5
On Wed, 2019-08-14 at 11:02 +0900, Masahiro Yamada wrote:
> Hi Knut,
> 
> On Wed, Aug 14, 2019 at 1:19 AM Knut Omang <knut.omang@oracle.com> wrote:
> > On Tue, 2019-08-13 at 23:01 +0900, Masahiro Yamada wrote:
> > > On Tue, Aug 13, 2019 at 3:13 PM Knut Omang <knut.omang@oracle.com> wrote:
> > > > C++ libraries interfacing to C APIs might sometimes need some glue
> > > > logic more easily written in C.
> > > > Allow a C++ library to also contain 0 or more C objects.
> > > > 
> > > > Also fix rules for both C and C++ shared libraries:
> > > > - C++ shared libraries depended on .c instead of .cc files
> > > > - Rules were referenced as -objs instead of the intended
> > > >   -cobjs and -cxxobjs following the pattern from hostprogs*.
> > > > 
> > > > Signed-off-by: Knut Omang <knut.omang@oracle.com>
> > > 
> > > How is this patch related to the rest of this series?
> > 
> > This is just my (likely naive) way I to get what I had working
> > using autotools in the Github version of KTF) translated into something
> > comparable using kbuild only. We need to build a shared library consisting
> > of a few C++ files and a very simple C file, and a couple of simple binaries,
> > and the rule in there does seem to take .c files and subject them to the
> > C++ compiler, which makes this difficult to achieve?
> 
> Looking at the diff stat of the cover-letter,
> the rest of this patch series is touching only
> Documentation/ and tools/testing/kselftests/.
> 
> So, this one is unused by the rest of the changes, isn't it?
> Am I missing something?
> 
> 
> 
> > > This patch breaks GCC-plugins.
> > > Did you really compile-test this patch before the submission?
> > 
> > Sorry for my ignorance here:
> > I ran through the kernel build and installed the resulting kernel
> > on a VM that I used to test this, if that's what you are asking
> > about?
> > 
> > Do I need some unusual .config options or run a special make target
> > to trigger the problem you see?
> > 
> > I used a recent Fedora config with default values for new options,
> > and ran the normal default make target (also with O=) and make selftests
> > to test the patch itself.
> 
> I just built allmodconfig for arm.
> 
> (The 0-day bot tests allmodconfig for most of architectures,
> so you may receive error reports anyway.)
> 
> 
> With your patch, I got the following:
> 
> 
> masahiro@grover:~/ref/linux$ make  ARCH=arm
> CROSS_COMPILE=arm-linux-gnueabihf-  allmodconfig all
>   HOSTCC  scripts/basic/fixdep
>   HOSTCC  scripts/kconfig/conf.o
>   HOSTCC  scripts/kconfig/confdata.o
>   HOSTCC  scripts/kconfig/expr.o
>   LEX     scripts/kconfig/lexer.lex.c
>   YACC    scripts/kconfig/parser.tab.h
>   HOSTCC  scripts/kconfig/lexer.lex.o
>   YACC    scripts/kconfig/parser.tab.c
>   HOSTCC  scripts/kconfig/parser.tab.o
>   HOSTCC  scripts/kconfig/preprocess.o
>   HOSTCC  scripts/kconfig/symbol.o
>   HOSTLD  scripts/kconfig/conf
> scripts/kconfig/conf  --allmodconfig Kconfig
> #
> # configuration written to .config
> #
>   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-common.h
>   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-oabi.h
>   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-eabi.h
>   HOSTCC  scripts/dtc/dtc.o
>   HOSTCC  scripts/dtc/flattree.o
>   HOSTCC  scripts/dtc/fstree.o
>   HOSTCC  scripts/dtc/data.o
>   HOSTCC  scripts/dtc/livetree.o
>   HOSTCC  scripts/dtc/treesource.o
>   HOSTCC  scripts/dtc/srcpos.o
>   HOSTCC  scripts/dtc/checks.o
>   HOSTCC  scripts/dtc/util.o
>   LEX     scripts/dtc/dtc-lexer.lex.c
>   YACC    scripts/dtc/dtc-parser.tab.h
>   HOSTCC  scripts/dtc/dtc-lexer.lex.o
>   YACC    scripts/dtc/dtc-parser.tab.c
>   HOSTCC  scripts/dtc/dtc-parser.tab.o
>   HOSTCC  scripts/dtc/yamltree.o
>   HOSTLD  scripts/dtc/dtc
>   CC      scripts/gcc-plugins/latent_entropy_plugin.o
> cc1: error: cannot load plugin ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so
>    ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: cannot open
> shared object file: No such file or directory
> cc1: error: cannot load plugin ./scripts/gcc-plugins/structleak_plugin.so
>    ./scripts/gcc-plugins/structleak_plugin.so: cannot open shared
> object file: No such file or directory
> cc1: error: cannot load plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
>    ./scripts/gcc-plugins/latent_entropy_plugin.so: cannot open shared
> object file: No such file or directory
> cc1: error: cannot load plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
>    ./scripts/gcc-plugins/randomize_layout_plugin.so: cannot open
> shared object file: No such file or directory
> make[3]: *** [scripts/Makefile.build;281:
> scripts/gcc-plugins/latent_entropy_plugin.o] Error 1
> make[2]: *** [scripts/Makefile.build;497: scripts/gcc-plugins] Error 2
> make[1]: *** [Makefile;1097: scripts] Error 2
> make: *** [Makefile;330: __build_one_by_one] Error 2

Ok, I see!

I'll recall this target and look into it!

Thanks!
Knut

> 
> 
> 
> 
> 
> 
>
Knut Omang Aug. 14, 2019, 12:52 p.m. UTC | #6
On Wed, 2019-08-14 at 07:52 +0200, Knut Omang wrote:
> On Wed, 2019-08-14 at 11:02 +0900, Masahiro Yamada wrote:
> > Hi Knut,
> > 
> > On Wed, Aug 14, 2019 at 1:19 AM Knut Omang <knut.omang@oracle.com> wrote:
> > > On Tue, 2019-08-13 at 23:01 +0900, Masahiro Yamada wrote:
> > > > On Tue, Aug 13, 2019 at 3:13 PM Knut Omang <knut.omang@oracle.com> wrote:
> > > > > C++ libraries interfacing to C APIs might sometimes need some glue
> > > > > logic more easily written in C.
> > > > > Allow a C++ library to also contain 0 or more C objects.
> > > > > 
> > > > > Also fix rules for both C and C++ shared libraries:
> > > > > - C++ shared libraries depended on .c instead of .cc files
> > > > > - Rules were referenced as -objs instead of the intended
> > > > >   -cobjs and -cxxobjs following the pattern from hostprogs*.
> > > > > 
> > > > > Signed-off-by: Knut Omang <knut.omang@oracle.com>
> > > > 
> > > > How is this patch related to the rest of this series?
> > > 
> > > This is just my (likely naive) way I to get what I had working
> > > using autotools in the Github version of KTF) translated into something
> > > comparable using kbuild only. We need to build a shared library consisting
> > > of a few C++ files and a very simple C file, and a couple of simple binaries,
> > > and the rule in there does seem to take .c files and subject them to the
> > > C++ compiler, which makes this difficult to achieve?
> > 
> > Looking at the diff stat of the cover-letter,
> > the rest of this patch series is touching only
> > Documentation/ and tools/testing/kselftests/.
> > 
> > So, this one is unused by the rest of the changes, isn't it?
> > Am I missing something?
> > 
> > 
> > 
> > > > This patch breaks GCC-plugins.
> > > > Did you really compile-test this patch before the submission?
> > > 
> > > Sorry for my ignorance here:
> > > I ran through the kernel build and installed the resulting kernel
> > > on a VM that I used to test this, if that's what you are asking
> > > about?
> > > 
> > > Do I need some unusual .config options or run a special make target
> > > to trigger the problem you see?
> > > 
> > > I used a recent Fedora config with default values for new options,
> > > and ran the normal default make target (also with O=) and make selftests
> > > to test the patch itself.
> > 
> > I just built allmodconfig for arm.
> > 
> > (The 0-day bot tests allmodconfig for most of architectures,
> > so you may receive error reports anyway.)
> > 
> > 
> > With your patch, I got the following:
> > 
> > 
> > masahiro@grover:~/ref/linux$ make  ARCH=arm
> > CROSS_COMPILE=-  allmodconfig all
> >   HOSTCC  scripts/basic/fixdep
> >   HOSTCC  scripts/kconfig/conf.o
> >   HOSTCC  scripts/kconfig/confdata.o
> >   HOSTCC  scripts/kconfig/expr.o
> >   LEX     scripts/kconfig/lexer.lex.c
> >   YACC    scripts/kconfig/parser.tab.h
> >   HOSTCC  scripts/kconfig/lexer.lex.o
> >   YACC    scripts/kconfig/parser.tab.c
> >   HOSTCC  scripts/kconfig/parser.tab.o
> >   HOSTCC  scripts/kconfig/preprocess.o
> >   HOSTCC  scripts/kconfig/symbol.o
> >   HOSTLD  scripts/kconfig/conf
> > scripts/kconfig/conf  --allmodconfig Kconfig
> > #
> > # configuration written to .config
> > #
> >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-common.h
> >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-oabi.h
> >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-eabi.h
> >   HOSTCC  scripts/dtc/dtc.o
> >   HOSTCC  scripts/dtc/flattree.o
> >   HOSTCC  scripts/dtc/fstree.o
> >   HOSTCC  scripts/dtc/data.o
> >   HOSTCC  scripts/dtc/livetree.o
> >   HOSTCC  scripts/dtc/treesource.o
> >   HOSTCC  scripts/dtc/srcpos.o
> >   HOSTCC  scripts/dtc/checks.o
> >   HOSTCC  scripts/dtc/util.o
> >   LEX     scripts/dtc/dtc-lexer.lex.c
> >   YACC    scripts/dtc/dtc-parser.tab.h
> >   HOSTCC  scripts/dtc/dtc-lexer.lex.o
> >   YACC    scripts/dtc/dtc-parser.tab.c
> >   HOSTCC  scripts/dtc/dtc-parser.tab.o
> >   HOSTCC  scripts/dtc/yamltree.o
> >   HOSTLD  scripts/dtc/dtc
> >   CC      scripts/gcc-plugins/latent_entropy_plugin.o
> > cc1: error: cannot load plugin ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so
> >    ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: cannot open
> > shared object file: No such file or directory
> > cc1: error: cannot load plugin ./scripts/gcc-plugins/structleak_plugin.so
> >    ./scripts/gcc-plugins/structleak_plugin.so: cannot open shared
> > object file: No such file or directory
> > cc1: error: cannot load plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
> >    ./scripts/gcc-plugins/latent_entropy_plugin.so: cannot open shared
> > object file: No such file or directory
> > cc1: error: cannot load plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
> >    ./scripts/gcc-plugins/randomize_layout_plugin.so: cannot open
> > shared object file: No such file or directory
> > make[3]: *** [scripts/Makefile.build;281:
> > scripts/gcc-plugins/latent_entropy_plugin.o] Error 1
> > make[2]: *** [scripts/Makefile.build;497: scripts/gcc-plugins] Error 2
> > make[1]: *** [Makefile;1097: scripts] Error 2
> > make: *** [Makefile;330: __build_one_by_one] Error 2
> 
> Ok, I see!
> 
> I'll recall this target and look into it!

Ok, so I have tried installing the arm-linux-gnueabihf cross compiler and compiled the kernel for arm,
but allmodconfig does not seem to enable any GCC plugins per default even on ARM and I haven't been able
to figure out how to enable any. 

A plain allmodconfig generated config compiles perfectly for me both native x86 and w/arm cross compile,
but it doesn't seem to enable any gcc plugins.

Anyway, maybe I am getting this wrong anyway: 
Having played with cross compile, it starts to become clear to me that HOSTCC rules
might not be the right rules to use, as it will generate host user land binaries as opposed to 
target user land binaries (in a cross compile world obviously these differ)

Now, I started off with using the rules in the selftests makefiles for this, but they do not play that well with 
kernel module building. My goal is to be able to do both user land and kernel module **target** compiles 
from the same subtree. Any hints on how to accomplish this appreciated :-)

Thanks,
Knut
Masahiro Yamada Aug. 21, 2019, 1:47 a.m. UTC | #7
On Wed, Aug 14, 2019 at 9:53 PM Knut Omang <knut.omang@oracle.com> wrote:
>
> On Wed, 2019-08-14 at 07:52 +0200, Knut Omang wrote:
> > On Wed, 2019-08-14 at 11:02 +0900, Masahiro Yamada wrote:
> > > Hi Knut,
> > >
> > > On Wed, Aug 14, 2019 at 1:19 AM Knut Omang <knut.omang@oracle.com> wrote:
> > > > On Tue, 2019-08-13 at 23:01 +0900, Masahiro Yamada wrote:
> > > > > On Tue, Aug 13, 2019 at 3:13 PM Knut Omang <knut.omang@oracle.com> wrote:
> > > > > > C++ libraries interfacing to C APIs might sometimes need some glue
> > > > > > logic more easily written in C.
> > > > > > Allow a C++ library to also contain 0 or more C objects.
> > > > > >
> > > > > > Also fix rules for both C and C++ shared libraries:
> > > > > > - C++ shared libraries depended on .c instead of .cc files
> > > > > > - Rules were referenced as -objs instead of the intended
> > > > > >   -cobjs and -cxxobjs following the pattern from hostprogs*.
> > > > > >
> > > > > > Signed-off-by: Knut Omang <knut.omang@oracle.com>
> > > > >
> > > > > How is this patch related to the rest of this series?
> > > >
> > > > This is just my (likely naive) way I to get what I had working
> > > > using autotools in the Github version of KTF) translated into something
> > > > comparable using kbuild only. We need to build a shared library consisting
> > > > of a few C++ files and a very simple C file, and a couple of simple binaries,
> > > > and the rule in there does seem to take .c files and subject them to the
> > > > C++ compiler, which makes this difficult to achieve?
> > >
> > > Looking at the diff stat of the cover-letter,
> > > the rest of this patch series is touching only
> > > Documentation/ and tools/testing/kselftests/.
> > >
> > > So, this one is unused by the rest of the changes, isn't it?
> > > Am I missing something?
> > >
> > >
> > >
> > > > > This patch breaks GCC-plugins.
> > > > > Did you really compile-test this patch before the submission?
> > > >
> > > > Sorry for my ignorance here:
> > > > I ran through the kernel build and installed the resulting kernel
> > > > on a VM that I used to test this, if that's what you are asking
> > > > about?
> > > >
> > > > Do I need some unusual .config options or run a special make target
> > > > to trigger the problem you see?
> > > >
> > > > I used a recent Fedora config with default values for new options,
> > > > and ran the normal default make target (also with O=) and make selftests
> > > > to test the patch itself.
> > >
> > > I just built allmodconfig for arm.
> > >
> > > (The 0-day bot tests allmodconfig for most of architectures,
> > > so you may receive error reports anyway.)
> > >
> > >
> > > With your patch, I got the following:
> > >
> > >
> > > masahiro@grover:~/ref/linux$ make  ARCH=arm
> > > CROSS_COMPILE=-  allmodconfig all
> > >   HOSTCC  scripts/basic/fixdep
> > >   HOSTCC  scripts/kconfig/conf.o
> > >   HOSTCC  scripts/kconfig/confdata.o
> > >   HOSTCC  scripts/kconfig/expr.o
> > >   LEX     scripts/kconfig/lexer.lex.c
> > >   YACC    scripts/kconfig/parser.tab.h
> > >   HOSTCC  scripts/kconfig/lexer.lex.o
> > >   YACC    scripts/kconfig/parser.tab.c
> > >   HOSTCC  scripts/kconfig/parser.tab.o
> > >   HOSTCC  scripts/kconfig/preprocess.o
> > >   HOSTCC  scripts/kconfig/symbol.o
> > >   HOSTLD  scripts/kconfig/conf
> > > scripts/kconfig/conf  --allmodconfig Kconfig
> > > #
> > > # configuration written to .config
> > > #
> > >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-common.h
> > >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-oabi.h
> > >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-eabi.h
> > >   HOSTCC  scripts/dtc/dtc.o
> > >   HOSTCC  scripts/dtc/flattree.o
> > >   HOSTCC  scripts/dtc/fstree.o
> > >   HOSTCC  scripts/dtc/data.o
> > >   HOSTCC  scripts/dtc/livetree.o
> > >   HOSTCC  scripts/dtc/treesource.o
> > >   HOSTCC  scripts/dtc/srcpos.o
> > >   HOSTCC  scripts/dtc/checks.o
> > >   HOSTCC  scripts/dtc/util.o
> > >   LEX     scripts/dtc/dtc-lexer.lex.c
> > >   YACC    scripts/dtc/dtc-parser.tab.h
> > >   HOSTCC  scripts/dtc/dtc-lexer.lex.o
> > >   YACC    scripts/dtc/dtc-parser.tab.c
> > >   HOSTCC  scripts/dtc/dtc-parser.tab.o
> > >   HOSTCC  scripts/dtc/yamltree.o
> > >   HOSTLD  scripts/dtc/dtc
> > >   CC      scripts/gcc-plugins/latent_entropy_plugin.o
> > > cc1: error: cannot load plugin ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so
> > >    ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: cannot open
> > > shared object file: No such file or directory
> > > cc1: error: cannot load plugin ./scripts/gcc-plugins/structleak_plugin.so
> > >    ./scripts/gcc-plugins/structleak_plugin.so: cannot open shared
> > > object file: No such file or directory
> > > cc1: error: cannot load plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
> > >    ./scripts/gcc-plugins/latent_entropy_plugin.so: cannot open shared
> > > object file: No such file or directory
> > > cc1: error: cannot load plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
> > >    ./scripts/gcc-plugins/randomize_layout_plugin.so: cannot open
> > > shared object file: No such file or directory
> > > make[3]: *** [scripts/Makefile.build;281:
> > > scripts/gcc-plugins/latent_entropy_plugin.o] Error 1
> > > make[2]: *** [scripts/Makefile.build;497: scripts/gcc-plugins] Error 2
> > > make[1]: *** [Makefile;1097: scripts] Error 2
> > > make: *** [Makefile;330: __build_one_by_one] Error 2
> >
> > Ok, I see!
> >
> > I'll recall this target and look into it!
>
> Ok, so I have tried installing the arm-linux-gnueabihf cross compiler and compiled the kernel for arm,
> but allmodconfig does not seem to enable any GCC plugins per default even on ARM and I haven't been able
> to figure out how to enable any.


Linaro toolchain supports gcc plugins.

https://releases.linaro.org/components/toolchain/binaries/latest-7/arm-linux-gnueabihf/


kernel.org one supports it as well.

https://mirrors.edge.kernel.org/pub/tools/crosstool/





> A plain allmodconfig generated config compiles perfectly for me both native x86 and w/arm cross compile,
> but it doesn't seem to enable any gcc plugins.
>
> Anyway, maybe I am getting this wrong anyway:
> Having played with cross compile, it starts to become clear to me that HOSTCC rules
> might not be the right rules to use, as it will generate host user land binaries as opposed to
> target user land binaries (in a cross compile world obviously these differ)
>
> Now, I started off with using the rules in the selftests makefiles for this, but they do not play that well with
> kernel module building. My goal is to be able to do both user land and kernel module **target** compiles
> from the same subtree. Any hints on how to accomplish this appreciated :-)


tools/ is out of scope of kbuild because it adopted a different (more
adhoc) build system.
I have no idea. Please talk to the kselftest maintainer.



>
> Thanks,
> Knut
>
Knut Omang Aug. 21, 2019, 4:03 a.m. UTC | #8
On Wed, 2019-08-21 at 10:47 +0900, Masahiro Yamada wrote:
> On Wed, Aug 14, 2019 at 9:53 PM Knut Omang <knut.omang@oracle.com> wrote:
> >
> > On Wed, 2019-08-14 at 07:52 +0200, Knut Omang wrote:
> > > On Wed, 2019-08-14 at 11:02 +0900, Masahiro Yamada wrote:
> > > > Hi Knut,
> > > >
> > > > On Wed, Aug 14, 2019 at 1:19 AM Knut Omang <knut.omang@oracle.com> wrote:
> > > > > On Tue, 2019-08-13 at 23:01 +0900, Masahiro Yamada wrote:
> > > > > > On Tue, Aug 13, 2019 at 3:13 PM Knut Omang <knut.omang@oracle.com> wrote:
> > > > > > > C++ libraries interfacing to C APIs might sometimes need some glue
> > > > > > > logic more easily written in C.
> > > > > > > Allow a C++ library to also contain 0 or more C objects.
> > > > > > >
> > > > > > > Also fix rules for both C and C++ shared libraries:
> > > > > > > - C++ shared libraries depended on .c instead of .cc files
> > > > > > > - Rules were referenced as -objs instead of the intended
> > > > > > >   -cobjs and -cxxobjs following the pattern from hostprogs*.
> > > > > > >
> > > > > > > Signed-off-by: Knut Omang <knut.omang@oracle.com>
> > > > > >
> > > > > > How is this patch related to the rest of this series?
> > > > >
> > > > > This is just my (likely naive) way I to get what I had working
> > > > > using autotools in the Github version of KTF) translated into something
> > > > > comparable using kbuild only. We need to build a shared library consisting
> > > > > of a few C++ files and a very simple C file, and a couple of simple binaries,
> > > > > and the rule in there does seem to take .c files and subject them to the
> > > > > C++ compiler, which makes this difficult to achieve?
> > > >
> > > > Looking at the diff stat of the cover-letter,
> > > > the rest of this patch series is touching only
> > > > Documentation/ and tools/testing/kselftests/.
> > > >
> > > > So, this one is unused by the rest of the changes, isn't it?
> > > > Am I missing something?
> > > >
> > > >
> > > >
> > > > > > This patch breaks GCC-plugins.
> > > > > > Did you really compile-test this patch before the submission?
> > > > >
> > > > > Sorry for my ignorance here:
> > > > > I ran through the kernel build and installed the resulting kernel
> > > > > on a VM that I used to test this, if that's what you are asking
> > > > > about?
> > > > >
> > > > > Do I need some unusual .config options or run a special make target
> > > > > to trigger the problem you see?
> > > > >
> > > > > I used a recent Fedora config with default values for new options,
> > > > > and ran the normal default make target (also with O=) and make selftests
> > > > > to test the patch itself.
> > > >
> > > > I just built allmodconfig for arm.
> > > >
> > > > (The 0-day bot tests allmodconfig for most of architectures,
> > > > so you may receive error reports anyway.)
> > > >
> > > >
> > > > With your patch, I got the following:
> > > >
> > > >
> > > > masahiro@grover:~/ref/linux$ make  ARCH=arm
> > > > CROSS_COMPILE=-  allmodconfig all
> > > >   HOSTCC  scripts/basic/fixdep
> > > >   HOSTCC  scripts/kconfig/conf.o
> > > >   HOSTCC  scripts/kconfig/confdata.o
> > > >   HOSTCC  scripts/kconfig/expr.o
> > > >   LEX     scripts/kconfig/lexer.lex.c
> > > >   YACC    scripts/kconfig/parser.tab.h
> > > >   HOSTCC  scripts/kconfig/lexer.lex.o
> > > >   YACC    scripts/kconfig/parser.tab.c
> > > >   HOSTCC  scripts/kconfig/parser.tab.o
> > > >   HOSTCC  scripts/kconfig/preprocess.o
> > > >   HOSTCC  scripts/kconfig/symbol.o
> > > >   HOSTLD  scripts/kconfig/conf
> > > > scripts/kconfig/conf  --allmodconfig Kconfig
> > > > #
> > > > # configuration written to .config
> > > > #
> > > >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-common.h
> > > >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-oabi.h
> > > >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-eabi.h
> > > >   HOSTCC  scripts/dtc/dtc.o
> > > >   HOSTCC  scripts/dtc/flattree.o
> > > >   HOSTCC  scripts/dtc/fstree.o
> > > >   HOSTCC  scripts/dtc/data.o
> > > >   HOSTCC  scripts/dtc/livetree.o
> > > >   HOSTCC  scripts/dtc/treesource.o
> > > >   HOSTCC  scripts/dtc/srcpos.o
> > > >   HOSTCC  scripts/dtc/checks.o
> > > >   HOSTCC  scripts/dtc/util.o
> > > >   LEX     scripts/dtc/dtc-lexer.lex.c
> > > >   YACC    scripts/dtc/dtc-parser.tab.h
> > > >   HOSTCC  scripts/dtc/dtc-lexer.lex.o
> > > >   YACC    scripts/dtc/dtc-parser.tab.c
> > > >   HOSTCC  scripts/dtc/dtc-parser.tab.o
> > > >   HOSTCC  scripts/dtc/yamltree.o
> > > >   HOSTLD  scripts/dtc/dtc
> > > >   CC      scripts/gcc-plugins/latent_entropy_plugin.o
> > > > cc1: error: cannot load plugin ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so
> > > >    ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: cannot open
> > > > shared object file: No such file or directory
> > > > cc1: error: cannot load plugin ./scripts/gcc-plugins/structleak_plugin.so
> > > >    ./scripts/gcc-plugins/structleak_plugin.so: cannot open shared
> > > > object file: No such file or directory
> > > > cc1: error: cannot load plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
> > > >    ./scripts/gcc-plugins/latent_entropy_plugin.so: cannot open shared
> > > > object file: No such file or directory
> > > > cc1: error: cannot load plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
> > > >    ./scripts/gcc-plugins/randomize_layout_plugin.so: cannot open
> > > > shared object file: No such file or directory
> > > > make[3]: *** [scripts/Makefile.build;281:
> > > > scripts/gcc-plugins/latent_entropy_plugin.o] Error 1
> > > > make[2]: *** [scripts/Makefile.build;497: scripts/gcc-plugins] Error 2
> > > > make[1]: *** [Makefile;1097: scripts] Error 2
> > > > make: *** [Makefile;330: __build_one_by_one] Error 2
> > >
> > > Ok, I see!
> > >
> > > I'll recall this target and look into it!
> >
> > Ok, so I have tried installing the arm-linux-gnueabihf cross compiler and compiled the
> kernel for arm,
> > but allmodconfig does not seem to enable any GCC plugins per default even on ARM and I
> haven't been able
> > to figure out how to enable any.
> 
> 
> Linaro toolchain supports gcc plugins.
> 
> https://releases.linaro.org/components/toolchain/binaries/latest-7/arm-linux-gnueabihf/
> 
> 
> kernel.org one supports it as well.
> 
> https://mirrors.edge.kernel.org/pub/tools/crosstool/
> 
> 
> 
> 
> 
> > A plain allmodconfig generated config compiles perfectly for me both native x86 and
> w/arm cross compile,
> > but it doesn't seem to enable any gcc plugins.
> >
> > Anyway, maybe I am getting this wrong anyway:
> > Having played with cross compile, it starts to become clear to me that HOSTCC rules
> > might not be the right rules to use, as it will generate host user land binaries as
> opposed to
> > target user land binaries (in a cross compile world obviously these differ)
> >
> > Now, I started off with using the rules in the selftests makefiles for this, but they
> do not play that well with
> > kernel module building. My goal is to be able to do both user land and kernel module
> **target** compiles
> > from the same subtree. Any hints on how to accomplish this appreciated :-)
> 
> 
> tools/ is out of scope of kbuild because it adopted a different (more
> adhoc) build system.

That resonates with my understanding, thanks!
I think that's what I am trying to understand how to improve:

To sum up, my conclusion so far is that there's a need for a set of more integrated Kbuild
rules for target platform binaries and libraries similar to the host* rules. The rules in
tools/ could then be modified to use that generic support instead of the current adhoc
approach. That would allow the needs of tests that have both user space and kernel space
code to be more nicely co-located, with several benefits IMHO.

Any further insights appreciated!

> I have no idea. Please talk to the kselftest maintainer.

She's on the Cc:-list, I was hoping to get a some of the challenges/choices highlighted
here ;-)

Thanks!
Knut

Patch
diff mbox series

diff --git a/scripts/Makefile.host b/scripts/Makefile.host
index b6a54bd..4e9bb21 100644
--- a/scripts/Makefile.host
+++ b/scripts/Makefile.host
@@ -46,8 +46,10 @@  host-cxxmulti	:= $(foreach m,$(__hostprogs),$(if $($(m)-cxxobjs),$(m)))
 host-cxxobjs	:= $(sort $(foreach m,$(host-cxxmulti),$($(m)-cxxobjs)))
 
 # Object (.o) files used by the shared libaries
-host-cshobjs	:= $(sort $(foreach m,$(host-cshlib),$($(m:.so=-objs))))
-host-cxxshobjs	:= $(sort $(foreach m,$(host-cxxshlib),$($(m:.so=-objs))))
+# Note: C++ libraries may contain both C and C++ objects, compiled differently:
+host-cshobjs	:= $(sort $(foreach m,$(host-cshlib),$($(m:.so=-cshobjs))))
+host-cshobjs	+= $(sort $(foreach m,$(host-cxxshlib),$($(m:.so=-cshobjs))))
+host-cxxshobjs	:= $(sort $(foreach m,$(host-cxxshlib),$($(m:.so=-cxxshobjs))))
 
 host-csingle	:= $(addprefix $(obj)/,$(host-csingle))
 host-cmulti	:= $(addprefix $(obj)/,$(host-cmulti))
@@ -130,14 +132,10 @@  quiet_cmd_host-cshobjs	= HOSTCC  -fPIC $@
 $(host-cshobjs): $(obj)/%.o: $(src)/%.c FORCE
 	$(call if_changed_dep,host-cshobjs)
 
-# Compile .c file, create position independent .o file
-# Note that plugin capable gcc versions can be either C or C++ based
-# therefore plugin source files have to be compilable in both C and C++ mode.
-# This is why a C++ compiler is invoked on a .c file.
-# host-cxxshobjs -> .o
+# Compile .cc (C++) file, create position independent .o file
 quiet_cmd_host-cxxshobjs	= HOSTCXX -fPIC $@
       cmd_host-cxxshobjs	= $(HOSTCXX) $(hostcxx_flags) -fPIC -c -o $@ $<
-$(host-cxxshobjs): $(obj)/%.o: $(src)/%.c FORCE
+$(host-cxxshobjs): $(obj)/%.o: $(src)/%.cc FORCE
 	$(call if_changed_dep,host-cxxshobjs)
 
 # Link a shared library, based on position independent .o files
@@ -154,7 +152,8 @@  $(call multi_depend, $(host-cshlib), .so, -objs)
 # *.o -> .so shared library (host-cxxshlib)
 quiet_cmd_host-cxxshlib	= HOSTLLD -shared $@
       cmd_host-cxxshlib	= $(HOSTCXX) $(KBUILD_HOSTLDFLAGS) -shared -o $@ \
-			  $(addprefix $(obj)/,$($(@F:.so=-objs))) \
+			  $(addprefix $(obj)/,$($(@F:.so=-cshobjs))) \
+			  $(addprefix $(obj)/,$($(@F:.so=-cxxshobjs))) \
 			  $(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(@F))
 $(host-cxxshlib): FORCE
 	$(call if_changed,host-cxxshlib)