| Message ID | 20210412142905.266942-1-yhs@fb.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | bpf: tools: support build selftests/bpf with clang | expand |
On Mon, Apr 12, 2021 at 7:29 AM Yonghong Song <yhs@fb.com> wrote: > > To build kernel with clang, people typically use > make -j60 LLVM=1 LLVM_IAS=1 > LLVM_IAS=1 is not required for non-LTO build but > is required for LTO build. In my environment, > I am always having LLVM_IAS=1 regardless of > whether LTO is enabled or not. > > After kernel is build with clang, the following command > can be used to build selftests with clang: > make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 Thank you for the series Yonghong. When I test the above command with your series applied, I observe: /usr/bin/ld: cannot find -lcap clang-13: error: linker command failed with exit code 1 (use -v to see invocation) I need to install libcap-dev, but this also seems to imply that BFD is being used as the linker, not LLD. Perhaps if the compiler is being used as the "driver" to also link executables, `-fuse-ld=lld` is needed for the compiler flags. Then there's: tools/include/tools/libc_compat.h:11:21: error: static declaration of 'reallocarray' follows non-static declaration static inline void *reallocarray(void *ptr, size_t nmemb, size_t size) ^ /usr/include/stdlib.h:559:14: note: previous declaration is here extern void *reallocarray (void *__ptr, size_t __nmemb, size_t __size) ^ so perhaps the detection of COMPAT_NEED_REALLOCARRAY/feature-reallocarray is incorrect? > > But currently, using the above command, some compilations > still use gcc and there are also compilation errors and warnings. > This patch set intends to fix these issues. > Patch #1 and #2 fixed the issue so clang/clang++ is > used instead of gcc/g++. Patch #3 fixed a compilation > failure. Patch #4 and #5 fixed various compiler warnings. > > Changelog: > v1 -> v2: > . add -Wno-unused-command-line-argument and -Wno-format-security > for clang only as (1). gcc does not exhibit those > warnings, and (2). -Wno-unused-command-line-argument is > only supported by clang. (Sedat) > > Yonghong Song (5): > selftests: set CC to clang in lib.mk if LLVM is set > tools: allow proper CC/CXX/... override with LLVM=1 in > Makefile.include > selftests/bpf: fix test_cpp compilation failure with clang > selftests/bpf: silence clang compilation warnings > bpftool: fix a clang compilation warning > > tools/bpf/bpftool/net.c | 2 +- > tools/scripts/Makefile.include | 12 ++++++++++-- > tools/testing/selftests/bpf/Makefile | 7 ++++++- > tools/testing/selftests/lib.mk | 4 ++++ > 4 files changed, 21 insertions(+), 4 deletions(-) > > -- > 2.30.2 > -- Thanks, ~Nick Desaulniers
On Mon, Apr 12, 2021 at 4:58 PM Nick Desaulniers <ndesaulniers@google.com> wrote: > > On Mon, Apr 12, 2021 at 7:29 AM Yonghong Song <yhs@fb.com> wrote: > > > > To build kernel with clang, people typically use > > make -j60 LLVM=1 LLVM_IAS=1 > > LLVM_IAS=1 is not required for non-LTO build but > > is required for LTO build. In my environment, > > I am always having LLVM_IAS=1 regardless of > > whether LTO is enabled or not. > > > > After kernel is build with clang, the following command > > can be used to build selftests with clang: > > make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 > > Thank you for the series Yonghong. When I test the above command with > your series applied, I observe: > tools/include/tools/libc_compat.h:11:21: error: static declaration of > 'reallocarray' follows non-static declaration > static inline void *reallocarray(void *ptr, size_t nmemb, size_t size) > ^ > /usr/include/stdlib.h:559:14: note: previous declaration is here > extern void *reallocarray (void *__ptr, size_t __nmemb, size_t __size) > ^ > so perhaps the detection of > COMPAT_NEED_REALLOCARRAY/feature-reallocarray is incorrect? Is this related to _DEFAULT_SOURCE vs _GNU_SOURCE. via man 3 reallocarray: reallocarray(): Since glibc 2.29: _DEFAULT_SOURCE Glibc 2.28 and earlier: _GNU_SOURCE $ cd tools/testing/selftests/bpf $ grep -rn _DEFAULT_SOURCE | wc -l 0 $ grep -rn _GNU_SOURCE | wc -l 37 $ ldd --version | head -n1 ldd (Debian GLIBC 2.31-9+build1) 2.31
On 4/12/21 4:58 PM, Nick Desaulniers wrote: > On Mon, Apr 12, 2021 at 7:29 AM Yonghong Song <yhs@fb.com> wrote: >> >> To build kernel with clang, people typically use >> make -j60 LLVM=1 LLVM_IAS=1 >> LLVM_IAS=1 is not required for non-LTO build but >> is required for LTO build. In my environment, >> I am always having LLVM_IAS=1 regardless of >> whether LTO is enabled or not. >> >> After kernel is build with clang, the following command >> can be used to build selftests with clang: >> make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 > > Thank you for the series Yonghong. When I test the above command with > your series applied, I observe: > /usr/bin/ld: cannot find -lcap > clang-13: error: linker command failed with exit code 1 (use -v to see > invocation) > > I need to install libcap-dev, but this also seems to imply that BFD is > being used as the linker, not LLD. Perhaps if the compiler is being > used as the "driver" to also link executables, `-fuse-ld=lld` is > needed for the compiler flags. Yes, bfd is needed to build selftests/bpf. This is due to a dependency on bpftool which needs uses libbfd to disassemble the jited code. > > Then there's: > tools/include/tools/libc_compat.h:11:21: error: static declaration of > 'reallocarray' follows non-static declaration > static inline void *reallocarray(void *ptr, size_t nmemb, size_t size) > ^ > /usr/include/stdlib.h:559:14: note: previous declaration is here > extern void *reallocarray (void *__ptr, size_t __nmemb, size_t __size) > ^ > so perhaps the detection of > COMPAT_NEED_REALLOCARRAY/feature-reallocarray is incorrect? libbpf already stopped to use system reallocarray(), but bpftool still uses it. In bpftool makefile, we have ifeq ($(feature-reallocarray), 0) CFLAGS += -DCOMPAT_NEED_REALLOCARRAY endif I guess probably detection of feature-reallocarray is not correct? Could you take a look at your system? My system supports reallocarray, so the above -DCOMPAT_NEED_REALLOCARRAY is not added to compilation flags. > > >> >> But currently, using the above command, some compilations >> still use gcc and there are also compilation errors and warnings. >> This patch set intends to fix these issues. >> Patch #1 and #2 fixed the issue so clang/clang++ is >> used instead of gcc/g++. Patch #3 fixed a compilation >> failure. Patch #4 and #5 fixed various compiler warnings. >> >> Changelog: >> v1 -> v2: >> . add -Wno-unused-command-line-argument and -Wno-format-security >> for clang only as (1). gcc does not exhibit those >> warnings, and (2). -Wno-unused-command-line-argument is >> only supported by clang. (Sedat) >> >> Yonghong Song (5): >> selftests: set CC to clang in lib.mk if LLVM is set >> tools: allow proper CC/CXX/... override with LLVM=1 in >> Makefile.include >> selftests/bpf: fix test_cpp compilation failure with clang >> selftests/bpf: silence clang compilation warnings >> bpftool: fix a clang compilation warning >> >> tools/bpf/bpftool/net.c | 2 +- >> tools/scripts/Makefile.include | 12 ++++++++++-- >> tools/testing/selftests/bpf/Makefile | 7 ++++++- >> tools/testing/selftests/lib.mk | 4 ++++ >> 4 files changed, 21 insertions(+), 4 deletions(-) >> >> -- >> 2.30.2 >> > > > -- > Thanks, > ~Nick Desaulniers >
On 4/12/21 5:02 PM, Nick Desaulniers wrote: > On Mon, Apr 12, 2021 at 4:58 PM Nick Desaulniers > <ndesaulniers@google.com> wrote: >> >> On Mon, Apr 12, 2021 at 7:29 AM Yonghong Song <yhs@fb.com> wrote: >>> >>> To build kernel with clang, people typically use >>> make -j60 LLVM=1 LLVM_IAS=1 >>> LLVM_IAS=1 is not required for non-LTO build but >>> is required for LTO build. In my environment, >>> I am always having LLVM_IAS=1 regardless of >>> whether LTO is enabled or not. >>> >>> After kernel is build with clang, the following command >>> can be used to build selftests with clang: >>> make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 >> >> Thank you for the series Yonghong. When I test the above command with >> your series applied, I observe: >> tools/include/tools/libc_compat.h:11:21: error: static declaration of >> 'reallocarray' follows non-static declaration >> static inline void *reallocarray(void *ptr, size_t nmemb, size_t size) >> ^ >> /usr/include/stdlib.h:559:14: note: previous declaration is here >> extern void *reallocarray (void *__ptr, size_t __nmemb, size_t __size) >> ^ >> so perhaps the detection of >> COMPAT_NEED_REALLOCARRAY/feature-reallocarray is incorrect? > > Is this related to _DEFAULT_SOURCE vs _GNU_SOURCE. via man 3 reallocarray: > reallocarray(): > Since glibc 2.29: > _DEFAULT_SOURCE > Glibc 2.28 and earlier: > _GNU_SOURCE > You can try the following patch to see whether it works or not. diff --git a/tools/build/feature/test-reallocarray.c b/tools/build/feature/test-reallocarray.c index 8f6743e31da7..500cdeca07a7 100644 --- a/tools/build/feature/test-reallocarray.c +++ b/tools/build/feature/test-reallocarray.c @@ -1,5 +1,5 @@ // SPDX-License-Identifier: GPL-2.0 -#define _GNU_SOURCE +#define _DEFAULT_SOURCE #include <stdlib.h> int main(void) @@ -7,4 +7,4 @@ int main(void) return !!reallocarray(NULL, 1, 1); } -#undef _GNU_SOURCE +#undef _DEFAULT_SOURCE [yhs@devbig003.ftw2 ~/work/bpf-next/tools/build]$ > $ cd tools/testing/selftests/bpf > $ grep -rn _DEFAULT_SOURCE | wc -l > 0 > $ grep -rn _GNU_SOURCE | wc -l > 37 > $ ldd --version | head -n1 > ldd (Debian GLIBC 2.31-9+build1) 2.31 >
On Mon, Apr 12, 2021 at 4:29 PM Yonghong Song <yhs@fb.com> wrote: > > To build kernel with clang, people typically use > make -j60 LLVM=1 LLVM_IAS=1 > LLVM_IAS=1 is not required for non-LTO build but > is required for LTO build. In my environment, > I am always having LLVM_IAS=1 regardless of > whether LTO is enabled or not. > > After kernel is build with clang, the following command > can be used to build selftests with clang: > make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 > > But currently, using the above command, some compilations > still use gcc and there are also compilation errors and warnings. > This patch set intends to fix these issues. > Patch #1 and #2 fixed the issue so clang/clang++ is > used instead of gcc/g++. Patch #3 fixed a compilation > failure. Patch #4 and #5 fixed various compiler warnings. > Might be good to add some hints like when the build stops or errors: Like in my case when I had no CONFIG_DEBUG_INFO_BTF set. Of course in combination with Clang-LTO a pointer "you need pahole version 1.21." and CONFIG_DEBUG_INFO_BTF=y. Finally, a hint for missing xxx-dev(el) packages (see Nick's report). The tools directory has its own build rules thus I cannot say how to check for specific Kconfigs. I have not looked. NOTE: I have not checked without setting CONFIG_DEBUG_INFO. - Sedat - > Changelog: > v1 -> v2: > . add -Wno-unused-command-line-argument and -Wno-format-security > for clang only as (1). gcc does not exhibit those > warnings, and (2). -Wno-unused-command-line-argument is > only supported by clang. (Sedat) > > Yonghong Song (5): > selftests: set CC to clang in lib.mk if LLVM is set > tools: allow proper CC/CXX/... override with LLVM=1 in > Makefile.include > selftests/bpf: fix test_cpp compilation failure with clang > selftests/bpf: silence clang compilation warnings > bpftool: fix a clang compilation warning > > tools/bpf/bpftool/net.c | 2 +- > tools/scripts/Makefile.include | 12 ++++++++++-- > tools/testing/selftests/bpf/Makefile | 7 ++++++- > tools/testing/selftests/lib.mk | 4 ++++ > 4 files changed, 21 insertions(+), 4 deletions(-) > > -- > 2.30.2 >
On Mon, Apr 12, 2021 at 4:29 PM Yonghong Song <yhs@fb.com> wrote: > > To build kernel with clang, people typically use > make -j60 LLVM=1 LLVM_IAS=1 Just as a side note: When building with a high parallel-make-job like above "-j60" I saw that "test_core_extern.skel.h" was not generated which is a pre-req header file for building stuff like test_cpp.cpp. I have in my build-script: MAXCPUS="$(($(getconf _NPROCESSORS_ONLN)))" MAKE_JOBS="${MAXCPUS}" Thanks. - Sedat - > LLVM_IAS=1 is not required for non-LTO build but > is required for LTO build. In my environment, > I am always having LLVM_IAS=1 regardless of > whether LTO is enabled or not. > > After kernel is build with clang, the following command > can be used to build selftests with clang: > make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 > > But currently, using the above command, some compilations > still use gcc and there are also compilation errors and warnings. > This patch set intends to fix these issues. > Patch #1 and #2 fixed the issue so clang/clang++ is > used instead of gcc/g++. Patch #3 fixed a compilation > failure. Patch #4 and #5 fixed various compiler warnings. > > Changelog: > v1 -> v2: > . add -Wno-unused-command-line-argument and -Wno-format-security > for clang only as (1). gcc does not exhibit those > warnings, and (2). -Wno-unused-command-line-argument is > only supported by clang. (Sedat) > > Yonghong Song (5): > selftests: set CC to clang in lib.mk if LLVM is set > tools: allow proper CC/CXX/... override with LLVM=1 in > Makefile.include > selftests/bpf: fix test_cpp compilation failure with clang > selftests/bpf: silence clang compilation warnings > bpftool: fix a clang compilation warning > > tools/bpf/bpftool/net.c | 2 +- > tools/scripts/Makefile.include | 12 ++++++++++-- > tools/testing/selftests/bpf/Makefile | 7 ++++++- > tools/testing/selftests/lib.mk | 4 ++++ > 4 files changed, 21 insertions(+), 4 deletions(-) > > -- > 2.30.2 >
On 4/12/21 6:44 PM, Sedat Dilek wrote: > On Mon, Apr 12, 2021 at 4:29 PM Yonghong Song <yhs@fb.com> wrote: >> >> To build kernel with clang, people typically use >> make -j60 LLVM=1 LLVM_IAS=1 >> LLVM_IAS=1 is not required for non-LTO build but >> is required for LTO build. In my environment, >> I am always having LLVM_IAS=1 regardless of >> whether LTO is enabled or not. >> >> After kernel is build with clang, the following command >> can be used to build selftests with clang: >> make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 >> >> But currently, using the above command, some compilations >> still use gcc and there are also compilation errors and warnings. >> This patch set intends to fix these issues. >> Patch #1 and #2 fixed the issue so clang/clang++ is >> used instead of gcc/g++. Patch #3 fixed a compilation >> failure. Patch #4 and #5 fixed various compiler warnings. >> > > Might be good to add some hints like when the build stops or errors: > > Like in my case when I had no CONFIG_DEBUG_INFO_BTF set. > > Of course in combination with Clang-LTO a pointer "you need pahole > version 1.21." and CONFIG_DEBUG_INFO_BTF=y. > > Finally, a hint for missing xxx-dev(el) packages (see Nick's report). > > The tools directory has its own build rules thus I cannot say how to > check for specific Kconfigs. > I have not looked. I will add some more information about how I have tested in the cover letter. > > NOTE: I have not checked without setting CONFIG_DEBUG_INFO. > > - Sedat - > >> Changelog: >> v1 -> v2: >> . add -Wno-unused-command-line-argument and -Wno-format-security >> for clang only as (1). gcc does not exhibit those >> warnings, and (2). -Wno-unused-command-line-argument is >> only supported by clang. (Sedat) >> >> Yonghong Song (5): >> selftests: set CC to clang in lib.mk if LLVM is set >> tools: allow proper CC/CXX/... override with LLVM=1 in >> Makefile.include >> selftests/bpf: fix test_cpp compilation failure with clang >> selftests/bpf: silence clang compilation warnings >> bpftool: fix a clang compilation warning >> >> tools/bpf/bpftool/net.c | 2 +- >> tools/scripts/Makefile.include | 12 ++++++++++-- >> tools/testing/selftests/bpf/Makefile | 7 ++++++- >> tools/testing/selftests/lib.mk | 4 ++++ >> 4 files changed, 21 insertions(+), 4 deletions(-) >> >> -- >> 2.30.2 >>
On Mon, Apr 12, 2021 at 5:31 PM Yonghong Song <yhs@fb.com> wrote: > > > > On 4/12/21 5:02 PM, Nick Desaulniers wrote: > > On Mon, Apr 12, 2021 at 4:58 PM Nick Desaulniers > > <ndesaulniers@google.com> wrote: > >> > >> On Mon, Apr 12, 2021 at 7:29 AM Yonghong Song <yhs@fb.com> wrote: > >>> > >>> To build kernel with clang, people typically use > >>> make -j60 LLVM=1 LLVM_IAS=1 > >>> LLVM_IAS=1 is not required for non-LTO build but > >>> is required for LTO build. In my environment, > >>> I am always having LLVM_IAS=1 regardless of > >>> whether LTO is enabled or not. > >>> > >>> After kernel is build with clang, the following command > >>> can be used to build selftests with clang: > >>> make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 > >> > >> Thank you for the series Yonghong. When I test the above command with > >> your series applied, I observe: > >> tools/include/tools/libc_compat.h:11:21: error: static declaration of > >> 'reallocarray' follows non-static declaration > >> static inline void *reallocarray(void *ptr, size_t nmemb, size_t size) > >> ^ > >> /usr/include/stdlib.h:559:14: note: previous declaration is here > >> extern void *reallocarray (void *__ptr, size_t __nmemb, size_t __size) > >> ^ > >> so perhaps the detection of > >> COMPAT_NEED_REALLOCARRAY/feature-reallocarray is incorrect? > > > > Is this related to _DEFAULT_SOURCE vs _GNU_SOURCE. via man 3 reallocarray: > > reallocarray(): > > Since glibc 2.29: > > _DEFAULT_SOURCE > > Glibc 2.28 and earlier: > > _GNU_SOURCE > > > > You can try the following patch to see whether it works or not. > > diff --git a/tools/build/feature/test-reallocarray.c > b/tools/build/feature/test-reallocarray.c > index 8f6743e31da7..500cdeca07a7 100644 > --- a/tools/build/feature/test-reallocarray.c > +++ b/tools/build/feature/test-reallocarray.c > @@ -1,5 +1,5 @@ > // SPDX-License-Identifier: GPL-2.0 > -#define _GNU_SOURCE > +#define _DEFAULT_SOURCE > #include <stdlib.h> > > int main(void) > @@ -7,4 +7,4 @@ int main(void) > return !!reallocarray(NULL, 1, 1); > } > > -#undef _GNU_SOURCE > +#undef _DEFAULT_SOURCE Yeah, I had tried that. No luck though; same error message. Even: $ cat foo.c #define _DEFAULT_SOURCE #include <stdlib.h> void *reallocarray(void *ptr, size_t nmemb, size_t size) { return (void*)0; }; $ clang -c foo.c $ echo $? 0 So I'm not sure precisely what's going on here. I probably have to go digging around to understand tools/build/feature/ anyways. With your v3 applied, I consistently see: No zlib found and yet, I certainly do have zlib on my host. https://stackoverflow.com/a/54558861 > [yhs@devbig003.ftw2 ~/work/bpf-next/tools/build]$ > > > $ cd tools/testing/selftests/bpf > > $ grep -rn _DEFAULT_SOURCE | wc -l > > 0 > > $ grep -rn _GNU_SOURCE | wc -l > > 37 > > $ ldd --version | head -n1 > > ldd (Debian GLIBC 2.31-9+build1) 2.31 > >
On Tue, Apr 13, 2021 at 11:46 AM Nick Desaulniers <ndesaulniers@google.com> wrote: > > On Mon, Apr 12, 2021 at 5:31 PM Yonghong Song <yhs@fb.com> wrote: > > > > > > > > On 4/12/21 5:02 PM, Nick Desaulniers wrote: > > > On Mon, Apr 12, 2021 at 4:58 PM Nick Desaulniers > > > <ndesaulniers@google.com> wrote: > > >> > > >> On Mon, Apr 12, 2021 at 7:29 AM Yonghong Song <yhs@fb.com> wrote: > > >>> > > >>> To build kernel with clang, people typically use > > >>> make -j60 LLVM=1 LLVM_IAS=1 > > >>> LLVM_IAS=1 is not required for non-LTO build but > > >>> is required for LTO build. In my environment, > > >>> I am always having LLVM_IAS=1 regardless of > > >>> whether LTO is enabled or not. > > >>> > > >>> After kernel is build with clang, the following command > > >>> can be used to build selftests with clang: > > >>> make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 > > >> > > >> Thank you for the series Yonghong. When I test the above command with > > >> your series applied, I observe: > > >> tools/include/tools/libc_compat.h:11:21: error: static declaration of > > >> 'reallocarray' follows non-static declaration > > >> static inline void *reallocarray(void *ptr, size_t nmemb, size_t size) > > >> ^ > > >> /usr/include/stdlib.h:559:14: note: previous declaration is here > > >> extern void *reallocarray (void *__ptr, size_t __nmemb, size_t __size) > > >> ^ > > >> so perhaps the detection of > > >> COMPAT_NEED_REALLOCARRAY/feature-reallocarray is incorrect? > > > > > > Is this related to _DEFAULT_SOURCE vs _GNU_SOURCE. via man 3 reallocarray: > > > reallocarray(): > > > Since glibc 2.29: > > > _DEFAULT_SOURCE > > > Glibc 2.28 and earlier: > > > _GNU_SOURCE > > > > > > > You can try the following patch to see whether it works or not. > > > > diff --git a/tools/build/feature/test-reallocarray.c > > b/tools/build/feature/test-reallocarray.c > > index 8f6743e31da7..500cdeca07a7 100644 > > --- a/tools/build/feature/test-reallocarray.c > > +++ b/tools/build/feature/test-reallocarray.c > > @@ -1,5 +1,5 @@ > > // SPDX-License-Identifier: GPL-2.0 > > -#define _GNU_SOURCE > > +#define _DEFAULT_SOURCE > > #include <stdlib.h> > > > > int main(void) > > @@ -7,4 +7,4 @@ int main(void) > > return !!reallocarray(NULL, 1, 1); > > } > > > > -#undef _GNU_SOURCE > > +#undef _DEFAULT_SOURCE > > Yeah, I had tried that. No luck though; same error message. Even: > > $ cat foo.c > #define _DEFAULT_SOURCE > #include <stdlib.h> > void *reallocarray(void *ptr, size_t nmemb, size_t size) { return (void*)0; }; > $ clang -c foo.c > $ echo $? > 0 > > So I'm not sure precisely what's going on here. I probably have to go > digging around to understand tools/build/feature/ anyways. With your > v3 applied, I consistently see: > No zlib found > and yet, I certainly do have zlib on my host. > https://stackoverflow.com/a/54558861 Jiri, any tips on how to debug feature detection in tools/build/feature/Makefile? > > > [yhs@devbig003.ftw2 ~/work/bpf-next/tools/build]$ > > > > > $ cd tools/testing/selftests/bpf > > > $ grep -rn _DEFAULT_SOURCE | wc -l > > > 0 > > > $ grep -rn _GNU_SOURCE | wc -l > > > 37 > > > $ ldd --version | head -n1 > > > ldd (Debian GLIBC 2.31-9+build1) 2.31 > > > > > > > -- > Thanks, > ~Nick Desaulniers
On Tue, Apr 13, 2021 at 11:56:33AM -0700, Nick Desaulniers wrote: > On Tue, Apr 13, 2021 at 11:46 AM Nick Desaulniers > <ndesaulniers@google.com> wrote: > > > > On Mon, Apr 12, 2021 at 5:31 PM Yonghong Song <yhs@fb.com> wrote: > > > > > > > > > > > > On 4/12/21 5:02 PM, Nick Desaulniers wrote: > > > > On Mon, Apr 12, 2021 at 4:58 PM Nick Desaulniers > > > > <ndesaulniers@google.com> wrote: > > > >> > > > >> On Mon, Apr 12, 2021 at 7:29 AM Yonghong Song <yhs@fb.com> wrote: > > > >>> > > > >>> To build kernel with clang, people typically use > > > >>> make -j60 LLVM=1 LLVM_IAS=1 > > > >>> LLVM_IAS=1 is not required for non-LTO build but > > > >>> is required for LTO build. In my environment, > > > >>> I am always having LLVM_IAS=1 regardless of > > > >>> whether LTO is enabled or not. > > > >>> > > > >>> After kernel is build with clang, the following command > > > >>> can be used to build selftests with clang: > > > >>> make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 > > > >> > > > >> Thank you for the series Yonghong. When I test the above command with > > > >> your series applied, I observe: > > > >> tools/include/tools/libc_compat.h:11:21: error: static declaration of > > > >> 'reallocarray' follows non-static declaration > > > >> static inline void *reallocarray(void *ptr, size_t nmemb, size_t size) > > > >> ^ > > > >> /usr/include/stdlib.h:559:14: note: previous declaration is here > > > >> extern void *reallocarray (void *__ptr, size_t __nmemb, size_t __size) > > > >> ^ > > > >> so perhaps the detection of > > > >> COMPAT_NEED_REALLOCARRAY/feature-reallocarray is incorrect? > > > > > > > > Is this related to _DEFAULT_SOURCE vs _GNU_SOURCE. via man 3 reallocarray: > > > > reallocarray(): > > > > Since glibc 2.29: > > > > _DEFAULT_SOURCE > > > > Glibc 2.28 and earlier: > > > > _GNU_SOURCE > > > > > > > > > > You can try the following patch to see whether it works or not. > > > > > > diff --git a/tools/build/feature/test-reallocarray.c > > > b/tools/build/feature/test-reallocarray.c > > > index 8f6743e31da7..500cdeca07a7 100644 > > > --- a/tools/build/feature/test-reallocarray.c > > > +++ b/tools/build/feature/test-reallocarray.c > > > @@ -1,5 +1,5 @@ > > > // SPDX-License-Identifier: GPL-2.0 > > > -#define _GNU_SOURCE > > > +#define _DEFAULT_SOURCE > > > #include <stdlib.h> > > > > > > int main(void) > > > @@ -7,4 +7,4 @@ int main(void) > > > return !!reallocarray(NULL, 1, 1); > > > } > > > > > > -#undef _GNU_SOURCE > > > +#undef _DEFAULT_SOURCE > > > > Yeah, I had tried that. No luck though; same error message. Even: > > > > $ cat foo.c > > #define _DEFAULT_SOURCE > > #include <stdlib.h> > > void *reallocarray(void *ptr, size_t nmemb, size_t size) { return (void*)0; }; > > $ clang -c foo.c > > $ echo $? > > 0 > > > > So I'm not sure precisely what's going on here. I probably have to go > > digging around to understand tools/build/feature/ anyways. With your > > v3 applied, I consistently see: > > No zlib found > > and yet, I certainly do have zlib on my host. > > https://stackoverflow.com/a/54558861 > > Jiri, any tips on how to debug feature detection in > tools/build/feature/Makefile? for quick check, there's output file for each test, like: [jolsa@krava feature]$ ls -l *.make.output -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-all.make.output -rw-rw-r--. 1 jolsa jolsa 182 Apr 9 15:52 test-bionic.make.output -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-cplus-demangle.make.output -rw-rw-r--. 1 jolsa jolsa 145 Apr 9 15:52 test-jvmti.make.output -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbabeltrace.make.output -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbpf.make.output -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libdebuginfod.make.output -rw-rw-r--. 1 jolsa jolsa 193 Apr 9 15:52 test-libunwind-aarch64.make.output -rw-rw-r--. 1 jolsa jolsa 177 Apr 9 15:52 test-libunwind-x86.make.output [jolsa@krava feature]$ cat test-libunwind-aarch64.make.output test-libunwind-aarch64.c:2:10: fatal error: libunwind-aarch64.h: No such file or directory 2 | #include <libunwind-aarch64.h> | ^~~~~~~~~~~~~~~~~~~~~ compilation terminated. [jolsa@krava feature]$ cat test-libunwind-x86.make.output test-libunwind-x86.c:2:10: fatal error: libunwind-x86.h: No such file or directory 2 | #include <libunwind-x86.h> | ^~~~~~~~~~~~~~~~~ zlib should be done by: [jolsa@krava feature]$ make test-zlib.bin gcc -MD -Wall -Werror -o test-zlib.bin test-zlib.c > test-zlib.make.output 2>&1 -lz I can try to recreate, how do you build? jirka
On Tue, Apr 13, 2021 at 1:36 PM Jiri Olsa <jolsa@redhat.com> wrote: > > On Tue, Apr 13, 2021 at 11:56:33AM -0700, Nick Desaulniers wrote: > > On Tue, Apr 13, 2021 at 11:46 AM Nick Desaulniers > > <ndesaulniers@google.com> wrote: > > > > > > On Mon, Apr 12, 2021 at 5:31 PM Yonghong Song <yhs@fb.com> wrote: > > > > > > > > > > > > > > > > On 4/12/21 5:02 PM, Nick Desaulniers wrote: > > > > > On Mon, Apr 12, 2021 at 4:58 PM Nick Desaulniers > > > > > <ndesaulniers@google.com> wrote: > > > > >> > > > > >> On Mon, Apr 12, 2021 at 7:29 AM Yonghong Song <yhs@fb.com> wrote: > > > > >>> > > > > >>> To build kernel with clang, people typically use > > > > >>> make -j60 LLVM=1 LLVM_IAS=1 > > > > >>> LLVM_IAS=1 is not required for non-LTO build but > > > > >>> is required for LTO build. In my environment, > > > > >>> I am always having LLVM_IAS=1 regardless of > > > > >>> whether LTO is enabled or not. > > > > >>> > > > > >>> After kernel is build with clang, the following command > > > > >>> can be used to build selftests with clang: > > > > >>> make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 ^ note below > > > > >> > > > > >> Thank you for the series Yonghong. When I test the above command with > > > > >> your series applied, I observe: > > > > >> tools/include/tools/libc_compat.h:11:21: error: static declaration of > > > > >> 'reallocarray' follows non-static declaration > > > > >> static inline void *reallocarray(void *ptr, size_t nmemb, size_t size) > > > > >> ^ > > > > >> /usr/include/stdlib.h:559:14: note: previous declaration is here > > > > >> extern void *reallocarray (void *__ptr, size_t __nmemb, size_t __size) > > > > >> ^ > > > > >> so perhaps the detection of > > > > >> COMPAT_NEED_REALLOCARRAY/feature-reallocarray is incorrect? > > > > > > > > > > Is this related to _DEFAULT_SOURCE vs _GNU_SOURCE. via man 3 reallocarray: > > > > > reallocarray(): > > > > > Since glibc 2.29: > > > > > _DEFAULT_SOURCE > > > > > Glibc 2.28 and earlier: > > > > > _GNU_SOURCE > > > > > > > > > > > > > You can try the following patch to see whether it works or not. > > > > > > > > diff --git a/tools/build/feature/test-reallocarray.c > > > > b/tools/build/feature/test-reallocarray.c > > > > index 8f6743e31da7..500cdeca07a7 100644 > > > > --- a/tools/build/feature/test-reallocarray.c > > > > +++ b/tools/build/feature/test-reallocarray.c > > > > @@ -1,5 +1,5 @@ > > > > // SPDX-License-Identifier: GPL-2.0 > > > > -#define _GNU_SOURCE > > > > +#define _DEFAULT_SOURCE > > > > #include <stdlib.h> > > > > > > > > int main(void) > > > > @@ -7,4 +7,4 @@ int main(void) > > > > return !!reallocarray(NULL, 1, 1); > > > > } > > > > > > > > -#undef _GNU_SOURCE > > > > +#undef _DEFAULT_SOURCE > > > > > > Yeah, I had tried that. No luck though; same error message. Even: > > > > > > $ cat foo.c > > > #define _DEFAULT_SOURCE > > > #include <stdlib.h> > > > void *reallocarray(void *ptr, size_t nmemb, size_t size) { return (void*)0; }; > > > $ clang -c foo.c > > > $ echo $? > > > 0 > > > > > > So I'm not sure precisely what's going on here. I probably have to go > > > digging around to understand tools/build/feature/ anyways. With your > > > v3 applied, I consistently see: > > > No zlib found > > > and yet, I certainly do have zlib on my host. > > > https://stackoverflow.com/a/54558861 > > > > Jiri, any tips on how to debug feature detection in > > tools/build/feature/Makefile? > > for quick check, there's output file for each test, like: > > [jolsa@krava feature]$ ls -l *.make.output > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-all.make.output > -rw-rw-r--. 1 jolsa jolsa 182 Apr 9 15:52 test-bionic.make.output > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-cplus-demangle.make.output > -rw-rw-r--. 1 jolsa jolsa 145 Apr 9 15:52 test-jvmti.make.output > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbabeltrace.make.output > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbpf.make.output > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libdebuginfod.make.output > -rw-rw-r--. 1 jolsa jolsa 193 Apr 9 15:52 test-libunwind-aarch64.make.output > -rw-rw-r--. 1 jolsa jolsa 177 Apr 9 15:52 test-libunwind-x86.make.output > [jolsa@krava feature]$ cat test-libunwind-aarch64.make.output > test-libunwind-aarch64.c:2:10: fatal error: libunwind-aarch64.h: No such file or directory > 2 | #include <libunwind-aarch64.h> > | ^~~~~~~~~~~~~~~~~~~~~ > compilation terminated. > [jolsa@krava feature]$ cat test-libunwind-x86.make.output > test-libunwind-x86.c:2:10: fatal error: libunwind-x86.h: No such file or directory > 2 | #include <libunwind-x86.h> > | ^~~~~~~~~~~~~~~~~ > > zlib should be done by: > [jolsa@krava feature]$ make test-zlib.bin > gcc -MD -Wall -Werror -o test-zlib.bin test-zlib.c > test-zlib.make.output 2>&1 -lz > > > I can try to recreate, how do you build? See note above, I'm similarly running precisely: $ make LLVM=1 LLVM_IAS=1 -j72 defconfig $ make LLVM=1 LLVM_IAS=1 -j72 clean $ make LLVM=1 LLVM_IAS=1 -j72 -C tools/testing/selftests/bpf
On Tue, Apr 13, 2021 at 01:45:39PM -0700, Nick Desaulniers wrote: SNIP > > > > > > > > So I'm not sure precisely what's going on here. I probably have to go > > > > digging around to understand tools/build/feature/ anyways. With your > > > > v3 applied, I consistently see: > > > > No zlib found > > > > and yet, I certainly do have zlib on my host. > > > > https://stackoverflow.com/a/54558861 > > > > > > Jiri, any tips on how to debug feature detection in > > > tools/build/feature/Makefile? > > > > for quick check, there's output file for each test, like: > > > > [jolsa@krava feature]$ ls -l *.make.output > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-all.make.output > > -rw-rw-r--. 1 jolsa jolsa 182 Apr 9 15:52 test-bionic.make.output > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-cplus-demangle.make.output > > -rw-rw-r--. 1 jolsa jolsa 145 Apr 9 15:52 test-jvmti.make.output > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbabeltrace.make.output > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbpf.make.output > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libdebuginfod.make.output > > -rw-rw-r--. 1 jolsa jolsa 193 Apr 9 15:52 test-libunwind-aarch64.make.output > > -rw-rw-r--. 1 jolsa jolsa 177 Apr 9 15:52 test-libunwind-x86.make.output > > [jolsa@krava feature]$ cat test-libunwind-aarch64.make.output > > test-libunwind-aarch64.c:2:10: fatal error: libunwind-aarch64.h: No such file or directory > > 2 | #include <libunwind-aarch64.h> > > | ^~~~~~~~~~~~~~~~~~~~~ > > compilation terminated. > > [jolsa@krava feature]$ cat test-libunwind-x86.make.output > > test-libunwind-x86.c:2:10: fatal error: libunwind-x86.h: No such file or directory > > 2 | #include <libunwind-x86.h> > > | ^~~~~~~~~~~~~~~~~ > > > > zlib should be done by: > > [jolsa@krava feature]$ make test-zlib.bin > > gcc -MD -Wall -Werror -o test-zlib.bin test-zlib.c > test-zlib.make.output 2>&1 -lz > > > > > > I can try to recreate, how do you build? > > See note above, I'm similarly running precisely: > $ make LLVM=1 LLVM_IAS=1 -j72 defconfig > $ make LLVM=1 LLVM_IAS=1 -j72 clean > $ make LLVM=1 LLVM_IAS=1 -j72 -C tools/testing/selftests/bpf for some reason I'm stuck with this error on latest bpf-next/master $ make LLVM=1 LLVM_IAS=1 -C tools/testing/selftests/bpf make[1]: Nothing to be done for 'docs'. CLNG-BPF [test_maps] test_lwt_ip_encap.o CLNG-BPF [test_maps] test_tc_edt.o CLNG-BPF [test_maps] local_storage.o progs/local_storage.c:41:15: error: use of undeclared identifier 'BPF_MAP_TYPE_TASK_STORAGE'; did you mean 'BPF_MAP_TYPE_SK_STORAGE'? __uint(type, BPF_MAP_TYPE_TASK_STORAGE); ^~~~~~~~~~~~~~~~~~~~~~~~~ BPF_MAP_TYPE_SK_STORAGE /home/jolsa/linux/tools/testing/selftests/bpf/tools/include/bpf/bpf_helpers.h:13:39: note: expanded from macro '__uint' #define __uint(name, val) int (*name)[val] ^ /home/jolsa/linux/tools/testing/selftests/bpf/tools/include/vmlinux.h:10317:2: note: 'BPF_MAP_TYPE_SK_STORAGE' declared here BPF_MAP_TYPE_SK_STORAGE = 24, ^ 1 error generated. make: *** [Makefile:448: /home/jolsa/linux/tools/testing/selftests/bpf/local_storage.o] Error 1 make: Leaving directory '/home/jolsa/linux/tools/testing/selftests/bpf' jirka
On Wed, Apr 14, 2021 at 6:18 AM Jiri Olsa <jolsa@redhat.com> wrote: > > On Tue, Apr 13, 2021 at 01:45:39PM -0700, Nick Desaulniers wrote: > > SNIP > > > > > > > > > > > So I'm not sure precisely what's going on here. I probably have to go > > > > > digging around to understand tools/build/feature/ anyways. With your > > > > > v3 applied, I consistently see: > > > > > No zlib found > > > > > and yet, I certainly do have zlib on my host. > > > > > https://stackoverflow.com/a/54558861 > > > > > > > > Jiri, any tips on how to debug feature detection in > > > > tools/build/feature/Makefile? > > > > > > for quick check, there's output file for each test, like: > > > > > > [jolsa@krava feature]$ ls -l *.make.output > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-all.make.output > > > -rw-rw-r--. 1 jolsa jolsa 182 Apr 9 15:52 test-bionic.make.output > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-cplus-demangle.make.output > > > -rw-rw-r--. 1 jolsa jolsa 145 Apr 9 15:52 test-jvmti.make.output > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbabeltrace.make.output > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbpf.make.output > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libdebuginfod.make.output > > > -rw-rw-r--. 1 jolsa jolsa 193 Apr 9 15:52 test-libunwind-aarch64.make.output > > > -rw-rw-r--. 1 jolsa jolsa 177 Apr 9 15:52 test-libunwind-x86.make.output > > > [jolsa@krava feature]$ cat test-libunwind-aarch64.make.output > > > test-libunwind-aarch64.c:2:10: fatal error: libunwind-aarch64.h: No such file or directory > > > 2 | #include <libunwind-aarch64.h> > > > | ^~~~~~~~~~~~~~~~~~~~~ > > > compilation terminated. > > > [jolsa@krava feature]$ cat test-libunwind-x86.make.output > > > test-libunwind-x86.c:2:10: fatal error: libunwind-x86.h: No such file or directory > > > 2 | #include <libunwind-x86.h> > > > | ^~~~~~~~~~~~~~~~~ > > > > > > zlib should be done by: > > > [jolsa@krava feature]$ make test-zlib.bin > > > gcc -MD -Wall -Werror -o test-zlib.bin test-zlib.c > test-zlib.make.output 2>&1 -lz > > > > > > > > > I can try to recreate, how do you build? > > > > See note above, I'm similarly running precisely: > > $ make LLVM=1 LLVM_IAS=1 -j72 defconfig > > $ make LLVM=1 LLVM_IAS=1 -j72 clean > > $ make LLVM=1 LLVM_IAS=1 -j72 -C tools/testing/selftests/bpf > > for some reason I'm stuck with this error on latest bpf-next/master did you build vmlinux image before building selftests? those enums should come through vmlinux.h from up-to-date vmlinux > > $ make LLVM=1 LLVM_IAS=1 -C tools/testing/selftests/bpf > > make[1]: Nothing to be done for 'docs'. > CLNG-BPF [test_maps] test_lwt_ip_encap.o > CLNG-BPF [test_maps] test_tc_edt.o > CLNG-BPF [test_maps] local_storage.o > progs/local_storage.c:41:15: error: use of undeclared identifier 'BPF_MAP_TYPE_TASK_STORAGE'; did you mean 'BPF_MAP_TYPE_SK_STORAGE'? > __uint(type, BPF_MAP_TYPE_TASK_STORAGE); > ^~~~~~~~~~~~~~~~~~~~~~~~~ > BPF_MAP_TYPE_SK_STORAGE > /home/jolsa/linux/tools/testing/selftests/bpf/tools/include/bpf/bpf_helpers.h:13:39: note: expanded from macro '__uint' > #define __uint(name, val) int (*name)[val] > ^ > /home/jolsa/linux/tools/testing/selftests/bpf/tools/include/vmlinux.h:10317:2: note: 'BPF_MAP_TYPE_SK_STORAGE' declared here > BPF_MAP_TYPE_SK_STORAGE = 24, > ^ > 1 error generated. > make: *** [Makefile:448: /home/jolsa/linux/tools/testing/selftests/bpf/local_storage.o] Error 1 > make: Leaving directory '/home/jolsa/linux/tools/testing/selftests/bpf' > > > jirka >
On Wed, Apr 14, 2021 at 05:16:01PM -0700, Andrii Nakryiko wrote: > On Wed, Apr 14, 2021 at 6:18 AM Jiri Olsa <jolsa@redhat.com> wrote: > > > > On Tue, Apr 13, 2021 at 01:45:39PM -0700, Nick Desaulniers wrote: > > > > SNIP > > > > > > > > > > > > > > So I'm not sure precisely what's going on here. I probably have to go > > > > > > digging around to understand tools/build/feature/ anyways. With your > > > > > > v3 applied, I consistently see: > > > > > > No zlib found > > > > > > and yet, I certainly do have zlib on my host. > > > > > > https://stackoverflow.com/a/54558861 > > > > > > > > > > Jiri, any tips on how to debug feature detection in > > > > > tools/build/feature/Makefile? > > > > > > > > for quick check, there's output file for each test, like: > > > > > > > > [jolsa@krava feature]$ ls -l *.make.output > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-all.make.output > > > > -rw-rw-r--. 1 jolsa jolsa 182 Apr 9 15:52 test-bionic.make.output > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-cplus-demangle.make.output > > > > -rw-rw-r--. 1 jolsa jolsa 145 Apr 9 15:52 test-jvmti.make.output > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbabeltrace.make.output > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbpf.make.output > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libdebuginfod.make.output > > > > -rw-rw-r--. 1 jolsa jolsa 193 Apr 9 15:52 test-libunwind-aarch64.make.output > > > > -rw-rw-r--. 1 jolsa jolsa 177 Apr 9 15:52 test-libunwind-x86.make.output > > > > [jolsa@krava feature]$ cat test-libunwind-aarch64.make.output > > > > test-libunwind-aarch64.c:2:10: fatal error: libunwind-aarch64.h: No such file or directory > > > > 2 | #include <libunwind-aarch64.h> > > > > | ^~~~~~~~~~~~~~~~~~~~~ > > > > compilation terminated. > > > > [jolsa@krava feature]$ cat test-libunwind-x86.make.output > > > > test-libunwind-x86.c:2:10: fatal error: libunwind-x86.h: No such file or directory > > > > 2 | #include <libunwind-x86.h> > > > > | ^~~~~~~~~~~~~~~~~ > > > > > > > > zlib should be done by: > > > > [jolsa@krava feature]$ make test-zlib.bin > > > > gcc -MD -Wall -Werror -o test-zlib.bin test-zlib.c > test-zlib.make.output 2>&1 -lz > > > > > > > > > > > > I can try to recreate, how do you build? > > > > > > See note above, I'm similarly running precisely: > > > $ make LLVM=1 LLVM_IAS=1 -j72 defconfig > > > $ make LLVM=1 LLVM_IAS=1 -j72 clean > > > $ make LLVM=1 LLVM_IAS=1 -j72 -C tools/testing/selftests/bpf > > > > for some reason I'm stuck with this error on latest bpf-next/master > > did you build vmlinux image before building selftests? those enums > should come through vmlinux.h from up-to-date vmlinux it was there.. but I found the clang/lld I compiled can't link properly, which is probably unrelated to the error below, but I need to solve it first ;-) jirka > > > > > $ make LLVM=1 LLVM_IAS=1 -C tools/testing/selftests/bpf > > > > make[1]: Nothing to be done for 'docs'. > > CLNG-BPF [test_maps] test_lwt_ip_encap.o > > CLNG-BPF [test_maps] test_tc_edt.o > > CLNG-BPF [test_maps] local_storage.o > > progs/local_storage.c:41:15: error: use of undeclared identifier 'BPF_MAP_TYPE_TASK_STORAGE'; did you mean 'BPF_MAP_TYPE_SK_STORAGE'? > > __uint(type, BPF_MAP_TYPE_TASK_STORAGE); > > ^~~~~~~~~~~~~~~~~~~~~~~~~ > > BPF_MAP_TYPE_SK_STORAGE > > /home/jolsa/linux/tools/testing/selftests/bpf/tools/include/bpf/bpf_helpers.h:13:39: note: expanded from macro '__uint' > > #define __uint(name, val) int (*name)[val] > > ^ > > /home/jolsa/linux/tools/testing/selftests/bpf/tools/include/vmlinux.h:10317:2: note: 'BPF_MAP_TYPE_SK_STORAGE' declared here > > BPF_MAP_TYPE_SK_STORAGE = 24, > > ^ > > 1 error generated. > > make: *** [Makefile:448: /home/jolsa/linux/tools/testing/selftests/bpf/local_storage.o] Error 1 > > make: Leaving directory '/home/jolsa/linux/tools/testing/selftests/bpf' > > > > > > jirka > > >
On Thu, Apr 15, 2021 at 6:23 AM Jiri Olsa <jolsa@redhat.com> wrote: > > On Wed, Apr 14, 2021 at 05:16:01PM -0700, Andrii Nakryiko wrote: > > On Wed, Apr 14, 2021 at 6:18 AM Jiri Olsa <jolsa@redhat.com> wrote: > > > > > > On Tue, Apr 13, 2021 at 01:45:39PM -0700, Nick Desaulniers wrote: > > > > > > SNIP > > > > > > > > > > > > > > > > > So I'm not sure precisely what's going on here. I probably have to go > > > > > > > digging around to understand tools/build/feature/ anyways. With your > > > > > > > v3 applied, I consistently see: > > > > > > > No zlib found > > > > > > > and yet, I certainly do have zlib on my host. > > > > > > > https://stackoverflow.com/a/54558861 > > > > > > > > > > > > Jiri, any tips on how to debug feature detection in > > > > > > tools/build/feature/Makefile? > > > > > > > > > > for quick check, there's output file for each test, like: > > > > > > > > > > [jolsa@krava feature]$ ls -l *.make.output > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-all.make.output > > > > > -rw-rw-r--. 1 jolsa jolsa 182 Apr 9 15:52 test-bionic.make.output > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-cplus-demangle.make.output > > > > > -rw-rw-r--. 1 jolsa jolsa 145 Apr 9 15:52 test-jvmti.make.output > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbabeltrace.make.output > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbpf.make.output > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libdebuginfod.make.output > > > > > -rw-rw-r--. 1 jolsa jolsa 193 Apr 9 15:52 test-libunwind-aarch64.make.output > > > > > -rw-rw-r--. 1 jolsa jolsa 177 Apr 9 15:52 test-libunwind-x86.make.output > > > > > [jolsa@krava feature]$ cat test-libunwind-aarch64.make.output > > > > > test-libunwind-aarch64.c:2:10: fatal error: libunwind-aarch64.h: No such file or directory > > > > > 2 | #include <libunwind-aarch64.h> > > > > > | ^~~~~~~~~~~~~~~~~~~~~ > > > > > compilation terminated. > > > > > [jolsa@krava feature]$ cat test-libunwind-x86.make.output > > > > > test-libunwind-x86.c:2:10: fatal error: libunwind-x86.h: No such file or directory > > > > > 2 | #include <libunwind-x86.h> > > > > > | ^~~~~~~~~~~~~~~~~ > > > > > > > > > > zlib should be done by: > > > > > [jolsa@krava feature]$ make test-zlib.bin > > > > > gcc -MD -Wall -Werror -o test-zlib.bin test-zlib.c > test-zlib.make.output 2>&1 -lz > > > > > > > > > > > > > > > I can try to recreate, how do you build? > > > > > > > > See note above, I'm similarly running precisely: > > > > $ make LLVM=1 LLVM_IAS=1 -j72 defconfig > > > > $ make LLVM=1 LLVM_IAS=1 -j72 clean > > > > $ make LLVM=1 LLVM_IAS=1 -j72 -C tools/testing/selftests/bpf > > > > > > for some reason I'm stuck with this error on latest bpf-next/master > > > > did you build vmlinux image before building selftests? those enums > > should come through vmlinux.h from up-to-date vmlinux > > it was there.. but I found the clang/lld I compiled can't link properly, > which is probably unrelated to the error below, but I need to solve it > first ;-) Do you have more info about what command you're running, or what error you're observing? The kernel itself is not linking for you? > > jirka > > > > > > > > > $ make LLVM=1 LLVM_IAS=1 -C tools/testing/selftests/bpf > > > > > > make[1]: Nothing to be done for 'docs'. > > > CLNG-BPF [test_maps] test_lwt_ip_encap.o > > > CLNG-BPF [test_maps] test_tc_edt.o > > > CLNG-BPF [test_maps] local_storage.o > > > progs/local_storage.c:41:15: error: use of undeclared identifier 'BPF_MAP_TYPE_TASK_STORAGE'; did you mean 'BPF_MAP_TYPE_SK_STORAGE'? > > > __uint(type, BPF_MAP_TYPE_TASK_STORAGE); > > > ^~~~~~~~~~~~~~~~~~~~~~~~~ > > > BPF_MAP_TYPE_SK_STORAGE > > > /home/jolsa/linux/tools/testing/selftests/bpf/tools/include/bpf/bpf_helpers.h:13:39: note: expanded from macro '__uint' > > > #define __uint(name, val) int (*name)[val] > > > ^ > > > /home/jolsa/linux/tools/testing/selftests/bpf/tools/include/vmlinux.h:10317:2: note: 'BPF_MAP_TYPE_SK_STORAGE' declared here > > > BPF_MAP_TYPE_SK_STORAGE = 24, > > > ^ > > > 1 error generated. > > > make: *** [Makefile:448: /home/jolsa/linux/tools/testing/selftests/bpf/local_storage.o] Error 1 > > > make: Leaving directory '/home/jolsa/linux/tools/testing/selftests/bpf' > > > > > > > > > jirka > > > > > >
On Thu, Apr 15, 2021 at 09:55:50AM -0700, Nick Desaulniers wrote: > On Thu, Apr 15, 2021 at 6:23 AM Jiri Olsa <jolsa@redhat.com> wrote: > > > > On Wed, Apr 14, 2021 at 05:16:01PM -0700, Andrii Nakryiko wrote: > > > On Wed, Apr 14, 2021 at 6:18 AM Jiri Olsa <jolsa@redhat.com> wrote: > > > > > > > > On Tue, Apr 13, 2021 at 01:45:39PM -0700, Nick Desaulniers wrote: > > > > > > > > SNIP > > > > > > > > > > > > > > > > > > > > So I'm not sure precisely what's going on here. I probably have to go > > > > > > > > digging around to understand tools/build/feature/ anyways. With your > > > > > > > > v3 applied, I consistently see: > > > > > > > > No zlib found > > > > > > > > and yet, I certainly do have zlib on my host. > > > > > > > > https://stackoverflow.com/a/54558861 > > > > > > > > > > > > > > Jiri, any tips on how to debug feature detection in > > > > > > > tools/build/feature/Makefile? > > > > > > > > > > > > for quick check, there's output file for each test, like: > > > > > > > > > > > > [jolsa@krava feature]$ ls -l *.make.output > > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-all.make.output > > > > > > -rw-rw-r--. 1 jolsa jolsa 182 Apr 9 15:52 test-bionic.make.output > > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-cplus-demangle.make.output > > > > > > -rw-rw-r--. 1 jolsa jolsa 145 Apr 9 15:52 test-jvmti.make.output > > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbabeltrace.make.output > > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbpf.make.output > > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libdebuginfod.make.output > > > > > > -rw-rw-r--. 1 jolsa jolsa 193 Apr 9 15:52 test-libunwind-aarch64.make.output > > > > > > -rw-rw-r--. 1 jolsa jolsa 177 Apr 9 15:52 test-libunwind-x86.make.output > > > > > > [jolsa@krava feature]$ cat test-libunwind-aarch64.make.output > > > > > > test-libunwind-aarch64.c:2:10: fatal error: libunwind-aarch64.h: No such file or directory > > > > > > 2 | #include <libunwind-aarch64.h> > > > > > > | ^~~~~~~~~~~~~~~~~~~~~ > > > > > > compilation terminated. > > > > > > [jolsa@krava feature]$ cat test-libunwind-x86.make.output > > > > > > test-libunwind-x86.c:2:10: fatal error: libunwind-x86.h: No such file or directory > > > > > > 2 | #include <libunwind-x86.h> > > > > > > | ^~~~~~~~~~~~~~~~~ > > > > > > > > > > > > zlib should be done by: > > > > > > [jolsa@krava feature]$ make test-zlib.bin > > > > > > gcc -MD -Wall -Werror -o test-zlib.bin test-zlib.c > test-zlib.make.output 2>&1 -lz > > > > > > > > > > > > > > > > > > I can try to recreate, how do you build? > > > > > > > > > > See note above, I'm similarly running precisely: > > > > > $ make LLVM=1 LLVM_IAS=1 -j72 defconfig > > > > > $ make LLVM=1 LLVM_IAS=1 -j72 clean > > > > > $ make LLVM=1 LLVM_IAS=1 -j72 -C tools/testing/selftests/bpf > > > > > > > > for some reason I'm stuck with this error on latest bpf-next/master > > > > > > did you build vmlinux image before building selftests? those enums > > > should come through vmlinux.h from up-to-date vmlinux > > > > it was there.. but I found the clang/lld I compiled can't link properly, > > which is probably unrelated to the error below, but I need to solve it > > first ;-) > > Do you have more info about what command you're running, or what error > you're observing? The kernel itself is not linking for you? I did the commands you sent, but I might have but llvm/clang build, I'm rebuilding now, it takes forever.. jirka
On Thu, Apr 15, 2021 at 07:17:35PM +0200, Jiri Olsa wrote: > On Thu, Apr 15, 2021 at 09:55:50AM -0700, Nick Desaulniers wrote: > > On Thu, Apr 15, 2021 at 6:23 AM Jiri Olsa <jolsa@redhat.com> wrote: > > > > > > On Wed, Apr 14, 2021 at 05:16:01PM -0700, Andrii Nakryiko wrote: > > > > On Wed, Apr 14, 2021 at 6:18 AM Jiri Olsa <jolsa@redhat.com> wrote: > > > > > > > > > > On Tue, Apr 13, 2021 at 01:45:39PM -0700, Nick Desaulniers wrote: > > > > > > > > > > SNIP > > > > > > > > > > > > > > > > > > > > > > > So I'm not sure precisely what's going on here. I probably have to go > > > > > > > > > digging around to understand tools/build/feature/ anyways. With your > > > > > > > > > v3 applied, I consistently see: > > > > > > > > > No zlib found > > > > > > > > > and yet, I certainly do have zlib on my host. > > > > > > > > > https://stackoverflow.com/a/54558861 > > > > > > > > > > > > > > > > Jiri, any tips on how to debug feature detection in > > > > > > > > tools/build/feature/Makefile? > > > > > > > > > > > > > > for quick check, there's output file for each test, like: > > > > > > > > > > > > > > [jolsa@krava feature]$ ls -l *.make.output > > > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-all.make.output > > > > > > > -rw-rw-r--. 1 jolsa jolsa 182 Apr 9 15:52 test-bionic.make.output > > > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-cplus-demangle.make.output > > > > > > > -rw-rw-r--. 1 jolsa jolsa 145 Apr 9 15:52 test-jvmti.make.output > > > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbabeltrace.make.output > > > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libbpf.make.output > > > > > > > -rw-rw-r--. 1 jolsa jolsa 0 Apr 8 20:25 test-libdebuginfod.make.output > > > > > > > -rw-rw-r--. 1 jolsa jolsa 193 Apr 9 15:52 test-libunwind-aarch64.make.output > > > > > > > -rw-rw-r--. 1 jolsa jolsa 177 Apr 9 15:52 test-libunwind-x86.make.output > > > > > > > [jolsa@krava feature]$ cat test-libunwind-aarch64.make.output > > > > > > > test-libunwind-aarch64.c:2:10: fatal error: libunwind-aarch64.h: No such file or directory > > > > > > > 2 | #include <libunwind-aarch64.h> > > > > > > > | ^~~~~~~~~~~~~~~~~~~~~ > > > > > > > compilation terminated. > > > > > > > [jolsa@krava feature]$ cat test-libunwind-x86.make.output > > > > > > > test-libunwind-x86.c:2:10: fatal error: libunwind-x86.h: No such file or directory > > > > > > > 2 | #include <libunwind-x86.h> > > > > > > > | ^~~~~~~~~~~~~~~~~ > > > > > > > > > > > > > > zlib should be done by: > > > > > > > [jolsa@krava feature]$ make test-zlib.bin > > > > > > > gcc -MD -Wall -Werror -o test-zlib.bin test-zlib.c > test-zlib.make.output 2>&1 -lz > > > > > > > > > > > > > > > > > > > > > I can try to recreate, how do you build? > > > > > > > > > > > > See note above, I'm similarly running precisely: > > > > > > $ make LLVM=1 LLVM_IAS=1 -j72 defconfig > > > > > > $ make LLVM=1 LLVM_IAS=1 -j72 clean > > > > > > $ make LLVM=1 LLVM_IAS=1 -j72 -C tools/testing/selftests/bpf > > > > > > > > > > for some reason I'm stuck with this error on latest bpf-next/master > > > > > > > > did you build vmlinux image before building selftests? those enums > > > > should come through vmlinux.h from up-to-date vmlinux > > > > > > it was there.. but I found the clang/lld I compiled can't link properly, > > > which is probably unrelated to the error below, but I need to solve it > > > first ;-) > > > > Do you have more info about what command you're running, or what error > > you're observing? The kernel itself is not linking for you? > > I did the commands you sent, but I might have but llvm/clang build, > I'm rebuilding now, it takes forever.. ok, finished all commands finaly.. did not reproduce the issue :-\ any luck with those build make.output files for zlib? jirka
To build kernel with clang, people typically use make -j60 LLVM=1 LLVM_IAS=1 LLVM_IAS=1 is not required for non-LTO build but is required for LTO build. In my environment, I am always having LLVM_IAS=1 regardless of whether LTO is enabled or not. After kernel is build with clang, the following command can be used to build selftests with clang: make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 But currently, using the above command, some compilations still use gcc and there are also compilation errors and warnings. This patch set intends to fix these issues. Patch #1 and #2 fixed the issue so clang/clang++ is used instead of gcc/g++. Patch #3 fixed a compilation failure. Patch #4 and #5 fixed various compiler warnings. Changelog: v1 -> v2: . add -Wno-unused-command-line-argument and -Wno-format-security for clang only as (1). gcc does not exhibit those warnings, and (2). -Wno-unused-command-line-argument is only supported by clang. (Sedat) Yonghong Song (5): selftests: set CC to clang in lib.mk if LLVM is set tools: allow proper CC/CXX/... override with LLVM=1 in Makefile.include selftests/bpf: fix test_cpp compilation failure with clang selftests/bpf: silence clang compilation warnings bpftool: fix a clang compilation warning tools/bpf/bpftool/net.c | 2 +- tools/scripts/Makefile.include | 12 ++++++++++-- tools/testing/selftests/bpf/Makefile | 7 ++++++- tools/testing/selftests/lib.mk | 4 ++++ 4 files changed, 21 insertions(+), 4 deletions(-)