| Message ID | 872b64e93de9a6cd6a7a10e6a5c5e7893704f743.1738276344.git.dxu@dxuuu.xyz (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | selftests: bpf: Support dynamic linking LLVM if static not available | expand |
On Thu, 2025-01-30 at 15:33 -0700, Daniel Xu wrote: > Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM > libraries"), only statically linking test_progs is supported. However, > some distros only provide a dynamically linkable LLVM. > > This commit adds a fallback for dynamically linking LLVM if static > linking is not available. If both options are available, static linking > is chosen. > > Signed-off-by: Daniel Xu <dxu@dxuuu.xyz> > --- Tested in two configuration: - when both static and dynamic libraries are available (linked against static); - when only static libraries are available. Tested-by: Eduard Zingerman <eddyz87@gmail.com> [...]
On 1/30/25 2:33 PM, Daniel Xu wrote: > Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM > libraries"), only statically linking test_progs is supported. However, > some distros only provide a dynamically linkable LLVM. > > This commit adds a fallback for dynamically linking LLVM if static > linking is not available. If both options are available, static linking > is chosen. > > Signed-off-by: Daniel Xu <dxu@dxuuu.xyz> LGTM. Acked-by: Yonghong Song <yonghong.song@linux.dev>
On 1/30/25 2:33 PM, Daniel Xu wrote: > Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM > libraries"), only statically linking test_progs is supported. However, > some distros only provide a dynamically linkable LLVM. > > This commit adds a fallback for dynamically linking LLVM if static > linking is not available. If both options are available, static linking > is chosen. > > Signed-off-by: Daniel Xu <dxu@dxuuu.xyz> > --- > tools/testing/selftests/bpf/Makefile | 11 ++++++++--- > 1 file changed, 8 insertions(+), 3 deletions(-) > > diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile > index 6722080b2107..da514030a153 100644 > --- a/tools/testing/selftests/bpf/Makefile > +++ b/tools/testing/selftests/bpf/Makefile > @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1) > LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets > # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict > LLVM_CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags)) > - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS)) > - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) > - LLVM_LDLIBS += -lstdc++ > + # Prefer linking statically if it's available, otherwise fallback to shared > + ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static) > + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS)) > + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) > + LLVM_LDLIBS += -lstdc++ > + else > + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS)) > + endif > LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags) > endif Although your change looks good, but maybe you can look at bpftool Makefile? # If LLVM is available, use it for JIT disassembly CFLAGS += -DHAVE_LLVM_SUPPORT LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS # (e.g. when bpftool build is called from selftests build as selftests # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause # compilation error due to redefinition. Let's filter it out here. CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags)) LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS)) ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static) LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) LIBS += -lstdc++ endif LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags) It would be great if the selftests shared library handling to be the same as bpftool's.
Hi Yonghong, On Thu, Jan 30, 2025 at 10:28:11PM -0800, Yonghong Song wrote: > > > > On 1/30/25 2:33 PM, Daniel Xu wrote: > > Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM > > libraries"), only statically linking test_progs is supported. However, > > some distros only provide a dynamically linkable LLVM. > > > > This commit adds a fallback for dynamically linking LLVM if static > > linking is not available. If both options are available, static linking > > is chosen. > > > > Signed-off-by: Daniel Xu <dxu@dxuuu.xyz> > > --- > > tools/testing/selftests/bpf/Makefile | 11 ++++++++--- > > 1 file changed, 8 insertions(+), 3 deletions(-) > > > > diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile > > index 6722080b2107..da514030a153 100644 > > --- a/tools/testing/selftests/bpf/Makefile > > +++ b/tools/testing/selftests/bpf/Makefile > > @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1) > > LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets > > # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict > > LLVM_CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags)) > > - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS)) > > - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) > > - LLVM_LDLIBS += -lstdc++ > > + # Prefer linking statically if it's available, otherwise fallback to shared > > + ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static) > > + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS)) > > + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) > > + LLVM_LDLIBS += -lstdc++ > > + else > > + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS)) > > + endif > > LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags) > > endif > > Although your change looks good, but maybe you can look at bpftool Makefile? > > # If LLVM is available, use it for JIT disassembly > CFLAGS += -DHAVE_LLVM_SUPPORT > LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets > # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS > # (e.g. when bpftool build is called from selftests build as selftests > # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause > # compilation error due to redefinition. Let's filter it out here. > CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags)) > LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS)) > ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static) > LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) > LIBS += -lstdc++ > endif > LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags) > > It would be great if the selftests shared library handling to be the same as bpftool's. So bpftool is both an internally consumed (from selftests) dependency as well as a tool packaged up by distros. For the latter case, distros prefer dynamic linking. So unfortunately, I think these probably need to be defined separately. The code looks similar but the use cases are different. Thanks, Daniel
On 2/1/25 12:23 AM, Daniel Xu wrote: > Hi Yonghong, > > On Thu, Jan 30, 2025 at 10:28:11PM -0800, Yonghong Song wrote: >> >> >> On 1/30/25 2:33 PM, Daniel Xu wrote: >>> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM >>> libraries"), only statically linking test_progs is supported. However, >>> some distros only provide a dynamically linkable LLVM. >>> >>> This commit adds a fallback for dynamically linking LLVM if static >>> linking is not available. If both options are available, static linking >>> is chosen. >>> >>> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz> >>> --- >>> tools/testing/selftests/bpf/Makefile | 11 ++++++++--- >>> 1 file changed, 8 insertions(+), 3 deletions(-) >>> >>> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile >>> index 6722080b2107..da514030a153 100644 >>> --- a/tools/testing/selftests/bpf/Makefile >>> +++ b/tools/testing/selftests/bpf/Makefile >>> @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1) >>> LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets >>> # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict >>> LLVM_CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags)) >>> - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS)) >>> - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) >>> - LLVM_LDLIBS += -lstdc++ >>> + # Prefer linking statically if it's available, otherwise fallback to shared >>> + ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static) >>> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS)) >>> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) >>> + LLVM_LDLIBS += -lstdc++ >>> + else >>> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS)) >>> + endif >>> LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags) >>> endif >> Although your change looks good, but maybe you can look at bpftool Makefile? >> >> # If LLVM is available, use it for JIT disassembly >> CFLAGS += -DHAVE_LLVM_SUPPORT >> LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets >> # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS >> # (e.g. when bpftool build is called from selftests build as selftests >> # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause >> # compilation error due to redefinition. Let's filter it out here. >> CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags)) >> LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS)) >> ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static) >> LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) >> LIBS += -lstdc++ >> endif >> LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags) >> >> It would be great if the selftests shared library handling to be the same as bpftool's. > So bpftool is both an internally consumed (from selftests) dependency as > well as a tool packaged up by distros. For the latter case, distros > prefer dynamic linking. I hacked llvm to have both static and shared libraries installed and indeed `llvm-config --shared-mode` prefers shared mode. So yes, your existing change looks good. Thanks. > > So unfortunately, I think these probably need to be defined separately. > The code looks similar but the use cases are different. > > Thanks, > Daniel
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 6722080b2107..da514030a153 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1) LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict LLVM_CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags)) - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS)) - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) - LLVM_LDLIBS += -lstdc++ + # Prefer linking statically if it's available, otherwise fallback to shared + ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static) + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS)) + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) + LLVM_LDLIBS += -lstdc++ + else + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS)) + endif LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags) endif
Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM libraries"), only statically linking test_progs is supported. However, some distros only provide a dynamically linkable LLVM. This commit adds a fallback for dynamically linking LLVM if static linking is not available. If both options are available, static linking is chosen. Signed-off-by: Daniel Xu <dxu@dxuuu.xyz> --- tools/testing/selftests/bpf/Makefile | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)