diff mbox series

[RFC] bpf: preload: Fix build error when O= is set

Message ID 20201119085022.3606135-1-davidgow@google.com (mailing list archive)
State RFC
Delegated to: BPF
Headers show
Series [RFC] bpf: preload: Fix build error when O= is set | expand

Checks

Context Check Description
netdev/tree_selection success Not a local patch

Commit Message

David Gow Nov. 19, 2020, 8:50 a.m. UTC
If BPF_PRELOAD is enabled, and an out-of-tree build is requested with
make O=<path>, compilation seems to fail with:

tools/scripts/Makefile.include:4: *** O=.kunit does not exist.  Stop.
make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2
make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2
make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [.../Makefile:1799: kernel] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:185: __sub-make] Error 2

By the looks of things, this is because the (relative path) O= passed on
the command line is being passed to the libbpf Makefile, which then
can't find the directory. Given OUTPUT= is being passed anyway, we can
work around this by explicitly setting an empty O=, which will be
ignored in favour of OUTPUT= in tools/scripts/Makefile.include.

Signed-off-by: David Gow <davidgow@google.com>
---

Hi all,

I'm not 100% sure this is the correct fix here -- it seems to work for
me, and makes some sense, but let me know if there's a better way.

One other thing worth noting is that I've been hitting this with
make allyesconfig on ARCH=um, but there's a comment in the Kconfig
suggesting that, because BPF_PRELOAD depends on !COMPILE_TEST, that
maybe it shouldn't be being built at all. I figured that it was worth
trying to fix this anyway.

Cheers,
-- David


 kernel/bpf/preload/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Brendan Higgins Nov. 19, 2020, 9:14 p.m. UTC | #1
On Thu, Nov 19, 2020 at 12:50 AM David Gow <davidgow@google.com> wrote:
>
> If BPF_PRELOAD is enabled, and an out-of-tree build is requested with
> make O=<path>, compilation seems to fail with:
>
> tools/scripts/Makefile.include:4: *** O=.kunit does not exist.  Stop.
> make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2
> make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2
> make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [.../Makefile:1799: kernel] Error 2
> make[1]: *** Waiting for unfinished jobs....
> make: *** [Makefile:185: __sub-make] Error 2
>
> By the looks of things, this is because the (relative path) O= passed on
> the command line is being passed to the libbpf Makefile, which then
> can't find the directory. Given OUTPUT= is being passed anyway, we can
> work around this by explicitly setting an empty O=, which will be
> ignored in favour of OUTPUT= in tools/scripts/Makefile.include.
>
> Signed-off-by: David Gow <davidgow@google.com>

Seems sensible to me. I have no strong feeling as to whether we just
turn this off on UML or whether we do the fix you proposed here
though. Nevertheless, I would like to see *some* fix go in before
v5.10 is released.

Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Andrii Nakryiko Nov. 21, 2020, 7:37 a.m. UTC | #2
On Thu, Nov 19, 2020 at 12:51 AM David Gow <davidgow@google.com> wrote:
>
> If BPF_PRELOAD is enabled, and an out-of-tree build is requested with
> make O=<path>, compilation seems to fail with:
>
> tools/scripts/Makefile.include:4: *** O=.kunit does not exist.  Stop.
> make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2
> make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2
> make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [.../Makefile:1799: kernel] Error 2
> make[1]: *** Waiting for unfinished jobs....
> make: *** [Makefile:185: __sub-make] Error 2
>
> By the looks of things, this is because the (relative path) O= passed on
> the command line is being passed to the libbpf Makefile, which then
> can't find the directory. Given OUTPUT= is being passed anyway, we can
> work around this by explicitly setting an empty O=, which will be
> ignored in favour of OUTPUT= in tools/scripts/Makefile.include.

Strange, but I can't repro it. I use make O=<absolute path> all the
time with no issues. I just tried specifically with a make O=.build,
where .build is inside Linux repo, and it still worked fine. See also
be40920fbf10 ("tools: Let O= makes handle a relative path with -C
option") which was supposed to address such an issue. So I'm wondering
what exactly is causing this problem.

>
> Signed-off-by: David Gow <davidgow@google.com>
> ---
>
> Hi all,
>
> I'm not 100% sure this is the correct fix here -- it seems to work for
> me, and makes some sense, but let me know if there's a better way.
>
> One other thing worth noting is that I've been hitting this with
> make allyesconfig on ARCH=um, but there's a comment in the Kconfig
> suggesting that, because BPF_PRELOAD depends on !COMPILE_TEST, that
> maybe it shouldn't be being built at all. I figured that it was worth
> trying to fix this anyway.
>
> Cheers,
> -- David
>
>
>  kernel/bpf/preload/Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
> index 23ee310b6eb4..39848d296097 100644
> --- a/kernel/bpf/preload/Makefile
> +++ b/kernel/bpf/preload/Makefile
> @@ -5,7 +5,7 @@ LIBBPF_A = $(obj)/libbpf.a
>  LIBBPF_OUT = $(abspath $(obj))
>
>  $(LIBBPF_A):
> -       $(Q)$(MAKE) -C $(LIBBPF_SRCS) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
> +       $(Q)$(MAKE) -C $(LIBBPF_SRCS) O= OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
>
>  userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
>         -I $(srctree)/tools/lib/ -Wno-unused-result
> --
> 2.29.2.454.gaff20da3a2-goog
>
David Gow Nov. 21, 2020, 9:48 a.m. UTC | #3
On Sat, Nov 21, 2020 at 3:38 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Nov 19, 2020 at 12:51 AM David Gow <davidgow@google.com> wrote:
> >
> > If BPF_PRELOAD is enabled, and an out-of-tree build is requested with
> > make O=<path>, compilation seems to fail with:
> >
> > tools/scripts/Makefile.include:4: *** O=.kunit does not exist.  Stop.
> > make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2
> > make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2
> > make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2
> > make[2]: *** Waiting for unfinished jobs....
> > make[1]: *** [.../Makefile:1799: kernel] Error 2
> > make[1]: *** Waiting for unfinished jobs....
> > make: *** [Makefile:185: __sub-make] Error 2
> >
> > By the looks of things, this is because the (relative path) O= passed on
> > the command line is being passed to the libbpf Makefile, which then
> > can't find the directory. Given OUTPUT= is being passed anyway, we can
> > work around this by explicitly setting an empty O=, which will be
> > ignored in favour of OUTPUT= in tools/scripts/Makefile.include.
>
> Strange, but I can't repro it. I use make O=<absolute path> all the
> time with no issues. I just tried specifically with a make O=.build,
> where .build is inside Linux repo, and it still worked fine. See also
> be40920fbf10 ("tools: Let O= makes handle a relative path with -C
> option") which was supposed to address such an issue. So I'm wondering
> what exactly is causing this problem.
>
[+ linux-um list]

Hmm... From a quick check, I can't reproduce this on x86, so it's
possibly a UML-specific issue.

The problem here seems to be that $PWD is, for whatever reason, equal
to the srcdir on x86, but not on UML. In general, $PWD behaves pretty
weirdly -- I don't fully understand it -- but if I add a tactical "PWD
:= $(shell pwd)" or use $(CURDIR) instead, the issue shows up on x86
as well. I guess this is because PWD only gets updated when set by a
shell or something, and UML does this somewhere?

Thoughts?

Cheers,
-- David

> >
> > Signed-off-by: David Gow <davidgow@google.com>
> > ---
> >
> > Hi all,
> >
> > I'm not 100% sure this is the correct fix here -- it seems to work for
> > me, and makes some sense, but let me know if there's a better way.
> >
> > One other thing worth noting is that I've been hitting this with
> > make allyesconfig on ARCH=um, but there's a comment in the Kconfig
> > suggesting that, because BPF_PRELOAD depends on !COMPILE_TEST, that
> > maybe it shouldn't be being built at all. I figured that it was worth
> > trying to fix this anyway.
> >
> > Cheers,
> > -- David
> >
> >
> >  kernel/bpf/preload/Makefile | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
> > index 23ee310b6eb4..39848d296097 100644
> > --- a/kernel/bpf/preload/Makefile
> > +++ b/kernel/bpf/preload/Makefile
> > @@ -5,7 +5,7 @@ LIBBPF_A = $(obj)/libbpf.a
> >  LIBBPF_OUT = $(abspath $(obj))
> >
> >  $(LIBBPF_A):
> > -       $(Q)$(MAKE) -C $(LIBBPF_SRCS) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
> > +       $(Q)$(MAKE) -C $(LIBBPF_SRCS) O= OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
> >
> >  userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
> >         -I $(srctree)/tools/lib/ -Wno-unused-result
> > --
> > 2.29.2.454.gaff20da3a2-goog
> >
Quentin Monnet Dec. 16, 2020, 2:53 p.m. UTC | #4
2020-11-21 17:48 UTC+0800 ~ David Gow <davidgow@google.com>
> On Sat, Nov 21, 2020 at 3:38 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
>>
>> On Thu, Nov 19, 2020 at 12:51 AM David Gow <davidgow@google.com> wrote:
>>>
>>> If BPF_PRELOAD is enabled, and an out-of-tree build is requested with
>>> make O=<path>, compilation seems to fail with:
>>>
>>> tools/scripts/Makefile.include:4: *** O=.kunit does not exist.  Stop.
>>> make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2
>>> make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2
>>> make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2
>>> make[2]: *** Waiting for unfinished jobs....
>>> make[1]: *** [.../Makefile:1799: kernel] Error 2
>>> make[1]: *** Waiting for unfinished jobs....
>>> make: *** [Makefile:185: __sub-make] Error 2
>>>
>>> By the looks of things, this is because the (relative path) O= passed on
>>> the command line is being passed to the libbpf Makefile, which then
>>> can't find the directory. Given OUTPUT= is being passed anyway, we can
>>> work around this by explicitly setting an empty O=, which will be
>>> ignored in favour of OUTPUT= in tools/scripts/Makefile.include.
>>
>> Strange, but I can't repro it. I use make O=<absolute path> all the
>> time with no issues. I just tried specifically with a make O=.build,
>> where .build is inside Linux repo, and it still worked fine. See also
>> be40920fbf10 ("tools: Let O= makes handle a relative path with -C
>> option") which was supposed to address such an issue. So I'm wondering
>> what exactly is causing this problem.
>>
> [+ linux-um list]
> 
> Hmm... From a quick check, I can't reproduce this on x86, so it's
> possibly a UML-specific issue.
> 
> The problem here seems to be that $PWD is, for whatever reason, equal
> to the srcdir on x86, but not on UML. In general, $PWD behaves pretty
> weirdly -- I don't fully understand it -- but if I add a tactical "PWD
> := $(shell pwd)" or use $(CURDIR) instead, the issue shows up on x86
> as well. I guess this is because PWD only gets updated when set by a
> shell or something, and UML does this somewhere?
> 
> Thoughts?
> 
> Cheers,
> -- David

Hi David, Andrii,

David, did you use a different command for building for UML and x86? I'm
asking because I reproduce on x86, but only for some targets, in
particular when I tried bindeb-pkg.

With "make O=.build vmlinux", I have:
- $(O) for "dummy" check in tools/scripts/Makefile.include set to
/linux/.build
- $(PWD) for same check set to /linux/tools
- Since $(O) is an absolute path, the "dummy" check passes

With "make O=.build bindeb-pkg", I have instead:
- $(O) set to .build (relative path)
- $(PWD) set to /linux/.build
- "dummy" check changes to /linux/.build and searches for .build in it,
which fails and aborts the build

(tools/scripts/Makefile.include is included from libbpf's Makefile,
called from kernel/bpf/preload/Makefile.)

I'm not sure how exactly the bindeb-pkg target ends up passing these values.

For what it's worth, I have been solving this (before finding this
thread) with a fix close to yours, I pass "O=$(abspath .)" on the
command line for building libbpf in kernel/bpf/preload/Makefile. It
looked consistent to me with the "tools/:" target from the main
Makefile, where "O=$(abspath $(objtree))" is passed (and $(objtree) is ".").

I hope this helps,
Quentin
David Gow Dec. 17, 2020, 9:05 a.m. UTC | #5
On Wed, Dec 16, 2020 at 10:53 PM Quentin Monnet <quentin@isovalent.com> wrote:
>
> 2020-11-21 17:48 UTC+0800 ~ David Gow <davidgow@google.com>
> > On Sat, Nov 21, 2020 at 3:38 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> >>
> >> On Thu, Nov 19, 2020 at 12:51 AM David Gow <davidgow@google.com> wrote:
> >>>
> >>> If BPF_PRELOAD is enabled, and an out-of-tree build is requested with
> >>> make O=<path>, compilation seems to fail with:
> >>>
> >>> tools/scripts/Makefile.include:4: *** O=.kunit does not exist.  Stop.
> >>> make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2
> >>> make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2
> >>> make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2
> >>> make[2]: *** Waiting for unfinished jobs....
> >>> make[1]: *** [.../Makefile:1799: kernel] Error 2
> >>> make[1]: *** Waiting for unfinished jobs....
> >>> make: *** [Makefile:185: __sub-make] Error 2
> >>>
> >>> By the looks of things, this is because the (relative path) O= passed on
> >>> the command line is being passed to the libbpf Makefile, which then
> >>> can't find the directory. Given OUTPUT= is being passed anyway, we can
> >>> work around this by explicitly setting an empty O=, which will be
> >>> ignored in favour of OUTPUT= in tools/scripts/Makefile.include.
> >>
> >> Strange, but I can't repro it. I use make O=<absolute path> all the
> >> time with no issues. I just tried specifically with a make O=.build,
> >> where .build is inside Linux repo, and it still worked fine. See also
> >> be40920fbf10 ("tools: Let O= makes handle a relative path with -C
> >> option") which was supposed to address such an issue. So I'm wondering
> >> what exactly is causing this problem.
> >>
> > [+ linux-um list]
> >
> > Hmm... From a quick check, I can't reproduce this on x86, so it's
> > possibly a UML-specific issue.
> >
> > The problem here seems to be that $PWD is, for whatever reason, equal
> > to the srcdir on x86, but not on UML. In general, $PWD behaves pretty
> > weirdly -- I don't fully understand it -- but if I add a tactical "PWD
> > := $(shell pwd)" or use $(CURDIR) instead, the issue shows up on x86
> > as well. I guess this is because PWD only gets updated when set by a
> > shell or something, and UML does this somewhere?
> >
> > Thoughts?
> >
> > Cheers,
> > -- David
>
> Hi David, Andrii,
>
> David, did you use a different command for building for UML and x86? I'm
> asking because I reproduce on x86, but only for some targets, in
> particular when I tried bindeb-pkg.

I just ran "make ARCH={x86,um} O=.bpftest", with defconfig + enabling
BPF_PRELOAD and its dependencies. UML fails, x86 works. (Though I can
reproduce the failure if I make bindeb-pkg on x86).

(It also shows up when building UML with the allyesconfig-based KUnit
alltests option by running "./tools/testing/kunit/kunit.py run
--alltests", though this understandably takes a long time and is less
obvious)
>
> With "make O=.build vmlinux", I have:
> - $(O) for "dummy" check in tools/scripts/Makefile.include set to
> /linux/.build
> - $(PWD) for same check set to /linux/tools
> - Since $(O) is an absolute path, the "dummy" check passes
>
> With "make O=.build bindeb-pkg", I have instead:
> - $(O) set to .build (relative path)
> - $(PWD) set to /linux/.build
> - "dummy" check changes to /linux/.build and searches for .build in it,
> which fails and aborts the build
>
> (tools/scripts/Makefile.include is included from libbpf's Makefile,
> called from kernel/bpf/preload/Makefile.)
>
> I'm not sure how exactly the bindeb-pkg target ends up passing these values.

Yeah: I haven't been able to find where uml is changing them either:
I'm assuming there's something which changes directory and/or spawns a
shell/recursive make to change $(PWD) or something.

> For what it's worth, I have been solving this (before finding this
> thread) with a fix close to yours, I pass "O=$(abspath .)" on the
> command line for building libbpf in kernel/bpf/preload/Makefile. It
> looked consistent to me with the "tools/:" target from the main
> Makefile, where "O=$(abspath $(objtree))" is passed (and $(objtree) is ".").

Given that there are several targets being broken here, it's probably
worth having a fix like this which overrides O= rather than trying to
hunt down every target which could change $(PWD). I don't particularly
mind whether we use O= or O=$(abspath .), both are working in the UML
usecase as well.

Does anyone object to basically accepting either this patch as-is, or
using O=$(abspath .)?


Cheers,
-- David
diff mbox series

Patch

diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
index 23ee310b6eb4..39848d296097 100644
--- a/kernel/bpf/preload/Makefile
+++ b/kernel/bpf/preload/Makefile
@@ -5,7 +5,7 @@  LIBBPF_A = $(obj)/libbpf.a
 LIBBPF_OUT = $(abspath $(obj))
 
 $(LIBBPF_A):
-	$(Q)$(MAKE) -C $(LIBBPF_SRCS) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
+	$(Q)$(MAKE) -C $(LIBBPF_SRCS) O= OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
 
 userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
 	-I $(srctree)/tools/lib/ -Wno-unused-result