diff mbox

[v5,5/8] kbuild: add fine grained build dependencies for exported symbols

Message ID 20160305071208.GA25757@pobox.suse.cz (mailing list archive)
State New, archived
Headers show

Commit Message

Michal Marek March 5, 2016, 7:12 a.m. UTC
On Fri, Mar 04, 2016 at 11:53:53PM +0100, Michal Marek wrote:
> Dne 4.3.2016 v 23:51 Michal Marek napsal(a):
> > Dne 4.3.2016 v 06:40 Nicolas Pitre napsal(a):
> >> +cmd_and_fixdep =                                                             \
> >> +	$(echo-cmd) $(cmd_$(1));                                             \
> >> +	$(ksym_dep_filter) |                                                 \
> >> +		scripts/basic/fixdep -e $(depfile) $@ '$(make-cmd)'          \
> >> +			> $(dot-target).tmp;	                             \
> >> +	rm -f $(depfile);                                                    \
> >> +	mv -f $(dot-target).tmp $(dot-target).cmd;
> > 
> > While trying this, I got a SIGBUS from fixdep once. My theory is that
> > the depfile is mmap()ed by fixdep and modified by the preprocesor run at
> > the same time. I could not reproduce this so far (still trying). But if
> > it's really this race, the fix would be to disable dependency generation
> > in the preprocessor by passing -Wp,MD,/dev/null or somesuch. But we
> > never had this problem with genksyms, which is weird. It could as well
> > be that my build machine's memory is faulty :(.
> 
> Actually, genksyms does not ran in parallel. neither before nor after
                              run
> this patch.

I reproduced the SIGBUS after a few iterations, and it crashes in
parse_dep_file(). I'm now testing this


Michal
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Nicolas Pitre March 6, 2016, 2:33 a.m. UTC | #1
On Sat, 5 Mar 2016, Michal Marek wrote:

> On Fri, Mar 04, 2016 at 11:53:53PM +0100, Michal Marek wrote:
> > Dne 4.3.2016 v 23:51 Michal Marek napsal(a):
> > > Dne 4.3.2016 v 06:40 Nicolas Pitre napsal(a):
> > >> +cmd_and_fixdep =                                                             \
> > >> +	$(echo-cmd) $(cmd_$(1));                                             \
> > >> +	$(ksym_dep_filter) |                                                 \
> > >> +		scripts/basic/fixdep -e $(depfile) $@ '$(make-cmd)'          \
> > >> +			> $(dot-target).tmp;	                             \
> > >> +	rm -f $(depfile);                                                    \
> > >> +	mv -f $(dot-target).tmp $(dot-target).cmd;
> > > 
> > > While trying this, I got a SIGBUS from fixdep once. My theory is that
> > > the depfile is mmap()ed by fixdep and modified by the preprocesor run at
> > > the same time. I could not reproduce this so far (still trying). But if
> > > it's really this race, the fix would be to disable dependency generation
> > > in the preprocessor by passing -Wp,MD,/dev/null or somesuch. But we
> > > never had this problem with genksyms, which is weird. It could as well
> > > be that my build machine's memory is faulty :(.
> > 
> > Actually, genksyms does not ran in parallel. neither before nor after
>                               run
> > this patch.
> 
> I reproduced the SIGBUS after a few iterations, and it crashes in
> parse_dep_file(). I'm now testing this
> 
> diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
> index 0be7e09ba381..4fdd8348acf6 100644
> --- a/scripts/Kbuild.include
> +++ b/scripts/Kbuild.include
> @@ -270,10 +270,12 @@ else
>  
>  # Filter out exported kernel symbol names from the preprocessor output.
>  # See also __KSYM_DEPS__ in include/linux/export.h.
> +# We disable the depfile generation here, so as not to overwrite the existing
> +# depfile while fixdep is parsing it
>  ksym_dep_filter =                                                            \
>  	case "$(1)" in                                                       \
> -	cc_*_c) $(CPP) $(c_flags) -D__KSYM_DEPS__ $< ;;                      \
> -	as_*_S) $(CPP) $(a_flags) -D__KSYM_DEPS__ $< ;;                      \
> +	cc_*_c) $(CPP) $(filter-out -Wp$(comma)-M%, $(c_flags)) -D__KSYM_DEPS__ $< ;; \
> +	as_*_S) $(CPP) $(filter-out -Wp$(comma)-M%, $(a_flags)) -D__KSYM_DEPS__ $< ;; \
>  	cpp_lds_S) : ;;                                                      \
>  	*) echo "Don't know how to preprocess $(1)"; false ;;                \
>  	esac | sed -rn 's/^.*=== __KSYM_(.*) ===.*$$/KSYM_\1/p'

This makes perfect sense even if I can't reproduce on my side.

Are you willing to fold this patch in, or do you prefer that I repost?


Nicolas
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index 0be7e09ba381..4fdd8348acf6 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -270,10 +270,12 @@  else
 
 # Filter out exported kernel symbol names from the preprocessor output.
 # See also __KSYM_DEPS__ in include/linux/export.h.
+# We disable the depfile generation here, so as not to overwrite the existing
+# depfile while fixdep is parsing it
 ksym_dep_filter =                                                            \
 	case "$(1)" in                                                       \
-	cc_*_c) $(CPP) $(c_flags) -D__KSYM_DEPS__ $< ;;                      \
-	as_*_S) $(CPP) $(a_flags) -D__KSYM_DEPS__ $< ;;                      \
+	cc_*_c) $(CPP) $(filter-out -Wp$(comma)-M%, $(c_flags)) -D__KSYM_DEPS__ $< ;; \
+	as_*_S) $(CPP) $(filter-out -Wp$(comma)-M%, $(a_flags)) -D__KSYM_DEPS__ $< ;; \
 	cpp_lds_S) : ;;                                                      \
 	*) echo "Don't know how to preprocess $(1)"; false ;;                \
 	esac | sed -rn 's/^.*=== __KSYM_(.*) ===.*$$/KSYM_\1/p'