From patchwork Fri May 27 10:01:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 12863190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23567C433EF for ; Fri, 27 May 2022 10:04:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241297AbiE0KEh (ORCPT ); Fri, 27 May 2022 06:04:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350767AbiE0KEf (ORCPT ); Fri, 27 May 2022 06:04:35 -0400 Received: from conuserg-08.nifty.com (conuserg-08.nifty.com [210.131.2.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70C981116D2; Fri, 27 May 2022 03:04:32 -0700 (PDT) Received: from grover.sesame (133-32-177-133.west.xps.vectant.ne.jp [133.32.177.133]) (authenticated) by conuserg-08.nifty.com with ESMTP id 24RA2hPR029808; Fri, 27 May 2022 19:02:45 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-08.nifty.com 24RA2hPR029808 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1653645765; bh=qwd23UnZbO+5y7U7znABEi6dbpFFNIV0E++LnxqEYTU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mBTnB1DbKOnCmX5Ya77nr4wiY61xCAd2gDVGZleMKQiezhhRhnn5kcWqlcmPRVA68 UfK15SyTlR56+ygHUSR1HFmo5wuekHdC+np4qWSLtNaCqiwqeu+6qNM1GMGSII4a1y NI6ntzouNqgU8FYUcdnaZVDpeWLoUuIx5StuCA/S65ebk2424eIKQhY+iphgwiR/rG LDJyNveUt1Yg2z2YSQ+Ud66mKIjX+UpnItrhq9abbiz+3jY2dYxqeqBaInsuUv1Clq WM7nRpQAXN+6s5hbIQxRd1+8l26aYDPi3LTrXIqJFttc0dKni0YR/6G2WKOyCb/JHA F1CMKUgoAlaTA== X-Nifty-SrcIP: [133.32.177.133] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Josh Poimboeuf , llvm@lists.linux.dev, Helge Deller , linux-parisc@vger.kernel.org, Masahiro Yamada , Michal Marek , Nick Desaulniers Subject: [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options Date: Fri, 27 May 2022 19:01:48 +0900 Message-Id: <20220527100155.1996314-2-masahiroy@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220527100155.1996314-1-masahiroy@kernel.org> References: <20220527100155.1996314-1-masahiroy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org *.prelink.o is created when CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT is enabled. Replace $(linked-object) with $(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT) so you will get better idea when the --link option is passed. No functional change is intended. Signed-off-by: Masahiro Yamada Acked-by: Josh Poimboeuf Reviewed-by: Nick Desaulniers --- Changes in v7: - New patch scripts/Makefile.build | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 06400504150b..f80196eef03a 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -219,7 +219,7 @@ objtool_args = \ $(if $(CONFIG_STACK_VALIDATION), --stackval) \ $(if $(CONFIG_HAVE_STATIC_CALL_INLINE), --static-call) \ --uaccess \ - $(if $(linked-object), --link) \ + $(if $($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)), --link) \ $(if $(part-of-module), --module) \ $(if $(CONFIG_GCOV_KERNEL), --no-unreachable) @@ -284,7 +284,6 @@ quiet_cmd_cc_prelink_modules = LD [M] $@ # modules into native code $(obj)/%.prelink.o: objtool-enabled = y $(obj)/%.prelink.o: part-of-module := y -$(obj)/%.prelink.o: linked-object := y $(obj)/%.prelink.o: $(obj)/%.o FORCE $(call if_changed,cc_prelink_modules) From patchwork Fri May 27 10:01:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 12863196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0511C4332F for ; Fri, 27 May 2022 10:05:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350792AbiE0KF2 (ORCPT ); Fri, 27 May 2022 06:05:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350933AbiE0KFR (ORCPT ); Fri, 27 May 2022 06:05:17 -0400 Received: from conuserg-08.nifty.com (conuserg-08.nifty.com [210.131.2.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57B77106577; Fri, 27 May 2022 03:05:15 -0700 (PDT) Received: from grover.sesame (133-32-177-133.west.xps.vectant.ne.jp [133.32.177.133]) (authenticated) by conuserg-08.nifty.com with ESMTP id 24RA2hPS029808; Fri, 27 May 2022 19:02:46 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-08.nifty.com 24RA2hPS029808 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1653645767; bh=SoYyI3Vl79XcBz6rKgrVB3f/gawTaflZ8wkhRbY1iyg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XmDVyTj3RAdPhHU5BIen4d8chFB1j4LYP9aySeubfNAwwwx54tXgW0uenkt6cpMQl 9yENuYy1d66gB+F3GIQV49sgVIuEjVQDfH8OReZtfb/It9F8aIRVcZLuNYj28BKkhM qbVqEoIqTX3G7SzsawvdryZyxV3uIsZ0vIr4vXf0GDVjObrGqvnTASdYzhOPK0a3MD TgF5G4ubxLppVzjsF5kAGuijrMV8LMCUPVz9MbOyTMysOkOauMU2ITO65zN/VFqCis XMrYu7jZI51VjCihrFNt7eJOU/tqBFEKYqkh0yAKQAt5GUPKDso8RGPdgya0ItmQ6b PpUQdTCzHPMVg== X-Nifty-SrcIP: [133.32.177.133] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Josh Poimboeuf , llvm@lists.linux.dev, Helge Deller , linux-parisc@vger.kernel.org, Masahiro Yamada , Nicolas Schier , Nathan Chancellor , Sami Tolvanen , Sedat Dilek , Michal Marek , Nick Desaulniers , Tom Rix Subject: [PATCH v7 2/8] kbuild: do not create *.prelink.o for Clang LTO or IBT Date: Fri, 27 May 2022 19:01:49 +0900 Message-Id: <20220527100155.1996314-3-masahiroy@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220527100155.1996314-1-masahiroy@kernel.org> References: <20220527100155.1996314-1-masahiroy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org When CONFIG_LTO_CLANG=y, additional intermediate *.prelink.o is created for each module. Also, objtool is postponed until LLVM IR is converted to ELF. CONFIG_X86_KERNEL_IBT works in a similar way to postpone objtool until objects are merged together. This commit stops generating *.prelink.o, so the build flow will look similar with/without LTO. The following figures show how the LTO build currently works, and how this commit is changing it. Current build flow ================== [1] single-object module $(LD) $(CC) +objtool $(LD) foo.c --------------------> foo.o -----> foo.prelink.o -----> foo.ko (LLVM IR) (ELF) | | foo.mod.o --/ [2] multi-object module $(LD) $(CC) $(AR) +objtool $(LD) foo1.c -----> foo1.o -----> foo.o -----> foo.prelink.o -----> foo.ko | (archive) (ELF) | foo2.c -----> foo2.o --/ | (LLVM IR) foo.mod.o --/ One confusion is foo.o in multi-object module is an archive despite of its suffix. New build flow ============== [1] single-object module Since there is only one object, there is no need to keep the LLVM IR. Use $(CC)+$(LD) to generate an ELF object in one build rule. When LTO is disabled, $(LD) is unneeded because $(CC) produces an ELF object. $(CC)+$(LD)+objtool $(LD) foo.c ----------------------------> foo.o ---------> foo.ko (ELF) | | foo.mod.o --/ [2] multi-object module Previously, $(AR) was used to combine LLVM bitcode into an archive, but there was no technical reason to do so. Use $(LD) to merge them into a single ELF object. $(LD) $(CC) +objtool $(LD) foo1.c ---------> foo1.o ---------> foo.o ---------> foo.ko | (ELF) | foo2.c ---------> foo2.o ----/ | (LLVM IR) foo.mod.o --/ Signed-off-by: Masahiro Yamada Reviewed-by: Nicolas Schier Tested-by: Nathan Chancellor Reviewed-by: Sami Tolvanen Tested-by: Sedat Dilek # LLVM-14 (x86-64) Acked-by: Josh Poimboeuf --- (no changes since v2) Changes in v2: - replace the chain of $(if ...) with $(and ) scripts/Kbuild.include | 4 +++ scripts/Makefile.build | 60 ++++++++++++--------------------------- scripts/Makefile.lib | 7 ----- scripts/Makefile.modfinal | 5 ++-- scripts/Makefile.modpost | 9 ++---- scripts/mod/modpost.c | 7 ----- 6 files changed, 26 insertions(+), 66 deletions(-) diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include index 3514c2149e9d..455a0a6ce12d 100644 --- a/scripts/Kbuild.include +++ b/scripts/Kbuild.include @@ -15,6 +15,10 @@ pound := \# # Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o dot-target = $(dir $@).$(notdir $@) +### +# Name of target with a '.tmp_' as filename prefix. foo/bar.o => foo/.tmp_bar.o +tmp-target = $(dir $@).tmp_$(notdir $@) + ### # The temporary file to save gcc -MMD generated dependencies must not # contain a comma diff --git a/scripts/Makefile.build b/scripts/Makefile.build index f80196eef03a..23e1fb19f9ad 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -88,10 +88,6 @@ endif targets-for-modules := $(foreach x, o mod $(if $(CONFIG_TRIM_UNUSED_KSYMS), usyms), \ $(patsubst %.o, %.$x, $(filter %.o, $(obj-m)))) -ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),) -targets-for-modules += $(patsubst %.o, %.prelink.o, $(filter %.o, $(obj-m))) -endif - ifdef need-modorder targets-for-modules += $(obj)/modules.order endif @@ -152,8 +148,16 @@ $(obj)/%.ll: $(src)/%.c FORCE # The C file is compiled and updated dependency information is generated. # (See cmd_cc_o_c + relevant part of rule_cc_o_c) +is-single-obj-m = $(and $(part-of-module),$(filter $@, $(obj-m)),y) + +ifdef CONFIG_LTO_CLANG +cmd_ld_single_m = $(if $(is-single-obj-m), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@) +endif + quiet_cmd_cc_o_c = CC $(quiet_modtag) $@ - cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $< $(cmd_objtool) + cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $< \ + $(cmd_ld_single_m) \ + $(cmd_objtool) ifdef CONFIG_MODVERSIONS # When module versioning is enabled the following steps are executed: @@ -219,7 +223,7 @@ objtool_args = \ $(if $(CONFIG_STACK_VALIDATION), --stackval) \ $(if $(CONFIG_HAVE_STATIC_CALL_INLINE), --static-call) \ --uaccess \ - $(if $($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)), --link) \ + $(if $(delay-objtool), --link) \ $(if $(part-of-module), --module) \ $(if $(CONFIG_GCOV_KERNEL), --no-unreachable) @@ -228,21 +232,16 @@ cmd_gen_objtooldep = $(if $(objtool-enabled), { echo ; echo '$@: $$(wildcard $(o endif # CONFIG_OBJTOOL -ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),) - -# Skip objtool for LLVM bitcode -$(obj)/%.o: objtool-enabled := - -else # 'OBJECT_FILES_NON_STANDARD := y': skip objtool checking for a directory # 'OBJECT_FILES_NON_STANDARD_foo.o := 'y': skip objtool checking for a file # 'OBJECT_FILES_NON_STANDARD_foo.o := 'n': override directory skip for a file -$(obj)/%.o: objtool-enabled = $(if $(filter-out y%, \ - $(OBJECT_FILES_NON_STANDARD_$(basetarget).o)$(OBJECT_FILES_NON_STANDARD)n),y) +is-standard-object = $(if $(filter-out y%, $(OBJECT_FILES_NON_STANDARD_$(basetarget).o)$(OBJECT_FILES_NON_STANDARD)n),y) -endif +delay-objtool := $(or $(CONFIG_LTO_CLANG),$(CONFIG_X86_KERNEL_IBT)) + +$(obj)/%.o: objtool-enabled = $(if $(is-standard-object),$(if $(delay-objtool),$(is-single-obj-m),y)) ifdef CONFIG_TRIM_UNUSED_KSYMS cmd_gen_ksymdeps = \ @@ -271,24 +270,6 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE $(call if_changed_rule,cc_o_c) $(call cmd,force_checksrc) -ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),) -# Module .o files may contain LLVM bitcode, compile them into native code -# before ELF processing -quiet_cmd_cc_prelink_modules = LD [M] $@ - cmd_cc_prelink_modules = \ - $(LD) $(ld_flags) -r -o $@ \ - --whole-archive $(filter-out FORCE,$^) \ - $(cmd_objtool) - -# objtool was skipped for LLVM bitcode, run it now that we have compiled -# modules into native code -$(obj)/%.prelink.o: objtool-enabled = y -$(obj)/%.prelink.o: part-of-module := y - -$(obj)/%.prelink.o: $(obj)/%.o FORCE - $(call if_changed,cc_prelink_modules) -endif - cmd_mod = echo $(addprefix $(obj)/, $(call real-search, $*.o, .o, -objs -y -m)) | \ $(AWK) -v RS='( |\n)' '!x[$$0]++' > $@ @@ -298,7 +279,7 @@ $(obj)/%.mod: FORCE # List module undefined symbols cmd_undefined_syms = $(NM) $< | sed -n 's/^ *U //p' > $@ -$(obj)/%.usyms: $(obj)/%$(mod-prelink-ext).o FORCE +$(obj)/%.usyms: $(obj)/%.o FORCE $(call if_changed,undefined_syms) quiet_cmd_cc_lst_c = MKLST $@ @@ -420,16 +401,11 @@ $(obj)/modules.order: $(obj-m) FORCE $(obj)/lib.a: $(lib-y) FORCE $(call if_changed,ar) -ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),) -quiet_cmd_link_multi-m = AR [M] $@ -cmd_link_multi-m = \ - rm -f $@; \ - $(AR) cDPrsT $@ @$(patsubst %.o,%.mod,$@) -else quiet_cmd_link_multi-m = LD [M] $@ - cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) -endif + cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool) +$(multi-obj-m): objtool-enabled := $(delay-objtool) +$(multi-obj-m): part-of-module := y $(multi-obj-m): %.o: %.mod FORCE $(call if_changed,link_multi-m) $(call multi_depend, $(multi-obj-m), .o, -objs -y -m) diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 0453a1904646..f75138385449 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -225,13 +225,6 @@ dtc_cpp_flags = -Wp,-MMD,$(depfile).pre.tmp -nostdinc \ $(addprefix -I,$(DTC_INCLUDE)) \ -undef -D__DTS__ -ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),) -# With CONFIG_LTO_CLANG, .o files in modules might be LLVM bitcode, so we -# need to run LTO to compile them into native code (.lto.o) before further -# processing. -mod-prelink-ext := .prelink -endif - # Useful for describing the dependency of composite objects # Usage: # $(call multi_depend, multi_used_targets, suffix_to_remove, suffix_to_add) diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 7f39599e9fae..35100e981f4a 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -9,7 +9,7 @@ __modfinal: include include/config/auto.conf include $(srctree)/scripts/Kbuild.include -# for c_flags and mod-prelink-ext +# for c_flags include $(srctree)/scripts/Makefile.lib # find all modules listed in modules.order @@ -54,9 +54,8 @@ if_changed_except = $(if $(call newer_prereqs_except,$(2))$(cmd-check), \ $(cmd); \ printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:) - # Re-generate module BTFs if either module's .ko or vmlinux changed -$(modules): %.ko: %$(mod-prelink-ext).o %.mod.o scripts/module.lds $(if $(KBUILD_BUILTIN),vmlinux) FORCE +$(modules): %.ko: %.o %.mod.o scripts/module.lds $(if $(KBUILD_BUILTIN),vmlinux) FORCE +$(call if_changed_except,ld_ko_o,vmlinux) ifdef CONFIG_DEBUG_INFO_BTF_MODULES +$(if $(newer-prereqs),$(call cmd,btf_ko)) diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost index 8b343480813d..911606496341 100644 --- a/scripts/Makefile.modpost +++ b/scripts/Makefile.modpost @@ -41,9 +41,6 @@ __modpost: include include/config/auto.conf include $(srctree)/scripts/Kbuild.include -# for mod-prelink-ext -include $(srctree)/scripts/Makefile.lib - MODPOST = scripts/mod/modpost \ $(if $(CONFIG_MODVERSIONS),-m) \ $(if $(CONFIG_MODULE_SRCVERSION_ALL),-a) \ @@ -117,8 +114,6 @@ $(input-symdump): @echo >&2 ' Modules may not have dependencies or modversions.' @echo >&2 ' You may get many unresolved symbol warnings.' -modules := $(sort $(shell cat $(MODORDER))) - # KBUILD_MODPOST_WARN can be set to avoid error out in case of undefined symbols ifneq ($(KBUILD_MODPOST_WARN)$(filter-out $(existing-input-symdump), $(input-symdump)),) MODPOST += -w @@ -127,9 +122,9 @@ endif # Read out modules.order to pass in modpost. # Otherwise, allmodconfig would fail with "Argument list too long". quiet_cmd_modpost = MODPOST $@ - cmd_modpost = sed 's/\.ko$$/$(mod-prelink-ext)\.o/' $< | $(MODPOST) -T - + cmd_modpost = sed 's/ko$$/o/' $< | $(MODPOST) -T - -$(output-symdump): $(MODORDER) $(input-symdump) $(modules:.ko=$(mod-prelink-ext).o) FORCE +$(output-symdump): $(MODORDER) $(input-symdump) FORCE $(call if_changed,modpost) targets += $(output-symdump) diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index b70a31b4245a..e74224a6a8e8 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -1903,10 +1903,6 @@ static char *remove_dot(char *s) size_t m = strspn(s + n + 1, "0123456789"); if (m && (s[n + m + 1] == '.' || s[n + m + 1] == 0)) s[n] = 0; - - /* strip trailing .prelink */ - if (strends(s, ".prelink")) - s[strlen(s) - 8] = '\0'; } return s; } @@ -2028,9 +2024,6 @@ static void read_symbols(const char *modname) /* strip trailing .o */ tmp = NOFAIL(strdup(modname)); tmp[strlen(tmp) - 2] = '\0'; - /* strip trailing .prelink */ - if (strends(tmp, ".prelink")) - tmp[strlen(tmp) - 8] = '\0'; mod = new_module(tmp); free(tmp); } From patchwork Fri May 27 10:01:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 12863189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB2D7C433EF for ; Fri, 27 May 2022 10:04:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350560AbiE0KEL (ORCPT ); Fri, 27 May 2022 06:04:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231489AbiE0KEK (ORCPT ); Fri, 27 May 2022 06:04:10 -0400 Received: from conuserg-08.nifty.com (conuserg-08.nifty.com [210.131.2.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 746E6F137B; Fri, 27 May 2022 03:04:08 -0700 (PDT) Received: from grover.sesame (133-32-177-133.west.xps.vectant.ne.jp [133.32.177.133]) (authenticated) by conuserg-08.nifty.com with ESMTP id 24RA2hPT029808; Fri, 27 May 2022 19:02:47 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-08.nifty.com 24RA2hPT029808 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1653645767; bh=cTEaVsseOx/T6+9eTNjc+bnIPuNDfidAbsIlj5Z4+nw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XpE7bhc4dasDj31nobFoqPZGgWiHTils8FMsJswF8sT3w9ijw0iw3nlSSVTozzRMi 5lR95zCyn+Nb62hZxhVHIqd5ZAWWXF0Ki8UEf5wX/DKFCjachHKy+d4+EMx1QH29X9 oB0Auu2Zn99EC92w3maEIK2Yd/bYMipg/OlwRvvXPBFznVlCCZRQlpjuNIpRGWqhTJ yYD5OJZlt7Nj2wfepXpIJl4EV+2ROJWGCiqydkPK73l7GS0SbhKeNPsVdHn0TyShTS C9wi5Jten3kQqzhIwYCejZtkdGYT2xGN5xyt4Licsl4OkoREvNk0Ini6mrNgcEehAh Jky+sznG4p0wQ== X-Nifty-SrcIP: [133.32.177.133] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Josh Poimboeuf , llvm@lists.linux.dev, Helge Deller , linux-parisc@vger.kernel.org, Masahiro Yamada , "James E.J. Bottomley" Subject: [PATCH v7 3/8] parisc: fix the exit status of arch/parisc/nm Date: Fri, 27 May 2022 19:01:50 +0900 Message-Id: <20220527100155.1996314-4-masahiroy@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220527100155.1996314-1-masahiroy@kernel.org> References: <20220527100155.1996314-1-masahiroy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org parisc overrides 'nm' with a shell script. I do not know the reason, but anyway it is how it has worked since 2003. [1] A problem is that this script returns the exit code of grep instead of ${CROSS_COMPILE}nm. grep(1) says: Normally the exit status is 0 if a line is selected, 1 if no lines were selected, and 2 if an error occurred. However, if the -q or --quiet or --silent is used and a line is selected, the exit status is 0 even if an error occurred. When the given object has no symbol, grep returns 1, while the true nm returns 0. Hence, build rules using ${NM} fail on ARCH=parisc even if the given object is valid. This commit corrects the exit status of the script. - A pipeline returns the exit status of the last command (here, grep). The exit status of ${CROSS_COMPILE}nm is just ignored. Use bash's pipefail flag to catch errors of ${CROSS_COMPILE}nm. - If grep returns 1, this script should return 0 in order to mimic true nm. If grep returns 2, it is a real and fatal error. Return it as is. [1]: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/commit/?id=36eaa6e4c0e0b6950136b956b72fd08155b92ca3 Signed-off-by: Masahiro Yamada --- Changes in v7: - New patch arch/parisc/Makefile | 2 +- arch/parisc/nm | 12 ++++++++++-- 2 files changed, 11 insertions(+), 3 deletions(-) mode change 100644 => 100755 arch/parisc/nm diff --git a/arch/parisc/Makefile b/arch/parisc/Makefile index aca1710fd658..e7139955367d 100644 --- a/arch/parisc/Makefile +++ b/arch/parisc/Makefile @@ -18,7 +18,7 @@ boot := arch/parisc/boot KBUILD_IMAGE := $(boot)/bzImage -NM = sh $(srctree)/arch/parisc/nm +NM = $(srctree)/arch/parisc/nm CHECKFLAGS += -D__hppa__=1 ifdef CONFIG_64BIT diff --git a/arch/parisc/nm b/arch/parisc/nm old mode 100644 new mode 100755 index c788308de33f..3e72238a91f3 --- a/arch/parisc/nm +++ b/arch/parisc/nm @@ -1,6 +1,14 @@ -#!/bin/sh +#!/bin/bash ## # Hack to have an nm which removes the local symbols. We also rely # on this nm being hidden out of the ordinarily executable path ## -${CROSS_COMPILE}nm $* | grep -v '.LC*[0-9]*$' + +# use pipefail to catch error of ${CROSS_COMPILE}nm +set -o pipefail + +# grep exits with 1 if no lines were selected. +# If the given object has no symbol, grep returns 1, but it is not an error. + +${CROSS_COMPILE}nm "$@" | +{ grep -v '.LC*[0-9]*$' || { exit_code=$?; test $exit_code -eq 1 || exit $exit_code; } } From patchwork Fri May 27 10:01:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 12863197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6434DC433EF for ; Fri, 27 May 2022 10:06:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350789AbiE0KE6 (ORCPT ); Fri, 27 May 2022 06:04:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350791AbiE0KE4 (ORCPT ); Fri, 27 May 2022 06:04:56 -0400 Received: from conuserg-08.nifty.com (conuserg-08.nifty.com [210.131.2.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F28B1116EA; Fri, 27 May 2022 03:04:46 -0700 (PDT) Received: from grover.sesame (133-32-177-133.west.xps.vectant.ne.jp [133.32.177.133]) (authenticated) by conuserg-08.nifty.com with ESMTP id 24RA2hPU029808; Fri, 27 May 2022 19:02:48 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-08.nifty.com 24RA2hPU029808 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1653645768; bh=63AM9Y7KFJuXBvKXOmj6kJdz+ZNVbwHrp42ZdIFptX0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RW1wttQF3Oea8ArmryxA7c5OlRphy4XLSdTGnKXoDhQDvP5OS0CvMg6u22wNMneYx MmO+ShRV77S1L7Kw29gY8hdiRenlRehQa9VP/A7EW8Q74MIo5gh55tpvyRw/EjcNDt FtZa4M+gKZRkV1J7Xeaw1CpPiJZPNC+qWSvPi6SIOT+q7f3UQqnuWYNdCbUsSk7eeK VXAsUTwKMj0SR86wH6e551ECa1syR07zSaD6957dxD4pxsoSxokifNvrkHppOu8zAW mngXMEsnyFXgvcZ5tdQ8cgd6TbzUkBzBfMsFXGJzFu11tESBdN4bw0fStXUmgTILtj rmlIrYn4jxCTQ== X-Nifty-SrcIP: [133.32.177.133] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Josh Poimboeuf , llvm@lists.linux.dev, Helge Deller , linux-parisc@vger.kernel.org, Masahiro Yamada , Nick Desaulniers , Nathan Chancellor , Sedat Dilek , Michal Marek , Tom Rix Subject: [PATCH v7 4/8] kbuild: check static EXPORT_SYMBOL* by script instead of modpost Date: Fri, 27 May 2022 19:01:51 +0900 Message-Id: <20220527100155.1996314-5-masahiroy@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220527100155.1996314-1-masahiroy@kernel.org> References: <20220527100155.1996314-1-masahiroy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org The 'static' specifier and EXPORT_SYMBOL() are an odd combination. Commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL* functions") tried to detect it, but this check has false negatives. Here is the sample code. Makefile: obj-y += foo1.o foo2.o foo1.c: #include static void foo(void) {} EXPORT_SYMBOL(foo); foo2.c: void foo(void) {} foo1.c exports the static symbol 'foo', but modpost cannot catch it because it is fooled by foo2.c, which has a global symbol with the same name. s->is_static is cleared if a global symbol with the same name is found somewhere, but EXPORT_SYMBOL() and the global symbol do not necessarily belong to the same compilation unit. This check should be done per compilation unit, but I do not know how to do it in modpost. modpost runs against vmlinux.o or modules, which merges multiple objects, then forgets their origin. modpost cannot parse individual objects because they may not be ELF but LLVM IR when CONFIG_LTO_CLANG=y. Add a simple bash script to parse the output from ${NM}. This works for CONFIG_LTO_CLANG=y because llvm-nm can dump symbols of LLVM IR files. Revert 15bfc2348d54. Signed-off-by: Masahiro Yamada Reviewed-by: Nick Desaulniers Tested-by: Nathan Chancellor Tested-by: Sedat Dilek # LLVM-14 (x86-64) --- Changes in v7: - Fix sparc64 build (skip the line if the number of fields is less than 3) Changes in v5: - Add more comments in the script Changes in v4: - New patch scripts/Makefile.build | 4 +++ scripts/check-local-export | 65 ++++++++++++++++++++++++++++++++++++++ scripts/mod/modpost.c | 28 +--------------- 3 files changed, 70 insertions(+), 27 deletions(-) create mode 100755 scripts/check-local-export diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 23e1fb19f9ad..0135b4e7f78b 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -248,9 +248,12 @@ cmd_gen_ksymdeps = \ $(CONFIG_SHELL) $(srctree)/scripts/gen_ksymdeps.sh $@ >> $(dot-target).cmd endif +cmd_check_local_export = $(srctree)/scripts/check-local-export $@ + define rule_cc_o_c $(call cmd_and_fixdep,cc_o_c) $(call cmd,gen_ksymdeps) + $(call cmd,check_local_export) $(call cmd,checksrc) $(call cmd,checkdoc) $(call cmd,gen_objtooldep) @@ -261,6 +264,7 @@ endef define rule_as_o_S $(call cmd_and_fixdep,as_o_S) $(call cmd,gen_ksymdeps) + $(call cmd,check_local_export) $(call cmd,gen_objtooldep) $(call cmd,gen_symversions_S) endef diff --git a/scripts/check-local-export b/scripts/check-local-export new file mode 100755 index 000000000000..017119d89cd2 --- /dev/null +++ b/scripts/check-local-export @@ -0,0 +1,65 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0-only +# +# Copyright (C) 2022 Masahiro Yamada +# +# Exit with error if a local exported symbol is found. +# EXPORT_SYMBOL should be used for global symbols. + +set -e + +declare -A symbol_types +declare -a export_symbols + +exit_code=0 + +while read value type name +do + # Skip the number of fields is less than 3. + # + # case 1) + # For undefined symbols, the first field (value) is empty. + # The outout looks like this: + # " U _printk" + # It is unneeded to record undefined symbols. + # + # case 2) + # For Clang LTO, llvm-nm outputs a line with type 't' but empty name: + # "---------------- t" + if [[ -z ${name} ]]; then + continue + fi + + # save (name, type) in the associative array + symbol_types[${name}]=${type} + + # append the exported symbol to the array + if [[ ${name} == __ksymtab_* ]]; then + export_symbols+=(${name#__ksymtab_}) + fi + + # If there is no symbol in the object, ${NM} (both GNU nm and llvm-nm) + # shows 'no symbols' diagnostic (but exits with 0). It is harmless and + # hidden by '2>/dev/null'. However, it suppresses real error messages + # as well. Add a hand-crafted error message here. + # + # Use --quiet instead of 2>/dev/null when we upgrade the minimum version + # of binutils to 2.37, llvm to 13.0.0. + # + # Then, the following line will be really simple: + # done < <(${NM} --quiet ${1}) +done < <(${NM} ${1} 2>/dev/null || { echo "${0}: ${NM} failed" >&2; false; } ) + +# Catch error in the process substitution +wait $! + +for name in "${export_symbols[@]}" +do + # nm(3) says "If lowercase, the symbol is usually local" + if [[ ${symbol_types[$name]} =~ [a-z] ]]; then + echo "$@: error: local symbol '${name}' was exported" >&2 + exit_code=1 + fi +done + +exit ${exit_code} diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index e74224a6a8e8..9269735f85c5 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -212,7 +212,6 @@ struct symbol { unsigned int crc; bool crc_valid; bool weak; - bool is_static; /* true if symbol is not global */ bool is_gpl_only; /* exported by EXPORT_SYMBOL_GPL */ char name[]; }; @@ -242,7 +241,7 @@ static struct symbol *alloc_symbol(const char *name) memset(s, 0, sizeof(*s)); strcpy(s->name, name); - s->is_static = true; + return s; } @@ -2064,20 +2063,6 @@ static void read_symbols(const char *modname) sym_get_data(&info, sym)); } - // check for static EXPORT_SYMBOL_* functions && global vars - for (sym = info.symtab_start; sym < info.symtab_stop; sym++) { - unsigned char bind = ELF_ST_BIND(sym->st_info); - - if (bind == STB_GLOBAL || bind == STB_WEAK) { - struct symbol *s = - find_symbol(remove_dot(info.strtab + - sym->st_name)); - - if (s) - s->is_static = false; - } - } - check_sec_ref(modname, &info); if (!mod->is_vmlinux) { @@ -2507,7 +2492,6 @@ static void read_dump(const char *fname) mod->from_dump = true; } s = sym_add_exported(symname, mod, gpl_only); - s->is_static = false; sym_set_crc(s, crc); sym_update_namespace(symname, namespace); } @@ -2572,7 +2556,6 @@ int main(int argc, char **argv) char *missing_namespace_deps = NULL; char *dump_write = NULL, *files_source = NULL; int opt; - int n; LIST_HEAD(dump_lists); struct dump_list *dl, *dl2; @@ -2648,15 +2631,6 @@ int main(int argc, char **argv) if (sec_mismatch_count && !sec_mismatch_warn_only) error("Section mismatches detected.\n" "Set CONFIG_SECTION_MISMATCH_WARN_ONLY=y to allow them.\n"); - for (n = 0; n < SYMBOL_HASH_SIZE; n++) { - struct symbol *s; - - for (s = symbolhash[n]; s; s = s->next) { - if (s->is_static) - error("\"%s\" [%s] is a static EXPORT_SYMBOL\n", - s->name, s->module->name); - } - } if (nr_unresolved > MAX_UNRESOLVED_REPORTS) warn("suppressed %u unresolved symbol warnings because there were too many)\n", From patchwork Fri May 27 10:01:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 12863194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F89BC43219 for ; Fri, 27 May 2022 10:05:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350774AbiE0KF1 (ORCPT ); Fri, 27 May 2022 06:05:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350930AbiE0KFP (ORCPT ); Fri, 27 May 2022 06:05:15 -0400 Received: from conuserg-08.nifty.com (conuserg-08.nifty.com [210.131.2.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12D0110656D; Fri, 27 May 2022 03:05:14 -0700 (PDT) Received: from grover.sesame (133-32-177-133.west.xps.vectant.ne.jp [133.32.177.133]) (authenticated) by conuserg-08.nifty.com with ESMTP id 24RA2hPV029808; Fri, 27 May 2022 19:02:49 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-08.nifty.com 24RA2hPV029808 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1653645770; bh=aeTLWp41sB2px7vbsl+AoWvoxM63JuytaCqa9JH2QLY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HNBXNGONQ3XAXjEyNsT812P/sRM9yUOuKeEx6rV5MeMv1NxMHnZVXWp2u3jL3+YhO hHwEyE27UCcCCype5LopeDwcShy1TIjgvIzb35fNVES8P6DPUjAtO/oKmWQyrjKn7e bNLeFrwGDnwLBGzZ9ECYC+HfWtv1orzRaQxDXtS4bmogZCcZlcfc0L4a8I5xbtRerl AENHInpUJ2umDWLEQ3o3RVbecF3RavfGGRbduj+yTrY6zqZ1VeHFrt267AKy33Vb4M j/NpQV/JGiOhD8mPqsR5WAZPocvvNN8niceVYS/cuIu+CoXuwCESKCAm4kbrQ1JKic Dpe+I0j85Opbg== X-Nifty-SrcIP: [133.32.177.133] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Josh Poimboeuf , llvm@lists.linux.dev, Helge Deller , linux-parisc@vger.kernel.org, Masahiro Yamada , Nicolas Schier , Nathan Chancellor , Sedat Dilek , Michal Marek , Nick Desaulniers , Tom Rix Subject: [PATCH v7 5/8] kbuild: make built-in.a rule robust against too long argument error Date: Fri, 27 May 2022 19:01:52 +0900 Message-Id: <20220527100155.1996314-6-masahiroy@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220527100155.1996314-1-masahiroy@kernel.org> References: <20220527100155.1996314-1-masahiroy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org Kbuild runs at the top of objtree instead of changing the working directory to subdirectories. I think this design is nice overall but some commands have a scalability issue. The build command of built-in.a is one of them whose length scales with: O(D * N) Here, D is the length of the directory path (i.e. $(obj)/ prefix), N is the number of objects in the Makefile, O() is the big O notation. The deeper directory the Makefile directory is located, the more easily it will hit the too long argument error. We can make it better. Trim the $(obj)/ by Make's builtin function, and restore it by a shell command (sed). With this, the command length scales with: O(D + N) In-tree modules still have some room to the limit (ARG_MAX=2097152), but this is more future-proof for big modules in a deep directory. For example, you can build i915 as builtin (CONFIG_DRM_I915=y) and compare drivers/gpu/drm/i915/.built-in.a.cmd with/without this commit. Signed-off-by: Masahiro Yamada Reviewed-by: Nicolas Schier Tested-by: Nathan Chancellor Tested-by: Sedat Dilek # LLVM-14 (x86-64) --- Changes in v7: - Add comment to the code Changes in v2: - New patch scripts/Makefile.build | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 0135b4e7f78b..014ebee70fd6 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -376,9 +376,14 @@ $(subdir-modorder): $(obj)/%/modules.order: $(obj)/% ; # # Rule to compile a set of .o files into one .a file (without symbol table) # +# To make this rule robust against "Argument list too long" error, +# remove $(obj)/ prefix, and restore it by a shell command. quiet_cmd_ar_builtin = AR $@ - cmd_ar_builtin = rm -f $@; $(AR) cDPrST $@ $(real-prereqs) + cmd_ar_builtin = rm -f $@; \ + echo $(patsubst $(obj)/%,%,$(real-prereqs)) | \ + sed -E 's:([^ ]+):$(obj)/\1:g' | \ + xargs $(AR) cDPrST $@ $(obj)/built-in.a: $(real-obj-y) FORCE $(call if_changed,ar_builtin) From patchwork Fri May 27 10:01:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 12863195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37CEAC433FE for ; Fri, 27 May 2022 10:05:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350784AbiE0KF1 (ORCPT ); Fri, 27 May 2022 06:05:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350936AbiE0KFT (ORCPT ); Fri, 27 May 2022 06:05:19 -0400 Received: from conuserg-08.nifty.com (conuserg-08.nifty.com [210.131.2.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9BCB110656D; Fri, 27 May 2022 03:05:17 -0700 (PDT) Received: from grover.sesame (133-32-177-133.west.xps.vectant.ne.jp [133.32.177.133]) (authenticated) by conuserg-08.nifty.com with ESMTP id 24RA2hPW029808; Fri, 27 May 2022 19:02:50 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-08.nifty.com 24RA2hPW029808 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1653645771; bh=3Oj1DhJn8CmLNtWlFeZrpT6DZfeTSEt5AwsBsBHThdw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=v0PffPwf03F/kuXj7tkaHCU736AoC3p/iSQGFT4EuXdSk7N4AHNFlo+HaQjmoyMK5 MYXLldeol6bR3qKJAaweMayW9SJ7L6Ol6Gx2LFBAz6wUIBmKQeNtVMh+l6om88wy+U 9sXkPMptAwV2mpoMEItMwB2cESfr5Y0wkSPl1NUMRVGd1ntYQ/pbDemHSt8Yc71iwX ZDOd7CzWE/FqZ/+GM1ZY3esJH3uCAuKs1WCrHQ7JXPKRhsYCDRAqY01YdQQusFIhY8 jXOfjZIOwRY7MZIVVBUlPLoYWmd/bZ5dq9bXs3v3fdYJ4Uu2NTj8B9IZyukH/7O3RK JL/jIbsOZOSlQ== X-Nifty-SrcIP: [133.32.177.133] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Josh Poimboeuf , llvm@lists.linux.dev, Helge Deller , linux-parisc@vger.kernel.org, Masahiro Yamada , Nicolas Schier , Nathan Chancellor , Sedat Dilek , Michal Marek , Nick Desaulniers , Tom Rix Subject: [PATCH v7 6/8] kbuild: make *.mod rule robust against too long argument error Date: Fri, 27 May 2022 19:01:53 +0900 Message-Id: <20220527100155.1996314-7-masahiroy@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220527100155.1996314-1-masahiroy@kernel.org> References: <20220527100155.1996314-1-masahiroy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org Like built-in.a, the command length of the *.mod rule scales with the depth of the directory times the number of objects in the Makefile. Add $(obj)/ by the shell command (awk) instead of by Make's builtin function. In-tree modules still have some room to the limit (ARG_MAX=2097152), but this is more future-proof for big modules in a deep directory. For example, you can build i915 as a module (CONFIG_DRM_I915=m) and compare drivers/gpu/drm/i915/.i915.mod.cmd with/without this commit. The issue is more critical for external modules because the M= path can be very long as Jeff Johnson reported before [1]. [1] https://lore.kernel.org/linux-kbuild/4c02050c4e95e4cb8cc04282695f8404@codeaurora.org/ Signed-off-by: Masahiro Yamada Reviewed-by: Nicolas Schier Tested-by: Nathan Chancellor Tested-by: Sedat Dilek # LLVM-14 (x86-64) --- Changes in v7: - Add comment to the code scripts/Makefile.build | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 014ebee70fd6..31feb798e16e 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -274,8 +274,10 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE $(call if_changed_rule,cc_o_c) $(call cmd,force_checksrc) -cmd_mod = echo $(addprefix $(obj)/, $(call real-search, $*.o, .o, -objs -y -m)) | \ - $(AWK) -v RS='( |\n)' '!x[$$0]++' > $@ +# To make this rule robust against "Argument list too long" error, +# ensure to add $(obj)/ prefix by a shell command. +cmd_mod = echo $(call real-search, $*.o, .o, -objs -y -m) | \ + $(AWK) -v RS='( |\n)' '!x[$$0]++ { print("$(obj)/"$$0) }' > $@ $(obj)/%.mod: FORCE $(call if_changed,mod) From patchwork Fri May 27 10:01:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 12863193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B880DC43217 for ; Fri, 27 May 2022 10:05:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350770AbiE0KF1 (ORCPT ); Fri, 27 May 2022 06:05:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350915AbiE0KFL (ORCPT ); Fri, 27 May 2022 06:05:11 -0400 Received: from conuserg-08.nifty.com (conuserg-08.nifty.com [210.131.2.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 102691078B0; Fri, 27 May 2022 03:05:09 -0700 (PDT) Received: from grover.sesame (133-32-177-133.west.xps.vectant.ne.jp [133.32.177.133]) (authenticated) by conuserg-08.nifty.com with ESMTP id 24RA2hPX029808; Fri, 27 May 2022 19:02:51 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-08.nifty.com 24RA2hPX029808 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1653645772; bh=/LDGWcsQ86eGZ6AnM0sm302Ykct401VHcmgP1DL7HYM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rRtdN563IfYIa8UJiH9R6JJ+H+T8TUQiUgBziOGfpObJBS0vJgoUxq8MvApQYo5aV O71PUr/VsqixjttnfTjeyypDUIv9rUsWLb4ujUwS/UU5A03nVQausNmzgYsgiYd43r 05WZFD8zP5J7ZoAldPOyNxWUv3GVaKzQh/fXLSxQJdNBqhfrcpQX3n4pdcni0SDM0A 4+PlqZsMPV/nyTcr799AAberdlGpzsfkVc30H9yuFqKcDiiHcXF143AZUtVsD/9Vme 2qvC268JrtUk4pUuE7NcTuPbeplTB5qsk4W8ZllSNgK7otYw7cd9E5l9gvfBKOKbTn KnhUoIuWymdYA== X-Nifty-SrcIP: [133.32.177.133] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Josh Poimboeuf , llvm@lists.linux.dev, Helge Deller , linux-parisc@vger.kernel.org, Masahiro Yamada , Kees Cook , Nathan Chancellor , Nicolas Schier , Sedat Dilek , Michal Marek , Nick Desaulniers , Tom Rix Subject: [PATCH v7 7/8] kbuild: add cmd_and_savecmd macro Date: Fri, 27 May 2022 19:01:54 +0900 Message-Id: <20220527100155.1996314-8-masahiroy@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220527100155.1996314-1-masahiroy@kernel.org> References: <20220527100155.1996314-1-masahiroy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org Separate out the command execution part of if_changed, as we did for if_changed_dep. This allows us to reuse it in if_changed_rule. define rule_foo $(call cmd_and_savecmd,foo) $(call cmd,bar) endef Signed-off-by: Masahiro Yamada Reviewed-by: Kees Cook Tested-by: Nathan Chancellor Reviewed-by: Nicolas Schier Tested-by: Sedat Dilek # LLVM-14 (x86-64) --- (no changes since v4) Changes in v4: - New. Resent of my previous submission. https://lore.kernel.org/all/20210831074004.3195284-10-masahiroy@kernel.org/ scripts/Kbuild.include | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include index 455a0a6ce12d..ece44b735061 100644 --- a/scripts/Kbuild.include +++ b/scripts/Kbuild.include @@ -142,9 +142,11 @@ check-FORCE = $(if $(filter FORCE, $^),,$(warning FORCE prerequisite is missing) if-changed-cond = $(newer-prereqs)$(cmd-check)$(check-FORCE) # Execute command if command has changed or prerequisite(s) are updated. -if_changed = $(if $(if-changed-cond), \ +if_changed = $(if $(if-changed-cond),$(cmd_and_savecmd),@:) + +cmd_and_savecmd = \ $(cmd); \ - printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:) + printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd # Execute the command and also postprocess generated .d dependencies file. if_changed_dep = $(if $(if-changed-cond),$(cmd_and_fixdep),@:) From patchwork Fri May 27 10:01:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 12863192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40B81C433EF for ; Fri, 27 May 2022 10:05:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232955AbiE0KFZ (ORCPT ); Fri, 27 May 2022 06:05:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350917AbiE0KFL (ORCPT ); Fri, 27 May 2022 06:05:11 -0400 X-Greylist: delayed 62 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Fri, 27 May 2022 03:05:10 PDT Received: from conuserg-08.nifty.com (conuserg-08.nifty.com [210.131.2.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7EB33106A69; Fri, 27 May 2022 03:05:09 -0700 (PDT) Received: from grover.sesame (133-32-177-133.west.xps.vectant.ne.jp [133.32.177.133]) (authenticated) by conuserg-08.nifty.com with ESMTP id 24RA2hPY029808; Fri, 27 May 2022 19:02:52 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-08.nifty.com 24RA2hPY029808 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1653645773; bh=jCYP4PRhKOSnrfF8IZ7vFN4wM6bjQvDGDEXIDORQ8GA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jikzAhA331ol0PYeJLvjeBEbXdujI+kIf0o3ofXuKIEM+meJCBxUL0pXT+5ICkrrk WGEI+qfbQ/eYfPv2X/mCpePNalEwvJoQquANrW6/9yUIYvayiJG3pecAlquCklvEzu u9CAnTa2xAQfvS1milmRCkFvz1Ows9DtGa22empJVZwYjcSYwicxgZfullj8vV0NpX hX66B4UH9sI7lX+LKJvr0edE2Gsv1wZM/9PTdvP6Iw+GTJyitv6VLegEP4RdZeO1DD 0UWOpTu5TFRnHiim7arEAU9NCmnt1bLfqR+NSrzOK26eqN0h/T4CftjLGI9wKkrUH9 bW4oOJqflZ3eQ== X-Nifty-SrcIP: [133.32.177.133] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Josh Poimboeuf , llvm@lists.linux.dev, Helge Deller , linux-parisc@vger.kernel.org, Masahiro Yamada , Kees Cook , Nathan Chancellor , Nicolas Schier , Sedat Dilek , Michal Marek , Nick Desaulniers , Tom Rix Subject: [PATCH v7 8/8] kbuild: rebuild multi-object modules when objtool is updated Date: Fri, 27 May 2022 19:01:55 +0900 Message-Id: <20220527100155.1996314-9-masahiroy@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220527100155.1996314-1-masahiroy@kernel.org> References: <20220527100155.1996314-1-masahiroy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org When CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT is enabled, objtool for multi-object modules is postponed until the objects are linked together. Make sure to re-run objtool and re-link multi-object modules when objtool is updated. Signed-off-by: Masahiro Yamada Reviewed-by: Kees Cook Acked-by: Josh Poimboeuf Tested-by: Nathan Chancellor Reviewed-by: Nicolas Schier Tested-by: Sedat Dilek # LLVM-14 (x86-64) --- (no changes since v4) Changes in v4: - New Resent of my previous submission https://lore.kernel.org/linux-kbuild/20210831074004.3195284-11-masahiroy@kernel.org/ scripts/Makefile.build | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 31feb798e16e..bd5fc2b37387 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -412,13 +412,18 @@ $(obj)/modules.order: $(obj-m) FORCE $(obj)/lib.a: $(lib-y) FORCE $(call if_changed,ar) -quiet_cmd_link_multi-m = LD [M] $@ - cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool) +quiet_cmd_ld_multi_m = LD [M] $@ + cmd_ld_multi_m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool) + +define rule_ld_multi_m + $(call cmd_and_savecmd,ld_multi_m) + $(call cmd,gen_objtooldep) +endef $(multi-obj-m): objtool-enabled := $(delay-objtool) $(multi-obj-m): part-of-module := y $(multi-obj-m): %.o: %.mod FORCE - $(call if_changed,link_multi-m) + $(call if_changed_rule,ld_multi_m) $(call multi_depend, $(multi-obj-m), .o, -objs -y -m) targets := $(filter-out $(PHONY), $(targets))