From patchwork Tue Oct 16 09:10:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 10643241 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7A05917D4 for ; Tue, 16 Oct 2018 09:12:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 69782294AA for ; Tue, 16 Oct 2018 09:12:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5BBB3294B7; Tue, 16 Oct 2018 09:12:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6299A294AA for ; Tue, 16 Oct 2018 09:12:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727260AbeJPRBi (ORCPT ); Tue, 16 Oct 2018 13:01:38 -0400 Received: from conuserg-09.nifty.com ([210.131.2.76]:57571 "EHLO conuserg-09.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727032AbeJPRBh (ORCPT ); Tue, 16 Oct 2018 13:01:37 -0400 Received: from pug.e01.socionext.com (p14092-ipngnfx01kyoto.kyoto.ocn.ne.jp [153.142.97.92]) (authenticated) by conuserg-09.nifty.com with ESMTP id w9G9BLLv029692; Tue, 16 Oct 2018 18:11:22 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-09.nifty.com w9G9BLLv029692 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1539681083; bh=bXpviXkYAhvFT7qG6xXXNbPHFseOjHvbyF4Xhb2IpBA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=zn1dGnanzsIvhFqKk6xLja7en9LBahNuNgeJN7L/sUhQGIADqRV19Dwju/lZ7yj7c FJ815cyptAXC/wIoILrUDhKdtvHfl8Ow0fIjOMb5F+1yj/YSh1UENJ5P3I1PG6B9hZ 5tZQA+P1phhE3BAULEJB4oAtGMt1Tpj4/n2uU5uyGsdHlnrRFBfBiqAk1nT6o5hTMU /KOuh/47kD6h2tGmc+2/Nkx71OYInSFhSMU6PLGvc1bJeIROVU+atuxGI8Uw3/jz/Q dw0NJ9Bt7irfbb6UOm2AMj152Hm1w8DAgUJ39PXeqwqybE/QAZ36xFNoeNsiRZrbtM dtOncDU/HwwbA== X-Nifty-SrcIP: [153.142.97.92] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: Ingo Molnar , Josh Poimboeuf , Bernd Edlinger , Borislav Petkov , Sam Ravnborg , Masahiro Yamada , Michal Marek , linux-kernel@vger.kernel.org Subject: [PATCH 1/3] kbuild: provide a new place to check necessary host tools Date: Tue, 16 Oct 2018 18:10:51 +0900 Message-Id: <1539681053-24388-2-git-send-email-yamada.masahiro@socionext.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1539681053-24388-1-git-send-email-yamada.masahiro@socionext.com> References: <1539681053-24388-1-git-send-email-yamada.masahiro@socionext.com> Sender: linux-kbuild-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Enabling CONFIG options often requires additional features, like target compiler capabilities, host tools, etc. As for the target compiler capabilities, Kconfig is able to add proper dependencies. For example, config STACKPROTECTOR ... depends on $(cc-option,-fstack-protector) The CONFIG option is simply disabled if unsupported by the compiler. This seems sensible since otherwise, there is no way to avoid the build error. In the discussion [1], doing similar for host tools was unacceptable. Build errors caused by missing host tools are easy to fix; just install them to your build machine. So, let the build fail instead of disabling CONFIG options silently. We already have such checks in random places in makefiles. What I do not like is that people tend to do that in the top Makefile. The top Makefile is too cluttered, and what is worse, $(error ...) functions in the top Makefile could be false positive. (Please note the top Makefile includes the stale include/config/auto.conf before syncconfig updates it.) This adds a new file scripts/Makefile.toolcheck to check additional tools depending on the kernel configuration. This check is run immediately after syncconfig, i.e., when a user attempts to build something with a new set of CONFIG options. [1] https://patchwork.kernel.org/patch/10516049/ Signed-off-by: Masahiro Yamada --- Makefile | 1 + scripts/Makefile.toolcheck | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+) create mode 100644 scripts/Makefile.toolcheck diff --git a/Makefile b/Makefile index bf3786e..23a204a 100644 --- a/Makefile +++ b/Makefile @@ -633,6 +633,7 @@ $(KCONFIG_CONFIG) include/config/auto.conf.cmd: ; # include/config/auto.conf (which mirrors .config). include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig + $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.toolcheck else # External modules and some install targets need include/generated/autoconf.h # and include/config/auto.conf but do not care if they are up-to-date. diff --git a/scripts/Makefile.toolcheck b/scripts/Makefile.toolcheck new file mode 100644 index 0000000..f3c165d --- /dev/null +++ b/scripts/Makefile.toolcheck @@ -0,0 +1,35 @@ +# SPDX-License-Identifier: GPL-2.0 +# =========================================================================== +# Host tools check +# =========================================================================== +# +# Some additional tools might be required depending on the kernel configuration. +# Check if they are installed on the host machine, and if missing, error out +# with a user-friendly message. + +include include/config/auto.conf + +__toolcheck: + @: + +PHONY += $(toolcheck-y) +__toolcheck: $(toolcheck-y) + +define populate_toolcheck +__toolcheck: check_$(1) +PHONY += check_$(1) +check_$(1): + $(Q){ $(chk_$(1)); } >/dev/null 2>&1 || { \ + echo "*" >&2; \ + for msg in $(msg_$(1)); \ + do \ + echo "* $$$${msg}" >&2; \ + done; \ + echo "*" >&2; \ + /bin/false; \ + } +endef + +$(foreach c, $(toolcheck-y), $(eval $(call populate_toolcheck,$(c)))) + +.PHONY: $(PHONY) From patchwork Tue Oct 16 09:10:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 10643245 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09B29112B for ; Tue, 16 Oct 2018 09:12:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EDCFC294B0 for ; Tue, 16 Oct 2018 09:12:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E1D81294BE; Tue, 16 Oct 2018 09:12:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5EF64294B0 for ; Tue, 16 Oct 2018 09:12:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726541AbeJPRBh (ORCPT ); Tue, 16 Oct 2018 13:01:37 -0400 Received: from conuserg-09.nifty.com ([210.131.2.76]:57572 "EHLO conuserg-09.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726958AbeJPRBh (ORCPT ); Tue, 16 Oct 2018 13:01:37 -0400 Received: from pug.e01.socionext.com (p14092-ipngnfx01kyoto.kyoto.ocn.ne.jp [153.142.97.92]) (authenticated) by conuserg-09.nifty.com with ESMTP id w9G9BLLw029692; Tue, 16 Oct 2018 18:11:23 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-09.nifty.com w9G9BLLw029692 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1539681084; bh=CqiDXGBClPJg3L6ZPE6aRkpV3O7/PTqNTFi2pZNHYq8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kV/KIjbDjvBKMNqpBjOjPliwu/kMV3OXCe1rYCNAfeed8Gq/HGtuvjgEe6H4mJrAx QEtcVb+1J/UIMOlVk2ndoZCTBeap2GzeDIQTZ04nv3jh2EMi38SyF+h2Bgm37w5W0x A1PPBLwDW5vRDKgh33gEAOg1YO2eoz52YrB0yPuLJCMFW9zX8dz4HivQQiwJMq2UeT wSsRNbH1U1KxUhZ2ZtnZFEjWF22d9WP3MKbmAoNB4mIHzFQt2Spuvn3/4Rm5zrwy5u WSfKlt/NwwYC2trosg4jfuCFPFOUaSmZnbMlLlN+RBqdblRvIXV3/UTj6OPXla1kau n7Qh/p6XHG1cg== X-Nifty-SrcIP: [153.142.97.92] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: Ingo Molnar , Josh Poimboeuf , Bernd Edlinger , Borislav Petkov , Sam Ravnborg , Masahiro Yamada , Michal Marek , linux-kernel@vger.kernel.org Subject: [PATCH 2/3] objtool: move libelf check out of top Makefile Date: Tue, 16 Oct 2018 18:10:52 +0900 Message-Id: <1539681053-24388-3-git-send-email-yamada.masahiro@socionext.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1539681053-24388-1-git-send-email-yamada.masahiro@socionext.com> References: <1539681053-24388-1-git-send-email-yamada.masahiro@socionext.com> Sender: linux-kbuild-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Bernd Edlinger reports: The next make after an oldconfig reads in the outdated include/config/auto.conf which can kill the make before it is able to call the syncconfig target. $ make defconfig *** Default configuration is based on 'x86_64_defconfig' $ make scripts/kconfig/conf --syncconfig Kconfig Makefile:936: *** "Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel". Stop. $ sed -i s/CONFIG_UNWINDER_ORC=y// .configs $ make oldconfig Choose kernel unwinder > 1. ORC unwinder (UNWINDER_ORC) (NEW) 2. Frame pointer unwinder (UNWINDER_FRAME_POINTER) choice[1-2?]: 2 $ make Makefile:936: *** "Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel". Stop. I tried to fix it by moving the libelf evaluation to the Kconfig stage [1], but it was rejected. Kconfig is now able to run shell commands to evaluate any prerequisites needed to enable CONFIG options. For example, as shown in commit 2a61f4747eea ("stack-protector: test compiler capability in Kconfig and drop AUTO mode"), the new Kconfig syntax is useful to evaluate target compiler capabilities. However, disabling the CONFIG option silently just because of missing libelf would not be a preferred behavior; in this case, installing an appropriate package will solve the problem. Hence, this check will be kept in a makefile, but somewhere else than the top Makefile. Move the check to scripts/Makefile.toolcheck so that it is run after syncconfig. Another behavioral change is, missing libelf for CONFIG_STACK_VALIDATION was previously a warning, but now a error. [1] https://patchwork.kernel.org/patch/10516049/ Reported-by: Bernd Edlinger Signed-off-by: Masahiro Yamada --- Makefile | 21 +++------------------ scripts/Makefile.build | 2 -- scripts/Makefile.toolcheck | 5 +++++ 3 files changed, 8 insertions(+), 20 deletions(-) diff --git a/Makefile b/Makefile index 23a204a..71940b7 100644 --- a/Makefile +++ b/Makefile @@ -949,23 +949,6 @@ mod_sign_cmd = true endif export mod_sign_cmd -ifdef CONFIG_STACK_VALIDATION - has_libelf := $(call try-run,\ - echo "int main() {}" | $(HOSTCC) -xc -o /dev/null -lelf -,1,0) - ifeq ($(has_libelf),1) - objtool_target := tools/objtool FORCE - else - ifdef CONFIG_UNWINDER_ORC - $(error "Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel") - else - $(warning "Cannot use CONFIG_STACK_VALIDATION=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel") - endif - SKIP_STACK_VALIDATION := 1 - export SKIP_STACK_VALIDATION - endif -endif - - ifeq ($(KBUILD_EXTMOD),) core-y += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/ block/ @@ -1115,7 +1098,9 @@ uapi-asm-generic: src=uapi/asm obj=arch/$(SRCARCH)/include/generated/uapi/asm PHONY += prepare-objtool -prepare-objtool: $(objtool_target) +ifdef CONFIG_STACK_VALIDATION +prepare-objtool: tools/objtool +endif # Generate some files # --------------------------------------------------------------------------- diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 54da4b0..e9dabe4 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -233,7 +233,6 @@ endif # CC_USING_RECORD_MCOUNT endif # CONFIG_FTRACE_MCOUNT_RECORD ifdef CONFIG_STACK_VALIDATION -ifneq ($(SKIP_STACK_VALIDATION),1) __objtool_obj := $(objtree)/tools/objtool/objtool @@ -270,7 +269,6 @@ objtool_obj = $(if $(patsubst y%,, \ $(OBJECT_FILES_NON_STANDARD_$(basetarget).o)$(OBJECT_FILES_NON_STANDARD)n), \ $(__objtool_obj)) -endif # SKIP_STACK_VALIDATION endif # CONFIG_STACK_VALIDATION # Rebuild all objects when objtool changes, or is enabled/disabled. diff --git a/scripts/Makefile.toolcheck b/scripts/Makefile.toolcheck index f3c165d..bc26fc0 100644 --- a/scripts/Makefile.toolcheck +++ b/scripts/Makefile.toolcheck @@ -12,6 +12,11 @@ include include/config/auto.conf __toolcheck: @: +chk_stack_validation = echo "int main() {}" | $(HOSTCC) -xc -o /dev/null -lelf - +msg_stack_validation = "libelf is necessary for building the objtool." \ + "Please install libelf-dev, libelf-devel or elfutils-libelf-devel." +toolcheck-$(CONFIG_STACK_VALIDATION) += stack_validation + PHONY += $(toolcheck-y) __toolcheck: $(toolcheck-y) From patchwork Tue Oct 16 09:10:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 10643243 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E3F817D4 for ; Tue, 16 Oct 2018 09:12:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8EA6F294AA for ; Tue, 16 Oct 2018 09:12:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 825AD294B7; Tue, 16 Oct 2018 09:12:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2CAB7294AA for ; Tue, 16 Oct 2018 09:12:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726968AbeJPRBn (ORCPT ); Tue, 16 Oct 2018 13:01:43 -0400 Received: from conuserg-09.nifty.com ([210.131.2.76]:57743 "EHLO conuserg-09.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727042AbeJPRBm (ORCPT ); Tue, 16 Oct 2018 13:01:42 -0400 Received: from pug.e01.socionext.com (p14092-ipngnfx01kyoto.kyoto.ocn.ne.jp [153.142.97.92]) (authenticated) by conuserg-09.nifty.com with ESMTP id w9G9BLLx029692; Tue, 16 Oct 2018 18:11:24 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-09.nifty.com w9G9BLLx029692 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1539681084; bh=oOu070Ru7d+JJagPqx7fCHtw6LbcaDKL54LM4cesph4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=xu1LPVbJscQBy8rpYl0T7P+sapQdLxQs6liVk992m/r3XP3O3rAiF0aa/8ER7hh7O wYwsOxYSY8T/krf/Su1ZuFd4wRKrOMAHWxtPoxXfEos5vmJcL9IxtDM4/BFU85XpBv BGGkqRPasp4jQfzMJ7/0lsHkogLSSEZUY0w74EwZXFwj1vYbUC2O0Mdg6CTZ2qFRQd lAtZQ2rL19vMORhaggPfScKJXBW8Q89bNAsQ9LFMcak6ERw8gLKzZcc4Bvq6JdhL/o e0ZLgep1/pSQ8fHz9/rVU+npzFG3jBJd9S1XQtEAdqxteamYXKzdzY3vRX7xFVG2mk c/BSyJ5akimwQ== X-Nifty-SrcIP: [153.142.97.92] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: Ingo Molnar , Josh Poimboeuf , Bernd Edlinger , Borislav Petkov , Sam Ravnborg , Masahiro Yamada , Michal Marek , linux-kernel@vger.kernel.org Subject: [PATCH 3/3] kbuild: check the presence of lzo and lz4 tools when necessary Date: Tue, 16 Oct 2018 18:10:53 +0900 Message-Id: <1539681053-24388-4-git-send-email-yamada.masahiro@socionext.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1539681053-24388-1-git-send-email-yamada.masahiro@socionext.com> References: <1539681053-24388-1-git-send-email-yamada.masahiro@socionext.com> Sender: linux-kbuild-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If CONFIG_KERNEL_LZ4 is enabled without lz4 tool installed on the system, the build fails at the very last stage (reported by Borislav Petkov [1]). LZO arch/x86/boot/compressed/vmlinux.bin.lzo /bin/sh: 1: lzop: not found arch/x86/boot/compressed/Makefile:141: recipe for target 'arch/x86/boot/compressed/vmlinux.bin.lzo' failed make[2]: *** [arch/x86/boot/compressed/vmlinux.bin.lzo] Error 1 arch/x86/boot/Makefile:112: recipe for target 'arch/x86/boot/compressed/vmlinux' failed make[1]: *** [arch/x86/boot/compressed/vmlinux] Error 2 arch/x86/Makefile:284: recipe for target 'bzImage' failed make: *** [bzImage] Error 2 Check the tools in scripts/Makefile.toolcheck to fail the build earlier with a more readable message. [1] https://patchwork.kernel.org/patch/10635381/ Suggested-by: Borislav Petkov Signed-off-by: Masahiro Yamada --- scripts/Makefile.toolcheck | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/scripts/Makefile.toolcheck b/scripts/Makefile.toolcheck index bc26fc0..5e336e4 100644 --- a/scripts/Makefile.toolcheck +++ b/scripts/Makefile.toolcheck @@ -17,6 +17,14 @@ msg_stack_validation = "libelf is necessary for building the objtool." \ "Please install libelf-dev, libelf-devel or elfutils-libelf-devel." toolcheck-$(CONFIG_STACK_VALIDATION) += stack_validation +chk_lzo = command -v lzop +msg_lzo = "lzo tool not found. Please install it." +toolcheck-$(CONFIG_KERNEL_LZO) += lzo + +chk_lz4 = command -v lz4c +msg_lz4 = "lz4 tool not found. Please install it." +toolcheck-$(CONFIG_KERNEL_LZ4) += lz4 + PHONY += $(toolcheck-y) __toolcheck: $(toolcheck-y)