From patchwork Sat Mar 31 02:20:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 10318503 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BE588602D6 for ; Sat, 31 Mar 2018 02:21:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A4BD42A202 for ; Sat, 31 Mar 2018 02:21:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 995902A5CD; Sat, 31 Mar 2018 02:21:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4593E2A202 for ; Sat, 31 Mar 2018 02:21:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752805AbeCaCVJ (ORCPT ); Fri, 30 Mar 2018 22:21:09 -0400 Received: from conssluserg-02.nifty.com ([210.131.2.81]:23350 "EHLO conssluserg-02.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752784AbeCaCVH (ORCPT ); Fri, 30 Mar 2018 22:21:07 -0400 Received: from mail-ua0-f174.google.com (mail-ua0-f174.google.com [209.85.217.174]) (authenticated) by conssluserg-02.nifty.com with ESMTP id w2V2L3b7013469; Sat, 31 Mar 2018 11:21:04 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conssluserg-02.nifty.com w2V2L3b7013469 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1522462864; bh=jDrRkn+8oAb9u66Yb/kMcqnwkJTaLHpHFxhTRsS63e4=; h=In-Reply-To:References:From:Date:Subject:To:Cc:From; b=q1dUoCTqakyXql9zygoIqJ8AXnjrbcTo4a6siVg/NlLkd88OgYV3C64MJjcMu2tV7 8u++FnuUZcHiPDs1hm1QhsmVBQL7BrZNJRj+Mhc8IL8bPzB9xbpT5EdAbkETn1T7TG I/3dcqgu+w96qNUq9ozLwbs3ENJG8imP2vTAYgw196osahc5cgC633ty2OgYppFjfL VPe58H6sDAAqYIKmuIVN0DTuQcoz0PROkXoV4lUnnsIJwyGytvcGjT6prENp9qk/vy h3aGokyIzt5xKiPhUnrIctMeT+Tr3XcJXowN/wjtsxAza57n4e1Ypggoye6trHqdXD 0dDiPu43+APDg== X-Nifty-SrcIP: [209.85.217.174] Received: by mail-ua0-f174.google.com with SMTP id v4so5753044uaj.3; Fri, 30 Mar 2018 19:21:04 -0700 (PDT) X-Gm-Message-State: ALQs6tAgCKEiSHpC0mFFjWPJCSs+GyLgTeHIzRVVtKUu5gDngd58TKGK fvKakt31d9HzBOtsowo57Fss9eZFLiRQ8lcvkb8= X-Google-Smtp-Source: AIpwx4/xdPGYaKEfoZJ732kUUSMe7ODLT5wSqcBjFd0Oyd9NcljpE8BN+wFA8vQgQRVMv5bfhq/IJTdpB719SCXCe00= X-Received: by 10.159.58.238 with SMTP id q46mr712396uag.52.1522462863268; Fri, 30 Mar 2018 19:21:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.29.150 with HTTP; Fri, 30 Mar 2018 19:20:22 -0700 (PDT) In-Reply-To: <20180330222145.GC20025@outlook.office365.com> References: <1521810279-6282-3-git-send-email-yamada.masahiro@socionext.com> <20180330174022.GA8890@outlook.office365.com> <20180330222145.GC20025@outlook.office365.com> From: Masahiro Yamada Date: Sat, 31 Mar 2018 11:20:22 +0900 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [03/10] genksyms: generate lexer and parser during build instead of shipping To: Andrei Vagin Cc: Linux Kbuild mailing list , Sam Ravnborg , Michal Marek , Linux Kernel Mailing List , Arnd Bergmann , Laurent Pinchart , Frank Rowand , Geert Uytterhoeven Sender: linux-kbuild-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP 2018-03-31 7:21 GMT+09:00 Andrei Vagin : > On Fri, Mar 30, 2018 at 10:40:22AM -0700, Andrei Vagin wrote: >> On Fri, Mar 23, 2018 at 10:04:32PM +0900, Masahiro Yamada wrote: >> > Now that the kernel build supports flex and bison, remove the _shipped >> > files and generate them during the build instead. >> > >> > There are no more shipped lexer and parser, so I ripped off the rules >> > in scripts/Malefile.lib that were used for REGENERATE_PARSERS. >> > >> > The genksyms parser has ambiguous grammar, which would emit warnings: >> > >> > scripts/genksyms/parse.y: warning: 9 shift/reduce conflicts [-Wconflicts-sr] >> > scripts/genksyms/parse.y: warning: 5 reduce/reduce conflicts [-Wconflicts-rr] >> > >> > They are normally suppressed, but displayed when W=1 is given. >> > >> > Signed-off-by: Masahiro Yamada >> > --- >> > >> > scripts/Makefile.lib | 24 +- >> > scripts/genksyms/Makefile | 23 + >> > scripts/genksyms/lex.lex.c_shipped | 2291 -------------------------------- >> > scripts/genksyms/parse.tab.c_shipped | 2394 ---------------------------------- >> > scripts/genksyms/parse.tab.h_shipped | 119 -- >> > 5 files changed, 26 insertions(+), 4825 deletions(-) >> > delete mode 100644 scripts/genksyms/lex.lex.c_shipped >> > delete mode 100644 scripts/genksyms/parse.tab.c_shipped >> > delete mode 100644 scripts/genksyms/parse.tab.h_shipped >> > >> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib >> > index 2fde810..b7d2c97 100644 >> > --- a/scripts/Makefile.lib >> > +++ b/scripts/Makefile.lib >> > @@ -183,14 +183,8 @@ endef >> > quiet_cmd_flex = LEX $@ >> > cmd_flex = $(LEX) -o$@ -L $< >> > >> > -ifdef REGENERATE_PARSERS >> > -.PRECIOUS: $(src)/%.lex.c_shipped >> > -$(src)/%.lex.c_shipped: $(src)/%.l >> > - $(call cmd,flex) >> > -endif >> > - >> > .PRECIOUS: $(obj)/%.lex.c >> > -$(filter %.lex.c,$(targets)): $(obj)/%.lex.c: $(src)/%.l FORCE >> > +$(obj)/%.lex.c: $(src)/%.l FORCE >> > $(call if_changed,flex) >> > >> > # YACC >> > @@ -198,27 +192,15 @@ $(filter %.lex.c,$(targets)): $(obj)/%.lex.c: $(src)/%.l FORCE >> > quiet_cmd_bison = YACC $@ >> > cmd_bison = $(YACC) -o$@ -t -l $< >> > >> > -ifdef REGENERATE_PARSERS >> > -.PRECIOUS: $(src)/%.tab.c_shipped >> > -$(src)/%.tab.c_shipped: $(src)/%.y >> > - $(call cmd,bison) >> > -endif >> > - >> > .PRECIOUS: $(obj)/%.tab.c >> > -$(filter %.tab.c,$(targets)): $(obj)/%.tab.c: $(src)/%.y FORCE >> > +$(obj)/%.tab.c: $(src)/%.y FORCE >> > $(call if_changed,bison) >> > >> > quiet_cmd_bison_h = YACC $@ >> > cmd_bison_h = bison -o/dev/null --defines=$@ -t -l $< >> > >> > -ifdef REGENERATE_PARSERS >> > -.PRECIOUS: $(src)/%.tab.h_shipped >> > -$(src)/%.tab.h_shipped: $(src)/%.y >> > - $(call cmd,bison_h) >> > -endif >> > - >> > .PRECIOUS: $(obj)/%.tab.h >> > -$(filter %.tab.h,$(targets)): $(obj)/%.tab.h: $(src)/%.y FORCE >> > +$(obj)/%.tab.h: $(src)/%.y FORCE >> > $(call if_changed,bison_h) >> > >> > # Shipped files >> > diff --git a/scripts/genksyms/Makefile b/scripts/genksyms/Makefile >> > index 0ccac51..f4749e8 100644 >> > --- a/scripts/genksyms/Makefile >> > +++ b/scripts/genksyms/Makefile >> > @@ -5,9 +5,32 @@ always := $(hostprogs-y) >> > >> > genksyms-objs := genksyms.o parse.tab.o lex.lex.o >> > >> > +# FIXME: fix the ambiguous grammar in parse.y and delete this hack >> > +# >> > +# Suppress shift/reduce, reduce/reduce conflicts warnings >> > +# unless W=1 is specified. >> > +ifeq ($(findstring 1,$(KBUILD_ENABLE_EXTRA_GCC_CHECKS)),) >> > +SUPPRESS_BISON_WARNING := 2>/dev/null >> >> We have a robot which runs CRIU tests on linux-next. >> Yesterday it failed with this error: >> >> HOSTCC scripts/genksyms/genksyms.o >> make[2]: *** [scripts/genksyms/parse.tab.c] Error 127 >> >> cripts/genksyms/Makefile:20: recipe for target 'scripts/genksyms/parse.tab.c' failed >> scripts/Makefile.build:559: recipe for target 'scripts/genksyms' failed >> Makefile:1073: recipe for target 'scripts' failed >> make[1]: *** [scripts/genksyms] Error 2 >> make: *** [scripts] Error 2 >> make: *** Waiting for unfinished jobs.... >> >> https://travis-ci.org/avagin/linux/jobs/360056903 >> >> From this output, it is very hard to understand what was going wrong. > > > The reason was that bison and fles were not installed, but I think the > error message should be more clear. > >> >> Thanks, >> Andrei >> Thanks for the report. OK, I will apply the fix-up attached below. If bison is not installed, it will fail with clear message. HOSTCC scripts/genksyms/genksyms.o /bin/sh: 1: bison: not found make[2]: *** [scripts/genksyms/Makefile:18: scripts/genksyms/parse.tab.c] Error 127 make[1]: *** [scripts/Makefile.build:559: scripts/genksyms] Error 2 make: *** [Makefile:1073: scripts] Error 2 BTW, without flex and bison, how did you build Kconfig? Since commit 29c833061c1d8c2d1d23a62e7061561eadd76cdb, Kconfig requires flex and bison, and this commit is in Linus' tree for a couple of month. Without flex and bison, you cannot do even "make *config" Nobody has ever complaint about this. # -I needed for generated C source (shipped source) diff --git a/scripts/genksyms/Makefile b/scripts/genksyms/Makefile index aeefd47..7cb3d70 100644 --- a/scripts/genksyms/Makefile +++ b/scripts/genksyms/Makefile @@ -10,20 +10,20 @@ genksyms-objs := genksyms.o parse.tab.o lex.lex.o # Suppress shift/reduce, reduce/reduce conflicts warnings # unless W=1 is specified. ifeq ($(findstring 1,$(KBUILD_ENABLE_EXTRA_GCC_CHECKS)),) -SUPPRESS_BISON_WARNING := 2>/dev/null -endif quiet_cmd_bison_no_warn = $(quet_cmd_bison) - cmd_bison_no_warn = $(cmd_bison) $(SUPPRESS_BISON_WARNING) + cmd_bison_no_warn = $(YACC) --version >/dev/null; $(cmd_bison) 2>/dev/null $(obj)/parse.tab.c: $(src)/parse.y FORCE $(call if_changed,bison_no_warn) quiet_cmd_bison_h_no_warn = $(quet_cmd_bison_h) - cmd_bison_h_no_warn = $(cmd_bison_h) $(SUPPRESS_BISON_WARNING) + cmd_bison_h_no_warn = $(YACC) --version >/dev/null; $(cmd_bison_h) 2>/dev/null $(obj)/parse.tab.h: $(src)/parse.y FORCE $(call if_changed,bison_h_no_warn) + +endif # FIXME END