| Message ID | 20180628111623.3807fe9b@naga.suse.cz (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, 28 Jun 2018 11:16:23 +0200 Michal Suchánek <msuchanek@suse.de> wrote: > On Wed, 27 Jun 2018 23:07:21 +0900 > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > > Hi. > > > > > > 2018-06-27 21:37 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > > > Hello, > > > > > > in the x86 Kconfig we have this: > > > > > > # Select 32 or 64 bit > > > config 64BIT > > > bool "64-bit kernel" if "$(ARCH)" = "x86" > > > default "$(ARCH)" != "i386" > > > ---help--- > > > Say yes to build a 64-bit kernel - formerly known as > > > x86_64 Say no to build a 32-bit kernel - formerly known as i386 > > > > > > Since commit 104daea149c4 ("kconfig: reference environment > > > variables directly and remove 'option env='") the value of ARCH > > > is not saved in the kernel config. > > > > I think this commit is unrelated. It was just a syntax change. > > This does not look like syntax only change to me: > > diff --git a/init/Kconfig b/init/Kconfig > index 15aae32e0719..1217fc62ca61 100644 > --- a/init/Kconfig > +++ b/init/Kconfig > @@ -1,20 +1,12 @@ > -config ARCH > - string > - option env="ARCH" > - > -config KERNELVERSION > - string > - option env="KERNELVERSION" > - > > > > > Unless I am missing something, > > we have never saved ARCH in the .config in the past. > > There was a config symbol defined for it before the commit removed it. nonetheless its value was not saved anyway. > > > > > > > > Since commit f467c5640c29 ("kconfig: only write '# > > > CONFIG_FOO is not set' for visible symbols") the value of 64BIT is > > > not saved if the ARCH is set i386 or x86_64 because the symbol is > > > not visible. > > > > This is correct. > > > > It was discussed a few weeks ago. > > > > https://lkml.org/lkml/2018/6/5/847 And it went nowhere. Anyway, the observed issue with CONFIG_64BIT on x86 is the tip of a larger problem which was unnoticed for ages. The .config simply does not contain the whole kernel configuration. ie. make oldconfig (and make syncconfig) is *not* expected to just work. It used to work just by luck until f467c5640c29 ("kconfig: only write '# CONFIG_FOO is not set' for visible symbols") finally exposed the problem. So is .config supposed to contain the kernel configuration or is it just some byproduct of the kernel build which is meaningless outside of your build environment (the object tree, shell environment, etc). Thanks Michal -- To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2018-06-28 18:16 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > On Wed, 27 Jun 2018 23:07:21 +0900 > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > >> Hi. >> >> >> 2018-06-27 21:37 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: >> > Hello, >> > >> > in the x86 Kconfig we have this: >> > >> > # Select 32 or 64 bit >> > config 64BIT >> > bool "64-bit kernel" if "$(ARCH)" = "x86" >> > default "$(ARCH)" != "i386" >> > ---help--- >> > Say yes to build a 64-bit kernel - formerly known as >> > x86_64 Say no to build a 32-bit kernel - formerly known as i386 >> > >> > Since commit 104daea149c4 ("kconfig: reference environment variables >> > directly and remove 'option env='") the value of ARCH is not saved >> > in the kernel config. >> >> I think this commit is unrelated. It was just a syntax change. > > This does not look like syntax only change to me: > > diff --git a/init/Kconfig b/init/Kconfig > index 15aae32e0719..1217fc62ca61 100644 > --- a/init/Kconfig > +++ b/init/Kconfig > @@ -1,20 +1,12 @@ > -config ARCH > - string > - option env="ARCH" > - > -config KERNELVERSION > - string > - option env="KERNELVERSION" > - This is just syntax change. 'option env=' was used to reference an environment variable. Now, $(ARCH), $(KERNELVERSION) are simpler forms. >> >> Unless I am missing something, >> we have never saved ARCH in the .config in the past. > > There was a config symbol defined for it before the commit removed it. No. CONFIG symbols with'option env=' are not written out to the .config file. We have never had CONFIG_ARCH or CONFIG_KERNELVERSION. >> >> >> > Since commit f467c5640c29 ("kconfig: only write '# >> > CONFIG_FOO is not set' for visible symbols") the value of 64BIT is >> > not saved if the ARCH is set i386 or x86_64 because the symbol is >> > not visible. >> >> This is correct. >> >> It was discussed a few weeks ago. >> >> https://lkml.org/lkml/2018/6/5/847 >> >> >> > There is a number of ways to hack this particular case to work. >> > >> > However, there is a more general problem with this. Some config >> > options may depend on the environment, may not be saved, and the >> > environment is not saved either. >> >> Which environment variables in particular are in your mind? > > Any that is used in Kconfig. They are provided from outside of Kconfig. This is the behavior we keep since a long time ago. ARCH is given by the environment variable or the command line. KERNELVERSION is supplied by the top Makefile. >> >> As for ARCH, you need to pass the same ARCH as you used for building >> the kernel. (For native building, you do not have to pass ARCH >> explicitly, though.) > > Except if you do pass it to make config you may need to pass it to to > make later as well. Right. For exmaple 'make ARCH=arm config' will create the config suitable only for ARM architecture. Then, you need to do 'make ARCH=arm' to build the kernel. If it is tedious to give 'ARCH=arm' to every make command, you can do 'export ARCH=arm' in your shell. Again, this is the behavior we have for a long time. >> >> As for CC, HOSTCC, etc. >> yes, these are new 'unsaved' environments. >> >> CONFIG options now depend on the compiler. >> This is the concept suggested by Linus Torvalds. >> >> >> > So in the end all the infrastructure with symlinks >> > from module directory pointing to the kernel source and object >> > directory is useless. To interpret the config stored there you need >> > the environment and that is not saved anywhere. So if you try to >> > build out-of-tree module it might end up reconfiguring your kernel >> > and producing useless modules. >> >> No. out-of-tree module building never ever re-configures the kernel. > > It does implicitly because the config values depend on environment that > is not saved and the values themselves are not saved either. If that > happens to expose a new variable it is even explicitly reconfigured. You should have a built kernel tree before building external modules. The .config is already there. The .config works for external modules, given that - ARCH is the same - the compiler is the same >> >> out-of-tree modules are built with exactly the same configuration >> as used for the kernel. > > It is not true. And that is the problem. You need the config file and > dump of the environment passed to the make command at configuration > time to get the exact same configuration. The environment is not saved > anywhere, though. Why dump of the environment? If you are building external modules natively your distribution provides /lib/modules/$(uname -r)/build, which contains files enough for building external modules. You can pass the directory path to M=... parameter. That's it. If you are cross-building external modules, you also need to - pass ARCH= - use the same compiler with CROSS_COMPILE= You should know both because you have built the kernel by your self. You do not need any other information, do you? > And it went nowhere. > > Anyway, the observed issue with CONFIG_64BIT on x86 is the tip of a > larger problem which was unnoticed for ages. The .config simply does > not contain the whole kernel configuration. ie. make oldconfig (and > make syncconfig) is *not* expected to just work. It used to work just by > luck until f467c5640c29 ("kconfig: only write '# CONFIG_FOO is not set' > for visible symbols") finally exposed the problem. If you want to build the kernel for an architecture other than the host machine architecture, you need to pass ARCH=. Building the i386 kernel on a x86_64 machine, it is a _kind_ of cross-compiling. So, passing ARCH=i386 is not so weird. > So is .config supposed to contain the kernel configuration or is it > just some byproduct of the kernel build which is meaningless outside of > your build environment (the object tree, shell environment, etc). The .config is supposed to contain the kernel configuration, 'ARCH' and the compiler are exceptions. 'ARCH' must be passed separately. The .config now depends on the compiler. So, if you pass your .config to somebody else, some symbols that depend on the compiler support might be configured differently. 'make syncconfig' will notice the compiler difference, and show prompts for user input as needed.
On Mon, 30 Jul 2018 17:02:42 +0900 Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > 2018-06-28 18:16 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > > On Wed, 27 Jun 2018 23:07:21 +0900 > > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > > >> Hi. > >> > >> > >> 2018-06-27 21:37 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > >> > Hello, > >> > > >> > in the x86 Kconfig we have this: > >> > > >> > # Select 32 or 64 bit > >> > config 64BIT > >> > bool "64-bit kernel" if "$(ARCH)" = "x86" > >> > default "$(ARCH)" != "i386" > >> > ---help--- > >> > Say yes to build a 64-bit kernel - formerly known as > >> > x86_64 Say no to build a 32-bit kernel - formerly known as i386 > >> > > >> > Since commit 104daea149c4 ("kconfig: reference environment > >> > variables directly and remove 'option env='") the value of ARCH > >> > is not saved in the kernel config. > >> > >> I think this commit is unrelated. It was just a syntax change. > > > > This does not look like syntax only change to me: > > > > diff --git a/init/Kconfig b/init/Kconfig > > index 15aae32e0719..1217fc62ca61 100644 > > --- a/init/Kconfig > > +++ b/init/Kconfig > > @@ -1,20 +1,12 @@ > > -config ARCH > > - string > > - option env="ARCH" > > - > > -config KERNELVERSION > > - string > > - option env="KERNELVERSION" > > - > > This is just syntax change. > > 'option env=' was used to reference an environment variable. > > Now, $(ARCH), $(KERNELVERSION) are simpler forms. > > > >> > >> Unless I am missing something, > >> we have never saved ARCH in the .config in the past. > > > > There was a config symbol defined for it before the commit removed > > it. > > No. > > CONFIG symbols with'option env=' > are not written out to the .config file. > > We have never had CONFIG_ARCH or CONFIG_KERNELVERSION. > > > > > >> > >> > >> > Since commit f467c5640c29 ("kconfig: only write '# > >> > CONFIG_FOO is not set' for visible symbols") the value of 64BIT > >> > is not saved if the ARCH is set i386 or x86_64 because the > >> > symbol is not visible. > >> > >> This is correct. > >> > >> It was discussed a few weeks ago. > >> > >> https://lkml.org/lkml/2018/6/5/847 > >> > >> > >> > There is a number of ways to hack this particular case to work. > >> > > >> > However, there is a more general problem with this. Some config > >> > options may depend on the environment, may not be saved, and the > >> > environment is not saved either. > >> > >> Which environment variables in particular are in your mind? > > > > Any that is used in Kconfig. > > They are provided from outside of Kconfig. > This is the behavior we keep since a long time ago. > > ARCH is given by the environment variable or the command line. > KERNELVERSION is supplied by the top Makefile. > > >> > >> As for ARCH, you need to pass the same ARCH as you used for > >> building the kernel. (For native building, you do not have to pass > >> ARCH explicitly, though.) > > > > Except if you do pass it to make config you may need to pass it to > > to make later as well. > > Right. > > For exmaple 'make ARCH=arm config' will create the config suitable > only for ARM architecture. > Then, you need to do 'make ARCH=arm' to build the kernel. > > If it is tedious to give 'ARCH=arm' to every make command, > you can do 'export ARCH=arm' in your shell. > > Again, this is the behavior we have for a long time. No, that's not what we had. The kernel build would fail instead of reconfiguring the kernel for the current arch. At least it used to work that way at some point. > > > > >> > >> As for CC, HOSTCC, etc. > >> yes, these are new 'unsaved' environments. > >> > >> CONFIG options now depend on the compiler. > >> This is the concept suggested by Linus Torvalds. > >> > >> > >> > So in the end all the infrastructure with symlinks > >> > from module directory pointing to the kernel source and object > >> > directory is useless. To interpret the config stored there you > >> > need the environment and that is not saved anywhere. So if you > >> > try to build out-of-tree module it might end up reconfiguring > >> > your kernel and producing useless modules. > >> > >> No. out-of-tree module building never ever re-configures the > >> kernel. > > > > It does implicitly because the config values depend on environment > > that is not saved and the values themselves are not saved either. > > If that happens to expose a new variable it is even explicitly > > reconfigured. > > > You should have a built kernel tree > before building external modules. > > The .config is already there. > > The .config works for external modules, given that > > - ARCH is the same > - the compiler is the same - the compiler additional plugins and/or external libraries used to implement advanced features are the same > > > > >> > >> out-of-tree modules are built with exactly the same configuration > >> as used for the kernel. > > > > It is not true. And that is the problem. You need the config file > > and dump of the environment passed to the make command at > > configuration time to get the exact same configuration. The > > environment is not saved anywhere, though. > > > Why dump of the environment? > > > If you are building external modules natively > your distribution provides /lib/modules/$(uname -r)/build, > which contains files enough for building external modules. > > You can pass the directory path to M=... parameter. That's it. No, that's not it. Since passing ARCH=i386 is the de-facto standard to configure a 32bit kernel and the result of passing that was not saved you need to pass it to make as well. And you need to patch a number of 3rd party build scripts that build a kernel module as part of a bigger project. > > > > If you are cross-building external modules, > you also need to > > - pass ARCH= > - use the same compiler with CROSS_COMPILE= > > You should know both > because you have built the kernel by your self. No, I am not cross-compiling. I am building 32bit modules and because the kconfig system did not save the information that the kernel is 32bit I get 64bit modules instead. > > You do not need any other information, do you? I need the information what ARCH was passed to the x86 kernel at configuration time when building on x86. This is new. To avoid this and similar surprises in the future I suggest to flag any option that depends on something dynamic (compiler option, environment variable) so that kconfig saves it even if it would not be saved otherwise. Then kconfig can do whatever seems sensible for oldconfig but syncconfig should not change any options based on dynamic input. It should verify that the option matches (ie compiler can accept -foobar if the option is on and defaults to cc-option -foobar) and fail if it does not. > > > And it went nowhere. > > > > Anyway, the observed issue with CONFIG_64BIT on x86 is the tip of a > > larger problem which was unnoticed for ages. The .config simply does > > not contain the whole kernel configuration. ie. make oldconfig (and > > make syncconfig) is *not* expected to just work. It used to work > > just by luck until f467c5640c29 ("kconfig: only write '# CONFIG_FOO > > is not set' for visible symbols") finally exposed the problem. > > If you want to build the kernel for an architecture > other than the host machine architecture, you need to pass ARCH=. > > Building the i386 kernel on a x86_64 machine, it is a _kind_ of > cross-compiling. So, passing ARCH=i386 is not so weird. > > > > So is .config supposed to contain the kernel configuration or is it > > just some byproduct of the kernel build which is meaningless > > outside of your build environment (the object tree, shell > > environment, etc). > > The .config is supposed to contain the kernel configuration, > 'ARCH' and the compiler are exceptions. > > 'ARCH' must be passed separately. > > The .config now depends on the compiler. So, if you pass your .config > to somebody else, some symbols that depend on the compiler support > might be configured differently. > > 'make syncconfig' will notice the compiler difference, > and show prompts for user input as needed. It should not prompt. I want to build the kernel non-interactively. I do not want kernel rpm build to wait for user input or silently change the kernel ABI just because I installed a new gcc plugin. Thanks Michal -- To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jul 30, 2018 at 05:02:42PM +0900, Masahiro Yamada wrote: > > For exmaple 'make ARCH=arm config' will create the config suitable > only for ARM architecture. > Then, you need to do 'make ARCH=arm' to build the kernel. > > If it is tedious to give 'ARCH=arm' to every make command, > you can do 'export ARCH=arm' in your shell. > > Again, this is the behavior we have for a long time. Actually, this no longer works reliably. For example, when I run ARCH=powerpc make oldconfig with our ppc64le config on x86_64 system, I get different result than when I run it on an actual ppc64le system (or when using a ppc64le cross compiler on x86_64). Since .config started to mix user configuration and build environment capabilities, maintaining distribution configs became real pain. And it's getting progressively worse. Michal Kubecek -- To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2018-08-07 3:07 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > On Mon, 30 Jul 2018 17:02:42 +0900 > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > >> 2018-06-28 18:16 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: >> > On Wed, 27 Jun 2018 23:07:21 +0900 >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: >> > >> >> Hi. >> >> >> >> >> >> 2018-06-27 21:37 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: >> >> > Hello, >> >> > >> >> > in the x86 Kconfig we have this: >> >> > >> >> > # Select 32 or 64 bit >> >> > config 64BIT >> >> > bool "64-bit kernel" if "$(ARCH)" = "x86" >> >> > default "$(ARCH)" != "i386" >> >> > ---help--- >> >> > Say yes to build a 64-bit kernel - formerly known as >> >> > x86_64 Say no to build a 32-bit kernel - formerly known as i386 >> >> > >> >> > Since commit 104daea149c4 ("kconfig: reference environment >> >> > variables directly and remove 'option env='") the value of ARCH >> >> > is not saved in the kernel config. >> >> >> >> I think this commit is unrelated. It was just a syntax change. >> > >> > This does not look like syntax only change to me: >> > >> > diff --git a/init/Kconfig b/init/Kconfig >> > index 15aae32e0719..1217fc62ca61 100644 >> > --- a/init/Kconfig >> > +++ b/init/Kconfig >> > @@ -1,20 +1,12 @@ >> > -config ARCH >> > - string >> > - option env="ARCH" >> > - >> > -config KERNELVERSION >> > - string >> > - option env="KERNELVERSION" >> > - >> >> This is just syntax change. >> >> 'option env=' was used to reference an environment variable. >> >> Now, $(ARCH), $(KERNELVERSION) are simpler forms. >> >> >> >> >> >> Unless I am missing something, >> >> we have never saved ARCH in the .config in the past. >> > >> > There was a config symbol defined for it before the commit removed >> > it. >> >> No. >> >> CONFIG symbols with'option env=' >> are not written out to the .config file. >> >> We have never had CONFIG_ARCH or CONFIG_KERNELVERSION. >> >> >> >> >> >> >> >> >> >> > Since commit f467c5640c29 ("kconfig: only write '# >> >> > CONFIG_FOO is not set' for visible symbols") the value of 64BIT >> >> > is not saved if the ARCH is set i386 or x86_64 because the >> >> > symbol is not visible. >> >> >> >> This is correct. >> >> >> >> It was discussed a few weeks ago. >> >> >> >> https://lkml.org/lkml/2018/6/5/847 >> >> >> >> >> >> > There is a number of ways to hack this particular case to work. >> >> > >> >> > However, there is a more general problem with this. Some config >> >> > options may depend on the environment, may not be saved, and the >> >> > environment is not saved either. >> >> >> >> Which environment variables in particular are in your mind? >> > >> > Any that is used in Kconfig. >> >> They are provided from outside of Kconfig. >> This is the behavior we keep since a long time ago. >> >> ARCH is given by the environment variable or the command line. >> KERNELVERSION is supplied by the top Makefile. >> >> >> >> >> As for ARCH, you need to pass the same ARCH as you used for >> >> building the kernel. (For native building, you do not have to pass >> >> ARCH explicitly, though.) >> > >> > Except if you do pass it to make config you may need to pass it to >> > to make later as well. >> >> Right. >> >> For exmaple 'make ARCH=arm config' will create the config suitable >> only for ARM architecture. >> Then, you need to do 'make ARCH=arm' to build the kernel. >> >> If it is tedious to give 'ARCH=arm' to every make command, >> you can do 'export ARCH=arm' in your shell. >> >> Again, this is the behavior we have for a long time. > > No, that's not what we had. The kernel build would fail instead of > reconfiguring the kernel for the current arch. At least it used to work > that way at some point. This is not true for most architectures. It worked for i386 prior to commit f467c5640c29 ("kconfig: only write '# CONFIG_FOO is not set' for visible symbols"). This was already discussed, and this can be solved by setting ARCH=i386. https://lkml.org/lkml/2018/6/7/1119 https://lkml.org/lkml/2018/6/8/149 You complaint about commit 104daea149c4 ("kconfig: reference environment variables directly and remove 'option env='"), but it is unrelated. >> >> >> >> >> >> >> As for CC, HOSTCC, etc. >> >> yes, these are new 'unsaved' environments. >> >> >> >> CONFIG options now depend on the compiler. >> >> This is the concept suggested by Linus Torvalds. >> >> >> >> >> >> > So in the end all the infrastructure with symlinks >> >> > from module directory pointing to the kernel source and object >> >> > directory is useless. To interpret the config stored there you >> >> > need the environment and that is not saved anywhere. So if you >> >> > try to build out-of-tree module it might end up reconfiguring >> >> > your kernel and producing useless modules. >> >> >> >> No. out-of-tree module building never ever re-configures the >> >> kernel. >> > >> > It does implicitly because the config values depend on environment >> > that is not saved and the values themselves are not saved either. >> > If that happens to expose a new variable it is even explicitly >> > reconfigured. >> >> >> You should have a built kernel tree >> before building external modules. >> >> The .config is already there. >> >> The .config works for external modules, given that >> >> - ARCH is the same >> - the compiler is the same > > - the compiler additional plugins and/or external libraries used to > implement advanced features are the same > >> >> >> >> >> >> >> out-of-tree modules are built with exactly the same configuration >> >> as used for the kernel. >> > >> > It is not true. And that is the problem. You need the config file >> > and dump of the environment passed to the make command at >> > configuration time to get the exact same configuration. The >> > environment is not saved anywhere, though. >> >> >> Why dump of the environment? >> >> >> If you are building external modules natively >> your distribution provides /lib/modules/$(uname -r)/build, >> which contains files enough for building external modules. >> >> You can pass the directory path to M=... parameter. That's it. > > No, that's not it. Since passing ARCH=i386 is the de-facto standard to > configure a 32bit kernel and the result of passing that was not saved > you need to pass it to make as well. If you pass ARCH= for the configuration phase, you need to pass the same ARCH= in the build phase. What I can suggest for you is: $ make ARCH=i386 defconfig $ make ARCH=i386 OR $ make i386_defconfig $ make > And you need to patch a number of > 3rd party build scripts that build a kernel module as part of a bigger > project. You do not need to patch a number of scripts. Just one liner fix. Add 'export ARCH=i386' in the top level script. >> >> >> >> If you are cross-building external modules, >> you also need to >> >> - pass ARCH= >> - use the same compiler with CROSS_COMPILE= >> >> You should know both >> because you have built the kernel by your self. > > No, I am not cross-compiling. I am building 32bit modules and because > the kconfig system did not save the information that the kernel is > 32bit I get 64bit modules instead. You are semi-cross-compiling if you build 32-bit modules on a 64-bit host machine. >> >> You do not need any other information, do you? > > I need the information what ARCH was passed to the x86 kernel at > configuration time when building on x86. This is new. > > To avoid this and similar surprises in the future I suggest to flag any > option that depends on something dynamic (compiler option, environment > variable) so that kconfig saves it even if it would not be saved > otherwise. It is saved in include/config/auto.conf.cmd > Then kconfig can do whatever seems sensible for oldconfig but > syncconfig should not change any options based on dynamic input. It > should verify that the option matches (ie compiler can accept -foobar > if the option is on and defaults to cc-option -foobar) and fail if it > does not. Kconfig already supports this. Try KCONFIG_NOSILENTUPDATE=1 > >> >> > And it went nowhere. >> > >> > Anyway, the observed issue with CONFIG_64BIT on x86 is the tip of a >> > larger problem which was unnoticed for ages. The .config simply does >> > not contain the whole kernel configuration. ie. make oldconfig (and >> > make syncconfig) is *not* expected to just work. It used to work >> > just by luck until f467c5640c29 ("kconfig: only write '# CONFIG_FOO >> > is not set' for visible symbols") finally exposed the problem. >> >> If you want to build the kernel for an architecture >> other than the host machine architecture, you need to pass ARCH=. >> >> Building the i386 kernel on a x86_64 machine, it is a _kind_ of >> cross-compiling. So, passing ARCH=i386 is not so weird. >> >> >> > So is .config supposed to contain the kernel configuration or is it >> > just some byproduct of the kernel build which is meaningless >> > outside of your build environment (the object tree, shell >> > environment, etc). >> >> The .config is supposed to contain the kernel configuration, >> 'ARCH' and the compiler are exceptions. >> >> 'ARCH' must be passed separately. >> >> The .config now depends on the compiler. So, if you pass your .config >> to somebody else, some symbols that depend on the compiler support >> might be configured differently. >> >> 'make syncconfig' will notice the compiler difference, >> and show prompts for user input as needed. > > It should not prompt. I want to build the kernel non-interactively. I > do not want kernel rpm build to wait for user input or silently > change the kernel ABI just because I installed a new gcc plugin. 'make syncconfig' shows prompt. This is the behavior since a long time ago. I cannot change it to meet your requirement. If you really want to avoid prompt, run "make olddefconfig" beforehand in your script. > Thanks > > Michal > -- > To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html
2018-08-07 3:33 GMT+09:00 Michal Kubecek <mkubecek@suse.cz>: > On Mon, Jul 30, 2018 at 05:02:42PM +0900, Masahiro Yamada wrote: >> >> For exmaple 'make ARCH=arm config' will create the config suitable >> only for ARM architecture. >> Then, you need to do 'make ARCH=arm' to build the kernel. >> >> If it is tedious to give 'ARCH=arm' to every make command, >> you can do 'export ARCH=arm' in your shell. >> >> Again, this is the behavior we have for a long time. > > Actually, this no longer works reliably. For example, when I run > > ARCH=powerpc make oldconfig > > with our ppc64le config on x86_64 system, I get different result than > when I run it on an actual ppc64le system (or when using a ppc64le cross > compiler on x86_64). This is because you used different compilers. > Since .config started to mix user configuration and build environment > capabilities, maintaining distribution configs became real pain. And > it's getting progressively worse. This is improvement requested by Linus Torvalds. Prior to the change, the user configuration in the .config did not reflect what you get. Even if you enable CONFIG_FOO=y, it might be silently disabled by $(call cc-option, ...) in Makefile if the feature is not supported by the compiler. By moving the compiler evaluation to the Kconfig phase, the .config matches to what you get. > Michal Kubecek > > -- > To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 20 Aug 2018 20:15:12 +0200, Masahiro Yamada wrote: > > 2018-08-07 3:07 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > > On Mon, 30 Jul 2018 17:02:42 +0900 > > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > > >> 2018-06-28 18:16 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > >> > On Wed, 27 Jun 2018 23:07:21 +0900 > >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > >> > > >> >> Hi. > >> >> > >> >> > >> >> 2018-06-27 21:37 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > >> >> > Hello, > >> >> > > >> >> > in the x86 Kconfig we have this: > >> >> > > >> >> > # Select 32 or 64 bit > >> >> > config 64BIT > >> >> > bool "64-bit kernel" if "$(ARCH)" = "x86" > >> >> > default "$(ARCH)" != "i386" > >> >> > ---help--- > >> >> > Say yes to build a 64-bit kernel - formerly known as > >> >> > x86_64 Say no to build a 32-bit kernel - formerly known as i386 > >> >> > > >> >> > Since commit 104daea149c4 ("kconfig: reference environment > >> >> > variables directly and remove 'option env='") the value of ARCH > >> >> > is not saved in the kernel config. > >> >> > >> >> I think this commit is unrelated. It was just a syntax change. > >> > > >> > This does not look like syntax only change to me: > >> > > >> > diff --git a/init/Kconfig b/init/Kconfig > >> > index 15aae32e0719..1217fc62ca61 100644 > >> > --- a/init/Kconfig > >> > +++ b/init/Kconfig > >> > @@ -1,20 +1,12 @@ > >> > -config ARCH > >> > - string > >> > - option env="ARCH" > >> > - > >> > -config KERNELVERSION > >> > - string > >> > - option env="KERNELVERSION" > >> > - > >> > >> This is just syntax change. > >> > >> 'option env=' was used to reference an environment variable. > >> > >> Now, $(ARCH), $(KERNELVERSION) are simpler forms. > >> > >> > >> >> > >> >> Unless I am missing something, > >> >> we have never saved ARCH in the .config in the past. > >> > > >> > There was a config symbol defined for it before the commit removed > >> > it. > >> > >> No. > >> > >> CONFIG symbols with'option env=' > >> are not written out to the .config file. > >> > >> We have never had CONFIG_ARCH or CONFIG_KERNELVERSION. Maybe it sounds like a stupid question, but... if passing ARCH= is almost mandatory for distinguishing the bi-arch or cross-compile cases, why don't we save it in .config? The whole "regression" we've seen can be worked around by passing ARCH at each time. So, by having it in .config, everything would work more easily, no? thanks, Takashi
Hello, On Tue, 21 Aug 2018 03:15:12 +0900 Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > 2018-08-07 3:07 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > > On Mon, 30 Jul 2018 17:02:42 +0900 > > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > > >> 2018-06-28 18:16 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > >> > On Wed, 27 Jun 2018 23:07:21 +0900 > >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > >> > > >> > >> > >> > >> >> > >> >> out-of-tree modules are built with exactly the same > >> >> configuration as used for the kernel. > >> > > >> > It is not true. And that is the problem. You need the config file > >> > and dump of the environment passed to the make command at > >> > configuration time to get the exact same configuration. The > >> > environment is not saved anywhere, though. > >> > >> > >> Why dump of the environment? > >> > >> > >> If you are building external modules natively > >> your distribution provides /lib/modules/$(uname -r)/build, > >> which contains files enough for building external modules. > >> > >> You can pass the directory path to M=... parameter. That's it. > > > > No, that's not it. Since passing ARCH=i386 is the de-facto standard > > to configure a 32bit kernel and the result of passing that was not > > saved you need to pass it to make as well. > > > If you pass ARCH= for the configuration phase, > you need to pass the same ARCH= in the build phase. > > > > > What I can suggest for you is: > > > $ make ARCH=i386 defconfig > $ make ARCH=i386 > > OR > > $ make i386_defconfig > $ make Maybe you missed that but I do not want to build the defconfig. I want to build a *particular* config which has been saved in a file beforehand. > > > And you need to patch a number of > > 3rd party build scripts that build a kernel module as part of a > > bigger project. > > > You do not need to patch a number of scripts. > > Just one liner fix. > Add 'export ARCH=i386' in the top level script. And that does not work because there is no toplevel script. rpm has three toplevel scripts - prep, build and install. Building out-of-tree module is another toplevel script. And it does not suffice to 'export ARCH=i386'. What if it was a 64bit config? How do I tell? With more and more options left out of the config that depend on compiler features, environment variables, phase of the moon or whatnot you can't even tell what kind of kernel the user is running by looking at the config. Thanks Michal
Hi, 2018-08-21 6:33 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: > Hello, > > On Tue, 21 Aug 2018 03:15:12 +0900 > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > >> 2018-08-07 3:07 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: >> > On Mon, 30 Jul 2018 17:02:42 +0900 >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: >> > >> >> 2018-06-28 18:16 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: >> >> > On Wed, 27 Jun 2018 23:07:21 +0900 >> >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: >> >> > > >> >> >> >> >> >> >> >> >> >> >> >> out-of-tree modules are built with exactly the same >> >> >> configuration as used for the kernel. >> >> > >> >> > It is not true. And that is the problem. You need the config file >> >> > and dump of the environment passed to the make command at >> >> > configuration time to get the exact same configuration. The >> >> > environment is not saved anywhere, though. >> >> >> >> >> >> Why dump of the environment? >> >> >> >> >> >> If you are building external modules natively >> >> your distribution provides /lib/modules/$(uname -r)/build, >> >> which contains files enough for building external modules. >> >> >> >> You can pass the directory path to M=... parameter. That's it. >> > >> > No, that's not it. Since passing ARCH=i386 is the de-facto standard >> > to configure a 32bit kernel and the result of passing that was not >> > saved you need to pass it to make as well. >> >> >> If you pass ARCH= for the configuration phase, >> you need to pass the same ARCH= in the build phase. >> >> >> >> >> What I can suggest for you is: >> >> >> $ make ARCH=i386 defconfig >> $ make ARCH=i386 >> >> OR >> >> $ make i386_defconfig >> $ make > > Maybe you missed that but I do not want to build the defconfig. I want > to build a *particular* config which has been saved in a file > beforehand. >> >> > And you need to patch a number of >> > 3rd party build scripts that build a kernel module as part of a >> > bigger project. >> >> >> You do not need to patch a number of scripts. >> >> Just one liner fix. >> Add 'export ARCH=i386' in the top level script. > > And that does not work because there is no toplevel script. rpm has > three toplevel scripts - prep, build and install. OK, I understood real problem cases. I wrote patches to support non-interactive .config updates. For package building like rpm, deb, etc. the target ARCH is determined when creating a source package. If so, it would not hurt to hard-code to record ARCH in the spec file. > Building out-of-tree module is another toplevel script. As I said before, the .config is not reconfigured when building out-of-tree modules. Out-of-tree modules are built with the .config that has been used to build vmlinux regardless of whether you pass ARCH=i386, ARCH=x86_64, or nothing. If rpmbuild is given target option like 'rpmbuild --target i386 ...", it think it makes sense to propagate it to ARCH=. > And it does not suffice to 'export ARCH=i386'. What if it was a 64bit > config? How do I tell? > > With more and more options left out of the config that depend on > compiler features, environment variables, phase of the moon or whatnot > you can't even tell what kind of kernel the user is running by looking > at the config. I added non-interactive .config update. Please try it.
Hi. 2018-08-21 3:37 GMT+09:00 Takashi Iwai <tiwai@suse.de>: > On Mon, 20 Aug 2018 20:15:12 +0200, > Masahiro Yamada wrote: >> >> 2018-08-07 3:07 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: >> > On Mon, 30 Jul 2018 17:02:42 +0900 >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: >> > >> >> 2018-06-28 18:16 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: >> >> > On Wed, 27 Jun 2018 23:07:21 +0900 >> >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote: >> >> > >> >> >> Hi. >> >> >> >> >> >> >> >> >> 2018-06-27 21:37 GMT+09:00 Michal Suchánek <msuchanek@suse.de>: >> >> >> > Hello, >> >> >> > >> >> >> > in the x86 Kconfig we have this: >> >> >> > >> >> >> > # Select 32 or 64 bit >> >> >> > config 64BIT >> >> >> > bool "64-bit kernel" if "$(ARCH)" = "x86" >> >> >> > default "$(ARCH)" != "i386" >> >> >> > ---help--- >> >> >> > Say yes to build a 64-bit kernel - formerly known as >> >> >> > x86_64 Say no to build a 32-bit kernel - formerly known as i386 >> >> >> > >> >> >> > Since commit 104daea149c4 ("kconfig: reference environment >> >> >> > variables directly and remove 'option env='") the value of ARCH >> >> >> > is not saved in the kernel config. >> >> >> >> >> >> I think this commit is unrelated. It was just a syntax change. >> >> > >> >> > This does not look like syntax only change to me: >> >> > >> >> > diff --git a/init/Kconfig b/init/Kconfig >> >> > index 15aae32e0719..1217fc62ca61 100644 >> >> > --- a/init/Kconfig >> >> > +++ b/init/Kconfig >> >> > @@ -1,20 +1,12 @@ >> >> > -config ARCH >> >> > - string >> >> > - option env="ARCH" >> >> > - >> >> > -config KERNELVERSION >> >> > - string >> >> > - option env="KERNELVERSION" >> >> > - >> >> >> >> This is just syntax change. >> >> >> >> 'option env=' was used to reference an environment variable. >> >> >> >> Now, $(ARCH), $(KERNELVERSION) are simpler forms. >> >> >> >> >> >> >> >> >> >> Unless I am missing something, >> >> >> we have never saved ARCH in the .config in the past. >> >> > >> >> > There was a config symbol defined for it before the commit removed >> >> > it. >> >> >> >> No. >> >> >> >> CONFIG symbols with'option env=' >> >> are not written out to the .config file. >> >> >> >> We have never had CONFIG_ARCH or CONFIG_KERNELVERSION. > > Maybe it sounds like a stupid question, but... > if passing ARCH= is almost mandatory for distinguishing the bi-arch or > cross-compile cases, why don't we save it in .config? The .config may not exist, may not be loaded. For "make help", "make headers_install" etc. ARCH needs be given from the command line anyway. I do not want to obscure "when to pass ARCH". > The whole "regression" we've seen can be worked around by passing > ARCH at each time. So, by having it in .config, everything would work > more easily, no? Recording ARCH in .config does not solve the problem because the .config contents depend on the compiler. > > > thanks, > > Takashi
diff --git a/init/Kconfig b/init/Kconfig index 15aae32e0719..1217fc62ca61 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1,20 +1,12 @@ -config ARCH - string - option env="ARCH" - -config KERNELVERSION - string - option env="KERNELVERSION" - > > Unless I am missing something, > we have never saved ARCH in the .config in the past. There was a config symbol defined for it before the commit removed it. > > > > Since commit f467c5640c29 ("kconfig: only write '# > > CONFIG_FOO is not set' for visible symbols") the value of 64BIT is > > not saved if the ARCH is set i386 or x86_64 because the symbol is