mbox series

[RFC,0/2] use interpreters to invoke scripts

Message ID 2b00e566-112c-5657-c10f-7f210d3eae93@gmail.com (mailing list archive)
Headers show
Series use interpreters to invoke scripts | expand

Message

Ujjwal Kumar Oct. 3, 2020, 3:19 p.m. UTC
This patch series aims at removing the dependency on execute 
bit of the scripts in the kbuild system.

If not working with fresh clone of linux-next, clean the srctree:
make distclean
make tools/clean

To test the dependency on execute bits, I tried building the 
kernel after removing x-bits for all files in the repository.
Removing execute bits:
for i in $(find -executable -type f); do chmod -x $i; done

Any attempts to configure (or build) the kernel fail because of 
'Permission denied' on scripts with the following error:
$ make allmodconfig
sh: ./scripts/gcc-version.sh: Permission denied
init/Kconfig:34: syntax error
init/Kconfig:33: invalid statement
init/Kconfig:34: invalid statement
sh: ./scripts/ld-version.sh: Permission denied
init/Kconfig:39: syntax error
init/Kconfig:38: invalid statement
sh: ./scripts/clang-version.sh: Permission denied
init/Kconfig:49: syntax error
init/Kconfig:48: invalid statement
make[1]: *** [scripts/kconfig/Makefile:71: allmodconfig] Error 1
make: *** [Makefile:606: allmodconfig] Error 2

Changes:
1. Adds specific interpreters (in Kconfig) to invoke 
scripts.

After this patch I could successfully do a kernel build 
without any errors.

2. Again, adds specific interpreters to other parts of 
kbuild system.

I could successfully perform the following make targets after 
applying the PATCH 2/2:
make headerdep
make kselftest-merge
make rpm-pkg
make perf-tar-src-pkg
make ARCH=ia64 defconfig
ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make prepare

Following changes in PATCH 2/2 are not yet tested:
arch/arm64/kernel/vdso32/Makefile
arch/nds32/kernel/vdso/Makefile
scripts/Makefile.build

Ujjwal Kumar (2):
  kconfig: use interpreters to invoke scripts
  kbuild: use interpreters to invoke scripts

 Makefile                          |  4 ++--
 arch/arm64/kernel/vdso/Makefile   |  2 +-
 arch/arm64/kernel/vdso32/Makefile |  2 +-
 arch/ia64/Makefile                |  4 ++--
 arch/nds32/kernel/vdso/Makefile   |  2 +-
 init/Kconfig                      | 16 ++++++++--------
 scripts/Makefile.build            |  2 +-
 scripts/Makefile.package          |  4 ++--
 8 files changed, 18 insertions(+), 18 deletions(-)

Comments

Lukas Bulwahn Oct. 4, 2020, 6:51 a.m. UTC | #1
On Sat, 3 Oct 2020, Ujjwal Kumar wrote:

> This patch series aims at removing the dependency on execute 
> bit of the scripts in the kbuild system.
>

Ujjwal, your setup to send out the patch series created three unrelated 
emails rather than the default way, i.e., to have one cover letter 
and the patches in reply to the cover letter.

You can see the difference here:

https://lore.kernel.org/linux-kbuild/

The presentation of your patch series looks different than the other 
patch series on the list. Use the linux-kernel-mentees list for testing 
your setup.

For this version of the patch series, I guess it is okay; but this set up 
right for the next patch series.
 
> If not working with fresh clone of linux-next, clean the srctree:
> make distclean
> make tools/clean
>

I hit an unrelated issue on next-20201002 that make tools/clean fails.
Other than that, this is all good.
 
> To test the dependency on execute bits, I tried building the 
> kernel after removing x-bits for all files in the repository.
> Removing execute bits:
> for i in $(find -executable -type f); do chmod -x $i; done
>

Okay, I did that.
 
> Any attempts to configure (or build) the kernel fail because of 
> 'Permission denied' on scripts with the following error:
> $ make allmodconfig
> sh: ./scripts/gcc-version.sh: Permission denied
> init/Kconfig:34: syntax error
> init/Kconfig:33: invalid statement
> init/Kconfig:34: invalid statement
> sh: ./scripts/ld-version.sh: Permission denied
> init/Kconfig:39: syntax error
> init/Kconfig:38: invalid statement
> sh: ./scripts/clang-version.sh: Permission denied
> init/Kconfig:49: syntax error
> init/Kconfig:48: invalid statement
> make[1]: *** [scripts/kconfig/Makefile:71: allmodconfig] Error 1
> make: *** [Makefile:606: allmodconfig] Error 2
>

I can confirm that these errors are reported on next-20201002.
 
> Changes:
> 1. Adds specific interpreters (in Kconfig) to invoke 
> scripts.
> 
> After this patch I could successfully do a kernel build 
> without any errors.
>

With this first patch, I could then successfully do:

make allmodconfig && make

So far, so good. I did check the first patch.

Lukas

> 2. Again, adds specific interpreters to other parts of 
> kbuild system.
> 
> I could successfully perform the following make targets after 
> applying the PATCH 2/2:
> make headerdep
> make kselftest-merge
> make rpm-pkg
> make perf-tar-src-pkg
> make ARCH=ia64 defconfig
> ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make prepare
> 
> Following changes in PATCH 2/2 are not yet tested:
> arch/arm64/kernel/vdso32/Makefile
> arch/nds32/kernel/vdso/Makefile
> scripts/Makefile.build
> 
> Ujjwal Kumar (2):
>   kconfig: use interpreters to invoke scripts
>   kbuild: use interpreters to invoke scripts
> 
>  Makefile                          |  4 ++--
>  arch/arm64/kernel/vdso/Makefile   |  2 +-
>  arch/arm64/kernel/vdso32/Makefile |  2 +-
>  arch/ia64/Makefile                |  4 ++--
>  arch/nds32/kernel/vdso/Makefile   |  2 +-
>  init/Kconfig                      | 16 ++++++++--------
>  scripts/Makefile.build            |  2 +-
>  scripts/Makefile.package          |  4 ++--
>  8 files changed, 18 insertions(+), 18 deletions(-)
> 
> -- 
> 2.26.2
> 
>
Lukas Bulwahn Oct. 12, 2020, 1:40 p.m. UTC | #2
On Sat, 3 Oct 2020, Ujjwal Kumar wrote:

> This patch series aims at removing the dependency on execute 
> bit of the scripts in the kbuild system.
> 
> If not working with fresh clone of linux-next, clean the srctree:
> make distclean
> make tools/clean
> 
> To test the dependency on execute bits, I tried building the 
> kernel after removing x-bits for all files in the repository.
> Removing execute bits:
> for i in $(find -executable -type f); do chmod -x $i; done
> 
> Any attempts to configure (or build) the kernel fail because of 
> 'Permission denied' on scripts with the following error:
> $ make allmodconfig
> sh: ./scripts/gcc-version.sh: Permission denied
> init/Kconfig:34: syntax error
> init/Kconfig:33: invalid statement
> init/Kconfig:34: invalid statement
> sh: ./scripts/ld-version.sh: Permission denied
> init/Kconfig:39: syntax error
> init/Kconfig:38: invalid statement
> sh: ./scripts/clang-version.sh: Permission denied
> init/Kconfig:49: syntax error
> init/Kconfig:48: invalid statement
> make[1]: *** [scripts/kconfig/Makefile:71: allmodconfig] Error 1
> make: *** [Makefile:606: allmodconfig] Error 2
> 
> Changes:
> 1. Adds specific interpreters (in Kconfig) to invoke 
> scripts.
> 
> After this patch I could successfully do a kernel build 
> without any errors.
> 
> 2. Again, adds specific interpreters to other parts of 
> kbuild system.
> 
> I could successfully perform the following make targets after 
> applying the PATCH 2/2:
> make headerdep
> make kselftest-merge
> make rpm-pkg
> make perf-tar-src-pkg
> make ARCH=ia64 defconfig
> ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make prepare
> 
> Following changes in PATCH 2/2 are not yet tested:
> arch/arm64/kernel/vdso32/Makefile
> arch/nds32/kernel/vdso/Makefile
> scripts/Makefile.build
> 
> Ujjwal Kumar (2):
>   kconfig: use interpreters to invoke scripts
>   kbuild: use interpreters to invoke scripts
>

Ujjwal, I suggest that you continue to wait if you get any feedback from 
Masahiro-san within the next two weeks (although the merge window) and if
not, try to rebase to the the next rc1 and resend this patchset with 
Nathan's feedback tags added.

The merge window is busy time for maintainers; in the meantime, you might
look into if the build target 'make tools/clean' works on the current 
release and if there are fixes getting merged that fix that.

Lukas

>  Makefile                          |  4 ++--
>  arch/arm64/kernel/vdso/Makefile   |  2 +-
>  arch/arm64/kernel/vdso32/Makefile |  2 +-
>  arch/ia64/Makefile                |  4 ++--
>  arch/nds32/kernel/vdso/Makefile   |  2 +-
>  init/Kconfig                      | 16 ++++++++--------
>  scripts/Makefile.build            |  2 +-
>  scripts/Makefile.package          |  4 ++--
>  8 files changed, 18 insertions(+), 18 deletions(-)
> 
> -- 
> 2.26.2
> 
>
Masahiro Yamada Oct. 12, 2020, 4:18 p.m. UTC | #3
On Sun, Oct 4, 2020 at 12:19 AM Ujjwal Kumar <ujjwalkumar0501@gmail.com> wrote:
>
> This patch series aims at removing the dependency on execute
> bit of the scripts in the kbuild system.
>
> If not working with fresh clone of linux-next, clean the srctree:
> make distclean
> make tools/clean
>
> To test the dependency on execute bits, I tried building the
> kernel after removing x-bits for all files in the repository.
> Removing execute bits:
> for i in $(find -executable -type f); do chmod -x $i; done
>
> Any attempts to configure (or build) the kernel fail because of
> 'Permission denied' on scripts with the following error:
> $ make allmodconfig
> sh: ./scripts/gcc-version.sh: Permission denied
> init/Kconfig:34: syntax error
> init/Kconfig:33: invalid statement
> init/Kconfig:34: invalid statement
> sh: ./scripts/ld-version.sh: Permission denied
> init/Kconfig:39: syntax error
> init/Kconfig:38: invalid statement
> sh: ./scripts/clang-version.sh: Permission denied
> init/Kconfig:49: syntax error
> init/Kconfig:48: invalid statement
> make[1]: *** [scripts/kconfig/Makefile:71: allmodconfig] Error 1
> make: *** [Makefile:606: allmodconfig] Error 2
>
> Changes:
> 1. Adds specific interpreters (in Kconfig) to invoke
> scripts.
>
> After this patch I could successfully do a kernel build
> without any errors.
>
> 2. Again, adds specific interpreters to other parts of
> kbuild system.
>
> I could successfully perform the following make targets after
> applying the PATCH 2/2:
> make headerdep
> make kselftest-merge
> make rpm-pkg
> make perf-tar-src-pkg
> make ARCH=ia64 defconfig
> ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make prepare
>
> Following changes in PATCH 2/2 are not yet tested:
> arch/arm64/kernel/vdso32/Makefile
> arch/nds32/kernel/vdso/Makefile
> scripts/Makefile.build
>
> Ujjwal Kumar (2):
>   kconfig: use interpreters to invoke scripts
>   kbuild: use interpreters to invoke scripts
>
>  Makefile                          |  4 ++--
>  arch/arm64/kernel/vdso/Makefile   |  2 +-
>  arch/arm64/kernel/vdso32/Makefile |  2 +-
>  arch/ia64/Makefile                |  4 ++--
>  arch/nds32/kernel/vdso/Makefile   |  2 +-
>  init/Kconfig                      | 16 ++++++++--------
>  scripts/Makefile.build            |  2 +-
>  scripts/Makefile.package          |  4 ++--
>  8 files changed, 18 insertions(+), 18 deletions(-)
>
> --
> 2.26.2
>


Andrew Morton suggested and applied the doc patch
(commit e9aae7af4601688386 in linux-next),
but did not pick up this series.

It is difficult to predict which patch he would
pick up, and which he would not.


I can apply this series
together with Lukas' base patch.


I pointed out possible mistakes in 2/2.
I can locally fix them up if you agree.


BTW, Kees Cook suggested dropping the x bit
from all scripts, but I did not agree with that part.


In the doc change, Lukas mentioned
"further clean-up patches", but I hope
it does not mean dropping the x bits.


--
Best Regards

Masahiro Yamada
Ujjwal Kumar Oct. 12, 2020, 4:36 p.m. UTC | #4
On 12/10/20 9:48 pm, Masahiro Yamada wrote:
> On Sun, Oct 4, 2020 at 12:19 AM Ujjwal Kumar <ujjwalkumar0501@gmail.com> wrote:
>>
>> This patch series aims at removing the dependency on execute
>> bit of the scripts in the kbuild system.
>>
>> If not working with fresh clone of linux-next, clean the srctree:
>> make distclean
>> make tools/clean
>>
>> To test the dependency on execute bits, I tried building the
>> kernel after removing x-bits for all files in the repository.
>> Removing execute bits:
>> for i in $(find -executable -type f); do chmod -x $i; done
>>
>> Any attempts to configure (or build) the kernel fail because of
>> 'Permission denied' on scripts with the following error:
>> $ make allmodconfig
>> sh: ./scripts/gcc-version.sh: Permission denied
>> init/Kconfig:34: syntax error
>> init/Kconfig:33: invalid statement
>> init/Kconfig:34: invalid statement
>> sh: ./scripts/ld-version.sh: Permission denied
>> init/Kconfig:39: syntax error
>> init/Kconfig:38: invalid statement
>> sh: ./scripts/clang-version.sh: Permission denied
>> init/Kconfig:49: syntax error
>> init/Kconfig:48: invalid statement
>> make[1]: *** [scripts/kconfig/Makefile:71: allmodconfig] Error 1
>> make: *** [Makefile:606: allmodconfig] Error 2
>>
>> Changes:
>> 1. Adds specific interpreters (in Kconfig) to invoke
>> scripts.
>>
>> After this patch I could successfully do a kernel build
>> without any errors.
>>
>> 2. Again, adds specific interpreters to other parts of
>> kbuild system.
>>
>> I could successfully perform the following make targets after
>> applying the PATCH 2/2:
>> make headerdep
>> make kselftest-merge
>> make rpm-pkg
>> make perf-tar-src-pkg
>> make ARCH=ia64 defconfig
>> ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make prepare
>>
>> Following changes in PATCH 2/2 are not yet tested:
>> arch/arm64/kernel/vdso32/Makefile
>> arch/nds32/kernel/vdso/Makefile
>> scripts/Makefile.build
>>
>> Ujjwal Kumar (2):
>>   kconfig: use interpreters to invoke scripts
>>   kbuild: use interpreters to invoke scripts
>>
>>  Makefile                          |  4 ++--
>>  arch/arm64/kernel/vdso/Makefile   |  2 +-
>>  arch/arm64/kernel/vdso32/Makefile |  2 +-
>>  arch/ia64/Makefile                |  4 ++--
>>  arch/nds32/kernel/vdso/Makefile   |  2 +-
>>  init/Kconfig                      | 16 ++++++++--------
>>  scripts/Makefile.build            |  2 +-
>>  scripts/Makefile.package          |  4 ++--
>>  8 files changed, 18 insertions(+), 18 deletions(-)
>>
>> --
>> 2.26.2
>>
> 
> 
> Andrew Morton suggested and applied the doc patch
> (commit e9aae7af4601688386 in linux-next),
> but did not pick up this series.
> 
> It is difficult to predict which patch he would
> pick up, and which he would not.
> 
> 
> I can apply this series
> together with Lukas' base patch.
> 
> 
> I pointed out possible mistakes in 2/2.
> I can locally fix them up if you agree.

I agree with the changes you pointed out. I was in the process
of sending a V2 patch series (almost done). But if you prefer 
on locally fixing them, that is completely fine.

> 
> 
> BTW, Kees Cook suggested dropping the x bit
> from all scripts, but I did not agree with that part.

IIRC, in the discussion Kees Cook suggestion was not to drop
x bit but rather he meant to use that as a trick to catch
any existing dependency on x bit.

> 
> 
> In the doc change, Lukas mentioned
> "further clean-up patches", but I hope
> it does not mean dropping the x bits.

IMO, he did not mean to drop the x bits.
But rather I have many more small changes similar to these.
He must be referring to these two patches and any future
patches around this issue.

> 
> 
> --
> Best Regards
> 
> Masahiro Yamada
> 

Thanks
Ujjwal Kumar