| Message ID | 1614608540-1025-1-git-send-email-jshriram@codeaurora.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | scripts: Fix incremental build header re-generation | expand |
On Mon, Mar 1, 2021 at 11:23 PM Jeevan Shriram <jshriram@codeaurora.org> wrote: > > compile.h and autoconf.h are ignored when checking headers sha as they > are always re-generated for every kernel compilation. However, > these two headers are packaged into kheaders tar. During incremental > compilation of kernel, kheaders tar file is always generated and re-packaged > irrespective of the changes in headers. I do not see this problem. Could you describe the steps to reproduce it, please? > Change-Id: I7a64faebb81df44c32230b0fea1d6df09d7ce66f > Signed-off-by: Jeevan Shriram <jshriram@codeaurora.org> > --- > kernel/gen_kheaders.sh | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh > index c1510f0..5499f72 100755 > --- a/kernel/gen_kheaders.sh > +++ b/kernel/gen_kheaders.sh > @@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" > if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi > if [ -f kernel/kheaders.md5 ] && > [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] && > - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] && > - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then > + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then > exit > fi > > -- > The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, > a Linux Foundation Collaborative Project >
On 3/1/2021 7:36 PM, Masahiro Yamada wrote: > On Mon, Mar 1, 2021 at 11:23 PM Jeevan Shriram <jshriram@codeaurora.org> wrote: >> compile.h and autoconf.h are ignored when checking headers sha as they >> are always re-generated for every kernel compilation. However, >> these two headers are packaged into kheaders tar. During incremental >> compilation of kernel, kheaders tar file is always generated and re-packaged >> irrespective of the changes in headers. > > I do not see this problem. > Could you describe the steps to reproduce it, please? > Without making any changes in the kernel or it's headers, re-compile the kernel. i.e.,incremental kernel build without any changes. I have added following log in gen_kheaders.sh script for confirming the hash differences. diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh index b7425a0..ee542a0 100755 --- a/kernel/gen_kheaders.sh +++ b/kernel/gen_kheaders.sh @@ -40,6 +40,10 @@ obj_files_md5="$(find $dir_list -name "*.h" | # Any changes to this script will also cause a rebuild of the archive. this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi + +echo "Old Tar file $tarfile_md5" +echo "New Tar file hash $this_file_md5" + if [ -f kernel/kheaders.md5 ] && [ "$(cat kernel/kheaders.md5|head -1)" == "$src_files_md5" ] && [ "$(cat kernel/kheaders.md5|head -2|tail -1)" == "$obj_files_md5" ] && log output : 89306 19:29:02.109961 CHK kernel/kheaders_data.tar.xz 89307 19:29:02.109971 Old Tar file 2aa6990e4183c31a862951f4bcac037e 89308 19:29:02.109982 New Tar file hash ecf84e700c7cacfe8b35a0905859582d >> Change-Id: I7a64faebb81df44c32230b0fea1d6df09d7ce66f >> Signed-off-by: Jeevan Shriram <jshriram@codeaurora.org> >> --- >> kernel/gen_kheaders.sh | 3 +-- >> 1 file changed, 1 insertion(+), 2 deletions(-) >> >> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh >> index c1510f0..5499f72 100755 >> --- a/kernel/gen_kheaders.sh >> +++ b/kernel/gen_kheaders.sh >> @@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" >> if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi >> if [ -f kernel/kheaders.md5 ] && >> [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] && >> - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] && >> - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then >> + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then >> exit >> fi >> >> -- >> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, >> a Linux Foundation Collaborative Project >> >
On Thu, Mar 18, 2021 at 11:55 AM Jeevan Shriram <jshriram@codeaurora.org> wrote: > > > On 3/1/2021 7:36 PM, Masahiro Yamada wrote: > > On Mon, Mar 1, 2021 at 11:23 PM Jeevan Shriram <jshriram@codeaurora.org> wrote: > >> compile.h and autoconf.h are ignored when checking headers sha as they > >> are always re-generated for every kernel compilation. However, > >> these two headers are packaged into kheaders tar. During incremental > >> compilation of kernel, kheaders tar file is always generated and re-packaged > >> irrespective of the changes in headers. > > > > I do not see this problem. > > Could you describe the steps to reproduce it, please? > > > Without making any changes in the kernel or it's headers, re-compile the kernel. i.e.,incremental kernel build without any changes. > I have added following log in gen_kheaders.sh script for confirming the hash differences. Of course they are different because you are comparing the hashes of two different files. > > diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh > index b7425a0..ee542a0 100755 > --- a/kernel/gen_kheaders.sh > +++ b/kernel/gen_kheaders.sh > @@ -40,6 +40,10 @@ obj_files_md5="$(find $dir_list -name "*.h" | > # Any changes to this script will also cause a rebuild of the archive. > this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" > if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi > + > +echo "Old Tar file $tarfile_md5" This is the hash of kernel/kheaders_data.tar.xz > +echo "New Tar file hash $this_file_md5" This is the hash of kernel/gen_kheaders.sh > + > if [ -f kernel/kheaders.md5 ] && > [ "$(cat kernel/kheaders.md5|head -1)" == "$src_files_md5" ] && > [ "$(cat kernel/kheaders.md5|head -2|tail -1)" == "$obj_files_md5" ] && > > log output : > 89306 19:29:02.109961 CHK kernel/kheaders_data.tar.xz > 89307 19:29:02.109971 Old Tar file 2aa6990e4183c31a862951f4bcac037e > 89308 19:29:02.109982 New Tar file hash ecf84e700c7cacfe8b35a0905859582d I do not understand what your claim is. CHK kernel/kheaders_data.tar.xz is displayed when it is checking the hash in order to determine if the tarball should be updated. GEN kernel/kheaders_data.tar.xz is displayed when the tarball is really updated. [Incremental build with no change] masahiro@oscar:~/ref/linux$ make -j24 DESCEND objtool CALL scripts/atomic/check-atomics.sh CALL scripts/checksyscalls.sh CHK include/generated/compile.h CHK kernel/kheaders_data.tar.xz Kernel: arch/x86/boot/bzImage is ready (#2) [Touch one header and rebuild] masahiro@oscar:~/ref/linux$ touch include/uapi/drm/exynos_drm.h masahiro@oscar:~/ref/linux$ make -j24 DESCEND objtool CALL scripts/atomic/check-atomics.sh CALL scripts/checksyscalls.sh CHK include/generated/compile.h CHK kernel/kheaders_data.tar.xz GEN kernel/kheaders_data.tar.xz CC kernel/kheaders.o AR kernel/built-in.a GEN .version CHK include/generated/compile.h UPD include/generated/compile.h CC init/version.o AR init/built-in.a LD vmlinux.o MODPOST vmlinux.symvers MODINFO modules.builtin.modinfo GEN modules.builtin LD .tmp_vmlinux.kallsyms1 KSYMS .tmp_vmlinux.kallsyms1.S AS .tmp_vmlinux.kallsyms1.S LD .tmp_vmlinux.kallsyms2 KSYMS .tmp_vmlinux.kallsyms2.S AS .tmp_vmlinux.kallsyms2.S LD vmlinux SORTTAB vmlinux SYSMAP System.map MODPOST Module.symvers CC arch/x86/boot/version.o VOFFSET arch/x86/boot/compressed/../voffset.h OBJCOPY arch/x86/boot/compressed/vmlinux.bin RELOCS arch/x86/boot/compressed/vmlinux.relocs CC arch/x86/boot/compressed/kaslr.o CC arch/x86/boot/compressed/misc.o GZIP arch/x86/boot/compressed/vmlinux.bin.gz MKPIGGY arch/x86/boot/compressed/piggy.S AS arch/x86/boot/compressed/piggy.o LD arch/x86/boot/compressed/vmlinux ZOFFSET arch/x86/boot/zoffset.h OBJCOPY arch/x86/boot/vmlinux.bin AS arch/x86/boot/header.o LD arch/x86/boot/setup.elf OBJCOPY arch/x86/boot/setup.bin BUILD arch/x86/boot/bzImage Kernel: arch/x86/boot/bzImage is ready (#3) > >> Change-Id: I7a64faebb81df44c32230b0fea1d6df09d7ce66f > >> Signed-off-by: Jeevan Shriram <jshriram@codeaurora.org> > >> --- > >> kernel/gen_kheaders.sh | 3 +-- > >> 1 file changed, 1 insertion(+), 2 deletions(-) > >> > >> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh > >> index c1510f0..5499f72 100755 > >> --- a/kernel/gen_kheaders.sh > >> +++ b/kernel/gen_kheaders.sh > >> @@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" > >> if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi > >> if [ -f kernel/kheaders.md5 ] && > >> [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] && > >> - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] && > >> - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then > >> + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then > >> exit > >> fi > >> > >> -- > >> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, > >> a Linux Foundation Collaborative Project > >> > > -- Best Regards Masahiro Yamada
diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh index c1510f0..5499f72 100755 --- a/kernel/gen_kheaders.sh +++ b/kernel/gen_kheaders.sh @@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi if [ -f kernel/kheaders.md5 ] && [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] && - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] && - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then exit fi
compile.h and autoconf.h are ignored when checking headers sha as they are always re-generated for every kernel compilation. However, these two headers are packaged into kheaders tar. During incremental compilation of kernel, kheaders tar file is always generated and re-packaged irrespective of the changes in headers. Change-Id: I7a64faebb81df44c32230b0fea1d6df09d7ce66f Signed-off-by: Jeevan Shriram <jshriram@codeaurora.org> --- kernel/gen_kheaders.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)