diff mbox series

[RFC,bpf-next,09/12] kbuild: Header guards for types from include/uapi/*.h in kernel BTF

Message ID 20221025222802.2295103-10-eddyz87@gmail.com (mailing list archive)
State RFC
Delegated to: BPF
Headers show
Series Use uapi kernel headers with vmlinux.h | expand

Checks

Context Check Description
bpf/vmtest-bpf-next-PR fail PR summary
bpf/vmtest-bpf-next-VM_Test-1 pending Logs for ${{ matrix.test }} on ${{ matrix.arch }} with ${{ matrix.toolchain }}
bpf/vmtest-bpf-next-VM_Test-2 fail Logs for build for s390x with gcc
bpf/vmtest-bpf-next-VM_Test-3 fail Logs for build for x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-4 fail Logs for build for x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-5 success Logs for llvm-toolchain
bpf/vmtest-bpf-next-VM_Test-6 success Logs for set-matrix
netdev/tree_selection success Clearly marked for bpf-next, async
netdev/fixes_present success Fixes tag not required for -next series
netdev/subject_prefix success Link
netdev/cover_letter success Series has a cover letter
netdev/patch_count success Link
netdev/header_inline success No static functions without inline keyword in header files
netdev/build_32bit success Errors and warnings before: 2 this patch: 2
netdev/cc_maintainers fail 4 maintainers not CCed: ndesaulniers@google.com masahiroy@kernel.org michal.lkml@markovi.net linux-kbuild@vger.kernel.org
netdev/build_clang success Errors and warnings before: 0 this patch: 0
netdev/module_param success Was 0 now: 0
netdev/verify_signedoff success Signed-off-by tag matches author and committer
netdev/check_selftest success No net selftest shell script
netdev/verify_fixes success No Fixes tag
netdev/build_allmodconfig_warn success Errors and warnings before: 2 this patch: 2
netdev/checkpatch warning WARNING: line length of 82 exceeds 80 columns
netdev/kdoc success Errors and warnings before: 0 this patch: 0
netdev/source_inline success Was 0 now: 0

Commit Message

Eduard Zingerman Oct. 25, 2022, 10:27 p.m. UTC
Use pahole --header_guards_db flag to enable encoding of header guard
information in kernel BTF. The actual correspondence between header
file and guard string is computed by the scripts/infer_header_guards.pl.

The encoded header guard information could be used to restore the
original guards in the vmlinux.h, e.g.:

    include/uapi/linux/tcp.h:

      #ifndef _UAPI_LINUX_TCP_H
      #define _UAPI_LINUX_TCP_H
      ...
      union tcp_word_hdr {
    	struct tcphdr hdr;
    	__be32        words[5];
      };
      ...
      #endif /* _UAPI_LINUX_TCP_H */

    vmlinux.h:

      ...
      #ifndef _UAPI_LINUX_TCP_H

      union tcp_word_hdr {
    	struct tcphdr hdr;
    	__be32 words[5];
      };

      #endif /* _UAPI_LINUX_TCP_H */
      ...

Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
---
 scripts/link-vmlinux.sh | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

Comments

Yonghong Song Oct. 27, 2022, 6:43 p.m. UTC | #1
On 10/25/22 3:27 PM, Eduard Zingerman wrote:
> Use pahole --header_guards_db flag to enable encoding of header guard
> information in kernel BTF. The actual correspondence between header
> file and guard string is computed by the scripts/infer_header_guards.pl.
> 
> The encoded header guard information could be used to restore the
> original guards in the vmlinux.h, e.g.:
> 
>      include/uapi/linux/tcp.h:
> 
>        #ifndef _UAPI_LINUX_TCP_H
>        #define _UAPI_LINUX_TCP_H
>        ...
>        union tcp_word_hdr {
>      	struct tcphdr hdr;
>      	__be32        words[5];
>        };
>        ...
>        #endif /* _UAPI_LINUX_TCP_H */
> 
>      vmlinux.h:
> 
>        ...
>        #ifndef _UAPI_LINUX_TCP_H
> 
>        union tcp_word_hdr {
>      	struct tcphdr hdr;
>      	__be32 words[5];
>        };
> 
>        #endif /* _UAPI_LINUX_TCP_H */
>        ...
> 
> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
> ---
>   scripts/link-vmlinux.sh | 13 ++++++++++++-
>   1 file changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index 918470d768e9..f57f621eda1f 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -110,6 +110,7 @@ vmlinux_link()
>   gen_btf()
>   {
>   	local pahole_ver
> +	local extra_flags
>   
>   	if ! [ -x "$(command -v ${PAHOLE})" ]; then
>   		echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
> @@ -122,10 +123,20 @@ gen_btf()
>   		return 1
>   	fi
>   
> +	if [ "${pahole_ver}" -ge "124" ]; then
> +		scripts/infer_header_guards.pl \

We should have full path like
	${srctree}/scripts/infer_header_guards.pl
so it can work if build directory is different from source directory.

> +			include/uapi \
> +			include/generated/uapi \
> +			arch/${SRCARCH}/include/uapi \
> +			arch/${SRCARCH}/include/generated/uapi \
> +			> .btf.uapi_header_guards || return 1;
> +		extra_flags="--header_guards_db .btf.uapi_header_guards"
> +	fi
> +
>   	vmlinux_link ${1}
>   
>   	info "BTF" ${2}
> -	LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
> +	LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${extra_flags} ${1}
>   
>   	# Create ${2} which contains just .BTF section but no symbols. Add
>   	# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
Yonghong Song Oct. 27, 2022, 6:55 p.m. UTC | #2
On 10/27/22 11:43 AM, Yonghong Song wrote:
> 
> 
> On 10/25/22 3:27 PM, Eduard Zingerman wrote:
>> Use pahole --header_guards_db flag to enable encoding of header guard
>> information in kernel BTF. The actual correspondence between header
>> file and guard string is computed by the scripts/infer_header_guards.pl.
>>
>> The encoded header guard information could be used to restore the
>> original guards in the vmlinux.h, e.g.:
>>
>>      include/uapi/linux/tcp.h:
>>
>>        #ifndef _UAPI_LINUX_TCP_H
>>        #define _UAPI_LINUX_TCP_H
>>        ...
>>        union tcp_word_hdr {
>>          struct tcphdr hdr;
>>          __be32        words[5];
>>        };
>>        ...
>>        #endif /* _UAPI_LINUX_TCP_H */
>>
>>      vmlinux.h:
>>
>>        ...
>>        #ifndef _UAPI_LINUX_TCP_H
>>
>>        union tcp_word_hdr {
>>          struct tcphdr hdr;
>>          __be32 words[5];
>>        };
>>
>>        #endif /* _UAPI_LINUX_TCP_H */
>>        ...
>>
>> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
>> ---
>>   scripts/link-vmlinux.sh | 13 ++++++++++++-
>>   1 file changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
>> index 918470d768e9..f57f621eda1f 100755
>> --- a/scripts/link-vmlinux.sh
>> +++ b/scripts/link-vmlinux.sh
>> @@ -110,6 +110,7 @@ vmlinux_link()
>>   gen_btf()
>>   {
>>       local pahole_ver
>> +    local extra_flags
>>       if ! [ -x "$(command -v ${PAHOLE})" ]; then
>>           echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
>> @@ -122,10 +123,20 @@ gen_btf()
>>           return 1
>>       fi
>> +    if [ "${pahole_ver}" -ge "124" ]; then
>> +        scripts/infer_header_guards.pl \
> 
> We should have full path like
>      ${srctree}/scripts/infer_header_guards.pl
> so it can work if build directory is different from source directory.

handling arguments for infer_header_guards.pl should also take
care of full file path.

+ /home/yhs/work/bpf-next/scripts/infer_header_guards.pl include/uapi 
include/generated/uapi arch/x86/include/uapi arch/x86/include/generated/uapi
+ return 1

> 
>> +            include/uapi \
>> +            include/generated/uapi \
>> +            arch/${SRCARCH}/include/uapi \
>> +            arch/${SRCARCH}/include/generated/uapi \
>> +            > .btf.uapi_header_guards || return 1;
>> +        extra_flags="--header_guards_db .btf.uapi_header_guards"
>> +    fi
>> +
>>       vmlinux_link ${1}
>>       info "BTF" ${2}
>> -    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
>> +    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} 
>> ${extra_flags} ${1}
>>       # Create ${2} which contains just .BTF section but no symbols. Add
>>       # SHF_ALLOC because .BTF will be part of the vmlinux image. 
>> --strip-all
Yonghong Song Oct. 27, 2022, 10:44 p.m. UTC | #3
On 10/27/22 11:55 AM, Yonghong Song wrote:
> 
> 
> On 10/27/22 11:43 AM, Yonghong Song wrote:
>>
>>
>> On 10/25/22 3:27 PM, Eduard Zingerman wrote:
>>> Use pahole --header_guards_db flag to enable encoding of header guard
>>> information in kernel BTF. The actual correspondence between header
>>> file and guard string is computed by the scripts/infer_header_guards.pl.
>>>
>>> The encoded header guard information could be used to restore the
>>> original guards in the vmlinux.h, e.g.:
>>>
>>>      include/uapi/linux/tcp.h:
>>>
>>>        #ifndef _UAPI_LINUX_TCP_H
>>>        #define _UAPI_LINUX_TCP_H
>>>        ...
>>>        union tcp_word_hdr {
>>>          struct tcphdr hdr;
>>>          __be32        words[5];
>>>        };
>>>        ...
>>>        #endif /* _UAPI_LINUX_TCP_H */
>>>
>>>      vmlinux.h:
>>>
>>>        ...
>>>        #ifndef _UAPI_LINUX_TCP_H
>>>
>>>        union tcp_word_hdr {
>>>          struct tcphdr hdr;
>>>          __be32 words[5];
>>>        };
>>>
>>>        #endif /* _UAPI_LINUX_TCP_H */
>>>        ...
>>>
>>> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
>>> ---
>>>   scripts/link-vmlinux.sh | 13 ++++++++++++-
>>>   1 file changed, 12 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
>>> index 918470d768e9..f57f621eda1f 100755
>>> --- a/scripts/link-vmlinux.sh
>>> +++ b/scripts/link-vmlinux.sh
>>> @@ -110,6 +110,7 @@ vmlinux_link()
>>>   gen_btf()
>>>   {
>>>       local pahole_ver
>>> +    local extra_flags
>>>       if ! [ -x "$(command -v ${PAHOLE})" ]; then
>>>           echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
>>> @@ -122,10 +123,20 @@ gen_btf()
>>>           return 1
>>>       fi
>>> +    if [ "${pahole_ver}" -ge "124" ]; then
>>> +        scripts/infer_header_guards.pl \
>>
>> We should have full path like
>>      ${srctree}/scripts/infer_header_guards.pl
>> so it can work if build directory is different from source directory.
> 
> handling arguments for infer_header_guards.pl should also take
> care of full file path.
> 
> + /home/yhs/work/bpf-next/scripts/infer_header_guards.pl include/uapi 
> include/generated/uapi arch/x86/include/uapi 
> arch/x86/include/generated/uapi
> + return 1

Also, please pay attention to bpf selftest result. I see quite a
few selftest failures with this patch set.

>>
>>> +            include/uapi \
>>> +            include/generated/uapi \
>>> +            arch/${SRCARCH}/include/uapi \
>>> +            arch/${SRCARCH}/include/generated/uapi \
>>> +            > .btf.uapi_header_guards || return 1;
>>> +        extra_flags="--header_guards_db .btf.uapi_header_guards"
>>> +    fi
>>> +
>>>       vmlinux_link ${1}
>>>       info "BTF" ${2}
>>> -    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
>>> +    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} 
>>> ${extra_flags} ${1}
>>>       # Create ${2} which contains just .BTF section but no symbols. Add
>>>       # SHF_ALLOC because .BTF will be part of the vmlinux image. 
>>> --strip-all
Eduard Zingerman Oct. 28, 2022, midnight UTC | #4
On Thu, 2022-10-27 at 15:44 -0700, Yonghong Song wrote:
> 
> On 10/27/22 11:55 AM, Yonghong Song wrote:
> > 
> > 
> > On 10/27/22 11:43 AM, Yonghong Song wrote:
> > > 
> > > 
> > > On 10/25/22 3:27 PM, Eduard Zingerman wrote:
> > > > Use pahole --header_guards_db flag to enable encoding of header guard
> > > > information in kernel BTF. The actual correspondence between header
> > > > file and guard string is computed by the scripts/infer_header_guards.pl.
> > > > 
> > > > The encoded header guard information could be used to restore the
> > > > original guards in the vmlinux.h, e.g.:
> > > > 
> > > >      include/uapi/linux/tcp.h:
> > > > 
> > > >        #ifndef _UAPI_LINUX_TCP_H
> > > >        #define _UAPI_LINUX_TCP_H
> > > >        ...
> > > >        union tcp_word_hdr {
> > > >          struct tcphdr hdr;
> > > >          __be32        words[5];
> > > >        };
> > > >        ...
> > > >        #endif /* _UAPI_LINUX_TCP_H */
> > > > 
> > > >      vmlinux.h:
> > > > 
> > > >        ...
> > > >        #ifndef _UAPI_LINUX_TCP_H
> > > > 
> > > >        union tcp_word_hdr {
> > > >          struct tcphdr hdr;
> > > >          __be32 words[5];
> > > >        };
> > > > 
> > > >        #endif /* _UAPI_LINUX_TCP_H */
> > > >        ...
> > > > 
> > > > Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
> > > > ---
> > > >   scripts/link-vmlinux.sh | 13 ++++++++++++-
> > > >   1 file changed, 12 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> > > > index 918470d768e9..f57f621eda1f 100755
> > > > --- a/scripts/link-vmlinux.sh
> > > > +++ b/scripts/link-vmlinux.sh
> > > > @@ -110,6 +110,7 @@ vmlinux_link()
> > > >   gen_btf()
> > > >   {
> > > >       local pahole_ver
> > > > +    local extra_flags
> > > >       if ! [ -x "$(command -v ${PAHOLE})" ]; then
> > > >           echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
> > > > @@ -122,10 +123,20 @@ gen_btf()
> > > >           return 1
> > > >       fi
> > > > +    if [ "${pahole_ver}" -ge "124" ]; then
> > > > +        scripts/infer_header_guards.pl \
> > > 
> > > We should have full path like
> > >      ${srctree}/scripts/infer_header_guards.pl
> > > so it can work if build directory is different from source directory.
> > 
> > handling arguments for infer_header_guards.pl should also take
> > care of full file path.
> > 
> > + /home/yhs/work/bpf-next/scripts/infer_header_guards.pl include/uapi 
> > include/generated/uapi arch/x86/include/uapi 
> > arch/x86/include/generated/uapi
> > + return 1
> 
> Also, please pay attention to bpf selftest result. I see quite a
> few selftest failures with this patch set.

Hi Yonghong,

Could you please copy-paste some of the error reports? I just re-run
the selftests locally and have test_maps, test_verifier, test_progs
and test_progs-no_alu32 passing.

Thanks,
Eduard

> 
> > > 
> > > > +            include/uapi \
> > > > +            include/generated/uapi \
> > > > +            arch/${SRCARCH}/include/uapi \
> > > > +            arch/${SRCARCH}/include/generated/uapi \
> > > > +            > .btf.uapi_header_guards || return 1;
> > > > +        extra_flags="--header_guards_db .btf.uapi_header_guards"
> > > > +    fi
> > > > +
> > > >       vmlinux_link ${1}
> > > >       info "BTF" ${2}
> > > > -    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
> > > > +    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} 
> > > > ${extra_flags} ${1}
> > > >       # Create ${2} which contains just .BTF section but no symbols. Add
> > > >       # SHF_ALLOC because .BTF will be part of the vmlinux image. 
> > > > --strip-all
Mykola Lysenko Oct. 28, 2022, 12:14 a.m. UTC | #5
Yonghong,

build will be failing without merged pahole changes.

> On Oct 27, 2022, at 5:00 PM, Eduard Zingerman <eddyz87@gmail.com> wrote:
> 
> On Thu, 2022-10-27 at 15:44 -0700, Yonghong Song wrote:
>> 
>> On 10/27/22 11:55 AM, Yonghong Song wrote:
>>> 
>>> 
>>> On 10/27/22 11:43 AM, Yonghong Song wrote:
>>>> 
>>>> 
>>>> On 10/25/22 3:27 PM, Eduard Zingerman wrote:
>>>>> Use pahole --header_guards_db flag to enable encoding of header guard
>>>>> information in kernel BTF. The actual correspondence between header
>>>>> file and guard string is computed by the scripts/infer_header_guards.pl.
>>>>> 
>>>>> The encoded header guard information could be used to restore the
>>>>> original guards in the vmlinux.h, e.g.:
>>>>> 
>>>>>      include/uapi/linux/tcp.h:
>>>>> 
>>>>>        #ifndef _UAPI_LINUX_TCP_H
>>>>>        #define _UAPI_LINUX_TCP_H
>>>>>        ...
>>>>>        union tcp_word_hdr {
>>>>>          struct tcphdr hdr;
>>>>>          __be32        words[5];
>>>>>        };
>>>>>        ...
>>>>>        #endif /* _UAPI_LINUX_TCP_H */
>>>>> 
>>>>>      vmlinux.h:
>>>>> 
>>>>>        ...
>>>>>        #ifndef _UAPI_LINUX_TCP_H
>>>>> 
>>>>>        union tcp_word_hdr {
>>>>>          struct tcphdr hdr;
>>>>>          __be32 words[5];
>>>>>        };
>>>>> 
>>>>>        #endif /* _UAPI_LINUX_TCP_H */
>>>>>        ...
>>>>> 
>>>>> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
>>>>> ---
>>>>>   scripts/link-vmlinux.sh | 13 ++++++++++++-
>>>>>   1 file changed, 12 insertions(+), 1 deletion(-)
>>>>> 
>>>>> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
>>>>> index 918470d768e9..f57f621eda1f 100755
>>>>> --- a/scripts/link-vmlinux.sh
>>>>> +++ b/scripts/link-vmlinux.sh
>>>>> @@ -110,6 +110,7 @@ vmlinux_link()
>>>>>   gen_btf()
>>>>>   {
>>>>>       local pahole_ver
>>>>> +    local extra_flags
>>>>>       if ! [ -x "$(command -v ${PAHOLE})" ]; then
>>>>>           echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
>>>>> @@ -122,10 +123,20 @@ gen_btf()
>>>>>           return 1
>>>>>       fi
>>>>> +    if [ "${pahole_ver}" -ge "124" ]; then
>>>>> +        scripts/infer_header_guards.pl \
>>>> 
>>>> We should have full path like
>>>>      ${srctree}/scripts/infer_header_guards.pl
>>>> so it can work if build directory is different from source directory.
>>> 
>>> handling arguments for infer_header_guards.pl should also take
>>> care of full file path.
>>> 
>>> + /home/yhs/work/bpf-next/scripts/infer_header_guards.pl include/uapi 
>>> include/generated/uapi arch/x86/include/uapi 
>>> arch/x86/include/generated/uapi
>>> + return 1
>> 
>> Also, please pay attention to bpf selftest result. I see quite a
>> few selftest failures with this patch set.
> 
> Hi Yonghong,
> 
> Could you please copy-paste some of the error reports? I just re-run
> the selftests locally and have test_maps, test_verifier, test_progs
> and test_progs-no_alu32 passing.
> 
> Thanks,
> Eduard
> 
>> 
>>>> 
>>>>> +            include/uapi \
>>>>> +            include/generated/uapi \
>>>>> +            arch/${SRCARCH}/include/uapi \
>>>>> +            arch/${SRCARCH}/include/generated/uapi \
>>>>> +            > .btf.uapi_header_guards || return 1;
>>>>> +        extra_flags="--header_guards_db .btf.uapi_header_guards"
>>>>> +    fi
>>>>> +
>>>>>       vmlinux_link ${1}
>>>>>       info "BTF" ${2}
>>>>> -    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
>>>>> +    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} 
>>>>> ${extra_flags} ${1}
>>>>>       # Create ${2} which contains just .BTF section but no symbols. Add
>>>>>       # SHF_ALLOC because .BTF will be part of the vmlinux image. 
>>>>> --strip-all
Yonghong Song Oct. 28, 2022, 1:21 a.m. UTC | #6
On 10/27/22 5:00 PM, Eduard Zingerman wrote:
> On Thu, 2022-10-27 at 15:44 -0700, Yonghong Song wrote:
>>
>> On 10/27/22 11:55 AM, Yonghong Song wrote:
>>>
>>>
>>> On 10/27/22 11:43 AM, Yonghong Song wrote:
>>>>
>>>>
>>>> On 10/25/22 3:27 PM, Eduard Zingerman wrote:
>>>>> Use pahole --header_guards_db flag to enable encoding of header guard
>>>>> information in kernel BTF. The actual correspondence between header
>>>>> file and guard string is computed by the scripts/infer_header_guards.pl.
>>>>>
>>>>> The encoded header guard information could be used to restore the
>>>>> original guards in the vmlinux.h, e.g.:
>>>>>
>>>>>       include/uapi/linux/tcp.h:
>>>>>
>>>>>         #ifndef _UAPI_LINUX_TCP_H
>>>>>         #define _UAPI_LINUX_TCP_H
>>>>>         ...
>>>>>         union tcp_word_hdr {
>>>>>           struct tcphdr hdr;
>>>>>           __be32        words[5];
>>>>>         };
>>>>>         ...
>>>>>         #endif /* _UAPI_LINUX_TCP_H */
>>>>>
>>>>>       vmlinux.h:
>>>>>
>>>>>         ...
>>>>>         #ifndef _UAPI_LINUX_TCP_H
>>>>>
>>>>>         union tcp_word_hdr {
>>>>>           struct tcphdr hdr;
>>>>>           __be32 words[5];
>>>>>         };
>>>>>
>>>>>         #endif /* _UAPI_LINUX_TCP_H */
>>>>>         ...
>>>>>
>>>>> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
>>>>> ---
>>>>>    scripts/link-vmlinux.sh | 13 ++++++++++++-
>>>>>    1 file changed, 12 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
>>>>> index 918470d768e9..f57f621eda1f 100755
>>>>> --- a/scripts/link-vmlinux.sh
>>>>> +++ b/scripts/link-vmlinux.sh
>>>>> @@ -110,6 +110,7 @@ vmlinux_link()
>>>>>    gen_btf()
>>>>>    {
>>>>>        local pahole_ver
>>>>> +    local extra_flags
>>>>>        if ! [ -x "$(command -v ${PAHOLE})" ]; then
>>>>>            echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
>>>>> @@ -122,10 +123,20 @@ gen_btf()
>>>>>            return 1
>>>>>        fi
>>>>> +    if [ "${pahole_ver}" -ge "124" ]; then
>>>>> +        scripts/infer_header_guards.pl \
>>>>
>>>> We should have full path like
>>>>       ${srctree}/scripts/infer_header_guards.pl
>>>> so it can work if build directory is different from source directory.
>>>
>>> handling arguments for infer_header_guards.pl should also take
>>> care of full file path.
>>>
>>> + /home/yhs/work/bpf-next/scripts/infer_header_guards.pl include/uapi
>>> include/generated/uapi arch/x86/include/uapi
>>> arch/x86/include/generated/uapi
>>> + return 1
>>
>> Also, please pay attention to bpf selftest result. I see quite a
>> few selftest failures with this patch set.
> 
> Hi Yonghong,
> 
> Could you please copy-paste some of the error reports? I just re-run
> the selftests locally and have test_maps, test_verifier, test_progs
> and test_progs-no_alu32 passing.

Sorry about the noise. It is my fault. My default build is out of
source tree with KBUILD_OUTPUT=<path>. Since the current patch set
won't work with it, so I build a in-tree one for vmlinux but forgot
to adjust selftest build which still has KBUILD_OUTPUT=<path> and
it caused some selftest failures. Consistently doing in-tree build
for vmlinux and selftest results in the same Success/Failure rate
with and without this patch set.

> 
> Thanks,
> Eduard
> 
>>
>>>>
>>>>> +            include/uapi \
>>>>> +            include/generated/uapi \
>>>>> +            arch/${SRCARCH}/include/uapi \
>>>>> +            arch/${SRCARCH}/include/generated/uapi \
>>>>> +            > .btf.uapi_header_guards || return 1;
>>>>> +        extra_flags="--header_guards_db .btf.uapi_header_guards"
>>>>> +    fi
>>>>> +
>>>>>        vmlinux_link ${1}
>>>>>        info "BTF" ${2}
>>>>> -    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
>>>>> +    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS}
>>>>> ${extra_flags} ${1}
>>>>>        # Create ${2} which contains just .BTF section but no symbols. Add
>>>>>        # SHF_ALLOC because .BTF will be part of the vmlinux image.
>>>>> --strip-all
>
Yonghong Song Oct. 28, 2022, 1:23 a.m. UTC | #7
On 10/27/22 5:14 PM, Mykola Lysenko wrote:
> Yonghong,
> 
> build will be failing without merged pahole changes.

Yes, I do have pahole changes. My failure is due to
building in-tree vmlinux (KBUILD_OUTPUT= ) since
current patch set doesn't support out-of-tree build,
and out-of-tree selftest (KBUILD_OUTPUT=<path>).
Using the same in-tree build fixed the problem.

> 
>> On Oct 27, 2022, at 5:00 PM, Eduard Zingerman <eddyz87@gmail.com> wrote:
>>
>> On Thu, 2022-10-27 at 15:44 -0700, Yonghong Song wrote:
>>>
>>> On 10/27/22 11:55 AM, Yonghong Song wrote:
>>>>
>>>>
>>>> On 10/27/22 11:43 AM, Yonghong Song wrote:
>>>>>
>>>>>
>>>>> On 10/25/22 3:27 PM, Eduard Zingerman wrote:
>>>>>> Use pahole --header_guards_db flag to enable encoding of header guard
>>>>>> information in kernel BTF. The actual correspondence between header
>>>>>> file and guard string is computed by the scripts/infer_header_guards.pl.
>>>>>>
>>>>>> The encoded header guard information could be used to restore the
>>>>>> original guards in the vmlinux.h, e.g.:
>>>>>>
>>>>>>       include/uapi/linux/tcp.h:
>>>>>>
>>>>>>         #ifndef _UAPI_LINUX_TCP_H
>>>>>>         #define _UAPI_LINUX_TCP_H
>>>>>>         ...
>>>>>>         union tcp_word_hdr {
>>>>>>           struct tcphdr hdr;
>>>>>>           __be32        words[5];
>>>>>>         };
>>>>>>         ...
>>>>>>         #endif /* _UAPI_LINUX_TCP_H */
>>>>>>
>>>>>>       vmlinux.h:
>>>>>>
>>>>>>         ...
>>>>>>         #ifndef _UAPI_LINUX_TCP_H
>>>>>>
>>>>>>         union tcp_word_hdr {
>>>>>>           struct tcphdr hdr;
>>>>>>           __be32 words[5];
>>>>>>         };
>>>>>>
>>>>>>         #endif /* _UAPI_LINUX_TCP_H */
>>>>>>         ...
>>>>>>
>>>>>> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
>>>>>> ---
>>>>>>    scripts/link-vmlinux.sh | 13 ++++++++++++-
>>>>>>    1 file changed, 12 insertions(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
>>>>>> index 918470d768e9..f57f621eda1f 100755
>>>>>> --- a/scripts/link-vmlinux.sh
>>>>>> +++ b/scripts/link-vmlinux.sh
>>>>>> @@ -110,6 +110,7 @@ vmlinux_link()
>>>>>>    gen_btf()
>>>>>>    {
>>>>>>        local pahole_ver
>>>>>> +    local extra_flags
>>>>>>        if ! [ -x "$(command -v ${PAHOLE})" ]; then
>>>>>>            echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
>>>>>> @@ -122,10 +123,20 @@ gen_btf()
>>>>>>            return 1
>>>>>>        fi
>>>>>> +    if [ "${pahole_ver}" -ge "124" ]; then
>>>>>> +        scripts/infer_header_guards.pl \
>>>>>
>>>>> We should have full path like
>>>>>       ${srctree}/scripts/infer_header_guards.pl
>>>>> so it can work if build directory is different from source directory.
>>>>
>>>> handling arguments for infer_header_guards.pl should also take
>>>> care of full file path.
>>>>
>>>> + /home/yhs/work/bpf-next/scripts/infer_header_guards.pl include/uapi
>>>> include/generated/uapi arch/x86/include/uapi
>>>> arch/x86/include/generated/uapi
>>>> + return 1
>>>
>>> Also, please pay attention to bpf selftest result. I see quite a
>>> few selftest failures with this patch set.
>>
>> Hi Yonghong,
>>
>> Could you please copy-paste some of the error reports? I just re-run
>> the selftests locally and have test_maps, test_verifier, test_progs
>> and test_progs-no_alu32 passing.
>>
>> Thanks,
>> Eduard
>>
>>>
>>>>>
>>>>>> +            include/uapi \
>>>>>> +            include/generated/uapi \
>>>>>> +            arch/${SRCARCH}/include/uapi \
>>>>>> +            arch/${SRCARCH}/include/generated/uapi \
>>>>>> +            > .btf.uapi_header_guards || return 1;
>>>>>> +        extra_flags="--header_guards_db .btf.uapi_header_guards"
>>>>>> +    fi
>>>>>> +
>>>>>>        vmlinux_link ${1}
>>>>>>        info "BTF" ${2}
>>>>>> -    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
>>>>>> +    LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS}
>>>>>> ${extra_flags} ${1}
>>>>>>        # Create ${2} which contains just .BTF section but no symbols. Add
>>>>>>        # SHF_ALLOC because .BTF will be part of the vmlinux image.
>>>>>> --strip-all
>
diff mbox series

Patch

diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 918470d768e9..f57f621eda1f 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -110,6 +110,7 @@  vmlinux_link()
 gen_btf()
 {
 	local pahole_ver
+	local extra_flags
 
 	if ! [ -x "$(command -v ${PAHOLE})" ]; then
 		echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
@@ -122,10 +123,20 @@  gen_btf()
 		return 1
 	fi
 
+	if [ "${pahole_ver}" -ge "124" ]; then
+		scripts/infer_header_guards.pl \
+			include/uapi \
+			include/generated/uapi \
+			arch/${SRCARCH}/include/uapi \
+			arch/${SRCARCH}/include/generated/uapi \
+			> .btf.uapi_header_guards || return 1;
+		extra_flags="--header_guards_db .btf.uapi_header_guards"
+	fi
+
 	vmlinux_link ${1}
 
 	info "BTF" ${2}
-	LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
+	LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${extra_flags} ${1}
 
 	# Create ${2} which contains just .BTF section but no symbols. Add
 	# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all