Message ID | 20210907230050.1957493-1-yhs@fb.com (mailing list archive) |
---|---|
Headers | show |
Series | bpf: add support for new btf kind BTF_KIND_TAG | expand |
Hi Yonghong. > LLVM14 added support for a new C attribute ([1]) > __attribute__((btf_tag("arbitrary_str"))) > This attribute will be emitted to dwarf ([2]) and pahole > will convert it to BTF. Or for bpf target, this > attribute will be emitted to BTF directly ([3]). > The attribute is intended to provide additional > information for > - struct/union type or struct/union member > - static/global variables > - static/global function or function parameter. > > This new attribute can be used to add attributes > to kernel codes, e.g., pre- or post- conditions, > allow/deny info, or any other info in which only > the kernel is interested. Such attributes will > be processed by clang frontend and emitted to > dwarf, converting to BTF by pahole. Ultimiately > the verifier can use these information for > verification purpose. > > The new attribute can also be used for bpf > programs, e.g., tagging with __user attributes > for function parameters, specifying global > function preconditions, etc. Such information > may help verifier to detect user program > bugs. > > After this series, pahole dwarf->btf converter > will be enhanced to support new llvm tag > for btf_tag attribute. With pahole support, > we will then try to add a few real use case, > e.g., __user/__rcu tagging, allow/deny list, > some kernel function precondition, etc, > in the kernel. We are looking into implementing this in the GCC BPF port. Supporting the new C attribute in BPF programs as a target-specific attribute, and the new BTF kind, is straightforward enough. However, I am afraid it will be difficult to upstream to GCC support for a target-independent C attribute called `btf_tag' that emits a LLVM-specific DWARF tag. Even if we proposed to use a GCC-specific DWARF tag like DW_TAG_GNU_annotation using the same number, or better a compiler neutral tag like DW_TAG_annotation or DW_TAG_BPF_annotation, adding such an attribute for all targets would still likely to be much controversial... Would you be open to explore other, more generic, ways to convey these annotations to pahole, something that could be easily supported by GCC, and potentially other C compilers?
On 9/9/21 3:45 PM, Jose E. Marchesi wrote: > > Hi Yonghong. > >> LLVM14 added support for a new C attribute ([1]) >> __attribute__((btf_tag("arbitrary_str"))) >> This attribute will be emitted to dwarf ([2]) and pahole >> will convert it to BTF. Or for bpf target, this >> attribute will be emitted to BTF directly ([3]). >> The attribute is intended to provide additional >> information for >> - struct/union type or struct/union member >> - static/global variables >> - static/global function or function parameter. >> >> This new attribute can be used to add attributes >> to kernel codes, e.g., pre- or post- conditions, >> allow/deny info, or any other info in which only >> the kernel is interested. Such attributes will >> be processed by clang frontend and emitted to >> dwarf, converting to BTF by pahole. Ultimiately >> the verifier can use these information for >> verification purpose. >> >> The new attribute can also be used for bpf >> programs, e.g., tagging with __user attributes >> for function parameters, specifying global >> function preconditions, etc. Such information >> may help verifier to detect user program >> bugs. >> >> After this series, pahole dwarf->btf converter >> will be enhanced to support new llvm tag >> for btf_tag attribute. With pahole support, >> we will then try to add a few real use case, >> e.g., __user/__rcu tagging, allow/deny list, >> some kernel function precondition, etc, >> in the kernel. > > We are looking into implementing this in the GCC BPF port. Hi, Jose, thanks for your reply. It would be great if the btf_tag can be implemented in gcc. > > Supporting the new C attribute in BPF programs as a target-specific > attribute, and the new BTF kind, is straightforward enough. > > However, I am afraid it will be difficult to upstream to GCC support for > a target-independent C attribute called `btf_tag' that emits a > LLVM-specific DWARF tag. Even if we proposed to use a GCC-specific Are you concerned with the name? The btf_tag name cames from the discussion in https://lore.kernel.org/bpf/CAADnVQJa=b=hoMGU213wMxyZzycPEKjAPFArKNatbVe4FvzVUA@mail.gmail.com/ as llvm guys want this attribute to be explicitly referring to bpf echo system because we didn't implement for C++, and we didn't try to annotate everywhere. Since its main purpose is to eventually encode in btf (for different architectures), so we settled with btf_tag instead of bpf_tag. But if you have suggestion to change to a different name which can be acceptable by both gcc and llvm community, I am okay with that. > DWARF tag like DW_TAG_GNU_annotation using the same number, or better a > compiler neutral tag like DW_TAG_annotation or DW_TAG_BPF_annotation, > adding such an attribute for all targets would still likely to be much > controversial... This is okay too. If gcc settles with DW_TAG_GNU_annotation with a different number (not conflict with existing other llvm tag numbers), I think llvm can change to have the same name and number since we are still in the release. > > Would you be open to explore other, more generic, ways to convey these > annotations to pahole, something that could be easily supported by GCC, > and potentially other C compilers? Could you share your proposal in detail? I think some kind of difference might be okay if it is handled by pahole and invisible to users, although it would be good if pahole only needs to handle single interface w.r.t. btf_tag support.
> On 9/9/21 3:45 PM, Jose E. Marchesi wrote: >> Hi Yonghong. >> >>> LLVM14 added support for a new C attribute ([1]) >>> __attribute__((btf_tag("arbitrary_str"))) >>> This attribute will be emitted to dwarf ([2]) and pahole >>> will convert it to BTF. Or for bpf target, this >>> attribute will be emitted to BTF directly ([3]). >>> The attribute is intended to provide additional >>> information for >>> - struct/union type or struct/union member >>> - static/global variables >>> - static/global function or function parameter. >>> >>> This new attribute can be used to add attributes >>> to kernel codes, e.g., pre- or post- conditions, >>> allow/deny info, or any other info in which only >>> the kernel is interested. Such attributes will >>> be processed by clang frontend and emitted to >>> dwarf, converting to BTF by pahole. Ultimiately >>> the verifier can use these information for >>> verification purpose. >>> >>> The new attribute can also be used for bpf >>> programs, e.g., tagging with __user attributes >>> for function parameters, specifying global >>> function preconditions, etc. Such information >>> may help verifier to detect user program >>> bugs. >>> >>> After this series, pahole dwarf->btf converter >>> will be enhanced to support new llvm tag >>> for btf_tag attribute. With pahole support, >>> we will then try to add a few real use case, >>> e.g., __user/__rcu tagging, allow/deny list, >>> some kernel function precondition, etc, >>> in the kernel. >> We are looking into implementing this in the GCC BPF port. > > Hi, Jose, thanks for your reply. It would be great if the > btf_tag can be implemented in gcc. > >> Supporting the new C attribute in BPF programs as a target-specific >> attribute, and the new BTF kind, is straightforward enough. >> However, I am afraid it will be difficult to upstream to GCC support >> for >> a target-independent C attribute called `btf_tag' that emits a >> LLVM-specific DWARF tag. Even if we proposed to use a GCC-specific > > Are you concerned with the name? The btf_tag name cames from the > discussion in > https://lore.kernel.org/bpf/CAADnVQJa=b=hoMGU213wMxyZzycPEKjAPFArKNatbVe4FvzVUA@mail.gmail.com/ > as llvm guys want this attribute to be explicitly referring to bpf echo > system because we didn't implement for C++, and we didn't try to > annotate everywhere. Since its main purpose is to eventually encode in > btf (for different architectures), so we settled with btf_tag instead of > bpf_tag. > > But if you have suggestion to change to a different name which can > be acceptable by both gcc and llvm community, I am okay with that. I think the name of the attribute is very fine when BTF is generated directly, like when compiling BPF programs. My concern is that the connection `btf_tag () -> DWARF -> kernel/pahole -> BTF' may be seen as too indirect and application-specific (the kernel) for a general-purpose compiler attribute. >> DWARF tag like DW_TAG_GNU_annotation using the same number, or better a >> compiler neutral tag like DW_TAG_annotation or DW_TAG_BPF_annotation, >> adding such an attribute for all targets would still likely to be much >> controversial... > > This is okay too. If gcc settles with DW_TAG_GNU_annotation with a > different number (not conflict with existing other llvm tag numbers), > I think llvm can change to have the same name and number since we are > still in the release. Thanks, that is very nice and appreciated :) I don't think the particular number used to encode the tag matters much, provided it doesn't collide with any existing one of course... However, there may be a way to entirely avoid creating a new DWARF tag... see below. >> Would you be open to explore other, more generic, ways to convey >> these >> annotations to pahole, something that could be easily supported by GCC, >> and potentially other C compilers? > > Could you share your proposal in detail? I think some kind of difference > might be okay if it is handled by pahole and invisible to users, > although it would be good if pahole only needs to handle single > interface w.r.t. btf_tag support. GCC can currently generate BTF for any target, not just BPF. For example, you can compile an object foo.o with both DWARF and BTF with: $ gcc -c -gdwarf -gbtf foo.c or with just BTF with: $ gcc -c -gbtf foo.c Binutils (ld) also supports full type deduplication for CTF, which is very similar to BTF. We use it to build kernels in-house with CTF enabled for dtrace. It is certainly possible to add support to ld to also merge and deduplicate BTF sections... it is similar enough to CTF to (hopefully) not require much work, and we were already considering doing it anyway for other purposes. So the proposal would be: For GCC, we can implement the btf_tag for any target, but impacting only the BTF output as the name implies. No effect in DWARF. Once ld is able to merge and deduplicate BTF, it shall then be possible to build the kernel and obtain the BTF for it without the aid of pahole, just building it with -gdwarf -gbtf and linking normally. (We know this works with CTF.) For LLVM, nothing would have to be done in the short term: just use the DWARF DIE + pahole approach already implemented. But in the medium term LLVM could be made to 1) support emitting BTF for any target (not sure how difficult would that be, maybe it already can do that?) and 2) to support the -gbtf command-line option. Then the generation of BTF for the kernel could be done in the same way (same build rules) with both compilers, and there would be no need for conveying the extra tags (nor any future extensions to aid the verifier on the kernel side) to pahole via DWARF. Pure BTF all the way up (or down) without diversion to DWARF :) Does this make sense? WDYT?
On 9/9/21 7:19 PM, Jose E. Marchesi wrote: > >> On 9/9/21 3:45 PM, Jose E. Marchesi wrote: >>> Hi Yonghong. >>> >>>> LLVM14 added support for a new C attribute ([1]) >>>> __attribute__((btf_tag("arbitrary_str"))) >>>> This attribute will be emitted to dwarf ([2]) and pahole >>>> will convert it to BTF. Or for bpf target, this >>>> attribute will be emitted to BTF directly ([3]). >>>> The attribute is intended to provide additional >>>> information for >>>> - struct/union type or struct/union member >>>> - static/global variables >>>> - static/global function or function parameter. >>>> >>>> This new attribute can be used to add attributes >>>> to kernel codes, e.g., pre- or post- conditions, >>>> allow/deny info, or any other info in which only >>>> the kernel is interested. Such attributes will >>>> be processed by clang frontend and emitted to >>>> dwarf, converting to BTF by pahole. Ultimiately >>>> the verifier can use these information for >>>> verification purpose. >>>> >>>> The new attribute can also be used for bpf >>>> programs, e.g., tagging with __user attributes >>>> for function parameters, specifying global >>>> function preconditions, etc. Such information >>>> may help verifier to detect user program >>>> bugs. >>>> >>>> After this series, pahole dwarf->btf converter >>>> will be enhanced to support new llvm tag >>>> for btf_tag attribute. With pahole support, >>>> we will then try to add a few real use case, >>>> e.g., __user/__rcu tagging, allow/deny list, >>>> some kernel function precondition, etc, >>>> in the kernel. >>> We are looking into implementing this in the GCC BPF port. >> >> Hi, Jose, thanks for your reply. It would be great if the >> btf_tag can be implemented in gcc. >> >>> Supporting the new C attribute in BPF programs as a target-specific >>> attribute, and the new BTF kind, is straightforward enough. >>> However, I am afraid it will be difficult to upstream to GCC support >>> for >>> a target-independent C attribute called `btf_tag' that emits a >>> LLVM-specific DWARF tag. Even if we proposed to use a GCC-specific >> >> Are you concerned with the name? The btf_tag name cames from the >> discussion in >> https://lore.kernel.org/bpf/CAADnVQJa=b=hoMGU213wMxyZzycPEKjAPFArKNatbVe4FvzVUA@mail.gmail.com/ >> as llvm guys want this attribute to be explicitly referring to bpf echo >> system because we didn't implement for C++, and we didn't try to >> annotate everywhere. Since its main purpose is to eventually encode in >> btf (for different architectures), so we settled with btf_tag instead of >> bpf_tag. >> >> But if you have suggestion to change to a different name which can >> be acceptable by both gcc and llvm community, I am okay with that. > > I think the name of the attribute is very fine when BTF is generated > directly, like when compiling BPF programs. My concern is that the > connection `btf_tag () -> DWARF -> kernel/pahole -> BTF' may be seen as > too indirect and application-specific (the kernel) for a general-purpose > compiler attribute. For llvm, btf_tag implies implementation scope as it *only covers* btf use cases. There are some other use cases which may utilize the same IR/dwarf implementation, but they may use a flag to control or different attribute. And this has been agreed upon with llvm community, so we should be okay here. > >>> DWARF tag like DW_TAG_GNU_annotation using the same number, or better a >>> compiler neutral tag like DW_TAG_annotation or DW_TAG_BPF_annotation, >>> adding such an attribute for all targets would still likely to be much >>> controversial... >> >> This is okay too. If gcc settles with DW_TAG_GNU_annotation with a >> different number (not conflict with existing other llvm tag numbers), >> I think llvm can change to have the same name and number since we are >> still in the release. > > Thanks, that is very nice and appreciated :) I don't think the > particular number used to encode the tag matters much, provided it > doesn't collide with any existing one of course... > > However, there may be a way to entirely avoid creating a new DWARF > tag... see below. > >>> Would you be open to explore other, more generic, ways to convey >>> these >>> annotations to pahole, something that could be easily supported by GCC, >>> and potentially other C compilers? >> >> Could you share your proposal in detail? I think some kind of difference >> might be okay if it is handled by pahole and invisible to users, >> although it would be good if pahole only needs to handle single >> interface w.r.t. btf_tag support. > > GCC can currently generate BTF for any target, not just BPF. For > example, you can compile an object foo.o with both DWARF and BTF with: > > $ gcc -c -gdwarf -gbtf foo.c > > or with just BTF with: > > $ gcc -c -gbtf foo.c > > Binutils (ld) also supports full type deduplication for CTF, which is > very similar to BTF. We use it to build kernels in-house with CTF > enabled for dtrace. It is certainly possible to add support to ld to > also merge and deduplicate BTF sections... it is similar enough to CTF > to (hopefully) not require much work, and we were already considering > doing it anyway for other purposes. > > So the proposal would be: > > For GCC, we can implement the btf_tag for any target, but impacting only > the BTF output as the name implies. No effect in DWARF. Once ld is > able to merge and deduplicate BTF, it shall then be possible to build > the kernel and obtain the BTF for it without the aid of pahole, just > building it with -gdwarf -gbtf and linking normally. (We know this works > with CTF.) This should be okay. > > For LLVM, nothing would have to be done in the short term: just use the > DWARF DIE + pahole approach already implemented. But in the medium term > LLVM could be made to 1) support emitting BTF for any target (not sure > how difficult would that be, maybe it already can do that?) and 2) to > support the -gbtf command-line option. > > Then the generation of BTF for the kernel could be done in the same way > (same build rules) with both compilers, and there would be no need for > conveying the extra tags (nor any future extensions to aid the verifier > on the kernel side) to pahole via DWARF. Pure BTF all the way up (or > down) without diversion to DWARF :) > > Does this make sense? WDYT? During discussion to implement btf_tag attribute, I actually have a prototype to emit BTF with non-BPF targets in llvm. see https://reviews.llvm.org/D103549 But since we get a simpler solution to emit the info to llvm, so we went there. We will keep this in mind, it is totally possible in the future we may start to directly generate BTF from llvm for all architectures.
> On 9/9/21 7:19 PM, Jose E. Marchesi wrote: >> >>> On 9/9/21 3:45 PM, Jose E. Marchesi wrote: >>>> Hi Yonghong. >>>> >>>>> LLVM14 added support for a new C attribute ([1]) >>>>> __attribute__((btf_tag("arbitrary_str"))) >>>>> This attribute will be emitted to dwarf ([2]) and pahole >>>>> will convert it to BTF. Or for bpf target, this >>>>> attribute will be emitted to BTF directly ([3]). >>>>> The attribute is intended to provide additional >>>>> information for >>>>> - struct/union type or struct/union member >>>>> - static/global variables >>>>> - static/global function or function parameter. >>>>> >>>>> This new attribute can be used to add attributes >>>>> to kernel codes, e.g., pre- or post- conditions, >>>>> allow/deny info, or any other info in which only >>>>> the kernel is interested. Such attributes will >>>>> be processed by clang frontend and emitted to >>>>> dwarf, converting to BTF by pahole. Ultimiately >>>>> the verifier can use these information for >>>>> verification purpose. >>>>> >>>>> The new attribute can also be used for bpf >>>>> programs, e.g., tagging with __user attributes >>>>> for function parameters, specifying global >>>>> function preconditions, etc. Such information >>>>> may help verifier to detect user program >>>>> bugs. >>>>> >>>>> After this series, pahole dwarf->btf converter >>>>> will be enhanced to support new llvm tag >>>>> for btf_tag attribute. With pahole support, >>>>> we will then try to add a few real use case, >>>>> e.g., __user/__rcu tagging, allow/deny list, >>>>> some kernel function precondition, etc, >>>>> in the kernel. >>>> We are looking into implementing this in the GCC BPF port. >>> >>> Hi, Jose, thanks for your reply. It would be great if the >>> btf_tag can be implemented in gcc. >>> >>>> Supporting the new C attribute in BPF programs as a target-specific >>>> attribute, and the new BTF kind, is straightforward enough. >>>> However, I am afraid it will be difficult to upstream to GCC support >>>> for >>>> a target-independent C attribute called `btf_tag' that emits a >>>> LLVM-specific DWARF tag. Even if we proposed to use a GCC-specific >>> >>> Are you concerned with the name? The btf_tag name cames from the >>> discussion in >>> https://lore.kernel.org/bpf/CAADnVQJa=b=hoMGU213wMxyZzycPEKjAPFArKNatbVe4FvzVUA@mail.gmail.com/ >>> as llvm guys want this attribute to be explicitly referring to bpf echo >>> system because we didn't implement for C++, and we didn't try to >>> annotate everywhere. Since its main purpose is to eventually encode in >>> btf (for different architectures), so we settled with btf_tag instead of >>> bpf_tag. >>> >>> But if you have suggestion to change to a different name which can >>> be acceptable by both gcc and llvm community, I am okay with that. >> I think the name of the attribute is very fine when BTF is generated >> directly, like when compiling BPF programs. My concern is that the >> connection `btf_tag () -> DWARF -> kernel/pahole -> BTF' may be seen as >> too indirect and application-specific (the kernel) for a general-purpose >> compiler attribute. > > For llvm, btf_tag implies implementation scope as it *only covers* btf > use cases. There are some other use cases which may utilize the same > IR/dwarf implementation, but they may use a flag to control or different > attribute. And this has been agreed upon with llvm community, so we > should be okay here. > >>>> DWARF tag like DW_TAG_GNU_annotation using the same number, or better a >>>> compiler neutral tag like DW_TAG_annotation or DW_TAG_BPF_annotation, >>>> adding such an attribute for all targets would still likely to be much >>>> controversial... >>> >>> This is okay too. If gcc settles with DW_TAG_GNU_annotation with a >>> different number (not conflict with existing other llvm tag numbers), >>> I think llvm can change to have the same name and number since we are >>> still in the release. >> Thanks, that is very nice and appreciated :) I don't think the >> particular number used to encode the tag matters much, provided it >> doesn't collide with any existing one of course... >> However, there may be a way to entirely avoid creating a new DWARF >> tag... see below. >> >>>> Would you be open to explore other, more generic, ways to convey >>>> these >>>> annotations to pahole, something that could be easily supported by GCC, >>>> and potentially other C compilers? >>> >>> Could you share your proposal in detail? I think some kind of difference >>> might be okay if it is handled by pahole and invisible to users, >>> although it would be good if pahole only needs to handle single >>> interface w.r.t. btf_tag support. >> GCC can currently generate BTF for any target, not just BPF. For >> example, you can compile an object foo.o with both DWARF and BTF with: >> $ gcc -c -gdwarf -gbtf foo.c >> or with just BTF with: >> $ gcc -c -gbtf foo.c >> Binutils (ld) also supports full type deduplication for CTF, which >> is >> very similar to BTF. We use it to build kernels in-house with CTF >> enabled for dtrace. It is certainly possible to add support to ld to >> also merge and deduplicate BTF sections... it is similar enough to CTF >> to (hopefully) not require much work, and we were already considering >> doing it anyway for other purposes. >> So the proposal would be: >> For GCC, we can implement the btf_tag for any target, but impacting >> only >> the BTF output as the name implies. No effect in DWARF. Once ld is >> able to merge and deduplicate BTF, it shall then be possible to build >> the kernel and obtain the BTF for it without the aid of pahole, just >> building it with -gdwarf -gbtf and linking normally. (We know this works >> with CTF.) > > This should be okay. Super. We will work in this direction. >> For LLVM, nothing would have to be done in the short term: just use >> the >> DWARF DIE + pahole approach already implemented. But in the medium term >> LLVM could be made to 1) support emitting BTF for any target (not sure >> how difficult would that be, maybe it already can do that?) and 2) to >> support the -gbtf command-line option. >> Then the generation of BTF for the kernel could be done in the same >> way >> (same build rules) with both compilers, and there would be no need for >> conveying the extra tags (nor any future extensions to aid the verifier >> on the kernel side) to pahole via DWARF. Pure BTF all the way up (or >> down) without diversion to DWARF :) >> Does this make sense? WDYT? > > During discussion to implement btf_tag attribute, I actually have a > prototype to emit BTF with non-BPF targets in llvm. > see https://reviews.llvm.org/D103549 > > But since we get a simpler solution to emit the info to llvm, so we went > there. We will keep this in mind, it is totally possible in the future > we may start to directly generate BTF from llvm for all architectures. Nice :) Thanks.