mbox series

[dwarves,v2,0/4] btf: support btf_type_tag attribute

Message ID 20211123045612.1387544-1-yhs@fb.com (mailing list archive)
Headers show
Series btf: support btf_type_tag attribute | expand

Message

Yonghong Song Nov. 23, 2021, 4:56 a.m. UTC
btf_type_tag is a new llvm type attribute which is used similar
to kernel __user/__rcu attributes. The format of btf_type_tag looks like
  __attribute__((btf_type_tag("tag1")))
For the case where the attribute applied to a pointee like
  #define __tag1 __attribute__((btf_type_tag("tag1")))
  #define __tag2 __attribute__((btf_type_tag("tag2")))
  int __tag1 * __tag1 __tag2 *g;
the information will be encoded in dwarf.

In BTF, the attribute is encoded as a new kind
BTF_KIND_TYPE_TAG and latest bpf-next supports it.

The patch added support in pahole, specifically
converts llvm dwarf btf_type_tag attributes to
BTF types. Please see individual patches for details.

Changelog:
  v1 -> v2:
     - reorg an if condition to reduce nesting level.
     - add more comments to explain how to chain type tag types.

Yonghong Song (4):
  libbpf: sync with latest libbpf repo
  dutil: move DW_TAG_LLVM_annotation definition to dutil.h
  dwarf_loader: support btf_type_tag attribute
  btf_encoder: support btf_type_tag attribute

 btf_encoder.c  |   7 +++
 dutil.h        |   4 ++
 dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
 dwarves.h      |  38 +++++++++++++-
 lib/bpf        |   2 +-
 pahole.c       |   8 +++
 6 files changed, 190 insertions(+), 9 deletions(-)

Comments

Andrii Nakryiko Nov. 23, 2021, 6:32 p.m. UTC | #1
On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
>
> btf_type_tag is a new llvm type attribute which is used similar
> to kernel __user/__rcu attributes. The format of btf_type_tag looks like
>   __attribute__((btf_type_tag("tag1")))
> For the case where the attribute applied to a pointee like
>   #define __tag1 __attribute__((btf_type_tag("tag1")))
>   #define __tag2 __attribute__((btf_type_tag("tag2")))
>   int __tag1 * __tag1 __tag2 *g;
> the information will be encoded in dwarf.
>
> In BTF, the attribute is encoded as a new kind
> BTF_KIND_TYPE_TAG and latest bpf-next supports it.
>
> The patch added support in pahole, specifically
> converts llvm dwarf btf_type_tag attributes to
> BTF types. Please see individual patches for details.
>
> Changelog:
>   v1 -> v2:
>      - reorg an if condition to reduce nesting level.
>      - add more comments to explain how to chain type tag types.
>
> Yonghong Song (4):
>   libbpf: sync with latest libbpf repo
>   dutil: move DW_TAG_LLVM_annotation definition to dutil.h
>   dwarf_loader: support btf_type_tag attribute
>   btf_encoder: support btf_type_tag attribute
>

I thought that v1 was already applied, but either way LGTM. I'm not
super familiar with the DWARF loader parts, so I mostly just read it
very superficially :)

Acked-by: Andrii Nakryiko <andrii@kernel.org>


>  btf_encoder.c  |   7 +++
>  dutil.h        |   4 ++
>  dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
>  dwarves.h      |  38 +++++++++++++-
>  lib/bpf        |   2 +-
>  pahole.c       |   8 +++
>  6 files changed, 190 insertions(+), 9 deletions(-)
>
> --
> 2.30.2
>
Arnaldo Carvalho de Melo Nov. 23, 2021, 11:36 p.m. UTC | #2
Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:
> On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
> >
> > btf_type_tag is a new llvm type attribute which is used similar
> > to kernel __user/__rcu attributes. The format of btf_type_tag looks like
> >   __attribute__((btf_type_tag("tag1")))
> > For the case where the attribute applied to a pointee like
> >   #define __tag1 __attribute__((btf_type_tag("tag1")))
> >   #define __tag2 __attribute__((btf_type_tag("tag2")))
> >   int __tag1 * __tag1 __tag2 *g;
> > the information will be encoded in dwarf.
> >
> > In BTF, the attribute is encoded as a new kind
> > BTF_KIND_TYPE_TAG and latest bpf-next supports it.
> >
> > The patch added support in pahole, specifically
> > converts llvm dwarf btf_type_tag attributes to
> > BTF types. Please see individual patches for details.
> >
> > Changelog:
> >   v1 -> v2:
> >      - reorg an if condition to reduce nesting level.
> >      - add more comments to explain how to chain type tag types.
> >
> > Yonghong Song (4):
> >   libbpf: sync with latest libbpf repo
> >   dutil: move DW_TAG_LLVM_annotation definition to dutil.h
> >   dwarf_loader: support btf_type_tag attribute
> >   btf_encoder: support btf_type_tag attribute
> >
> 
> I thought that v1 was already applied, but either way LGTM. I'm not

To the next branch, and the libbpf pahole CI is failing, since a few
days, can you please take a look?

> super familiar with the DWARF loader parts, so I mostly just read it
> very superficially :)

I replaced the patches that changed, re-added the S-o-B for Yonghong and
tested it with llvm-project HEAD.

> Acked-by: Andrii Nakryiko <andrii@kernel.org>

Adding it to the csets.

Thanks!

- Arnaldo
 
> 
> >  btf_encoder.c  |   7 +++
> >  dutil.h        |   4 ++
> >  dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
> >  dwarves.h      |  38 +++++++++++++-
> >  lib/bpf        |   2 +-
> >  pahole.c       |   8 +++
> >  6 files changed, 190 insertions(+), 9 deletions(-)
> >
> > --
> > 2.30.2
> >
Yonghong Song Nov. 24, 2021, 3:56 a.m. UTC | #3
On 11/23/21 3:36 PM, Arnaldo Carvalho de Melo wrote:
> Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:
>> On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
>>>
>>> btf_type_tag is a new llvm type attribute which is used similar
>>> to kernel __user/__rcu attributes. The format of btf_type_tag looks like
>>>    __attribute__((btf_type_tag("tag1")))
>>> For the case where the attribute applied to a pointee like
>>>    #define __tag1 __attribute__((btf_type_tag("tag1")))
>>>    #define __tag2 __attribute__((btf_type_tag("tag2")))
>>>    int __tag1 * __tag1 __tag2 *g;
>>> the information will be encoded in dwarf.
>>>
>>> In BTF, the attribute is encoded as a new kind
>>> BTF_KIND_TYPE_TAG and latest bpf-next supports it.
>>>
>>> The patch added support in pahole, specifically
>>> converts llvm dwarf btf_type_tag attributes to
>>> BTF types. Please see individual patches for details.
>>>
>>> Changelog:
>>>    v1 -> v2:
>>>       - reorg an if condition to reduce nesting level.
>>>       - add more comments to explain how to chain type tag types.
>>>
>>> Yonghong Song (4):
>>>    libbpf: sync with latest libbpf repo
>>>    dutil: move DW_TAG_LLVM_annotation definition to dutil.h
>>>    dwarf_loader: support btf_type_tag attribute
>>>    btf_encoder: support btf_type_tag attribute
>>>
>>
>> I thought that v1 was already applied, but either way LGTM. I'm not
> 
> To the next branch, and the libbpf pahole CI is failing, since a few
> days, can you please take a look?
> 
>> super familiar with the DWARF loader parts, so I mostly just read it
>> very superficially :)
> 
> I replaced the patches that changed, re-added the S-o-B for Yonghong and
> tested it with llvm-project HEAD.

Thanks Arnaldo!

> 
>> Acked-by: Andrii Nakryiko <andrii@kernel.org>
> 
> Adding it to the csets.
> 
> Thanks!
> 
> - Arnaldo
>   
>>
>>>   btf_encoder.c  |   7 +++
>>>   dutil.h        |   4 ++
>>>   dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
>>>   dwarves.h      |  38 +++++++++++++-
>>>   lib/bpf        |   2 +-
>>>   pahole.c       |   8 +++
>>>   6 files changed, 190 insertions(+), 9 deletions(-)
>>>
>>> --
>>> 2.30.2
>>>
>
Andrii Nakryiko Nov. 24, 2021, 4:49 a.m. UTC | #4
On Tue, Nov 23, 2021 at 3:36 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:
> > On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
> > >
> > > btf_type_tag is a new llvm type attribute which is used similar
> > > to kernel __user/__rcu attributes. The format of btf_type_tag looks like
> > >   __attribute__((btf_type_tag("tag1")))
> > > For the case where the attribute applied to a pointee like
> > >   #define __tag1 __attribute__((btf_type_tag("tag1")))
> > >   #define __tag2 __attribute__((btf_type_tag("tag2")))
> > >   int __tag1 * __tag1 __tag2 *g;
> > > the information will be encoded in dwarf.
> > >
> > > In BTF, the attribute is encoded as a new kind
> > > BTF_KIND_TYPE_TAG and latest bpf-next supports it.
> > >
> > > The patch added support in pahole, specifically
> > > converts llvm dwarf btf_type_tag attributes to
> > > BTF types. Please see individual patches for details.
> > >
> > > Changelog:
> > >   v1 -> v2:
> > >      - reorg an if condition to reduce nesting level.
> > >      - add more comments to explain how to chain type tag types.
> > >
> > > Yonghong Song (4):
> > >   libbpf: sync with latest libbpf repo
> > >   dutil: move DW_TAG_LLVM_annotation definition to dutil.h
> > >   dwarf_loader: support btf_type_tag attribute
> > >   btf_encoder: support btf_type_tag attribute
> > >
> >
> > I thought that v1 was already applied, but either way LGTM. I'm not
>
> To the next branch, and the libbpf pahole CI is failing, since a few
> days, can you please take a look?

We've had Clang regression which Yonghong fixed very quickly, but then
we were blocked on Clang nightly builds being broken for days. Seems
like we got a new Clang today, so hopefully libbpf CI will be back to
green again.

>
> > super familiar with the DWARF loader parts, so I mostly just read it
> > very superficially :)
>
> I replaced the patches that changed, re-added the S-o-B for Yonghong and
> tested it with llvm-project HEAD.
>
> > Acked-by: Andrii Nakryiko <andrii@kernel.org>
>
> Adding it to the csets.
>
> Thanks!
>
> - Arnaldo
>
> >
> > >  btf_encoder.c  |   7 +++
> > >  dutil.h        |   4 ++
> > >  dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
> > >  dwarves.h      |  38 +++++++++++++-
> > >  lib/bpf        |   2 +-
> > >  pahole.c       |   8 +++
> > >  6 files changed, 190 insertions(+), 9 deletions(-)
> > >
> > > --
> > > 2.30.2
> > >
>
> --
>
> - Arnaldo
Arnaldo Carvalho de Melo Nov. 25, 2021, 9:33 p.m. UTC | #5
Em Tue, Nov 23, 2021 at 08:49:17PM -0800, Andrii Nakryiko escreveu:
> On Tue, Nov 23, 2021 at 3:36 PM Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> >
> > Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:
> > > On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
> > > >
> > > > btf_type_tag is a new llvm type attribute which is used similar
> > > > to kernel __user/__rcu attributes. The format of btf_type_tag looks like
> > > >   __attribute__((btf_type_tag("tag1")))
> > > > For the case where the attribute applied to a pointee like
> > > >   #define __tag1 __attribute__((btf_type_tag("tag1")))
> > > >   #define __tag2 __attribute__((btf_type_tag("tag2")))
> > > >   int __tag1 * __tag1 __tag2 *g;
> > > > the information will be encoded in dwarf.
> > > >
> > > > In BTF, the attribute is encoded as a new kind
> > > > BTF_KIND_TYPE_TAG and latest bpf-next supports it.
> > > >
> > > > The patch added support in pahole, specifically
> > > > converts llvm dwarf btf_type_tag attributes to
> > > > BTF types. Please see individual patches for details.
> > > >
> > > > Changelog:
> > > >   v1 -> v2:
> > > >      - reorg an if condition to reduce nesting level.
> > > >      - add more comments to explain how to chain type tag types.
> > > >
> > > > Yonghong Song (4):
> > > >   libbpf: sync with latest libbpf repo
> > > >   dutil: move DW_TAG_LLVM_annotation definition to dutil.h
> > > >   dwarf_loader: support btf_type_tag attribute
> > > >   btf_encoder: support btf_type_tag attribute
> > > >
> > >
> > > I thought that v1 was already applied, but either way LGTM. I'm not
> >
> > To the next branch, and the libbpf pahole CI is failing, since a few
> > days, can you please take a look?
> 
> We've had Clang regression which Yonghong fixed very quickly, but then
> we were blocked on Clang nightly builds being broken for days. Seems
> like we got a new Clang today, so hopefully libbpf CI will be back to
> green again.

It is back to green, so I moved next to master, i.e. this series is now
on master.

- Arnaldo
 
> >
> > > super familiar with the DWARF loader parts, so I mostly just read it
> > > very superficially :)
> >
> > I replaced the patches that changed, re-added the S-o-B for Yonghong and
> > tested it with llvm-project HEAD.
> >
> > > Acked-by: Andrii Nakryiko <andrii@kernel.org>
> >
> > Adding it to the csets.
> >
> > Thanks!
> >
> > - Arnaldo
> >
> > >
> > > >  btf_encoder.c  |   7 +++
> > > >  dutil.h        |   4 ++
> > > >  dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
> > > >  dwarves.h      |  38 +++++++++++++-
> > > >  lib/bpf        |   2 +-
> > > >  pahole.c       |   8 +++
> > > >  6 files changed, 190 insertions(+), 9 deletions(-)
> > > >
> > > > --
> > > > 2.30.2
> > > >
> >
> > --
> >
> > - Arnaldo
Yonghong Song Nov. 26, 2021, 5:37 a.m. UTC | #6
On 11/25/21 1:33 PM, Arnaldo Carvalho de Melo wrote:
> Em Tue, Nov 23, 2021 at 08:49:17PM -0800, Andrii Nakryiko escreveu:
>> On Tue, Nov 23, 2021 at 3:36 PM Arnaldo Carvalho de Melo
>> <acme@kernel.org> wrote:
>>>
>>> Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:
>>>> On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
>>>>>
>>>>> btf_type_tag is a new llvm type attribute which is used similar
>>>>> to kernel __user/__rcu attributes. The format of btf_type_tag looks like
>>>>>    __attribute__((btf_type_tag("tag1")))
>>>>> For the case where the attribute applied to a pointee like
>>>>>    #define __tag1 __attribute__((btf_type_tag("tag1")))
>>>>>    #define __tag2 __attribute__((btf_type_tag("tag2")))
>>>>>    int __tag1 * __tag1 __tag2 *g;
>>>>> the information will be encoded in dwarf.
>>>>>
>>>>> In BTF, the attribute is encoded as a new kind
>>>>> BTF_KIND_TYPE_TAG and latest bpf-next supports it.
>>>>>
>>>>> The patch added support in pahole, specifically
>>>>> converts llvm dwarf btf_type_tag attributes to
>>>>> BTF types. Please see individual patches for details.
>>>>>
>>>>> Changelog:
>>>>>    v1 -> v2:
>>>>>       - reorg an if condition to reduce nesting level.
>>>>>       - add more comments to explain how to chain type tag types.
>>>>>
>>>>> Yonghong Song (4):
>>>>>    libbpf: sync with latest libbpf repo
>>>>>    dutil: move DW_TAG_LLVM_annotation definition to dutil.h
>>>>>    dwarf_loader: support btf_type_tag attribute
>>>>>    btf_encoder: support btf_type_tag attribute
>>>>>
>>>>
>>>> I thought that v1 was already applied, but either way LGTM. I'm not
>>>
>>> To the next branch, and the libbpf pahole CI is failing, since a few
>>> days, can you please take a look?
>>
>> We've had Clang regression which Yonghong fixed very quickly, but then
>> we were blocked on Clang nightly builds being broken for days. Seems
>> like we got a new Clang today, so hopefully libbpf CI will be back to
>> green again.
> 
> It is back to green, so I moved next to master, i.e. this series is now
> on master.

Arnaldo,

Is it possible that we could cut a pahole release in the near future?
My kernel using btf_type_tag patch ([1]) needs a pahole version check. 
In old pahole versions, a warning about pointer type subtags not 
supported will be emitted and btf_type_tag attributes (as pointer type 
subtags) will be ignored.

   [1] https://lore.kernel.org/bpf/20211118184810.1846996-1-yhs@fb.com/

Thanks!

> 
> - Arnaldo
>   
>>>
>>>> super familiar with the DWARF loader parts, so I mostly just read it
>>>> very superficially :)
>>>
>>> I replaced the patches that changed, re-added the S-o-B for Yonghong and
>>> tested it with llvm-project HEAD.
>>>
>>>> Acked-by: Andrii Nakryiko <andrii@kernel.org>
>>>
>>> Adding it to the csets.
>>>
>>> Thanks!
>>>
>>> - Arnaldo
>>>
>>>>
>>>>>   btf_encoder.c  |   7 +++
>>>>>   dutil.h        |   4 ++
>>>>>   dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
>>>>>   dwarves.h      |  38 +++++++++++++-
>>>>>   lib/bpf        |   2 +-
>>>>>   pahole.c       |   8 +++
>>>>>   6 files changed, 190 insertions(+), 9 deletions(-)
>>>>>
>>>>> --
>>>>> 2.30.2
>>>>>
>>>
>>> --
>>>
>>> - Arnaldo
>