mbox series

[dwarves,v3,0/2] btf: support BTF_KIND_ENUM64

Message ID 20220629071213.3178592-1-yhs@fb.com (mailing list archive)
Headers show
Series btf: support BTF_KIND_ENUM64 | expand

Message

Yonghong Song June 29, 2022, 7:12 a.m. UTC
Add support for enum64. For 64-bit enumerator value,
previously, the value is truncated into 32bit, e.g.,
for the following enum in linux uapi bpf.h,
  enum {
        BPF_F_INDEX_MASK                = 0xffffffffULL,
        BPF_F_CURRENT_CPU               = BPF_F_INDEX_MASK,
  /* BPF_FUNC_perf_event_output for sk_buff input context. */
        BPF_F_CTXLEN_MASK               = (0xfffffULL << 32),
  };    

BPF_F_CTXLEN_MASK will be encoded with 0 with BTF_KIND_ENUM
after pahole dwarf-to-btf conversion.
With this patch, the BPF_F_CTXLEN_MASK will be encoded properly
with BTF_KIND_ENUM64.

This patch is on top of tmp.master since tmp.master has not
been sync'ed with master branch yet.

Changelogs:
  v2 -> v3:
    - pass struct type/conf_load pointers to btf_encoder__add_enum[_value]
      to make code easier to understand.
  v1 -> v2:
    - Add flag --skip_encoding_btf_enum64 to disable newly-added functionality.

Yonghong Song (2):
  libbpf: Sync with latest libbpf repo
  btf: Support BTF_KIND_ENUM64

 btf_encoder.c     | 67 +++++++++++++++++++++++++++++++++++------------
 btf_encoder.h     |  2 +-
 dwarf_loader.c    | 12 +++++++++
 dwarves.h         |  4 ++-
 dwarves_fprintf.c |  6 ++++-
 lib/bpf           |  2 +-
 pahole.c          | 10 ++++++-
 7 files changed, 81 insertions(+), 22 deletions(-)

Comments

Arnaldo Carvalho de Melo June 29, 2022, 1:16 p.m. UTC | #1
Em Wed, Jun 29, 2022 at 12:12:13AM -0700, Yonghong Song escreveu:
> Add support for enum64. For 64-bit enumerator value,
> previously, the value is truncated into 32bit, e.g.,
> for the following enum in linux uapi bpf.h,
>   enum {
>         BPF_F_INDEX_MASK                = 0xffffffffULL,
>         BPF_F_CURRENT_CPU               = BPF_F_INDEX_MASK,
>   /* BPF_FUNC_perf_event_output for sk_buff input context. */
>         BPF_F_CTXLEN_MASK               = (0xfffffULL << 32),
>   };    
> 
> BPF_F_CTXLEN_MASK will be encoded with 0 with BTF_KIND_ENUM
> after pahole dwarf-to-btf conversion.
> With this patch, the BPF_F_CTXLEN_MASK will be encoded properly
> with BTF_KIND_ENUM64.
> 
> This patch is on top of tmp.master since tmp.master has not
> been sync'ed with master branch yet.
> 
> Changelogs:
>   v2 -> v3:
>     - pass struct type/conf_load pointers to btf_encoder__add_enum[_value]
>       to make code easier to understand.

Yeah, that is more clear indeed.

- Arnaldo

>   v1 -> v2:
>     - Add flag --skip_encoding_btf_enum64 to disable newly-added functionality.
> 
> Yonghong Song (2):
>   libbpf: Sync with latest libbpf repo
>   btf: Support BTF_KIND_ENUM64
> 
>  btf_encoder.c     | 67 +++++++++++++++++++++++++++++++++++------------
>  btf_encoder.h     |  2 +-
>  dwarf_loader.c    | 12 +++++++++
>  dwarves.h         |  4 ++-
>  dwarves_fprintf.c |  6 ++++-
>  lib/bpf           |  2 +-
>  pahole.c          | 10 ++++++-
>  7 files changed, 81 insertions(+), 22 deletions(-)
> 
> -- 
> 2.30.2
Arnaldo Carvalho de Melo June 29, 2022, 9:44 p.m. UTC | #2
Em Wed, Jun 29, 2022 at 12:12:13AM -0700, Yonghong Song escreveu:
> Add support for enum64. For 64-bit enumerator value,
> previously, the value is truncated into 32bit, e.g.,
> for the following enum in linux uapi bpf.h,
>   enum {
>         BPF_F_INDEX_MASK                = 0xffffffffULL,
>         BPF_F_CURRENT_CPU               = BPF_F_INDEX_MASK,
>   /* BPF_FUNC_perf_event_output for sk_buff input context. */
>         BPF_F_CTXLEN_MASK               = (0xfffffULL << 32),
>   };    

Applied, added the entry for skip generating enums to the man page,
added support to the pahole BTF loader, used the new pahole to build
bpf-next/master, all seems ok, pushing to next on git.kernel.org so that
the libbpf github CI can give it a go.

To build with torvalds/master one has to add --skip_encoding_btf_enum64,
I think, haven't tested with it, without it isn't working, libbpf
complains at that btfids tool.

Please check/test what is in there now:

  git://git.kernel.org/pub/scm/devel/pahole/pahole.git next
  https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next

Unless someone screams I plan pushing out a new release, update fedora
packages, etc early next week its overdue by now.

- Arnaldo
 
> BPF_F_CTXLEN_MASK will be encoded with 0 with BTF_KIND_ENUM
> after pahole dwarf-to-btf conversion.
> With this patch, the BPF_F_CTXLEN_MASK will be encoded properly
> with BTF_KIND_ENUM64.
> 
> This patch is on top of tmp.master since tmp.master has not
> been sync'ed with master branch yet.
> 
> Changelogs:
>   v2 -> v3:
>     - pass struct type/conf_load pointers to btf_encoder__add_enum[_value]
>       to make code easier to understand.
>   v1 -> v2:
>     - Add flag --skip_encoding_btf_enum64 to disable newly-added functionality.
> 
> Yonghong Song (2):
>   libbpf: Sync with latest libbpf repo
>   btf: Support BTF_KIND_ENUM64
> 
>  btf_encoder.c     | 67 +++++++++++++++++++++++++++++++++++------------
>  btf_encoder.h     |  2 +-
>  dwarf_loader.c    | 12 +++++++++
>  dwarves.h         |  4 ++-
>  dwarves_fprintf.c |  6 ++++-
>  lib/bpf           |  2 +-
>  pahole.c          | 10 ++++++-
>  7 files changed, 81 insertions(+), 22 deletions(-)
> 
> -- 
> 2.30.2
Yonghong Song June 30, 2022, 2:33 a.m. UTC | #3
On 6/29/22 2:44 PM, Arnaldo Carvalho de Melo wrote:
> Em Wed, Jun 29, 2022 at 12:12:13AM -0700, Yonghong Song escreveu:
>> Add support for enum64. For 64-bit enumerator value,
>> previously, the value is truncated into 32bit, e.g.,
>> for the following enum in linux uapi bpf.h,
>>    enum {
>>          BPF_F_INDEX_MASK                = 0xffffffffULL,
>>          BPF_F_CURRENT_CPU               = BPF_F_INDEX_MASK,
>>    /* BPF_FUNC_perf_event_output for sk_buff input context. */
>>          BPF_F_CTXLEN_MASK               = (0xfffffULL << 32),
>>    };
> 
> Applied, added the entry for skip generating enums to the man page,
> added support to the pahole BTF loader, used the new pahole to build
> bpf-next/master, all seems ok, pushing to next on git.kernel.org so that
> the libbpf github CI can give it a go.
> 
> To build with torvalds/master one has to add --skip_encoding_btf_enum64,
> I think, haven't tested with it, without it isn't working, libbpf
> complains at that btfids tool.

The patch to support enum64 in btf_laoder looks good to me too.
Thanks for checking and additional testing!

> 
> Please check/test what is in there now:
> 
>    git://git.kernel.org/pub/scm/devel/pahole/pahole.git next
>    https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next
> 
> Unless someone screams I plan pushing out a new release, update fedora
> packages, etc early next week its overdue by now.
> 
> - Arnaldo
>   
>> BPF_F_CTXLEN_MASK will be encoded with 0 with BTF_KIND_ENUM
>> after pahole dwarf-to-btf conversion.
>> With this patch, the BPF_F_CTXLEN_MASK will be encoded properly
>> with BTF_KIND_ENUM64.
>>
>> This patch is on top of tmp.master since tmp.master has not
>> been sync'ed with master branch yet.
>>
>> Changelogs:
>>    v2 -> v3:
>>      - pass struct type/conf_load pointers to btf_encoder__add_enum[_value]
>>        to make code easier to understand.
>>    v1 -> v2:
>>      - Add flag --skip_encoding_btf_enum64 to disable newly-added functionality.
>>
>> Yonghong Song (2):
>>    libbpf: Sync with latest libbpf repo
>>    btf: Support BTF_KIND_ENUM64
>>
>>   btf_encoder.c     | 67 +++++++++++++++++++++++++++++++++++------------
>>   btf_encoder.h     |  2 +-
>>   dwarf_loader.c    | 12 +++++++++
>>   dwarves.h         |  4 ++-
>>   dwarves_fprintf.c |  6 ++++-
>>   lib/bpf           |  2 +-
>>   pahole.c          | 10 ++++++-
>>   7 files changed, 81 insertions(+), 22 deletions(-)
>>
>> -- 
>> 2.30.2
>
Jiri Olsa July 11, 2022, 9:16 a.m. UTC | #4
On Wed, Jun 29, 2022 at 06:44:52PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Wed, Jun 29, 2022 at 12:12:13AM -0700, Yonghong Song escreveu:
> > Add support for enum64. For 64-bit enumerator value,
> > previously, the value is truncated into 32bit, e.g.,
> > for the following enum in linux uapi bpf.h,
> >   enum {
> >         BPF_F_INDEX_MASK                = 0xffffffffULL,
> >         BPF_F_CURRENT_CPU               = BPF_F_INDEX_MASK,
> >   /* BPF_FUNC_perf_event_output for sk_buff input context. */
> >         BPF_F_CTXLEN_MASK               = (0xfffffULL << 32),
> >   };    
> 
> Applied, added the entry for skip generating enums to the man page,
> added support to the pahole BTF loader, used the new pahole to build
> bpf-next/master, all seems ok, pushing to next on git.kernel.org so that
> the libbpf github CI can give it a go.
> 
> To build with torvalds/master one has to add --skip_encoding_btf_enum64,
> I think, haven't tested with it, without it isn't working, libbpf
> complains at that btfids tool.
> 
> Please check/test what is in there now:
> 
>   git://git.kernel.org/pub/scm/devel/pahole/pahole.git next
>   https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next
> 
> Unless someone screams I plan pushing out a new release, update fedora
> packages, etc early next week its overdue by now.

I used this new pahole in kernel build and it looks ok,
but I'm getting following warning:

  BTFIDS  vmlinux
WARN: resolve_btfids: unresolved symbol mptcp_sock

might be specific to my .config, I'll check and let you know

jirka
Jiri Olsa July 11, 2022, 9:37 a.m. UTC | #5
On Mon, Jul 11, 2022 at 11:16:21AM +0200, Jiri Olsa wrote:
> On Wed, Jun 29, 2022 at 06:44:52PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Wed, Jun 29, 2022 at 12:12:13AM -0700, Yonghong Song escreveu:
> > > Add support for enum64. For 64-bit enumerator value,
> > > previously, the value is truncated into 32bit, e.g.,
> > > for the following enum in linux uapi bpf.h,
> > >   enum {
> > >         BPF_F_INDEX_MASK                = 0xffffffffULL,
> > >         BPF_F_CURRENT_CPU               = BPF_F_INDEX_MASK,
> > >   /* BPF_FUNC_perf_event_output for sk_buff input context. */
> > >         BPF_F_CTXLEN_MASK               = (0xfffffULL << 32),
> > >   };    
> > 
> > Applied, added the entry for skip generating enums to the man page,
> > added support to the pahole BTF loader, used the new pahole to build
> > bpf-next/master, all seems ok, pushing to next on git.kernel.org so that
> > the libbpf github CI can give it a go.
> > 
> > To build with torvalds/master one has to add --skip_encoding_btf_enum64,
> > I think, haven't tested with it, without it isn't working, libbpf
> > complains at that btfids tool.
> > 
> > Please check/test what is in there now:
> > 
> >   git://git.kernel.org/pub/scm/devel/pahole/pahole.git next
> >   https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next
> > 
> > Unless someone screams I plan pushing out a new release, update fedora
> > packages, etc early next week its overdue by now.
> 
> I used this new pahole in kernel build and it looks ok,
> but I'm getting following warning:
> 
>   BTFIDS  vmlinux
> WARN: resolve_btfids: unresolved symbol mptcp_sock
> 
> might be specific to my .config, I'll check and let you know

ok I have only FWD declaration of mptcp_sock struct, that's why it
can't get resolved.. when I enabled CONFIG_MPTCP it goes away

so no problem with pahole ;-)

Tested-by: Jiri Olsa <jolsa@kernel.org>


the problem is with the btf_sock_ids array that carries mptcp_sock
BTF id even when the CONFIG_MPTCP option is not enabled

I'll follow up on that in separate email

thanks,
jirka