| Message ID | 20210914223004.244411-1-yhs@fb.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | bpf: add support for new btf kind BTF_KIND_TAG | expand |
On Tue, Sep 14, 2021 at 3:30 PM Yonghong Song <yhs@fb.com> wrote: > > LLVM14 added support for a new C attribute ([1]) > __attribute__((btf_tag("arbitrary_str"))) > This attribute will be emitted to dwarf ([2]) and pahole > will convert it to BTF. Or for bpf target, this > attribute will be emitted to BTF directly ([3], [4]). > The attribute is intended to provide additional > information for > - struct/union type or struct/union member > - static/global variables > - static/global function or function parameter. > > This new attribute can be used to add attributes > to kernel codes, e.g., pre- or post- conditions, > allow/deny info, or any other info in which only > the kernel is interested. Such attributes will > be processed by clang frontend and emitted to > dwarf, converting to BTF by pahole. Ultimiately > the verifier can use these information for > verification purpose. > > The new attribute can also be used for bpf > programs, e.g., tagging with __user attributes > for function parameters, specifying global > function preconditions, etc. Such information > may help verifier to detect user program > bugs. > > After this series, pahole dwarf->btf converter > will be enhanced to support new llvm tag > for btf_tag attribute. With pahole support, > we will then try to add a few real use case, > e.g., __user/__rcu tagging, allow/deny list, > some kernel function precondition, etc, > in the kernel. > > In the rest of the series, Patches 1-2 had > kernel support. Patches 3-4 added > libbpf support. Patch 5 added bpftool > support. Patches 6-10 added various selftests. > Patch 11 added documentation for the new kind. > > [1] https://reviews.llvm.org/D106614 > [2] https://reviews.llvm.org/D106621 > [3] https://reviews.llvm.org/D106622 > [4] https://reviews.llvm.org/D109560 > > Changelog: > v2 -> v3: > - put NR_BTF_KINDS and BTF_KIND_MAX into enum as well > - check component_idx earlier (check_meta stage) in kernel > - add more tests > - fix misc nits > v1 -> v2: > - BTF ELF format changed in llvm ([4] above), > so cross-board change to use the new format. > - Clarified in commit message that BTF_KIND_TAG > is not emitted by bpftool btf dump format c. > - Fix various comments from Andrii. > > Yonghong Song (11): > btf: change BTF_KIND_* macros to enums > bpf: support for new btf kind BTF_KIND_TAG > libbpf: rename btf_{hash,equal}_int to btf_{hash,equal}_int_tag > libbpf: add support for BTF_KIND_TAG > bpftool: add support for BTF_KIND_TAG > selftests/bpf: test libbpf API function btf__add_tag() > selftests/bpf: change NAME_NTH/IS_NAME_NTH for BTF_KIND_TAG format > selftests/bpf: add BTF_KIND_TAG unit tests > selftests/bpf: test BTF_KIND_TAG for deduplication > selftests/bpf: add a test with a bpf program with btf_tag attributes > docs/bpf: add documentation for BTF_KIND_TAG > > Documentation/bpf/btf.rst | 29 +- > include/uapi/linux/btf.h | 55 ++- > kernel/bpf/btf.c | 128 +++++ > tools/bpf/bpftool/btf.c | 12 + > tools/include/uapi/linux/btf.h | 55 ++- > tools/lib/bpf/btf.c | 84 +++- > tools/lib/bpf/btf.h | 15 + > tools/lib/bpf/btf_dump.c | 3 + > tools/lib/bpf/libbpf.c | 31 +- > tools/lib/bpf/libbpf.map | 2 + > tools/lib/bpf/libbpf_internal.h | 2 + > tools/testing/selftests/bpf/btf_helpers.c | 7 +- > tools/testing/selftests/bpf/prog_tests/btf.c | 441 +++++++++++++++++- > .../selftests/bpf/prog_tests/btf_tag.c | 14 + > .../selftests/bpf/prog_tests/btf_write.c | 21 + > tools/testing/selftests/bpf/progs/tag.c | 39 ++ > tools/testing/selftests/bpf/test_btf.h | 3 + > 17 files changed, 869 insertions(+), 72 deletions(-) > create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_tag.c > create mode 100644 tools/testing/selftests/bpf/progs/tag.c > > -- > 2.30.2 > I've acked every individual patch, but just to make it more clear, for the series: Acked-by: Andrii Nakryiko <andrii@kernel.org>
On Tue, Sep 14, 2021 at 3:30 PM Yonghong Song <yhs@fb.com> wrote: > > LLVM14 added support for a new C attribute ([1]) > __attribute__((btf_tag("arbitrary_str"))) > This attribute will be emitted to dwarf ([2]) and pahole > will convert it to BTF. Or for bpf target, this > attribute will be emitted to BTF directly ([3], [4]). > The attribute is intended to provide additional > information for > - struct/union type or struct/union member > - static/global variables > - static/global function or function parameter. > > This new attribute can be used to add attributes > to kernel codes, e.g., pre- or post- conditions, > allow/deny info, or any other info in which only > the kernel is interested. Such attributes will > be processed by clang frontend and emitted to > dwarf, converting to BTF by pahole. Ultimiately > the verifier can use these information for > verification purpose. > > The new attribute can also be used for bpf > programs, e.g., tagging with __user attributes > for function parameters, specifying global > function preconditions, etc. Such information > may help verifier to detect user program > bugs. > > After this series, pahole dwarf->btf converter > will be enhanced to support new llvm tag > for btf_tag attribute. With pahole support, > we will then try to add a few real use case, > e.g., __user/__rcu tagging, allow/deny list, > some kernel function precondition, etc, > in the kernel. > > In the rest of the series, Patches 1-2 had > kernel support. Patches 3-4 added > libbpf support. Patch 5 added bpftool > support. Patches 6-10 added various selftests. > Patch 11 added documentation for the new kind. > > [1] https://reviews.llvm.org/D106614 > [2] https://reviews.llvm.org/D106621 > [3] https://reviews.llvm.org/D106622 > [4] https://reviews.llvm.org/D109560 > > Changelog: > v2 -> v3: > - put NR_BTF_KINDS and BTF_KIND_MAX into enum as well > - check component_idx earlier (check_meta stage) in kernel > - add more tests > - fix misc nits Applied. Please send an update to selftests/bpf/README. Since folks will be puzzled with messages: progs/tag.c:23:20: warning: unknown attribute 'btf_tag' ignored [-Wunknown-attributes] Even with old clang: ./test_progs -t tag #21 btf_tag:OK Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED The test probably should fail with old clang ?
Hello: This series was applied to bpf/bpf-next.git (refs/heads/master): On Tue, 14 Sep 2021 15:30:04 -0700 you wrote: > LLVM14 added support for a new C attribute ([1]) > __attribute__((btf_tag("arbitrary_str"))) > This attribute will be emitted to dwarf ([2]) and pahole > will convert it to BTF. Or for bpf target, this > attribute will be emitted to BTF directly ([3], [4]). > The attribute is intended to provide additional > information for > - struct/union type or struct/union member > - static/global variables > - static/global function or function parameter. > > [...] Here is the summary with links: - [bpf-next,v3,01/11] btf: change BTF_KIND_* macros to enums https://git.kernel.org/bpf/bpf-next/c/41ced4cd8802 - [bpf-next,v3,02/11] bpf: support for new btf kind BTF_KIND_TAG https://git.kernel.org/bpf/bpf-next/c/b5ea834dde6b - [bpf-next,v3,03/11] libbpf: rename btf_{hash,equal}_int to btf_{hash,equal}_int_tag https://git.kernel.org/bpf/bpf-next/c/30025e8bd80f - [bpf-next,v3,04/11] libbpf: add support for BTF_KIND_TAG https://git.kernel.org/bpf/bpf-next/c/5b84bd10363e - [bpf-next,v3,05/11] bpftool: add support for BTF_KIND_TAG https://git.kernel.org/bpf/bpf-next/c/5c07f2fec003 - [bpf-next,v3,06/11] selftests/bpf: test libbpf API function btf__add_tag() https://git.kernel.org/bpf/bpf-next/c/71d29c2d47d1 - [bpf-next,v3,07/11] selftests/bpf: change NAME_NTH/IS_NAME_NTH for BTF_KIND_TAG format https://git.kernel.org/bpf/bpf-next/c/3df3bd68d481 - [bpf-next,v3,08/11] selftests/bpf: add BTF_KIND_TAG unit tests https://git.kernel.org/bpf/bpf-next/c/35baba7a832f - [bpf-next,v3,09/11] selftests/bpf: test BTF_KIND_TAG for deduplication https://git.kernel.org/bpf/bpf-next/c/ad526474aec1 - [bpf-next,v3,10/11] selftests/bpf: add a test with a bpf program with btf_tag attributes https://git.kernel.org/bpf/bpf-next/c/c240ba287890 - [bpf-next,v3,11/11] docs/bpf: add documentation for BTF_KIND_TAG https://git.kernel.org/bpf/bpf-next/c/48f5a6c41627 You are awesome, thank you! -- Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
On 9/14/21 6:55 PM, Alexei Starovoitov wrote: > On Tue, Sep 14, 2021 at 3:30 PM Yonghong Song <yhs@fb.com> wrote: >> >> LLVM14 added support for a new C attribute ([1]) >> __attribute__((btf_tag("arbitrary_str"))) >> This attribute will be emitted to dwarf ([2]) and pahole >> will convert it to BTF. Or for bpf target, this >> attribute will be emitted to BTF directly ([3], [4]). >> The attribute is intended to provide additional >> information for >> - struct/union type or struct/union member >> - static/global variables >> - static/global function or function parameter. >> >> This new attribute can be used to add attributes >> to kernel codes, e.g., pre- or post- conditions, >> allow/deny info, or any other info in which only >> the kernel is interested. Such attributes will >> be processed by clang frontend and emitted to >> dwarf, converting to BTF by pahole. Ultimiately >> the verifier can use these information for >> verification purpose. >> >> The new attribute can also be used for bpf >> programs, e.g., tagging with __user attributes >> for function parameters, specifying global >> function preconditions, etc. Such information >> may help verifier to detect user program >> bugs. >> >> After this series, pahole dwarf->btf converter >> will be enhanced to support new llvm tag >> for btf_tag attribute. With pahole support, >> we will then try to add a few real use case, >> e.g., __user/__rcu tagging, allow/deny list, >> some kernel function precondition, etc, >> in the kernel. >> >> In the rest of the series, Patches 1-2 had >> kernel support. Patches 3-4 added >> libbpf support. Patch 5 added bpftool >> support. Patches 6-10 added various selftests. >> Patch 11 added documentation for the new kind. >> >> [1] https://reviews.llvm.org/D106614 >> [2] https://reviews.llvm.org/D106621 >> [3] https://reviews.llvm.org/D106622 >> [4] https://reviews.llvm.org/D109560 >> >> Changelog: >> v2 -> v3: >> - put NR_BTF_KINDS and BTF_KIND_MAX into enum as well >> - check component_idx earlier (check_meta stage) in kernel >> - add more tests >> - fix misc nits > > Applied. Please send an update to selftests/bpf/README. > Since folks will be puzzled with messages: > progs/tag.c:23:20: warning: unknown attribute 'btf_tag' ignored > [-Wunknown-attributes] Ya, this is not good. I too focused on the latest clang which has btf_tag support. > > Even with old clang: > ./test_progs -t tag > #21 btf_tag:OK > Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED > > The test probably should fail with old clang ? I will follow atomics example, if btf_tag is not supported, the test will be marked as SKIP. Will also update selftests/bpf/README for when SKIP message may appear. Will send the followup patch soon.
LLVM14 added support for a new C attribute ([1]) __attribute__((btf_tag("arbitrary_str"))) This attribute will be emitted to dwarf ([2]) and pahole will convert it to BTF. Or for bpf target, this attribute will be emitted to BTF directly ([3], [4]). The attribute is intended to provide additional information for - struct/union type or struct/union member - static/global variables - static/global function or function parameter. This new attribute can be used to add attributes to kernel codes, e.g., pre- or post- conditions, allow/deny info, or any other info in which only the kernel is interested. Such attributes will be processed by clang frontend and emitted to dwarf, converting to BTF by pahole. Ultimiately the verifier can use these information for verification purpose. The new attribute can also be used for bpf programs, e.g., tagging with __user attributes for function parameters, specifying global function preconditions, etc. Such information may help verifier to detect user program bugs. After this series, pahole dwarf->btf converter will be enhanced to support new llvm tag for btf_tag attribute. With pahole support, we will then try to add a few real use case, e.g., __user/__rcu tagging, allow/deny list, some kernel function precondition, etc, in the kernel. In the rest of the series, Patches 1-2 had kernel support. Patches 3-4 added libbpf support. Patch 5 added bpftool support. Patches 6-10 added various selftests. Patch 11 added documentation for the new kind. [1] https://reviews.llvm.org/D106614 [2] https://reviews.llvm.org/D106621 [3] https://reviews.llvm.org/D106622 [4] https://reviews.llvm.org/D109560 Changelog: v2 -> v3: - put NR_BTF_KINDS and BTF_KIND_MAX into enum as well - check component_idx earlier (check_meta stage) in kernel - add more tests - fix misc nits v1 -> v2: - BTF ELF format changed in llvm ([4] above), so cross-board change to use the new format. - Clarified in commit message that BTF_KIND_TAG is not emitted by bpftool btf dump format c. - Fix various comments from Andrii. Yonghong Song (11): btf: change BTF_KIND_* macros to enums bpf: support for new btf kind BTF_KIND_TAG libbpf: rename btf_{hash,equal}_int to btf_{hash,equal}_int_tag libbpf: add support for BTF_KIND_TAG bpftool: add support for BTF_KIND_TAG selftests/bpf: test libbpf API function btf__add_tag() selftests/bpf: change NAME_NTH/IS_NAME_NTH for BTF_KIND_TAG format selftests/bpf: add BTF_KIND_TAG unit tests selftests/bpf: test BTF_KIND_TAG for deduplication selftests/bpf: add a test with a bpf program with btf_tag attributes docs/bpf: add documentation for BTF_KIND_TAG Documentation/bpf/btf.rst | 29 +- include/uapi/linux/btf.h | 55 ++- kernel/bpf/btf.c | 128 +++++ tools/bpf/bpftool/btf.c | 12 + tools/include/uapi/linux/btf.h | 55 ++- tools/lib/bpf/btf.c | 84 +++- tools/lib/bpf/btf.h | 15 + tools/lib/bpf/btf_dump.c | 3 + tools/lib/bpf/libbpf.c | 31 +- tools/lib/bpf/libbpf.map | 2 + tools/lib/bpf/libbpf_internal.h | 2 + tools/testing/selftests/bpf/btf_helpers.c | 7 +- tools/testing/selftests/bpf/prog_tests/btf.c | 441 +++++++++++++++++- .../selftests/bpf/prog_tests/btf_tag.c | 14 + .../selftests/bpf/prog_tests/btf_write.c | 21 + tools/testing/selftests/bpf/progs/tag.c | 39 ++ tools/testing/selftests/bpf/test_btf.h | 3 + 17 files changed, 869 insertions(+), 72 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_tag.c create mode 100644 tools/testing/selftests/bpf/progs/tag.c