| Message ID | 20230115071613.125791-1-danieltimlee@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | samples/bpf: modernize BPF functionality test programs | expand |
On Sat, Jan 14, 2023 at 11:16 PM Daniel T. Lee <danieltimlee@gmail.com> wrote: > > Currently, there are many programs under samples/bpf to test the > various functionality of BPF that have been developed for a long time. > However, the kernel (BPF) has changed a lot compared to the 2016 when > some of these test programs were first introduced. > > Therefore, some of these programs use the deprecated function of BPF, > and some programs no longer work normally due to changes in the API. > > To list some of the kernel changes that this patch set is focusing on, > - legacy BPF map declaration syntax support had been dropped [1] > - bpf_trace_printk() always append newline at the end [2] > - deprecated styled BPF section header (bpf_load style) [3] > - urandom_read tracepoint is removed (used for testing overhead) [4] > - ping sends packet with SOCK_DGRAM instead of SOCK_RAW [5]* > - use "vmlinux.h" instead of including individual headers > > In addition to this, this patchset tries to modernize the existing > testing scripts a bit. And for network-related testing programs, > a separate header file was created and applied. (To use the > Endianness conversion function from xdp_sample and bunch of constants) Nice set of cleanups. Applied. As a follow up could you convert some of them to proper selftests/bpf ? Unfortunately samples/bpf will keep bit rotting despite your herculean efforts.
On Mon, Jan 16, 2023 at 6:38 AM Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote: > > On Sat, Jan 14, 2023 at 11:16 PM Daniel T. Lee <danieltimlee@gmail.com> wrote: > > > > Currently, there are many programs under samples/bpf to test the > > various functionality of BPF that have been developed for a long time. > > However, the kernel (BPF) has changed a lot compared to the 2016 when > > some of these test programs were first introduced. > > > > Therefore, some of these programs use the deprecated function of BPF, > > and some programs no longer work normally due to changes in the API. > > > > To list some of the kernel changes that this patch set is focusing on, > > - legacy BPF map declaration syntax support had been dropped [1] > > - bpf_trace_printk() always append newline at the end [2] > > - deprecated styled BPF section header (bpf_load style) [3] > > - urandom_read tracepoint is removed (used for testing overhead) [4] > > - ping sends packet with SOCK_DGRAM instead of SOCK_RAW [5]* > > - use "vmlinux.h" instead of including individual headers > > > > In addition to this, this patchset tries to modernize the existing > > testing scripts a bit. And for network-related testing programs, > > a separate header file was created and applied. (To use the > > Endianness conversion function from xdp_sample and bunch of constants) > > Nice set of cleanups. Applied. > As a follow up could you convert some of them to proper selftests/bpf ? > Unfortunately samples/bpf will keep bit rotting despite your herculean efforts. I really appreciate for your compliment! I'll try to convert the existing sample to selftest in the next patch.
"Daniel T. Lee" <danieltimlee@gmail.com> writes: > On Mon, Jan 16, 2023 at 6:38 AM Alexei Starovoitov > <alexei.starovoitov@gmail.com> wrote: >> >> On Sat, Jan 14, 2023 at 11:16 PM Daniel T. Lee <danieltimlee@gmail.com> wrote: >> > >> > Currently, there are many programs under samples/bpf to test the >> > various functionality of BPF that have been developed for a long time. >> > However, the kernel (BPF) has changed a lot compared to the 2016 when >> > some of these test programs were first introduced. >> > >> > Therefore, some of these programs use the deprecated function of BPF, >> > and some programs no longer work normally due to changes in the API. >> > >> > To list some of the kernel changes that this patch set is focusing on, >> > - legacy BPF map declaration syntax support had been dropped [1] >> > - bpf_trace_printk() always append newline at the end [2] >> > - deprecated styled BPF section header (bpf_load style) [3] >> > - urandom_read tracepoint is removed (used for testing overhead) [4] >> > - ping sends packet with SOCK_DGRAM instead of SOCK_RAW [5]* >> > - use "vmlinux.h" instead of including individual headers >> > >> > In addition to this, this patchset tries to modernize the existing >> > testing scripts a bit. And for network-related testing programs, >> > a separate header file was created and applied. (To use the >> > Endianness conversion function from xdp_sample and bunch of constants) >> >> Nice set of cleanups. Applied. >> As a follow up could you convert some of them to proper selftests/bpf ? >> Unfortunately samples/bpf will keep bit rotting despite your herculean efforts. > > I really appreciate for your compliment! > I'll try to convert the existing sample to selftest in the next patch. Maybe this is a good time to mention that we recently ported some of the XDP utilities from samples/bpf to xdp-tools, in the form of the 'xdp-bench' utility: https://github.com/xdp-project/xdp-tools/tree/master/xdp-bench It's basically a combination of all the xdp_redirect* samples, but also includes the functionality from the xdp_rxq_info sample program (i.e., the ability to monitor RXQs and use other return codes). I'm planning to submit a patch to remove those utilities from samples/bpf after we tag the next release of xdp-tools (have one or two outstanding issues to clear before we do that), but wanted to give you a head's up so you don't spend any time on those particular utilities when you're cleaning up samples :) -Toke
On 1/16/23 2:35 PM, Toke Høiland-Jørgensen wrote: > "Daniel T. Lee" <danieltimlee@gmail.com> writes: >> On Mon, Jan 16, 2023 at 6:38 AM Alexei Starovoitov >> <alexei.starovoitov@gmail.com> wrote: >>> On Sat, Jan 14, 2023 at 11:16 PM Daniel T. Lee <danieltimlee@gmail.com> wrote: >>>> >>>> Currently, there are many programs under samples/bpf to test the >>>> various functionality of BPF that have been developed for a long time. >>>> However, the kernel (BPF) has changed a lot compared to the 2016 when >>>> some of these test programs were first introduced. >>>> >>>> Therefore, some of these programs use the deprecated function of BPF, >>>> and some programs no longer work normally due to changes in the API. >>>> >>>> To list some of the kernel changes that this patch set is focusing on, >>>> - legacy BPF map declaration syntax support had been dropped [1] >>>> - bpf_trace_printk() always append newline at the end [2] >>>> - deprecated styled BPF section header (bpf_load style) [3] >>>> - urandom_read tracepoint is removed (used for testing overhead) [4] >>>> - ping sends packet with SOCK_DGRAM instead of SOCK_RAW [5]* >>>> - use "vmlinux.h" instead of including individual headers >>>> >>>> In addition to this, this patchset tries to modernize the existing >>>> testing scripts a bit. And for network-related testing programs, >>>> a separate header file was created and applied. (To use the >>>> Endianness conversion function from xdp_sample and bunch of constants) >>> >>> Nice set of cleanups. Applied. >>> As a follow up could you convert some of them to proper selftests/bpf ? >>> Unfortunately samples/bpf will keep bit rotting despite your herculean efforts. >> >> I really appreciate for your compliment! >> I'll try to convert the existing sample to selftest in the next patch. This would be awesome, thanks a lot Daniel! > Maybe this is a good time to mention that we recently ported some of the > XDP utilities from samples/bpf to xdp-tools, in the form of the > 'xdp-bench' utility: > https://github.com/xdp-project/xdp-tools/tree/master/xdp-bench > > It's basically a combination of all the xdp_redirect* samples, but also > includes the functionality from the xdp_rxq_info sample program (i.e., > the ability to monitor RXQs and use other return codes). > > I'm planning to submit a patch to remove those utilities from > samples/bpf after we tag the next release of xdp-tools (have one or two > outstanding issues to clear before we do that), but wanted to give you a > head's up so you don't spend any time on those particular utilities when > you're cleaning up samples :) Nice! Once we're through with most relevant ones from samples/bpf, it would be great to only have a readme in that dir (and that's really all) with pointers for developers on how to get started.. including BPF selftests, xdp tools, links to ebpf.io/applications and ebpf.io/infrastructure, etc where more resources can be found, essentially a small getting started doc for BPF devs.
Daniel Borkmann <daniel@iogearbox.net> writes: > On 1/16/23 2:35 PM, Toke Høiland-Jørgensen wrote: >> "Daniel T. Lee" <danieltimlee@gmail.com> writes: >>> On Mon, Jan 16, 2023 at 6:38 AM Alexei Starovoitov >>> <alexei.starovoitov@gmail.com> wrote: >>>> On Sat, Jan 14, 2023 at 11:16 PM Daniel T. Lee <danieltimlee@gmail.com> wrote: >>>>> >>>>> Currently, there are many programs under samples/bpf to test the >>>>> various functionality of BPF that have been developed for a long time. >>>>> However, the kernel (BPF) has changed a lot compared to the 2016 when >>>>> some of these test programs were first introduced. >>>>> >>>>> Therefore, some of these programs use the deprecated function of BPF, >>>>> and some programs no longer work normally due to changes in the API. >>>>> >>>>> To list some of the kernel changes that this patch set is focusing on, >>>>> - legacy BPF map declaration syntax support had been dropped [1] >>>>> - bpf_trace_printk() always append newline at the end [2] >>>>> - deprecated styled BPF section header (bpf_load style) [3] >>>>> - urandom_read tracepoint is removed (used for testing overhead) [4] >>>>> - ping sends packet with SOCK_DGRAM instead of SOCK_RAW [5]* >>>>> - use "vmlinux.h" instead of including individual headers >>>>> >>>>> In addition to this, this patchset tries to modernize the existing >>>>> testing scripts a bit. And for network-related testing programs, >>>>> a separate header file was created and applied. (To use the >>>>> Endianness conversion function from xdp_sample and bunch of constants) >>>> >>>> Nice set of cleanups. Applied. >>>> As a follow up could you convert some of them to proper selftests/bpf ? >>>> Unfortunately samples/bpf will keep bit rotting despite your herculean efforts. >>> >>> I really appreciate for your compliment! >>> I'll try to convert the existing sample to selftest in the next patch. > > This would be awesome, thanks a lot Daniel! > >> Maybe this is a good time to mention that we recently ported some of the >> XDP utilities from samples/bpf to xdp-tools, in the form of the >> 'xdp-bench' utility: >> https://github.com/xdp-project/xdp-tools/tree/master/xdp-bench >> >> It's basically a combination of all the xdp_redirect* samples, but also >> includes the functionality from the xdp_rxq_info sample program (i.e., >> the ability to monitor RXQs and use other return codes). >> >> I'm planning to submit a patch to remove those utilities from >> samples/bpf after we tag the next release of xdp-tools (have one or two >> outstanding issues to clear before we do that), but wanted to give you a >> head's up so you don't spend any time on those particular utilities when >> you're cleaning up samples :) > > Nice! Once we're through with most relevant ones from samples/bpf, it would > be great to only have a readme in that dir (and that's really all) with pointers > for developers on how to get started.. including BPF selftests, xdp tools, links > to ebpf.io/applications and ebpf.io/infrastructure, etc where more resources > can be found, essentially a small getting started doc for BPF devs. Yeah, good point! Will keep that in mind :) -Toke
Currently, there are many programs under samples/bpf to test the various functionality of BPF that have been developed for a long time. However, the kernel (BPF) has changed a lot compared to the 2016 when some of these test programs were first introduced. Therefore, some of these programs use the deprecated function of BPF, and some programs no longer work normally due to changes in the API. To list some of the kernel changes that this patch set is focusing on, - legacy BPF map declaration syntax support had been dropped [1] - bpf_trace_printk() always append newline at the end [2] - deprecated styled BPF section header (bpf_load style) [3] - urandom_read tracepoint is removed (used for testing overhead) [4] - ping sends packet with SOCK_DGRAM instead of SOCK_RAW [5]* - use "vmlinux.h" instead of including individual headers In addition to this, this patchset tries to modernize the existing testing scripts a bit. And for network-related testing programs, a separate header file was created and applied. (To use the Endianness conversion function from xdp_sample and bunch of constants) [1]: https://github.com/libbpf/libbpf/issues/282 [2]: commit ac5a72ea5c89 ("bpf: Use dedicated bpf_trace_printk event instead of trace_printk()") [3]: commit ceb5dea56543 ("samples: bpf: Remove bpf_load loader completely") [4]: commit 14c174633f34 ("random: remove unused tracepoints") [5]: https://lwn.net/Articles/422330/ *: This is quite old, but I'm not sure why the code was initially developed to filter only SOCK_RAW. Daniel T. Lee (10): samples/bpf: ensure ipv6 is enabled before running tests samples/bpf: refactor BPF functionality testing scripts samples/bpf: fix broken lightweight tunnel testing samples/bpf: fix broken cgroup socket testing samples/bpf: replace broken overhead microbenchmark with fib_table_lookup samples/bpf: replace legacy map with the BTF-defined map samples/bpf: split common macros to net_shared.h samples/bpf: replace BPF programs header with net_shared.h samples/bpf: use vmlinux.h instead of implicit headers in BPF test program samples/bpf: change _kern suffix to .bpf with BPF test programs samples/bpf/Makefile | 14 +++--- ...lwt_len_hist_kern.c => lwt_len_hist.bpf.c} | 29 +++-------- samples/bpf/lwt_len_hist.sh | 4 +- samples/bpf/net_shared.h | 32 ++++++++++++ .../{sock_flags_kern.c => sock_flags.bpf.c} | 24 ++++----- samples/bpf/tc_l2_redirect.sh | 3 ++ samples/bpf/test_cgrp2_sock.sh | 16 +++--- samples/bpf/test_cgrp2_sock2.sh | 9 +++- ...st_cgrp2_tc_kern.c => test_cgrp2_tc.bpf.c} | 34 ++++--------- samples/bpf/test_cgrp2_tc.sh | 8 +-- samples/bpf/test_lwt_bpf.c | 50 ++++++++----------- samples/bpf/test_lwt_bpf.sh | 19 ++++--- ...ap_in_map_kern.c => test_map_in_map.bpf.c} | 7 +-- samples/bpf/test_map_in_map_user.c | 2 +- ...robe_kern.c => test_overhead_kprobe.bpf.c} | 6 +-- ...w_tp_kern.c => test_overhead_raw_tp.bpf.c} | 4 +- ...rhead_tp_kern.c => test_overhead_tp.bpf.c} | 29 +++++++---- samples/bpf/test_overhead_user.c | 34 ++++++++----- samples/bpf/xdp_sample.bpf.h | 22 +------- 19 files changed, 179 insertions(+), 167 deletions(-) rename samples/bpf/{lwt_len_hist_kern.c => lwt_len_hist.bpf.c} (75%) create mode 100644 samples/bpf/net_shared.h rename samples/bpf/{sock_flags_kern.c => sock_flags.bpf.c} (66%) rename samples/bpf/{test_cgrp2_tc_kern.c => test_cgrp2_tc.bpf.c} (70%) rename samples/bpf/{test_map_in_map_kern.c => test_map_in_map.bpf.c} (97%) rename samples/bpf/{test_overhead_kprobe_kern.c => test_overhead_kprobe.bpf.c} (92%) rename samples/bpf/{test_overhead_raw_tp_kern.c => test_overhead_raw_tp.bpf.c} (82%) rename samples/bpf/{test_overhead_tp_kern.c => test_overhead_tp.bpf.c} (61%)