| Message ID | 20210928093100.27124-1-lmb@cloudflare.com (mailing list archive) |
|---|---|
| State | Accepted |
| Delegated to: | BPF |
| Headers | show |
| Series | [bpf-next] bpf: do not invoke the XDP dispatcher for PROG_RUN with single repeat | expand |
| Context | Check | Description |
|---|---|---|
| netdev/cover_letter | success | Link |
| netdev/fixes_present | success | Link |
| netdev/patch_count | success | Link |
| netdev/tree_selection | success | Clearly marked for bpf-next |
| netdev/subject_prefix | success | Link |
| netdev/cc_maintainers | warning | 4 maintainers not CCed: kpsingh@kernel.org kafai@fb.com yhs@fb.com songliubraving@fb.com |
| netdev/source_inline | success | Was 0 now: 0 |
| netdev/verify_signedoff | success | Link |
| netdev/module_param | success | Was 0 now: 0 |
| netdev/build_32bit | success | Errors and warnings before: 13 this patch: 13 |
| netdev/kdoc | success | Errors and warnings before: 0 this patch: 0 |
| netdev/verify_fixes | success | Link |
| netdev/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 18 lines checked |
| netdev/build_allmodconfig_warn | success | Errors and warnings before: 13 this patch: 13 |
| netdev/header_inline | success | Link |
| bpf/vmtest-bpf-next-PR | success | PR summary |
| bpf/vmtest-bpf-next | success | VM_Test |
On Tue, Sep 28, 2021 at 2:31 AM Lorenz Bauer <lmb@cloudflare.com> wrote: > > We have a unit test that invokes an XDP program with 1m different > inputs, aka 1m BPF_PROG_RUN syscalls. We run this test concurrently > with slight variations in how we generated the input. > > Since commit f23c4b3924d2 ("bpf: Start using the BPF dispatcher in BPF_TEST_RUN") > the unit test has slowed down significantly. Digging deeper reveals that > the concurrent tests are serialised in the kernel on the XDP dispatcher. > This is a global resource that is protected by a mutex, on which we contend. > > Fix this by not calling into the XDP dispatcher if we only want to perform > a single run of the BPF program. > > See: https://lore.kernel.org/bpf/CACAyw9_y4QumOW35qpgTbLsJ532uGq-kVW-VESJzGyiZkypnvw@mail.gmail.com/ > > Signed-off-by: Lorenz Bauer <lmb@cloudflare.com> Applied. Thanks
diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c index fcb2f493f710..6593a71dba5f 100644 --- a/net/bpf/test_run.c +++ b/net/bpf/test_run.c @@ -803,7 +803,8 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, if (ret) goto free_data; - bpf_prog_change_xdp(NULL, prog); + if (repeat > 1) + bpf_prog_change_xdp(NULL, prog); ret = bpf_test_run(prog, &xdp, repeat, &retval, &duration, true); /* We convert the xdp_buff back to an xdp_md before checking the return * code so the reference count of any held netdevice will be decremented @@ -824,7 +825,8 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, sizeof(struct xdp_md)); out: - bpf_prog_change_xdp(prog, NULL); + if (repeat > 1) + bpf_prog_change_xdp(prog, NULL); free_data: kfree(data); free_ctx:
We have a unit test that invokes an XDP program with 1m different inputs, aka 1m BPF_PROG_RUN syscalls. We run this test concurrently with slight variations in how we generated the input. Since commit f23c4b3924d2 ("bpf: Start using the BPF dispatcher in BPF_TEST_RUN") the unit test has slowed down significantly. Digging deeper reveals that the concurrent tests are serialised in the kernel on the XDP dispatcher. This is a global resource that is protected by a mutex, on which we contend. Fix this by not calling into the XDP dispatcher if we only want to perform a single run of the BPF program. See: https://lore.kernel.org/bpf/CACAyw9_y4QumOW35qpgTbLsJ532uGq-kVW-VESJzGyiZkypnvw@mail.gmail.com/ Signed-off-by: Lorenz Bauer <lmb@cloudflare.com> --- net/bpf/test_run.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)