[bpf-next,0/3] Reduce kmalloc / kfree churn in the verifier

Message

Lorenz Bauer April 29, 2021, 1:46 p.m. UTC

github.com/cilium/ebpf runs integration tests with libbpf in a vm on CI.
I recently did some work to increase the code coverage from that, and
started experiencing OOM-kills in the VM. That led me down a rabbit
hole looking at verifier memory allocation patterns. I didn't figure out
what triggered the OOM-kills but refactored some often called memory
allocation code.

The key insight is that often times we don't need to do a full kfree /
kmalloc, but can instead just reallocate. The first patch adds two helpers
which do just that for the use cases in the verifier, which are sufficiently
different that they can't use stock krealloc_array and friends.

The series makes bpf_verif_scale about 10% faster in my VM set up, which
is especially noticeable when running with KASAN enabled.

Lorenz Bauer (3):
  bpf: verifier: improve function state reallocation
  bpf: verifier: use copy_array for jmp_history
  bpf: verifier: allocate idmap scratch in verifier env

 include/linux/bpf_verifier.h |   8 ++
 kernel/bpf/verifier.c        | 254 +++++++++++++++++------------------
 2 files changed, 128 insertions(+), 134 deletions(-)

Comments

Alexei Starovoitov May 11, 2021, 12:55 a.m. UTC | #1

On Thu, Apr 29, 2021 at 6:47 AM Lorenz Bauer <[e-mail]> wrote:
>
> github.com/cilium/ebpf runs integration tests with libbpf in a vm on CI.
> I recently did some work to increase the code coverage from that, and
> started experiencing OOM-kills in the VM. That led me down a rabbit
> hole looking at verifier memory allocation patterns. I didn't figure out
> what triggered the OOM-kills but refactored some often called memory
> allocation code.
>
> The key insight is that often times we don't need to do a full kfree /
> kmalloc, but can instead just reallocate. The first patch adds two helpers
> which do just that for the use cases in the verifier, which are sufficiently
> different that they can't use stock krealloc_array and friends.
>
> The series makes bpf_verif_scale about 10% faster in my VM set up, which
> is especially noticeable when running with KASAN enabled.

The series looks great. Applied to bpf-next.