Message ID | 20250318143318.656785-9-aspsk@isovalent.com (mailing list archive) |
---|---|
State | RFC |
Delegated to: | BPF |
Headers | show |
Series | instruction sets and static keys | expand |
Context | Check | Description |
---|---|---|
bpf/vmtest-bpf-next-VM_Test-0 | success | Logs for Lint |
bpf/vmtest-bpf-next-VM_Test-1 | success | Logs for ShellCheck |
bpf/vmtest-bpf-next-VM_Test-2 | success | Logs for Unittests |
bpf/vmtest-bpf-next-VM_Test-3 | success | Logs for Validate matrix.py |
bpf/vmtest-bpf-next-PR | fail | PR summary |
bpf/vmtest-bpf-next-VM_Test-6 | success | Logs for aarch64-gcc / build-release |
bpf/vmtest-bpf-next-VM_Test-7 | success | Logs for aarch64-gcc / veristat-kernel |
bpf/vmtest-bpf-next-VM_Test-8 | success | Logs for aarch64-gcc / veristat-meta |
bpf/vmtest-bpf-next-VM_Test-9 | success | Logs for s390x-gcc / GCC BPF |
bpf/vmtest-bpf-next-VM_Test-10 | fail | Logs for s390x-gcc / build / build for s390x with gcc |
bpf/vmtest-bpf-next-VM_Test-11 | success | Logs for s390x-gcc / build-release |
bpf/vmtest-bpf-next-VM_Test-12 | success | Logs for s390x-gcc / veristat-kernel |
bpf/vmtest-bpf-next-VM_Test-4 | success | Logs for aarch64-gcc / GCC BPF |
bpf/vmtest-bpf-next-VM_Test-13 | success | Logs for s390x-gcc / veristat-meta |
bpf/vmtest-bpf-next-VM_Test-5 | fail | Logs for aarch64-gcc / build / build for aarch64 with gcc |
bpf/vmtest-bpf-next-VM_Test-14 | success | Logs for set-matrix |
bpf/vmtest-bpf-next-VM_Test-15 | success | Logs for test |
bpf/vmtest-bpf-next-VM_Test-16 | success | Logs for test |
bpf/vmtest-bpf-next-VM_Test-17 | success | Logs for test |
bpf/vmtest-bpf-next-VM_Test-18 | success | Logs for test |
bpf/vmtest-bpf-next-VM_Test-19 | success | Logs for test |
bpf/vmtest-bpf-next-VM_Test-20 | success | Logs for x86_64-gcc / GCC BPF |
bpf/vmtest-bpf-next-VM_Test-21 | fail | Logs for x86_64-gcc / build / build for x86_64 with gcc |
bpf/vmtest-bpf-next-VM_Test-22 | success | Logs for x86_64-gcc / build-release |
bpf/vmtest-bpf-next-VM_Test-23 | success | Logs for x86_64-gcc / veristat-kernel |
bpf/vmtest-bpf-next-VM_Test-24 | success | Logs for x86_64-gcc / veristat-meta |
bpf/vmtest-bpf-next-VM_Test-25 | success | Logs for x86_64-llvm-17 / GCC BPF |
bpf/vmtest-bpf-next-VM_Test-26 | fail | Logs for x86_64-llvm-17 / build / build for x86_64 with llvm-17 |
bpf/vmtest-bpf-next-VM_Test-27 | fail | Logs for x86_64-llvm-17 / build-release / build for x86_64 with llvm-17-O2 |
bpf/vmtest-bpf-next-VM_Test-28 | success | Logs for x86_64-llvm-17 / veristat-kernel |
bpf/vmtest-bpf-next-VM_Test-29 | success | Logs for x86_64-llvm-17 / veristat-meta |
bpf/vmtest-bpf-next-VM_Test-30 | success | Logs for x86_64-llvm-18 / GCC BPF |
bpf/vmtest-bpf-next-VM_Test-31 | fail | Logs for x86_64-llvm-18 / build / build for x86_64 with llvm-18 |
bpf/vmtest-bpf-next-VM_Test-32 | fail | Logs for x86_64-llvm-18 / build-release / build for x86_64 with llvm-18-O2 |
bpf/vmtest-bpf-next-VM_Test-33 | success | Logs for x86_64-llvm-18 / veristat-kernel |
bpf/vmtest-bpf-next-VM_Test-34 | success | Logs for x86_64-llvm-18 / veristat-meta |
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 5856ac1aab80..31cadded820b 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -1569,6 +1569,8 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image const s32 imm32 = insn->imm; u32 dst_reg = insn->dst_reg; u32 src_reg = insn->src_reg; + int adjust_off = 0; + int abs_xlated_off; u8 b2 = 0, b3 = 0; u8 *start_of_ldx; s64 jmp_offset; @@ -1724,6 +1726,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image emit_mov_imm64(&prog, dst_reg, insn[1].imm, insn[0].imm); insn++; i++; + adjust_off = 1; break; /* dst %= src, dst /= src, dst %= imm32, dst /= imm32 */ @@ -2595,6 +2598,14 @@ st: if (is_imm8(insn->off)) return -EFAULT; } memcpy(rw_image + proglen, temp, ilen); + + /* + * Static keys need to know how the xlated code + * of static ja instructions maps to jited code + */ + abs_xlated_off = bpf_prog->aux->subprog_start + i - 1 - adjust_off; + bpf_prog_update_insn_ptr(bpf_prog, abs_xlated_off, proglen, ilen, + jmp_offset, image + proglen); } proglen += ilen; addrs[i] = proglen; @@ -3880,3 +3891,38 @@ bool bpf_jit_supports_timed_may_goto(void) { return true; } + +int bpf_arch_poke_static_branch(struct bpf_insn_ptr *ptr, bool on) +{ + int jmp_offset = ptr->jitted_jump_offset; + void *ip = ptr->jitted_ip; + u32 len = ptr->jitted_len; + u8 op[5]; + + if (WARN_ON_ONCE(!ip)) + return -EINVAL; + + if (WARN_ON_ONCE(is_imm8(jmp_offset) && len != 2)) + return -EINVAL; + + if (WARN_ON_ONCE(!is_imm8(jmp_offset) && len != 5)) + return -EINVAL; + + if (on) { + if (len == 2) { + op[0] = 0xEB; + op[1] = jmp_offset; + } else { + op[0] = 0xE9; + memcpy(&op[1], &jmp_offset, 4); + } + } else { + memcpy(op, x86_nops[len], len); + } + + mutex_lock(&text_mutex); + text_poke_bp(ip, op, len, NULL); + mutex_unlock(&text_mutex); + + return 0; +}
Implement bpf_arch_poke_static_branch() for x86. Namely, during each JIT loop, save IP values and sizes of jump instructions pointed by static keys. Then use the text_poke_bp() to toggle jumps/nops. Signed-off-by: Anton Protopopov <aspsk@isovalent.com> --- arch/x86/net/bpf_jit_comp.c | 46 +++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+)