From patchwork Tue Feb 8 19:13:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 12739252 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E83CC433F5 for ; Tue, 8 Feb 2022 19:13:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385633AbiBHTNO (ORCPT ); Tue, 8 Feb 2022 14:13:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58158 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1385605AbiBHTNM (ORCPT ); Tue, 8 Feb 2022 14:13:12 -0500 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B94A9C0612C1 for ; Tue, 8 Feb 2022 11:13:11 -0800 (PST) Received: by mail-pj1-x1032.google.com with SMTP id my12-20020a17090b4c8c00b001b528ba1cd7so3889246pjb.1 for ; Tue, 08 Feb 2022 11:13:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KaEbUFRJyIgT3B5jjuOV+HjcH9SCqSe/5cpM0kVQMrA=; b=B2BbQ6dADYysJydxqi1pXzQGaW49DV6QcF7hshP4ss5OkaHQMvJkhrpjhePbR7SGQb BVRNqpjWNZMguQj8JlxP2a+TmZxOIpBQzLmjLO1MwfvN1yG7U2YY6+sgrIq4IxUKeWI7 h5KWNdYs+6uviIx+ayD3l9IwTvkFFcK8WVM54npxKvp/DVUbyU0CCCQnleBNIFjk/K6G CXXxR9ymnL5afGoIc6C3WcMLRrfL7I3s4B547u8JANjNQwiaKzSCJ8ploK54WO9s9HyY ixHPoYm45Y3tjcKo7esBv62qTPuqmdWmamvCIQMNS5zfVr3gL7ONnvSrjgTgEzvn4CUU aijA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KaEbUFRJyIgT3B5jjuOV+HjcH9SCqSe/5cpM0kVQMrA=; b=kSFF756GJFZ8RxOYQB+hC3GEeHXD5SHw37OwuDUhJeMwTm93YViu6ku8Do9C/iOXTg zosILiGG5QuHXb3JFS5LNywa1OC+0SXdSPZCMGiUvCpcjxGl2M5qe9ZhilMUhONsulrT m4dKtV+1JNP4N+6SRWB99K+jt2rIpdO3bV6V3nhnF4BIe7wc0erVfuGF3Ocy6Zao6AfC KIZAquFpB9RWIvthFrWwQRf85HtuYoNd3EETCD6v+J31kDpRVOyC9nFyAvUmcx3yAT0n nL30mZ6IciDP+jhOKSD/A3wK+DGEAyRnph45LQzTPU4aXPk3BSyrvuFQRJngk8dNmq33 FsZg== X-Gm-Message-State: AOAM530aMhay7+sR4BoYNEIO8ncX+h2VqAB0Nk2oy825QFyvrCCRTPw4 xED7v3Cw2EUgUqTW1Mi8I+E= X-Google-Smtp-Source: ABdhPJzIcCAtMR77lvPc58feKCnTFdkf5oxpwU0NVT7yXezcshwcipIthq/n6OP2TUA4n2uaJnKKPA== X-Received: by 2002:a17:902:6903:: with SMTP id j3mr5866700plk.23.1644347591180; Tue, 08 Feb 2022 11:13:11 -0800 (PST) Received: from ast-mbp.thefacebook.com ([2620:10d:c090:500::2:72b2]) by smtp.gmail.com with ESMTPSA id gx10sm3508459pjb.7.2022.02.08.11.13.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 11:13:10 -0800 (PST) From: Alexei Starovoitov To: davem@davemloft.net Cc: daniel@iogearbox.net, andrii@kernel.org, bpf@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 bpf-next 1/5] bpf: Extend sys_bpf commands for bpf_syscall programs. Date: Tue, 8 Feb 2022 11:13:02 -0800 Message-Id: <20220208191306.6136-2-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208191306.6136-1-alexei.starovoitov@gmail.com> References: <20220208191306.6136-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Alexei Starovoitov bpf_sycall programs can be used directly by the kernel modules to load programs and create maps via kernel skeleton. . Export bpf_sys_bpf syscall wrapper to be used in kernel skeleton. . Export bpf_map_get to be used in kernel skeleton. . Allow prog_run cmd for bpf_syscall programs with recursion check. . Enable link_create and raw_tp_open cmds. Acked-by: Yonghong Song Signed-off-by: Alexei Starovoitov --- kernel/bpf/syscall.c | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 72ce1edde950..49f88b30662a 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -985,6 +985,7 @@ struct bpf_map *bpf_map_get(u32 ufd) return map; } +EXPORT_SYMBOL(bpf_map_get); struct bpf_map *bpf_map_get_with_uref(u32 ufd) { @@ -4756,23 +4757,52 @@ static bool syscall_prog_is_valid_access(int off, int size, return true; } -BPF_CALL_3(bpf_sys_bpf, int, cmd, void *, attr, u32, attr_size) +BPF_CALL_3(bpf_sys_bpf, int, cmd, union bpf_attr *, attr, u32, attr_size) { + struct bpf_prog * __maybe_unused prog; + switch (cmd) { case BPF_MAP_CREATE: case BPF_MAP_UPDATE_ELEM: case BPF_MAP_FREEZE: case BPF_PROG_LOAD: case BPF_BTF_LOAD: + case BPF_LINK_CREATE: + case BPF_RAW_TRACEPOINT_OPEN: break; - /* case BPF_PROG_TEST_RUN: - * is not part of this list to prevent recursive test_run - */ +#ifdef CONFIG_BPF_JIT /* __bpf_prog_enter_sleepable used by trampoline and JIT */ + case BPF_PROG_TEST_RUN: + if (attr->test.data_in || attr->test.data_out || + attr->test.ctx_out || attr->test.duration || + attr->test.repeat || attr->test.flags) + return -EINVAL; + + prog = bpf_prog_get_type(attr->test.prog_fd, BPF_PROG_TYPE_SYSCALL); + if (IS_ERR(prog)) + return PTR_ERR(prog); + + if (attr->test.ctx_size_in < prog->aux->max_ctx_offset || + attr->test.ctx_size_in > U16_MAX) { + bpf_prog_put(prog); + return -EINVAL; + } + + if (!__bpf_prog_enter_sleepable(prog)) { + /* recursion detected */ + bpf_prog_put(prog); + return -EBUSY; + } + attr->test.retval = bpf_prog_run(prog, (void *) (long) attr->test.ctx_in); + __bpf_prog_exit_sleepable(prog, 0 /* bpf_prog_run does runtime stats */); + bpf_prog_put(prog); + return 0; +#endif default: return -EINVAL; } return __sys_bpf(cmd, KERNEL_BPFPTR(attr), attr_size); } +EXPORT_SYMBOL(bpf_sys_bpf); static const struct bpf_func_proto bpf_sys_bpf_proto = { .func = bpf_sys_bpf, From patchwork Tue Feb 8 19:13:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 12739253 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28FD1C433EF for ; Tue, 8 Feb 2022 19:13:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385883AbiBHTNQ (ORCPT ); Tue, 8 Feb 2022 14:13:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1385876AbiBHTNP (ORCPT ); Tue, 8 Feb 2022 14:13:15 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1ADEC0612C0 for ; Tue, 8 Feb 2022 11:13:14 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id my12-20020a17090b4c8c00b001b528ba1cd7so3889391pjb.1 for ; Tue, 08 Feb 2022 11:13:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SHhi0nf9DwDJT6jlxRE82NHhaQvbXywIwRW67/6Cw64=; b=A3isYZUmhdEOOBMHNeGJqXEEoRCjACjFwq8puowHpKlGRzzxM8vutM8+wUGoGjB83Q XBsbraUTc4ySKe4h3sJ9j7/zjyVAy30ylE6rrUUvj/j0TmOQGT+aWa2FO+x4XdqcGFJ/ 1CgDLKs7vBbxqpGELvohQXkAR095/ZeAH7vbu9RtVV7KtvwXbDdMDNEq3zdKuxEuSxeB SJdwex6yVQe8iHJVn/Nb8GNFkPx1m4m8UC0NNDUFyNNHs4hACQqWg6TRywTWVPxskk80 HlXF4wH63bl9H9G+vGgTT864DZUKW8McHh2KXgKfr9spTxO5rN1gUrSwC9vnjW/NvTyM 453w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SHhi0nf9DwDJT6jlxRE82NHhaQvbXywIwRW67/6Cw64=; b=7hfvwFbPQ6SMHbumZv6+j3ZLyNffSfovjDAj3/KYgpaejWXzltiiwUB9XKAa0DBZlt ok47f5OvagZDwg7+iqPEikfImdjD5vnSghguCP750pB/Qk01UUaxJVfSFlQv9i6lmwvb RFWmAHloSFw8184ztwhAlb92wqKg5AxfAp7P5i4PH1WGCkKMe1fXYCx0rfewIgFCym6H 5VmLaq5j6dqGY8pOBD4ssqOAx5872F+jaJqQbcccXdF2YBig2oh4og0FCRWvxvhHO4Hg Nc92n0LO86QzXKdF5ovkLTjLz9ZelgKlzaxvnYaY/XT2IPsaq5X1qWiYywpZxs3m5g+0 Erew== X-Gm-Message-State: AOAM530dc08d24fgPxuxRqEj/GH0jR3aSRIESOib1fFe0NfBs9jgmc27 gByHoGgDjQPGDNP+7AcLafA= X-Google-Smtp-Source: ABdhPJzWv7rXFIyX69Qw5OnR8talXT3PwF0eq25UszwJcHILIT10CO+hi0UUjEASAPg2YoBsmRubww== X-Received: by 2002:a17:903:234c:: with SMTP id c12mr5974786plh.55.1644347594062; Tue, 08 Feb 2022 11:13:14 -0800 (PST) Received: from ast-mbp.thefacebook.com ([2620:10d:c090:500::2:72b2]) by smtp.gmail.com with ESMTPSA id f8sm17044697pfe.204.2022.02.08.11.13.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 11:13:13 -0800 (PST) From: Alexei Starovoitov To: davem@davemloft.net Cc: daniel@iogearbox.net, andrii@kernel.org, bpf@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 bpf-next 2/5] libbpf: Prepare light skeleton for the kernel. Date: Tue, 8 Feb 2022 11:13:03 -0800 Message-Id: <20220208191306.6136-3-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208191306.6136-1-alexei.starovoitov@gmail.com> References: <20220208191306.6136-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Alexei Starovoitov Prepare light skeleton to be used in the kernel module and in the user space. The look and feel of lskel.h is mostly the same with the difference that for user space the skel->rodata is the same pointer before and after skel_load operation, while in the kernel the skel->rodata after skel_open and the skel->rodata after skel_load are different pointers. Typical usage of skeleton remains the same for kernel and user space: skel = my_bpf__open(); skel->rodata->my_global_var = init_val; err = my_bpf__load(skel); err = my_bpf__attach(skel); // access skel->rodata->my_global_var; // access skel->bss->another_var; Signed-off-by: Alexei Starovoitov Acked-by: Yonghong Song --- tools/lib/bpf/skel_internal.h | 193 +++++++++++++++++++++++++++++++--- 1 file changed, 176 insertions(+), 17 deletions(-) diff --git a/tools/lib/bpf/skel_internal.h b/tools/lib/bpf/skel_internal.h index dcd3336512d4..d16544666341 100644 --- a/tools/lib/bpf/skel_internal.h +++ b/tools/lib/bpf/skel_internal.h @@ -3,9 +3,19 @@ #ifndef __SKEL_INTERNAL_H #define __SKEL_INTERNAL_H +#ifdef __KERNEL__ +#include +#include +#include +#include +#include +#else #include #include #include +#include +#include "bpf.h" +#endif #ifndef __NR_bpf # if defined(__mips__) && defined(_ABIO32) @@ -25,17 +35,11 @@ * requested during loader program generation. */ struct bpf_map_desc { - union { - /* input for the loader prog */ - struct { - __aligned_u64 initial_value; - __u32 max_entries; - }; - /* output of the loader prog */ - struct { - int map_fd; - }; - }; + /* output of the loader prog */ + int map_fd; + /* input for the loader prog */ + __u32 max_entries; + __aligned_u64 initial_value; }; struct bpf_prog_desc { int prog_fd; @@ -57,12 +61,159 @@ struct bpf_load_and_run_opts { const char *errstr; }; +long bpf_sys_bpf(__u32 cmd, void *attr, __u32 attr_size); + static inline int skel_sys_bpf(enum bpf_cmd cmd, union bpf_attr *attr, unsigned int size) { +#ifdef __KERNEL__ + return bpf_sys_bpf(cmd, attr, size); +#else return syscall(__NR_bpf, cmd, attr, size); +#endif +} + +#ifdef __KERNEL__ +static inline int close(int fd) +{ + return close_fd(fd); +} + +static inline void *skel_alloc(size_t size) +{ + return kcalloc(1, size, GFP_KERNEL); +} + +static inline void skel_free(const void *p) +{ + kfree(p); +} + +/* skel->bss/rodata maps are populated in three steps. + * + * For kernel use: + * skel_prep_map_data() allocates kernel memory that kernel module can directly access. + * skel_prep_init_value() allocates a region in user space process and copies + * potentially modified initial map value into it. + * The loader program will perform copy_from_user() from maps.rodata.initial_value. + * skel_finalize_map_data() sets skel->rodata to point to actual value in a bpf map and + * does maps.rodata.initial_value = ~0ULL to signal skel_free_map_data() that kvfree + * is not nessary. + * + * For user space: + * skel_prep_map_data() mmaps anon memory into skel->rodata that can be accessed directly. + * skel_prep_init_value() copies rodata pointer into map.rodata.initial_value. + * The loader program will perform copy_from_user() from maps.rodata.initial_value. + * skel_finalize_map_data() remaps bpf array map value from the kernel memory into + * skel->rodata address. + * + * The "bpftool gen skeleton -L" command generates lskel.h that is suitable for + * both kernel and user space. The generated loader program does + * copy_from_user() from intial_value. Therefore the vm_mmap+copy_to_user step + * is need when lskel is used from the kernel module. + */ +static inline void skel_free_map_data(void *p, __u64 addr, size_t sz) +{ + if (addr && addr != ~0ULL) + vm_munmap(addr, sz); + if (addr != ~0ULL) + kvfree(p); + /* When addr == ~0ULL the 'p' points to + * ((struct bpf_array *)map)->value. See skel_finalize_map_data. + */ +} + +static inline void *skel_prep_map_data(const void *val, size_t mmap_sz, size_t val_sz) +{ + void *addr; + + addr = kvmalloc(val_sz, GFP_KERNEL); + if (!addr) + return NULL; + memcpy(addr, val, val_sz); + return addr; +} + +static inline __u64 skel_prep_init_value(void **addr, size_t mmap_sz, size_t val_sz) +{ + __u64 ret = 0; + void *uaddr; + + uaddr = (void *) vm_mmap(NULL, 0, mmap_sz, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_ANONYMOUS, 0); + if (IS_ERR(uaddr)) + goto out; + if (copy_to_user(uaddr, *addr, val_sz)) { + vm_munmap((long) uaddr, mmap_sz); + goto out; + } + ret = (__u64) (long) uaddr; +out: + kvfree(*addr); + *addr = NULL; + return ret; } +static inline void *skel_finalize_map_data(__u64 *addr, size_t mmap_sz, int flags, int fd) +{ + struct bpf_map *map; + void *ptr = NULL; + + vm_munmap(*addr, mmap_sz); + *addr = ~0ULL; + + map = bpf_map_get(fd); + if (IS_ERR(map)) + return NULL; + if (map->map_type != BPF_MAP_TYPE_ARRAY) + goto out; + ptr = ((struct bpf_array *)map)->value; + /* the ptr stays valid, since FD is not closed */ +out: + bpf_map_put(map); + return ptr; +} + +#else + +static inline void *skel_alloc(size_t size) +{ + return calloc(1, size); +} + +static inline void skel_free(void *p) +{ + free(p); +} + +static inline void skel_free_map_data(void *p, __u64 addr, size_t sz) +{ + munmap(p, sz); +} + +static inline void *skel_prep_map_data(const void *val, size_t mmap_sz, size_t val_sz) +{ + void *addr; + + addr = mmap(NULL, mmap_sz, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_ANONYMOUS, -1, 0); + if (addr == (void *) -1) + return NULL; + memcpy(addr, val, val_sz); + return addr; +} + +static inline __u64 skel_prep_init_value(void **addr, size_t mmap_sz, size_t val_sz) +{ + return (__u64) (long) *addr; +} + +static inline void *skel_finalize_map_data(__u64 *addr, size_t mmap_sz, int flags, int fd) +{ + return mmap((void *)*addr, mmap_sz, flags, MAP_SHARED | MAP_FIXED, fd, 0); +} +#endif + static inline int skel_closenz(int fd) { if (fd > 0) @@ -136,22 +287,28 @@ static inline int skel_link_create(int prog_fd, int target_fd, return skel_sys_bpf(BPF_LINK_CREATE, &attr, attr_sz); } +#ifdef __KERNEL__ +#define set_err +#else +#define set_err err = -errno +#endif + static inline int bpf_load_and_run(struct bpf_load_and_run_opts *opts) { int map_fd = -1, prog_fd = -1, key = 0, err; union bpf_attr attr; - map_fd = skel_map_create(BPF_MAP_TYPE_ARRAY, "__loader.map", 4, opts->data_sz, 1); + err = map_fd = skel_map_create(BPF_MAP_TYPE_ARRAY, "__loader.map", 4, opts->data_sz, 1); if (map_fd < 0) { opts->errstr = "failed to create loader map"; - err = -errno; + set_err; goto out; } err = skel_map_update_elem(map_fd, &key, opts->data, 0); if (err < 0) { opts->errstr = "failed to update loader map"; - err = -errno; + set_err; goto out; } @@ -166,10 +323,10 @@ static inline int bpf_load_and_run(struct bpf_load_and_run_opts *opts) attr.log_size = opts->ctx->log_size; attr.log_buf = opts->ctx->log_buf; attr.prog_flags = BPF_F_SLEEPABLE; - prog_fd = skel_sys_bpf(BPF_PROG_LOAD, &attr, sizeof(attr)); + err = prog_fd = skel_sys_bpf(BPF_PROG_LOAD, &attr, sizeof(attr)); if (prog_fd < 0) { opts->errstr = "failed to load loader prog"; - err = -errno; + set_err; goto out; } @@ -181,10 +338,12 @@ static inline int bpf_load_and_run(struct bpf_load_and_run_opts *opts) if (err < 0 || (int)attr.test.retval < 0) { opts->errstr = "failed to execute loader prog"; if (err < 0) { - err = -errno; + set_err; } else { err = (int)attr.test.retval; +#ifndef __KERNEL__ errno = -err; +#endif } goto out; } From patchwork Tue Feb 8 19:13:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 12739254 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82D55C433F5 for ; Tue, 8 Feb 2022 19:13:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385896AbiBHTNW (ORCPT ); Tue, 8 Feb 2022 14:13:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58206 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1385647AbiBHTNU (ORCPT ); Tue, 8 Feb 2022 14:13:20 -0500 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 907DDC0613CB for ; Tue, 8 Feb 2022 11:13:17 -0800 (PST) Received: by mail-pf1-x42b.google.com with SMTP id z35so177450pfw.2 for ; Tue, 08 Feb 2022 11:13:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Imcqa94K6qmkHgNev/2e7jXS6AGjkc6K2IMNqaaw6eM=; b=WhHwMVMwPjza6VuLYbwUUsPEb4XQQ2WA+VXiZikUEunjhLv9SFXwdNLCZQs1f1ia8b gZYJVFro9fErOHZHD2yA/kRL37mjVjyNwIlxVVI2T8QYwhqHBrB0dbk0e8Gr0OjW31Xo UejDyHpEvVOo1TWDczUYTxydoISwxi5e2CAjCdjnCSWZJgGOne/MGW/njeaBPlSHk99j K3qVcdoCGN/gZnyjb/eMSHiFL5s5PF5LyXvm/qdOwu+qDtZhpP4E2Nn/O7c+gkzcmPSD LI9jiQkSSOJGvu/o31AnScvlEAyNUNDR+TuYgNAcC3Oa+CxYSzyOmXMhpct8p0uSe7xh IZBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Imcqa94K6qmkHgNev/2e7jXS6AGjkc6K2IMNqaaw6eM=; b=Cdiz2LnYvyQs7JmHn6w5NqXW9nh4aHcd2KqhmLN48cuqQ7LpHrgGKEEYMT+T7nkhoL gVBi/BI2aH8lwm4rxCl0fWKwB+GcOEzVcsBo+l8L1reUEjvZRry3C7/nkGwvArBc5evT lsLEahNkRwpNNNiq0LP+i31yr74Bkxjn5iMOfM9dY6Ss4BdgRwJsseZm9ISZ9GpHSMr8 3sTX6kFg7QMPRyFCRmSfmJL078XUfhxC5fGSVitxBWIl39oDfHxzeK5DwacnOxXs0Ru/ AzyDzsDQR1V0faM3ik/UyGvkYU525pE7ST4Yj+ZkrHh/UCuCxcQdFkrlSJQt2Ls66Fwd tNkw== X-Gm-Message-State: AOAM530uOkWma36hmBgOwVxmATE9RmOOu9xlVwpzALsydt+EytUzrwBj Y+Pltj+/DtP7kv/yCR/DIVCpPNTCmmA= X-Google-Smtp-Source: ABdhPJx5XUfrYs+GJppXTzij6c1gDSVE0DLisYhgozzfIiX13yFcZI/pxzfsf5jLKbzxQz3R5wea3Q== X-Received: by 2002:a05:6a00:8c5:: with SMTP id s5mr5884723pfu.34.1644347596986; Tue, 08 Feb 2022 11:13:16 -0800 (PST) Received: from ast-mbp.thefacebook.com ([2620:10d:c090:500::2:72b2]) by smtp.gmail.com with ESMTPSA id j8sm3541080pjc.11.2022.02.08.11.13.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 11:13:16 -0800 (PST) From: Alexei Starovoitov To: davem@davemloft.net Cc: daniel@iogearbox.net, andrii@kernel.org, bpf@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 bpf-next 3/5] bpftool: Generalize light skeleton generation. Date: Tue, 8 Feb 2022 11:13:04 -0800 Message-Id: <20220208191306.6136-4-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208191306.6136-1-alexei.starovoitov@gmail.com> References: <20220208191306.6136-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Alexei Starovoitov Generealize light skeleton by hiding mmap details in skel_internal.h In this form generated lskel.h is usable both by user space and by the kernel. Signed-off-by: Alexei Starovoitov Acked-by: Yonghong Song --- tools/bpf/bpftool/gen.c | 45 ++++++++++++++++++++++++----------------- 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c index eacfc6a2060d..903abbf077ce 100644 --- a/tools/bpf/bpftool/gen.c +++ b/tools/bpf/bpftool/gen.c @@ -472,7 +472,7 @@ static void codegen_destroy(struct bpf_object *obj, const char *obj_name) continue; if (bpf_map__is_internal(map) && (bpf_map__map_flags(map) & BPF_F_MMAPABLE)) - printf("\tmunmap(skel->%1$s, %2$zd);\n", + printf("\tskel_free_map_data(skel->%1$s, skel->maps.%1$s.initial_value, %2$zd);\n", ident, bpf_map_mmap_sz(map)); codegen("\ \n\ @@ -481,7 +481,7 @@ static void codegen_destroy(struct bpf_object *obj, const char *obj_name) } codegen("\ \n\ - free(skel); \n\ + skel_free(skel); \n\ } \n\ ", obj_name); @@ -525,7 +525,7 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h { \n\ struct %1$s *skel; \n\ \n\ - skel = calloc(sizeof(*skel), 1); \n\ + skel = skel_alloc(sizeof(*skel)); \n\ if (!skel) \n\ goto cleanup; \n\ skel->ctx.sz = (void *)&skel->links - (void *)skel; \n\ @@ -544,18 +544,12 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h codegen("\ \n\ - skel->%1$s = \n\ - mmap(NULL, %2$zd, PROT_READ | PROT_WRITE,\n\ - MAP_SHARED | MAP_ANONYMOUS, -1, 0); \n\ - if (skel->%1$s == (void *) -1) \n\ - goto cleanup; \n\ - memcpy(skel->%1$s, (void *)\"\\ \n\ - ", ident, bpf_map_mmap_sz(map)); + skel->%1$s = skel_prep_map_data((void *)\"\\ \n\ + ", ident); mmap_data = bpf_map__initial_value(map, &mmap_size); print_hex(mmap_data, mmap_size); - printf("\", %2$zd);\n" - "\tskel->maps.%1$s.initial_value = (__u64)(long)skel->%1$s;\n", - ident, mmap_size); + printf("\", %1$zd, %2$zd);\n", + bpf_map_mmap_sz(map), mmap_size); } codegen("\ \n\ @@ -592,6 +586,24 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h codegen("\ \n\ \"; \n\ + "); + bpf_object__for_each_map(map, obj) { + size_t mmap_size = 0; + + if (!get_map_ident(map, ident, sizeof(ident))) + continue; + + if (!bpf_map__is_internal(map) || + !(bpf_map__map_flags(map) & BPF_F_MMAPABLE)) + continue; + + bpf_map__initial_value(map, &mmap_size); + printf("\tskel->maps.%1$s.initial_value =" + " skel_prep_init_value((void **)&skel->%1$s, %2$zd, %3$zd);\n", + ident, bpf_map_mmap_sz(map), mmap_size); + } + codegen("\ + \n\ err = bpf_load_and_run(&opts); \n\ if (err < 0) \n\ return err; \n\ @@ -611,9 +623,8 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h else mmap_flags = "PROT_READ | PROT_WRITE"; - printf("\tskel->%1$s =\n" - "\t\tmmap(skel->%1$s, %2$zd, %3$s, MAP_SHARED | MAP_FIXED,\n" - "\t\t\tskel->maps.%1$s.map_fd, 0);\n", + printf("\tskel->%1$s = skel_finalize_map_data(&skel->maps.%1$s.initial_value,\n" + "\t\t\t%2$zd, %3$s, skel->maps.%1$s.map_fd);\n", ident, bpf_map_mmap_sz(map), mmap_flags); } codegen("\ @@ -751,8 +762,6 @@ static int do_skeleton(int argc, char **argv) #ifndef %2$s \n\ #define %2$s \n\ \n\ - #include \n\ - #include \n\ #include \n\ \n\ struct %1$s { \n\ From patchwork Tue Feb 8 19:13:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 12739255 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19CA2C433EF for ; Tue, 8 Feb 2022 19:13:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385647AbiBHTNY (ORCPT ); Tue, 8 Feb 2022 14:13:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58218 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1385892AbiBHTNU (ORCPT ); Tue, 8 Feb 2022 14:13:20 -0500 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4049AC0612C3 for ; Tue, 8 Feb 2022 11:13:20 -0800 (PST) Received: by mail-pj1-x102d.google.com with SMTP id on2so66390pjb.4 for ; Tue, 08 Feb 2022 11:13:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ENmm2RIFAUVug724BIM6PFSnweNoJNbJmLqC9jMYHHM=; b=mdm+DE9VTPEO6c/lf6AdDArz6A12vLDARiW4EDr7ozJCy1mZctrRZJcy7qbR9gwQQZ H7bcRekC3QyLxRM85kDeEz1JBmMpD95vbkCVxYsYg/lPZzM5fcjiMs6mJUqWGgONv8Fx ACXtVjVXJVlZa2emHrj9G+lk30uYpGrED4aWYFtcfDOH6ZOZSnZ1o8oHEGBZ6BzPUIVh aWcMXUriZcrLrBjgrhzjU0/8kVUN2i2QIdGBczrds33rUVxGB1MDhEhlZBnwzvy+WbAc 90HgI83HoVpiVR2qOPHZoqbuWFF9T4c/67ZKcdmWMRQTWDlhjWd3UchEBPN+FMmSFV0X Ts0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ENmm2RIFAUVug724BIM6PFSnweNoJNbJmLqC9jMYHHM=; b=b0/Xv27S/b3J3XIPOmvDABv6gL4ykEQphyl9cowmvr4WIhLOxgwaH86IY0DnZItgIt av3/A6I+Ku2Dm8AltiiUBzi9rWq16ho9Hi2tAmUXGwD0EGJzZFpejHvJ5HytBp6vGkyR jwbkHzBvOFxUCZnOgjGiDij0FLLUpg7Mt1FA87UrYPzy6Sl9iPQk/CfZRSL/XbdbCrFg zsBFJ4OmCKVUVQlM1wbnAcxkS7uy47jZbTkHnxpEHGfbnWm2EoYSWqZeECEzYurkseU3 SIY8W8ybpfTgM3P2yKgtdpb8gjZl8iLWZntRTfEiKdlIjImpyXd3J6IgOFLrpwwRfn0Z 1H/w== X-Gm-Message-State: AOAM533TzdrpXf0ZZBX7DG5JZ2sewjkQAKMK8CbgdXW4iBZAkpHXKfIg emJPUUZOb1AarPZPxtdRwlc= X-Google-Smtp-Source: ABdhPJxVjCBbiZGeUe/BGUYteSlEXY0WCLfzEmmSnorUeO1ouAow+QjYu+/T4PIk2cbWQQguDTycRA== X-Received: by 2002:a17:90b:4f46:: with SMTP id pj6mr2965975pjb.43.1644347599749; Tue, 08 Feb 2022 11:13:19 -0800 (PST) Received: from ast-mbp.thefacebook.com ([2620:10d:c090:500::2:72b2]) by smtp.gmail.com with ESMTPSA id f8sm17044822pfe.204.2022.02.08.11.13.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 11:13:19 -0800 (PST) From: Alexei Starovoitov To: davem@davemloft.net Cc: daniel@iogearbox.net, andrii@kernel.org, bpf@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 bpf-next 4/5] bpf: Update iterators.lskel.h. Date: Tue, 8 Feb 2022 11:13:05 -0800 Message-Id: <20220208191306.6136-5-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208191306.6136-1-alexei.starovoitov@gmail.com> References: <20220208191306.6136-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Alexei Starovoitov Light skeleton and skel_internal.h have changed. Update iterators.lskel.h. Signed-off-by: Alexei Starovoitov Acked-by: Yonghong Song --- .../bpf/preload/iterators/iterators.lskel.h | 28 +++++++------------ 1 file changed, 10 insertions(+), 18 deletions(-) diff --git a/kernel/bpf/preload/iterators/iterators.lskel.h b/kernel/bpf/preload/iterators/iterators.lskel.h index d90562d672d2..3e45237f59f4 100644 --- a/kernel/bpf/preload/iterators/iterators.lskel.h +++ b/kernel/bpf/preload/iterators/iterators.lskel.h @@ -3,8 +3,6 @@ #ifndef __ITERATORS_BPF_SKEL_H__ #define __ITERATORS_BPF_SKEL_H__ -#include -#include #include struct iterators_bpf { @@ -70,31 +68,25 @@ iterators_bpf__destroy(struct iterators_bpf *skel) iterators_bpf__detach(skel); skel_closenz(skel->progs.dump_bpf_map.prog_fd); skel_closenz(skel->progs.dump_bpf_prog.prog_fd); - munmap(skel->rodata, 4096); + skel_free_map_data(skel->rodata, skel->maps.rodata.initial_value, 4096); skel_closenz(skel->maps.rodata.map_fd); - free(skel); + skel_free(skel); } static inline struct iterators_bpf * iterators_bpf__open(void) { struct iterators_bpf *skel; - skel = calloc(sizeof(*skel), 1); + skel = skel_alloc(sizeof(*skel)); if (!skel) goto cleanup; skel->ctx.sz = (void *)&skel->links - (void *)skel; - skel->rodata = - mmap(NULL, 4096, PROT_READ | PROT_WRITE, - MAP_SHARED | MAP_ANONYMOUS, -1, 0); - if (skel->rodata == (void *) -1) - goto cleanup; - memcpy(skel->rodata, (void *)"\ + skel->rodata = skel_prep_map_data((void *)"\ \x20\x20\x69\x64\x20\x6e\x61\x6d\x65\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\ \x20\x20\x20\x6d\x61\x78\x5f\x65\x6e\x74\x72\x69\x65\x73\x0a\0\x25\x34\x75\x20\ \x25\x2d\x31\x36\x73\x25\x36\x64\x0a\0\x20\x20\x69\x64\x20\x6e\x61\x6d\x65\x20\ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x61\x74\x74\x61\x63\x68\x65\ -\x64\x0a\0\x25\x34\x75\x20\x25\x2d\x31\x36\x73\x20\x25\x73\x20\x25\x73\x0a\0", 98); - skel->maps.rodata.initial_value = (__u64)(long)skel->rodata; +\x64\x0a\0\x25\x34\x75\x20\x25\x2d\x31\x36\x73\x20\x25\x73\x20\x25\x73\x0a\0", 4096, 98); return skel; cleanup: iterators_bpf__destroy(skel); @@ -343,11 +335,11 @@ iterators_bpf__load(struct iterators_bpf *skel) \0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\x30\x0e\0\0\xb7\x03\0\0\x1c\0\0\0\x85\0\0\0\ \xa6\0\0\0\xbf\x07\0\0\0\0\0\0\xc5\x07\xd4\xff\0\0\0\0\x63\x7a\x78\xff\0\0\0\0\ \x61\xa0\x78\xff\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x80\x0e\0\0\x63\x01\0\0\0\ -\0\0\0\x61\x60\x20\0\0\0\0\0\x15\0\x03\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\ +\0\0\0\x61\x60\x1c\0\0\0\0\0\x15\0\x03\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\ \x5c\x0e\0\0\x63\x01\0\0\0\0\0\0\xb7\x01\0\0\0\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\ \0\x50\x0e\0\0\xb7\x03\0\0\x48\0\0\0\x85\0\0\0\xa6\0\0\0\xbf\x07\0\0\0\0\0\0\ \xc5\x07\xc3\xff\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x63\x71\0\0\0\0\0\ -\0\x79\x63\x18\0\0\0\0\0\x15\x03\x04\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x98\ +\0\x79\x63\x20\0\0\0\0\0\x15\x03\x04\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x98\ \x0e\0\0\xb7\x02\0\0\x62\0\0\0\x85\0\0\0\x94\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\ \0\0\0\0\x61\x20\0\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x08\x0f\0\0\x63\x01\0\ \0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\0\x0f\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\ @@ -401,12 +393,12 @@ iterators_bpf__load(struct iterators_bpf *skel) \x28\0\0\0\0\0\x61\xa0\x84\xff\0\0\0\0\x63\x06\x2c\0\0\0\0\0\x18\x61\0\0\0\0\0\ \0\0\0\0\0\0\0\0\0\x61\x10\0\0\0\0\0\0\x63\x06\x18\0\0\0\0\0\xb7\0\0\0\0\0\0\0\ \x95\0\0\0\0\0\0\0"; + skel->maps.rodata.initial_value = skel_prep_init_value((void **)&skel->rodata, 4096, 98); err = bpf_load_and_run(&opts); if (err < 0) return err; - skel->rodata = - mmap(skel->rodata, 4096, PROT_READ, MAP_SHARED | MAP_FIXED, - skel->maps.rodata.map_fd, 0); + skel->rodata = skel_finalize_map_data(&skel->maps.rodata.initial_value, + 4096, PROT_READ, skel->maps.rodata.map_fd); return 0; } From patchwork Tue Feb 8 19:13:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 12739256 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A511C433FE for ; Tue, 8 Feb 2022 19:13:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385895AbiBHTN0 (ORCPT ); Tue, 8 Feb 2022 14:13:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1385903AbiBHTNZ (ORCPT ); Tue, 8 Feb 2022 14:13:25 -0500 Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55C51C0612C3 for ; Tue, 8 Feb 2022 11:13:23 -0800 (PST) Received: by mail-pf1-x430.google.com with SMTP id e28so133826pfj.5 for ; Tue, 08 Feb 2022 11:13:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CPHEEDxKrsWPcFazq/e5fUcM5898tZQ9nd+iRZtSPlM=; b=kMvvcu+GoG35z6VwYdAGWF2li5YhsHuTaAbbZzwti/V4u6uJEW+YxbGwcUPrFFJjnw 6Us4bIY/EqfpHIq738OrKzrdDdk2P1bmF7nAX0CnvX4EoaL8nOOI7oY3cH3moC0c9e61 xg3fMEjnKrKRlyNtIXrbwvm5Gd6H6GJIrA1Ynb15l08w0yepuG59ypUQgJOTHQa3Z7BB OoSBjt34um0LGZ4VclsZL5b7YoFAHxUiudLmsEhEpyG03H2raVGEj+86j+mPdCjuT0Vp 7yrhAswM+CbEQ4k+qMpkI2t9szO5aD2Pg77QmC2O6OZRLmeBeHAxnBCeFr4snlkWJvo/ dVVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CPHEEDxKrsWPcFazq/e5fUcM5898tZQ9nd+iRZtSPlM=; b=kaM604bMEEZmymrMHpxqphogLMBC+hqqOSGCE6rz+oAOIOaS4BCh7BArjTvDGDyIh7 6Y25gVtjYzSQwutxsNgVruDi4IOn71sksVg5aJcVXFUKJXnVZlRlOqceyLDX25+daC39 X9WOZCO4hxP7X7KZqIX39OLpMVRKT1m7Vp1OiAm97MKvZ9zJBleuPw+MvgtUlru7Q5V3 xk/zuIgx0vDr5TUO4b+eGk+++slHCyfwzGU7yt3b3RaukwnS9aXZQeRP8SKLI8i0OSvO adyTDW1szMta9wQijaCGR6dRlLjP/rw24NTKT7fVxxEzA3/+UUCGl5Jo+kCroZoUwdx1 nIWQ== X-Gm-Message-State: AOAM532v4rurgSM5NwzeHveVUJA6Cr7Z7yfUQQ7a65ZDZRapdxwyY520 OKOFjMMq4XTk1qRzpj3CHsw= X-Google-Smtp-Source: ABdhPJwQOBbThVPjNXpsEPjNCX41SzeUsxO7OItsgnpbePAvPdkgcvNzyShStor2RlPyRV5tdxdhQA== X-Received: by 2002:a05:6a00:1914:: with SMTP id y20mr5561969pfi.41.1644347602677; Tue, 08 Feb 2022 11:13:22 -0800 (PST) Received: from ast-mbp.thefacebook.com ([2620:10d:c090:500::2:72b2]) by smtp.gmail.com with ESMTPSA id q1sm484918pfs.112.2022.02.08.11.13.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 11:13:22 -0800 (PST) From: Alexei Starovoitov To: davem@davemloft.net Cc: daniel@iogearbox.net, andrii@kernel.org, bpf@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 bpf-next 5/5] bpf: Convert bpf_preload.ko to use light skeleton. Date: Tue, 8 Feb 2022 11:13:06 -0800 Message-Id: <20220208191306.6136-6-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208191306.6136-1-alexei.starovoitov@gmail.com> References: <20220208191306.6136-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Alexei Starovoitov The main change is a move of the single line #include "iterators.lskel.h" from iterators/iterators.c to bpf_preload_kern.c. Which means that generated light skeleton can be used from user space or user mode driver like iterators.c or from the kernel module. The direct use of light skeleton from the kernel module simplifies the code, since UMD is no longer necessary. The libbpf.a required user space and UMD. The CO-RE in the kernel and generated "loader bpf program" used by the light skeleton are capable to perform complex loading operations traditionally provided by libbpf. In addition UMD approach was launching UMD process every time bpffs has to be mounted. With light skeleton in the kernel the bpf_preload kernel module loads bpf iterators once and pins them multiple times into different bpffs mounts. Note the light skeleton cannot be used during early boot or out of kthread since light skeleton needs a valid mm. This limitation could be lifted in the future. Signed-off-by: Alexei Starovoitov Acked-by: Yonghong Song --- kernel/bpf/inode.c | 39 ++---- kernel/bpf/preload/Kconfig | 9 +- kernel/bpf/preload/Makefile | 14 +-- kernel/bpf/preload/bpf_preload.h | 8 +- kernel/bpf/preload/bpf_preload_kern.c | 119 ++++++++---------- kernel/bpf/preload/bpf_preload_umd_blob.S | 7 -- .../preload/iterators/bpf_preload_common.h | 13 -- kernel/bpf/preload/iterators/iterators.c | 108 ---------------- kernel/bpf/syscall.c | 2 + 9 files changed, 72 insertions(+), 247 deletions(-) delete mode 100644 kernel/bpf/preload/bpf_preload_umd_blob.S delete mode 100644 kernel/bpf/preload/iterators/bpf_preload_common.h delete mode 100644 kernel/bpf/preload/iterators/iterators.c diff --git a/kernel/bpf/inode.c b/kernel/bpf/inode.c index 5a8d9f7467bf..4f841e16779e 100644 --- a/kernel/bpf/inode.c +++ b/kernel/bpf/inode.c @@ -710,11 +710,10 @@ static DEFINE_MUTEX(bpf_preload_lock); static int populate_bpffs(struct dentry *parent) { struct bpf_preload_info objs[BPF_PRELOAD_LINKS] = {}; - struct bpf_link *links[BPF_PRELOAD_LINKS] = {}; int err = 0, i; /* grab the mutex to make sure the kernel interactions with bpf_preload - * UMD are serialized + * are serialized */ mutex_lock(&bpf_preload_lock); @@ -722,40 +721,22 @@ static int populate_bpffs(struct dentry *parent) if (!bpf_preload_mod_get()) goto out; - if (!bpf_preload_ops->info.tgid) { - /* preload() will start UMD that will load BPF iterator programs */ - err = bpf_preload_ops->preload(objs); - if (err) + err = bpf_preload_ops->preload(objs); + if (err) + goto out_put; + for (i = 0; i < BPF_PRELOAD_LINKS; i++) { + bpf_link_inc(objs[i].link); + err = bpf_iter_link_pin_kernel(parent, + objs[i].link_name, objs[i].link); + if (err) { + bpf_link_put(objs[i].link); goto out_put; - for (i = 0; i < BPF_PRELOAD_LINKS; i++) { - links[i] = bpf_link_by_id(objs[i].link_id); - if (IS_ERR(links[i])) { - err = PTR_ERR(links[i]); - goto out_put; - } } - for (i = 0; i < BPF_PRELOAD_LINKS; i++) { - err = bpf_iter_link_pin_kernel(parent, - objs[i].link_name, links[i]); - if (err) - goto out_put; - /* do not unlink successfully pinned links even - * if later link fails to pin - */ - links[i] = NULL; - } - /* finish() will tell UMD process to exit */ - err = bpf_preload_ops->finish(); - if (err) - goto out_put; } out_put: bpf_preload_mod_put(); out: mutex_unlock(&bpf_preload_lock); - for (i = 0; i < BPF_PRELOAD_LINKS && err; i++) - if (!IS_ERR_OR_NULL(links[i])) - bpf_link_put(links[i]); return err; } diff --git a/kernel/bpf/preload/Kconfig b/kernel/bpf/preload/Kconfig index 26bced262473..9de6cfa5dbb1 100644 --- a/kernel/bpf/preload/Kconfig +++ b/kernel/bpf/preload/Kconfig @@ -18,10 +18,11 @@ menuconfig BPF_PRELOAD if BPF_PRELOAD config BPF_PRELOAD_UMD - tristate "bpf_preload kernel module with user mode driver" - depends on CC_CAN_LINK - depends on m || CC_CAN_LINK_STATIC + tristate "bpf_preload kernel module" + # light skeleton cannot run out of kthread without mm + depends on m default m help - This builds bpf_preload kernel module with embedded user mode driver. + This builds bpf_preload kernel module with embedded BPF programs for + introspection in bpffs. endif diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile index baf47d9c7557..167534e3b0b4 100644 --- a/kernel/bpf/preload/Makefile +++ b/kernel/bpf/preload/Makefile @@ -3,16 +3,6 @@ LIBBPF_SRCS = $(srctree)/tools/lib/bpf/ LIBBPF_INCLUDE = $(LIBBPF_SRCS)/.. -userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \ - -I $(LIBBPF_INCLUDE) -Wno-unused-result - -userprogs := bpf_preload_umd - -bpf_preload_umd-objs := iterators/iterators.o - -$(obj)/bpf_preload_umd: - -$(obj)/bpf_preload_umd_blob.o: $(obj)/bpf_preload_umd - obj-$(CONFIG_BPF_PRELOAD_UMD) += bpf_preload.o -bpf_preload-objs += bpf_preload_kern.o bpf_preload_umd_blob.o +CFLAGS_bpf_preload_kern.o += -I $(LIBBPF_INCLUDE) +bpf_preload-objs += bpf_preload_kern.o diff --git a/kernel/bpf/preload/bpf_preload.h b/kernel/bpf/preload/bpf_preload.h index 2f9932276f2e..f065c91213a0 100644 --- a/kernel/bpf/preload/bpf_preload.h +++ b/kernel/bpf/preload/bpf_preload.h @@ -2,13 +2,13 @@ #ifndef _BPF_PRELOAD_H #define _BPF_PRELOAD_H -#include -#include "iterators/bpf_preload_common.h" +struct bpf_preload_info { + char link_name[16]; + struct bpf_link *link; +}; struct bpf_preload_ops { - struct umd_info info; int (*preload)(struct bpf_preload_info *); - int (*finish)(void); struct module *owner; }; extern struct bpf_preload_ops *bpf_preload_ops; diff --git a/kernel/bpf/preload/bpf_preload_kern.c b/kernel/bpf/preload/bpf_preload_kern.c index 53736e52c1df..30207c048d36 100644 --- a/kernel/bpf/preload/bpf_preload_kern.c +++ b/kernel/bpf/preload/bpf_preload_kern.c @@ -2,101 +2,80 @@ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt #include #include -#include -#include -#include #include "bpf_preload.h" +#include "iterators/iterators.lskel.h" -extern char bpf_preload_umd_start; -extern char bpf_preload_umd_end; +static struct bpf_link *maps_link, *progs_link; +static struct iterators_bpf *skel; -static int preload(struct bpf_preload_info *obj); -static int finish(void); +static void free_links_and_skel(void) +{ + if (!IS_ERR_OR_NULL(maps_link)) + bpf_link_put(maps_link); + if (!IS_ERR_OR_NULL(progs_link)) + bpf_link_put(progs_link); + iterators_bpf__destroy(skel); +} + +static int preload(struct bpf_preload_info *obj) +{ + strlcpy(obj[0].link_name, "maps.debug", sizeof(obj[0].link_name)); + obj[0].link = maps_link; + strlcpy(obj[1].link_name, "progs.debug", sizeof(obj[1].link_name)); + obj[1].link = progs_link; + return 0; +} -static struct bpf_preload_ops umd_ops = { - .info.driver_name = "bpf_preload", +static struct bpf_preload_ops ops = { .preload = preload, - .finish = finish, .owner = THIS_MODULE, }; -static int preload(struct bpf_preload_info *obj) +static int load_skel(void) { - int magic = BPF_PRELOAD_START; - loff_t pos = 0; - int i, err; - ssize_t n; + int err; - err = fork_usermode_driver(&umd_ops.info); + skel = iterators_bpf__open(); + if (!skel) + return -ENOMEM; + err = iterators_bpf__load(skel); if (err) - return err; - - /* send the start magic to let UMD proceed with loading BPF progs */ - n = kernel_write(umd_ops.info.pipe_to_umh, - &magic, sizeof(magic), &pos); - if (n != sizeof(magic)) - return -EPIPE; - - /* receive bpf_link IDs and names from UMD */ - pos = 0; - for (i = 0; i < BPF_PRELOAD_LINKS; i++) { - n = kernel_read(umd_ops.info.pipe_from_umh, - &obj[i], sizeof(*obj), &pos); - if (n != sizeof(*obj)) - return -EPIPE; + goto out; + err = iterators_bpf__attach(skel); + if (err) + goto out; + maps_link = bpf_link_get_from_fd(skel->links.dump_bpf_map_fd); + if (IS_ERR(maps_link)) { + err = PTR_ERR(maps_link); + goto out; } - return 0; -} - -static int finish(void) -{ - int magic = BPF_PRELOAD_END; - struct pid *tgid; - loff_t pos = 0; - ssize_t n; - - /* send the last magic to UMD. It will do a normal exit. */ - n = kernel_write(umd_ops.info.pipe_to_umh, - &magic, sizeof(magic), &pos); - if (n != sizeof(magic)) - return -EPIPE; - - tgid = umd_ops.info.tgid; - if (tgid) { - wait_event(tgid->wait_pidfd, thread_group_exited(tgid)); - umd_cleanup_helper(&umd_ops.info); + progs_link = bpf_link_get_from_fd(skel->links.dump_bpf_prog_fd); + if (IS_ERR(progs_link)) { + err = PTR_ERR(progs_link); + goto out; } return 0; +out: + free_links_and_skel(); + return err; } -static int __init load_umd(void) +static int __init load(void) { int err; - err = umd_load_blob(&umd_ops.info, &bpf_preload_umd_start, - &bpf_preload_umd_end - &bpf_preload_umd_start); + err = load_skel(); if (err) return err; - bpf_preload_ops = &umd_ops; + bpf_preload_ops = &ops; return err; } -static void __exit fini_umd(void) +static void __exit fini(void) { - struct pid *tgid; - bpf_preload_ops = NULL; - - /* kill UMD in case it's still there due to earlier error */ - tgid = umd_ops.info.tgid; - if (tgid) { - kill_pid(tgid, SIGKILL, 1); - - wait_event(tgid->wait_pidfd, thread_group_exited(tgid)); - umd_cleanup_helper(&umd_ops.info); - } - umd_unload_blob(&umd_ops.info); + free_links_and_skel(); } -late_initcall(load_umd); -module_exit(fini_umd); +late_initcall(load); +module_exit(fini); MODULE_LICENSE("GPL"); diff --git a/kernel/bpf/preload/bpf_preload_umd_blob.S b/kernel/bpf/preload/bpf_preload_umd_blob.S deleted file mode 100644 index f1f40223b5c3..000000000000 --- a/kernel/bpf/preload/bpf_preload_umd_blob.S +++ /dev/null @@ -1,7 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ - .section .init.rodata, "a" - .global bpf_preload_umd_start -bpf_preload_umd_start: - .incbin "kernel/bpf/preload/bpf_preload_umd" - .global bpf_preload_umd_end -bpf_preload_umd_end: diff --git a/kernel/bpf/preload/iterators/bpf_preload_common.h b/kernel/bpf/preload/iterators/bpf_preload_common.h deleted file mode 100644 index 8464d1a48c05..000000000000 --- a/kernel/bpf/preload/iterators/bpf_preload_common.h +++ /dev/null @@ -1,13 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef _BPF_PRELOAD_COMMON_H -#define _BPF_PRELOAD_COMMON_H - -#define BPF_PRELOAD_START 0x5555 -#define BPF_PRELOAD_END 0xAAAA - -struct bpf_preload_info { - char link_name[16]; - int link_id; -}; - -#endif diff --git a/kernel/bpf/preload/iterators/iterators.c b/kernel/bpf/preload/iterators/iterators.c deleted file mode 100644 index 4dafe0f4f2b2..000000000000 --- a/kernel/bpf/preload/iterators/iterators.c +++ /dev/null @@ -1,108 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* Copyright (c) 2020 Facebook */ -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include "iterators.lskel.h" -#include "bpf_preload_common.h" - -int to_kernel = -1; -int from_kernel = 0; - -static int __bpf_obj_get_info_by_fd(int bpf_fd, void *info, __u32 *info_len) -{ - union bpf_attr attr; - int err; - - memset(&attr, 0, sizeof(attr)); - attr.info.bpf_fd = bpf_fd; - attr.info.info_len = *info_len; - attr.info.info = (long) info; - - err = skel_sys_bpf(BPF_OBJ_GET_INFO_BY_FD, &attr, sizeof(attr)); - if (!err) - *info_len = attr.info.info_len; - return err; -} - -static int send_link_to_kernel(int link_fd, const char *link_name) -{ - struct bpf_preload_info obj = {}; - struct bpf_link_info info = {}; - __u32 info_len = sizeof(info); - int err; - - err = __bpf_obj_get_info_by_fd(link_fd, &info, &info_len); - if (err) - return err; - obj.link_id = info.id; - if (strlen(link_name) >= sizeof(obj.link_name)) - return -E2BIG; - strcpy(obj.link_name, link_name); - if (write(to_kernel, &obj, sizeof(obj)) != sizeof(obj)) - return -EPIPE; - return 0; -} - -int main(int argc, char **argv) -{ - struct iterators_bpf *skel; - int err, magic; - int debug_fd; - - debug_fd = open("/dev/console", O_WRONLY | O_NOCTTY | O_CLOEXEC); - if (debug_fd < 0) - return 1; - to_kernel = dup(1); - close(1); - dup(debug_fd); - /* now stdin and stderr point to /dev/console */ - - read(from_kernel, &magic, sizeof(magic)); - if (magic != BPF_PRELOAD_START) { - printf("bad start magic %d\n", magic); - return 1; - } - /* libbpf opens BPF object and loads it into the kernel */ - skel = iterators_bpf__open_and_load(); - if (!skel) { - /* iterators.skel.h is little endian. - * libbpf doesn't support automatic little->big conversion - * of BPF bytecode yet. - * The program load will fail in such case. - */ - printf("Failed load could be due to wrong endianness\n"); - return 1; - } - err = iterators_bpf__attach(skel); - if (err) - goto cleanup; - - /* send two bpf_link IDs with names to the kernel */ - err = send_link_to_kernel(skel->links.dump_bpf_map_fd, "maps.debug"); - if (err) - goto cleanup; - err = send_link_to_kernel(skel->links.dump_bpf_prog_fd, "progs.debug"); - if (err) - goto cleanup; - - /* The kernel will proceed with pinnging the links in bpffs. - * UMD will wait on read from pipe. - */ - read(from_kernel, &magic, sizeof(magic)); - if (magic != BPF_PRELOAD_END) { - printf("bad final magic %d\n", magic); - err = -EINVAL; - } -cleanup: - iterators_bpf__destroy(skel); - - return err != 0; -} diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 49f88b30662a..35646db3d950 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -2490,6 +2490,7 @@ void bpf_link_put(struct bpf_link *link) bpf_link_free(link); } } +EXPORT_SYMBOL(bpf_link_put); static int bpf_link_release(struct inode *inode, struct file *filp) { @@ -2632,6 +2633,7 @@ struct bpf_link *bpf_link_get_from_fd(u32 ufd) return link; } +EXPORT_SYMBOL(bpf_link_get_from_fd); struct bpf_tracing_link { struct bpf_link link;