| Message ID | 20210313193537.1548766-9-andrii@kernel.org (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | BPF |
| Headers | show |
| Series | BPF static linking | expand |
2021-03-13 11:35 UTC-0800 ~ Andrii Nakryiko <andrii@kernel.org> > Add `bpftool gen object <output-file> <input_file>...` command to statically > link multiple BPF ELF object files into a single output BPF ELF object file. > > Similarly to existing '*.o' convention, bpftool is establishing a '*.bpfo' > convention for statically-linked BPF object files. Both .o and .bpfo suffixes > will be stripped out during BPF skeleton generation to infer BPF object name. > > This patch also updates bash completions and man page. Man page gets a short > section on `gen object` command, but also updates the skeleton example to show > off workflow for BPF application with two .bpf.c files, compiled individually > with Clang, then resulting object files are linked together with `gen object`, > and then final object file is used to generate usable BPF skeleton. This > should help new users understand realistic workflow w.r.t. compiling > mutli-file BPF application. > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org> > --- > .../bpf/bpftool/Documentation/bpftool-gen.rst | 65 +++++++++++++++---- > tools/bpf/bpftool/bash-completion/bpftool | 2 +- > tools/bpf/bpftool/gen.c | 49 +++++++++++++- > 3 files changed, 99 insertions(+), 17 deletions(-) > > diff --git a/tools/bpf/bpftool/Documentation/bpftool-gen.rst b/tools/bpf/bpftool/Documentation/bpftool-gen.rst > index 84cf0639696f..4cdce187c393 100644 > --- a/tools/bpf/bpftool/Documentation/bpftool-gen.rst > +++ b/tools/bpf/bpftool/Documentation/bpftool-gen.rst > @@ -14,16 +14,37 @@ SYNOPSIS > > *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] } > > - *COMMAND* := { **skeleton** | **help** } > + *COMMAND* := { **object** | **skeleton** | **help** } > > GEN COMMANDS > ============= > > +| **bpftool** **gen object** *OUTPUT_FILE* *INPUT_FILE* [*INPUT_FILE*...] > | **bpftool** **gen skeleton** *FILE* > | **bpftool** **gen help** > > DESCRIPTION > =========== > + **bpftool gen object** *OUTPUT_FILE* *INPUT_FILE* [*INPUT_FILE*...] > + Statically link (combine) together one or more *INPUT_FILE*'s > + into a single resulting *OUTPUT_FILE*. All the files involed Typo: "involed" > + are BPF ELF object files. > + > + The rules of BPF static linking are mostly the same as for > + user-space object files, but in addition to combining data > + and instruction sections, .BTF and .BTF.ext (if present in > + any of the input files) data are combined together. .BTF > + data is deduplicated, so all the common types across > + *INPUT_FILE*'s will only be represented once in the resulting > + BTF information. > + > + BPF static linking allows to partition BPF source code into > + individually compiled files that are then linked into > + a single resulting BPF object file, which can be used to > + generated BPF skeleton (with **gen skeleton** command) or > + passed directly into **libbpf** (using **bpf_object__open()** > + family of APIs). > + > **bpftool gen skeleton** *FILE* > Generate BPF skeleton C header file for a given *FILE*. > > diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool > index fdffbc64c65c..7ca23c58f2c0 100644 > --- a/tools/bpf/bpftool/bash-completion/bpftool > +++ b/tools/bpf/bpftool/bash-completion/bpftool > @@ -981,7 +981,7 @@ _bpftool() > ;; > gen) > case $command in > - skeleton) > + object|skeleton) > _filedir > ;; > *) Suggesting the "object" keyword for completing "bpftool gen [tab]" is missing. It is just a few lines below: ------ diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool index fdffbc64c65c..223438e86932 100644 --- a/tools/bpf/bpftool/bash-completion/bpftool +++ b/tools/bpf/bpftool/bash-completion/bpftool @@ -981,12 +981,12 @@ _bpftool() ;; gen) case $command in - skeleton) + object|skeleton) _filedir ;; *) [[ $prev == $object ]] && \ - COMPREPLY=( $( compgen -W 'skeleton help' -- "$cur" ) ) + COMPREPLY=( $( compgen -W 'object skeleton help' -- "$cur" ) ) ;; esac ;; ------ Looks good otherwise. Thanks for the documentation, it's great to have the example in the man page. Pending the two nits above are fixed: Reviewed-by: Quentin Monnet <quentin@isovalent.com> Quentin
On Mon, Mar 15, 2021 at 2:24 AM Quentin Monnet <quentin@isovalent.com> wrote: > > 2021-03-13 11:35 UTC-0800 ~ Andrii Nakryiko <andrii@kernel.org> > > Add `bpftool gen object <output-file> <input_file>...` command to statically > > link multiple BPF ELF object files into a single output BPF ELF object file. > > > > Similarly to existing '*.o' convention, bpftool is establishing a '*.bpfo' > > convention for statically-linked BPF object files. Both .o and .bpfo suffixes > > will be stripped out during BPF skeleton generation to infer BPF object name. > > > > This patch also updates bash completions and man page. Man page gets a short > > section on `gen object` command, but also updates the skeleton example to show > > off workflow for BPF application with two .bpf.c files, compiled individually > > with Clang, then resulting object files are linked together with `gen object`, > > and then final object file is used to generate usable BPF skeleton. This > > should help new users understand realistic workflow w.r.t. compiling > > mutli-file BPF application. > > > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org> > > --- > > .../bpf/bpftool/Documentation/bpftool-gen.rst | 65 +++++++++++++++---- > > tools/bpf/bpftool/bash-completion/bpftool | 2 +- > > tools/bpf/bpftool/gen.c | 49 +++++++++++++- > > 3 files changed, 99 insertions(+), 17 deletions(-) > > > > diff --git a/tools/bpf/bpftool/Documentation/bpftool-gen.rst b/tools/bpf/bpftool/Documentation/bpftool-gen.rst > > index 84cf0639696f..4cdce187c393 100644 > > --- a/tools/bpf/bpftool/Documentation/bpftool-gen.rst > > +++ b/tools/bpf/bpftool/Documentation/bpftool-gen.rst > > @@ -14,16 +14,37 @@ SYNOPSIS > > > > *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] } > > > > - *COMMAND* := { **skeleton** | **help** } > > + *COMMAND* := { **object** | **skeleton** | **help** } > > > > GEN COMMANDS > > ============= > > > > +| **bpftool** **gen object** *OUTPUT_FILE* *INPUT_FILE* [*INPUT_FILE*...] > > | **bpftool** **gen skeleton** *FILE* > > | **bpftool** **gen help** > > > > DESCRIPTION > > =========== > > + **bpftool gen object** *OUTPUT_FILE* *INPUT_FILE* [*INPUT_FILE*...] > > + Statically link (combine) together one or more *INPUT_FILE*'s > > + into a single resulting *OUTPUT_FILE*. All the files involed > > Typo: "involed" > > > + are BPF ELF object files. > > + > > + The rules of BPF static linking are mostly the same as for > > + user-space object files, but in addition to combining data > > + and instruction sections, .BTF and .BTF.ext (if present in > > + any of the input files) data are combined together. .BTF > > + data is deduplicated, so all the common types across > > + *INPUT_FILE*'s will only be represented once in the resulting > > + BTF information. > > + > > + BPF static linking allows to partition BPF source code into > > + individually compiled files that are then linked into > > + a single resulting BPF object file, which can be used to > > + generated BPF skeleton (with **gen skeleton** command) or > > + passed directly into **libbpf** (using **bpf_object__open()** > > + family of APIs). > > + > > **bpftool gen skeleton** *FILE* > > Generate BPF skeleton C header file for a given *FILE*. > > > > > diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool > > index fdffbc64c65c..7ca23c58f2c0 100644 > > --- a/tools/bpf/bpftool/bash-completion/bpftool > > +++ b/tools/bpf/bpftool/bash-completion/bpftool > > @@ -981,7 +981,7 @@ _bpftool() > > ;; > > gen) > > case $command in > > - skeleton) > > + object|skeleton) > > _filedir > > ;; > > *) > > Suggesting the "object" keyword for completing "bpftool gen [tab]" > is missing. It is just a few lines below: > > ------ > diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool > index fdffbc64c65c..223438e86932 100644 > --- a/tools/bpf/bpftool/bash-completion/bpftool > +++ b/tools/bpf/bpftool/bash-completion/bpftool > @@ -981,12 +981,12 @@ _bpftool() > ;; > gen) > case $command in > - skeleton) > + object|skeleton) > _filedir > ;; > *) > [[ $prev == $object ]] && \ > - COMPREPLY=( $( compgen -W 'skeleton help' -- "$cur" ) ) > + COMPREPLY=( $( compgen -W 'object skeleton help' -- "$cur" ) ) > ;; > esac > ;; > ------ > > Looks good otherwise. Thanks for the documentation, it's great to > have the example in the man page. > > Pending the two nits above are fixed: > Reviewed-by: Quentin Monnet <quentin@isovalent.com> Thanks! I'll wait a bit for some more feedback on other patches and will fix it in the next version (unless my other patches are perfect, of course ;). > > Quentin
diff --git a/tools/bpf/bpftool/Documentation/bpftool-gen.rst b/tools/bpf/bpftool/Documentation/bpftool-gen.rst index 84cf0639696f..4cdce187c393 100644 --- a/tools/bpf/bpftool/Documentation/bpftool-gen.rst +++ b/tools/bpf/bpftool/Documentation/bpftool-gen.rst @@ -14,16 +14,37 @@ SYNOPSIS *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] } - *COMMAND* := { **skeleton** | **help** } + *COMMAND* := { **object** | **skeleton** | **help** } GEN COMMANDS ============= +| **bpftool** **gen object** *OUTPUT_FILE* *INPUT_FILE* [*INPUT_FILE*...] | **bpftool** **gen skeleton** *FILE* | **bpftool** **gen help** DESCRIPTION =========== + **bpftool gen object** *OUTPUT_FILE* *INPUT_FILE* [*INPUT_FILE*...] + Statically link (combine) together one or more *INPUT_FILE*'s + into a single resulting *OUTPUT_FILE*. All the files involed + are BPF ELF object files. + + The rules of BPF static linking are mostly the same as for + user-space object files, but in addition to combining data + and instruction sections, .BTF and .BTF.ext (if present in + any of the input files) data are combined together. .BTF + data is deduplicated, so all the common types across + *INPUT_FILE*'s will only be represented once in the resulting + BTF information. + + BPF static linking allows to partition BPF source code into + individually compiled files that are then linked into + a single resulting BPF object file, which can be used to + generated BPF skeleton (with **gen skeleton** command) or + passed directly into **libbpf** (using **bpf_object__open()** + family of APIs). + **bpftool gen skeleton** *FILE* Generate BPF skeleton C header file for a given *FILE*. @@ -130,26 +151,19 @@ OPTIONS EXAMPLES ======== -**$ cat example.c** +**$ cat example1.bpf.c** :: #include <stdbool.h> #include <linux/ptrace.h> #include <linux/bpf.h> - #include "bpf_helpers.h" + #include <bpf/bpf_helpers.h> const volatile int param1 = 42; bool global_flag = true; struct { int x; } data = {}; - struct { - __uint(type, BPF_MAP_TYPE_HASH); - __uint(max_entries, 128); - __type(key, int); - __type(value, long); - } my_map SEC(".maps"); - SEC("raw_tp/sys_enter") int handle_sys_enter(struct pt_regs *ctx) { @@ -161,6 +175,21 @@ EXAMPLES return 0; } +**$ cat example2.bpf.c** + +:: + + #include <linux/ptrace.h> + #include <linux/bpf.h> + #include <bpf/bpf_helpers.h> + + struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(max_entries, 128); + __type(key, int); + __type(value, long); + } my_map SEC(".maps"); + SEC("raw_tp/sys_exit") int handle_sys_exit(struct pt_regs *ctx) { @@ -170,9 +199,17 @@ EXAMPLES } This is example BPF application with two BPF programs and a mix of BPF maps -and global variables. +and global variables. Source code is split across two source code files. -**$ bpftool gen skeleton example.o** +**$ clang -target bpf -g example1.bpf.c -o example1.bpf.o** +**$ clang -target bpf -g example2.bpf.c -o example2.bpf.o** +**$ bpftool gen object example.bpfo example1.bpf.o example2.bpf.o** + +This set of commands compiles *example1.bpf.c* and *example2.bpf.c* +individually and then statically links respective object files into the final +BPF ELF object file *example.bpfo*. + +**$ bpftool gen skeleton example.bpfo** :: @@ -227,7 +264,7 @@ and global variables. #endif /* __EXAMPLE_SKEL_H__ */ -**$ cat example_user.c** +**$ cat example.c** :: @@ -270,7 +307,7 @@ and global variables. return err; } -**# ./example_user** +**# ./example** :: diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool index fdffbc64c65c..7ca23c58f2c0 100644 --- a/tools/bpf/bpftool/bash-completion/bpftool +++ b/tools/bpf/bpftool/bash-completion/bpftool @@ -981,7 +981,7 @@ _bpftool() ;; gen) case $command in - skeleton) + object|skeleton) _filedir ;; *) diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c index 4033c46d83e7..123cca7b400b 100644 --- a/tools/bpf/bpftool/gen.c +++ b/tools/bpf/bpftool/gen.c @@ -54,7 +54,9 @@ static void get_obj_name(char *name, const char *file) strncpy(name, basename(file), MAX_OBJ_NAME_LEN - 1); name[MAX_OBJ_NAME_LEN - 1] = '\0'; if (str_has_suffix(name, ".o")) - name[strlen(name) - 2] = '\0'; + name[strlen(name) - sizeof(".o") + 1] = '\0'; + else if (str_has_suffix(name, ".bpfo")) + name[strlen(name) - sizeof(".bpfo") + 1] = '\0'; sanitize_identifier(name); } @@ -591,6 +593,47 @@ static int do_skeleton(int argc, char **argv) return err; } +static int do_object(int argc, char **argv) +{ + struct bpf_linker *linker; + const char *output_file, *file; + int err = 0; + + if (!REQ_ARGS(2)) { + usage(); + return -1; + } + + output_file = GET_ARG(); + + linker = bpf_linker__new(output_file, NULL); + if (!linker) { + p_err("failed to create BPF linker instance"); + return -1; + } + + while (argc) { + file = GET_ARG(); + + err = bpf_linker__add_file(linker, file); + if (err) { + p_err("failed to link '%s': %s (%d)", file, strerror(err), err); + goto out; + } + } + + err = bpf_linker__finalize(linker); + if (err) { + p_err("failed to finalize ELF file: %s (%d)", strerror(err), err); + goto out; + } + + err = 0; +out: + bpf_linker__free(linker); + return err; +} + static int do_help(int argc, char **argv) { if (json_output) { @@ -599,7 +642,8 @@ static int do_help(int argc, char **argv) } fprintf(stderr, - "Usage: %1$s %2$s skeleton FILE\n" + "Usage: %1$s %2$s object OUTPUT_FILE INPUT_FILE [INPUT_FILE...]\n" + " %1$s %2$s skeleton FILE\n" " %1$s %2$s help\n" "\n" " " HELP_SPEC_OPTIONS "\n" @@ -610,6 +654,7 @@ static int do_help(int argc, char **argv) } static const struct cmd cmds[] = { + { "object", do_object }, { "skeleton", do_skeleton }, { "help", do_help }, { 0 }
Add `bpftool gen object <output-file> <input_file>...` command to statically link multiple BPF ELF object files into a single output BPF ELF object file. Similarly to existing '*.o' convention, bpftool is establishing a '*.bpfo' convention for statically-linked BPF object files. Both .o and .bpfo suffixes will be stripped out during BPF skeleton generation to infer BPF object name. This patch also updates bash completions and man page. Man page gets a short section on `gen object` command, but also updates the skeleton example to show off workflow for BPF application with two .bpf.c files, compiled individually with Clang, then resulting object files are linked together with `gen object`, and then final object file is used to generate usable BPF skeleton. This should help new users understand realistic workflow w.r.t. compiling mutli-file BPF application. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> --- .../bpf/bpftool/Documentation/bpftool-gen.rst | 65 +++++++++++++++---- tools/bpf/bpftool/bash-completion/bpftool | 2 +- tools/bpf/bpftool/gen.c | 49 +++++++++++++- 3 files changed, 99 insertions(+), 17 deletions(-)