Message ID | 20210531195553.168298-2-grantseltzer@gmail.com (mailing list archive) |
---|---|
State | Changes Requested |
Delegated to: | BPF |
Headers | show |
Series | Autogenerating libbpf API documentation | expand |
Context | Check | Description |
---|---|---|
netdev/cover_letter | success | Link |
netdev/fixes_present | success | Link |
netdev/patch_count | success | Link |
netdev/tree_selection | success | Clearly marked for bpf-next |
netdev/subject_prefix | success | Link |
netdev/cc_maintainers | warning | 10 maintainers not CCed: netdev@vger.kernel.org yhs@fb.com hawk@kernel.org kpsingh@kernel.org kafai@fb.com ast@kernel.org john.fastabend@gmail.com songliubraving@fb.com davem@davemloft.net kuba@kernel.org |
netdev/source_inline | success | Was 0 now: 0 |
netdev/verify_signedoff | success | Link |
netdev/module_param | success | Was 0 now: 0 |
netdev/build_32bit | success | Errors and warnings before: 0 this patch: 0 |
netdev/kdoc | success | Errors and warnings before: 0 this patch: 0 |
netdev/verify_fixes | success | Link |
netdev/checkpatch | warning | WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? |
netdev/build_allmodconfig_warn | success | Errors and warnings before: 0 this patch: 0 |
netdev/header_inline | success | Link |
On Mon, May 31, 2021 at 07:55:52PM +0000, grantseltzer wrote: > +++ b/Documentation/bpf/libbpf.rst > @@ -0,0 +1,14 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > +libbpf > +====== > + > +This is documentation for libbpf, a userspace library for loading and > +interacting with bpf programs. > + > +All general BPF questions, including kernel functionality, libbpf APIs and > +their application, should be sent to bpf@vger.kernel.org mailing list. > +You can subscribe to it `here <http://vger.kernel.org/vger-lists.html#bpf>`_ > +and search its archive `here <https://lore.kernel.org/bpf/>`_. https://www.w3.org/QA/Tips/noClickHere I suggest: You can `subscribe <http://vger.kernel.org/vger-lists.html#bpf>`_ to the mailing list or search its `archives <https://lore.kernel.org/bpf/>`_.
On Mon, May 31, 2021 at 12:56 PM grantseltzer <grantseltzer@gmail.com> wrote: > > This adds rst files containing documentation for libbpf. This includes > the addition of libbpf_api.rst which pulls comment documentation from > header files in libbpf under tools/lib/bpf/. The comment docs would be > of the standard kernel doc format. > > Signed-off-by: grantseltzer <grantseltzer@gmail.com> > --- Looks good, thanks! See few comments below. Let's figure out what to do with libbpf docs versioning and land it through bpf-next tree. > Documentation/bpf/index.rst | 13 ++ > Documentation/bpf/libbpf.rst | 14 ++ > Documentation/bpf/libbpf_api.rst | 18 ++ > Documentation/bpf/libbpf_build.rst | 37 ++++ > .../bpf/libbpf_naming_convention.rst | 170 ++++++++++++++++++ > 5 files changed, 252 insertions(+) > create mode 100644 Documentation/bpf/libbpf.rst > create mode 100644 Documentation/bpf/libbpf_api.rst > create mode 100644 Documentation/bpf/libbpf_build.rst > create mode 100644 Documentation/bpf/libbpf_naming_convention.rst > [...] > +API > +=== > + > +This documentation is autogenerated from header files in libbpf, tools/lib/bpf > + > +.. kernel-doc:: tools/lib/bpf/libbpf.h > + :internal: > + > +.. kernel-doc:: tools/lib/bpf/bpf.h > + :internal: > + > +.. kernel-doc:: tools/lib/bpf/btf.h > + :internal: > + > +.. kernel-doc:: tools/lib/bpf/xsk.h > + :internal: Libbpf API has a BPF side as well (bpf_helpers.h which pulls in auto-generated bpf_helper_defs.h with all BPF helper definitions, bpf_tracing.h, bpf_core_read.h, bpf_endian.h), we should probably expose them as well? > diff --git a/Documentation/bpf/libbpf_build.rst b/Documentation/bpf/libbpf_build.rst > new file mode 100644 > index 000000000..b8240eaaa > --- /dev/null > +++ b/Documentation/bpf/libbpf_build.rst > @@ -0,0 +1,37 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > +Building libbpf > +=============== > + > +libelf is an internal dependency of libbpf and thus it is required to link zlib is another dependency, can you please mention it as well? > +against and must be installed on the system for applications to work. > +pkg-config is used by default to find libelf, and the program called > +can be overridden with PKG_CONFIG. > + [...] > +API naming convention > +===================== > + > +libbpf API provides access to a few logically separated groups of > +functions and types. Every group has its own naming convention > +described here. It's recommended to follow these conventions whenever a > +new function or type is added to keep libbpf API clean and consistent. > + > +All types and functions provided by libbpf API should have one of the > +following prefixes: ``bpf_``, ``btf_``, ``libbpf_``, ``xsk_``, > +``perf_buffer_``. ring_buffer_ and btf_dump_ are two others that we use. But I don't know how important it is to have an exhaustive list here. > + > +System call wrappers > +-------------------- > + > +System call wrappers are simple wrappers for commands supported by > +sys_bpf system call. These wrappers should go to ``bpf.h`` header file > +and map one-on-one to corresponding commands. typo: one-to-one? > + > +For example ``bpf_map_lookup_elem`` wraps ``BPF_MAP_LOOKUP_ELEM`` > +command of sys_bpf, ``bpf_prog_attach`` wraps ``BPF_PROG_ATTACH``, etc. > + > +Objects > +------- > + > +Another class of types and functions provided by libbpf API is "objects" > +and functions to work with them. Objects are high-level abstractions > +such as BPF program or BPF map. They're represented by corresponding > +structures such as ``struct bpf_object``, ``struct bpf_program``, > +``struct bpf_map``, etc. > + > +Structures are forward declared and access to their fields should be > +provided via corresponding getters and setters rather than directly. > + > +These objects are associated with corresponding parts of ELF object that > +contains compiled BPF programs. > + > +For example ``struct bpf_object`` represents ELF object itself created > +from an ELF file or from a buffer, ``struct bpf_program`` represents a > +program in ELF object and ``struct bpf_map`` is a map. > + > +Functions that work with an object have names built from object name, > +double underscore and part that describes function purpose. > + > +For example ``bpf_object__open`` consists of the name of corresponding > +object, ``bpf_object``, double underscore and ``open`` that defines the > +purpose of the function to open ELF file and create ``bpf_object`` from > +it. > + > +Another example: ``bpf_program__load`` is named for corresponding > +object, ``bpf_program``, that is separated from other part of the name > +by double underscore. let's drop this example, bpf_program__load is a bad example and is going to be deprecated. We can use btf__parse() as an example here. > + > +All objects and corresponding functions other than BTF related should go > +to ``libbpf.h``. BTF types and functions should go to ``btf.h``. > + > +Auxiliary functions > +------------------- > + > +Auxiliary functions and types that don't fit well in any of categories > +described above should have ``libbpf_`` prefix, e.g. > +``libbpf_get_error`` or ``libbpf_prog_type_by_name``. > + > +AF_XDP functions > +------------------- > + > +AF_XDP functions should have an ``xsk_`` prefix, e.g. > +``xsk_umem__get_data`` or ``xsk_umem__create``. The interface consists > +of both low-level ring access functions and high-level configuration > +functions. These can be mixed and matched. Note that these functions > +are not reentrant for performance reasons. > + > +Please take a look at Documentation/networking/af_xdp.rst in the Linux > +kernel source tree on how to use XDP sockets and for some common > +mistakes in case you do not get any traffic up to user space. I'd probably drop this section, given we move xsk.{c,h} into libxdp. > + > +ABI > +========== > + > +libbpf can be both linked statically or used as DSO. To avoid possible > +conflicts with other libraries an application is linked with, all > +non-static libbpf symbols should have one of the prefixes mentioned in > +API documentation above. See API naming convention to choose the right > +name for a new symbol. > + [...]
On Wed, Jun 2, 2021 at 4:36 PM Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote: > > On Mon, May 31, 2021 at 12:56 PM grantseltzer <grantseltzer@gmail.com> wrote: > > > > This adds rst files containing documentation for libbpf. This includes > > the addition of libbpf_api.rst which pulls comment documentation from > > header files in libbpf under tools/lib/bpf/. The comment docs would be > > of the standard kernel doc format. > > > > Signed-off-by: grantseltzer <grantseltzer@gmail.com> > > --- > > Looks good, thanks! See few comments below. Let's figure out what to > do with libbpf docs versioning and land it through bpf-next tree. > > > Documentation/bpf/index.rst | 13 ++ > > Documentation/bpf/libbpf.rst | 14 ++ > > Documentation/bpf/libbpf_api.rst | 18 ++ > > Documentation/bpf/libbpf_build.rst | 37 ++++ > > .../bpf/libbpf_naming_convention.rst | 170 ++++++++++++++++++ > > 5 files changed, 252 insertions(+) > > create mode 100644 Documentation/bpf/libbpf.rst > > create mode 100644 Documentation/bpf/libbpf_api.rst > > create mode 100644 Documentation/bpf/libbpf_build.rst > > create mode 100644 Documentation/bpf/libbpf_naming_convention.rst > > > > [...] > > > +API > > +=== > > + > > +This documentation is autogenerated from header files in libbpf, tools/lib/bpf > > + > > +.. kernel-doc:: tools/lib/bpf/libbpf.h > > + :internal: > > + > > +.. kernel-doc:: tools/lib/bpf/bpf.h > > + :internal: > > + > > +.. kernel-doc:: tools/lib/bpf/btf.h > > + :internal: > > + > > +.. kernel-doc:: tools/lib/bpf/xsk.h > > + :internal: > > Libbpf API has a BPF side as well (bpf_helpers.h which pulls in > auto-generated bpf_helper_defs.h with all BPF helper definitions, > bpf_tracing.h, bpf_core_read.h, bpf_endian.h), we should probably > expose them as well? > > > diff --git a/Documentation/bpf/libbpf_build.rst b/Documentation/bpf/libbpf_build.rst > > new file mode 100644 > > index 000000000..b8240eaaa > > --- /dev/null > > +++ b/Documentation/bpf/libbpf_build.rst > > @@ -0,0 +1,37 @@ > > +.. SPDX-License-Identifier: GPL-2.0 > > + > > +Building libbpf > > +=============== > > + > > +libelf is an internal dependency of libbpf and thus it is required to link > > zlib is another dependency, can you please mention it as well? > > > +against and must be installed on the system for applications to work. > > +pkg-config is used by default to find libelf, and the program called > > +can be overridden with PKG_CONFIG. > > + > > [...] > > > +API naming convention > > +===================== > > + > > +libbpf API provides access to a few logically separated groups of > > +functions and types. Every group has its own naming convention > > +described here. It's recommended to follow these conventions whenever a > > +new function or type is added to keep libbpf API clean and consistent. > > + > > +All types and functions provided by libbpf API should have one of the > > +following prefixes: ``bpf_``, ``btf_``, ``libbpf_``, ``xsk_``, > > +``perf_buffer_``. > > ring_buffer_ and btf_dump_ are two others that we use. But I don't > know how important it is to have an exhaustive list here. > > > + > > +System call wrappers > > +-------------------- > > + > > +System call wrappers are simple wrappers for commands supported by > > +sys_bpf system call. These wrappers should go to ``bpf.h`` header file > > +and map one-on-one to corresponding commands. > > typo: one-to-one? > > > + > > +For example ``bpf_map_lookup_elem`` wraps ``BPF_MAP_LOOKUP_ELEM`` > > +command of sys_bpf, ``bpf_prog_attach`` wraps ``BPF_PROG_ATTACH``, etc. > > + > > +Objects > > +------- > > + > > +Another class of types and functions provided by libbpf API is "objects" > > +and functions to work with them. Objects are high-level abstractions > > +such as BPF program or BPF map. They're represented by corresponding > > +structures such as ``struct bpf_object``, ``struct bpf_program``, > > +``struct bpf_map``, etc. > > + > > +Structures are forward declared and access to their fields should be > > +provided via corresponding getters and setters rather than directly. > > + > > +These objects are associated with corresponding parts of ELF object that > > +contains compiled BPF programs. > > + > > +For example ``struct bpf_object`` represents ELF object itself created > > +from an ELF file or from a buffer, ``struct bpf_program`` represents a > > +program in ELF object and ``struct bpf_map`` is a map. > > + > > +Functions that work with an object have names built from object name, > > +double underscore and part that describes function purpose. > > + > > +For example ``bpf_object__open`` consists of the name of corresponding > > +object, ``bpf_object``, double underscore and ``open`` that defines the > > +purpose of the function to open ELF file and create ``bpf_object`` from > > +it. > > + > > +Another example: ``bpf_program__load`` is named for corresponding > > +object, ``bpf_program``, that is separated from other part of the name > > +by double underscore. > > let's drop this example, bpf_program__load is a bad example and is > going to be deprecated. We can use btf__parse() as an example here. > > > + > > +All objects and corresponding functions other than BTF related should go > > +to ``libbpf.h``. BTF types and functions should go to ``btf.h``. > > + > > +Auxiliary functions > > +------------------- > > + > > +Auxiliary functions and types that don't fit well in any of categories > > +described above should have ``libbpf_`` prefix, e.g. > > +``libbpf_get_error`` or ``libbpf_prog_type_by_name``. > > + > > +AF_XDP functions > > +------------------- > > + > > +AF_XDP functions should have an ``xsk_`` prefix, e.g. > > +``xsk_umem__get_data`` or ``xsk_umem__create``. The interface consists > > +of both low-level ring access functions and high-level configuration > > +functions. These can be mixed and matched. Note that these functions > > +are not reentrant for performance reasons. > > + > > +Please take a look at Documentation/networking/af_xdp.rst in the Linux > > +kernel source tree on how to use XDP sockets and for some common > > +mistakes in case you do not get any traffic up to user space. > > I'd probably drop this section, given we move xsk.{c,h} into libxdp. > > > + > > +ABI > > +========== > > + > > +libbpf can be both linked statically or used as DSO. To avoid possible > > +conflicts with other libraries an application is linked with, all > > +non-static libbpf symbols should have one of the prefixes mentioned in > > +API documentation above. See API naming convention to choose the right > > +name for a new symbol. > > + > > [...] Thanks for the feedback on this, I've fixed them on a local copy and will incorporate them into my final patch/PR depending on what we go with. See my most recent response on '[PATCH bpf-next 0/3] Autogenerating API documentation'
diff --git a/Documentation/bpf/index.rst b/Documentation/bpf/index.rst index a702f67dd..44f646735 100644 --- a/Documentation/bpf/index.rst +++ b/Documentation/bpf/index.rst @@ -12,6 +12,19 @@ BPF instruction-set. The Cilium project also maintains a `BPF and XDP Reference Guide`_ that goes into great technical depth about the BPF Architecture. +libbpf +====== + +Libbpf is a userspace library for loading and interacting with bpf programs. + +.. toctree:: + :maxdepth: 1 + + libbpf + libbpf_api + libbpf_build + libbpf_naming_convention + BPF Type Format (BTF) ===================== diff --git a/Documentation/bpf/libbpf.rst b/Documentation/bpf/libbpf.rst new file mode 100644 index 000000000..515e5f341 --- /dev/null +++ b/Documentation/bpf/libbpf.rst @@ -0,0 +1,14 @@ +.. SPDX-License-Identifier: GPL-2.0 + +libbpf +====== + +This is documentation for libbpf, a userspace library for loading and +interacting with bpf programs. + +All general BPF questions, including kernel functionality, libbpf APIs and +their application, should be sent to bpf@vger.kernel.org mailing list. +You can subscribe to it `here <http://vger.kernel.org/vger-lists.html#bpf>`_ +and search its archive `here <https://lore.kernel.org/bpf/>`_. +Please search the archive before asking new questions. It very well might +be that this was already addressed or answered before. diff --git a/Documentation/bpf/libbpf_api.rst b/Documentation/bpf/libbpf_api.rst new file mode 100644 index 000000000..fd6b25e03 --- /dev/null +++ b/Documentation/bpf/libbpf_api.rst @@ -0,0 +1,18 @@ +.. SPDX-License-Identifier: GPL-2.0 + +API +=== + +This documentation is autogenerated from header files in libbpf, tools/lib/bpf + +.. kernel-doc:: tools/lib/bpf/libbpf.h + :internal: + +.. kernel-doc:: tools/lib/bpf/bpf.h + :internal: + +.. kernel-doc:: tools/lib/bpf/btf.h + :internal: + +.. kernel-doc:: tools/lib/bpf/xsk.h + :internal: diff --git a/Documentation/bpf/libbpf_build.rst b/Documentation/bpf/libbpf_build.rst new file mode 100644 index 000000000..b8240eaaa --- /dev/null +++ b/Documentation/bpf/libbpf_build.rst @@ -0,0 +1,37 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Building libbpf +=============== + +libelf is an internal dependency of libbpf and thus it is required to link +against and must be installed on the system for applications to work. +pkg-config is used by default to find libelf, and the program called +can be overridden with PKG_CONFIG. + +If using pkg-config at build time is not desired, it can be disabled by +setting NO_PKG_CONFIG=1 when calling make. + +To build both static libbpf.a and shared libbpf.so: + +.. code-block:: bash + + $ cd src + $ make + +To build only static libbpf.a library in directory build/ and install them +together with libbpf headers in a staging directory root/: + +.. code-block:: bash + + $ cd src + $ mkdir build root + $ BUILD_STATIC_ONLY=y OBJDIR=build DESTDIR=root make install + +To build both static libbpf.a and shared libbpf.so against a custom libelf +dependency installed in /build/root/ and install them together with libbpf +headers in a build directory /build/root/: + +.. code-block:: bash + + $ cd src + $ PKG_CONFIG_PATH=/build/root/lib64/pkgconfig DESTDIR=/build/root make \ No newline at end of file diff --git a/Documentation/bpf/libbpf_naming_convention.rst b/Documentation/bpf/libbpf_naming_convention.rst new file mode 100644 index 000000000..048be7933 --- /dev/null +++ b/Documentation/bpf/libbpf_naming_convention.rst @@ -0,0 +1,170 @@ +.. SPDX-License-Identifier: GPL-2.0 + +API naming convention +===================== + +libbpf API provides access to a few logically separated groups of +functions and types. Every group has its own naming convention +described here. It's recommended to follow these conventions whenever a +new function or type is added to keep libbpf API clean and consistent. + +All types and functions provided by libbpf API should have one of the +following prefixes: ``bpf_``, ``btf_``, ``libbpf_``, ``xsk_``, +``perf_buffer_``. + +System call wrappers +-------------------- + +System call wrappers are simple wrappers for commands supported by +sys_bpf system call. These wrappers should go to ``bpf.h`` header file +and map one-on-one to corresponding commands. + +For example ``bpf_map_lookup_elem`` wraps ``BPF_MAP_LOOKUP_ELEM`` +command of sys_bpf, ``bpf_prog_attach`` wraps ``BPF_PROG_ATTACH``, etc. + +Objects +------- + +Another class of types and functions provided by libbpf API is "objects" +and functions to work with them. Objects are high-level abstractions +such as BPF program or BPF map. They're represented by corresponding +structures such as ``struct bpf_object``, ``struct bpf_program``, +``struct bpf_map``, etc. + +Structures are forward declared and access to their fields should be +provided via corresponding getters and setters rather than directly. + +These objects are associated with corresponding parts of ELF object that +contains compiled BPF programs. + +For example ``struct bpf_object`` represents ELF object itself created +from an ELF file or from a buffer, ``struct bpf_program`` represents a +program in ELF object and ``struct bpf_map`` is a map. + +Functions that work with an object have names built from object name, +double underscore and part that describes function purpose. + +For example ``bpf_object__open`` consists of the name of corresponding +object, ``bpf_object``, double underscore and ``open`` that defines the +purpose of the function to open ELF file and create ``bpf_object`` from +it. + +Another example: ``bpf_program__load`` is named for corresponding +object, ``bpf_program``, that is separated from other part of the name +by double underscore. + +All objects and corresponding functions other than BTF related should go +to ``libbpf.h``. BTF types and functions should go to ``btf.h``. + +Auxiliary functions +------------------- + +Auxiliary functions and types that don't fit well in any of categories +described above should have ``libbpf_`` prefix, e.g. +``libbpf_get_error`` or ``libbpf_prog_type_by_name``. + +AF_XDP functions +------------------- + +AF_XDP functions should have an ``xsk_`` prefix, e.g. +``xsk_umem__get_data`` or ``xsk_umem__create``. The interface consists +of both low-level ring access functions and high-level configuration +functions. These can be mixed and matched. Note that these functions +are not reentrant for performance reasons. + +Please take a look at Documentation/networking/af_xdp.rst in the Linux +kernel source tree on how to use XDP sockets and for some common +mistakes in case you do not get any traffic up to user space. + +ABI +========== + +libbpf can be both linked statically or used as DSO. To avoid possible +conflicts with other libraries an application is linked with, all +non-static libbpf symbols should have one of the prefixes mentioned in +API documentation above. See API naming convention to choose the right +name for a new symbol. + +Symbol visibility +----------------- + +libbpf follow the model when all global symbols have visibility "hidden" +by default and to make a symbol visible it has to be explicitly +attributed with ``LIBBPF_API`` macro. For example: + +.. code-block:: c + + LIBBPF_API int bpf_prog_get_fd_by_id(__u32 id); + +This prevents from accidentally exporting a symbol, that is not supposed +to be a part of ABI what, in turn, improves both libbpf developer- and +user-experiences. + +ABI versionning +--------------- + +To make future ABI extensions possible libbpf ABI is versioned. +Versioning is implemented by ``libbpf.map`` version script that is +passed to linker. + +Version name is ``LIBBPF_`` prefix + three-component numeric version, +starting from ``0.0.1``. + +Every time ABI is being changed, e.g. because a new symbol is added or +semantic of existing symbol is changed, ABI version should be bumped. +This bump in ABI version is at most once per kernel development cycle. + +For example, if current state of ``libbpf.map`` is: + +.. code-block:: c + + LIBBPF_0.0.1 { + global: + bpf_func_a; + bpf_func_b; + local: + \*; + }; + +, and a new symbol ``bpf_func_c`` is being introduced, then +``libbpf.map`` should be changed like this: + +.. code-block:: c + + LIBBPF_0.0.1 { + global: + bpf_func_a; + bpf_func_b; + local: + \*; + }; + LIBBPF_0.0.2 { + global: + bpf_func_c; + } LIBBPF_0.0.1; + +, where new version ``LIBBPF_0.0.2`` depends on the previous +``LIBBPF_0.0.1``. + +Format of version script and ways to handle ABI changes, including +incompatible ones, described in details in [1]. + +Stand-alone build +------------------- + +Under https://github.com/libbpf/libbpf there is a (semi-)automated +mirror of the mainline's version of libbpf for a stand-alone build. + +However, all changes to libbpf's code base must be upstreamed through +the mainline kernel tree. + +License +------------------- + +libbpf is dual-licensed under LGPL 2.1 and BSD 2-Clause. + +Links +------------------- + +[1] https://www.akkadia.org/drepper/dsohowto.pdf + (Chapter 3. Maintaining APIs and ABIs).
This adds rst files containing documentation for libbpf. This includes the addition of libbpf_api.rst which pulls comment documentation from header files in libbpf under tools/lib/bpf/. The comment docs would be of the standard kernel doc format. Signed-off-by: grantseltzer <grantseltzer@gmail.com> --- Documentation/bpf/index.rst | 13 ++ Documentation/bpf/libbpf.rst | 14 ++ Documentation/bpf/libbpf_api.rst | 18 ++ Documentation/bpf/libbpf_build.rst | 37 ++++ .../bpf/libbpf_naming_convention.rst | 170 ++++++++++++++++++ 5 files changed, 252 insertions(+) create mode 100644 Documentation/bpf/libbpf.rst create mode 100644 Documentation/bpf/libbpf_api.rst create mode 100644 Documentation/bpf/libbpf_build.rst create mode 100644 Documentation/bpf/libbpf_naming_convention.rst