[bpf-next,1/2] libbpf: fix LDX/STX/ST CO-RE relocation size adjustment logic

Commit Message

Andrii Nakryiko Feb. 7, 2025, 1:48 a.m. UTC

Libbpf has a somewhat obscure feature of automatically adjusting the
"size" of LDX/STX/ST instruction (memory store and load instructions),
based on originally recorded access size (u8, u16, u32, or u64) and the
actual size of the field on target kernel. This is meant to facilitate
using BPF CO-RE on 32-bit architectures (pointers are always 64-bit in
BPF, but host kernel's BTF will have it as 32-bit type), as well as
generally supporting safe type changes (unsigned integer type changes
can be transparently "relocated").

One issue that surfaced only now, 5 years after this logic was
implemented, is how this all works when dealing with fields that are
arrays. This isn't all that easy and straightforward to hit (see
selftests that reproduce this condition), but one of sched_ext BPF
programs did hit it with innocent looking loop.

Long story short, libbpf used to calculate entire array size, instead of
making sure to only calculate array's element size. But it's the element
that is loaded by LDX/STX/ST instructions (1, 2, 4, or 8 bytes), so
that's what libbpf should check. This patch adjusts the logic for
arrays and fixed the issue.

Reported-by: Emil Tsalapatis <[e-mail]>
Signed-off-by: Andrii Nakryiko <[e-mail]>
---
 tools/lib/bpf/relo_core.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

Comments

Eduard Zingerman Feb. 7, 2025, 9:45 p.m. UTC | #1

On Thu, 2025-02-06 at 17:48 -0800, Andrii Nakryiko wrote:
> Libbpf has a somewhat obscure feature of automatically adjusting the
> "size" of LDX/STX/ST instruction (memory store and load instructions),
> based on originally recorded access size (u8, u16, u32, or u64) and the
> actual size of the field on target kernel. This is meant to facilitate
> using BPF CO-RE on 32-bit architectures (pointers are always 64-bit in
> BPF, but host kernel's BTF will have it as 32-bit type), as well as
> generally supporting safe type changes (unsigned integer type changes
> can be transparently "relocated").
> 
> One issue that surfaced only now, 5 years after this logic was
> implemented, is how this all works when dealing with fields that are
> arrays. This isn't all that easy and straightforward to hit (see
> selftests that reproduce this condition), but one of sched_ext BPF
> programs did hit it with innocent looking loop.
> 
> Long story short, libbpf used to calculate entire array size, instead of
> making sure to only calculate array's element size. But it's the element
> that is loaded by LDX/STX/ST instructions (1, 2, 4, or 8 bytes), so
> that's what libbpf should check. This patch adjusts the logic for
> arrays and fixed the issue.
> 
> Reported-by: Emil Tsalapatis <[e-mail]>
> Signed-off-by: Andrii Nakryiko <[e-mail]>
> ---

Do I understand correctly, that for nested arrays relocation size
would be resolved to the innermost element size?
To allow e.g.:

    struct { int a[2][3]; }
    ...
    int *a = __builtin_preserve_access_index(({ in->a; }));
    a[0] = 42;

With a justification that nothing useful could be done with 'int **a'
type when dimensions are not known?
I guess this makes sense.

Acked-by: Eduard Zingerman <[e-mail]>?

>  tools/lib/bpf/relo_core.c | 24 ++++++++++++++++++++----
>  1 file changed, 20 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/lib/bpf/relo_core.c b/tools/lib/bpf/relo_core.c
> index 7632e9d41827..2b83c98a1137 100644
> --- a/tools/lib/bpf/relo_core.c
> +++ b/tools/lib/bpf/relo_core.c
> @@ -683,7 +683,7 @@ static int bpf_core_calc_field_relo(const char *prog_name,
>  {
>  	const struct bpf_core_accessor *acc;
>  	const struct btf_type *t;
> -	__u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id;
> +	__u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id, elem_id;
>  	const struct btf_member *m;
>  	const struct btf_type *mt;
>  	bool bitfield;
> @@ -706,8 +706,14 @@ static int bpf_core_calc_field_relo(const char *prog_name,
>  	if (!acc->name) {
>  		if (relo->kind == BPF_CORE_FIELD_BYTE_OFFSET) {
>  			*val = spec->bit_offset / 8;
> -			/* remember field size for load/store mem size */
> -			sz = btf__resolve_size(spec->btf, acc->type_id);
> +			/* remember field size for load/store mem size;
> +			 * note, for arrays we care about individual element
> +			 * sizes, not the overall array size
> +			 */
> +			t = skip_mods_and_typedefs(spec->btf, acc->type_id, &elem_id);
> +			while (btf_is_array(t))
> +				t = skip_mods_and_typedefs(spec->btf, btf_array(t)->type, &elem_id);
> +			sz = btf__resolve_size(spec->btf, elem_id);

Nit: while trying to figure out what this change is about
     I commented out the above hunk and this did not trigger any test failures.

[...]

Andrii Nakryiko Feb. 10, 2025, 8:05 p.m. UTC | #2

On Fri, Feb 7, 2025 at 1:45 PM Eduard Zingerman <[e-mail]> wrote:
>
> On Thu, 2025-02-06 at 17:48 -0800, Andrii Nakryiko wrote:
> > Libbpf has a somewhat obscure feature of automatically adjusting the
> > "size" of LDX/STX/ST instruction (memory store and load instructions),
> > based on originally recorded access size (u8, u16, u32, or u64) and the
> > actual size of the field on target kernel. This is meant to facilitate
> > using BPF CO-RE on 32-bit architectures (pointers are always 64-bit in
> > BPF, but host kernel's BTF will have it as 32-bit type), as well as
> > generally supporting safe type changes (unsigned integer type changes
> > can be transparently "relocated").
> >
> > One issue that surfaced only now, 5 years after this logic was
> > implemented, is how this all works when dealing with fields that are
> > arrays. This isn't all that easy and straightforward to hit (see
> > selftests that reproduce this condition), but one of sched_ext BPF
> > programs did hit it with innocent looking loop.
> >
> > Long story short, libbpf used to calculate entire array size, instead of
> > making sure to only calculate array's element size. But it's the element
> > that is loaded by LDX/STX/ST instructions (1, 2, 4, or 8 bytes), so
> > that's what libbpf should check. This patch adjusts the logic for
> > arrays and fixed the issue.
> >
> > Reported-by: Emil Tsalapatis <[e-mail]>
> > Signed-off-by: Andrii Nakryiko <[e-mail]>
> > ---
>
> Do I understand correctly, that for nested arrays relocation size
> would be resolved to the innermost element size?
> To allow e.g.:
>
>     struct { int a[2][3]; }
>     ...
>     int *a = __builtin_preserve_access_index(({ in->a; }));
>     a[0] = 42;
>
> With a justification that nothing useful could be done with 'int **a'
> type when dimensions are not known?
> I guess this makes sense.

Known or not, a multi-dimensional array at the lowest level is still
an array of elements, and it is the elements that will be read (up to
u64), so that's why I'm flattening the array and getting to the actual
item.

>
> Acked-by: Eduard Zingerman <[e-mail]>?
>
> >  tools/lib/bpf/relo_core.c | 24 ++++++++++++++++++++----
> >  1 file changed, 20 insertions(+), 4 deletions(-)
> >
> > diff --git a/tools/lib/bpf/relo_core.c b/tools/lib/bpf/relo_core.c
> > index 7632e9d41827..2b83c98a1137 100644
> > --- a/tools/lib/bpf/relo_core.c
> > +++ b/tools/lib/bpf/relo_core.c
> > @@ -683,7 +683,7 @@ static int bpf_core_calc_field_relo(const char *prog_name,
> >  {
> >       const struct bpf_core_accessor *acc;
> >       const struct btf_type *t;
> > -     __u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id;
> > +     __u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id, elem_id;
> >       const struct btf_member *m;
> >       const struct btf_type *mt;
> >       bool bitfield;
> > @@ -706,8 +706,14 @@ static int bpf_core_calc_field_relo(const char *prog_name,
> >       if (!acc->name) {
> >               if (relo->kind == BPF_CORE_FIELD_BYTE_OFFSET) {
> >                       *val = spec->bit_offset / 8;
> > -                     /* remember field size for load/store mem size */
> > -                     sz = btf__resolve_size(spec->btf, acc->type_id);
> > +                     /* remember field size for load/store mem size;
> > +                      * note, for arrays we care about individual element
> > +                      * sizes, not the overall array size
> > +                      */
> > +                     t = skip_mods_and_typedefs(spec->btf, acc->type_id, &elem_id);
> > +                     while (btf_is_array(t))
> > +                             t = skip_mods_and_typedefs(spec->btf, btf_array(t)->type, &elem_id);
> > +                     sz = btf__resolve_size(spec->btf, elem_id);
>
> Nit: while trying to figure out what this change is about
>      I commented out the above hunk and this did not trigger any test failures.

I don't remember exactly under which conditions we'll have this
branch, something about array element access. But this whole logic has
to stay in sync with non-array-element CO-RE relocation.

>
> [...]
>

[e-mail] Feb. 15, 2025, 4:10 a.m. UTC | #3

Hello:

This series was applied to bpf/bpf-next.git (master)
by Alexei Starovoitov <[e-mail]>:

On Thu,  6 Feb 2025 17:48:08 -0800 you wrote:
> Libbpf has a somewhat obscure feature of automatically adjusting the
> "size" of LDX/STX/ST instruction (memory store and load instructions),
> based on originally recorded access size (u8, u16, u32, or u64) and the
> actual size of the field on target kernel. This is meant to facilitate
> using BPF CO-RE on 32-bit architectures (pointers are always 64-bit in
> BPF, but host kernel's BTF will have it as 32-bit type), as well as
> generally supporting safe type changes (unsigned integer type changes
> can be transparently "relocated").
> 
> [...]

Here is the summary with links:
  - [bpf-next,1/2] libbpf: fix LDX/STX/ST CO-RE relocation size adjustment logic
    https://git.kernel.org/bpf/bpf-next/c/06096d19ee38
  - [bpf-next,2/2] selftests/bpf: add test for LDX/STX/ST relocations over array field
    https://git.kernel.org/bpf/bpf-next/c/4eb93fea5919

You are awesome, thank you!

Patch

diff --git a/tools/lib/bpf/relo_core.c b/tools/lib/bpf/relo_core.c
index 7632e9d41827..2b83c98a1137 100644
--- a/tools/lib/bpf/relo_core.c
+++ b/tools/lib/bpf/relo_core.c
@@ -683,7 +683,7 @@  static int bpf_core_calc_field_relo(const char *prog_name,
 {
 	const struct bpf_core_accessor *acc;
 	const struct btf_type *t;
-	__u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id;
+	__u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id, elem_id;
 	const struct btf_member *m;
 	const struct btf_type *mt;
 	bool bitfield;
@@ -706,8 +706,14 @@  static int bpf_core_calc_field_relo(const char *prog_name,
 	if (!acc->name) {
 		if (relo->kind == BPF_CORE_FIELD_BYTE_OFFSET) {
 			*val = spec->bit_offset / 8;
-			/* remember field size for load/store mem size */
-			sz = btf__resolve_size(spec->btf, acc->type_id);
+			/* remember field size for load/store mem size;
+			 * note, for arrays we care about individual element
+			 * sizes, not the overall array size
+			 */
+			t = skip_mods_and_typedefs(spec->btf, acc->type_id, &elem_id);
+			while (btf_is_array(t))
+				t = skip_mods_and_typedefs(spec->btf, btf_array(t)->type, &elem_id);
+			sz = btf__resolve_size(spec->btf, elem_id);
 			if (sz < 0)
 				return -EINVAL;
 			*field_sz = sz;
@@ -767,7 +773,17 @@  static int bpf_core_calc_field_relo(const char *prog_name,
 	case BPF_CORE_FIELD_BYTE_OFFSET:
 		*val = byte_off;
 		if (!bitfield) {
-			*field_sz = byte_sz;
+			/* remember field size for load/store mem size;
+			 * note, for arrays we care about individual element
+			 * sizes, not the overall array size
+			 */
+			t = skip_mods_and_typedefs(spec->btf, field_type_id, &elem_id);
+			while (btf_is_array(t))
+				t = skip_mods_and_typedefs(spec->btf, btf_array(t)->type, &elem_id);
+			sz = btf__resolve_size(spec->btf, elem_id);
+			if (sz < 0)
+				return -EINVAL;
+			*field_sz = sz;
 			*type_id = field_type_id;
 		}
 		break;