[v2] bpf: Replace bpf_lpm_trie_key 0-length array with flexible array

Commit Message

Kees Cook Feb. 16, 2024, 11:55 p.m. UTC

Replace deprecated 0-length array in struct bpf_lpm_trie_key with
flexible array. Found with GCC 13:

../kernel/bpf/lpm_trie.c:207:51: warning: array subscript i is outside array bounds of 'const __u8[0]' {aka 'const unsigned char[]'} [-Warray-bounds=]
  207 |                                        *(__be16 *)&key->data[i]);
      |                                                   ^~~~~~~~~~~~~
../include/uapi/linux/swab.h:102:54: note: in definition of macro '__swab16'
  102 | #define __swab16(x) (__u16)__builtin_bswap16((__u16)(x))
      |                                                      ^
../include/linux/byteorder/generic.h:97:21: note: in expansion of macro '__be16_to_cpu'
   97 | #define be16_to_cpu __be16_to_cpu
      |                     ^~~~~~~~~~~~~
../kernel/bpf/lpm_trie.c:206:28: note: in expansion of macro 'be16_to_cpu'
  206 |                 u16 diff = be16_to_cpu(*(__be16 *)&node->data[i]
^
      |                            ^~~~~~~~~~~
In file included from ../include/linux/bpf.h:7:
../include/uapi/linux/bpf.h:82:17: note: while referencing 'data'
   82 |         __u8    data[0];        /* Arbitrary size */
      |                 ^~~~

And found at run-time under CONFIG_FORTIFY_SOURCE:

  UBSAN: array-index-out-of-bounds in kernel/bpf/lpm_trie.c:218:49
  index 0 is out of range for type '__u8 [*]'

This includes fixing the selftest which was incorrectly using a
variable length struct as a header, identified earlier[1]. Avoid this
by just explicitly including the prefixlen member instead of struct
bpf_lpm_trie_key.

Note that it is not possible to simply remove the "data" member, as it
is referenced by userspace

cilium:
        struct egress_gw_policy_key in_key = {
                .lpm_key = { 32 + 24, {} },
                .saddr   = CLIENT_IP,
                .daddr   = EXTERNAL_SVC_IP & 0Xffffff,
        };

systemd:
	ipv6_map_fd = bpf_map_new(
			BPF_MAP_TYPE_LPM_TRIE,
			offsetof(struct bpf_lpm_trie_key, data) + sizeof(uint32_t)*4,
			sizeof(uint64_t),
			...

The only risk to UAPI would be if sizeof() were used directly on the
data member, which it does not seem to be. It is only used as a static
initializer destination and to find its location via offsetof().

Link: https://lore.kernel.org/all/202206281009.4332AA33@keescook/ [1]
Reported-by: Mark Rutland <[e-mail]>
Closes: https://paste.debian.net/hidden/ca500597/
Signed-off-by: Kees Cook <[e-mail]>
---
Cc: Alexei Starovoitov <[e-mail]>
Cc: Daniel Borkmann <[e-mail]>
Cc: Andrii Nakryiko <[e-mail]>
Cc: Martin KaFai Lau <[e-mail]>
Cc: Song Liu <[e-mail]>
Cc: Yonghong Song <[e-mail]>
Cc: John Fastabend <[e-mail]>
Cc: KP Singh <[e-mail]>
Cc: Stanislav Fomichev <[e-mail]>
Cc: Hao Luo <[e-mail]>
Cc: Jiri Olsa <[e-mail]>
Cc: Mykola Lysenko <[e-mail]>
Cc: Shuah Khan <[e-mail]>
Cc: Haowen Bai <[e-mail]>
Cc: [e-mail]
Cc: [e-mail]

v2- clarify commit log, add more failure examples
v1- https://lore.kernel.org/all/[e-mail]/
---
 include/uapi/linux/bpf.h                         | 2 +-
 tools/testing/selftests/bpf/progs/map_ptr_kern.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

Comments

Gustavo A. R. Silva Feb. 17, 2024, 12:27 a.m. UTC | #1

On 2/16/24 17:55, Kees Cook wrote:
> Replace deprecated 0-length array in struct bpf_lpm_trie_key with
> flexible array. Found with GCC 13:
> 
> ../kernel/bpf/lpm_trie.c:207:51: warning: array subscript i is outside array bounds of 'const __u8[0]' {aka 'const unsigned char[]'} [-Warray-bounds=]
>    207 |                                        *(__be16 *)&key->data[i]);
>        |                                                   ^~~~~~~~~~~~~
> ../include/uapi/linux/swab.h:102:54: note: in definition of macro '__swab16'
>    102 | #define __swab16(x) (__u16)__builtin_bswap16((__u16)(x))
>        |                                                      ^
> ../include/linux/byteorder/generic.h:97:21: note: in expansion of macro '__be16_to_cpu'
>     97 | #define be16_to_cpu __be16_to_cpu
>        |                     ^~~~~~~~~~~~~
> ../kernel/bpf/lpm_trie.c:206:28: note: in expansion of macro 'be16_to_cpu'
>    206 |                 u16 diff = be16_to_cpu(*(__be16 *)&node->data[i]
> ^
>        |                            ^~~~~~~~~~~
> In file included from ../include/linux/bpf.h:7:
> ../include/uapi/linux/bpf.h:82:17: note: while referencing 'data'
>     82 |         __u8    data[0];        /* Arbitrary size */
>        |                 ^~~~
> 
> And found at run-time under CONFIG_FORTIFY_SOURCE:
> 
>    UBSAN: array-index-out-of-bounds in kernel/bpf/lpm_trie.c:218:49
>    index 0 is out of range for type '__u8 [*]'
> 
> This includes fixing the selftest which was incorrectly using a
> variable length struct as a header, identified earlier[1]. Avoid this
> by just explicitly including the prefixlen member instead of struct
> bpf_lpm_trie_key.
> 
> Note that it is not possible to simply remove the "data" member, as it
> is referenced by userspace
> 
> cilium:
>          struct egress_gw_policy_key in_key = {
>                  .lpm_key = { 32 + 24, {} },
>                  .saddr   = CLIENT_IP,
>                  .daddr   = EXTERNAL_SVC_IP & 0Xffffff,
>          };
> 
> systemd:
> 	ipv6_map_fd = bpf_map_new(
> 			BPF_MAP_TYPE_LPM_TRIE,
> 			offsetof(struct bpf_lpm_trie_key, data) + sizeof(uint32_t)*4,
> 			sizeof(uint64_t),
> 			...
> 
> The only risk to UAPI would be if sizeof() were used directly on the
> data member, which it does not seem to be. It is only used as a static
> initializer destination and to find its location via offsetof().
> 
> Link: https://lore.kernel.org/all/202206281009.4332AA33@keescook/ [1]
> Reported-by: Mark Rutland <[e-mail]>
> Closes: https://paste.debian.net/hidden/ca500597/

mmh... this URL expires: 2024-05-15

> Signed-off-by: Kees Cook <[e-mail]>

Acked-by: Gustavo A. R. Silva <[e-mail]>

Thanks!
--
Gustavo

> ---
> Cc: Alexei Starovoitov <[e-mail]>
> Cc: Daniel Borkmann <[e-mail]>
> Cc: Andrii Nakryiko <[e-mail]>
> Cc: Martin KaFai Lau <[e-mail]>
> Cc: Song Liu <[e-mail]>
> Cc: Yonghong Song <[e-mail]>
> Cc: John Fastabend <[e-mail]>
> Cc: KP Singh <[e-mail]>
> Cc: Stanislav Fomichev <[e-mail]>
> Cc: Hao Luo <[e-mail]>
> Cc: Jiri Olsa <[e-mail]>
> Cc: Mykola Lysenko <[e-mail]>
> Cc: Shuah Khan <[e-mail]>
> Cc: Haowen Bai <[e-mail]>
> Cc: [e-mail]
> Cc: [e-mail]
> 
> v2- clarify commit log, add more failure examples
> v1- https://lore.kernel.org/all/[e-mail]/
> ---
>   include/uapi/linux/bpf.h                         | 2 +-
>   tools/testing/selftests/bpf/progs/map_ptr_kern.c | 2 +-
>   2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 754e68ca8744..359dd8a429c1 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -80,7 +80,7 @@ struct bpf_insn {
>   /* Key of an a BPF_MAP_TYPE_LPM_TRIE entry */
>   struct bpf_lpm_trie_key {
>   	__u32	prefixlen;	/* up to 32 for AF_INET, 128 for AF_INET6 */
> -	__u8	data[0];	/* Arbitrary size */
> +	__u8	data[];		/* Arbitrary size */
>   };
>   
>   struct bpf_cgroup_storage_key {
> diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c
> index 3325da17ec81..1d476c6ae284 100644
> --- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c
> +++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c
> @@ -316,7 +316,7 @@ struct lpm_trie {
>   } __attribute__((preserve_access_index));
>   
>   struct lpm_key {
> -	struct bpf_lpm_trie_key trie_key;
> +	__u32 prefixlen;
>   	__u32 data;
>   };
>

Kees Cook Feb. 17, 2024, 3:03 a.m. UTC | #2

On Fri, Feb 16, 2024 at 06:27:08PM -0600, Gustavo A. R. Silva wrote:
> 
> 
> On 2/16/24 17:55, Kees Cook wrote:
> > Replace deprecated 0-length array in struct bpf_lpm_trie_key with
> > flexible array. Found with GCC 13:
> > 
> > ../kernel/bpf/lpm_trie.c:207:51: warning: array subscript i is outside array bounds of 'const __u8[0]' {aka 'const unsigned char[]'} [-Warray-bounds=]
> >    207 |                                        *(__be16 *)&key->data[i]);
> >        |                                                   ^~~~~~~~~~~~~
> > ../include/uapi/linux/swab.h:102:54: note: in definition of macro '__swab16'
> >    102 | #define __swab16(x) (__u16)__builtin_bswap16((__u16)(x))
> >        |                                                      ^
> > ../include/linux/byteorder/generic.h:97:21: note: in expansion of macro '__be16_to_cpu'
> >     97 | #define be16_to_cpu __be16_to_cpu
> >        |                     ^~~~~~~~~~~~~
> > ../kernel/bpf/lpm_trie.c:206:28: note: in expansion of macro 'be16_to_cpu'
> >    206 |                 u16 diff = be16_to_cpu(*(__be16 *)&node->data[i]
> > ^
> >        |                            ^~~~~~~~~~~
> > In file included from ../include/linux/bpf.h:7:
> > ../include/uapi/linux/bpf.h:82:17: note: while referencing 'data'
> >     82 |         __u8    data[0];        /* Arbitrary size */
> >        |                 ^~~~
> > 
> > And found at run-time under CONFIG_FORTIFY_SOURCE:
> > 
> >    UBSAN: array-index-out-of-bounds in kernel/bpf/lpm_trie.c:218:49
> >    index 0 is out of range for type '__u8 [*]'
> > 
> > This includes fixing the selftest which was incorrectly using a
> > variable length struct as a header, identified earlier[1]. Avoid this
> > by just explicitly including the prefixlen member instead of struct
> > bpf_lpm_trie_key.
> > 
> > Note that it is not possible to simply remove the "data" member, as it
> > is referenced by userspace
> > 
> > cilium:
> >          struct egress_gw_policy_key in_key = {
> >                  .lpm_key = { 32 + 24, {} },
> >                  .saddr   = CLIENT_IP,
> >                  .daddr   = EXTERNAL_SVC_IP & 0Xffffff,
> >          };
> > 
> > systemd:
> > 	ipv6_map_fd = bpf_map_new(
> > 			BPF_MAP_TYPE_LPM_TRIE,
> > 			offsetof(struct bpf_lpm_trie_key, data) + sizeof(uint32_t)*4,
> > 			sizeof(uint64_t),
> > 			...
> > 
> > The only risk to UAPI would be if sizeof() were used directly on the
> > data member, which it does not seem to be. It is only used as a static
> > initializer destination and to find its location via offsetof().
> > 
> > Link: https://lore.kernel.org/all/202206281009.4332AA33@keescook/ [1]
> > Reported-by: Mark Rutland <[e-mail]>
> > Closes: https://paste.debian.net/hidden/ca500597/
> 
> mmh... this URL expires: 2024-05-15

Yup, but that's why I included the run-time splat above too. :)

Daniel Borkmann Feb. 19, 2024, 5:48 p.m. UTC | #3

On 2/17/24 4:03 AM, Kees Cook wrote:
> On Fri, Feb 16, 2024 at 06:27:08PM -0600, Gustavo A. R. Silva wrote:
>> On 2/16/24 17:55, Kees Cook wrote:
>>> Replace deprecated 0-length array in struct bpf_lpm_trie_key with
>>> flexible array. Found with GCC 13:
>>>
>>> ../kernel/bpf/lpm_trie.c:207:51: warning: array subscript i is outside array bounds of 'const __u8[0]' {aka 'const unsigned char[]'} [-Warray-bounds=]
>>>     207 |                                        *(__be16 *)&key->data[i]);
>>>         |                                                   ^~~~~~~~~~~~~
>>> ../include/uapi/linux/swab.h:102:54: note: in definition of macro '__swab16'
>>>     102 | #define __swab16(x) (__u16)__builtin_bswap16((__u16)(x))
>>>         |                                                      ^
>>> ../include/linux/byteorder/generic.h:97:21: note: in expansion of macro '__be16_to_cpu'
>>>      97 | #define be16_to_cpu __be16_to_cpu
>>>         |                     ^~~~~~~~~~~~~
>>> ../kernel/bpf/lpm_trie.c:206:28: note: in expansion of macro 'be16_to_cpu'
>>>     206 |                 u16 diff = be16_to_cpu(*(__be16 *)&node->data[i]
>>> ^
>>>         |                            ^~~~~~~~~~~
>>> In file included from ../include/linux/bpf.h:7:
>>> ../include/uapi/linux/bpf.h:82:17: note: while referencing 'data'
>>>      82 |         __u8    data[0];        /* Arbitrary size */
>>>         |                 ^~~~
>>>
>>> And found at run-time under CONFIG_FORTIFY_SOURCE:
>>>
>>>     UBSAN: array-index-out-of-bounds in kernel/bpf/lpm_trie.c:218:49
>>>     index 0 is out of range for type '__u8 [*]'
>>>
>>> This includes fixing the selftest which was incorrectly using a
>>> variable length struct as a header, identified earlier[1]. Avoid this
>>> by just explicitly including the prefixlen member instead of struct
>>> bpf_lpm_trie_key.
>>>
>>> Note that it is not possible to simply remove the "data" member, as it
>>> is referenced by userspace
>>>
>>> cilium:
>>>           struct egress_gw_policy_key in_key = {
>>>                   .lpm_key = { 32 + 24, {} },
>>>                   .saddr   = CLIENT_IP,
>>>                   .daddr   = EXTERNAL_SVC_IP & 0Xffffff,
>>>           };
>>>
>>> systemd:
>>> 	ipv6_map_fd = bpf_map_new(
>>> 			BPF_MAP_TYPE_LPM_TRIE,
>>> 			offsetof(struct bpf_lpm_trie_key, data) + sizeof(uint32_t)*4,
>>> 			sizeof(uint64_t),
>>> 			...
>>>
>>> The only risk to UAPI would be if sizeof() were used directly on the
>>> data member, which it does not seem to be. It is only used as a static
>>> initializer destination and to find its location via offsetof().
>>>
>>> Link: https://lore.kernel.org/all/202206281009.4332AA33@keescook/ [1]
>>> Reported-by: Mark Rutland <[e-mail]>
>>> Closes: https://paste.debian.net/hidden/ca500597/
>>
>> mmh... this URL expires: 2024-05-15
> 
> Yup, but that's why I included the run-time splat above too. :)

I don't quite follow, this basically undoes 3024d95a4c52 ("bpf: Partially revert
flexible-array member replacement") again with the small change that this 'fixes'
up the BPF selftest to not embed struct bpf_lpm_trie_key.

Outside of BPF selftests though aren't we readding the same error that we fixed
earlier for BPF programs in the wild which embed struct bpf_lpm_trie_key into their
key structure?

Thanks,
Daniel

Kees Cook Feb. 19, 2024, 7:48 p.m. UTC | #4

On Mon, Feb 19, 2024 at 06:48:41PM +0100, Daniel Borkmann wrote:
> On 2/17/24 4:03 AM, Kees Cook wrote:
> > On Fri, Feb 16, 2024 at 06:27:08PM -0600, Gustavo A. R. Silva wrote:
> > > On 2/16/24 17:55, Kees Cook wrote:
> > > > Replace deprecated 0-length array in struct bpf_lpm_trie_key with
> > > > flexible array. Found with GCC 13:
> > > > 
> > > > ../kernel/bpf/lpm_trie.c:207:51: warning: array subscript i is outside array bounds of 'const __u8[0]' {aka 'const unsigned char[]'} [-Warray-bounds=]
> > > >     207 |                                        *(__be16 *)&key->data[i]);
> > > >         |                                                   ^~~~~~~~~~~~~
> > > > ../include/uapi/linux/swab.h:102:54: note: in definition of macro '__swab16'
> > > >     102 | #define __swab16(x) (__u16)__builtin_bswap16((__u16)(x))
> > > >         |                                                      ^
> > > > ../include/linux/byteorder/generic.h:97:21: note: in expansion of macro '__be16_to_cpu'
> > > >      97 | #define be16_to_cpu __be16_to_cpu
> > > >         |                     ^~~~~~~~~~~~~
> > > > ../kernel/bpf/lpm_trie.c:206:28: note: in expansion of macro 'be16_to_cpu'
> > > >     206 |                 u16 diff = be16_to_cpu(*(__be16 *)&node->data[i]
> > > > ^
> > > >         |                            ^~~~~~~~~~~
> > > > In file included from ../include/linux/bpf.h:7:
> > > > ../include/uapi/linux/bpf.h:82:17: note: while referencing 'data'
> > > >      82 |         __u8    data[0];        /* Arbitrary size */
> > > >         |                 ^~~~
> > > > 
> > > > And found at run-time under CONFIG_FORTIFY_SOURCE:
> > > > 
> > > >     UBSAN: array-index-out-of-bounds in kernel/bpf/lpm_trie.c:218:49
> > > >     index 0 is out of range for type '__u8 [*]'
> > > > 
> > > > This includes fixing the selftest which was incorrectly using a
> > > > variable length struct as a header, identified earlier[1]. Avoid this
> > > > by just explicitly including the prefixlen member instead of struct
> > > > bpf_lpm_trie_key.
> > > > 
> > > > Note that it is not possible to simply remove the "data" member, as it
> > > > is referenced by userspace
> > > > 
> > > > cilium:
> > > >           struct egress_gw_policy_key in_key = {
> > > >                   .lpm_key = { 32 + 24, {} },
> > > >                   .saddr   = CLIENT_IP,
> > > >                   .daddr   = EXTERNAL_SVC_IP & 0Xffffff,
> > > >           };
> > > > 
> > > > systemd:
> > > > 	ipv6_map_fd = bpf_map_new(
> > > > 			BPF_MAP_TYPE_LPM_TRIE,
> > > > 			offsetof(struct bpf_lpm_trie_key, data) + sizeof(uint32_t)*4,
> > > > 			sizeof(uint64_t),
> > > > 			...
> > > > 
> > > > The only risk to UAPI would be if sizeof() were used directly on the
> > > > data member, which it does not seem to be. It is only used as a static
> > > > initializer destination and to find its location via offsetof().
> > > > 
> > > > Link: https://lore.kernel.org/all/202206281009.4332AA33@keescook/ [1]
> > > > Reported-by: Mark Rutland <[e-mail]>
> > > > Closes: https://paste.debian.net/hidden/ca500597/
> > > 
> > > mmh... this URL expires: 2024-05-15
> > 
> > Yup, but that's why I included the run-time splat above too. :)
> 
> I don't quite follow, this basically undoes 3024d95a4c52 ("bpf: Partially revert
> flexible-array member replacement") again with the small change that this 'fixes'
> up the BPF selftest to not embed struct bpf_lpm_trie_key.
>
> Outside of BPF selftests though aren't we readding the same error that we fixed
> earlier for BPF programs in the wild which embed struct bpf_lpm_trie_key into their
> key structure?

Oops, yes, sorry. I see how that cilium does include it in the same
fashion. I will adjust this patch again. Thanks for double-checking!

struct egress_gw_policy_key {
        struct bpf_lpm_trie_key lpm_key;
        __u32 saddr;
        __u32 daddr;
};

Patch

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 754e68ca8744..359dd8a429c1 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -80,7 +80,7 @@  struct bpf_insn {
 /* Key of an a BPF_MAP_TYPE_LPM_TRIE entry */
 struct bpf_lpm_trie_key {
 	__u32	prefixlen;	/* up to 32 for AF_INET, 128 for AF_INET6 */
-	__u8	data[0];	/* Arbitrary size */
+	__u8	data[];		/* Arbitrary size */
 };
 
 struct bpf_cgroup_storage_key {
diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c
index 3325da17ec81..1d476c6ae284 100644
--- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c
+++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c
@@ -316,7 +316,7 @@  struct lpm_trie {
 } __attribute__((preserve_access_index));
 
 struct lpm_key {
-	struct bpf_lpm_trie_key trie_key;
+	__u32 prefixlen;
 	__u32 data;
 };