Message ID | 20230113-sockmap-fix-v1-1-d3cad092ee10@cloudflare.com (mailing list archive) |
---|---|
State | Superseded |
Delegated to: | BPF |
Headers | show |
Series | bpf, sockmap: Fix infinite recursion in sock_map_close | expand |
Hi Jakub, url: https://github.com/intel-lab-lkp/linux/commits/Jakub-Sitnicki/bpf-sockmap-Check-for-any-of-tcp_bpf_prots-when-cloning-a-listener/20230113-230728 base: e7895f017b79410bf4591396a733b876dc1e0e9d patch link: https://lore.kernel.org/r/20230113-sockmap-fix-v1-1-d3cad092ee10%40cloudflare.com patch subject: [PATCH bpf 1/3] bpf, sockmap: Check for any of tcp_bpf_prots when cloning a listener config: i386-randconfig-m021 compiler: gcc-11 (Debian 11.3.0-8) 11.3.0 If you fix the issue, kindly add following tag where applicable | Reported-by: kernel test robot <lkp@intel.com> | Reported-by: Dan Carpenter <error27@gmail.com> smatch warnings: net/ipv4/tcp_bpf.c:644 tcp_bpf_clone() error: buffer overflow 'tcp_bpf_prots' 2 <= 2 vim +/tcp_bpf_prots +644 net/ipv4/tcp_bpf.c 604326b41a6fb9 Daniel Borkmann 2018-10-13 634 e80251555f0bef Jakub Sitnicki 2020-02-18 635 /* If a child got cloned from a listening socket that had tcp_bpf e80251555f0bef Jakub Sitnicki 2020-02-18 636 * protocol callbacks installed, we need to restore the callbacks to e80251555f0bef Jakub Sitnicki 2020-02-18 637 * the default ones because the child does not inherit the psock state e80251555f0bef Jakub Sitnicki 2020-02-18 638 * that tcp_bpf callbacks expect. e80251555f0bef Jakub Sitnicki 2020-02-18 639 */ e80251555f0bef Jakub Sitnicki 2020-02-18 640 void tcp_bpf_clone(const struct sock *sk, struct sock *newsk) e80251555f0bef Jakub Sitnicki 2020-02-18 641 { e80251555f0bef Jakub Sitnicki 2020-02-18 642 struct proto *prot = newsk->sk_prot; e80251555f0bef Jakub Sitnicki 2020-02-18 643 c2e74657613125 Jakub Sitnicki 2023-01-13 @644 if (tcp_bpf_prots[0] <= prot && prot < tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ What? Also I suspect this might cause a compile error for Clang builds.
Hi Dan, On Sat, Jan 14, 2023 at 11:04 AM +03, Dan Carpenter wrote: > Hi Jakub, > > url: https://github.com/intel-lab-lkp/linux/commits/Jakub-Sitnicki/bpf-sockmap-Check-for-any-of-tcp_bpf_prots-when-cloning-a-listener/20230113-230728 > base: e7895f017b79410bf4591396a733b876dc1e0e9d > patch link: https://lore.kernel.org/r/20230113-sockmap-fix-v1-1-d3cad092ee10%40cloudflare.com > patch subject: [PATCH bpf 1/3] bpf, sockmap: Check for any of tcp_bpf_prots when cloning a listener > config: i386-randconfig-m021 > compiler: gcc-11 (Debian 11.3.0-8) 11.3.0 > > If you fix the issue, kindly add following tag where applicable > | Reported-by: kernel test robot <lkp@intel.com> > | Reported-by: Dan Carpenter <error27@gmail.com> > > smatch warnings: > net/ipv4/tcp_bpf.c:644 tcp_bpf_clone() error: buffer overflow 'tcp_bpf_prots' 2 <= 2 > > vim +/tcp_bpf_prots +644 net/ipv4/tcp_bpf.c > > 604326b41a6fb9 Daniel Borkmann 2018-10-13 634 > e80251555f0bef Jakub Sitnicki 2020-02-18 635 /* If a child got cloned from a listening socket that had tcp_bpf > e80251555f0bef Jakub Sitnicki 2020-02-18 636 * protocol callbacks installed, we need to restore the callbacks to > e80251555f0bef Jakub Sitnicki 2020-02-18 637 * the default ones because the child does not inherit the psock state > e80251555f0bef Jakub Sitnicki 2020-02-18 638 * that tcp_bpf callbacks expect. > e80251555f0bef Jakub Sitnicki 2020-02-18 639 */ > e80251555f0bef Jakub Sitnicki 2020-02-18 640 void tcp_bpf_clone(const struct sock *sk, struct sock *newsk) > e80251555f0bef Jakub Sitnicki 2020-02-18 641 { > e80251555f0bef Jakub Sitnicki 2020-02-18 642 struct proto *prot = newsk->sk_prot; > e80251555f0bef Jakub Sitnicki 2020-02-18 643 > c2e74657613125 Jakub Sitnicki 2023-01-13 @644 if (tcp_bpf_prots[0] <= prot && prot < tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)]) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > What? Also I suspect this might cause a compile error for Clang builds. Can't say I see a problem B-) tcp_bpf_prots is a 2D array: static struct proto tcp_bpf_prots[TCP_BPF_NUM_PROTS][TCP_BPF_NUM_CFGS]; ... so tcp_bpf_prots[0] is the base address of the first array, while tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)] is the base address of the array one past the last one. Smatch doesn't seem to graps the 2D array concept here. We can make it happy by being explicit but harder on the eyes: if (&tcp_bpf_prots[0][0] <= prot && prot < &tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)][0]) newsk->sk_prot = sk->sk_prot_creator; Clang can do pointer arithmetic on 2D arrays just fine :-)
On Mon, Jan 16, 2023 at 11:13 AM Jakub Sitnicki <jakub@cloudflare.com> wrote: > > Hi Dan, > > On Sat, Jan 14, 2023 at 11:04 AM +03, Dan Carpenter wrote: > > Hi Jakub, > > > > url: https://github.com/intel-lab-lkp/linux/commits/Jakub-Sitnicki/bpf-sockmap-Check-for-any-of-tcp_bpf_prots-when-cloning-a-listener/20230113-230728 > > base: e7895f017b79410bf4591396a733b876dc1e0e9d > > patch link: https://lore.kernel.org/r/20230113-sockmap-fix-v1-1-d3cad092ee10%40cloudflare.com > > patch subject: [PATCH bpf 1/3] bpf, sockmap: Check for any of tcp_bpf_prots when cloning a listener > > config: i386-randconfig-m021 > > compiler: gcc-11 (Debian 11.3.0-8) 11.3.0 > > > > If you fix the issue, kindly add following tag where applicable > > | Reported-by: kernel test robot <lkp@intel.com> > > | Reported-by: Dan Carpenter <error27@gmail.com> > > > > smatch warnings: > > net/ipv4/tcp_bpf.c:644 tcp_bpf_clone() error: buffer overflow 'tcp_bpf_prots' 2 <= 2 > > > > vim +/tcp_bpf_prots +644 net/ipv4/tcp_bpf.c > > > > 604326b41a6fb9 Daniel Borkmann 2018-10-13 634 > > e80251555f0bef Jakub Sitnicki 2020-02-18 635 /* If a child got cloned from a listening socket that had tcp_bpf > > e80251555f0bef Jakub Sitnicki 2020-02-18 636 * protocol callbacks installed, we need to restore the callbacks to > > e80251555f0bef Jakub Sitnicki 2020-02-18 637 * the default ones because the child does not inherit the psock state > > e80251555f0bef Jakub Sitnicki 2020-02-18 638 * that tcp_bpf callbacks expect. > > e80251555f0bef Jakub Sitnicki 2020-02-18 639 */ > > e80251555f0bef Jakub Sitnicki 2020-02-18 640 void tcp_bpf_clone(const struct sock *sk, struct sock *newsk) > > e80251555f0bef Jakub Sitnicki 2020-02-18 641 { > > e80251555f0bef Jakub Sitnicki 2020-02-18 642 struct proto *prot = newsk->sk_prot; > > e80251555f0bef Jakub Sitnicki 2020-02-18 643 > > c2e74657613125 Jakub Sitnicki 2023-01-13 @644 if (tcp_bpf_prots[0] <= prot && prot < tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)]) > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > What? Also I suspect this might cause a compile error for Clang builds. > > Can't say I see a problem B-) > > tcp_bpf_prots is a 2D array: > > static struct proto tcp_bpf_prots[TCP_BPF_NUM_PROTS][TCP_BPF_NUM_CFGS]; > > ... so tcp_bpf_prots[0] is the base address of the first array, while > tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)] is the base address of the > array one past the last one. > > Smatch doesn't seem to graps the 2D array concept here. We can make it > happy by being explicit but harder on the eyes: > > if (&tcp_bpf_prots[0][0] <= prot && prot < &tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)][0]) > newsk->sk_prot = sk->sk_prot_creator; > > Clang can do pointer arithmetic on 2D arrays just fine :-) We might add a generic helper to make all this a bit more clear ? +#define is_insidevar(PTR, VAR) ( \ + ((void *)(PTR) <= (void *)(VAR)) && \ + ((void *)(PTR) <= (void *)(VAR) + sizeof(VAR))) + ... if (is_insidevar(prot, tcp_bpf_prots)) newsk->sk_prot = sk->sk_prot_creator;
On Mon, Jan 16, 2023 at 11:09:02AM +0100, Jakub Sitnicki wrote: > Hi Dan, > > On Sat, Jan 14, 2023 at 11:04 AM +03, Dan Carpenter wrote: > > Hi Jakub, > > > > url: https://github.com/intel-lab-lkp/linux/commits/Jakub-Sitnicki/bpf-sockmap-Check-for-any-of-tcp_bpf_prots-when-cloning-a-listener/20230113-230728 > > base: e7895f017b79410bf4591396a733b876dc1e0e9d > > patch link: https://lore.kernel.org/r/20230113-sockmap-fix-v1-1-d3cad092ee10%40cloudflare.com > > patch subject: [PATCH bpf 1/3] bpf, sockmap: Check for any of tcp_bpf_prots when cloning a listener > > config: i386-randconfig-m021 > > compiler: gcc-11 (Debian 11.3.0-8) 11.3.0 > > > > If you fix the issue, kindly add following tag where applicable > > | Reported-by: kernel test robot <lkp@intel.com> > > | Reported-by: Dan Carpenter <error27@gmail.com> > > > > smatch warnings: > > net/ipv4/tcp_bpf.c:644 tcp_bpf_clone() error: buffer overflow 'tcp_bpf_prots' 2 <= 2 > > > > vim +/tcp_bpf_prots +644 net/ipv4/tcp_bpf.c > > > > 604326b41a6fb9 Daniel Borkmann 2018-10-13 634 > > e80251555f0bef Jakub Sitnicki 2020-02-18 635 /* If a child got cloned from a listening socket that had tcp_bpf > > e80251555f0bef Jakub Sitnicki 2020-02-18 636 * protocol callbacks installed, we need to restore the callbacks to > > e80251555f0bef Jakub Sitnicki 2020-02-18 637 * the default ones because the child does not inherit the psock state > > e80251555f0bef Jakub Sitnicki 2020-02-18 638 * that tcp_bpf callbacks expect. > > e80251555f0bef Jakub Sitnicki 2020-02-18 639 */ > > e80251555f0bef Jakub Sitnicki 2020-02-18 640 void tcp_bpf_clone(const struct sock *sk, struct sock *newsk) > > e80251555f0bef Jakub Sitnicki 2020-02-18 641 { > > e80251555f0bef Jakub Sitnicki 2020-02-18 642 struct proto *prot = newsk->sk_prot; > > e80251555f0bef Jakub Sitnicki 2020-02-18 643 > > c2e74657613125 Jakub Sitnicki 2023-01-13 @644 if (tcp_bpf_prots[0] <= prot && prot < tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)]) > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > What? Also I suspect this might cause a compile error for Clang builds. > > Can't say I see a problem B-) > > tcp_bpf_prots is a 2D array: > > static struct proto tcp_bpf_prots[TCP_BPF_NUM_PROTS][TCP_BPF_NUM_CFGS]; > > ... so tcp_bpf_prots[0] is the base address of the first array, while > tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)] is the base address of the > array one past the last one. > > Smatch doesn't seem to graps the 2D array concept here. We can make it > happy by being explicit but harder on the eyes: > > if (&tcp_bpf_prots[0][0] <= prot && prot < &tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)][0]) > newsk->sk_prot = sk->sk_prot_creator; Huh. I can silence this false positive in Smatch... It never even occured to me that this was a two dimensional array (I only have the information in the email). > > Clang can do pointer arithmetic on 2D arrays just fine :-) Heh. I must have an older version of Clang. CC net/ipv4/tcp_bpf.o net/ipv4/tcp_bpf.c:644:41: warning: array index 2 is past the end of the array (that has type 'struct proto[2][4]') [-Warray-bounds] if (tcp_bpf_prots[0] <= prot && prot < tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)]) ^ ~~~~~~~~~~~~~~~~~~~~~~~~~ net/ipv4/tcp_bpf.c:544:1: note: array 'tcp_bpf_prots' declared here static struct proto tcp_bpf_prots[TCP_BPF_NUM_PROTS][TCP_BPF_NUM_CFGS]; ^ 1 warning generated. regards, dan carpetner
On Mon, Jan 16, 2023 at 11:39 AM +01, Eric Dumazet wrote: > We might add a generic helper to make all this a bit more clear ? > > +#define is_insidevar(PTR, VAR) ( \ > + ((void *)(PTR) <= (void *)(VAR)) && \ > + ((void *)(PTR) <= (void *)(VAR) + sizeof(VAR))) > + > > > ... > > if (is_insidevar(prot, tcp_bpf_prots)) > newsk->sk_prot = sk->sk_prot_creator; Sure can do. Thanks for the suggestion. I adjusted it a bit: - added cast to char * so we don't offend -Wpointer-arith, - fixed the lower and upper bound check. Final form would look like: #define is_insidevar(ptr, var) \ ((void *)(ptr) >= (void *)(var)) && \ ((void *)(ptr) < (void *)((char *)(var) + sizeof(var))) Not sure where to stuff it. I propose include/linux/util_macros.h.
On Mon, Jan 16, 2023 at 02:13 PM +03, Dan Carpenter wrote: > On Mon, Jan 16, 2023 at 11:09:02AM +0100, Jakub Sitnicki wrote: [...] >> Smatch doesn't seem to graps the 2D array concept here. We can make it >> happy by being explicit but harder on the eyes: >> >> if (&tcp_bpf_prots[0][0] <= prot && prot < &tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)][0]) >> newsk->sk_prot = sk->sk_prot_creator; > > Huh. I can silence this false positive in Smatch... It never even > occured to me that this was a two dimensional array (I only have the > information in the email). > No need. Eric's macro helper makes Smatch happy. I'll use it in v2. >> >> Clang can do pointer arithmetic on 2D arrays just fine :-) > > Heh. I must have an older version of Clang. > > CC net/ipv4/tcp_bpf.o > net/ipv4/tcp_bpf.c:644:41: warning: array index 2 is past the end of the array (that has type 'struct proto[2][4]') [-Warray-bounds] > if (tcp_bpf_prots[0] <= prot && prot < tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)]) > ^ ~~~~~~~~~~~~~~~~~~~~~~~~~ > net/ipv4/tcp_bpf.c:544:1: note: array 'tcp_bpf_prots' declared here > static struct proto tcp_bpf_prots[TCP_BPF_NUM_PROTS][TCP_BPF_NUM_CFGS]; > ^ > 1 warning generated. FWIW, I've checked against: $ clang --version clang version 15.0.6 (Fedora 15.0.6-2.fc37) Gotta keep it fresh to be able to build bpf selftests ;-) But I sure don't want to break builds with older Clangs. Thanks for pointing it out.
On Mon, Jan 16, 2023 at 12:31:11PM +0100, Jakub Sitnicki wrote: > >> Clang can do pointer arithmetic on 2D arrays just fine :-) > > > > Heh. I must have an older version of Clang. > > > > CC net/ipv4/tcp_bpf.o > > net/ipv4/tcp_bpf.c:644:41: warning: array index 2 is past the end of the array (that has type 'struct proto[2][4]') [-Warray-bounds] > > if (tcp_bpf_prots[0] <= prot && prot < tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)]) > > ^ ~~~~~~~~~~~~~~~~~~~~~~~~~ > > net/ipv4/tcp_bpf.c:544:1: note: array 'tcp_bpf_prots' declared here > > static struct proto tcp_bpf_prots[TCP_BPF_NUM_PROTS][TCP_BPF_NUM_CFGS]; > > ^ > > 1 warning generated. > > FWIW, I've checked against: > > $ clang --version > clang version 15.0.6 (Fedora 15.0.6-2.fc37) > > Gotta keep it fresh to be able to build bpf selftests ;-) > But I sure don't want to break builds with older Clangs. I'm actually on a newer 16.x something version from git. Btw, it made me outrageously happy that Clang was one for one bug compatible with Smatch on this. With this kind of warning you could either print a warning when there is a read but that's not what either Smatch or Clang do. Smatch looks at the offset and then checks to see if the code is just doing pointer math to find the &(array + 1) address. So Smatch checks is the offset known to be exactly ARRAY_SIZE() and are we taking the address of that. I have updated that check. regards, dan carpenter
diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index 94aad3870c5f..8db00647e0a4 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -639,10 +639,9 @@ EXPORT_SYMBOL_GPL(tcp_bpf_update_proto); */ void tcp_bpf_clone(const struct sock *sk, struct sock *newsk) { - int family = sk->sk_family == AF_INET6 ? TCP_BPF_IPV6 : TCP_BPF_IPV4; struct proto *prot = newsk->sk_prot; - if (prot == &tcp_bpf_prots[family][TCP_BPF_BASE]) + if (tcp_bpf_prots[0] <= prot && prot < tcp_bpf_prots[ARRAY_SIZE(tcp_bpf_prots)]) newsk->sk_prot = sk->sk_prot_creator; } #endif /* CONFIG_BPF_SYSCALL */
A listening socket linked to a sockmap has its sk_prot overridden. It points to one of the struct proto variants in tcp_bpf_prots. The variant depends on the socket's family and which sockmap programs are attached. A child socket cloned from a TCP listener initially inherits their sk_prot. But before cloning is finished, we restore the child's proto to the listener's original non-tcp_bpf_prots one. This happens in tcp_create_openreq_child -> tcp_bpf_clone. Today, in tcp_bpf_clone we detect if the child's proto should be restored by checking only for the TCP_BPF_BASE proto variant. This is not correct. The sk_prot of listening socket linked to a sockmap can point to to any variant in tcp_bpf_prots. If the listeners sk_prot happens to be not the TCP_BPF_BASE variant, then the child socket unintentionally is left if the inherited sk_prot by tcp_bpf_clone. This leads to issues like infinite recursion on close [1], because the child state is otherwise not set up for use with tcp_bpf_prot operations. Adjust the check in tcp_bpf_clone to detect all of tcp_bpf_prots variants. Note that it wouldn't be sufficient to check the socket state when overriding the sk_prot in tcp_bpf_update_proto in order to always use the TCP_BPF_BASE variant for listening sockets. Since commit b8b8315e39ff ("bpf, sockmap: Remove unhash handler for BPF sockmap usage") it is possible for a socket to transition to TCP_LISTEN state while already linked to a sockmap, e.g. connect() -> insert into map -> connect(AF_UNSPEC) -> listen(). [1]: https://lore.kernel.org/all/00000000000073b14905ef2e7401@google.com/ Fixes: e80251555f0b ("tcp_bpf: Don't let child socket inherit parent protocol ops on copy") Reported-by: syzbot+04c21ed96d861dccc5cd@syzkaller.appspotmail.com Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com> --- net/ipv4/tcp_bpf.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)