| Message ID | 21423fd3-708f-fa57-c496-e50057db78e0@163.com (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Delegated to: | Paolo Abeni |
| Headers | show |
| Series | mptcp: fix warning in __skb_flow_dissect() when do syn cookie for subflow join | expand |
Hello, On Sat, 2021-06-05 at 11:24 +0800, Jianguo Wu wrote: > From: Jianguo Wu <wujianguo@chinatelecom.cn> > > I got the following warning message while doing the test: > > [ 55.552626] TCP: request_sock_subflow: Possible SYN flooding on port 8099. Sending cookies. Check SNMP counters. > [ 55.553024] ------------[ cut here ]------------ > [ 55.553027] WARNING: CPU: 0 PID: 10 at net/core/flow_dissector.c:984 __skb_flow_dissect+0x280/0x1650 > ... > [ 55.553117] CPU: 0 PID: 10 Comm: ksoftirqd/0 Not tainted 5.12.0+ #18 > [ 55.553121] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020 > [ 55.553124] RIP: 0010:__skb_flow_dissect+0x280/0x1650 > ... > [ 55.553133] RSP: 0018:ffffb79580087770 EFLAGS: 00010246 > [ 55.553137] RAX: 0000000000000000 RBX: ffffffff8ddb58e0 RCX: ffffb79580087888 > [ 55.553139] RDX: ffffffff8ddb58e0 RSI: ffff8f7e4652b600 RDI: 0000000000000000 > [ 55.553141] RBP: ffffb79580087858 R08: 0000000000000000 R09: 0000000000000008 > [ 55.553143] R10: 000000008c622965 R11: 00000000d3313a5b R12: ffff8f7e4652b600 > [ 55.553146] R13: ffff8f7e465c9062 R14: 0000000000000000 R15: ffffb79580087888 > [ 55.553149] FS: 0000000000000000(0000) GS:ffff8f7f75e00000(0000) knlGS:0000000000000000 > [ 55.553152] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 55.553154] CR2: 00007f73d1d19000 CR3: 0000000135e10004 CR4: 00000000003706f0 > [ 55.553160] Call Trace: > [ 55.553166] ? __sha256_final+0x67/0xd0 > [ 55.553173] ? sha256+0x7e/0xa0 > [ 55.553177] __skb_get_hash+0x57/0x210 > [ 55.553182] subflow_init_req_cookie_join_save+0xac/0xc0 > [ 55.553189] subflow_check_req+0x474/0x550 > [ 55.553195] ? ip_route_output_key_hash+0x67/0x90 > [ 55.553200] ? xfrm_lookup_route+0x1d/0xa0 > [ 55.553207] subflow_v4_route_req+0x8e/0xd0 > [ 55.553212] tcp_conn_request+0x31e/0xab0 > [ 55.553218] ? selinux_socket_sock_rcv_skb+0x116/0x210 > [ 55.553224] ? tcp_rcv_state_process+0x179/0x6d0 > [ 55.553229] tcp_rcv_state_process+0x179/0x6d0 > [ 55.553235] tcp_v4_do_rcv+0xaf/0x220 > [ 55.553239] tcp_v4_rcv+0xce4/0xd80 > [ 55.553243] ? ip_route_input_rcu+0x246/0x260 > [ 55.553248] ip_protocol_deliver_rcu+0x35/0x1b0 > [ 55.553253] ip_local_deliver_finish+0x44/0x50 > [ 55.553258] ip_local_deliver+0x6c/0x110 > [ 55.553262] ? ip_rcv_finish_core.isra.19+0x5a/0x400 > [ 55.553267] ip_rcv+0xd1/0xe0 > ... > > After debugging, I found in __skb_flow_dissect(), skb->dev and skb->sk are both NULL, > then net is NULL, and trigger WARN_ON_ONCE(!net), actually net is always NULL in this > code path. > > Code snippet in __skb_flow_dissect() that trigger warning: > 975 if (skb) { > 976 if (!net) { > 977 if (skb->dev) > 978 net = dev_net(skb->dev); > 979 else if (skb->sk) > 980 net = sock_net(skb->sk); > 981 } > 982 } > 983 > 984 WARN_ON_ONCE(!net); > > skb->dev is set to NULL in tcp_v4_rcv(), and skb->sk is never set in this code path. > But for sk_state is TCP_NEW_SYN_RECV or TCP_TIME_WAIT, skb->dev is not set to NULL, > so we can remove skb->dev = NULL in tcp_v4_rcv()? Thank you for the detailed report! I'm unable to find easily any negative side effect to remove such assignment, still I'm more than a bit uneasy for such change. The issue is clearly an mptcp-specific one, so I think event the fix should be mptcp-specific. If the skb->hash is not available - !skb_get_hash_raw(skb) == 0, then we could fallback using an sk-derived hash, e.g. doing something alike what inet_ehashfn() does - with different seeds/secrets. Cheers, Paolo
Hi Paolo, On 2021/6/7 19:07, Paolo Abeni wrote: > Hello, > > On Sat, 2021-06-05 at 11:24 +0800, Jianguo Wu wrote: >> From: Jianguo Wu <wujianguo@chinatelecom.cn> >> >> I got the following warning message while doing the test: >> >> [ 55.552626] TCP: request_sock_subflow: Possible SYN flooding on port 8099. Sending cookies. Check SNMP counters. >> [ 55.553024] ------------[ cut here ]------------ >> [ 55.553027] WARNING: CPU: 0 PID: 10 at net/core/flow_dissector.c:984 __skb_flow_dissect+0x280/0x1650 >> ... >> [ 55.553117] CPU: 0 PID: 10 Comm: ksoftirqd/0 Not tainted 5.12.0+ #18 >> [ 55.553121] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020 >> [ 55.553124] RIP: 0010:__skb_flow_dissect+0x280/0x1650 >> ... >> [ 55.553133] RSP: 0018:ffffb79580087770 EFLAGS: 00010246 >> [ 55.553137] RAX: 0000000000000000 RBX: ffffffff8ddb58e0 RCX: ffffb79580087888 >> [ 55.553139] RDX: ffffffff8ddb58e0 RSI: ffff8f7e4652b600 RDI: 0000000000000000 >> [ 55.553141] RBP: ffffb79580087858 R08: 0000000000000000 R09: 0000000000000008 >> [ 55.553143] R10: 000000008c622965 R11: 00000000d3313a5b R12: ffff8f7e4652b600 >> [ 55.553146] R13: ffff8f7e465c9062 R14: 0000000000000000 R15: ffffb79580087888 >> [ 55.553149] FS: 0000000000000000(0000) GS:ffff8f7f75e00000(0000) knlGS:0000000000000000 >> [ 55.553152] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> [ 55.553154] CR2: 00007f73d1d19000 CR3: 0000000135e10004 CR4: 00000000003706f0 >> [ 55.553160] Call Trace: >> [ 55.553166] ? __sha256_final+0x67/0xd0 >> [ 55.553173] ? sha256+0x7e/0xa0 >> [ 55.553177] __skb_get_hash+0x57/0x210 >> [ 55.553182] subflow_init_req_cookie_join_save+0xac/0xc0 >> [ 55.553189] subflow_check_req+0x474/0x550 >> [ 55.553195] ? ip_route_output_key_hash+0x67/0x90 >> [ 55.553200] ? xfrm_lookup_route+0x1d/0xa0 >> [ 55.553207] subflow_v4_route_req+0x8e/0xd0 >> [ 55.553212] tcp_conn_request+0x31e/0xab0 >> [ 55.553218] ? selinux_socket_sock_rcv_skb+0x116/0x210 >> [ 55.553224] ? tcp_rcv_state_process+0x179/0x6d0 >> [ 55.553229] tcp_rcv_state_process+0x179/0x6d0 >> [ 55.553235] tcp_v4_do_rcv+0xaf/0x220 >> [ 55.553239] tcp_v4_rcv+0xce4/0xd80 >> [ 55.553243] ? ip_route_input_rcu+0x246/0x260 >> [ 55.553248] ip_protocol_deliver_rcu+0x35/0x1b0 >> [ 55.553253] ip_local_deliver_finish+0x44/0x50 >> [ 55.553258] ip_local_deliver+0x6c/0x110 >> [ 55.553262] ? ip_rcv_finish_core.isra.19+0x5a/0x400 >> [ 55.553267] ip_rcv+0xd1/0xe0 >> ... >> >> After debugging, I found in __skb_flow_dissect(), skb->dev and skb->sk are both NULL, >> then net is NULL, and trigger WARN_ON_ONCE(!net), actually net is always NULL in this >> code path. >> >> Code snippet in __skb_flow_dissect() that trigger warning: >> 975 if (skb) { >> 976 if (!net) { >> 977 if (skb->dev) >> 978 net = dev_net(skb->dev); >> 979 else if (skb->sk) >> 980 net = sock_net(skb->sk); >> 981 } >> 982 } >> 983 >> 984 WARN_ON_ONCE(!net); >> >> skb->dev is set to NULL in tcp_v4_rcv(), and skb->sk is never set in this code path. >> But for sk_state is TCP_NEW_SYN_RECV or TCP_TIME_WAIT, skb->dev is not set to NULL, >> so we can remove skb->dev = NULL in tcp_v4_rcv()? > > Thank you for the detailed report! > > I'm unable to find easily any negative side effect to remove such > assignment, still I'm more than a bit uneasy for such change. > > The issue is clearly an mptcp-specific one, so I think event the fix > should be mptcp-specific. > > If the skb->hash is not available - !skb_get_hash_raw(skb) == 0, then > we could fallback using an sk-derived hash, e.g. doing something alike > what inet_ehashfn() does - with different seeds/secrets. > I will try to prepare a new patch as your suggestion, Thanks! > Cheers, > > Paolo >
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 4f5b68a..b1f94a4 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -2061,8 +2061,6 @@ int tcp_v4_rcv(struct sk_buff *skb) iph = ip_hdr(skb); tcp_v4_fill_cb(skb, iph, th); - skb->dev = NULL; - if (sk->sk_state == TCP_LISTEN) { ret = tcp_v4_do_rcv(sk, skb); goto put_and_return;
From: Jianguo Wu <wujianguo@chinatelecom.cn> I got the following warning message while doing the test: [ 55.552626] TCP: request_sock_subflow: Possible SYN flooding on port 8099. Sending cookies. Check SNMP counters. [ 55.553024] ------------[ cut here ]------------ [ 55.553027] WARNING: CPU: 0 PID: 10 at net/core/flow_dissector.c:984 __skb_flow_dissect+0x280/0x1650 ... [ 55.553117] CPU: 0 PID: 10 Comm: ksoftirqd/0 Not tainted 5.12.0+ #18 [ 55.553121] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020 [ 55.553124] RIP: 0010:__skb_flow_dissect+0x280/0x1650 ... [ 55.553133] RSP: 0018:ffffb79580087770 EFLAGS: 00010246 [ 55.553137] RAX: 0000000000000000 RBX: ffffffff8ddb58e0 RCX: ffffb79580087888 [ 55.553139] RDX: ffffffff8ddb58e0 RSI: ffff8f7e4652b600 RDI: 0000000000000000 [ 55.553141] RBP: ffffb79580087858 R08: 0000000000000000 R09: 0000000000000008 [ 55.553143] R10: 000000008c622965 R11: 00000000d3313a5b R12: ffff8f7e4652b600 [ 55.553146] R13: ffff8f7e465c9062 R14: 0000000000000000 R15: ffffb79580087888 [ 55.553149] FS: 0000000000000000(0000) GS:ffff8f7f75e00000(0000) knlGS:0000000000000000 [ 55.553152] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 55.553154] CR2: 00007f73d1d19000 CR3: 0000000135e10004 CR4: 00000000003706f0 [ 55.553160] Call Trace: [ 55.553166] ? __sha256_final+0x67/0xd0 [ 55.553173] ? sha256+0x7e/0xa0 [ 55.553177] __skb_get_hash+0x57/0x210 [ 55.553182] subflow_init_req_cookie_join_save+0xac/0xc0 [ 55.553189] subflow_check_req+0x474/0x550 [ 55.553195] ? ip_route_output_key_hash+0x67/0x90 [ 55.553200] ? xfrm_lookup_route+0x1d/0xa0 [ 55.553207] subflow_v4_route_req+0x8e/0xd0 [ 55.553212] tcp_conn_request+0x31e/0xab0 [ 55.553218] ? selinux_socket_sock_rcv_skb+0x116/0x210 [ 55.553224] ? tcp_rcv_state_process+0x179/0x6d0 [ 55.553229] tcp_rcv_state_process+0x179/0x6d0 [ 55.553235] tcp_v4_do_rcv+0xaf/0x220 [ 55.553239] tcp_v4_rcv+0xce4/0xd80 [ 55.553243] ? ip_route_input_rcu+0x246/0x260 [ 55.553248] ip_protocol_deliver_rcu+0x35/0x1b0 [ 55.553253] ip_local_deliver_finish+0x44/0x50 [ 55.553258] ip_local_deliver+0x6c/0x110 [ 55.553262] ? ip_rcv_finish_core.isra.19+0x5a/0x400 [ 55.553267] ip_rcv+0xd1/0xe0 ... After debugging, I found in __skb_flow_dissect(), skb->dev and skb->sk are both NULL, then net is NULL, and trigger WARN_ON_ONCE(!net), actually net is always NULL in this code path. Code snippet in __skb_flow_dissect() that trigger warning: 975 if (skb) { 976 if (!net) { 977 if (skb->dev) 978 net = dev_net(skb->dev); 979 else if (skb->sk) 980 net = sock_net(skb->sk); 981 } 982 } 983 984 WARN_ON_ONCE(!net); skb->dev is set to NULL in tcp_v4_rcv(), and skb->sk is never set in this code path. But for sk_state is TCP_NEW_SYN_RECV or TCP_TIME_WAIT, skb->dev is not set to NULL, so we can remove skb->dev = NULL in tcp_v4_rcv()? Fixes: 9466a1ccebbe("mptcp: enable JOIN requests even if cookies are in use"). --- net/ipv4/tcp_ipv4.c | 2 -- 1 file changed, 2 deletions(-)