| Message ID | 20240718134348.289865-1-jchapman@katalix.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | d587d825424b6784ff7d7519efd1b827b09aa501 |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [net] l2tp: make session IDR and tunnel session list coherent | expand |
On Thu, Jul 18, 2024 at 02:43:48PM +0100, James Chapman wrote: > Modify l2tp_session_register and l2tp_session_unhash so that the > session IDR and tunnel session lists remain coherent. To do so, hold > the session IDR lock and the tunnel's session list lock when making > any changes to either list. > > Without this change, a rare race condition could hit the WARN_ON_ONCE > in l2tp_session_unhash if a thread replaced the IDR entry while > another thread was registering the same ID. > > [ 7126.151795][T17511] WARNING: CPU: 3 PID: 17511 at net/l2tp/l2tp_core.c:1282 l2tp_session_delete.part.0+0x87e/0xbc0 > [ 7126.163754][T17511] ? show_regs+0x93/0xa0 > [ 7126.164157][T17511] ? __warn+0xe5/0x3c0 > [ 7126.164536][T17511] ? l2tp_session_delete.part.0+0x87e/0xbc0 > [ 7126.165070][T17511] ? report_bug+0x2e1/0x500 > [ 7126.165486][T17511] ? l2tp_session_delete.part.0+0x87e/0xbc0 > [ 7126.166013][T17511] ? handle_bug+0x99/0x130 > [ 7126.166428][T17511] ? exc_invalid_op+0x35/0x80 > [ 7126.166890][T17511] ? asm_exc_invalid_op+0x1a/0x20 > [ 7126.167372][T17511] ? l2tp_session_delete.part.0+0x87d/0xbc0 > [ 7126.167900][T17511] ? l2tp_session_delete.part.0+0x87e/0xbc0 > [ 7126.168429][T17511] ? __local_bh_enable_ip+0xa4/0x120 > [ 7126.168917][T17511] l2tp_session_delete+0x40/0x50 > [ 7126.169369][T17511] pppol2tp_release+0x1a1/0x3f0 > [ 7126.169817][T17511] __sock_release+0xb3/0x270 > [ 7126.170247][T17511] ? __pfx_sock_close+0x10/0x10 > [ 7126.170697][T17511] sock_close+0x1c/0x30 > [ 7126.171087][T17511] __fput+0x40b/0xb90 > [ 7126.171470][T17511] task_work_run+0x16c/0x260 > [ 7126.171897][T17511] ? __pfx_task_work_run+0x10/0x10 > [ 7126.172362][T17511] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 7126.172863][T17511] ? do_raw_spin_unlock+0x174/0x230 > [ 7126.173348][T17511] do_exit+0xaae/0x2b40 > [ 7126.173730][T17511] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 7126.174235][T17511] ? __pfx_lock_release+0x10/0x10 > [ 7126.174690][T17511] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 7126.175190][T17511] ? do_raw_spin_lock+0x12c/0x2b0 > [ 7126.175650][T17511] ? __pfx_do_exit+0x10/0x10 > [ 7126.176072][T17511] ? _raw_spin_unlock_irq+0x23/0x50 > [ 7126.176543][T17511] do_group_exit+0xd3/0x2a0 > [ 7126.176990][T17511] __x64_sys_exit_group+0x3e/0x50 > [ 7126.177456][T17511] x64_sys_call+0x1821/0x1830 > [ 7126.177895][T17511] do_syscall_64+0xcb/0x250 > [ 7126.178317][T17511] entry_SYSCALL_64_after_hwframe+0x77/0x7f > > Fixes: aa5e17e1f5ec ("l2tp: store l2tpv3 sessions in per-net IDR") > Signed-off-by: James Chapman <jchapman@katalix.com> > Signed-off-by: Tom Parkin <tparkin@katalix.com> Thanks James, I agree that this addresses the issue described. And, FWIIW, I also checked that the locking order is consistent with that before this patch as for no reason in particular I was concerned about deadlocks. Reviewed-by: Simon Horman <horms@kernel.org>
On 7/18/24 15:43, James Chapman wrote: > Modify l2tp_session_register and l2tp_session_unhash so that the > session IDR and tunnel session lists remain coherent. To do so, hold > the session IDR lock and the tunnel's session list lock when making > any changes to either list. Looks good for net. AFAICS, after this patch, every time 'l2tp_session_idr_lock' is acquired, 'list_lock' is already held; as a possible net-next follow-up you could remove completely the 'l2tp_session_idr_lock' lock, and update accordingly the related lockdep_assert_held() assertion. Cheers, Paolo
Hello: This patch was applied to netdev/net.git (main) by Paolo Abeni <pabeni@redhat.com>: On Thu, 18 Jul 2024 14:43:48 +0100 you wrote: > Modify l2tp_session_register and l2tp_session_unhash so that the > session IDR and tunnel session lists remain coherent. To do so, hold > the session IDR lock and the tunnel's session list lock when making > any changes to either list. > > Without this change, a rare race condition could hit the WARN_ON_ONCE > in l2tp_session_unhash if a thread replaced the IDR entry while > another thread was registering the same ID. > > [...] Here is the summary with links: - [net] l2tp: make session IDR and tunnel session list coherent https://git.kernel.org/netdev/net/c/d587d825424b You are awesome, thank you!
diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c index 1c1decce7f06..c80ab3f26084 100644 --- a/net/l2tp/l2tp_core.c +++ b/net/l2tp/l2tp_core.c @@ -441,14 +441,15 @@ int l2tp_session_register(struct l2tp_session *session, int err; spin_lock_bh(&tunnel->list_lock); + spin_lock_bh(&pn->l2tp_session_idr_lock); + if (!tunnel->acpt_newsess) { err = -ENODEV; - goto err_tlock; + goto out; } if (tunnel->version == L2TP_HDR_VER_3) { session_key = session->session_id; - spin_lock_bh(&pn->l2tp_session_idr_lock); err = idr_alloc_u32(&pn->l2tp_v3_session_idr, NULL, &session_key, session_key, GFP_ATOMIC); /* IP encap expects session IDs to be globally unique, while @@ -462,43 +463,36 @@ int l2tp_session_register(struct l2tp_session *session, err = l2tp_session_collision_add(pn, session, other_session); } - spin_unlock_bh(&pn->l2tp_session_idr_lock); } else { session_key = l2tp_v2_session_key(tunnel->tunnel_id, session->session_id); - spin_lock_bh(&pn->l2tp_session_idr_lock); err = idr_alloc_u32(&pn->l2tp_v2_session_idr, NULL, &session_key, session_key, GFP_ATOMIC); - spin_unlock_bh(&pn->l2tp_session_idr_lock); } if (err) { if (err == -ENOSPC) err = -EEXIST; - goto err_tlock; + goto out; } l2tp_tunnel_inc_refcount(tunnel); - list_add(&session->list, &tunnel->session_list); - spin_unlock_bh(&tunnel->list_lock); - spin_lock_bh(&pn->l2tp_session_idr_lock); if (tunnel->version == L2TP_HDR_VER_3) { if (!other_session) idr_replace(&pn->l2tp_v3_session_idr, session, session_key); } else { idr_replace(&pn->l2tp_v2_session_idr, session, session_key); } - spin_unlock_bh(&pn->l2tp_session_idr_lock); - - trace_register_session(session); - return 0; - -err_tlock: +out: + spin_unlock_bh(&pn->l2tp_session_idr_lock); spin_unlock_bh(&tunnel->list_lock); + if (!err) + trace_register_session(session); + return err; } EXPORT_SYMBOL_GPL(l2tp_session_register); @@ -1260,13 +1254,13 @@ static void l2tp_session_unhash(struct l2tp_session *session) struct l2tp_net *pn = l2tp_pernet(tunnel->l2tp_net); struct l2tp_session *removed = session; - /* Remove from the per-tunnel list */ spin_lock_bh(&tunnel->list_lock); + spin_lock_bh(&pn->l2tp_session_idr_lock); + + /* Remove from the per-tunnel list */ list_del_init(&session->list); - spin_unlock_bh(&tunnel->list_lock); /* Remove from per-net IDR */ - spin_lock_bh(&pn->l2tp_session_idr_lock); if (tunnel->version == L2TP_HDR_VER_3) { if (hash_hashed(&session->hlist)) l2tp_session_collision_del(pn, session); @@ -1280,7 +1274,9 @@ static void l2tp_session_unhash(struct l2tp_session *session) session_key); } WARN_ON_ONCE(removed && removed != session); + spin_unlock_bh(&pn->l2tp_session_idr_lock); + spin_unlock_bh(&tunnel->list_lock); synchronize_rcu(); }
Modify l2tp_session_register and l2tp_session_unhash so that the session IDR and tunnel session lists remain coherent. To do so, hold the session IDR lock and the tunnel's session list lock when making any changes to either list. Without this change, a rare race condition could hit the WARN_ON_ONCE in l2tp_session_unhash if a thread replaced the IDR entry while another thread was registering the same ID. [ 7126.151795][T17511] WARNING: CPU: 3 PID: 17511 at net/l2tp/l2tp_core.c:1282 l2tp_session_delete.part.0+0x87e/0xbc0 [ 7126.163754][T17511] ? show_regs+0x93/0xa0 [ 7126.164157][T17511] ? __warn+0xe5/0x3c0 [ 7126.164536][T17511] ? l2tp_session_delete.part.0+0x87e/0xbc0 [ 7126.165070][T17511] ? report_bug+0x2e1/0x500 [ 7126.165486][T17511] ? l2tp_session_delete.part.0+0x87e/0xbc0 [ 7126.166013][T17511] ? handle_bug+0x99/0x130 [ 7126.166428][T17511] ? exc_invalid_op+0x35/0x80 [ 7126.166890][T17511] ? asm_exc_invalid_op+0x1a/0x20 [ 7126.167372][T17511] ? l2tp_session_delete.part.0+0x87d/0xbc0 [ 7126.167900][T17511] ? l2tp_session_delete.part.0+0x87e/0xbc0 [ 7126.168429][T17511] ? __local_bh_enable_ip+0xa4/0x120 [ 7126.168917][T17511] l2tp_session_delete+0x40/0x50 [ 7126.169369][T17511] pppol2tp_release+0x1a1/0x3f0 [ 7126.169817][T17511] __sock_release+0xb3/0x270 [ 7126.170247][T17511] ? __pfx_sock_close+0x10/0x10 [ 7126.170697][T17511] sock_close+0x1c/0x30 [ 7126.171087][T17511] __fput+0x40b/0xb90 [ 7126.171470][T17511] task_work_run+0x16c/0x260 [ 7126.171897][T17511] ? __pfx_task_work_run+0x10/0x10 [ 7126.172362][T17511] ? srso_alias_return_thunk+0x5/0xfbef5 [ 7126.172863][T17511] ? do_raw_spin_unlock+0x174/0x230 [ 7126.173348][T17511] do_exit+0xaae/0x2b40 [ 7126.173730][T17511] ? srso_alias_return_thunk+0x5/0xfbef5 [ 7126.174235][T17511] ? __pfx_lock_release+0x10/0x10 [ 7126.174690][T17511] ? srso_alias_return_thunk+0x5/0xfbef5 [ 7126.175190][T17511] ? do_raw_spin_lock+0x12c/0x2b0 [ 7126.175650][T17511] ? __pfx_do_exit+0x10/0x10 [ 7126.176072][T17511] ? _raw_spin_unlock_irq+0x23/0x50 [ 7126.176543][T17511] do_group_exit+0xd3/0x2a0 [ 7126.176990][T17511] __x64_sys_exit_group+0x3e/0x50 [ 7126.177456][T17511] x64_sys_call+0x1821/0x1830 [ 7126.177895][T17511] do_syscall_64+0xcb/0x250 [ 7126.178317][T17511] entry_SYSCALL_64_after_hwframe+0x77/0x7f Fixes: aa5e17e1f5ec ("l2tp: store l2tpv3 sessions in per-net IDR") Signed-off-by: James Chapman <jchapman@katalix.com> Signed-off-by: Tom Parkin <tparkin@katalix.com> --- net/l2tp/l2tp_core.c | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-)