| Message ID | c1ba274b19f6d1399636d018333d14a032d05454.1713967592.git.dcaratti@redhat.com (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [net,v2] netlabel: fix RCU annotation for IPv4 options on socket creation | expand |
On Apr 24, 2024 Davide Caratti <dcaratti@redhat.com> wrote: > > Xiumei reports the following splat when netlabel and TCP socket are used: > > ============================= > WARNING: suspicious RCU usage > 6.9.0-rc2+ #637 Not tainted > ----------------------------- > net/ipv4/cipso_ipv4.c:1880 suspicious rcu_dereference_protected() usage! > > other info that might help us debug this: > > rcu_scheduler_active = 2, debug_locks = 1 > 1 lock held by ncat/23333: > #0: ffffffff906030c0 (rcu_read_lock){....}-{1:2}, at: netlbl_sock_setattr+0x25/0x1b0 > > stack backtrace: > CPU: 11 PID: 23333 Comm: ncat Kdump: loaded Not tainted 6.9.0-rc2+ #637 > Hardware name: Supermicro SYS-6027R-72RF/X9DRH-7TF/7F/iTF/iF, BIOS 3.0 07/26/2013 > Call Trace: > <TASK> > dump_stack_lvl+0xa9/0xc0 > lockdep_rcu_suspicious+0x117/0x190 > cipso_v4_sock_setattr+0x1ab/0x1b0 > netlbl_sock_setattr+0x13e/0x1b0 > selinux_netlbl_socket_post_create+0x3f/0x80 > selinux_socket_post_create+0x1a0/0x460 > security_socket_post_create+0x42/0x60 > __sock_create+0x342/0x3a0 > __sys_socket_create.part.22+0x42/0x70 > __sys_socket+0x37/0xb0 > __x64_sys_socket+0x16/0x20 > do_syscall_64+0x96/0x180 > ? do_user_addr_fault+0x68d/0xa30 > ? exc_page_fault+0x171/0x280 > ? asm_exc_page_fault+0x22/0x30 > entry_SYSCALL_64_after_hwframe+0x71/0x79 > RIP: 0033:0x7fbc0ca3fc1b > Code: 73 01 c3 48 8b 0d 05 f2 1b 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d5 f1 1b 00 f7 d8 64 89 01 48 > RSP: 002b:00007fff18635208 EFLAGS: 00000246 ORIG_RAX: 0000000000000029 > RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fbc0ca3fc1b > RDX: 0000000000000006 RSI: 0000000000000001 RDI: 0000000000000002 > RBP: 000055d24f80f8a0 R08: 0000000000000003 R09: 0000000000000001 > > R10: 0000000000020000 R11: 0000000000000246 R12: 000055d24f80f8a0 > R13: 0000000000000000 R14: 000055d24f80fb88 R15: 0000000000000000 > </TASK> > > The current implementation of cipso_v4_sock_setattr() replaces IP options > under the assumption that the caller holds the socket lock; however, such > assumption is not true, nor needed, in selinux_socket_post_create() hook. > > Let all callers of cipso_v4_sock_setattr() specify the "socket lock held" > condition, except selinux_socket_post_create() _ where such condition can > safely be set as true even without holding the socket lock. > While at it: use rcu_replace_pointer() instead of open coding, and remove > useless NULL check of 'old' before kfree_rcu(old, ...). > > v2: > - pass lockdep_sock_is_held() through a boolean variable in the stack > (thanks Eric Dumazet, Paul Moore, Casey Schaufler) > - use rcu_replace_pointer() instead of rcu_dereference_protected() + > rcu_assign_pointer() > - remove NULL check of 'old' before kfree_rcu() > > Fixes: f6d8bd051c39 ("inet: add RCU protection to inet->opt") > Reported-by: Xiumei Mu <xmu@redhat.com> > Signed-off-by: Davide Caratti <dcaratti@redhat.com> > --- > include/net/cipso_ipv4.h | 6 ++++-- > include/net/netlabel.h | 6 ++++-- > net/ipv4/cipso_ipv4.c | 13 ++++++------- > net/netlabel/netlabel_kapi.c | 9 ++++++--- > security/selinux/netlabel.c | 5 ++++- > security/smack/smack_lsm.c | 3 ++- > 6 files changed, 26 insertions(+), 16 deletions(-) ... > diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c > index 8b17d83e5fde..c4ac704cbcc2 100644 > --- a/net/ipv4/cipso_ipv4.c > +++ b/net/ipv4/cipso_ipv4.c > @@ -1815,6 +1815,7 @@ static int cipso_v4_genopt(unsigned char *buf, u32 buf_len, > * @sk: the socket > * @doi_def: the CIPSO DOI to use > * @secattr: the specific security attributes of the socket > + * @slock_held: true if caller holds the socket lock > * > * Description: > * Set the CIPSO option on the given socket using the DOI definition and > @@ -1826,7 +1827,8 @@ static int cipso_v4_genopt(unsigned char *buf, u32 buf_len, > */ > int cipso_v4_sock_setattr(struct sock *sk, > const struct cipso_v4_doi *doi_def, > - const struct netlbl_lsm_secattr *secattr) > + const struct netlbl_lsm_secattr *secattr, > + bool slock_held) This is a nitpicky bikeshedding remark, but "slock_held" sounds really awkward to me, something like "sk_locked" sounds much better. > { > int ret_val = -EPERM; > unsigned char *buf = NULL; > @@ -1876,18 +1878,15 @@ int cipso_v4_sock_setattr(struct sock *sk, > > sk_inet = inet_sk(sk); > > - old = rcu_dereference_protected(sk_inet->inet_opt, > - lockdep_sock_is_held(sk)); > + old = rcu_replace_pointer(sk_inet->inet_opt, opt, slock_held); > if (inet_test_bit(IS_ICSK, sk)) { > sk_conn = inet_csk(sk); > if (old) > sk_conn->icsk_ext_hdr_len -= old->opt.optlen; > - sk_conn->icsk_ext_hdr_len += opt->opt.optlen; > + sk_conn->icsk_ext_hdr_len += opt_len; > sk_conn->icsk_sync_mss(sk, sk_conn->icsk_pmtu_cookie); > } > - rcu_assign_pointer(sk_inet->inet_opt, opt); > - if (old) > - kfree_rcu(old, rcu); > + kfree_rcu(old, rcu); Thanks for sticking with this and posting a v2. These changes look okay to me, but considering the 'Fixes:' tag and the RCU splat it is reasonable to expect that this is going to be backported to the various stable trees. With that in mind, I think we should try to keep the immediate fix as simple as possible, saving the other changes for a separate patch. This means sticking with rcu_dereference_protected() and omitting the opt_len optimization; both can be done in a second patch without the 'Fixes:' marking. Unless I missing something and those changes are somehow part of the fix? -- paul-moore.com
hello Paul, thanks for reviewing! On Thu, Apr 25, 2024 at 05:01:36PM -0400, Paul Moore wrote: > On Apr 24, 2024 Davide Caratti <dcaratti@redhat.com> wrote: > > [...] > > @@ -1826,7 +1827,8 @@ static int cipso_v4_genopt(unsigned char *buf, u32 buf_len, > > */ > > int cipso_v4_sock_setattr(struct sock *sk, > > const struct cipso_v4_doi *doi_def, > > - const struct netlbl_lsm_secattr *secattr) > > + const struct netlbl_lsm_secattr *secattr, > > + bool slock_held) > > This is a nitpicky bikeshedding remark, but "slock_held" sounds really > awkward to me, something like "sk_locked" sounds much better. ok, will fix that in v3. [...] > > @@ -1876,18 +1878,15 @@ int cipso_v4_sock_setattr(struct sock *sk, > > > > sk_inet = inet_sk(sk); > > > > - old = rcu_dereference_protected(sk_inet->inet_opt, > > - lockdep_sock_is_held(sk)); > > + old = rcu_replace_pointer(sk_inet->inet_opt, opt, slock_held); > > if (inet_test_bit(IS_ICSK, sk)) { > > sk_conn = inet_csk(sk); > > if (old) > > sk_conn->icsk_ext_hdr_len -= old->opt.optlen; > > - sk_conn->icsk_ext_hdr_len += opt->opt.optlen; > > + sk_conn->icsk_ext_hdr_len += opt_len; > > sk_conn->icsk_sync_mss(sk, sk_conn->icsk_pmtu_cookie); > > } > > - rcu_assign_pointer(sk_inet->inet_opt, opt); > > - if (old) > > - kfree_rcu(old, rcu); > > + kfree_rcu(old, rcu); > > Thanks for sticking with this and posting a v2. > > These changes look okay to me, but considering the 'Fixes:' tag and the > RCU splat it is reasonable to expect that this is going to be backported > to the various stable trees. With that in mind, I think we should try > to keep the immediate fix as simple as possible, saving the other > changes for a separate patch. This means sticking with > rcu_dereference_protected() and omitting the opt_len optimization; both > can be done in a second patch without the 'Fixes:' marking. > > Unless I missing something and those changes are somehow part of the > fix? just checked, rcu_replace_pointer() can be used also in the oldest LTS but I'm not sure if kfree_rcu(NULL, ...) is ok. I agree to keep rcu_dereference_protected(), and the useless NULL check - I will follow-up with another patch (targeting net-next), after this one is merged. -- davide
On Mon, Apr 29, 2024 at 6:10 AM Davide Caratti <dcaratti@redhat.com> wrote: > On Thu, Apr 25, 2024 at 05:01:36PM -0400, Paul Moore wrote: > > On Apr 24, 2024 Davide Caratti <dcaratti@redhat.com> wrote: ... > > > @@ -1876,18 +1878,15 @@ int cipso_v4_sock_setattr(struct sock *sk, > > > > > > sk_inet = inet_sk(sk); > > > > > > - old = rcu_dereference_protected(sk_inet->inet_opt, > > > - lockdep_sock_is_held(sk)); > > > + old = rcu_replace_pointer(sk_inet->inet_opt, opt, slock_held); > > > if (inet_test_bit(IS_ICSK, sk)) { > > > sk_conn = inet_csk(sk); > > > if (old) > > > sk_conn->icsk_ext_hdr_len -= old->opt.optlen; > > > - sk_conn->icsk_ext_hdr_len += opt->opt.optlen; > > > + sk_conn->icsk_ext_hdr_len += opt_len; > > > sk_conn->icsk_sync_mss(sk, sk_conn->icsk_pmtu_cookie); > > > } > > > - rcu_assign_pointer(sk_inet->inet_opt, opt); > > > - if (old) > > > - kfree_rcu(old, rcu); > > > + kfree_rcu(old, rcu); > > > > Thanks for sticking with this and posting a v2. > > > > These changes look okay to me, but considering the 'Fixes:' tag and the > > RCU splat it is reasonable to expect that this is going to be backported > > to the various stable trees. With that in mind, I think we should try > > to keep the immediate fix as simple as possible, saving the other > > changes for a separate patch. This means sticking with > > rcu_dereference_protected() and omitting the opt_len optimization; both > > can be done in a second patch without the 'Fixes:' marking. > > > > Unless I missing something and those changes are somehow part of the > > fix? > > just checked, rcu_replace_pointer() can be used also in the oldest LTS > but I'm not sure if kfree_rcu(NULL, ...) is ok. I agree to keep > rcu_dereference_protected(), and the useless NULL check - I will > follow-up with another patch (targeting net-next), after this one is > merged. The issue isn't so much about if a particular function is available in an older kernel, it is more about keeping the patch focused on a single immediate purpose both to limit any unintended behaviors and for the simple reason that smaller patches are almost always easier to port by hand if needed.
diff --git a/include/net/cipso_ipv4.h b/include/net/cipso_ipv4.h index 53dd7d988a2d..5f8193a9808c 100644 --- a/include/net/cipso_ipv4.h +++ b/include/net/cipso_ipv4.h @@ -183,7 +183,8 @@ int cipso_v4_getattr(const unsigned char *cipso, struct netlbl_lsm_secattr *secattr); int cipso_v4_sock_setattr(struct sock *sk, const struct cipso_v4_doi *doi_def, - const struct netlbl_lsm_secattr *secattr); + const struct netlbl_lsm_secattr *secattr, + bool slock_held); void cipso_v4_sock_delattr(struct sock *sk); int cipso_v4_sock_getattr(struct sock *sk, struct netlbl_lsm_secattr *secattr); int cipso_v4_req_setattr(struct request_sock *req, @@ -214,7 +215,8 @@ static inline int cipso_v4_getattr(const unsigned char *cipso, static inline int cipso_v4_sock_setattr(struct sock *sk, const struct cipso_v4_doi *doi_def, - const struct netlbl_lsm_secattr *secattr) + const struct netlbl_lsm_secattr *secattr, + bool slock_held) { return -ENOSYS; } diff --git a/include/net/netlabel.h b/include/net/netlabel.h index f3ab0b8a4b18..6f098b618461 100644 --- a/include/net/netlabel.h +++ b/include/net/netlabel.h @@ -470,7 +470,8 @@ void netlbl_bitmap_setbit(unsigned char *bitmap, u32 bit, u8 state); int netlbl_enabled(void); int netlbl_sock_setattr(struct sock *sk, u16 family, - const struct netlbl_lsm_secattr *secattr); + const struct netlbl_lsm_secattr *secattr, + bool slock_held); void netlbl_sock_delattr(struct sock *sk); int netlbl_sock_getattr(struct sock *sk, struct netlbl_lsm_secattr *secattr); @@ -614,7 +615,8 @@ static inline int netlbl_enabled(void) } static inline int netlbl_sock_setattr(struct sock *sk, u16 family, - const struct netlbl_lsm_secattr *secattr) + const struct netlbl_lsm_secattr *secattr, + bool slock_held) { return -ENOSYS; } diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c index 8b17d83e5fde..c4ac704cbcc2 100644 --- a/net/ipv4/cipso_ipv4.c +++ b/net/ipv4/cipso_ipv4.c @@ -1815,6 +1815,7 @@ static int cipso_v4_genopt(unsigned char *buf, u32 buf_len, * @sk: the socket * @doi_def: the CIPSO DOI to use * @secattr: the specific security attributes of the socket + * @slock_held: true if caller holds the socket lock * * Description: * Set the CIPSO option on the given socket using the DOI definition and @@ -1826,7 +1827,8 @@ static int cipso_v4_genopt(unsigned char *buf, u32 buf_len, */ int cipso_v4_sock_setattr(struct sock *sk, const struct cipso_v4_doi *doi_def, - const struct netlbl_lsm_secattr *secattr) + const struct netlbl_lsm_secattr *secattr, + bool slock_held) { int ret_val = -EPERM; unsigned char *buf = NULL; @@ -1876,18 +1878,15 @@ int cipso_v4_sock_setattr(struct sock *sk, sk_inet = inet_sk(sk); - old = rcu_dereference_protected(sk_inet->inet_opt, - lockdep_sock_is_held(sk)); + old = rcu_replace_pointer(sk_inet->inet_opt, opt, slock_held); if (inet_test_bit(IS_ICSK, sk)) { sk_conn = inet_csk(sk); if (old) sk_conn->icsk_ext_hdr_len -= old->opt.optlen; - sk_conn->icsk_ext_hdr_len += opt->opt.optlen; + sk_conn->icsk_ext_hdr_len += opt_len; sk_conn->icsk_sync_mss(sk, sk_conn->icsk_pmtu_cookie); } - rcu_assign_pointer(sk_inet->inet_opt, opt); - if (old) - kfree_rcu(old, rcu); + kfree_rcu(old, rcu); return 0; diff --git a/net/netlabel/netlabel_kapi.c b/net/netlabel/netlabel_kapi.c index 1ba4f58e1d35..0083749705c4 100644 --- a/net/netlabel/netlabel_kapi.c +++ b/net/netlabel/netlabel_kapi.c @@ -965,6 +965,7 @@ int netlbl_enabled(void) * @sk: the socket to label * @family: protocol family * @secattr: the security attributes + * @slock_held: true if caller holds the socket lock * * Description: * Attach the correct label to the given socket using the security attributes @@ -977,7 +978,8 @@ int netlbl_enabled(void) */ int netlbl_sock_setattr(struct sock *sk, u16 family, - const struct netlbl_lsm_secattr *secattr) + const struct netlbl_lsm_secattr *secattr, + bool slock_held) { int ret_val; struct netlbl_dom_map *dom_entry; @@ -997,7 +999,7 @@ int netlbl_sock_setattr(struct sock *sk, case NETLBL_NLTYPE_CIPSOV4: ret_val = cipso_v4_sock_setattr(sk, dom_entry->def.cipso, - secattr); + secattr, slock_held); break; case NETLBL_NLTYPE_UNLABELED: ret_val = 0; @@ -1126,7 +1128,8 @@ int netlbl_conn_setattr(struct sock *sk, switch (entry->type) { case NETLBL_NLTYPE_CIPSOV4: ret_val = cipso_v4_sock_setattr(sk, - entry->cipso, secattr); + entry->cipso, secattr, + lockdep_sock_is_held(sk)); break; case NETLBL_NLTYPE_UNLABELED: /* just delete the protocols we support for right now diff --git a/security/selinux/netlabel.c b/security/selinux/netlabel.c index 8f182800e412..55885634e880 100644 --- a/security/selinux/netlabel.c +++ b/security/selinux/netlabel.c @@ -402,7 +402,10 @@ int selinux_netlbl_socket_post_create(struct sock *sk, u16 family) secattr = selinux_netlbl_sock_genattr(sk); if (secattr == NULL) return -ENOMEM; - rc = netlbl_sock_setattr(sk, family, secattr); + /* On socket creation, replacement of IP options is safe even if + * the caller does not hold the socket lock. + */ + rc = netlbl_sock_setattr(sk, family, secattr, true); switch (rc) { case 0: sksec->nlbl_state = NLBL_LABELED; diff --git a/security/smack/smack_lsm.c b/security/smack/smack_lsm.c index 146667937811..1ab2125d352d 100644 --- a/security/smack/smack_lsm.c +++ b/security/smack/smack_lsm.c @@ -2565,7 +2565,8 @@ static int smack_netlbl_add(struct sock *sk) local_bh_disable(); bh_lock_sock_nested(sk); - rc = netlbl_sock_setattr(sk, sk->sk_family, &skp->smk_netlabel); + rc = netlbl_sock_setattr(sk, sk->sk_family, &skp->smk_netlabel, + lockdep_sock_is_held(sk)); switch (rc) { case 0: ssp->smk_state = SMK_NETLBL_LABELED;
Xiumei reports the following splat when netlabel and TCP socket are used: ============================= WARNING: suspicious RCU usage 6.9.0-rc2+ #637 Not tainted ----------------------------- net/ipv4/cipso_ipv4.c:1880 suspicious rcu_dereference_protected() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 1 lock held by ncat/23333: #0: ffffffff906030c0 (rcu_read_lock){....}-{1:2}, at: netlbl_sock_setattr+0x25/0x1b0 stack backtrace: CPU: 11 PID: 23333 Comm: ncat Kdump: loaded Not tainted 6.9.0-rc2+ #637 Hardware name: Supermicro SYS-6027R-72RF/X9DRH-7TF/7F/iTF/iF, BIOS 3.0 07/26/2013 Call Trace: <TASK> dump_stack_lvl+0xa9/0xc0 lockdep_rcu_suspicious+0x117/0x190 cipso_v4_sock_setattr+0x1ab/0x1b0 netlbl_sock_setattr+0x13e/0x1b0 selinux_netlbl_socket_post_create+0x3f/0x80 selinux_socket_post_create+0x1a0/0x460 security_socket_post_create+0x42/0x60 __sock_create+0x342/0x3a0 __sys_socket_create.part.22+0x42/0x70 __sys_socket+0x37/0xb0 __x64_sys_socket+0x16/0x20 do_syscall_64+0x96/0x180 ? do_user_addr_fault+0x68d/0xa30 ? exc_page_fault+0x171/0x280 ? asm_exc_page_fault+0x22/0x30 entry_SYSCALL_64_after_hwframe+0x71/0x79 RIP: 0033:0x7fbc0ca3fc1b Code: 73 01 c3 48 8b 0d 05 f2 1b 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d5 f1 1b 00 f7 d8 64 89 01 48 RSP: 002b:00007fff18635208 EFLAGS: 00000246 ORIG_RAX: 0000000000000029 RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fbc0ca3fc1b RDX: 0000000000000006 RSI: 0000000000000001 RDI: 0000000000000002 RBP: 000055d24f80f8a0 R08: 0000000000000003 R09: 0000000000000001 R10: 0000000000020000 R11: 0000000000000246 R12: 000055d24f80f8a0 R13: 0000000000000000 R14: 000055d24f80fb88 R15: 0000000000000000 </TASK> The current implementation of cipso_v4_sock_setattr() replaces IP options under the assumption that the caller holds the socket lock; however, such assumption is not true, nor needed, in selinux_socket_post_create() hook. Let all callers of cipso_v4_sock_setattr() specify the "socket lock held" condition, except selinux_socket_post_create() _ where such condition can safely be set as true even without holding the socket lock. While at it: use rcu_replace_pointer() instead of open coding, and remove useless NULL check of 'old' before kfree_rcu(old, ...). v2: - pass lockdep_sock_is_held() through a boolean variable in the stack (thanks Eric Dumazet, Paul Moore, Casey Schaufler) - use rcu_replace_pointer() instead of rcu_dereference_protected() + rcu_assign_pointer() - remove NULL check of 'old' before kfree_rcu() Fixes: f6d8bd051c39 ("inet: add RCU protection to inet->opt") Reported-by: Xiumei Mu <xmu@redhat.com> Signed-off-by: Davide Caratti <dcaratti@redhat.com> --- include/net/cipso_ipv4.h | 6 ++++-- include/net/netlabel.h | 6 ++++-- net/ipv4/cipso_ipv4.c | 13 ++++++------- net/netlabel/netlabel_kapi.c | 9 ++++++--- security/selinux/netlabel.c | 5 ++++- security/smack/smack_lsm.c | 3 ++- 6 files changed, 26 insertions(+), 16 deletions(-)