[v2] Bluetooth: Fix l2cap_disconnect_req deadlock

Commit Message

Ying Hsu May 22, 2023, 11:41 p.m. UTC

L2CAP assumes that the locks conn->chan_lock and chan->lock are
acquired in the order conn->chan_lock, chan->lock to avoid
potential deadlock.
For example, l2sock_shutdown acquires these locks in the order:
  mutex_lock(&conn->chan_lock)
  l2cap_chan_lock(chan)

However, l2cap_disconnect_req acquires chan->lock in
l2cap_get_chan_by_scid first and then acquires conn->chan_lock
before calling l2cap_chan_del. This means that these locks are
acquired in unexpected order, which leads to potential deadlock:
  l2cap_chan_lock(c)
  mutex_lock(&conn->chan_lock)

This patch uses __l2cap_get_chan_by_scid to replace
l2cap_get_chan_by_scid and adjusts the locking order to avoid the
potential deadlock.

Signed-off-by: Ying Hsu <[e-mail]>
---
This commit has been tested on a Chromebook device.

Changes in v2:
- Adding the prefix "Bluetooth:" to subject line.

 net/bluetooth/l2cap_core.c | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)

Comments

[e-mail] May 23, 2023, 1:17 a.m. UTC | #1

This is automated email and please do not reply to this email!

Dear submitter,

Thank you for submitting the patches to the linux bluetooth mailing list.
This is a CI test results with your patch series:
PW Link:https://patchwork.kernel.org/project/bluetooth/list/?series=749991

---Test result---

Test Summary:
CheckPatch                    PASS      0.75 seconds
GitLint                       PASS      0.31 seconds
SubjectPrefix                 PASS      0.10 seconds
BuildKernel                   PASS      43.74 seconds
CheckAllWarning               PASS      46.42 seconds
CheckSparse                   PASS      52.72 seconds
CheckSmatch                   PASS      141.11 seconds
BuildKernel32                 PASS      40.88 seconds
TestRunnerSetup               PASS      586.64 seconds
TestRunner_l2cap-tester       PASS      21.12 seconds
TestRunner_iso-tester         PASS      29.31 seconds
TestRunner_bnep-tester        PASS      7.97 seconds
TestRunner_mgmt-tester        PASS      143.13 seconds
TestRunner_rfcomm-tester      PASS      11.72 seconds
TestRunner_sco-tester         PASS      10.72 seconds
TestRunner_ioctl-tester       PASS      12.37 seconds
TestRunner_mesh-tester        PASS      9.30 seconds
TestRunner_smp-tester         PASS      10.29 seconds
TestRunner_userchan-tester    PASS      7.91 seconds
IncrementalBuild              PASS      39.00 seconds



---
Regards,
Linux Bluetooth

Simon Horman May 23, 2023, 10:04 a.m. UTC | #2

On Mon, May 22, 2023 at 11:41:51PM +0000, Ying Hsu wrote:
> L2CAP assumes that the locks conn->chan_lock and chan->lock are
> acquired in the order conn->chan_lock, chan->lock to avoid
> potential deadlock.
> For example, l2sock_shutdown acquires these locks in the order:
>   mutex_lock(&conn->chan_lock)
>   l2cap_chan_lock(chan)
> 
> However, l2cap_disconnect_req acquires chan->lock in
> l2cap_get_chan_by_scid first and then acquires conn->chan_lock
> before calling l2cap_chan_del. This means that these locks are
> acquired in unexpected order, which leads to potential deadlock:
>   l2cap_chan_lock(c)
>   mutex_lock(&conn->chan_lock)
> 
> This patch uses __l2cap_get_chan_by_scid to replace
> l2cap_get_chan_by_scid and adjusts the locking order to avoid the
> potential deadlock.
> 
> Signed-off-by: Ying Hsu <[e-mail]>
> ---
> This commit has been tested on a Chromebook device.
> 
> Changes in v2:
> - Adding the prefix "Bluetooth:" to subject line.
> 
>  net/bluetooth/l2cap_core.c | 26 ++++++++++++++++++++------
>  1 file changed, 20 insertions(+), 6 deletions(-)
> 
> diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
> index 376b523c7b26..8f08192b8fb1 100644
> --- a/net/bluetooth/l2cap_core.c
> +++ b/net/bluetooth/l2cap_core.c
> @@ -4651,8 +4651,16 @@ static inline int l2cap_disconnect_req(struct l2cap_conn *conn,
>  
>  	BT_DBG("scid 0x%4.4x dcid 0x%4.4x", scid, dcid);
>  
> -	chan = l2cap_get_chan_by_scid(conn, dcid);
> +	mutex_lock(&conn->chan_lock);
> +	chan = __l2cap_get_chan_by_scid(conn, dcid);
> +	if (chan) {
> +		chan = l2cap_chan_hold_unless_zero(chan);
> +		if (chan)
> +			l2cap_chan_lock(chan);
> +	}
> +
>  	if (!chan) {
> +		mutex_unlock(&conn->chan_lock);
>  		cmd_reject_invalid_cid(conn, cmd->ident, dcid, scid);
>  		return 0;
>  	}

Hi Ying,

The conditional setting of chan and calling l2cap_chan_lock()
is both non-trivial and repeated. It seems that it ought to be
in a helper.

Something like this (I'm sure a better function name can be chosen):

	chan = __l2cap_get_and_lock_chan_by_scid(conn, dcid);
	if (!chan) {
		...
	}

	...

Luiz Augusto von Dentz May 23, 2023, 7:06 p.m. UTC | #3

Hi Simon, Ying,

On Tue, May 23, 2023 at 3:04 AM Simon Horman <[e-mail]> wrote:
>
> On Mon, May 22, 2023 at 11:41:51PM +0000, Ying Hsu wrote:
> > L2CAP assumes that the locks conn->chan_lock and chan->lock are
> > acquired in the order conn->chan_lock, chan->lock to avoid
> > potential deadlock.
> > For example, l2sock_shutdown acquires these locks in the order:
> >   mutex_lock(&conn->chan_lock)
> >   l2cap_chan_lock(chan)
> >
> > However, l2cap_disconnect_req acquires chan->lock in
> > l2cap_get_chan_by_scid first and then acquires conn->chan_lock
> > before calling l2cap_chan_del. This means that these locks are
> > acquired in unexpected order, which leads to potential deadlock:
> >   l2cap_chan_lock(c)
> >   mutex_lock(&conn->chan_lock)
> >
> > This patch uses __l2cap_get_chan_by_scid to replace
> > l2cap_get_chan_by_scid and adjusts the locking order to avoid the
> > potential deadlock.

This needs the fixes tag so we can backport it properly.

> > Signed-off-by: Ying Hsu <[e-mail]>
> > ---
> > This commit has been tested on a Chromebook device.
> >
> > Changes in v2:
> > - Adding the prefix "Bluetooth:" to subject line.
> >
> >  net/bluetooth/l2cap_core.c | 26 ++++++++++++++++++++------
> >  1 file changed, 20 insertions(+), 6 deletions(-)
> >
> > diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
> > index 376b523c7b26..8f08192b8fb1 100644
> > --- a/net/bluetooth/l2cap_core.c
> > +++ b/net/bluetooth/l2cap_core.c
> > @@ -4651,8 +4651,16 @@ static inline int l2cap_disconnect_req(struct l2cap_conn *conn,
> >
> >       BT_DBG("scid 0x%4.4x dcid 0x%4.4x", scid, dcid);
> >
> > -     chan = l2cap_get_chan_by_scid(conn, dcid);
> > +     mutex_lock(&conn->chan_lock);
> > +     chan = __l2cap_get_chan_by_scid(conn, dcid);
> > +     if (chan) {
> > +             chan = l2cap_chan_hold_unless_zero(chan);
> > +             if (chan)
> > +                     l2cap_chan_lock(chan);
> > +     }
> > +
> >       if (!chan) {
> > +             mutex_unlock(&conn->chan_lock);
> >               cmd_reject_invalid_cid(conn, cmd->ident, dcid, scid);
> >               return 0;
> >       }
>
> Hi Ying,
>
> The conditional setting of chan and calling l2cap_chan_lock()
> is both non-trivial and repeated. It seems that it ought to be
> in a helper.
>
> Something like this (I'm sure a better function name can be chosen):
>
>         chan = __l2cap_get_and_lock_chan_by_scid(conn, dcid);
>         if (!chan) {
>                 ...
>         }
>
>         ...

Or perhaps we could do something like l2cap_del_chan_by_scid:

https://gist.github.com/Vudentz/e513859ecb31e79c947dfcb4b5c60453

Ying Hsu May 24, 2023, 10:53 a.m. UTC | #4

Hi Simon,

I understand your concern about the repeated code.
However, simply hiding the locking logic in another function
introduces hidden assumptions.
For this patch, I would like to fix the deadlock in a simple and easy
to understand way.
We can always refactor the l2cap_chan utility functions later.

Hi Luis,

I'll add a fixes tag in the next version.

Best regards,
Ying


On Wed, May 24, 2023 at 3:06 AM Luiz Augusto von Dentz
<[e-mail]> wrote:
>
> Hi Simon, Ying,
>
> On Tue, May 23, 2023 at 3:04 AM Simon Horman <[e-mail]> wrote:
> >
> > On Mon, May 22, 2023 at 11:41:51PM +0000, Ying Hsu wrote:
> > > L2CAP assumes that the locks conn->chan_lock and chan->lock are
> > > acquired in the order conn->chan_lock, chan->lock to avoid
> > > potential deadlock.
> > > For example, l2sock_shutdown acquires these locks in the order:
> > >   mutex_lock(&conn->chan_lock)
> > >   l2cap_chan_lock(chan)
> > >
> > > However, l2cap_disconnect_req acquires chan->lock in
> > > l2cap_get_chan_by_scid first and then acquires conn->chan_lock
> > > before calling l2cap_chan_del. This means that these locks are
> > > acquired in unexpected order, which leads to potential deadlock:
> > >   l2cap_chan_lock(c)
> > >   mutex_lock(&conn->chan_lock)
> > >
> > > This patch uses __l2cap_get_chan_by_scid to replace
> > > l2cap_get_chan_by_scid and adjusts the locking order to avoid the
> > > potential deadlock.
>
> This needs the fixes tag so we can backport it properly.
>
> > > Signed-off-by: Ying Hsu <[e-mail]>
> > > ---
> > > This commit has been tested on a Chromebook device.
> > >
> > > Changes in v2:
> > > - Adding the prefix "Bluetooth:" to subject line.
> > >
> > >  net/bluetooth/l2cap_core.c | 26 ++++++++++++++++++++------
> > >  1 file changed, 20 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
> > > index 376b523c7b26..8f08192b8fb1 100644
> > > --- a/net/bluetooth/l2cap_core.c
> > > +++ b/net/bluetooth/l2cap_core.c
> > > @@ -4651,8 +4651,16 @@ static inline int l2cap_disconnect_req(struct l2cap_conn *conn,
> > >
> > >       BT_DBG("scid 0x%4.4x dcid 0x%4.4x", scid, dcid);
> > >
> > > -     chan = l2cap_get_chan_by_scid(conn, dcid);
> > > +     mutex_lock(&conn->chan_lock);
> > > +     chan = __l2cap_get_chan_by_scid(conn, dcid);
> > > +     if (chan) {
> > > +             chan = l2cap_chan_hold_unless_zero(chan);
> > > +             if (chan)
> > > +                     l2cap_chan_lock(chan);
> > > +     }
> > > +
> > >       if (!chan) {
> > > +             mutex_unlock(&conn->chan_lock);
> > >               cmd_reject_invalid_cid(conn, cmd->ident, dcid, scid);
> > >               return 0;
> > >       }
> >
> > Hi Ying,
> >
> > The conditional setting of chan and calling l2cap_chan_lock()
> > is both non-trivial and repeated. It seems that it ought to be
> > in a helper.
> >
> > Something like this (I'm sure a better function name can be chosen):
> >
> >         chan = __l2cap_get_and_lock_chan_by_scid(conn, dcid);
> >         if (!chan) {
> >                 ...
> >         }
> >
> >         ...
>
> Or perhaps we could do something like l2cap_del_chan_by_scid:
>
> https://gist.github.com/Vudentz/e513859ecb31e79c947dfcb4b5c60453
>
> --
> Luiz Augusto von Dentz

Luiz Augusto von Dentz May 24, 2023, 6:56 p.m. UTC | #5

Hi Ying,

On Wed, May 24, 2023 at 3:54 AM Ying Hsu <[e-mail]> wrote:
>
> Hi Simon,
>
> I understand your concern about the repeated code.
> However, simply hiding the locking logic in another function
> introduces hidden assumptions.
> For this patch, I would like to fix the deadlock in a simple and easy
> to understand way.
> We can always refactor the l2cap_chan utility functions later.
>
> Hi Luis,
>
> I'll add a fixes tag in the next version.

And how about doing this:

https://gist.github.com/Vudentz/e513859ecb31e79c947dfcb4b5c60453

> Best regards,
> Ying
>
>
> On Wed, May 24, 2023 at 3:06 AM Luiz Augusto von Dentz
> <[e-mail]> wrote:
> >
> > Hi Simon, Ying,
> >
> > On Tue, May 23, 2023 at 3:04 AM Simon Horman <[e-mail]> wrote:
> > >
> > > On Mon, May 22, 2023 at 11:41:51PM +0000, Ying Hsu wrote:
> > > > L2CAP assumes that the locks conn->chan_lock and chan->lock are
> > > > acquired in the order conn->chan_lock, chan->lock to avoid
> > > > potential deadlock.
> > > > For example, l2sock_shutdown acquires these locks in the order:
> > > >   mutex_lock(&conn->chan_lock)
> > > >   l2cap_chan_lock(chan)
> > > >
> > > > However, l2cap_disconnect_req acquires chan->lock in
> > > > l2cap_get_chan_by_scid first and then acquires conn->chan_lock
> > > > before calling l2cap_chan_del. This means that these locks are
> > > > acquired in unexpected order, which leads to potential deadlock:
> > > >   l2cap_chan_lock(c)
> > > >   mutex_lock(&conn->chan_lock)
> > > >
> > > > This patch uses __l2cap_get_chan_by_scid to replace
> > > > l2cap_get_chan_by_scid and adjusts the locking order to avoid the
> > > > potential deadlock.
> >
> > This needs the fixes tag so we can backport it properly.
> >
> > > > Signed-off-by: Ying Hsu <[e-mail]>
> > > > ---
> > > > This commit has been tested on a Chromebook device.
> > > >
> > > > Changes in v2:
> > > > - Adding the prefix "Bluetooth:" to subject line.
> > > >
> > > >  net/bluetooth/l2cap_core.c | 26 ++++++++++++++++++++------
> > > >  1 file changed, 20 insertions(+), 6 deletions(-)
> > > >
> > > > diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
> > > > index 376b523c7b26..8f08192b8fb1 100644
> > > > --- a/net/bluetooth/l2cap_core.c
> > > > +++ b/net/bluetooth/l2cap_core.c
> > > > @@ -4651,8 +4651,16 @@ static inline int l2cap_disconnect_req(struct l2cap_conn *conn,
> > > >
> > > >       BT_DBG("scid 0x%4.4x dcid 0x%4.4x", scid, dcid);
> > > >
> > > > -     chan = l2cap_get_chan_by_scid(conn, dcid);
> > > > +     mutex_lock(&conn->chan_lock);
> > > > +     chan = __l2cap_get_chan_by_scid(conn, dcid);
> > > > +     if (chan) {
> > > > +             chan = l2cap_chan_hold_unless_zero(chan);
> > > > +             if (chan)
> > > > +                     l2cap_chan_lock(chan);
> > > > +     }
> > > > +
> > > >       if (!chan) {
> > > > +             mutex_unlock(&conn->chan_lock);
> > > >               cmd_reject_invalid_cid(conn, cmd->ident, dcid, scid);
> > > >               return 0;
> > > >       }
> > >
> > > Hi Ying,
> > >
> > > The conditional setting of chan and calling l2cap_chan_lock()
> > > is both non-trivial and repeated. It seems that it ought to be
> > > in a helper.
> > >
> > > Something like this (I'm sure a better function name can be chosen):
> > >
> > >         chan = __l2cap_get_and_lock_chan_by_scid(conn, dcid);
> > >         if (!chan) {
> > >                 ...
> > >         }
> > >
> > >         ...
> >
> > Or perhaps we could do something like l2cap_del_chan_by_scid:
> >
> > https://gist.github.com/Vudentz/e513859ecb31e79c947dfcb4b5c60453
> >
> > --
> > Luiz Augusto von Dentz

Ying Hsu May 25, 2023, 4:16 a.m. UTC | #6

Hi Luiz,

The proposal solves the deadlock but might introduce other problems as
it breaks the order of l2cap_chan_del.
There are another way to resolve the deadlock:
```
@@ -4663,7 +4663,9 @@ static inline int l2cap_disconnect_req(struct
l2cap_conn *conn,

        chan->ops->set_shutdown(chan);

+       l2cap_chan_unlock(chan);
        mutex_lock(&conn->chan_lock);
+       l2cap_chan_lock(chan);
        l2cap_chan_del(chan, ECONNRESET);
        mutex_unlock(&conn->chan_lock);
 ```

If you're okay with it, I'll do some verification and post a full patch.

Best regards,
Ying

On Thu, May 25, 2023 at 2:56 AM Luiz Augusto von Dentz
<[e-mail]> wrote:
>
> Hi Ying,
>
> On Wed, May 24, 2023 at 3:54 AM Ying Hsu <[e-mail]> wrote:
> >
> > Hi Simon,
> >
> > I understand your concern about the repeated code.
> > However, simply hiding the locking logic in another function
> > introduces hidden assumptions.
> > For this patch, I would like to fix the deadlock in a simple and easy
> > to understand way.
> > We can always refactor the l2cap_chan utility functions later.
> >
> > Hi Luis,
> >
> > I'll add a fixes tag in the next version.
>
> And how about doing this:
>
> https://gist.github.com/Vudentz/e513859ecb31e79c947dfcb4b5c60453
>
> > Best regards,
> > Ying
> >
> >
> > On Wed, May 24, 2023 at 3:06 AM Luiz Augusto von Dentz
> > <[e-mail]> wrote:
> > >
> > > Hi Simon, Ying,
> > >
> > > On Tue, May 23, 2023 at 3:04 AM Simon Horman <[e-mail]> wrote:
> > > >
> > > > On Mon, May 22, 2023 at 11:41:51PM +0000, Ying Hsu wrote:
> > > > > L2CAP assumes that the locks conn->chan_lock and chan->lock are
> > > > > acquired in the order conn->chan_lock, chan->lock to avoid
> > > > > potential deadlock.
> > > > > For example, l2sock_shutdown acquires these locks in the order:
> > > > >   mutex_lock(&conn->chan_lock)
> > > > >   l2cap_chan_lock(chan)
> > > > >
> > > > > However, l2cap_disconnect_req acquires chan->lock in
> > > > > l2cap_get_chan_by_scid first and then acquires conn->chan_lock
> > > > > before calling l2cap_chan_del. This means that these locks are
> > > > > acquired in unexpected order, which leads to potential deadlock:
> > > > >   l2cap_chan_lock(c)
> > > > >   mutex_lock(&conn->chan_lock)
> > > > >
> > > > > This patch uses __l2cap_get_chan_by_scid to replace
> > > > > l2cap_get_chan_by_scid and adjusts the locking order to avoid the
> > > > > potential deadlock.
> > >
> > > This needs the fixes tag so we can backport it properly.
> > >
> > > > > Signed-off-by: Ying Hsu <[e-mail]>
> > > > > ---
> > > > > This commit has been tested on a Chromebook device.
> > > > >
> > > > > Changes in v2:
> > > > > - Adding the prefix "Bluetooth:" to subject line.
> > > > >
> > > > >  net/bluetooth/l2cap_core.c | 26 ++++++++++++++++++++------
> > > > >  1 file changed, 20 insertions(+), 6 deletions(-)
> > > > >
> > > > > diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
> > > > > index 376b523c7b26..8f08192b8fb1 100644
> > > > > --- a/net/bluetooth/l2cap_core.c
> > > > > +++ b/net/bluetooth/l2cap_core.c
> > > > > @@ -4651,8 +4651,16 @@ static inline int l2cap_disconnect_req(struct l2cap_conn *conn,
> > > > >
> > > > >       BT_DBG("scid 0x%4.4x dcid 0x%4.4x", scid, dcid);
> > > > >
> > > > > -     chan = l2cap_get_chan_by_scid(conn, dcid);
> > > > > +     mutex_lock(&conn->chan_lock);
> > > > > +     chan = __l2cap_get_chan_by_scid(conn, dcid);
> > > > > +     if (chan) {
> > > > > +             chan = l2cap_chan_hold_unless_zero(chan);
> > > > > +             if (chan)
> > > > > +                     l2cap_chan_lock(chan);
> > > > > +     }
> > > > > +
> > > > >       if (!chan) {
> > > > > +             mutex_unlock(&conn->chan_lock);
> > > > >               cmd_reject_invalid_cid(conn, cmd->ident, dcid, scid);
> > > > >               return 0;
> > > > >       }
> > > >
> > > > Hi Ying,
> > > >
> > > > The conditional setting of chan and calling l2cap_chan_lock()
> > > > is both non-trivial and repeated. It seems that it ought to be
> > > > in a helper.
> > > >
> > > > Something like this (I'm sure a better function name can be chosen):
> > > >
> > > >         chan = __l2cap_get_and_lock_chan_by_scid(conn, dcid);
> > > >         if (!chan) {
> > > >                 ...
> > > >         }
> > > >
> > > >         ...
> > >
> > > Or perhaps we could do something like l2cap_del_chan_by_scid:
> > >
> > > https://gist.github.com/Vudentz/e513859ecb31e79c947dfcb4b5c60453
> > >
> > > --
> > > Luiz Augusto von Dentz
>
>
>
> --
> Luiz Augusto von Dentz

Ying Hsu May 30, 2023, 5:07 a.m. UTC | #7

Gentle ping, Luiz.


On Thu, May 25, 2023 at 12:16 PM Ying Hsu <[e-mail]> wrote:
>
> Hi Luiz,
>
> The proposal solves the deadlock but might introduce other problems as
> it breaks the order of l2cap_chan_del.
> There are another way to resolve the deadlock:
> ```
> @@ -4663,7 +4663,9 @@ static inline int l2cap_disconnect_req(struct
> l2cap_conn *conn,
>
>         chan->ops->set_shutdown(chan);
>
> +       l2cap_chan_unlock(chan);
>         mutex_lock(&conn->chan_lock);
> +       l2cap_chan_lock(chan);
>         l2cap_chan_del(chan, ECONNRESET);
>         mutex_unlock(&conn->chan_lock);
>  ```
>
> If you're okay with it, I'll do some verification and post a full patch.
>
> Best regards,
> Ying
>
> On Thu, May 25, 2023 at 2:56 AM Luiz Augusto von Dentz
> <[e-mail]> wrote:
> >
> > Hi Ying,
> >
> > On Wed, May 24, 2023 at 3:54 AM Ying Hsu <[e-mail]> wrote:
> > >
> > > Hi Simon,
> > >
> > > I understand your concern about the repeated code.
> > > However, simply hiding the locking logic in another function
> > > introduces hidden assumptions.
> > > For this patch, I would like to fix the deadlock in a simple and easy
> > > to understand way.
> > > We can always refactor the l2cap_chan utility functions later.
> > >
> > > Hi Luis,
> > >
> > > I'll add a fixes tag in the next version.
> >
> > And how about doing this:
> >
> > https://gist.github.com/Vudentz/e513859ecb31e79c947dfcb4b5c60453
> >
> > > Best regards,
> > > Ying
> > >
> > >
> > > On Wed, May 24, 2023 at 3:06 AM Luiz Augusto von Dentz
> > > <[e-mail]> wrote:
> > > >
> > > > Hi Simon, Ying,
> > > >
> > > > On Tue, May 23, 2023 at 3:04 AM Simon Horman <[e-mail]> wrote:
> > > > >
> > > > > On Mon, May 22, 2023 at 11:41:51PM +0000, Ying Hsu wrote:
> > > > > > L2CAP assumes that the locks conn->chan_lock and chan->lock are
> > > > > > acquired in the order conn->chan_lock, chan->lock to avoid
> > > > > > potential deadlock.
> > > > > > For example, l2sock_shutdown acquires these locks in the order:
> > > > > >   mutex_lock(&conn->chan_lock)
> > > > > >   l2cap_chan_lock(chan)
> > > > > >
> > > > > > However, l2cap_disconnect_req acquires chan->lock in
> > > > > > l2cap_get_chan_by_scid first and then acquires conn->chan_lock
> > > > > > before calling l2cap_chan_del. This means that these locks are
> > > > > > acquired in unexpected order, which leads to potential deadlock:
> > > > > >   l2cap_chan_lock(c)
> > > > > >   mutex_lock(&conn->chan_lock)
> > > > > >
> > > > > > This patch uses __l2cap_get_chan_by_scid to replace
> > > > > > l2cap_get_chan_by_scid and adjusts the locking order to avoid the
> > > > > > potential deadlock.
> > > >
> > > > This needs the fixes tag so we can backport it properly.
> > > >
> > > > > > Signed-off-by: Ying Hsu <[e-mail]>
> > > > > > ---
> > > > > > This commit has been tested on a Chromebook device.
> > > > > >
> > > > > > Changes in v2:
> > > > > > - Adding the prefix "Bluetooth:" to subject line.
> > > > > >
> > > > > >  net/bluetooth/l2cap_core.c | 26 ++++++++++++++++++++------
> > > > > >  1 file changed, 20 insertions(+), 6 deletions(-)
> > > > > >
> > > > > > diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
> > > > > > index 376b523c7b26..8f08192b8fb1 100644
> > > > > > --- a/net/bluetooth/l2cap_core.c
> > > > > > +++ b/net/bluetooth/l2cap_core.c
> > > > > > @@ -4651,8 +4651,16 @@ static inline int l2cap_disconnect_req(struct l2cap_conn *conn,
> > > > > >
> > > > > >       BT_DBG("scid 0x%4.4x dcid 0x%4.4x", scid, dcid);
> > > > > >
> > > > > > -     chan = l2cap_get_chan_by_scid(conn, dcid);
> > > > > > +     mutex_lock(&conn->chan_lock);
> > > > > > +     chan = __l2cap_get_chan_by_scid(conn, dcid);
> > > > > > +     if (chan) {
> > > > > > +             chan = l2cap_chan_hold_unless_zero(chan);
> > > > > > +             if (chan)
> > > > > > +                     l2cap_chan_lock(chan);
> > > > > > +     }
> > > > > > +
> > > > > >       if (!chan) {
> > > > > > +             mutex_unlock(&conn->chan_lock);
> > > > > >               cmd_reject_invalid_cid(conn, cmd->ident, dcid, scid);
> > > > > >               return 0;
> > > > > >       }
> > > > >
> > > > > Hi Ying,
> > > > >
> > > > > The conditional setting of chan and calling l2cap_chan_lock()
> > > > > is both non-trivial and repeated. It seems that it ought to be
> > > > > in a helper.
> > > > >
> > > > > Something like this (I'm sure a better function name can be chosen):
> > > > >
> > > > >         chan = __l2cap_get_and_lock_chan_by_scid(conn, dcid);
> > > > >         if (!chan) {
> > > > >                 ...
> > > > >         }
> > > > >
> > > > >         ...
> > > >
> > > > Or perhaps we could do something like l2cap_del_chan_by_scid:
> > > >
> > > > https://gist.github.com/Vudentz/e513859ecb31e79c947dfcb4b5c60453
> > > >
> > > > --
> > > > Luiz Augusto von Dentz
> >
> >
> >
> > --
> > Luiz Augusto von Dentz

Luiz Augusto von Dentz May 30, 2023, 8:50 p.m. UTC | #8

Hi Ying,

On Mon, May 29, 2023 at 10:08 PM Ying Hsu <[e-mail]> wrote:
>
> Gentle ping, Luiz.
>
>
> On Thu, May 25, 2023 at 12:16 PM Ying Hsu <[e-mail]> wrote:
> >
> > Hi Luiz,
> >
> > The proposal solves the deadlock but might introduce other problems as
> > it breaks the order of l2cap_chan_del.
> > There are another way to resolve the deadlock:
> > ```
> > @@ -4663,7 +4663,9 @@ static inline int l2cap_disconnect_req(struct
> > l2cap_conn *conn,
> >
> >         chan->ops->set_shutdown(chan);
> >
> > +       l2cap_chan_unlock(chan);
> >         mutex_lock(&conn->chan_lock);
> > +       l2cap_chan_lock(chan);
> >         l2cap_chan_del(chan, ECONNRESET);
> >         mutex_unlock(&conn->chan_lock);
> >  ```

Yeah, I kind of like this better, that said I don't think changing the
order of l2cap_chan_del matters that much but it does change the
callback teardown sequence so perhaps we should stick to a simpler
solution for now.

Please submit an updated version so we can move forward with it.

> > If you're okay with it, I'll do some verification and post a full patch.
> >
> > Best regards,
> > Ying
> >
> > On Thu, May 25, 2023 at 2:56 AM Luiz Augusto von Dentz
> > <[e-mail]> wrote:
> > >
> > > Hi Ying,
> > >
> > > On Wed, May 24, 2023 at 3:54 AM Ying Hsu <[e-mail]> wrote:
> > > >
> > > > Hi Simon,
> > > >
> > > > I understand your concern about the repeated code.
> > > > However, simply hiding the locking logic in another function
> > > > introduces hidden assumptions.
> > > > For this patch, I would like to fix the deadlock in a simple and easy
> > > > to understand way.
> > > > We can always refactor the l2cap_chan utility functions later.
> > > >
> > > > Hi Luis,
> > > >
> > > > I'll add a fixes tag in the next version.
> > >
> > > And how about doing this:
> > >
> > > https://gist.github.com/Vudentz/e513859ecb31e79c947dfcb4b5c60453
> > >
> > > > Best regards,
> > > > Ying
> > > >
> > > >
> > > > On Wed, May 24, 2023 at 3:06 AM Luiz Augusto von Dentz
> > > > <[e-mail]> wrote:
> > > > >
> > > > > Hi Simon, Ying,
> > > > >
> > > > > On Tue, May 23, 2023 at 3:04 AM Simon Horman <[e-mail]> wrote:
> > > > > >
> > > > > > On Mon, May 22, 2023 at 11:41:51PM +0000, Ying Hsu wrote:
> > > > > > > L2CAP assumes that the locks conn->chan_lock and chan->lock are
> > > > > > > acquired in the order conn->chan_lock, chan->lock to avoid
> > > > > > > potential deadlock.
> > > > > > > For example, l2sock_shutdown acquires these locks in the order:
> > > > > > >   mutex_lock(&conn->chan_lock)
> > > > > > >   l2cap_chan_lock(chan)
> > > > > > >
> > > > > > > However, l2cap_disconnect_req acquires chan->lock in
> > > > > > > l2cap_get_chan_by_scid first and then acquires conn->chan_lock
> > > > > > > before calling l2cap_chan_del. This means that these locks are
> > > > > > > acquired in unexpected order, which leads to potential deadlock:
> > > > > > >   l2cap_chan_lock(c)
> > > > > > >   mutex_lock(&conn->chan_lock)
> > > > > > >
> > > > > > > This patch uses __l2cap_get_chan_by_scid to replace
> > > > > > > l2cap_get_chan_by_scid and adjusts the locking order to avoid the
> > > > > > > potential deadlock.
> > > > >
> > > > > This needs the fixes tag so we can backport it properly.
> > > > >
> > > > > > > Signed-off-by: Ying Hsu <[e-mail]>
> > > > > > > ---
> > > > > > > This commit has been tested on a Chromebook device.
> > > > > > >
> > > > > > > Changes in v2:
> > > > > > > - Adding the prefix "Bluetooth:" to subject line.
> > > > > > >
> > > > > > >  net/bluetooth/l2cap_core.c | 26 ++++++++++++++++++++------
> > > > > > >  1 file changed, 20 insertions(+), 6 deletions(-)
> > > > > > >
> > > > > > > diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
> > > > > > > index 376b523c7b26..8f08192b8fb1 100644
> > > > > > > --- a/net/bluetooth/l2cap_core.c
> > > > > > > +++ b/net/bluetooth/l2cap_core.c
> > > > > > > @@ -4651,8 +4651,16 @@ static inline int l2cap_disconnect_req(struct l2cap_conn *conn,
> > > > > > >
> > > > > > >       BT_DBG("scid 0x%4.4x dcid 0x%4.4x", scid, dcid);
> > > > > > >
> > > > > > > -     chan = l2cap_get_chan_by_scid(conn, dcid);
> > > > > > > +     mutex_lock(&conn->chan_lock);
> > > > > > > +     chan = __l2cap_get_chan_by_scid(conn, dcid);
> > > > > > > +     if (chan) {
> > > > > > > +             chan = l2cap_chan_hold_unless_zero(chan);
> > > > > > > +             if (chan)
> > > > > > > +                     l2cap_chan_lock(chan);
> > > > > > > +     }
> > > > > > > +
> > > > > > >       if (!chan) {
> > > > > > > +             mutex_unlock(&conn->chan_lock);
> > > > > > >               cmd_reject_invalid_cid(conn, cmd->ident, dcid, scid);
> > > > > > >               return 0;
> > > > > > >       }
> > > > > >
> > > > > > Hi Ying,
> > > > > >
> > > > > > The conditional setting of chan and calling l2cap_chan_lock()
> > > > > > is both non-trivial and repeated. It seems that it ought to be
> > > > > > in a helper.
> > > > > >
> > > > > > Something like this (I'm sure a better function name can be chosen):
> > > > > >
> > > > > >         chan = __l2cap_get_and_lock_chan_by_scid(conn, dcid);
> > > > > >         if (!chan) {
> > > > > >                 ...
> > > > > >         }
> > > > > >
> > > > > >         ...
> > > > >
> > > > > Or perhaps we could do something like l2cap_del_chan_by_scid:
> > > > >
> > > > > https://gist.github.com/Vudentz/e513859ecb31e79c947dfcb4b5c60453
> > > > >
> > > > > --
> > > > > Luiz Augusto von Dentz
> > >
> > >
> > >
> > > --
> > > Luiz Augusto von Dentz

Patch

diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index 376b523c7b26..8f08192b8fb1 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -4651,8 +4651,16 @@  static inline int l2cap_disconnect_req(struct l2cap_conn *conn,
 
 	BT_DBG("scid 0x%4.4x dcid 0x%4.4x", scid, dcid);
 
-	chan = l2cap_get_chan_by_scid(conn, dcid);
+	mutex_lock(&conn->chan_lock);
+	chan = __l2cap_get_chan_by_scid(conn, dcid);
+	if (chan) {
+		chan = l2cap_chan_hold_unless_zero(chan);
+		if (chan)
+			l2cap_chan_lock(chan);
+	}
+
 	if (!chan) {
+		mutex_unlock(&conn->chan_lock);
 		cmd_reject_invalid_cid(conn, cmd->ident, dcid, scid);
 		return 0;
 	}
@@ -4663,14 +4671,13 @@  static inline int l2cap_disconnect_req(struct l2cap_conn *conn,
 
 	chan->ops->set_shutdown(chan);
 
-	mutex_lock(&conn->chan_lock);
 	l2cap_chan_del(chan, ECONNRESET);
-	mutex_unlock(&conn->chan_lock);
 
 	chan->ops->close(chan);
 
 	l2cap_chan_unlock(chan);
 	l2cap_chan_put(chan);
+	mutex_unlock(&conn->chan_lock);
 
 	return 0;
 }
@@ -4691,25 +4698,32 @@  static inline int l2cap_disconnect_rsp(struct l2cap_conn *conn,
 
 	BT_DBG("dcid 0x%4.4x scid 0x%4.4x", dcid, scid);
 
-	chan = l2cap_get_chan_by_scid(conn, scid);
+	mutex_lock(&conn->chan_lock);
+	chan = __l2cap_get_chan_by_scid(conn, scid);
+	if (chan) {
+		chan = l2cap_chan_hold_unless_zero(chan);
+		if (chan)
+			l2cap_chan_lock(chan);
+	}
 	if (!chan) {
+		mutex_unlock(&conn->chan_lock);
 		return 0;
 	}
 
 	if (chan->state != BT_DISCONN) {
 		l2cap_chan_unlock(chan);
 		l2cap_chan_put(chan);
+		mutex_unlock(&conn->chan_lock);
 		return 0;
 	}
 
-	mutex_lock(&conn->chan_lock);
 	l2cap_chan_del(chan, 0);
-	mutex_unlock(&conn->chan_lock);
 
 	chan->ops->close(chan);
 
 	l2cap_chan_unlock(chan);
 	l2cap_chan_put(chan);
+	mutex_unlock(&conn->chan_lock);
 
 	return 0;
 }