diff mbox series

[v2] net: ks8851: Fix deadlock with the SPI chip variant

Message ID 20240704174756.1225995-1-rwahl@gmx.de (mailing list archive)
State Superseded
Delegated to: Netdev Maintainers
Headers show
Series [v2] net: ks8851: Fix deadlock with the SPI chip variant | expand

Checks

Context Check Description
netdev/series_format warning Single patches do not need cover letters; Target tree name not specified in the subject
netdev/tree_selection success Guessed tree name to be net-next
netdev/ynl success Generated files up to date; no warnings/errors; no diff in generated;
netdev/fixes_present success Fixes tag not required for -next series
netdev/header_inline success No static functions without inline keyword in header files
netdev/build_32bit success Errors and warnings before: 839 this patch: 839
netdev/build_tools success No tools touched, skip
netdev/cc_maintainers warning 2 maintainers not CCed: marex@denx.de broonie@kernel.org
netdev/build_clang success Errors and warnings before: 846 this patch: 846
netdev/verify_signedoff success Signed-off-by tag matches author and committer
netdev/deprecated_api success None detected
netdev/check_selftest success No net selftest shell script
netdev/verify_fixes success Fixes tag looks correct
netdev/build_allmodconfig_warn success Errors and warnings before: 846 this patch: 846
netdev/checkpatch success total: 0 errors, 0 warnings, 0 checks, 29 lines checked
netdev/build_clang_rust success No Rust files in patch. Skipping build
netdev/kdoc success Errors and warnings before: 12 this patch: 12
netdev/source_inline success Was 0 now: 0
netdev/contest success net-next-2024-07-05--06-00 (tests: 695)

Commit Message

Ronald Wahl July 4, 2024, 5:47 p.m. UTC
From: Ronald Wahl <ronald.wahl@raritan.com>

When SMP is enabled and spinlocks are actually functional then there is
a deadlock with the 'statelock' spinlock between ks8851_start_xmit_spi
and ks8851_irq:

    watchdog: BUG: soft lockup - CPU#0 stuck for 27s!
    call trace:
      queued_spin_lock_slowpath+0x100/0x284
      do_raw_spin_lock+0x34/0x44
      ks8851_start_xmit_spi+0x30/0xb8
      ks8851_start_xmit+0x14/0x20
      netdev_start_xmit+0x40/0x6c
      dev_hard_start_xmit+0x6c/0xbc
      sch_direct_xmit+0xa4/0x22c
      __qdisc_run+0x138/0x3fc
      qdisc_run+0x24/0x3c
      net_tx_action+0xf8/0x130
      handle_softirqs+0x1ac/0x1f0
      __do_softirq+0x14/0x20
      ____do_softirq+0x10/0x1c
      call_on_irq_stack+0x3c/0x58
      do_softirq_own_stack+0x1c/0x28
      __irq_exit_rcu+0x54/0x9c
      irq_exit_rcu+0x10/0x1c
      el1_interrupt+0x38/0x50
      el1h_64_irq_handler+0x18/0x24
      el1h_64_irq+0x64/0x68
      __netif_schedule+0x6c/0x80
      netif_tx_wake_queue+0x38/0x48
      ks8851_irq+0xb8/0x2c8
      irq_thread_fn+0x2c/0x74
      irq_thread+0x10c/0x1b0
      kthread+0xc8/0xd8
      ret_from_fork+0x10/0x20

This issue has not been identified earlier because tests were done on
a device with SMP disabled and so spinlocks were actually NOPs.

Now use spin_(un)lock_bh for TX queue related locking to avoid execution
of softirq work synchronously that would lead to a deadlock.

Fixes: 3dc5d4454545 ("net: ks8851: Fix TX stall caused by TX buffer overrun")
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Jakub Kicinski <kuba@kernel.org>
Cc: Paolo Abeni <pabeni@redhat.com>
Cc: Simon Horman <horms@kernel.org>
Cc: netdev@vger.kernel.org
Cc: stable@vger.kernel.org # 5.10+
Signed-off-by: Ronald Wahl <ronald.wahl@raritan.com>
---
V2: - use spin_lock_bh instead of moving netif_wake_queue outside of
      locked region (doing the same in the start_xmit function)
    - add missing net: tag

 drivers/net/ethernet/micrel/ks8851_common.c | 4 ++--
 drivers/net/ethernet/micrel/ks8851_spi.c    | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

--
2.45.2

Comments

Jakub Kicinski July 6, 2024, 12:39 a.m. UTC | #1
On Thu,  4 Jul 2024 19:47:56 +0200 Ronald Wahl wrote:
> --- a/drivers/net/ethernet/micrel/ks8851_spi.c
> +++ b/drivers/net/ethernet/micrel/ks8851_spi.c
> @@ -385,7 +385,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,
>  	netif_dbg(ks, tx_queued, ks->netdev,
>  		  "%s: skb %p, %d@%p\n", __func__, skb, skb->len, skb->data);
> 
> -	spin_lock(&ks->statelock);
> +	spin_lock_bh(&ks->statelock);
> 
>  	if (ks->queued_len + needed > ks->tx_space) {
>  		netif_stop_queue(dev);
> @@ -395,7 +395,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,
>  		skb_queue_tail(&ks->txq, skb);
>  	}
> 
> -	spin_unlock(&ks->statelock);
> +	spin_unlock_bh(&ks->statelock);

this one probably can stay as spin_lock() since networking stack only
calls xmit in BH context. But I see 2 other spin_lock(statelock) in the
driver which I'm not as sure about. Any taking of this lock has to be
_bh() unless you're sure the caller is already in BH.
Ronald Wahl July 6, 2024, 8:38 a.m. UTC | #2
On 06.07.24 02:39, Jakub Kicinski wrote:
> On Thu,  4 Jul 2024 19:47:56 +0200 Ronald Wahl wrote:
>> --- a/drivers/net/ethernet/micrel/ks8851_spi.c
>> +++ b/drivers/net/ethernet/micrel/ks8851_spi.c
>> @@ -385,7 +385,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,
>>      netif_dbg(ks, tx_queued, ks->netdev,
>>                "%s: skb %p, %d@%p\n", __func__, skb, skb->len, skb->data);
>>
>> -    spin_lock(&ks->statelock);
>> +    spin_lock_bh(&ks->statelock);
>>
>>      if (ks->queued_len + needed > ks->tx_space) {
>>              netif_stop_queue(dev);
>> @@ -395,7 +395,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,
>>              skb_queue_tail(&ks->txq, skb);
>>      }
>>
>> -    spin_unlock(&ks->statelock);
>> +    spin_unlock_bh(&ks->statelock);
>
> this one probably can stay as spin_lock() since networking stack only
> calls xmit in BH context.

I already suspected this it was more a mental hint here. I will remove it.

> But I see 2 other spin_lock(statelock) in the
> driver which I'm not as sure about. Any taking of this lock has to be
> _bh() unless you're sure the caller is already in BH.

The other two instances are not in BH context as far as I know but also
do not interfere with BH. The one in ks8861_tx_work protects only
variable assignments used only inside the driver and the one in
ks8851_set_rx_mode also only does some driver local variable stuff and a
schedule_work which as far as I know has nothing to do with BH because
workqueues are running in process context. Am I wrong here?

- ron
Ronald Wahl July 6, 2024, 9:22 a.m. UTC | #3
On 06.07.24 10:38, Ronald Wahl wrote:
> On 06.07.24 02:39, Jakub Kicinski wrote:
>> On Thu,  4 Jul 2024 19:47:56 +0200 Ronald Wahl wrote:
>>> --- a/drivers/net/ethernet/micrel/ks8851_spi.c
>>> +++ b/drivers/net/ethernet/micrel/ks8851_spi.c
>>> @@ -385,7 +385,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct
>>> sk_buff *skb,
>>>       netif_dbg(ks, tx_queued, ks->netdev,
>>>             "%s: skb %p, %d@%p\n", __func__, skb, skb->len, skb->data);
>>>
>>> -    spin_lock(&ks->statelock);
>>> +    spin_lock_bh(&ks->statelock);
>>>
>>>       if (ks->queued_len + needed > ks->tx_space) {
>>>           netif_stop_queue(dev);
>>> @@ -395,7 +395,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct
>>> sk_buff *skb,
>>>           skb_queue_tail(&ks->txq, skb);
>>>       }
>>>
>>> -    spin_unlock(&ks->statelock);
>>> +    spin_unlock_bh(&ks->statelock);
>>
>> this one probably can stay as spin_lock() since networking stack only
>> calls xmit in BH context.
>
> I already suspected this it was more a mental hint here. I will remove it.
>
>> But I see 2 other spin_lock(statelock) in the
>> driver which I'm not as sure about. Any taking of this lock has to be
>> _bh() unless you're sure the caller is already in BH.
>
> The other two instances are not in BH context as far as I know but also
> do not interfere with BH. The one in ks8861_tx_work protects only
> variable assignments used only inside the driver and the one in
> ks8851_set_rx_mode also only does some driver local variable stuff and a
> schedule_work which as far as I know has nothing to do with BH because
> workqueues are running in process context. Am I wrong here?

I guess I found a misunderstanding on my side: I was assuming that a
softirq cannot asynchronously interrupt a spin lock protected section. Maybe
this is wrong. In the one place where I'm waking the queue again the
spin_lock_bh avoids synchronously triggering the BH processing while still
holding a spinlock.

I will use the _bh variants on the two other places.
  - ron
gregkh@linuxfoundation.org July 6, 2024, 9:27 a.m. UTC | #4
On Sat, Jul 06, 2024 at 11:22:11AM +0200, Ronald Wahl wrote:
> This e-mail, and any document attached hereby, may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized, direct or indirect, copying, disclosure, distribution or other use of the material or parts thereof is strictly forbidden.
> 

Now deleted
diff mbox series

Patch

diff --git a/drivers/net/ethernet/micrel/ks8851_common.c b/drivers/net/ethernet/micrel/ks8851_common.c
index 6453c92f0fa7..51fb6c27153e 100644
--- a/drivers/net/ethernet/micrel/ks8851_common.c
+++ b/drivers/net/ethernet/micrel/ks8851_common.c
@@ -352,11 +352,11 @@  static irqreturn_t ks8851_irq(int irq, void *_ks)
 		netif_dbg(ks, intr, ks->netdev,
 			  "%s: txspace %d\n", __func__, tx_space);

-		spin_lock(&ks->statelock);
+		spin_lock_bh(&ks->statelock);
 		ks->tx_space = tx_space;
 		if (netif_queue_stopped(ks->netdev))
 			netif_wake_queue(ks->netdev);
-		spin_unlock(&ks->statelock);
+		spin_unlock_bh(&ks->statelock);
 	}

 	if (status & IRQ_SPIBEI) {
diff --git a/drivers/net/ethernet/micrel/ks8851_spi.c b/drivers/net/ethernet/micrel/ks8851_spi.c
index 670c1de966db..818e1ce3227b 100644
--- a/drivers/net/ethernet/micrel/ks8851_spi.c
+++ b/drivers/net/ethernet/micrel/ks8851_spi.c
@@ -385,7 +385,7 @@  static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,
 	netif_dbg(ks, tx_queued, ks->netdev,
 		  "%s: skb %p, %d@%p\n", __func__, skb, skb->len, skb->data);

-	spin_lock(&ks->statelock);
+	spin_lock_bh(&ks->statelock);

 	if (ks->queued_len + needed > ks->tx_space) {
 		netif_stop_queue(dev);
@@ -395,7 +395,7 @@  static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,
 		skb_queue_tail(&ks->txq, skb);
 	}

-	spin_unlock(&ks->statelock);
+	spin_unlock_bh(&ks->statelock);
 	if (ret == NETDEV_TX_OK)
 		schedule_work(&kss->tx_work);