答复: [PATCH v3] usb: gadget: u_serial: Disable ep before setting port to null to fix the crash caused by port being null

Commit Message

胡连勤 Feb. 5, 2025, 6:38 a.m. UTC

Hi Greg, Dan,Jon, other kernel experts:

> On Tue, Jan 21, 2025 at 12:19:37PM +0000, Jon Hunter wrote:
> > > > > [   59.918177] tegra-xudc 3550000.usb: ep 3 disabled [
> > > > > 59.923097] tegra-xudc 3550000.usb: ep 2 disabled [   59.927955]
> > > > > tegra-xudc 3550000.usb: ep 5 disabled [   80.911432] rcu: INFO:
> > > > > rcu_preempt self-detected stall on CPU [   80.917354] rcu:
> > > > > 6-....: (5248 ticks this GP)
> > > > > idle=ec24/1/0x4000000000000000 softirq=1213/1213 fqs=2623 [
> > > > > 80.927146] rcu:     (t=5253 jiffies g=3781 q=1490 ncpus=12) [
> > > > > 80.932704] Sending NMI from CPU 6 to CPUs 2:
> > > > > [   90.981555] CPU: 6 UID: 0 PID: 18 Comm: rcu_exp_gp_kthr Not
> > > > > tainted 6.13.0-rc7-00043-g619f0b6fad52 #1 [   90.981558]
> > > > > Hardware name: NVIDIA NVIDIA Jetson AGX Orin Developer
> > > > > Kit/Jetson, BIOS 00.0.0-dev-main_92e5ae_88fd1_296de
> > > > > 12/16/2024
> > > > > [   90.981559] pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT
> > > > > -SSBS BTYPE=--) [   90.981562] pc :
> > > > > smp_call_function_single+0xdc/0x1a0
> > > > > [   90.981574] lr : __sync_rcu_exp_select_node_cpus+0x228/0x3c0
> > > > > [   90.981578] sp : ffff800082eb3cd0 [   90.981579] x29:
> > > > > ffff800082eb3cd0 x28: 0000000000000010 x27:
> > > > > ffff0000802933c0
> > > > > [   90.981582] x26: ffff0007a8a1d700 x25: ffff800082895500 x24:
> > > > > ffff800080132018
> > > > > [   90.981584] x23: 0000000000000014 x22: ffff800081fb7700 x21:
> > > > > ffff80008280d970
> > > > > [   90.981586] x20: 0000000000000feb x19: ffff800082eb3d00 x18:
> > > > > 0000000000000000
> > > > > [   90.981588] x17: 0000000000000000 x16: 0000000000000000 x15:
> > > > > 0000000000000000
> > > > > [   90.981590] x14: ffff000080293440 x13: 0000000000000001 x12:
> > > > > 0000000000000000
> > > > > [   90.981591] x11: ffff800081fb2388 x10: ffff0000802933c0 x9 :
> > > > > 0000000000000001
> > > > > [   90.981593] x8 : 0000000000000040 x7 : 0000000000017068 x6 :
> > > > > ffff800080132018
> > > > > [   90.981595] x5 : 0000000000000000 x4 : ffff0007a8a4f9c8 x3 :
> > > > > 0000000000000001
> > > > > [   90.981597] x2 : 0000000000000000 x1 : ffff0007a8a4f9c0 x0 :
> > > > > 0000000000000004
> > > > > [   90.981599] Call trace:
> > > > > [   90.981601]  smp_call_function_single+0xdc/0x1a0 (P) [
> > > > > 90.981605]  __sync_rcu_exp_select_node_cpus+0x228/0x3c0
> > > > > [   90.981607]  sync_rcu_exp_select_cpus+0x13c/0x2a0
> > > > > [   90.981609]  wait_rcu_exp_gp+0x18/0x30 [   90.981611]
> > > > > kthread_worker_fn+0xd0/0x188 [   90.981614]  kthread+0x118/0x11c
> > > > > [   90.981619]  ret_from_fork+0x10/0x20 [  101.416347] sched: DL
> > > > > replenish lagged too much
> > > > >
> > > >
> > > > Odd, you have a usb-serial gadget device in this system that is
> > > > disconnecting somehow?  That oops doesn't point to anything in the
> > > > usb gadget codebase, "all" we have done is move the call to
> > > > shutdown the endpoints to earlier in the disconnect function.
> > >
> > > Yes the board starts usb-serial and usb-ethernet gadget and on
> > > reboot when tearing it down I am seeing the above. As soon as it
> > > disables the tegra-xudc endpoints (as seen above) the board appears to
> stall.
> > >
> > > > I'm glad to revert this, but it feels really odd that this is
> > > > causing you an rcu stall issue.
> > >
> > > Thanks. I can't say I understand it either, but I am certain it is
> > > caused by this change.
> > >
> > > Happy to run any tests to narrow this down a bit.
> >
> >
> > I did a bit more looking at this and I see that we setup a USB gadget
> > device via the configfs as described in this doc [0]. The RCU stall
> > occurs when we attempt to disable the gadget on shutdown by ...
> >
> >  $ echo "" > /path/to/UDC
> >
> > Jon
> >
> > [0] https://docs.kernel.org/usb/gadget_configfs.html
> 
> It's an RCU stall so it's probably a locking bug.  The original code dropped the
> port->port_lock, called usb_ep_disable() then took the lock again.  There
> must have been a reason to drop the lock although I haven't found it yet.
> The new code moved the usb_ep_disable() under the lock.
> 
>   1431          port->suspended = false;
>   1432          spin_unlock(&port->port_lock);
>   1433          spin_unlock_irqrestore(&serial_port_lock, flags);
>   1434
>   1435          /* disable endpoints, aborting down any active I/O */
>   1436          usb_ep_disable(gser->out);
>   1437          usb_ep_disable(gser->in);
>   1438
>   1439          /* finally, free any unused/unusable I/O buffers */
>   1440          spin_lock_irqsave(&port->port_lock, flags);
>   1441          if (port->port.count == 0)
>   1442                  kfifo_free(&port->port_write_buf);
> 
> 
> It might help to try testing again with lockdep enabled.
> 

To solve this problem, I plan to refer to the methods in gserial_suspend and gserial_resume functions
and use spin lock to solve the competition between gs_read_complete and gserial_disconnect.

The patch is as follows:

Considering that in some extreme cases, when the unbind operation
is being executed, gserial_disconnect has already cleared gser->ioport,
triggering a gadget reconfiguration at this time and gs_read_complete
gets called afterwards, which results in accessing null pointer,
add a null pointer check to prevent this situation.

Added a static spinlock to prevent gser->ioport from becoming
null after the newly added check.

Call trace:
 gs_read_complete+0x58/0x240
 usb_gadget_giveback_request+0x40/0x160
 dwc3_remove_requests+0x170/0x484
 dwc3_ep0_out_start+0xb0/0x1d4
 __dwc3_gadget_start+0x25c/0x720
 kretprobe_trampoline.cfi_jt+0x0/0x8
 kretprobe_trampoline.cfi_jt+0x0/0x8
 udc_bind_to_driver+0x1d8/0x300
 usb_gadget_probe_driver+0xa8/0x1dc
 gadget_dev_desc_UDC_store+0x13c/0x188
 configfs_write_iter+0x160/0x1f4
 vfs_write+0x2d0/0x40c
 ksys_write+0x7c/0xf0
 __arm64_sys_write+0x20/0x30
 invoke_syscall+0x60/0x150
 el0_svc_common+0x8c/0xf8
 do_el0_svc+0x28/0xa0
 el0_svc+0x24/0x84

Fixes: c1dca562be8a ("usb gadget: split out serial core")
Cc: [e-mail]
Suggested-by: Prashanth K <[e-mail]>
Signed-off-by: Lianqin Hu <[e-mail]>

 drivers/usb/gadget/function/u_serial.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

Patch

diff --git a/drivers/usb/gadget/function/u_serial.c b/drivers/usb/gadget/function/u_serial.c
index 0a8c05b2746b..6f2c9715e8df 100644
--- a/drivers/usb/gadget/function/u_serial.c
+++ b/drivers/usb/gadget/function/u_serial.c
@@ -456,13 +456,24 @@  static void gs_rx_push(struct work_struct *work)
 
 static void gs_read_complete(struct usb_ep *ep, struct usb_request *req)
 {
-	struct gs_port	*port = ep->driver_data;
+	struct gs_port	*port;
+	unsigned long  flags;
+
+	spin_lock_irqsave(&serial_port_lock, flags);
+	port = ep->driver_data;
+
+	if (!port) {
+		spin_unlock_irqrestore(&serial_port_lock, flags);
+		return;
+	}
 
-	/* Queue all received data until the tty layer is ready for it. */
 	spin_lock(&port->port_lock);
+	spin_unlock(&serial_port_lock);
+
+	/* Queue all received data until the tty layer is ready for it. */
 	list_add_tail(&req->list, &port->read_queue);
 	schedule_delayed_work(&port->push, 0);
-	spin_unlock(&port->port_lock);
+	spin_unlock_irqrestore(&port->port_lock, flags);
 }
 
 static void gs_write_complete(struct usb_ep *ep, struct usb_request *req)