| Message ID | 20190425160540.10036-2-johan@kernel.org (mailing list archive) |
|---|---|
| State | Mainlined |
| Commit | 3f5edd58d040bfa4b74fb89bc02f0bc6b9cd06ab |
| Headers | show |
| Series | USB: fix tty unthrottle races | expand |
On Do, 2019-04-25 at 18:05 +0200, Johan Hovold wrote: > @@ -484,6 +503,12 @@ void usb_serial_generic_unthrottle(struct tty_struct *tty) > port->throttled = port->throttle_req = 0; > spin_unlock_irq(&port->lock); > > + /* > + * Matches the smp_mb__after_atomic() in > + * usb_serial_generic_read_bulk_callback(). > + */ > + smp_mb(); > + > if (was_throttled) > usb_serial_generic_submit_read_urbs(port, GFP_KERNEL); Doesn't the spin_unlock_irq() imply smp_mb()? Otherwise it looks correct to me. Regards Oliver
On Mon, Apr 29, 2019 at 11:50:58AM +0200, Oliver Neukum wrote: > On Do, 2019-04-25 at 18:05 +0200, Johan Hovold wrote: > > @@ -484,6 +503,12 @@ void usb_serial_generic_unthrottle(struct tty_struct *tty) > > port->throttled = port->throttle_req = 0; > > spin_unlock_irq(&port->lock); > > > > + /* > > + * Matches the smp_mb__after_atomic() in > > + * usb_serial_generic_read_bulk_callback(). > > + */ > > + smp_mb(); > > + > > if (was_throttled) > > usb_serial_generic_submit_read_urbs(port, GFP_KERNEL); > > > Doesn't the spin_unlock_irq() imply smp_mb()? > Otherwise it looks correct to me. No, spin_unlock_irq() is only a one-way barrier, and doesn't prevent later accesses from "moving" into the locked section. Johan
On Thu, Apr 25, 2019 at 06:05:36PM +0200, Johan Hovold wrote: > Fix two long-standing bugs which could potentially lead to memory > corruption or leave the port throttled until it is reopened (on weakly > ordered systems), respectively, when read-URB completion races with > unthrottle(). > > First, the URB must not be marked as free before processing is complete > to prevent it from being submitted by unthrottle() on another CPU. > > CPU 1 CPU 2 > ================ ================ > complete() unthrottle() > process_urb(); > smp_mb__before_atomic(); > set_bit(i, free); if (test_and_clear_bit(i, free)) > submit_urb(); > > Second, the URB must be marked as free before checking the throttled > flag to prevent unthrottle() on another CPU from failing to observe that > the URB needs to be submitted if complete() sees that the throttled flag > is set. > > CPU 1 CPU 2 > ================ ================ > complete() unthrottle() > set_bit(i, free); throttled = 0; > smp_mb__after_atomic(); smp_mb(); > if (throttled) if (test_and_clear_bit(i, free)) > return; submit_urb(); > > Note that test_and_clear_bit() only implies barriers when the test is > successful. To handle the case where the URB is still in use an explicit > barrier needs to be added to unthrottle() for the second race condition. > > Fixes: d83b405383c9 ("USB: serial: add support for multiple read urbs") > Signed-off-by: Johan Hovold <johan@kernel.org> Greg, I noticed you added a stable tag to the corresponding cdc-acm fix and think I should have added on one from the start to this one as well. Would you mind queuing this one up for stable? Upstream commit 3f5edd58d040bfa4b74fb89bc02f0bc6b9cd06ab. Thanks, Johan
On Mon, May 13, 2019 at 12:43:39PM +0200, Johan Hovold wrote: > On Thu, Apr 25, 2019 at 06:05:36PM +0200, Johan Hovold wrote: > > Fix two long-standing bugs which could potentially lead to memory > > corruption or leave the port throttled until it is reopened (on weakly > > ordered systems), respectively, when read-URB completion races with > > unthrottle(). > > > > First, the URB must not be marked as free before processing is complete > > to prevent it from being submitted by unthrottle() on another CPU. > > > > CPU 1 CPU 2 > > ================ ================ > > complete() unthrottle() > > process_urb(); > > smp_mb__before_atomic(); > > set_bit(i, free); if (test_and_clear_bit(i, free)) > > submit_urb(); > > > > Second, the URB must be marked as free before checking the throttled > > flag to prevent unthrottle() on another CPU from failing to observe that > > the URB needs to be submitted if complete() sees that the throttled flag > > is set. > > > > CPU 1 CPU 2 > > ================ ================ > > complete() unthrottle() > > set_bit(i, free); throttled = 0; > > smp_mb__after_atomic(); smp_mb(); > > if (throttled) if (test_and_clear_bit(i, free)) > > return; submit_urb(); > > > > Note that test_and_clear_bit() only implies barriers when the test is > > successful. To handle the case where the URB is still in use an explicit > > barrier needs to be added to unthrottle() for the second race condition. > > > > Fixes: d83b405383c9 ("USB: serial: add support for multiple read urbs") > > Signed-off-by: Johan Hovold <johan@kernel.org> > > Greg, I noticed you added a stable tag to the corresponding cdc-acm fix > and think I should have added on one from the start to this one as well. > > Would you mind queuing this one up for stable? > > Upstream commit 3f5edd58d040bfa4b74fb89bc02f0bc6b9cd06ab. Sure, now queued up for 4.9+ thanks, greg k-h
On Mon, May 13, 2019 at 12:56:06PM +0200, Greg Kroah-Hartman wrote: > On Mon, May 13, 2019 at 12:43:39PM +0200, Johan Hovold wrote: > > On Thu, Apr 25, 2019 at 06:05:36PM +0200, Johan Hovold wrote: > > > Fix two long-standing bugs which could potentially lead to memory > > > corruption or leave the port throttled until it is reopened (on weakly > > > ordered systems), respectively, when read-URB completion races with > > > unthrottle(). > > > Fixes: d83b405383c9 ("USB: serial: add support for multiple read urbs") > > > Signed-off-by: Johan Hovold <johan@kernel.org> > > > > Greg, I noticed you added a stable tag to the corresponding cdc-acm fix > > and think I should have added on one from the start to this one as well. > > > > Would you mind queuing this one up for stable? > > > > Upstream commit 3f5edd58d040bfa4b74fb89bc02f0bc6b9cd06ab. > > Sure, now queued up for 4.9+ Thanks. The issue has been there since v3.3 so I guess you could queue it for all stable trees. Johan
On Mon, May 13, 2019 at 01:46:01PM +0200, Johan Hovold wrote: > On Mon, May 13, 2019 at 12:56:06PM +0200, Greg Kroah-Hartman wrote: > > On Mon, May 13, 2019 at 12:43:39PM +0200, Johan Hovold wrote: > > > On Thu, Apr 25, 2019 at 06:05:36PM +0200, Johan Hovold wrote: > > > > Fix two long-standing bugs which could potentially lead to memory > > > > corruption or leave the port throttled until it is reopened (on weakly > > > > ordered systems), respectively, when read-URB completion races with > > > > unthrottle(). > > > > > Fixes: d83b405383c9 ("USB: serial: add support for multiple read urbs") > > > > Signed-off-by: Johan Hovold <johan@kernel.org> > > > > > > Greg, I noticed you added a stable tag to the corresponding cdc-acm fix > > > and think I should have added on one from the start to this one as well. > > > > > > Would you mind queuing this one up for stable? > > > > > > Upstream commit 3f5edd58d040bfa4b74fb89bc02f0bc6b9cd06ab. > > > > Sure, now queued up for 4.9+ > > Thanks. The issue has been there since v3.3 so I guess you could queue > it for all stable trees. Doesn't apply cleanly for 4.4.y or 3.18.y, so if it's really worth adding there (and I kind of doubt it), I would need a backport :) thanks, greg k-h
On Mon, May 13, 2019 at 02:51:31PM +0200, Greg Kroah-Hartman wrote: > On Mon, May 13, 2019 at 01:46:01PM +0200, Johan Hovold wrote: > > Thanks. The issue has been there since v3.3 so I guess you could queue > > it for all stable trees. > > Doesn't apply cleanly for 4.4.y or 3.18.y, so if it's really worth > adding there (and I kind of doubt it), I would need a backport :) Ah ok, just wasn't sure why you chose 4.9+. Johan
On Mon, May 13, 2019 at 02:59:09PM +0200, Johan Hovold wrote: >On Mon, May 13, 2019 at 02:51:31PM +0200, Greg Kroah-Hartman wrote: >> On Mon, May 13, 2019 at 01:46:01PM +0200, Johan Hovold wrote: > >> > Thanks. The issue has been there since v3.3 so I guess you could queue >> > it for all stable trees. >> >> Doesn't apply cleanly for 4.4.y or 3.18.y, so if it's really worth >> adding there (and I kind of doubt it), I would need a backport :) > >Ah ok, just wasn't sure why you chose 4.9+. On 4.4 and 3.18 it just had a contextual conflict because of 3161da970d38c ("USB: serial: use variable for status"), I can queue both 3161da970d38c and 3f5edd58d040b for 4.4 and 3.18. -- Thanks, Sasha
On Tue, May 14, 2019 at 08:53:53AM -0400, Sasha Levin wrote: > On Mon, May 13, 2019 at 02:59:09PM +0200, Johan Hovold wrote: > >On Mon, May 13, 2019 at 02:51:31PM +0200, Greg Kroah-Hartman wrote: > >> On Mon, May 13, 2019 at 01:46:01PM +0200, Johan Hovold wrote: > > > >> > Thanks. The issue has been there since v3.3 so I guess you could queue > >> > it for all stable trees. > >> > >> Doesn't apply cleanly for 4.4.y or 3.18.y, so if it's really worth > >> adding there (and I kind of doubt it), I would need a backport :) > > > >Ah ok, just wasn't sure why you chose 4.9+. > > On 4.4 and 3.18 it just had a contextual conflict because of > 3161da970d38c ("USB: serial: use variable for status"), I can queue both > 3161da970d38c and 3f5edd58d040b for 4.4 and 3.18. Sounds good, thanks! Johan
diff --git a/drivers/usb/serial/generic.c b/drivers/usb/serial/generic.c index 2274d9625f63..0fff4968ea1b 100644 --- a/drivers/usb/serial/generic.c +++ b/drivers/usb/serial/generic.c @@ -376,6 +376,7 @@ void usb_serial_generic_read_bulk_callback(struct urb *urb) struct usb_serial_port *port = urb->context; unsigned char *data = urb->transfer_buffer; unsigned long flags; + bool stopped = false; int status = urb->status; int i; @@ -383,33 +384,51 @@ void usb_serial_generic_read_bulk_callback(struct urb *urb) if (urb == port->read_urbs[i]) break; } - set_bit(i, &port->read_urbs_free); dev_dbg(&port->dev, "%s - urb %d, len %d\n", __func__, i, urb->actual_length); switch (status) { case 0: + usb_serial_debug_data(&port->dev, __func__, urb->actual_length, + data); + port->serial->type->process_read_urb(urb); break; case -ENOENT: case -ECONNRESET: case -ESHUTDOWN: dev_dbg(&port->dev, "%s - urb stopped: %d\n", __func__, status); - return; + stopped = true; + break; case -EPIPE: dev_err(&port->dev, "%s - urb stopped: %d\n", __func__, status); - return; + stopped = true; + break; default: dev_dbg(&port->dev, "%s - nonzero urb status: %d\n", __func__, status); - goto resubmit; + break; } - usb_serial_debug_data(&port->dev, __func__, urb->actual_length, data); - port->serial->type->process_read_urb(urb); + /* + * Make sure URB processing is done before marking as free to avoid + * racing with unthrottle() on another CPU. Matches the barriers + * implied by the test_and_clear_bit() in + * usb_serial_generic_submit_read_urb(). + */ + smp_mb__before_atomic(); + set_bit(i, &port->read_urbs_free); + /* + * Make sure URB is marked as free before checking the throttled flag + * to avoid racing with unthrottle() on another CPU. Matches the + * smp_mb() in unthrottle(). + */ + smp_mb__after_atomic(); + + if (stopped) + return; -resubmit: /* Throttle the device if requested by tty */ spin_lock_irqsave(&port->lock, flags); port->throttled = port->throttle_req; @@ -484,6 +503,12 @@ void usb_serial_generic_unthrottle(struct tty_struct *tty) port->throttled = port->throttle_req = 0; spin_unlock_irq(&port->lock); + /* + * Matches the smp_mb__after_atomic() in + * usb_serial_generic_read_bulk_callback(). + */ + smp_mb(); + if (was_throttled) usb_serial_generic_submit_read_urbs(port, GFP_KERNEL); }
Fix two long-standing bugs which could potentially lead to memory corruption or leave the port throttled until it is reopened (on weakly ordered systems), respectively, when read-URB completion races with unthrottle(). First, the URB must not be marked as free before processing is complete to prevent it from being submitted by unthrottle() on another CPU. CPU 1 CPU 2 ================ ================ complete() unthrottle() process_urb(); smp_mb__before_atomic(); set_bit(i, free); if (test_and_clear_bit(i, free)) submit_urb(); Second, the URB must be marked as free before checking the throttled flag to prevent unthrottle() on another CPU from failing to observe that the URB needs to be submitted if complete() sees that the throttled flag is set. CPU 1 CPU 2 ================ ================ complete() unthrottle() set_bit(i, free); throttled = 0; smp_mb__after_atomic(); smp_mb(); if (throttled) if (test_and_clear_bit(i, free)) return; submit_urb(); Note that test_and_clear_bit() only implies barriers when the test is successful. To handle the case where the URB is still in use an explicit barrier needs to be added to unthrottle() for the second race condition. Fixes: d83b405383c9 ("USB: serial: add support for multiple read urbs") Signed-off-by: Johan Hovold <johan@kernel.org> --- drivers/usb/serial/generic.c | 39 +++++++++++++++++++++++++++++------- 1 file changed, 32 insertions(+), 7 deletions(-)