| Message ID | 20230904121706.7132-420-nic_swsd@realtek.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [net] r8152: avoid the driver drops a lot of packets | expand |
Hello, On Mon, 2023-09-04 at 20:17 +0800, Hayes Wang wrote: > Stop submitting rx, if the driver queue more than 256 packets. > > If the hardware is more fast than the software, the driver would start > queuing the packets. And, the driver starts dropping the packets, if it > queues more than 1000 packets. > > Increase the weight of NAPI could improve the situation. However, the > weight has been changed to 64, so we have to stop submitting rx when the > driver queues too many packets. Then,the device may send the pause frame > to slow down the receiving, when the FIFO of the device is full. > > Fixes: cf74eb5a5bc8 ("eth: r8152: try to use a normal budget") > Signed-off-by: Hayes Wang <hayeswang@realtek.com> > --- > drivers/net/usb/r8152.c | 6 +----- > 1 file changed, 1 insertion(+), 5 deletions(-) > > diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c > index 332c853ca99b..b5ed55938b1c 100644 > --- a/drivers/net/usb/r8152.c > +++ b/drivers/net/usb/r8152.c > @@ -2484,10 +2484,6 @@ static int rx_bottom(struct r8152 *tp, int budget) > unsigned int pkt_len, rx_frag_head_sz; > struct sk_buff *skb; > > - /* limit the skb numbers for rx_queue */ > - if (unlikely(skb_queue_len(&tp->rx_queue) >= 1000)) > - break; > - Dropping this check looks dangerous to me. What if pause frames are disabled on the other end or dropped? It looks like this would cause unlimited memory consumption?!? If this limit is not supposed to be reached under normal conditions, perhaps is worthy changing it into a WARN_ON_ONCE()? Thanks! Paolo
Paolo Abeni <pabeni@redhat.com> > Sent: Tuesday, September 5, 2023 6:11 PM [...] > > - /* limit the skb numbers for rx_queue */ > > - if (unlikely(skb_queue_len(&tp->rx_queue) >= > 1000)) > > - break; > > - > > Dropping this check looks dangerous to me. What if pause frames are > disabled on the other end or dropped? It looks like this would cause > unlimited memory consumption?!? When the driver stops submitting rx, the driver wouldn't get any packet from the device after the previous urbs which have been submitted return. That is, skb_queue_len(&tp->rx_queue) wouldn't increase any more until the driver starts submitting rx again. Now, the driver stops submitting rx when the skb_queue_len more than 256, so the check becomes redundant. The skb_queue_len has been limited less than 1000. Besides, if the flow control is disabled, the packets may be dropped by the hardware when the FIFO of the device is full, after the driver stops submitting rx. Best Regards, Hayes
On Tue, 2023-09-05 at 10:37 +0000, Hayes Wang wrote: > Paolo Abeni <pabeni@redhat.com> > > Sent: Tuesday, September 5, 2023 6:11 PM > [...] > > > - /* limit the skb numbers for rx_queue */ > > > - if (unlikely(skb_queue_len(&tp->rx_queue) >= > > 1000)) > > > - break; > > > - > > > > Dropping this check looks dangerous to me. What if pause frames are > > disabled on the other end or dropped? It looks like this would cause > > unlimited memory consumption?!? > > When the driver stops submitting rx, the driver wouldn't get any packet > from the device after the previous urbs which have been submitted return. > That is, skb_queue_len(&tp->rx_queue) wouldn't increase any more until > the driver starts submitting rx again. > > Now, the driver stops submitting rx when the skb_queue_len more than 256, > so the check becomes redundant. The skb_queue_len has been limited less > than 1000. I'm sorry, I have a very superficial knowledge of the USB layer, but it looks like that when such condition is reached, in the worst condition there could be up to urbs in flight. AFAICS each of them carries a 16K buffer, can be up to 10 standard-mtu packets - or much more small ones. Setting an upper limits to the rx_queue still looks like a reasonable safeguard. > Besides, if the flow control is disabled, the packets may be dropped by > the hardware when the FIFO of the device is full, after the driver stops > submitting rx. If the incoming rate exceeds the H/W processing capacity, packets are dropped: that is expected and unavoidable. Possibly exposing the root cause for such drops to user space via appropriate stats would be useful. Cheers, Paolo
Paolo Abeni <pabeni@redhat.com> > Sent: Tuesday, September 5, 2023 7:21 PM [...] > I'm sorry, I have a very superficial knowledge of the USB layer, but it > looks like that when such condition is reached, in the worst condition > there could be up to urbs in flight. AFAICS each of them carries a 16K > buffer, can be up to 10 standard-mtu packets - or much more small ones. > > Setting an upper limits to the rx_queue still looks like a reasonable > safeguard. I think it is very hard to queue more than 1000 packets. The NAPI continues consuming the queued packets. And, the hardware wouldn't complete all urbs at one time. However, I would add WARN_ON_ONCE() to observe if any exception would occur. > > Besides, if the flow control is disabled, the packets may be dropped by > > the hardware when the FIFO of the device is full, after the driver stops > > submitting rx. > > If the incoming rate exceeds the H/W processing capacity, packets are > dropped: that is expected and unavoidable. > > Possibly exposing the root cause for such drops to user space via > appropriate stats would be useful. The number of packet which the device drops could be got through ethtool. Best Regards, Hayes
diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c index 332c853ca99b..b5ed55938b1c 100644 --- a/drivers/net/usb/r8152.c +++ b/drivers/net/usb/r8152.c @@ -2484,10 +2484,6 @@ static int rx_bottom(struct r8152 *tp, int budget) unsigned int pkt_len, rx_frag_head_sz; struct sk_buff *skb; - /* limit the skb numbers for rx_queue */ - if (unlikely(skb_queue_len(&tp->rx_queue) >= 1000)) - break; - pkt_len = le32_to_cpu(rx_desc->opts1) & RX_LEN_MASK; if (pkt_len < ETH_ZLEN) break; @@ -2556,7 +2552,7 @@ static int rx_bottom(struct r8152 *tp, int budget) } submit: - if (!ret) { + if (!ret && likely(skb_queue_len(&tp->rx_queue) < 256)) { ret = r8152_submit_rx(tp, agg, GFP_ATOMIC); } else { urb->actual_length = 0;
Stop submitting rx, if the driver queue more than 256 packets. If the hardware is more fast than the software, the driver would start queuing the packets. And, the driver starts dropping the packets, if it queues more than 1000 packets. Increase the weight of NAPI could improve the situation. However, the weight has been changed to 64, so we have to stop submitting rx when the driver queues too many packets. Then,the device may send the pause frame to slow down the receiving, when the FIFO of the device is full. Fixes: cf74eb5a5bc8 ("eth: r8152: try to use a normal budget") Signed-off-by: Hayes Wang <hayeswang@realtek.com> --- drivers/net/usb/r8152.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)