| Message ID | 20221221044230.1012787-1-lsahn@ooseel.net (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [v2] usbnet: optimize usbnet_bh() to reduce CPU load | expand |
On Wed, Dec 21, 2022 at 01:42:30PM +0900, Leesoo Ahn wrote: > The current source pushes skb into dev->done queue by calling > skb_queue_tail() and then pop it by calling skb_dequeue() to branch to > rx_cleanup state for freeing urb/skb in usbnet_bh(). It takes extra CPU > load, 2.21% (skb_queue_tail) as follows. > > - 11.58% 0.26% swapper [k] usbnet_bh > - 11.32% usbnet_bh > - 6.43% skb_dequeue > 6.34% _raw_spin_unlock_irqrestore > - 2.21% skb_queue_tail > 2.19% _raw_spin_unlock_irqrestore > - 1.68% consume_skb > - 0.97% kfree_skbmem > 0.80% kmem_cache_free > 0.53% skb_release_data > > To reduce the extra CPU load use return values jumping to rx_cleanup > state directly to free them instead of calling skb_queue_tail() and > skb_dequeue() for push/pop respectively. > > - 7.87% 0.25% swapper [k] usbnet_bh > - 7.62% usbnet_bh > - 4.81% skb_dequeue > 4.74% _raw_spin_unlock_irqrestore > - 1.75% consume_skb > - 0.98% kfree_skbmem > 0.78% kmem_cache_free > 0.58% skb_release_data > 0.53% smsc95xx_rx_fixup > > Signed-off-by: Leesoo Ahn <lsahn@ooseel.net> > --- > v2: > - Replace goto label with return statement to reduce goto entropy > - Add CPU load information by perf in commit message > > v1 at: > https://patchwork.kernel.org/project/netdevbpf/patch/20221217161851.829497-1-lsahn@ooseel.net/ > --- > drivers/net/usb/usbnet.c | 19 +++++++++---------- > 1 file changed, 9 insertions(+), 10 deletions(-) > > diff --git a/drivers/net/usb/usbnet.c b/drivers/net/usb/usbnet.c > index 64a9a80b2309..6e82fef90dd9 100644 > --- a/drivers/net/usb/usbnet.c > +++ b/drivers/net/usb/usbnet.c > @@ -555,32 +555,30 @@ static int rx_submit (struct usbnet *dev, struct urb *urb, gfp_t flags) > > /*-------------------------------------------------------------------------*/ > > -static inline void rx_process (struct usbnet *dev, struct sk_buff *skb) > +static inline int rx_process(struct usbnet *dev, struct sk_buff *skb) > { > if (dev->driver_info->rx_fixup && > !dev->driver_info->rx_fixup (dev, skb)) { > /* With RX_ASSEMBLE, rx_fixup() must update counters */ > if (!(dev->driver_info->flags & FLAG_RX_ASSEMBLE)) > dev->net->stats.rx_errors++; > - goto done; > + return 1; "1" means that you processed 1 byte, not that this is an error, which is what you want to say here, right? Please return a negative error value like I asked this to be changed to last time :( thanks, greg k-h
On 22. 12. 21. 15:32, Greg KH wrote: > On Wed, Dec 21, 2022 at 01:42:30PM +0900, Leesoo Ahn wrote: >> The current source pushes skb into dev->done queue by calling >> skb_queue_tail() and then pop it by calling skb_dequeue() to branch to >> rx_cleanup state for freeing urb/skb in usbnet_bh(). It takes extra CPU >> load, 2.21% (skb_queue_tail) as follows. >> >> - 11.58% 0.26% swapper [k] usbnet_bh >> - 11.32% usbnet_bh >> - 6.43% skb_dequeue >> 6.34% _raw_spin_unlock_irqrestore >> - 2.21% skb_queue_tail >> 2.19% _raw_spin_unlock_irqrestore >> - 1.68% consume_skb >> - 0.97% kfree_skbmem >> 0.80% kmem_cache_free >> 0.53% skb_release_data >> >> To reduce the extra CPU load use return values jumping to rx_cleanup >> state directly to free them instead of calling skb_queue_tail() and >> skb_dequeue() for push/pop respectively. >> >> - 7.87% 0.25% swapper [k] usbnet_bh >> - 7.62% usbnet_bh >> - 4.81% skb_dequeue >> 4.74% _raw_spin_unlock_irqrestore >> - 1.75% consume_skb >> - 0.98% kfree_skbmem >> 0.78% kmem_cache_free >> 0.58% skb_release_data >> 0.53% smsc95xx_rx_fixup >> >> Signed-off-by: Leesoo Ahn <lsahn@ooseel.net> >> --- >> v2: >> - Replace goto label with return statement to reduce goto entropy >> - Add CPU load information by perf in commit message >> >> v1 at: >> https://patchwork.kernel.org/project/netdevbpf/patch/20221217161851.829497-1-lsahn@ooseel.net/ >> --- >> drivers/net/usb/usbnet.c | 19 +++++++++---------- >> 1 file changed, 9 insertions(+), 10 deletions(-) >> >> diff --git a/drivers/net/usb/usbnet.c b/drivers/net/usb/usbnet.c >> index 64a9a80b2309..6e82fef90dd9 100644 >> --- a/drivers/net/usb/usbnet.c >> +++ b/drivers/net/usb/usbnet.c >> @@ -555,32 +555,30 @@ static int rx_submit (struct usbnet *dev, struct urb *urb, gfp_t flags) >> >> /*-------------------------------------------------------------------------*/ >> >> -static inline void rx_process (struct usbnet *dev, struct sk_buff *skb) >> +static inline int rx_process(struct usbnet *dev, struct sk_buff *skb) >> { >> if (dev->driver_info->rx_fixup && >> !dev->driver_info->rx_fixup (dev, skb)) { >> /* With RX_ASSEMBLE, rx_fixup() must update counters */ >> if (!(dev->driver_info->flags & FLAG_RX_ASSEMBLE)) >> dev->net->stats.rx_errors++; >> - goto done; >> + return 1; > "1" means that you processed 1 byte, not that this is an error, which is > what you want to say here, right? No not at all.. > Please return a negative error value > like I asked this to be changed to last time :( Could you help me to decide the message type at this point please? I am confused. The return value totally depends on how rx_fixup() is. For instance, in smsc95xx.c, smsc95xx_rx_fixup() function returns 0 in two cases that 1) frame size is greater than ETH_FRAME_LEN(1526 bytes) as follows 1853 /* ETH_FRAME_LEN + 4(CRC) + 2(COE) + 4(Vlan) */ 1854 if (unlikely(size > (ETH_FRAME_LEN + 12))) { 1855 netif_dbg(dev, rx_err, dev->net, 1856 "size err header=0x%08x\n", header); 1857 return 0; 1858 } 2) it is failed for skb allocation, but memory? 1870 ax_skb = skb_clone(skb, GFP_ATOMIC); 1871 if (unlikely(!ax_skb)) { 1872 netdev_warn(dev->net, "Error allocating skb\n"); 1873 return 0; 1874 } I guess EPROTO or ENOMEM, one of them could be the value at the point but I have no ideas.. Best regards, Leesoo
On Wed, Dec 21, 2022 at 04:19:45PM +0900, Leesoo Ahn wrote: > > On 22. 12. 21. 15:32, Greg KH wrote: > > On Wed, Dec 21, 2022 at 01:42:30PM +0900, Leesoo Ahn wrote: > > > The current source pushes skb into dev->done queue by calling > > > skb_queue_tail() and then pop it by calling skb_dequeue() to branch to > > > rx_cleanup state for freeing urb/skb in usbnet_bh(). It takes extra CPU > > > load, 2.21% (skb_queue_tail) as follows. > > > > > > - 11.58% 0.26% swapper [k] usbnet_bh > > > - 11.32% usbnet_bh > > > - 6.43% skb_dequeue > > > 6.34% _raw_spin_unlock_irqrestore > > > - 2.21% skb_queue_tail > > > 2.19% _raw_spin_unlock_irqrestore > > > - 1.68% consume_skb > > > - 0.97% kfree_skbmem > > > 0.80% kmem_cache_free > > > 0.53% skb_release_data > > > > > > To reduce the extra CPU load use return values jumping to rx_cleanup > > > state directly to free them instead of calling skb_queue_tail() and > > > skb_dequeue() for push/pop respectively. > > > > > > - 7.87% 0.25% swapper [k] usbnet_bh > > > - 7.62% usbnet_bh > > > - 4.81% skb_dequeue > > > 4.74% _raw_spin_unlock_irqrestore > > > - 1.75% consume_skb > > > - 0.98% kfree_skbmem > > > 0.78% kmem_cache_free > > > 0.58% skb_release_data > > > 0.53% smsc95xx_rx_fixup > > > > > > Signed-off-by: Leesoo Ahn <lsahn@ooseel.net> > > > --- > > > v2: > > > - Replace goto label with return statement to reduce goto entropy > > > - Add CPU load information by perf in commit message > > > > > > v1 at: > > > https://patchwork.kernel.org/project/netdevbpf/patch/20221217161851.829497-1-lsahn@ooseel.net/ > > > --- > > > drivers/net/usb/usbnet.c | 19 +++++++++---------- > > > 1 file changed, 9 insertions(+), 10 deletions(-) > > > > > > diff --git a/drivers/net/usb/usbnet.c b/drivers/net/usb/usbnet.c > > > index 64a9a80b2309..6e82fef90dd9 100644 > > > --- a/drivers/net/usb/usbnet.c > > > +++ b/drivers/net/usb/usbnet.c > > > @@ -555,32 +555,30 @@ static int rx_submit (struct usbnet *dev, struct urb *urb, gfp_t flags) > > > /*-------------------------------------------------------------------------*/ > > > -static inline void rx_process (struct usbnet *dev, struct sk_buff *skb) > > > +static inline int rx_process(struct usbnet *dev, struct sk_buff *skb) > > > { > > > if (dev->driver_info->rx_fixup && > > > !dev->driver_info->rx_fixup (dev, skb)) { > > > /* With RX_ASSEMBLE, rx_fixup() must update counters */ > > > if (!(dev->driver_info->flags & FLAG_RX_ASSEMBLE)) > > > dev->net->stats.rx_errors++; > > > - goto done; > > > + return 1; > > "1" means that you processed 1 byte, not that this is an error, which is > > what you want to say here, right? > No not at all.. > > Please return a negative error value > > like I asked this to be changed to last time :( > Could you help me to decide the message type at this point please? I am > confused. I do not know, pick something that seems correct and we can go from there. The important thing is that it is a -ERR value, not a positive one as that makes no sense for kernel functions. thanks, greg k-h
On 22. 12. 21. 16:30, Greg KH wrote: > On Wed, Dec 21, 2022 at 04:19:45PM +0900, Leesoo Ahn wrote: >> On 22. 12. 21. 15:32, Greg KH wrote: >>> On Wed, Dec 21, 2022 at 01:42:30PM +0900, Leesoo Ahn wrote: >>>> The current source pushes skb into dev->done queue by calling >>>> skb_queue_tail() and then pop it by calling skb_dequeue() to branch to >>>> rx_cleanup state for freeing urb/skb in usbnet_bh(). It takes extra CPU >>>> load, 2.21% (skb_queue_tail) as follows. >>>> >>>> - 11.58% 0.26% swapper [k] usbnet_bh >>>> - 11.32% usbnet_bh >>>> - 6.43% skb_dequeue >>>> 6.34% _raw_spin_unlock_irqrestore >>>> - 2.21% skb_queue_tail >>>> 2.19% _raw_spin_unlock_irqrestore >>>> - 1.68% consume_skb >>>> - 0.97% kfree_skbmem >>>> 0.80% kmem_cache_free >>>> 0.53% skb_release_data >>>> >>>> To reduce the extra CPU load use return values jumping to rx_cleanup >>>> state directly to free them instead of calling skb_queue_tail() and >>>> skb_dequeue() for push/pop respectively. >>>> >>>> - 7.87% 0.25% swapper [k] usbnet_bh >>>> - 7.62% usbnet_bh >>>> - 4.81% skb_dequeue >>>> 4.74% _raw_spin_unlock_irqrestore >>>> - 1.75% consume_skb >>>> - 0.98% kfree_skbmem >>>> 0.78% kmem_cache_free >>>> 0.58% skb_release_data >>>> 0.53% smsc95xx_rx_fixup >>>> >>>> Signed-off-by: Leesoo Ahn <lsahn@ooseel.net> >>>> --- >>>> v2: >>>> - Replace goto label with return statement to reduce goto entropy >>>> - Add CPU load information by perf in commit message >>>> >>>> v1 at: >>>> https://patchwork.kernel.org/project/netdevbpf/patch/20221217161851.829497-1-lsahn@ooseel.net/ >>>> --- >>>> drivers/net/usb/usbnet.c | 19 +++++++++---------- >>>> 1 file changed, 9 insertions(+), 10 deletions(-) >>>> >>>> diff --git a/drivers/net/usb/usbnet.c b/drivers/net/usb/usbnet.c >>>> index 64a9a80b2309..6e82fef90dd9 100644 >>>> --- a/drivers/net/usb/usbnet.c >>>> +++ b/drivers/net/usb/usbnet.c >>>> @@ -555,32 +555,30 @@ static int rx_submit (struct usbnet *dev, struct urb *urb, gfp_t flags) >>>> /*-------------------------------------------------------------------------*/ >>>> -static inline void rx_process (struct usbnet *dev, struct sk_buff *skb) >>>> +static inline int rx_process(struct usbnet *dev, struct sk_buff *skb) >>>> { >>>> if (dev->driver_info->rx_fixup && >>>> !dev->driver_info->rx_fixup (dev, skb)) { >>>> /* With RX_ASSEMBLE, rx_fixup() must update counters */ >>>> if (!(dev->driver_info->flags & FLAG_RX_ASSEMBLE)) >>>> dev->net->stats.rx_errors++; >>>> - goto done; >>>> + return 1; >>> "1" means that you processed 1 byte, not that this is an error, which is >>> what you want to say here, right? >> No not at all.. >>> Please return a negative error value >>> like I asked this to be changed to last time :( >> Could you help me to decide the message type at this point please? I am >> confused. > I do not know, pick something that seems correct and we can go from > there. The important thing is that it is a -ERR value, not a positive > one as that makes no sense for kernel functions. Thank you for reviewing, v3 will be sent soon. Best regards, Leesoo
diff --git a/drivers/net/usb/usbnet.c b/drivers/net/usb/usbnet.c index 64a9a80b2309..6e82fef90dd9 100644 --- a/drivers/net/usb/usbnet.c +++ b/drivers/net/usb/usbnet.c @@ -555,32 +555,30 @@ static int rx_submit (struct usbnet *dev, struct urb *urb, gfp_t flags) /*-------------------------------------------------------------------------*/ -static inline void rx_process (struct usbnet *dev, struct sk_buff *skb) +static inline int rx_process(struct usbnet *dev, struct sk_buff *skb) { if (dev->driver_info->rx_fixup && !dev->driver_info->rx_fixup (dev, skb)) { /* With RX_ASSEMBLE, rx_fixup() must update counters */ if (!(dev->driver_info->flags & FLAG_RX_ASSEMBLE)) dev->net->stats.rx_errors++; - goto done; + return 1; } // else network stack removes extra byte if we forced a short packet /* all data was already cloned from skb inside the driver */ if (dev->driver_info->flags & FLAG_MULTI_PACKET) - goto done; + return 1; if (skb->len < ETH_HLEN) { dev->net->stats.rx_errors++; dev->net->stats.rx_length_errors++; netif_dbg(dev, rx_err, dev->net, "rx length %d\n", skb->len); - } else { - usbnet_skb_return(dev, skb); - return; + return 1; } -done: - skb_queue_tail(&dev->done, skb); + usbnet_skb_return(dev, skb); + return 0; } /*-------------------------------------------------------------------------*/ @@ -1528,13 +1526,14 @@ static void usbnet_bh (struct timer_list *t) entry = (struct skb_data *) skb->cb; switch (entry->state) { case rx_done: - entry->state = rx_cleanup; - rx_process (dev, skb); + if (rx_process(dev, skb)) + goto cleanup; continue; case tx_done: kfree(entry->urb->sg); fallthrough; case rx_cleanup: +cleanup: usb_free_urb (entry->urb); dev_kfree_skb (skb); continue;
The current source pushes skb into dev->done queue by calling skb_queue_tail() and then pop it by calling skb_dequeue() to branch to rx_cleanup state for freeing urb/skb in usbnet_bh(). It takes extra CPU load, 2.21% (skb_queue_tail) as follows. - 11.58% 0.26% swapper [k] usbnet_bh - 11.32% usbnet_bh - 6.43% skb_dequeue 6.34% _raw_spin_unlock_irqrestore - 2.21% skb_queue_tail 2.19% _raw_spin_unlock_irqrestore - 1.68% consume_skb - 0.97% kfree_skbmem 0.80% kmem_cache_free 0.53% skb_release_data To reduce the extra CPU load use return values jumping to rx_cleanup state directly to free them instead of calling skb_queue_tail() and skb_dequeue() for push/pop respectively. - 7.87% 0.25% swapper [k] usbnet_bh - 7.62% usbnet_bh - 4.81% skb_dequeue 4.74% _raw_spin_unlock_irqrestore - 1.75% consume_skb - 0.98% kfree_skbmem 0.78% kmem_cache_free 0.58% skb_release_data 0.53% smsc95xx_rx_fixup Signed-off-by: Leesoo Ahn <lsahn@ooseel.net> --- v2: - Replace goto label with return statement to reduce goto entropy - Add CPU load information by perf in commit message v1 at: https://patchwork.kernel.org/project/netdevbpf/patch/20221217161851.829497-1-lsahn@ooseel.net/ --- drivers/net/usb/usbnet.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-)