| Message ID | 20240415020247.2207781-1-lei.chen@smartx.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | f8bbc07ac535593139c875ffa19af924b1084540 |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [net-next,v5] tun: limit printing rate when illegal packet received by tun dev | expand |
On Mon, Apr 15, 2024 at 4:03 AM Lei Chen <lei.chen@smartx.com> wrote: > > vhost_worker will call tun call backs to receive packets. If too many > illegal packets arrives, tun_do_read will keep dumping packet contents. > When console is enabled, it will costs much more cpu time to dump > packet and soft lockup will be detected. > > net_ratelimit mechanism can be used to limit the dumping rate. > > Fixes: ef3db4a59542 ("tun: avoid BUG, dump packet on GSO errors") > Signed-off-by: Lei Chen <lei.chen@smartx.com> > Reviewed-by: Willem de Bruijn <willemb@google.com> > Acked-by: Jason Wang <jasowang@redhat.com> > --- Reviewed-by: Eric Dumazet <edumazet@google.com>
On Sun, Apr 14, 2024 at 10:02:46PM -0400, Lei Chen wrote: > vhost_worker will call tun call backs to receive packets. If too many > illegal packets arrives, tun_do_read will keep dumping packet contents. > When console is enabled, it will costs much more cpu time to dump > packet and soft lockup will be detected. > > net_ratelimit mechanism can be used to limit the dumping rate. > > PID: 33036 TASK: ffff949da6f20000 CPU: 23 COMMAND: "vhost-32980" > #0 [fffffe00003fce50] crash_nmi_callback at ffffffff89249253 > #1 [fffffe00003fce58] nmi_handle at ffffffff89225fa3 > #2 [fffffe00003fceb0] default_do_nmi at ffffffff8922642e > #3 [fffffe00003fced0] do_nmi at ffffffff8922660d > #4 [fffffe00003fcef0] end_repeat_nmi at ffffffff89c01663 > [exception RIP: io_serial_in+20] > RIP: ffffffff89792594 RSP: ffffa655314979e8 RFLAGS: 00000002 > RAX: ffffffff89792500 RBX: ffffffff8af428a0 RCX: 0000000000000000 > RDX: 00000000000003fd RSI: 0000000000000005 RDI: ffffffff8af428a0 > RBP: 0000000000002710 R8: 0000000000000004 R9: 000000000000000f > R10: 0000000000000000 R11: ffffffff8acbf64f R12: 0000000000000020 > R13: ffffffff8acbf698 R14: 0000000000000058 R15: 0000000000000000 > ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 > #5 [ffffa655314979e8] io_serial_in at ffffffff89792594 > #6 [ffffa655314979e8] wait_for_xmitr at ffffffff89793470 > #7 [ffffa65531497a08] serial8250_console_putchar at ffffffff897934f6 > #8 [ffffa65531497a20] uart_console_write at ffffffff8978b605 > #9 [ffffa65531497a48] serial8250_console_write at ffffffff89796558 > #10 [ffffa65531497ac8] console_unlock at ffffffff89316124 > #11 [ffffa65531497b10] vprintk_emit at ffffffff89317c07 > #12 [ffffa65531497b68] printk at ffffffff89318306 > #13 [ffffa65531497bc8] print_hex_dump at ffffffff89650765 > #14 [ffffa65531497ca8] tun_do_read at ffffffffc0b06c27 [tun] > #15 [ffffa65531497d38] tun_recvmsg at ffffffffc0b06e34 [tun] > #16 [ffffa65531497d68] handle_rx at ffffffffc0c5d682 [vhost_net] > #17 [ffffa65531497ed0] vhost_worker at ffffffffc0c644dc [vhost] > #18 [ffffa65531497f10] kthread at ffffffff892d2e72 > #19 [ffffa65531497f50] ret_from_fork at ffffffff89c0022f > > Fixes: ef3db4a59542 ("tun: avoid BUG, dump packet on GSO errors") > Signed-off-by: Lei Chen <lei.chen@smartx.com> > Reviewed-by: Willem de Bruijn <willemb@google.com> > Acked-by: Jason Wang <jasowang@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> > --- > Changes from v4: > https://lore.kernel.org/all/20240414081806.2173098-1-lei.chen@smartx.com/ > 1. Adjust code indentation > > Changes from v3: > https://lore.kernel.org/all/20240412065841.2148691-1-lei.chen@smartx.com/ > 1. Change patch target from net tun to tun. > 2. Move change log below the seperator "---". > 3. Remove escaped parentheses in the Fixes string. > > Changes from v2: > https://lore.kernel.org/netdev/20240410042245.2044516-1-lei.chen@smartx.com/ > 1. Add net-dev to patch subject-prefix. > 2. Add fix tag. > > Changes from v1: > https://lore.kernel.org/all/20240409062407.1952728-1-lei.chen@smartx.com/ > 1. Use net_ratelimit instead of raw __ratelimit. > 2. Use netdev_err instead of pr_err to print more info abort net dev. > 3. Adjust git commit message to make git am happy. > drivers/net/tun.c | 18 ++++++++++-------- > 1 file changed, 10 insertions(+), 8 deletions(-) > > drivers/net/tun.c | 18 ++++++++++-------- > 1 file changed, 10 insertions(+), 8 deletions(-) > > diff --git a/drivers/net/tun.c b/drivers/net/tun.c > index 0b3f21cba552..92da8c03d960 100644 > --- a/drivers/net/tun.c > +++ b/drivers/net/tun.c > @@ -2125,14 +2125,16 @@ static ssize_t tun_put_user(struct tun_struct *tun, > tun_is_little_endian(tun), true, > vlan_hlen)) { > struct skb_shared_info *sinfo = skb_shinfo(skb); > - pr_err("unexpected GSO type: " > - "0x%x, gso_size %d, hdr_len %d\n", > - sinfo->gso_type, tun16_to_cpu(tun, gso.gso_size), > - tun16_to_cpu(tun, gso.hdr_len)); > - print_hex_dump(KERN_ERR, "tun: ", > - DUMP_PREFIX_NONE, > - 16, 1, skb->head, > - min((int)tun16_to_cpu(tun, gso.hdr_len), 64), true); > + > + if (net_ratelimit()) { > + netdev_err(tun->dev, "unexpected GSO type: 0x%x, gso_size %d, hdr_len %d\n", > + sinfo->gso_type, tun16_to_cpu(tun, gso.gso_size), > + tun16_to_cpu(tun, gso.hdr_len)); > + print_hex_dump(KERN_ERR, "tun: ", > + DUMP_PREFIX_NONE, > + 16, 1, skb->head, > + min((int)tun16_to_cpu(tun, gso.hdr_len), 64), true); > + } > WARN_ON_ONCE(1); > return -EINVAL; > } > -- > 2.44.0 >
Hello: This patch was applied to netdev/net.git (main) by Jakub Kicinski <kuba@kernel.org>: On Sun, 14 Apr 2024 22:02:46 -0400 you wrote: > vhost_worker will call tun call backs to receive packets. If too many > illegal packets arrives, tun_do_read will keep dumping packet contents. > When console is enabled, it will costs much more cpu time to dump > packet and soft lockup will be detected. > > net_ratelimit mechanism can be used to limit the dumping rate. > > [...] Here is the summary with links: - [net-next,v5] tun: limit printing rate when illegal packet received by tun dev https://git.kernel.org/netdev/net/c/f8bbc07ac535 You are awesome, thank you!
diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 0b3f21cba552..92da8c03d960 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -2125,14 +2125,16 @@ static ssize_t tun_put_user(struct tun_struct *tun, tun_is_little_endian(tun), true, vlan_hlen)) { struct skb_shared_info *sinfo = skb_shinfo(skb); - pr_err("unexpected GSO type: " - "0x%x, gso_size %d, hdr_len %d\n", - sinfo->gso_type, tun16_to_cpu(tun, gso.gso_size), - tun16_to_cpu(tun, gso.hdr_len)); - print_hex_dump(KERN_ERR, "tun: ", - DUMP_PREFIX_NONE, - 16, 1, skb->head, - min((int)tun16_to_cpu(tun, gso.hdr_len), 64), true); + + if (net_ratelimit()) { + netdev_err(tun->dev, "unexpected GSO type: 0x%x, gso_size %d, hdr_len %d\n", + sinfo->gso_type, tun16_to_cpu(tun, gso.gso_size), + tun16_to_cpu(tun, gso.hdr_len)); + print_hex_dump(KERN_ERR, "tun: ", + DUMP_PREFIX_NONE, + 16, 1, skb->head, + min((int)tun16_to_cpu(tun, gso.hdr_len), 64), true); + } WARN_ON_ONCE(1); return -EINVAL; }
vhost_worker will call tun call backs to receive packets. If too many illegal packets arrives, tun_do_read will keep dumping packet contents. When console is enabled, it will costs much more cpu time to dump packet and soft lockup will be detected. net_ratelimit mechanism can be used to limit the dumping rate. PID: 33036 TASK: ffff949da6f20000 CPU: 23 COMMAND: "vhost-32980" #0 [fffffe00003fce50] crash_nmi_callback at ffffffff89249253 #1 [fffffe00003fce58] nmi_handle at ffffffff89225fa3 #2 [fffffe00003fceb0] default_do_nmi at ffffffff8922642e #3 [fffffe00003fced0] do_nmi at ffffffff8922660d #4 [fffffe00003fcef0] end_repeat_nmi at ffffffff89c01663 [exception RIP: io_serial_in+20] RIP: ffffffff89792594 RSP: ffffa655314979e8 RFLAGS: 00000002 RAX: ffffffff89792500 RBX: ffffffff8af428a0 RCX: 0000000000000000 RDX: 00000000000003fd RSI: 0000000000000005 RDI: ffffffff8af428a0 RBP: 0000000000002710 R8: 0000000000000004 R9: 000000000000000f R10: 0000000000000000 R11: ffffffff8acbf64f R12: 0000000000000020 R13: ffffffff8acbf698 R14: 0000000000000058 R15: 0000000000000000 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #5 [ffffa655314979e8] io_serial_in at ffffffff89792594 #6 [ffffa655314979e8] wait_for_xmitr at ffffffff89793470 #7 [ffffa65531497a08] serial8250_console_putchar at ffffffff897934f6 #8 [ffffa65531497a20] uart_console_write at ffffffff8978b605 #9 [ffffa65531497a48] serial8250_console_write at ffffffff89796558 #10 [ffffa65531497ac8] console_unlock at ffffffff89316124 #11 [ffffa65531497b10] vprintk_emit at ffffffff89317c07 #12 [ffffa65531497b68] printk at ffffffff89318306 #13 [ffffa65531497bc8] print_hex_dump at ffffffff89650765 #14 [ffffa65531497ca8] tun_do_read at ffffffffc0b06c27 [tun] #15 [ffffa65531497d38] tun_recvmsg at ffffffffc0b06e34 [tun] #16 [ffffa65531497d68] handle_rx at ffffffffc0c5d682 [vhost_net] #17 [ffffa65531497ed0] vhost_worker at ffffffffc0c644dc [vhost] #18 [ffffa65531497f10] kthread at ffffffff892d2e72 #19 [ffffa65531497f50] ret_from_fork at ffffffff89c0022f Fixes: ef3db4a59542 ("tun: avoid BUG, dump packet on GSO errors") Signed-off-by: Lei Chen <lei.chen@smartx.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Acked-by: Jason Wang <jasowang@redhat.com> --- Changes from v4: https://lore.kernel.org/all/20240414081806.2173098-1-lei.chen@smartx.com/ 1. Adjust code indentation Changes from v3: https://lore.kernel.org/all/20240412065841.2148691-1-lei.chen@smartx.com/ 1. Change patch target from net tun to tun. 2. Move change log below the seperator "---". 3. Remove escaped parentheses in the Fixes string. Changes from v2: https://lore.kernel.org/netdev/20240410042245.2044516-1-lei.chen@smartx.com/ 1. Add net-dev to patch subject-prefix. 2. Add fix tag. Changes from v1: https://lore.kernel.org/all/20240409062407.1952728-1-lei.chen@smartx.com/ 1. Use net_ratelimit instead of raw __ratelimit. 2. Use netdev_err instead of pr_err to print more info abort net dev. 3. Adjust git commit message to make git am happy. drivers/net/tun.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) drivers/net/tun.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)