From patchwork Wed May 26 08:24:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 12280875 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C0EFC2B9F7 for ; Wed, 26 May 2021 08:24:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 471C16143E for ; Wed, 26 May 2021 08:24:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232526AbhEZI0N (ORCPT ); Wed, 26 May 2021 04:26:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22032 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232342AbhEZI0M (ORCPT ); Wed, 26 May 2021 04:26:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622017481; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=7hXo1aGEjC6ReUwtGmib1qGT2W6hQwxMtry8MQ3+Q5s=; b=U47H+rRtJmY8DaAWHr8udjajhdh1IzezqaeIFgZt8SOxSuPmxbgKF5QahW2EgYocOyLCXn jo0QQTm69X2Nf8T3PAat0VzW6shreM2w2rjwo5hVGACBCXZXcZXzalnkBWTMLL9oKo29G6 f972H5GzY5iG1Kz7Pwn95hHOUNiAllg= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-320-XDFqhoWXNrebVYPJbEnodg-1; Wed, 26 May 2021 04:24:39 -0400 X-MC-Unique: XDFqhoWXNrebVYPJbEnodg-1 Received: by mail-wr1-f72.google.com with SMTP id f16-20020a5d50d00000b0290114238aa727so47023wrt.5 for ; Wed, 26 May 2021 01:24:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=7hXo1aGEjC6ReUwtGmib1qGT2W6hQwxMtry8MQ3+Q5s=; b=hNGKjVmBiYtlyTh0HQ/S8tadzgGWFXc2LGbj7yo7SRCkJjbNOEnaan7rdJJzolre4E Dk816jTn9Z3oH+YeabJ39UFJ/5Tc6cNPxa7sWOcihu9nWI6P45eNtVhupyvnAZemZt7E 5iyjpV/K6S2EiVF9p8eFVVvCQ3s4mbjzOprzyMyDGh5+qhtlR8Cn1nF+tjEI3Cgv1zYe rIFi0/b6AjL60npRgDbhG20rtwnVBmQxW+f/JrJvGlrOyoTxTEMnsYsk8vKCj7Z3KFRY YkZPeOSdqx2lB+ZzYBytBeOiVJAWt+bk10+nk0gPo1O8NE9VcyUCK5WLktgxKl/ip3aV rqhA== X-Gm-Message-State: AOAM532Q0uW1N+9fpK5/V26nzNOuipAzamCOtqLVO2v+LqraRblj9wCU rIi8UhZS5GB3DHin5vjI5lt8Lzmym6IzrhTwRI5WJ8uqPF8k1loUrYfG0bqoaKXHhW73240KuK/ R2KpnZxIGjr02aNaA X-Received: by 2002:a1c:3d05:: with SMTP id k5mr1578835wma.9.1622017477883; Wed, 26 May 2021 01:24:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxU85p8gyLrf0vuWCAB2u049QkVe1v5Zh6lkfS/y/mIz9iaRTlYd6RVbAfTvvyvkld+1u+lAw== X-Received: by 2002:a1c:3d05:: with SMTP id k5mr1578819wma.9.1622017477718; Wed, 26 May 2021 01:24:37 -0700 (PDT) Received: from redhat.com ([2a10:8006:fcda:0:90d:c7e7:9e26:b297]) by smtp.gmail.com with ESMTPSA id g6sm5469484wmg.10.2021.05.26.01.24.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 May 2021 01:24:37 -0700 (PDT) Date: Wed, 26 May 2021 04:24:35 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Jakub Kicinski , Wei Wang , David Miller , netdev@vger.kernel.org, Willem de Bruijn , virtualization@lists.linux-foundation.org, Jason Wang Subject: [PATCH v3 1/4] virtio_net: move tx vq operation under tx queue lock Message-ID: <20210526082423.47837-2-mst@redhat.com> References: <20210526082423.47837-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210526082423.47837-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org It's unsafe to operate a vq from multiple threads. Unfortunately this is exactly what we do when invoking clean tx poll from rx napi. Same happens with napi-tx even without the opportunistic cleaning from the receive interrupt: that races with processing the vq in start_xmit. As a fix move everything that deals with the vq to under tx lock. Fixes: b92f1e6751a6 ("virtio-net: transmit napi") Signed-off-by: Michael S. Tsirkin --- drivers/net/virtio_net.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index ac0c143f97b4..12512d1002ec 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -1508,6 +1508,8 @@ static int virtnet_poll_tx(struct napi_struct *napi, int budget) struct virtnet_info *vi = sq->vq->vdev->priv; unsigned int index = vq2txq(sq->vq); struct netdev_queue *txq; + int opaque; + bool done; if (unlikely(is_xdp_raw_buffer_queue(vi, index))) { /* We don't need to enable cb for XDP */ @@ -1517,10 +1519,28 @@ static int virtnet_poll_tx(struct napi_struct *napi, int budget) txq = netdev_get_tx_queue(vi->dev, index); __netif_tx_lock(txq, raw_smp_processor_id()); + virtqueue_disable_cb(sq->vq); free_old_xmit_skbs(sq, true); + + opaque = virtqueue_enable_cb_prepare(sq->vq); + + done = napi_complete_done(napi, 0); + + if (!done) + virtqueue_disable_cb(sq->vq); + __netif_tx_unlock(txq); - virtqueue_napi_complete(napi, sq->vq, 0); + if (done) { + if (unlikely(virtqueue_poll(sq->vq, opaque))) { + if (napi_schedule_prep(napi)) { + __netif_tx_lock(txq, raw_smp_processor_id()); + virtqueue_disable_cb(sq->vq); + __netif_tx_unlock(txq); + __napi_schedule(napi); + } + } + } if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) netif_tx_wake_queue(txq); From patchwork Wed May 26 08:24:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 12280877 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3719C47089 for ; Wed, 26 May 2021 08:24:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DC6606143E for ; Wed, 26 May 2021 08:24:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232412AbhEZI0R (ORCPT ); Wed, 26 May 2021 04:26:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:48042 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232349AbhEZI0O (ORCPT ); Wed, 26 May 2021 04:26:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622017483; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZS7ZD35dU/eELTvMx06tUPCxlf74xPaW7VsnOKz2lgA=; b=YoXA9nNFldikGUcYUNr161UObTN2s2JqW8GjgSmskkTkjKjZhbAKJ3vyF6LAtXIsSBM8mN IJ690VeOpN0Ve4gmmbjerq5ZpH+k+sz2PQh2yxQylUh2ik/2NUZhrXc7u3DzTNk1veHr1L QjPZdyXfe3kW3DMU2ZdovObzLygKjZE= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-189-XLSv131bN4yBcGfts4Ji1Q-1; Wed, 26 May 2021 04:24:41 -0400 X-MC-Unique: XLSv131bN4yBcGfts4Ji1Q-1 Received: by mail-wm1-f70.google.com with SMTP id h9-20020a1cb7090000b029016d3f0b6ce4so115229wmf.9 for ; Wed, 26 May 2021 01:24:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=ZS7ZD35dU/eELTvMx06tUPCxlf74xPaW7VsnOKz2lgA=; b=QePkZ6ccAkH/M0l1S7SVlNjfykUbxE+2sI27+/OR8GUORVTVuRRtzSeTcnBbh69iWy 2Nk8yUPWfDH68fSyPFIdJqyoNJnHC2ybAi03f8BOehRb86U7dzQUE+Z/hQaSqZ5ibvXE O7juIeRKCbzv6dRgUJ4GkBZF/Q6FF5cV+g02R89XkkxrLwyIVj89T1pIDrHeYjAFr2tq WpczYfCvwAOrPzw8OMEbKRVaPyFmgdeO4aYHCgJZxyPhmbNzKKAYhRNqkhoiHyU1OfjY p3EBznibrn2ZAuC1g6/KnsBaOj7dbBLJA3MVLYtfVCSWKuXG4gVaHcgBrMVOWLrl/jJG F9lw== X-Gm-Message-State: AOAM531Kk9ZToguGzsuJy0LqUGw6+uM0Yd05DdzHBlCRzy0GAg2V94C1 crjAYCZLIYvPYTmXSbP3GEnWtPgOEuXL5xu0n/DdHcOU3MkTy8z68i01gZIRqduT+3/I1g616Ag tzhcaJGzhenIKbz00 X-Received: by 2002:a1c:4444:: with SMTP id r65mr2151369wma.127.1622017480213; Wed, 26 May 2021 01:24:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxXdpzBs/WOwiOoCYBUHQSDzRAcpfSmL78loNWZAQmEL/+Swns93afcI95zX4GVL88WJjsVEw== X-Received: by 2002:a1c:4444:: with SMTP id r65mr2151356wma.127.1622017480077; Wed, 26 May 2021 01:24:40 -0700 (PDT) Received: from redhat.com ([2a10:8006:fcda:0:90d:c7e7:9e26:b297]) by smtp.gmail.com with ESMTPSA id n20sm13641956wmk.12.2021.05.26.01.24.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 May 2021 01:24:39 -0700 (PDT) Date: Wed, 26 May 2021 04:24:37 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Jakub Kicinski , Wei Wang , David Miller , netdev@vger.kernel.org, Willem de Bruijn , virtualization@lists.linux-foundation.org, Jason Wang Subject: [PATCH v3 2/4] virtio_net: move txq wakeups under tx q lock Message-ID: <20210526082423.47837-3-mst@redhat.com> References: <20210526082423.47837-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210526082423.47837-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org We currently check num_free outside tx q lock which is unsafe: new packets can arrive meanwhile and there won't be space in the queue. Thus a spurious queue wakeup causing overhead and even packet drops. Move the check under the lock to fix that. Signed-off-by: Michael S. Tsirkin Acked-by: Jason Wang --- drivers/net/virtio_net.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 12512d1002ec..c29f42d1e04f 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -1434,11 +1434,12 @@ static void virtnet_poll_cleantx(struct receive_queue *rq) if (__netif_tx_trylock(txq)) { free_old_xmit_skbs(sq, true); + + if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) + netif_tx_wake_queue(txq); + __netif_tx_unlock(txq); } - - if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) - netif_tx_wake_queue(txq); } static int virtnet_poll(struct napi_struct *napi, int budget) @@ -1522,6 +1523,9 @@ static int virtnet_poll_tx(struct napi_struct *napi, int budget) virtqueue_disable_cb(sq->vq); free_old_xmit_skbs(sq, true); + if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) + netif_tx_wake_queue(txq); + opaque = virtqueue_enable_cb_prepare(sq->vq); done = napi_complete_done(napi, 0); @@ -1542,9 +1546,6 @@ static int virtnet_poll_tx(struct napi_struct *napi, int budget) } } - if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) - netif_tx_wake_queue(txq); - return 0; } From patchwork Wed May 26 08:24:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 12280879 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6CFAC2B9F7 for ; Wed, 26 May 2021 08:24:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 94BBA61441 for ; Wed, 26 May 2021 08:24:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232913AbhEZI0T (ORCPT ); Wed, 26 May 2021 04:26:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:35055 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232541AbhEZI0R (ORCPT ); Wed, 26 May 2021 04:26:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622017486; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=xQ4VlAVqH25DzMLpMhdVuF4QUuvJ8Qp+k4bIKP2VcN4=; b=VbCJBpN5sfVfjhpT6yHTPPLNGbOld9cjVPoWj8oDGuQH01AKHcHD6EVRu2ToO13KBmOGTJ MoKHfJhGv2ohFRIVdtaGbMYtEICvHYV6h8/N7d5LT1MBYv0NbAIW9sPg9g2jAREEzDC3ef tvF9ldPM9xspWWn1rCNVKMb4JdtDoF0= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-588-hxBnVuniOSmHR9GV2Gtjaw-1; Wed, 26 May 2021 04:24:44 -0400 X-MC-Unique: hxBnVuniOSmHR9GV2Gtjaw-1 Received: by mail-wr1-f71.google.com with SMTP id g3-20020adfd1e30000b02901122a4b850aso22867wrd.20 for ; Wed, 26 May 2021 01:24:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=xQ4VlAVqH25DzMLpMhdVuF4QUuvJ8Qp+k4bIKP2VcN4=; b=BP5v7f9d+NMdvFNTFe+B+y7D2yEYtYhL2oJDZBSXP3F9FEDdpD/IimH22yC42Qn3cD kVWONm+inzVeYYVPQm0L4gxAsdElbpUntDQrQHcbyUl1WY47yUWNBY6pmOZst801zGqK qzZY0C6xN49C20toCTXRrAEkC30Re+fiJ3oq/Y90iNp4nVZ02fuI2m/DS9sboM0mggFU iRndS+CYag34deHF3fgtxpUaFf4HqQ8+dqy9zpAa0X6JSoEdGOe/B1sdLwIvDc4E+I27 zH7u9XpoebSB9C6uvp+OFTMyyzZpKfAHUfYa3TqXrho/8WgwmygMgwLmnrp/hXRM/Wst dAYQ== X-Gm-Message-State: AOAM5309P8tzWPL3hhYKOiT33JZnEgowxHLPuPlQRz5YovKngocJFdi6 xn7Jh0vCBXobVOuraaY3dGiUveX6LTLIJTerfqY3XGVJDJMI1d328/0LBBzoFrn9HGETREhrmUi 4mylPusEug8cFsK4D X-Received: by 2002:a05:6000:192:: with SMTP id p18mr31087409wrx.252.1622017483279; Wed, 26 May 2021 01:24:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwBruCkcm4GOWWMpK9+L6RBQgOo0DFFffIWCQmTsoaoIsXJIdCDaUTvqMzWEVx0KNyBacvMzw== X-Received: by 2002:a05:6000:192:: with SMTP id p18mr31087397wrx.252.1622017483114; Wed, 26 May 2021 01:24:43 -0700 (PDT) Received: from redhat.com ([2a10:8006:fcda:0:90d:c7e7:9e26:b297]) by smtp.gmail.com with ESMTPSA id r7sm5281221wmq.3.2021.05.26.01.24.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 May 2021 01:24:42 -0700 (PDT) Date: Wed, 26 May 2021 04:24:40 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Jakub Kicinski , Wei Wang , David Miller , netdev@vger.kernel.org, Willem de Bruijn , virtualization@lists.linux-foundation.org, Jason Wang Subject: [PATCH v3 3/4] virtio: fix up virtio_disable_cb Message-ID: <20210526082423.47837-4-mst@redhat.com> References: <20210526082423.47837-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210526082423.47837-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org virtio_disable_cb is currently a nop for split ring with event index. This is because it used to be always called from a callback when we know device won't trigger more events until we update the index. However, now that we run with interrupts enabled a lot we also poll without a callback so that is different: disabling callbacks will help reduce the number of spurious interrupts. Further, if using event index with a packed ring, and if being called from a callback, we actually do disable interrupts which is unnecessary. Fix both issues by tracking whenever we get a callback. If that is the case disabling interrupts with event index can be a nop. If not the case disable interrupts. Note: with a split ring there's no explicit "no interrupts" value. For now we write a fixed value so our chance of triggering an interupt is 1/ring size. It's probably better to write something related to the last used index there to reduce the chance even further. For now I'm keeping it simple. Signed-off-by: Michael S. Tsirkin --- drivers/virtio/virtio_ring.c | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 71e16b53e9c1..88f0b16b11b8 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -113,6 +113,9 @@ struct vring_virtqueue { /* Last used index we've seen. */ u16 last_used_idx; + /* Hint for event idx: already triggered no need to disable. */ + bool event_triggered; + union { /* Available for split ring */ struct { @@ -739,7 +742,10 @@ static void virtqueue_disable_cb_split(struct virtqueue *_vq) if (!(vq->split.avail_flags_shadow & VRING_AVAIL_F_NO_INTERRUPT)) { vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT; - if (!vq->event) + if (vq->event) + /* TODO: this is a hack. Figure out a cleaner value to write. */ + vring_used_event(&vq->split.vring) = 0x0; + else vq->split.vring.avail->flags = cpu_to_virtio16(_vq->vdev, vq->split.avail_flags_shadow); @@ -1605,6 +1611,7 @@ static struct virtqueue *vring_create_virtqueue_packed( vq->weak_barriers = weak_barriers; vq->broken = false; vq->last_used_idx = 0; + vq->event_triggered = false; vq->num_added = 0; vq->packed_ring = true; vq->use_dma_api = vring_use_dma_api(vdev); @@ -1919,6 +1926,12 @@ void virtqueue_disable_cb(struct virtqueue *_vq) { struct vring_virtqueue *vq = to_vvq(_vq); + /* If device triggered an event already it won't trigger one again: + * no need to disable. + */ + if (vq->event_triggered) + return; + if (vq->packed_ring) virtqueue_disable_cb_packed(_vq); else @@ -1942,6 +1955,9 @@ unsigned virtqueue_enable_cb_prepare(struct virtqueue *_vq) { struct vring_virtqueue *vq = to_vvq(_vq); + if (vq->event_triggered) + vq->event_triggered = false; + return vq->packed_ring ? virtqueue_enable_cb_prepare_packed(_vq) : virtqueue_enable_cb_prepare_split(_vq); } @@ -2005,6 +2021,9 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *_vq) { struct vring_virtqueue *vq = to_vvq(_vq); + if (vq->event_triggered) + vq->event_triggered = false; + return vq->packed_ring ? virtqueue_enable_cb_delayed_packed(_vq) : virtqueue_enable_cb_delayed_split(_vq); } @@ -2044,6 +2063,10 @@ irqreturn_t vring_interrupt(int irq, void *_vq) if (unlikely(vq->broken)) return IRQ_HANDLED; + /* Just a hint for performance: so it's ok that this can be racy! */ + if (vq->event) + vq->event_triggered = true; + pr_debug("virtqueue callback for %p (%p)\n", vq, vq->vq.callback); if (vq->vq.callback) vq->vq.callback(&vq->vq); @@ -2083,6 +2106,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index, vq->weak_barriers = weak_barriers; vq->broken = false; vq->last_used_idx = 0; + vq->event_triggered = false; vq->num_added = 0; vq->use_dma_api = vring_use_dma_api(vdev); #ifdef DEBUG From patchwork Wed May 26 08:24:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 12280881 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 601B0C47082 for ; Wed, 26 May 2021 08:24:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 421DF613C7 for ; Wed, 26 May 2021 08:24:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233103AbhEZI0Y (ORCPT ); Wed, 26 May 2021 04:26:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:57494 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232958AbhEZI0V (ORCPT ); Wed, 26 May 2021 04:26:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622017490; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=o40Fmid6nNIkbFe8iYePDZhpBrn8pMAYTwjh787uFdE=; b=bMgWZzvxoCCUOHiqAt0w139I9D42aQf9cd8BYUDxKX/uDjIEkOzpeo62tkp5FvqyWDP7Aa rm3CPN2Qp+lzFBggvoHlokDF1QVUFH9JQyXMZbeqpgWtE4xc36908E1BjbXIIrrgdrujnt iKk9y9eolWhyG7h//Rh6E2PzNDkgCno= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-123-mifakwWHP-6pm-AzVN-kGA-1; Wed, 26 May 2021 04:24:46 -0400 X-MC-Unique: mifakwWHP-6pm-AzVN-kGA-1 Received: by mail-wm1-f72.google.com with SMTP id n20-20020a05600c4f94b029017f371265feso4634946wmq.5 for ; Wed, 26 May 2021 01:24:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=o40Fmid6nNIkbFe8iYePDZhpBrn8pMAYTwjh787uFdE=; b=LaZUDnplN9yqp7Gwstn1P4MYQXH5Lw6XhXCPMo9pGbc62FKFQ0AaEvZLzw+GvfTO6Q b8vwuN8FO2gVzkI7nfx/scwOQtwxgQAQNyuj+WJId66drZYiXFFGPDMLC1Z1sUX2n2QG 5AFi4ORaFdXAn3ObvNMABsJNSOFBkfU7DTF40BtsrX8pQAaUT53pCaO+piP016lMaVnD cah7UXXTdGXUAU229TdyyvHg/cg8W5qfVIoi4eoBsbMnFHrgUZihy+qO3ZjCGEgom8J/ 8FKJCDM+XXllxNRSq8vhBptL2pL6fa0wQgtVT+W2dNrazfoJu6eFlO6wHz35Tz4f1ASr NOzw== X-Gm-Message-State: AOAM533U4/qUj0rvmXGRMkdaZXENb5HtuKit/WOgmlKNRRfVKw9Th4Ef a0/ob55xAF6P4ToDbdAEYM66rgwkPbJE4XGHBi6hFjaEuQtp8n7fBfeEb/WQ7OD6CzehAUqrc0i 5vd+3B9sGL+FeM9t9 X-Received: by 2002:a1c:f303:: with SMTP id q3mr2246260wmq.9.1622017485572; Wed, 26 May 2021 01:24:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy3YaY8d7OxUasPPsyIXLBzSHdDVjW4txQgjFtjWKP91pjY4SLYxtZjAEHxMjzrOAh+TdWZzQ== X-Received: by 2002:a1c:f303:: with SMTP id q3mr2246253wmq.9.1622017485441; Wed, 26 May 2021 01:24:45 -0700 (PDT) Received: from redhat.com ([2a10:8006:fcda:0:90d:c7e7:9e26:b297]) by smtp.gmail.com with ESMTPSA id l188sm769420wmf.27.2021.05.26.01.24.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 May 2021 01:24:45 -0700 (PDT) Date: Wed, 26 May 2021 04:24:43 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Jakub Kicinski , Wei Wang , David Miller , netdev@vger.kernel.org, Willem de Bruijn , virtualization@lists.linux-foundation.org, Jason Wang Subject: [PATCH v3 4/4] virtio_net: disable cb aggressively Message-ID: <20210526082423.47837-5-mst@redhat.com> References: <20210526082423.47837-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210526082423.47837-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org There are currently two cases where we poll TX vq not in response to a callback: start xmit and rx napi. We currently do this with callbacks enabled which can cause extra interrupts from the card. Used not to be a big issue as we run with interrupts disabled but that is no longer the case, and in some cases the rate of spurious interrupts is so high linux detects this and actually kills the interrupt. Fix up by disabling the callbacks before polling the tx vq. Signed-off-by: Michael S. Tsirkin --- drivers/net/virtio_net.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index c29f42d1e04f..a83dc038d8af 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -1433,7 +1433,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq) return; if (__netif_tx_trylock(txq)) { - free_old_xmit_skbs(sq, true); + do { + virtqueue_disable_cb(sq->vq); + free_old_xmit_skbs(sq, true); + } while (unlikely(!virtqueue_enable_cb_delayed(sq->vq))); if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) netif_tx_wake_queue(txq); @@ -1605,12 +1608,17 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev) struct netdev_queue *txq = netdev_get_tx_queue(dev, qnum); bool kick = !netdev_xmit_more(); bool use_napi = sq->napi.weight; + unsigned int bytes = skb->len; /* Free up any pending old buffers before queueing new ones. */ - free_old_xmit_skbs(sq, false); + do { + if (use_napi) + virtqueue_disable_cb(sq->vq); - if (use_napi && kick) - virtqueue_enable_cb_delayed(sq->vq); + free_old_xmit_skbs(sq, false); + + } while (use_napi && kick && + unlikely(!virtqueue_enable_cb_delayed(sq->vq))); /* timestamp packet in software */ skb_tx_timestamp(skb);