From patchwork Tue Feb 4 23:00:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 13960144 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34208206F16 for ; Tue, 4 Feb 2025 23:01:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738710063; cv=none; b=aazxlqq9dLDGoQddGTxk9h117MIomVWONvobptQI79nqrn5f9b93xhOS1G6E3xorxHHhV422LPt9gg6tyrLi1NYZrxyTvqZFayG+hF5aPpGVm3x5hGWzewzHA2wLt58t7wdiPuvMHNeZh0EZFSw/vkxRBh/tM8bYFkf0fDEzeE0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738710063; c=relaxed/simple; bh=r4pLZP27SnGk+NUHU8GerlNYnqnEDtaakLxWmssmaXg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CeebNc3BBHHOsUqSCQ5oNHF2FmHrtTSG3Wga4LVqcwTvYB3JZTiD9AzP40uI9RMETcbxxsVKEMPLH0yTfjSVNmRWqHPEEdolzTB/+e/kHj2eLJ9bu0qJvCaYGW/OrBXbdk6A9JAYH+X4Wf0hOx8B+9E452tW/D4q0D7NdAVJUXg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=fomichev.me; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=fomichev.me Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2163dc5155fso108573235ad.0 for ; Tue, 04 Feb 2025 15:01:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738710060; x=1739314860; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ElQb9U4uQw6ven++d30UUeDXa84HifTkFMszzPFiP4M=; b=lSYC/NZpsAtpUExgiwQ5N1UV96tlZ1n8kA4VX7KHxmx/UeFVVRZilnSU+XEBkYKxIw uuKUZYhxxkgxXKdKHawAAIh9z5ofecMRQ/b15aTT7NFvx7buamue6OM2lWXy5rKWX7EK QGQlAXP04xsA8eZnvI+g6iQ8lVvGmZxjArEtd8fuIHbb3CgMunaRFZKJu9/TEfBMsTY6 OtRXBI/eki5SaCC/PUgM0aUUi1Cc3eQAV3S/xPN8JfriVcoM4IO59P0xVvKX7h2vQfnF ZRuqBgcLFb6lvr45MRMvriXvi0OnsOvdLxqabT762Hp/gPu5lb+XHWBGf7vMbSlQrdQg TFVg== X-Gm-Message-State: AOJu0Yxmpy+/Z9ISnnXESrACgcr61b1mZlpszUgV04HxuZCQqZr8jaUX h8TLLXJVMUF2wr3lBA8FEbwpS+ZsGurrfCrQDb9j4+JSGHgqz4u2n5wn X-Gm-Gg: ASbGncv1hHOoCqqPgSnMBJj20qrvOu0DIfBzLbLuNnwk7tXgQehTCJ8c1YhYr8+Khjl pEpa0/HlGdLZDiuD+wmqd+8zPjOh7NkCl2yYMU9XR413mLIuLXOgIBZ99arK1th6icQtPOpgq8W TWsq55SBhhUjYdpcpZQH/gfk52MASFeHvHPGoFimQj4UR2CApc83eabtnTG9aV7sab2vxW5/p5p tr++aAxvML/pPjXnJvRKXWgkmgPP+n4cBgbrqbxRceHbm6PvpzgTjg0vp0joAxVK75t9EdqWJGl RsF8U6bX3xXM2sY= X-Google-Smtp-Source: AGHT+IEmOnfAJ9my4yJ3mm8rM7h5TDMuOxxQs5p8g6UUpb/82MJzF1bynncwi1SDlIZ2wtNL2ZNmIg== X-Received: by 2002:a05:6a00:28cc:b0:725:f1b1:cbc5 with SMTP id d2e1a72fcca58-730350f96demr1316232b3a.3.1738710059931; Tue, 04 Feb 2025 15:00:59 -0800 (PST) Received: from localhost ([2601:646:9e00:f56e:123b:cea3:439a:b3e3]) by smtp.gmail.com with UTF8SMTPSA id 41be03b00d2f7-acebe384272sm9110022a12.18.2025.02.04.15.00.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2025 15:00:59 -0800 (PST) From: Stanislav Fomichev To: netdev@vger.kernel.org Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, Saeed Mahameed Subject: [RFC net-next 1/4] net: Hold netdev instance lock during ndo_open/ndo_stop Date: Tue, 4 Feb 2025 15:00:54 -0800 Message-ID: <20250204230057.1270362-2-sdf@fomichev.me> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250204230057.1270362-1-sdf@fomichev.me> References: <20250204230057.1270362-1-sdf@fomichev.me> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC For the drivers that use shaper API, switch to the mode where core stack holds the netdev lock. This affects two drivers: * iavf - already grabs netdev lock in ndo_open/ndo_stop, so mostly remove these * netdevsim - switch to _locked APIs to avoid deadlock Cc: Saeed Mahameed Signed-off-by: Stanislav Fomichev --- Documentation/networking/netdevices.rst | 6 ++++-- drivers/net/ethernet/intel/iavf/iavf_main.c | 14 ++++++------- drivers/net/netdevsim/netdev.c | 14 ++++++++----- include/linux/netdevice.h | 23 +++++++++++++++++++++ net/core/dev.c | 12 +++++++++++ net/core/dev.h | 6 ++++-- 6 files changed, 58 insertions(+), 17 deletions(-) diff --git a/Documentation/networking/netdevices.rst b/Documentation/networking/netdevices.rst index 1d37038e9fbe..78213e476ce6 100644 --- a/Documentation/networking/netdevices.rst +++ b/Documentation/networking/netdevices.rst @@ -210,11 +210,13 @@ packets is preferred. struct net_device synchronization rules ======================================= ndo_open: - Synchronization: rtnl_lock() semaphore. + Synchronization: rtnl_lock() semaphore. In addition, netdev instance + lock if the driver implements shaper API. Context: process ndo_stop: - Synchronization: rtnl_lock() semaphore. + Synchronization: rtnl_lock() semaphore. In addition, netdev instance + lock if the driver implements shaper API. Context: process Note: netif_running() is guaranteed false diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index 2d7a18fcc3be..176f9bb871d0 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -4375,22 +4375,21 @@ static int iavf_open(struct net_device *netdev) struct iavf_adapter *adapter = netdev_priv(netdev); int err; + netdev_assert_locked(netdev); + if (adapter->flags & IAVF_FLAG_PF_COMMS_FAILED) { dev_err(&adapter->pdev->dev, "Unable to open device due to PF driver failure.\n"); return -EIO; } - netdev_lock(netdev); while (!mutex_trylock(&adapter->crit_lock)) { /* If we are in __IAVF_INIT_CONFIG_ADAPTER state the crit_lock * is already taken and iavf_open is called from an upper * device's notifier reacting on NETDEV_REGISTER event. * We have to leave here to avoid dead lock. */ - if (adapter->state == __IAVF_INIT_CONFIG_ADAPTER) { - netdev_unlock(netdev); + if (adapter->state == __IAVF_INIT_CONFIG_ADAPTER) return -EBUSY; - } usleep_range(500, 1000); } @@ -4439,7 +4438,6 @@ static int iavf_open(struct net_device *netdev) iavf_irq_enable(adapter, true); mutex_unlock(&adapter->crit_lock); - netdev_unlock(netdev); return 0; @@ -4452,7 +4450,6 @@ static int iavf_open(struct net_device *netdev) iavf_free_all_tx_resources(adapter); err_unlock: mutex_unlock(&adapter->crit_lock); - netdev_unlock(netdev); return err; } @@ -4474,12 +4471,12 @@ static int iavf_close(struct net_device *netdev) u64 aq_to_restore; int status; - netdev_lock(netdev); + netdev_assert_locked(netdev); + mutex_lock(&adapter->crit_lock); if (adapter->state <= __IAVF_DOWN_PENDING) { mutex_unlock(&adapter->crit_lock); - netdev_unlock(netdev); return 0; } @@ -4532,6 +4529,7 @@ static int iavf_close(struct net_device *netdev) if (!status) netdev_warn(netdev, "Device resources not yet released\n"); + netdev_lock(netdev); mutex_lock(&adapter->crit_lock); adapter->aq_required |= aq_to_restore; mutex_unlock(&adapter->crit_lock); diff --git a/drivers/net/netdevsim/netdev.c b/drivers/net/netdevsim/netdev.c index 42f247cbdcee..efec03b34c9f 100644 --- a/drivers/net/netdevsim/netdev.c +++ b/drivers/net/netdevsim/netdev.c @@ -401,7 +401,7 @@ static int nsim_init_napi(struct netdevsim *ns) for (i = 0; i < dev->num_rx_queues; i++) { rq = ns->rq[i]; - netif_napi_add_config(dev, &rq->napi, nsim_poll, i); + netif_napi_add_config_locked(dev, &rq->napi, nsim_poll, i); } for (i = 0; i < dev->num_rx_queues; i++) { @@ -421,7 +421,7 @@ static int nsim_init_napi(struct netdevsim *ns) } for (i = 0; i < dev->num_rx_queues; i++) - __netif_napi_del(&ns->rq[i]->napi); + __netif_napi_del_locked(&ns->rq[i]->napi); return err; } @@ -435,7 +435,7 @@ static void nsim_enable_napi(struct netdevsim *ns) struct nsim_rq *rq = ns->rq[i]; netif_queue_set_napi(dev, i, NETDEV_QUEUE_TYPE_RX, &rq->napi); - napi_enable(&rq->napi); + napi_enable_locked(&rq->napi); } } @@ -444,6 +444,8 @@ static int nsim_open(struct net_device *dev) struct netdevsim *ns = netdev_priv(dev); int err; + netdev_assert_locked(dev); + err = nsim_init_napi(ns); if (err) return err; @@ -461,8 +463,8 @@ static void nsim_del_napi(struct netdevsim *ns) for (i = 0; i < dev->num_rx_queues; i++) { struct nsim_rq *rq = ns->rq[i]; - napi_disable(&rq->napi); - __netif_napi_del(&rq->napi); + napi_disable_locked(&rq->napi); + __netif_napi_del_locked(&rq->napi); } synchronize_net(); @@ -477,6 +479,8 @@ static int nsim_stop(struct net_device *dev) struct netdevsim *ns = netdev_priv(dev); struct netdevsim *peer; + netdev_assert_locked(dev); + netif_carrier_off(dev); peer = rtnl_dereference(ns->peer); if (peer) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 2a59034a5fa2..962774cbce55 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2717,6 +2717,29 @@ static inline void netdev_assert_locked_or_invisible(struct net_device *dev) netdev_assert_locked(dev); } +static inline bool need_netdev_ops_lock(struct net_device *dev) +{ + bool ret = false; + +#if IS_ENABLED(CONFIG_NET_SHAPER) + ret |= !!(dev)->netdev_ops->net_shaper_ops; +#endif + + return ret; +} + +static inline void netdev_lock_ops(struct net_device *dev) +{ + if (need_netdev_ops_lock(dev)) + netdev_lock(dev); +} + +static inline void netdev_unlock_ops(struct net_device *dev) +{ + if (need_netdev_ops_lock(dev)) + netdev_unlock(dev); +} + static inline void netif_napi_set_irq_locked(struct napi_struct *napi, int irq) { napi->irq = irq; diff --git a/net/core/dev.c b/net/core/dev.c index c0021cbd28fc..fda42b2415fc 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -1595,6 +1595,8 @@ static int __dev_open(struct net_device *dev, struct netlink_ext_ack *extack) if (ret) return ret; + netdev_lock_ops(dev); + set_bit(__LINK_STATE_START, &dev->state); if (ops->ndo_validate_addr) @@ -1614,6 +1616,8 @@ static int __dev_open(struct net_device *dev, struct netlink_ext_ack *extack) add_device_randomness(dev->dev_addr, dev->addr_len); } + netdev_unlock_ops(dev); + return ret; } @@ -1684,11 +1688,19 @@ static void __dev_close_many(struct list_head *head) * We allow it to be called even after a DETACH hot-plug * event. */ + + /* TODO: move the lock up before clearing __LINK_STATE_START. + * Generates spurious lockdep warning. + */ + netdev_lock_ops(dev); + if (ops->ndo_stop) ops->ndo_stop(dev); netif_set_up(dev, false); netpoll_poll_enable(dev); + + netdev_unlock_ops(dev); } } diff --git a/net/core/dev.h b/net/core/dev.h index a5b166bbd169..18070c0452e3 100644 --- a/net/core/dev.h +++ b/net/core/dev.h @@ -134,9 +134,11 @@ static inline void netif_set_up(struct net_device *dev, bool value) else dev->flags &= ~IFF_UP; - netdev_lock(dev); + if (!need_netdev_ops_lock(dev)) + netdev_lock(dev); dev->up = value; - netdev_unlock(dev); + if (!need_netdev_ops_lock(dev)) + netdev_unlock(dev); } static inline void netif_set_gso_max_size(struct net_device *dev, From patchwork Tue Feb 4 23:00:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 13960145 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 813D221C185 for ; Tue, 4 Feb 2025 23:01:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738710064; cv=none; b=pQ/uXAuoTGuigpsIGAIWE6IDGU3ypIoswlBky23hZQvCz91Q7OgIDUzxnIpqy4lq7jWUp5DYbul3CWnDml6vKRhZiM978VuMD0KvJS8Ib6ht7Pu0w76K4mGQfn7mRECN5UMtu/8jlg+ytZezab4PnGVH83WvRLwZiOkbpIvFsDA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738710064; c=relaxed/simple; bh=qPpc7VR+hEywtv1QnaIqa7vYsxQXbcdHIsG0J6R3r3Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bECA6MsValMFxtYojnin0xB0dF1038z2knmgIVKak77BurKO9TFaCE32/Aks/YsBDfN7Eev54EipebEOJevhEIStNW02ETmZ0hjnfd7e/Am7IY0sRMlRBSvHKEuMdtrudW57GXmJ6nxfbaKyNYqHe0ttar64BVWdgNpiKmGlG0g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=fomichev.me; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=fomichev.me Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-216426b0865so107726705ad.0 for ; Tue, 04 Feb 2025 15:01:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738710061; x=1739314861; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=86ytV2o0r7olPN40xHsUg9fRYAMG6anGPFiwgPfhGXY=; b=eBh+CJH738Ap8ZN6d5etpty6HBFXurxRTsmq+5bquKrk3Z/1ckMA2l54hJQwC60713 rFJI6yDu+TpfahnVZV9EI2EIkxjU43yr7J2fIUvVBORph71u/dIDOKlclG6YgON9ykbo RGW2bAwAxAhQHeY4CLylkbgeInOzWMlZIki2JJu+wopWOyNlWr2XgNafvFldMYQH67ap GlDop6hoL8WtjMulkML8zmX18nQ3ipEeG5JVm87JlIsbfljOT6SefKiJDrVPN0r7ZFR8 4dC9TSft7kSvGe/stsYkkGSk3/6TnMEskbKt0Z4CDShUCPblt3DtwnxdnDAw4pmpo+c0 NutA== X-Gm-Message-State: AOJu0YwzXAc9NsP/+1xtg1nJ4T9jj86yz/DO2snlgHXn5h1CCaAS+Oij b+GubqNZ0FnepVMWTdMpuSrXDtgIwAEKzKS1+sljsaatsJxPRwAYna2K X-Gm-Gg: ASbGncuBMlCjFBRvE5lK9WsuNkXiSIFcRTfT0xQ/0Z6+mNaip+JqCLYTo6Ct+TrHviU Jcv4XUckpt/wLt87/zKkMazusCv+bFT8CIulLFqNKHOdF9W0R6suLEmreusPMSSEvzevjD7bneJ JAnQmfK81lPsllVW2XO1wOKbSmHzgByMftbgsES6hTyIvpl2gGUKSIjaElJr1q5tYQa+42DjP63 pHnvsev9V6IHdggIwEW5fi6scKWt5t1UPxRy50FBy8kTR8DJILAW0owx6SHLmdyDWWFIWrdST8v m9RUBSZj7m8QBT0= X-Google-Smtp-Source: AGHT+IE9yLB8IpGZdqlvY/A5sLyDm+GALV7t7M5bEEv7/AChC3pm2fGXfYVpc48FPLGjQjfV3ocdIA== X-Received: by 2002:a05:6a21:6da4:b0:1ed:75f4:d289 with SMTP id adf61e73a8af0-1ede8845ecamr698985637.19.1738710061209; Tue, 04 Feb 2025 15:01:01 -0800 (PST) Received: from localhost ([2601:646:9e00:f56e:123b:cea3:439a:b3e3]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72fe6424f5dsm11071807b3a.42.2025.02.04.15.01.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2025 15:01:00 -0800 (PST) From: Stanislav Fomichev To: netdev@vger.kernel.org Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, Saeed Mahameed Subject: [RFC net-next 2/4] net: Hold netdev instance lock during ndo_setup_tc Date: Tue, 4 Feb 2025 15:00:55 -0800 Message-ID: <20250204230057.1270362-3-sdf@fomichev.me> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250204230057.1270362-1-sdf@fomichev.me> References: <20250204230057.1270362-1-sdf@fomichev.me> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Introduce new dev_setup_tc that handles the details and call it from all qdiscs/classifiers. The instance lock is still applied only to the drivers that implement shaper API so only iavf is affected. Cc: Saeed Mahameed Signed-off-by: Stanislav Fomichev --- Documentation/networking/netdevices.rst | 4 ++++ drivers/net/ethernet/intel/iavf/iavf_main.c | 2 -- include/linux/netdevice.h | 2 ++ net/core/dev.c | 22 +++++++++++++++++++++ net/dsa/user.c | 5 +---- net/netfilter/nf_flow_table_offload.c | 2 +- net/netfilter/nf_tables_offload.c | 2 +- net/sched/cls_api.c | 2 +- net/sched/sch_api.c | 13 +++--------- net/sched/sch_cbs.c | 9 ++------- net/sched/sch_etf.c | 9 ++------- net/sched/sch_ets.c | 10 ++-------- net/sched/sch_fifo.c | 10 ++-------- net/sched/sch_gred.c | 5 +---- net/sched/sch_htb.c | 2 +- net/sched/sch_mq.c | 5 +---- net/sched/sch_mqprio.c | 6 ++---- net/sched/sch_prio.c | 5 +---- net/sched/sch_red.c | 8 ++------ net/sched/sch_taprio.c | 16 +++------------ net/sched/sch_tbf.c | 10 ++-------- 21 files changed, 56 insertions(+), 93 deletions(-) diff --git a/Documentation/networking/netdevices.rst b/Documentation/networking/netdevices.rst index 78213e476ce6..c6087d92d740 100644 --- a/Documentation/networking/netdevices.rst +++ b/Documentation/networking/netdevices.rst @@ -257,6 +257,10 @@ struct net_device synchronization rules Synchronization: rtnl_lock() semaphore, or RCU. Context: atomic (can't sleep under RCU) +ndo_setup_tc: + Synchronization: rtnl_lock() semaphore. In addition, netdev instance + lock if the driver implements shaper API. + ndo_start_xmit: Synchronization: __netif_tx_lock spinlock. diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index 176f9bb871d0..4fe481433842 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -3707,10 +3707,8 @@ static int __iavf_setup_tc(struct net_device *netdev, void *type_data) if (test_bit(__IAVF_IN_REMOVE_TASK, &adapter->crit_section)) return 0; - netdev_lock(netdev); netif_set_real_num_rx_queues(netdev, total_qps); netif_set_real_num_tx_queues(netdev, total_qps); - netdev_unlock(netdev); return ret; } diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 962774cbce55..6f2eb129ef3e 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3315,6 +3315,8 @@ int dev_alloc_name(struct net_device *dev, const char *name); int dev_open(struct net_device *dev, struct netlink_ext_ack *extack); void dev_close(struct net_device *dev); void dev_close_many(struct list_head *head, bool unlink); +int dev_setup_tc(struct net_device *dev, enum tc_setup_type type, + void *type_data); void dev_disable_lro(struct net_device *dev); int dev_loopback_xmit(struct net *net, struct sock *sk, struct sk_buff *newskb); u16 dev_pick_tx_zero(struct net_device *dev, struct sk_buff *skb, diff --git a/net/core/dev.c b/net/core/dev.c index fda42b2415fc..e55da80d24e2 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -1754,6 +1754,28 @@ void dev_close(struct net_device *dev) } EXPORT_SYMBOL(dev_close); +int dev_setup_tc(struct net_device *dev, enum tc_setup_type type, + void *type_data) +{ + const struct net_device_ops *ops = dev->netdev_ops; + + ASSERT_RTNL(); + + if (tc_can_offload(dev) && ops->ndo_setup_tc) { + int ret = -ENODEV; + + if (netif_device_present(dev)) { + netdev_lock_ops(dev); + ret = ops->ndo_setup_tc(dev, type, type_data); + netdev_unlock_ops(dev); + } + + return ret; + } + + return -EOPNOTSUPP; +} +EXPORT_SYMBOL(dev_setup_tc); /** * dev_disable_lro - disable Large Receive Offload on a device diff --git a/net/dsa/user.c b/net/dsa/user.c index 291ab1b4acc4..f2ac7662e4cc 100644 --- a/net/dsa/user.c +++ b/net/dsa/user.c @@ -1729,10 +1729,7 @@ static int dsa_user_setup_ft_block(struct dsa_switch *ds, int port, { struct net_device *conduit = dsa_port_to_conduit(dsa_to_port(ds, port)); - if (!conduit->netdev_ops->ndo_setup_tc) - return -EOPNOTSUPP; - - return conduit->netdev_ops->ndo_setup_tc(conduit, TC_SETUP_FT, type_data); + return dev_setup_tc(conduit, TC_SETUP_FT, type_data); } static int dsa_user_setup_tc(struct net_device *dev, enum tc_setup_type type, diff --git a/net/netfilter/nf_flow_table_offload.c b/net/netfilter/nf_flow_table_offload.c index e06bc36f49fe..0ec4abded10d 100644 --- a/net/netfilter/nf_flow_table_offload.c +++ b/net/netfilter/nf_flow_table_offload.c @@ -1175,7 +1175,7 @@ static int nf_flow_table_offload_cmd(struct flow_block_offload *bo, nf_flow_table_block_offload_init(bo, dev_net(dev), cmd, flowtable, extack); down_write(&flowtable->flow_block_lock); - err = dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_FT, bo); + err = dev_setup_tc(dev, TC_SETUP_FT, bo); up_write(&flowtable->flow_block_lock); if (err < 0) return err; diff --git a/net/netfilter/nf_tables_offload.c b/net/netfilter/nf_tables_offload.c index 64675f1c7f29..b761899c143c 100644 --- a/net/netfilter/nf_tables_offload.c +++ b/net/netfilter/nf_tables_offload.c @@ -390,7 +390,7 @@ static int nft_block_offload_cmd(struct nft_base_chain *chain, nft_flow_block_offload_init(&bo, dev_net(dev), cmd, chain, &extack); - err = dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_BLOCK, &bo); + err = dev_setup_tc(dev, TC_SETUP_BLOCK, &bo); if (err < 0) return err; diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c index 8e47e5355be6..082b355be8e6 100644 --- a/net/sched/cls_api.c +++ b/net/sched/cls_api.c @@ -835,7 +835,7 @@ static int tcf_block_offload_cmd(struct tcf_block *block, if (dev->netdev_ops->ndo_setup_tc) { int err; - err = dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_BLOCK, &bo); + err = dev_setup_tc(dev, TC_SETUP_BLOCK, &bo); if (err < 0) { if (err != -EOPNOTSUPP) NL_SET_ERR_MSG(extack, "Driver ndo_setup_tc failed"); diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c index e3e91cf867eb..6d0728cfc19f 100644 --- a/net/sched/sch_api.c +++ b/net/sched/sch_api.c @@ -833,10 +833,8 @@ int qdisc_offload_dump_helper(struct Qdisc *sch, enum tc_setup_type type, int err; sch->flags &= ~TCQ_F_OFFLOADED; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return 0; - err = dev->netdev_ops->ndo_setup_tc(dev, type, type_data); + err = dev_setup_tc(dev, type, type_data); if (err == -EOPNOTSUPP) return 0; @@ -855,10 +853,7 @@ void qdisc_offload_graft_helper(struct net_device *dev, struct Qdisc *sch, bool any_qdisc_is_offloaded; int err; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return; - - err = dev->netdev_ops->ndo_setup_tc(dev, type, type_data); + err = dev_setup_tc(dev, type, type_data); /* Don't report error if the graft is part of destroy operation. */ if (!err || !new || new == &noop_qdisc) @@ -880,7 +875,6 @@ void qdisc_offload_query_caps(struct net_device *dev, enum tc_setup_type type, void *caps, size_t caps_len) { - const struct net_device_ops *ops = dev->netdev_ops; struct tc_query_caps_base base = { .type = type, .caps = caps, @@ -888,8 +882,7 @@ void qdisc_offload_query_caps(struct net_device *dev, memset(caps, 0, caps_len); - if (ops->ndo_setup_tc) - ops->ndo_setup_tc(dev, TC_QUERY_CAPS, &base); + dev_setup_tc(dev, TC_QUERY_CAPS, &base); } EXPORT_SYMBOL(qdisc_offload_query_caps); diff --git a/net/sched/sch_cbs.c b/net/sched/sch_cbs.c index 8c9a0400c862..492a98fa209b 100644 --- a/net/sched/sch_cbs.c +++ b/net/sched/sch_cbs.c @@ -251,7 +251,6 @@ static void cbs_disable_offload(struct net_device *dev, struct cbs_sched_data *q) { struct tc_cbs_qopt_offload cbs = { }; - const struct net_device_ops *ops; int err; if (!q->offload) @@ -260,14 +259,10 @@ static void cbs_disable_offload(struct net_device *dev, q->enqueue = cbs_enqueue_soft; q->dequeue = cbs_dequeue_soft; - ops = dev->netdev_ops; - if (!ops->ndo_setup_tc) - return; - cbs.queue = q->queue; cbs.enable = 0; - err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_CBS, &cbs); + err = dev_setup_tc(dev, TC_SETUP_QDISC_CBS, &cbs); if (err < 0) pr_warn("Couldn't disable CBS offload for queue %d\n", cbs.queue); @@ -294,7 +289,7 @@ static int cbs_enable_offload(struct net_device *dev, struct cbs_sched_data *q, cbs.idleslope = opt->idleslope; cbs.sendslope = opt->sendslope; - err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_CBS, &cbs); + err = dev_setup_tc(dev, TC_SETUP_QDISC_CBS, &cbs); if (err < 0) { NL_SET_ERR_MSG(extack, "Specified device failed to setup cbs hardware offload"); return err; diff --git a/net/sched/sch_etf.c b/net/sched/sch_etf.c index c74d778c32a1..f183c2e9a4e8 100644 --- a/net/sched/sch_etf.c +++ b/net/sched/sch_etf.c @@ -297,20 +297,15 @@ static void etf_disable_offload(struct net_device *dev, struct etf_sched_data *q) { struct tc_etf_qopt_offload etf = { }; - const struct net_device_ops *ops; int err; if (!q->offload) return; - ops = dev->netdev_ops; - if (!ops->ndo_setup_tc) - return; - etf.queue = q->queue; etf.enable = 0; - err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_ETF, &etf); + err = dev_setup_tc(dev, TC_SETUP_QDISC_ETF, &etf); if (err < 0) pr_warn("Couldn't disable ETF offload for queue %d\n", etf.queue); @@ -331,7 +326,7 @@ static int etf_enable_offload(struct net_device *dev, struct etf_sched_data *q, etf.queue = q->queue; etf.enable = 1; - err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_ETF, &etf); + err = dev_setup_tc(dev, TC_SETUP_QDISC_ETF, &etf); if (err < 0) { NL_SET_ERR_MSG(extack, "Specified device failed to setup ETF hardware offload"); return err; diff --git a/net/sched/sch_ets.c b/net/sched/sch_ets.c index 516038a44163..1757dd3b0552 100644 --- a/net/sched/sch_ets.c +++ b/net/sched/sch_ets.c @@ -117,9 +117,6 @@ static void ets_offload_change(struct Qdisc *sch) unsigned int weight; unsigned int i; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return; - qopt.command = TC_ETS_REPLACE; qopt.handle = sch->handle; qopt.parent = sch->parent; @@ -142,7 +139,7 @@ static void ets_offload_change(struct Qdisc *sch) qopt.replace_params.weights[i] = weight; } - dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_ETS, &qopt); + dev_setup_tc(dev, TC_SETUP_QDISC_ETS, &qopt); } static void ets_offload_destroy(struct Qdisc *sch) @@ -150,13 +147,10 @@ static void ets_offload_destroy(struct Qdisc *sch) struct net_device *dev = qdisc_dev(sch); struct tc_ets_qopt_offload qopt; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return; - qopt.command = TC_ETS_DESTROY; qopt.handle = sch->handle; qopt.parent = sch->parent; - dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_ETS, &qopt); + dev_setup_tc(dev, TC_SETUP_QDISC_ETS, &qopt); } static void ets_offload_graft(struct Qdisc *sch, struct Qdisc *new, diff --git a/net/sched/sch_fifo.c b/net/sched/sch_fifo.c index b50b2c2cc09b..4729a090c876 100644 --- a/net/sched/sch_fifo.c +++ b/net/sched/sch_fifo.c @@ -58,13 +58,10 @@ static void fifo_offload_init(struct Qdisc *sch) struct net_device *dev = qdisc_dev(sch); struct tc_fifo_qopt_offload qopt; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return; - qopt.command = TC_FIFO_REPLACE; qopt.handle = sch->handle; qopt.parent = sch->parent; - dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_FIFO, &qopt); + dev_setup_tc(dev, TC_SETUP_QDISC_FIFO, &qopt); } static void fifo_offload_destroy(struct Qdisc *sch) @@ -72,13 +69,10 @@ static void fifo_offload_destroy(struct Qdisc *sch) struct net_device *dev = qdisc_dev(sch); struct tc_fifo_qopt_offload qopt; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return; - qopt.command = TC_FIFO_DESTROY; qopt.handle = sch->handle; qopt.parent = sch->parent; - dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_FIFO, &qopt); + dev_setup_tc(dev, TC_SETUP_QDISC_FIFO, &qopt); } static int fifo_offload_dump(struct Qdisc *sch) diff --git a/net/sched/sch_gred.c b/net/sched/sch_gred.c index ab6234b4fcd5..d1a29842a2b4 100644 --- a/net/sched/sch_gred.c +++ b/net/sched/sch_gred.c @@ -314,9 +314,6 @@ static void gred_offload(struct Qdisc *sch, enum tc_gred_command command) struct net_device *dev = qdisc_dev(sch); struct tc_gred_qopt_offload *opt = table->opt; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return; - memset(opt, 0, sizeof(*opt)); opt->command = command; opt->handle = sch->handle; @@ -348,7 +345,7 @@ static void gred_offload(struct Qdisc *sch, enum tc_gred_command command) opt->set.qstats = &sch->qstats; } - dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_GRED, opt); + dev_setup_tc(dev, TC_SETUP_QDISC_GRED, opt); } static int gred_offload_dump_stats(struct Qdisc *sch) diff --git a/net/sched/sch_htb.c b/net/sched/sch_htb.c index c31bc5489bdd..ed406b3ceb7b 100644 --- a/net/sched/sch_htb.c +++ b/net/sched/sch_htb.c @@ -1041,7 +1041,7 @@ static void htb_work_func(struct work_struct *work) static int htb_offload(struct net_device *dev, struct tc_htb_qopt_offload *opt) { - return dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_HTB, opt); + return dev_setup_tc(dev, TC_SETUP_QDISC_HTB, opt); } static int htb_init(struct Qdisc *sch, struct nlattr *opt, diff --git a/net/sched/sch_mq.c b/net/sched/sch_mq.c index c860119a8f09..a5ba59728d63 100644 --- a/net/sched/sch_mq.c +++ b/net/sched/sch_mq.c @@ -29,10 +29,7 @@ static int mq_offload(struct Qdisc *sch, enum tc_mq_command cmd) .handle = sch->handle, }; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return -EOPNOTSUPP; - - return dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_MQ, &opt); + return dev_setup_tc(dev, TC_SETUP_QDISC_MQ, &opt); } static int mq_offload_stats(struct Qdisc *sch) diff --git a/net/sched/sch_mqprio.c b/net/sched/sch_mqprio.c index 51d4013b6121..2c9a90b83c2b 100644 --- a/net/sched/sch_mqprio.c +++ b/net/sched/sch_mqprio.c @@ -67,8 +67,7 @@ static int mqprio_enable_offload(struct Qdisc *sch, mqprio_fp_to_offload(priv->fp, &mqprio); - err = dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_MQPRIO, - &mqprio); + err = dev_setup_tc(dev, TC_SETUP_QDISC_MQPRIO, &mqprio); if (err) return err; @@ -86,8 +85,7 @@ static void mqprio_disable_offload(struct Qdisc *sch) switch (priv->mode) { case TC_MQPRIO_MODE_DCB: case TC_MQPRIO_MODE_CHANNEL: - dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_MQPRIO, - &mqprio); + dev_setup_tc(dev, TC_SETUP_QDISC_MQPRIO, &mqprio); break; } } diff --git a/net/sched/sch_prio.c b/net/sched/sch_prio.c index cc30f7a32f1a..276e290b4071 100644 --- a/net/sched/sch_prio.c +++ b/net/sched/sch_prio.c @@ -145,9 +145,6 @@ static int prio_offload(struct Qdisc *sch, struct tc_prio_qopt *qopt) .parent = sch->parent, }; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return -EOPNOTSUPP; - if (qopt) { opt.command = TC_PRIO_REPLACE; opt.replace_params.bands = qopt->bands; @@ -158,7 +155,7 @@ static int prio_offload(struct Qdisc *sch, struct tc_prio_qopt *qopt) opt.command = TC_PRIO_DESTROY; } - return dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_PRIO, &opt); + return dev_setup_tc(dev, TC_SETUP_QDISC_PRIO, &opt); } static void diff --git a/net/sched/sch_red.c b/net/sched/sch_red.c index ef8a2afed26b..235ec57b29b8 100644 --- a/net/sched/sch_red.c +++ b/net/sched/sch_red.c @@ -192,9 +192,6 @@ static int red_offload(struct Qdisc *sch, bool enable) .parent = sch->parent, }; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return -EOPNOTSUPP; - if (enable) { opt.command = TC_RED_REPLACE; opt.set.min = q->parms.qth_min >> q->parms.Wlog; @@ -209,7 +206,7 @@ static int red_offload(struct Qdisc *sch, bool enable) opt.command = TC_RED_DESTROY; } - return dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_RED, &opt); + return dev_setup_tc(dev, TC_SETUP_QDISC_RED, &opt); } static void red_destroy(struct Qdisc *sch) @@ -460,8 +457,7 @@ static int red_dump_stats(struct Qdisc *sch, struct gnet_dump *d) .xstats = &q->stats, }, }; - dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_RED, - &hw_stats_request); + dev_setup_tc(dev, TC_SETUP_QDISC_RED, &hw_stats_request); } st.early = q->stats.prob_drop + q->stats.forced_drop; st.pdrop = q->stats.pdrop; diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c index a68e17891b0b..18b7e59df786 100644 --- a/net/sched/sch_taprio.c +++ b/net/sched/sch_taprio.c @@ -1539,7 +1539,7 @@ static int taprio_enable_offload(struct net_device *dev, for (tc = 0; tc < TC_MAX_QUEUE; tc++) offload->max_sdu[tc] = q->max_sdu[tc]; - err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_TAPRIO, offload); + err = dev_setup_tc(dev, TC_SETUP_QDISC_TAPRIO, offload); if (err < 0) { NL_SET_ERR_MSG_WEAK(extack, "Device failed to setup taprio offload"); @@ -1579,7 +1579,7 @@ static int taprio_disable_offload(struct net_device *dev, } offload->cmd = TAPRIO_CMD_DESTROY; - err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_TAPRIO, offload); + err = dev_setup_tc(dev, TC_SETUP_QDISC_TAPRIO, offload); if (err < 0) { NL_SET_ERR_MSG(extack, "Device failed to disable offload"); @@ -2314,24 +2314,14 @@ static int taprio_dump_xstats(struct Qdisc *sch, struct gnet_dump *d, struct tc_taprio_qopt_stats *stats) { struct net_device *dev = qdisc_dev(sch); - const struct net_device_ops *ops; struct sk_buff *skb = d->skb; struct nlattr *xstats; int err; - ops = qdisc_dev(sch)->netdev_ops; - - /* FIXME I could use qdisc_offload_dump_helper(), but that messes - * with sch->flags depending on whether the device reports taprio - * stats, and I'm not sure whether that's a good idea, considering - * that stats are optional to the offload itself - */ - if (!ops->ndo_setup_tc) - return 0; memset(stats, 0xff, sizeof(*stats)); - err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_TAPRIO, offload); + err = dev_setup_tc(dev, TC_SETUP_QDISC_TAPRIO, offload); if (err == -EOPNOTSUPP) return 0; if (err) diff --git a/net/sched/sch_tbf.c b/net/sched/sch_tbf.c index dc26b22d53c7..5a2133716446 100644 --- a/net/sched/sch_tbf.c +++ b/net/sched/sch_tbf.c @@ -145,9 +145,6 @@ static void tbf_offload_change(struct Qdisc *sch) struct net_device *dev = qdisc_dev(sch); struct tc_tbf_qopt_offload qopt; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return; - qopt.command = TC_TBF_REPLACE; qopt.handle = sch->handle; qopt.parent = sch->parent; @@ -155,7 +152,7 @@ static void tbf_offload_change(struct Qdisc *sch) qopt.replace_params.max_size = q->max_size; qopt.replace_params.qstats = &sch->qstats; - dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_TBF, &qopt); + dev_setup_tc(dev, TC_SETUP_QDISC_TBF, &qopt); } static void tbf_offload_destroy(struct Qdisc *sch) @@ -163,13 +160,10 @@ static void tbf_offload_destroy(struct Qdisc *sch) struct net_device *dev = qdisc_dev(sch); struct tc_tbf_qopt_offload qopt; - if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc) - return; - qopt.command = TC_TBF_DESTROY; qopt.handle = sch->handle; qopt.parent = sch->parent; - dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_TBF, &qopt); + dev_setup_tc(dev, TC_SETUP_QDISC_TBF, &qopt); } static int tbf_offload_dump(struct Qdisc *sch) From patchwork Tue Feb 4 23:00:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 13960146 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C1D521C19F for ; Tue, 4 Feb 2025 23:01:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738710065; cv=none; b=SfJUzNrpUC6yrYRIk27ULRmh8nUIYctRrF3y9x1wCcSqVm6oUCxMlXUnAx/HBGElVK8YubCPPuaxTrd7bju8UslxAAyTRmprm6HSlkxhoilAvXC7fuyBj76Ou3XVI7Zn/qhh1ApdnRPoZAGYd/oycsxC5An0vndDwie0zMk8kFw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738710065; c=relaxed/simple; bh=kElv2szNBWBbxKK8NRkVYveCTgrstoch1qBTR699a6M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T/+c66pBgOIgRtvLfNJSiZIYB2W3fCGxwFvQMNEt3e9acuzVKA6z+kKsszRAq0wOLNz+M5JipmdXLFLPRgPJ8bXWKEbxtOsJOD4XgRkooDgK1jEJZ5pgAWoROu5QN69wSj+J16OqFcOv9NKTM8bnJ9ixHz68HriUwBVT0kCnOM8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=fomichev.me; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=fomichev.me Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-2f9cd9601b8so1681389a91.3 for ; Tue, 04 Feb 2025 15:01:03 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738710063; x=1739314863; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3CsxBP0ibbJQrbtR9vPHba3ffwbyUFme6Ne8I4mjKAc=; b=MzAhCqyCx9lsnMyoSbIcB2SfYY1u6eHNJ9fGdWh06OqeKF80I2s7uvI7u+9RlH04wE UU3PQ+P/VxwHzg7P0tKbMkCQ8hVaJFdZ6n0w0hJtP3HmFeNZgd9mQBvw1ukQEuJsqM9a 0TxZyOdxOetUs7on07bfh5N3aFrfqeW6V2SCEYYg5RJOBDMI6FAP/sDL33phU5RCDA2A XWGFucoc7sJxnmXsW1FfrfKPLCiKVGplezvCUo122qopLeNQmimnJqwyaVctF7ReTZla PaxVPe/Uw/Ln+xZrohjGveJQDFv5T7ai4MSUBhOXEAYx/dnBybsPKd5VFpZdJqud+ojP 5asQ== X-Gm-Message-State: AOJu0YypNIhELs11GUHizE6PWAXcDS1h4aOerAJz82398iA4zlWQEvtv jN3eczjeDrhWXJctVSoSJ+vbxoo7m4auKEW5LKrMAR/V1xYjjGRo8kwQ X-Gm-Gg: ASbGncsTllzHvpj0GXcybkKtiv3sjaAdKQdQxF/A/rqDjOcd2VeReCopePJbinS9uYI JP4FSvFlVXtxBy+LOEUvAycC+0IdiMdaSS54+/fqlO5Z09UBJ9y+UIuENMLf8qKEj23cd0n3fZb TcH+qsc2uWJELMfPoxmHookBXzBMLR1iaM705i4Bdq7By3rYS8WexLl1Z7zbkwkn+GJbnUnpaxY yIxR48gD0QGL13Ax4jRITqkDnc+zhRSyxEEqRJzDssiXw0JNso9RQxeAlYXYODIMm6jvVh75w8c UzYooE/QEjzqPk4= X-Google-Smtp-Source: AGHT+IHP3kzsE/1q+yYSQcspjnLr24+TWIXSIdLRi/kET42qT0UfXwQqNIgUC+cmi5cnGOhqUCWd3w== X-Received: by 2002:a17:90b:2fcf:b0:2ee:8ea0:6b9c with SMTP id 98e67ed59e1d1-2f9e076d86cmr1027813a91.12.1738710062506; Tue, 04 Feb 2025 15:01:02 -0800 (PST) Received: from localhost ([2601:646:9e00:f56e:123b:cea3:439a:b3e3]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21f06e7d4f6sm18939255ad.112.2025.02.04.15.01.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2025 15:01:02 -0800 (PST) From: Stanislav Fomichev To: netdev@vger.kernel.org Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, Saeed Mahameed Subject: [RFC net-next 3/4] net: Hold netdev instance lock for more NDOs Date: Tue, 4 Feb 2025 15:00:56 -0800 Message-ID: <20250204230057.1270362-4-sdf@fomichev.me> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250204230057.1270362-1-sdf@fomichev.me> References: <20250204230057.1270362-1-sdf@fomichev.me> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Convert all ndo_eth_ioctl invocations to dev_eth_ioctl which does the locking. Reflow some of the dev_siocxxx to drop else clause. Fix tabs vs spaces in neighboring lines while at it.. Remove rtnl_lock from ndo_get_stats and clarify that read path can race with the write path. Still shaper-only drivers (iavf/netdevim). Cc: Saeed Mahameed Signed-off-by: Stanislav Fomichev --- Documentation/networking/netdevices.rst | 28 +++++++++------- drivers/net/bonding/bond_main.c | 9 +++-- include/linux/netdevice.h | 2 ++ net/8021q/vlan_dev.c | 4 +-- net/core/dev_ioctl.c | 44 +++++++++++++++++-------- net/ieee802154/socket.c | 2 ++ net/phonet/pn_dev.c | 2 ++ 7 files changed, 58 insertions(+), 33 deletions(-) diff --git a/Documentation/networking/netdevices.rst b/Documentation/networking/netdevices.rst index c6087d92d740..3ed1bf322a5c 100644 --- a/Documentation/networking/netdevices.rst +++ b/Documentation/networking/netdevices.rst @@ -221,40 +221,46 @@ struct net_device synchronization rules Note: netif_running() is guaranteed false ndo_do_ioctl: - Synchronization: rtnl_lock() semaphore. + Synchronization: rtnl_lock() semaphore. In addition, netdev instance + lock if the driver implements shaper API. Context: process - This is only called by network subsystems internally, - not by user space calling ioctl as it was in before - linux-5.14. + This is only called by network subsystems internally, + not by user space calling ioctl as it was in before + linux-5.14. ndo_siocbond: - Synchronization: rtnl_lock() semaphore. + Synchronization: rtnl_lock() semaphore. In addition, netdev instance + lock if the driver implements shaper API. Context: process - Used by the bonding driver for the SIOCBOND family of - ioctl commands. + Used by the bonding driver for the SIOCBOND family of + ioctl commands. ndo_siocwandev: - Synchronization: rtnl_lock() semaphore. + Synchronization: rtnl_lock() semaphore. In addition, netdev instance + lock if the driver implements shaper API. Context: process Used by the drivers/net/wan framework to handle the SIOCWANDEV ioctl with the if_settings structure. ndo_siocdevprivate: - Synchronization: rtnl_lock() semaphore. + Synchronization: rtnl_lock() semaphore. In addition, netdev instance + lock if the driver implements shaper API. Context: process This is used to implement SIOCDEVPRIVATE ioctl helpers. These should not be added to new drivers, so don't use. ndo_eth_ioctl: - Synchronization: rtnl_lock() semaphore. + Synchronization: rtnl_lock() semaphore. In addition, netdev instance + lock if the driver implements shaper API. Context: process ndo_get_stats: - Synchronization: rtnl_lock() semaphore, or RCU. + Synchronization: RCU (can be called concurrently with the stats + update path). Context: atomic (can't sleep under RCU) ndo_setup_tc: diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index e45bba240cbc..025d605166c3 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -858,7 +858,6 @@ static int bond_check_dev_link(struct bonding *bond, struct net_device *slave_dev, int reporting) { const struct net_device_ops *slave_ops = slave_dev->netdev_ops; - int (*ioctl)(struct net_device *, struct ifreq *, int); struct ifreq ifr; struct mii_ioctl_data *mii; @@ -874,8 +873,7 @@ static int bond_check_dev_link(struct bonding *bond, BMSR_LSTATUS : 0; /* Ethtool can't be used, fallback to MII ioctls. */ - ioctl = slave_ops->ndo_eth_ioctl; - if (ioctl) { + if (slave_ops->ndo_eth_ioctl) { /* TODO: set pointer to correct ioctl on a per team member * bases to make this more efficient. that is, once * we determine the correct ioctl, we will always @@ -891,9 +889,10 @@ static int bond_check_dev_link(struct bonding *bond, /* Yes, the mii is overlaid on the ifreq.ifr_ifru */ strscpy_pad(ifr.ifr_name, slave_dev->name, IFNAMSIZ); mii = if_mii(&ifr); - if (ioctl(slave_dev, &ifr, SIOCGMIIPHY) == 0) { + + if (dev_eth_ioctl(slave_dev, &ifr, SIOCGMIIPHY) == 0) { mii->reg_num = MII_BMSR; - if (ioctl(slave_dev, &ifr, SIOCGMIIREG) == 0) + if (dev_eth_ioctl(slave_dev, &ifr, SIOCGMIIREG) == 0) return mii->val_out & BMSR_LSTATUS; } } diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 6f2eb129ef3e..e49b818054b9 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -4159,6 +4159,8 @@ int put_user_ifreq(struct ifreq *ifr, void __user *arg); int dev_ioctl(struct net *net, unsigned int cmd, struct ifreq *ifr, void __user *data, bool *need_copyout); int dev_ifconf(struct net *net, struct ifconf __user *ifc); +int dev_eth_ioctl(struct net_device *dev, + struct ifreq *ifr, unsigned int cmd); int generic_hwtstamp_get_lower(struct net_device *dev, struct kernel_hwtstamp_config *kernel_cfg); int generic_hwtstamp_set_lower(struct net_device *dev, diff --git a/net/8021q/vlan_dev.c b/net/8021q/vlan_dev.c index 91d134961357..ee3283400716 100644 --- a/net/8021q/vlan_dev.c +++ b/net/8021q/vlan_dev.c @@ -377,7 +377,6 @@ static int vlan_hwtstamp_set(struct net_device *dev, static int vlan_dev_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd) { struct net_device *real_dev = vlan_dev_priv(dev)->real_dev; - const struct net_device_ops *ops = real_dev->netdev_ops; struct ifreq ifrr; int err = -EOPNOTSUPP; @@ -388,8 +387,7 @@ static int vlan_dev_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd) case SIOCGMIIPHY: case SIOCGMIIREG: case SIOCSMIIREG: - if (netif_device_present(real_dev) && ops->ndo_eth_ioctl) - err = ops->ndo_eth_ioctl(real_dev, &ifrr, cmd); + err = dev_eth_ioctl(real_dev, &ifrr, cmd); break; } diff --git a/net/core/dev_ioctl.c b/net/core/dev_ioctl.c index 4c2098ac9d72..8dc2c323fe58 100644 --- a/net/core/dev_ioctl.c +++ b/net/core/dev_ioctl.c @@ -240,19 +240,23 @@ int net_hwtstamp_validate(const struct kernel_hwtstamp_config *cfg) return 0; } -static int dev_eth_ioctl(struct net_device *dev, - struct ifreq *ifr, unsigned int cmd) +int dev_eth_ioctl(struct net_device *dev, + struct ifreq *ifr, unsigned int cmd) { const struct net_device_ops *ops = dev->netdev_ops; + int ret = -ENODEV; if (!ops->ndo_eth_ioctl) return -EOPNOTSUPP; - if (!netif_device_present(dev)) - return -ENODEV; + netdev_lock_ops(dev); + if (netif_device_present(dev)) + ret = ops->ndo_eth_ioctl(dev, ifr, cmd); + netdev_unlock_ops(dev); - return ops->ndo_eth_ioctl(dev, ifr, cmd); + return ret; } +EXPORT_SYMBOL(dev_eth_ioctl); /** * dev_get_hwtstamp_phylib() - Get hardware timestamping settings of NIC @@ -504,10 +508,14 @@ static int dev_siocbond(struct net_device *dev, const struct net_device_ops *ops = dev->netdev_ops; if (ops->ndo_siocbond) { + int ret = -ENODEV; + + netdev_lock_ops(dev); if (netif_device_present(dev)) - return ops->ndo_siocbond(dev, ifr, cmd); - else - return -ENODEV; + ret = ops->ndo_siocbond(dev, ifr, cmd); + netdev_unlock_ops(dev); + + return ret; } return -EOPNOTSUPP; @@ -519,10 +527,14 @@ static int dev_siocdevprivate(struct net_device *dev, struct ifreq *ifr, const struct net_device_ops *ops = dev->netdev_ops; if (ops->ndo_siocdevprivate) { + int ret = -ENODEV; + + netdev_lock_ops(dev); if (netif_device_present(dev)) - return ops->ndo_siocdevprivate(dev, ifr, data, cmd); - else - return -ENODEV; + ret = ops->ndo_siocdevprivate(dev, ifr, data, cmd); + netdev_unlock_ops(dev); + + return ret; } return -EOPNOTSUPP; @@ -533,10 +545,14 @@ static int dev_siocwandev(struct net_device *dev, struct if_settings *ifs) const struct net_device_ops *ops = dev->netdev_ops; if (ops->ndo_siocwandev) { + int ret = -ENODEV; + + netdev_lock_ops(dev); if (netif_device_present(dev)) - return ops->ndo_siocwandev(dev, ifs); - else - return -ENODEV; + ret = ops->ndo_siocwandev(dev, ifs); + netdev_unlock_ops(dev); + + return ret; } return -EOPNOTSUPP; diff --git a/net/ieee802154/socket.c b/net/ieee802154/socket.c index 18d267921bb5..b86d090a2ba5 100644 --- a/net/ieee802154/socket.c +++ b/net/ieee802154/socket.c @@ -139,8 +139,10 @@ static int ieee802154_dev_ioctl(struct sock *sk, struct ifreq __user *arg, if (!dev) return -ENODEV; + netdev_lock_ops(dev); if (dev->type == ARPHRD_IEEE802154 && dev->netdev_ops->ndo_do_ioctl) ret = dev->netdev_ops->ndo_do_ioctl(dev, &ifr, cmd); + netdev_unlock_ops(dev); if (!ret && put_user_ifreq(&ifr, arg)) ret = -EFAULT; diff --git a/net/phonet/pn_dev.c b/net/phonet/pn_dev.c index 5c36bae37b8f..945f2315499a 100644 --- a/net/phonet/pn_dev.c +++ b/net/phonet/pn_dev.c @@ -246,8 +246,10 @@ static int phonet_device_autoconf(struct net_device *dev) if (!dev->netdev_ops->ndo_siocdevprivate) return -EOPNOTSUPP; + netdev_lock_ops(dev); ret = dev->netdev_ops->ndo_siocdevprivate(dev, (struct ifreq *)&req, NULL, SIOCPNGAUTOCONF); + netdev_unlock_ops(dev); if (ret < 0) return ret; From patchwork Tue Feb 4 23:00:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 13960147 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0368021CA0D for ; Tue, 4 Feb 2025 23:01:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738710068; cv=none; b=PbXfcyKtLgl0d/v5nYK+MDMXIsf9NI60f7cwtWy3WcpDpLzCLIBW2kqAqXMcXC8iLvzNTwnkxoir2JiW995UYS8j0fYqBz6M4DMB/0VX2Dc+x2oyDgsPV6p5EsVkwfKGf2FhNzUB5m3odygiZIItLeKijysTRssfX6EWsHA8hsw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738710068; c=relaxed/simple; bh=pxdVJHsG5TtOmh2bYZk5tIk5Iyn/1I/7rQjtXEILnL0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YGT7ICNCSOzqvdmsAlyi6gEpqtQjTSMqd9Kia2e48N/tQh4zck+o4nxXeYDY3f8IABIQyU+ziNb1hGOhRNc1959bND20HaMsW09A+FEvbFjkWbxKJv+TOAIoTOzyG/7kno21DvYg7b+WAoInTdTG4iSLrkYq9SAHHrtJozpe11w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=fomichev.me; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.216.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=fomichev.me Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2f42992f608so8563800a91.0 for ; Tue, 04 Feb 2025 15:01:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738710064; x=1739314864; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xJTU5+TDbGyMnNZqTELfGW3+4rhP8OcNS2sTQLWkPv8=; b=bJi7C1Qkyl2JMJKLe9+FZzGyrVv50PhWt0N3YiDSDVZ3aQcP6ofvYqVcM8JaSV1ogQ FDp/DZPyw1KblGXeD9r+L0N90ctw3XR7ZDzBGnYEAm0jwgON7IMH8hElcTJEAub7P7aU gkWc7QTTkWCoYRC0K1HonktaJxQpWQUyhQ1icGAW2O2Ja0r4MmXEfZ+5k4HidKNdMV8I HyK2t4ffmIZ6YLMHhlU5JXulSZu0+WMCJdZu8U1BywR/L+sTOB7pge7HFph1IzC73mYE 9PUh2Q7gGDQ+eUGzVEB4EqYZEKE8KE/8E2d8Atqe/KWFHZ32wWfUYbolvnG47RoJVCK8 9Ynw== X-Gm-Message-State: AOJu0Yy6ZG20lyw9wl30FTEzzsFK6IjpXFUYpHIk0PJE51rRDCI3ISI4 ODI2ZVP6EuPH9kxgGv2rFMQBd+pbXwyz3gOofe8s0dV263UZTcOWS3kD X-Gm-Gg: ASbGncvYf1gGpiKhctJB2x39SxskGTlKpbp0I5Aq569TXZNlTdTQecOCEj5/fW+DgfX KuGPlX/I0tbFr2Scurz4HwCeRoTs7t0vZlIsLU4CN/XxvGrC31AZZ2mj0onctEJ8+/bqh2sSBKO wtjgQHx1WxoR8qdNu0DTTs/YDoBhKt3Q0XKN8ZeAJfzW/1U8TFC4xnraIv/lUywzymoKWMmxam5 mu/NqSvdjGFsgq6/s8VjEQ409abSDjkQgfGtf2hnY0tT3Vb8NvEWgmyVkLMZZxs9zygWxwkw4Iz mLkX25rh6nNnI+c= X-Google-Smtp-Source: AGHT+IEkF9OBWZJAFJueqBdExBP07IO57fbf+X/X1duuDeT86AKv7csyml6Xt21vjNjk3QeEf2xQZQ== X-Received: by 2002:a17:90b:3557:b0:2f4:434d:c7f0 with SMTP id 98e67ed59e1d1-2f9e076271fmr1034338a91.12.1738710063799; Tue, 04 Feb 2025 15:01:03 -0800 (PST) Received: from localhost ([2601:646:9e00:f56e:123b:cea3:439a:b3e3]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f9e1d60f72sm94231a91.3.2025.02.04.15.01.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2025 15:01:03 -0800 (PST) From: Stanislav Fomichev To: netdev@vger.kernel.org Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, Saeed Mahameed Subject: [RFC net-next 4/4] net: Hold netdev instance lock during queue operations Date: Tue, 4 Feb 2025 15:00:57 -0800 Message-ID: <20250204230057.1270362-5-sdf@fomichev.me> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250204230057.1270362-1-sdf@fomichev.me> References: <20250204230057.1270362-1-sdf@fomichev.me> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC For the drivers that use queue management API, switch to the mode where core stack holds the netdev instance lock. This affects the following drivers: - bnxt - gve - netdevsim Originally I locked only start/stop, but switched to holding the lock over all iterations to make them look atomic to the device (feels like it should be easier to reason about). Cc: Saeed Mahameed Signed-off-by: Stanislav Fomichev --- Documentation/networking/netdevices.rst | 30 ++++++++--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 54 +++++++++++++++---- .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 11 ++-- .../net/ethernet/broadcom/bnxt/bnxt_sriov.c | 2 + drivers/net/ethernet/google/gve/gve_main.c | 8 +-- drivers/net/ethernet/google/gve/gve_utils.c | 8 +-- drivers/net/netdevsim/netdev.c | 22 ++++---- include/linux/netdevice.h | 2 +- net/core/netdev_rx_queue.c | 5 ++ 9 files changed, 103 insertions(+), 39 deletions(-) diff --git a/Documentation/networking/netdevices.rst b/Documentation/networking/netdevices.rst index 3ed1bf322a5c..1ebd3bf011f3 100644 --- a/Documentation/networking/netdevices.rst +++ b/Documentation/networking/netdevices.rst @@ -211,18 +211,18 @@ struct net_device synchronization rules ======================================= ndo_open: Synchronization: rtnl_lock() semaphore. In addition, netdev instance - lock if the driver implements shaper API. + lock if the driver implements queue management or shaper API. Context: process ndo_stop: Synchronization: rtnl_lock() semaphore. In addition, netdev instance - lock if the driver implements shaper API. + lock if the driver implements queue management or shaper API. Context: process Note: netif_running() is guaranteed false ndo_do_ioctl: Synchronization: rtnl_lock() semaphore. In addition, netdev instance - lock if the driver implements shaper API. + lock if the driver implements queue management or shaper API. Context: process This is only called by network subsystems internally, @@ -231,7 +231,7 @@ struct net_device synchronization rules ndo_siocbond: Synchronization: rtnl_lock() semaphore. In addition, netdev instance - lock if the driver implements shaper API. + lock if the driver implements queue management or shaper API. Context: process Used by the bonding driver for the SIOCBOND family of @@ -239,7 +239,7 @@ struct net_device synchronization rules ndo_siocwandev: Synchronization: rtnl_lock() semaphore. In addition, netdev instance - lock if the driver implements shaper API. + lock if the driver implements queue management or shaper API. Context: process Used by the drivers/net/wan framework to handle @@ -247,7 +247,7 @@ struct net_device synchronization rules ndo_siocdevprivate: Synchronization: rtnl_lock() semaphore. In addition, netdev instance - lock if the driver implements shaper API. + lock if the driver implements queue management or shaper API. Context: process This is used to implement SIOCDEVPRIVATE ioctl helpers. @@ -255,7 +255,7 @@ struct net_device synchronization rules ndo_eth_ioctl: Synchronization: rtnl_lock() semaphore. In addition, netdev instance - lock if the driver implements shaper API. + lock if the driver implements queue management or shaper API. Context: process ndo_get_stats: @@ -265,7 +265,7 @@ struct net_device synchronization rules ndo_setup_tc: Synchronization: rtnl_lock() semaphore. In addition, netdev instance - lock if the driver implements shaper API. + lock if the driver implements queue management or shaper API. ndo_start_xmit: Synchronization: __netif_tx_lock spinlock. @@ -310,6 +310,20 @@ struct napi_struct synchronization rules softirq will be called with interrupts disabled by netconsole. +struct netdev_queue_mgmt_ops synchronization rules +================================================== +ndo_queue_mem_alloc: + Synchronization: Netdev instance lock. + +ndo_queue_mem_free: + Synchronization: Netdev instance lock. + +ndo_queue_start: + Synchronization: Netdev instance lock. + +ndo_queue_stop: + Synchronization: Netdev instance lock. + NETDEV_INTERNAL symbol namespace ================================ diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 7b8b5b39c7bb..a16dcccfb482 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -11301,7 +11301,7 @@ static int bnxt_request_irq(struct bnxt *bp) if (rc) break; - netif_napi_set_irq(&bp->bnapi[i]->napi, irq->vector); + netif_napi_set_irq_locked(&bp->bnapi[i]->napi, irq->vector); irq->requested = 1; if (zalloc_cpumask_var(&irq->cpu_mask, GFP_KERNEL)) { @@ -11337,9 +11337,9 @@ static void bnxt_del_napi(struct bnxt *bp) for (i = 0; i < bp->cp_nr_rings; i++) { struct bnxt_napi *bnapi = bp->bnapi[i]; - __netif_napi_del(&bnapi->napi); + __netif_napi_del_locked(&bnapi->napi); } - /* We called __netif_napi_del(), we need + /* We called __netif_napi_del_locked(), we need * to respect an RCU grace period before freeing napi structures. */ synchronize_net(); @@ -11352,18 +11352,20 @@ static void bnxt_init_napi(struct bnxt *bp) struct bnxt_napi *bnapi; int i; + netdev_assert_locked(bp->dev); + if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) poll_fn = bnxt_poll_p5; else if (BNXT_CHIP_TYPE_NITRO_A0(bp)) cp_nr_rings--; for (i = 0; i < cp_nr_rings; i++) { bnapi = bp->bnapi[i]; - netif_napi_add_config(bp->dev, &bnapi->napi, poll_fn, - bnapi->index); + netif_napi_add_config_locked(bp->dev, &bnapi->napi, poll_fn, + bnapi->index); } if (BNXT_CHIP_TYPE_NITRO_A0(bp)) { bnapi = bp->bnapi[cp_nr_rings]; - netif_napi_add(bp->dev, &bnapi->napi, bnxt_poll_nitroa0); + netif_napi_add_locked(bp->dev, &bnapi->napi, bnxt_poll_nitroa0); } } @@ -11375,6 +11377,8 @@ static void bnxt_disable_napi(struct bnxt *bp) test_and_set_bit(BNXT_STATE_NAPI_DISABLED, &bp->state)) return; + netdev_assert_locked(bp->dev); + for (i = 0; i < bp->cp_nr_rings; i++) { struct bnxt_napi *bnapi = bp->bnapi[i]; struct bnxt_cp_ring_info *cpr; @@ -11384,7 +11388,7 @@ static void bnxt_disable_napi(struct bnxt *bp) cpr->sw_stats->tx.tx_resets++; if (bnapi->in_reset) cpr->sw_stats->rx.rx_resets++; - napi_disable(&bnapi->napi); + napi_disable_locked(&bnapi->napi); } } @@ -11392,6 +11396,8 @@ static void bnxt_enable_napi(struct bnxt *bp) { int i; + netdev_assert_locked(bp->dev); + clear_bit(BNXT_STATE_NAPI_DISABLED, &bp->state); for (i = 0; i < bp->cp_nr_rings; i++) { struct bnxt_napi *bnapi = bp->bnapi[i]; @@ -11406,7 +11412,7 @@ static void bnxt_enable_napi(struct bnxt *bp) INIT_WORK(&cpr->dim.work, bnxt_dim_work); cpr->dim.mode = DIM_CQ_PERIOD_MODE_START_FROM_EQE; } - napi_enable(&bnapi->napi); + napi_enable_locked(&bnapi->napi); } } @@ -13291,11 +13297,17 @@ static netdev_features_t bnxt_fix_features(struct net_device *dev, static int bnxt_reinit_features(struct bnxt *bp, bool irq_re_init, bool link_re_init, u32 flags, bool update_tpa) { + int rc; + + netdev_lock(bp->dev); bnxt_close_nic(bp, irq_re_init, link_re_init); bp->flags = flags; if (update_tpa) bnxt_set_ring_params(bp); - return bnxt_open_nic(bp, irq_re_init, link_re_init); + rc = bnxt_open_nic(bp, irq_re_init, link_re_init); + netdev_unlock(bp->dev); + + return rc; } static int bnxt_set_features(struct net_device *dev, netdev_features_t features) @@ -13754,11 +13766,13 @@ static void bnxt_rtnl_lock_sp(struct bnxt *bp) */ clear_bit(BNXT_STATE_IN_SP_TASK, &bp->state); rtnl_lock(); + netdev_lock(bp->dev); } static void bnxt_rtnl_unlock_sp(struct bnxt *bp) { set_bit(BNXT_STATE_IN_SP_TASK, &bp->state); + netdev_unlock(bp->dev); rtnl_unlock(); } @@ -14622,8 +14636,10 @@ static void bnxt_fw_reset_task(struct work_struct *work) } bp->fw_reset_timestamp = jiffies; rtnl_lock(); + netdev_lock(bp->dev); if (test_bit(BNXT_STATE_ABORT_ERR, &bp->state)) { bnxt_fw_reset_abort(bp, rc); + netdev_unlock(bp->dev); rtnl_unlock(); goto ulp_start; } @@ -14635,6 +14651,7 @@ static void bnxt_fw_reset_task(struct work_struct *work) bp->fw_reset_state = BNXT_FW_RESET_STATE_ENABLE_DEV; tmo = bp->fw_reset_min_dsecs * HZ / 10; } + netdev_unlock(bp->dev); rtnl_unlock(); bnxt_queue_fw_reset_work(bp, tmo); return; @@ -14713,7 +14730,9 @@ static void bnxt_fw_reset_task(struct work_struct *work) bnxt_queue_fw_reset_work(bp, HZ / 10); return; } + netdev_lock(dev); rc = bnxt_open(bp->dev); + netdev_unlock(dev); if (rc) { netdev_err(bp->dev, "bnxt_open() failed during FW reset\n"); bnxt_fw_reset_abort(bp, rc); @@ -14868,10 +14887,12 @@ static int bnxt_change_mac_addr(struct net_device *dev, void *p) eth_hw_addr_set(dev, addr->sa_data); bnxt_clear_usr_fltrs(bp, true); + netdev_lock(dev); if (netif_running(dev)) { bnxt_close_nic(bp, false, false); rc = bnxt_open_nic(bp, false, false); } + netdev_unlock(dev); return rc; } @@ -14880,6 +14901,9 @@ static int bnxt_change_mac_addr(struct net_device *dev, void *p) static int bnxt_change_mtu(struct net_device *dev, int new_mtu) { struct bnxt *bp = netdev_priv(dev); + int rc = 0; + + netdev_lock(dev); if (netif_running(dev)) bnxt_close_nic(bp, true, false); @@ -14896,9 +14920,11 @@ static int bnxt_change_mtu(struct net_device *dev, int new_mtu) bnxt_set_ring_params(bp); if (netif_running(dev)) - return bnxt_open_nic(bp, true, false); + rc = bnxt_open_nic(bp, true, false); - return 0; + netdev_unlock(dev); + + return rc; } int bnxt_setup_mq_tc(struct net_device *dev, u8 tc) @@ -16426,6 +16452,7 @@ static int bnxt_suspend(struct device *device) bnxt_ulp_stop(bp); rtnl_lock(); + netdev_lock(dev); if (netif_running(dev)) { netif_device_detach(dev); rc = bnxt_close(dev); @@ -16434,6 +16461,7 @@ static int bnxt_suspend(struct device *device) bnxt_ptp_clear(bp); pci_disable_device(bp->pdev); bnxt_free_ctx_mem(bp, false); + netdev_unlock(dev); rtnl_unlock(); return rc; } @@ -16445,6 +16473,7 @@ static int bnxt_resume(struct device *device) int rc = 0; rtnl_lock(); + netdev_lock(dev); rc = pci_enable_device(bp->pdev); if (rc) { netdev_err(dev, "Cannot re-enable PCI device during resume, err = %d\n", @@ -16487,6 +16516,7 @@ static int bnxt_resume(struct device *device) } resume_exit: + netdev_unlock(dev); rtnl_unlock(); bnxt_ulp_start(bp, rc); if (!rc) @@ -16655,6 +16685,7 @@ static void bnxt_io_resume(struct pci_dev *pdev) netdev_info(bp->dev, "PCI Slot Resume\n"); rtnl_lock(); + netdev_lock(dev); err = bnxt_hwrm_func_qcaps(bp); if (!err) { @@ -16667,6 +16698,7 @@ static void bnxt_io_resume(struct pci_dev *pdev) if (!err) netif_device_attach(netdev); + netdev_unlock(dev); rtnl_unlock(); bnxt_ulp_start(bp, err); if (!err) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c index 9c5820839514..2246afcdcea2 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c @@ -4929,10 +4929,12 @@ static void bnxt_self_test(struct net_device *dev, struct ethtool_test *etest, return; } + netdev_lock(dev); + memset(buf, 0, sizeof(u64) * bp->num_tests); if (!netif_running(dev)) { etest->flags |= ETH_TEST_FL_FAILED; - return; + goto unlock; } if ((etest->flags & ETH_TEST_FL_EXTERNAL_LB) && @@ -4943,7 +4945,7 @@ static void bnxt_self_test(struct net_device *dev, struct ethtool_test *etest, if (bp->pf.active_vfs || !BNXT_SINGLE_PF(bp)) { etest->flags |= ETH_TEST_FL_FAILED; netdev_warn(dev, "Offline tests cannot be run with active VFs or on shared PF\n"); - return; + goto unlock; } offline = true; } @@ -4965,7 +4967,7 @@ static void bnxt_self_test(struct net_device *dev, struct ethtool_test *etest, rc = bnxt_half_open_nic(bp); if (rc) { etest->flags |= ETH_TEST_FL_FAILED; - return; + goto unlock; } buf[BNXT_MACLPBK_TEST_IDX] = 1; if (bp->mac_flags & BNXT_MAC_FL_NO_MAC_LPBK) @@ -5017,6 +5019,9 @@ static void bnxt_self_test(struct net_device *dev, struct ethtool_test *etest, etest->flags |= ETH_TEST_FL_FAILED; } } + +unlock: + netdev_unlock(dev); } static int bnxt_reset(struct net_device *dev, u32 *flags) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c index 12b6ed51fd88..dc61cf63fe3f 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c @@ -946,7 +946,9 @@ void bnxt_sriov_disable(struct bnxt *bp) /* Reclaim all resources for the PF. */ rtnl_lock(); + netdev_lock(bp->dev); bnxt_restore_pf_fw_resources(bp); + netdev_unlock(bp->dev); rtnl_unlock(); } diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index 533e659b15b3..cf9bd08d04b2 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -1886,7 +1886,7 @@ static void gve_turndown(struct gve_priv *priv) netif_queue_set_napi(priv->dev, idx, NETDEV_QUEUE_TYPE_TX, NULL); - napi_disable(&block->napi); + napi_disable_locked(&block->napi); } for (idx = 0; idx < priv->rx_cfg.num_queues; idx++) { int ntfy_idx = gve_rx_idx_to_ntfy(priv, idx); @@ -1897,7 +1897,7 @@ static void gve_turndown(struct gve_priv *priv) netif_queue_set_napi(priv->dev, idx, NETDEV_QUEUE_TYPE_RX, NULL); - napi_disable(&block->napi); + napi_disable_locked(&block->napi); } /* Stop tx queues */ @@ -1925,7 +1925,7 @@ static void gve_turnup(struct gve_priv *priv) if (!gve_tx_was_added_to_block(priv, idx)) continue; - napi_enable(&block->napi); + napi_enable_locked(&block->napi); if (idx < priv->tx_cfg.num_queues) netif_queue_set_napi(priv->dev, idx, @@ -1953,7 +1953,7 @@ static void gve_turnup(struct gve_priv *priv) if (!gve_rx_was_added_to_block(priv, idx)) continue; - napi_enable(&block->napi); + napi_enable_locked(&block->napi); netif_queue_set_napi(priv->dev, idx, NETDEV_QUEUE_TYPE_RX, &block->napi); diff --git a/drivers/net/ethernet/google/gve/gve_utils.c b/drivers/net/ethernet/google/gve/gve_utils.c index 30fef100257e..fa21d240806b 100644 --- a/drivers/net/ethernet/google/gve/gve_utils.c +++ b/drivers/net/ethernet/google/gve/gve_utils.c @@ -110,13 +110,15 @@ void gve_add_napi(struct gve_priv *priv, int ntfy_idx, { struct gve_notify_block *block = &priv->ntfy_blocks[ntfy_idx]; - netif_napi_add(priv->dev, &block->napi, gve_poll); - netif_napi_set_irq(&block->napi, block->irq); + netdev_assert_locked(priv->dev); + netif_napi_add_locked(priv->dev, &block->napi, gve_poll); + netif_napi_set_irq_locked(&block->napi, block->irq); } void gve_remove_napi(struct gve_priv *priv, int ntfy_idx) { struct gve_notify_block *block = &priv->ntfy_blocks[ntfy_idx]; - netif_napi_del(&block->napi); + netdev_assert_locked(priv->dev); + netif_napi_del_locked(&block->napi); } diff --git a/drivers/net/netdevsim/netdev.c b/drivers/net/netdevsim/netdev.c index efec03b34c9f..8faa5d22289c 100644 --- a/drivers/net/netdevsim/netdev.c +++ b/drivers/net/netdevsim/netdev.c @@ -661,7 +661,7 @@ nsim_queue_mem_alloc(struct net_device *dev, void *per_queue_mem, int idx) goto err_free; if (!ns->rq_reset_mode) - netif_napi_add_config(dev, &qmem->rq->napi, nsim_poll, idx); + netif_napi_add_config_locked(dev, &qmem->rq->napi, nsim_poll, idx); return 0; @@ -678,7 +678,7 @@ static void nsim_queue_mem_free(struct net_device *dev, void *per_queue_mem) page_pool_destroy(qmem->pp); if (qmem->rq) { if (!ns->rq_reset_mode) - netif_napi_del(&qmem->rq->napi); + netif_napi_del_locked(&qmem->rq->napi); page_pool_destroy(qmem->rq->page_pool); nsim_queue_free(qmem->rq); } @@ -690,9 +690,11 @@ nsim_queue_start(struct net_device *dev, void *per_queue_mem, int idx) struct nsim_queue_mem *qmem = per_queue_mem; struct netdevsim *ns = netdev_priv(dev); + netdev_assert_locked(dev); + if (ns->rq_reset_mode == 1) { ns->rq[idx]->page_pool = qmem->pp; - napi_enable(&ns->rq[idx]->napi); + napi_enable_locked(&ns->rq[idx]->napi); return 0; } @@ -700,15 +702,15 @@ nsim_queue_start(struct net_device *dev, void *per_queue_mem, int idx) * here we want to test various call orders. */ if (ns->rq_reset_mode == 2) { - netif_napi_del(&ns->rq[idx]->napi); - netif_napi_add_config(dev, &qmem->rq->napi, nsim_poll, idx); + netif_napi_del_locked(&ns->rq[idx]->napi); + netif_napi_add_config_locked(dev, &qmem->rq->napi, nsim_poll, idx); } else if (ns->rq_reset_mode == 3) { - netif_napi_add_config(dev, &qmem->rq->napi, nsim_poll, idx); - netif_napi_del(&ns->rq[idx]->napi); + netif_napi_add_config_locked(dev, &qmem->rq->napi, nsim_poll, idx); + netif_napi_del_locked(&ns->rq[idx]->napi); } ns->rq[idx] = qmem->rq; - napi_enable(&ns->rq[idx]->napi); + napi_enable_locked(&ns->rq[idx]->napi); return 0; } @@ -718,7 +720,9 @@ static int nsim_queue_stop(struct net_device *dev, void *per_queue_mem, int idx) struct nsim_queue_mem *qmem = per_queue_mem; struct netdevsim *ns = netdev_priv(dev); - napi_disable(&ns->rq[idx]->napi); + netdev_assert_locked(dev); + + napi_disable_locked(&ns->rq[idx]->napi); if (ns->rq_reset_mode == 1) { qmem->pp = ns->rq[idx]->page_pool; diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index e49b818054b9..cafa587233fd 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2719,7 +2719,7 @@ static inline void netdev_assert_locked_or_invisible(struct net_device *dev) static inline bool need_netdev_ops_lock(struct net_device *dev) { - bool ret = false; + bool ret = !!(dev)->queue_mgmt_ops; #if IS_ENABLED(CONFIG_NET_SHAPER) ret |= !!(dev)->netdev_ops->net_shaper_ops; diff --git a/net/core/netdev_rx_queue.c b/net/core/netdev_rx_queue.c index db82786fa0c4..04a681aef907 100644 --- a/net/core/netdev_rx_queue.c +++ b/net/core/netdev_rx_queue.c @@ -30,6 +30,8 @@ int netdev_rx_queue_restart(struct net_device *dev, unsigned int rxq_idx) goto err_free_new_mem; } + netdev_lock(dev); + err = dev->queue_mgmt_ops->ndo_queue_mem_alloc(dev, new_mem, rxq_idx); if (err) goto err_free_old_mem; @@ -48,6 +50,8 @@ int netdev_rx_queue_restart(struct net_device *dev, unsigned int rxq_idx) dev->queue_mgmt_ops->ndo_queue_mem_free(dev, old_mem); + netdev_unlock(dev); + kvfree(old_mem); kvfree(new_mem); @@ -72,6 +76,7 @@ int netdev_rx_queue_restart(struct net_device *dev, unsigned int rxq_idx) dev->queue_mgmt_ops->ndo_queue_mem_free(dev, new_mem); err_free_old_mem: + netdev_unlock(dev); kvfree(old_mem); err_free_new_mem: