From patchwork Tue Jan 14 03:51:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938382 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD519230985 for ; Tue, 14 Jan 2025 03:51:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826691; cv=none; b=nzOXKW38CmU/y6fuSJeNnfs5EoDyzyJ8hwrWG+4hTtBfXN3OStGDfRxcfoMQVQZaMkp5xDLuUpKMITBFRwkWANMmkCLPMF+sOxFAL5vfCflC1/vVETpzxLnv4X5Og9t9WwM94bT1vyRkQ7D+HCdTtsAl9CP3xlU/tOBPm9EXVAk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826691; c=relaxed/simple; bh=1cMjQBYAGQdst9IgSAnbXaq3v++n0uJW90xcc+ub/2I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XOHMLtd0EplxMO/o9iVbOeSROnyrKIQdpybg64jYiL7VaZOWOIm1h37qV9yjjJtEodeiKhM+47wj0JNiCLcqN2yjZBAYH0MWm11563/YsgZ/7IFH17iIU3o+om41JWD+rKKp9zcaqlKhAqgjuCOGE48zneQz16buh1oOmskFeEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=C6BieY7R; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="C6BieY7R" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1BB18C4CEE3; Tue, 14 Jan 2025 03:51:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826690; bh=1cMjQBYAGQdst9IgSAnbXaq3v++n0uJW90xcc+ub/2I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C6BieY7R5CCwcHsN0vGaV6m7b/0S4lmsXIAv5v07e/9umXEjMHMHb4LJ/RE8Xd+RB af8BukeD/lwQ+NRwp9IP7y+BdqzI3JDv1Ha/IxRlA/4rZ74vWbmyN6QXloz3pLPo3e qtSoVgxkweGzTHlWKny28EO0T2KOH8ivLgn/hfluFODqEVo2vxDs9RuOen6K8mJm5l i1gercupCjIQuxH/poNYWLmq0VQeX90EWbQAUS3rPyu0ObpJdqP0BUlUIH44Spl9Y+ ZZoQSOBY/4EUdJzzZIa/HC6nJH/1RpZTTEz21Cu2FX6MUOMaCX5OHfwRcibVRFnjHo UPnflrz0D/lKQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski , anthony.l.nguyen@intel.com, przemyslaw.kitszel@intel.com, jiri@resnulli.us Subject: [PATCH net-next 01/11] net: add netdev_lock() / netdev_unlock() helpers Date: Mon, 13 Jan 2025 19:51:07 -0800 Message-ID: <20250114035118.110297-2-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Add helpers for locking the netdev instance and use it in drivers and the shaper code. This will make grepping for the lock usage much easier, as we extend the lock to cover more fields. Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet Reviewed-by: Joe Damato --- CC: anthony.l.nguyen@intel.com CC: przemyslaw.kitszel@intel.com CC: jiri@resnulli.us --- include/linux/netdevice.h | 23 ++++++- drivers/net/ethernet/intel/iavf/iavf_main.c | 74 ++++++++++----------- drivers/net/netdevsim/ethtool.c | 4 +- net/shaper/shaper.c | 6 +- 4 files changed, 63 insertions(+), 44 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index dd8f6f8991fe..0e008ce9d5ee 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2444,8 +2444,12 @@ struct net_device { u32 napi_defer_hard_irqs; /** - * @lock: protects @net_shaper_hierarchy, feel free to use for other - * netdev-scope protection. Ordering: take after rtnl_lock. + * @lock: netdev-scope lock, protects a small selection of fields. + * Should always be taken using netdev_lock() / netdev_unlock() helpers. + * Drivers are free to use it for other protection. + * + * Protects: @net_shaper_hierarchy. + * Ordering: take after rtnl_lock. */ struct mutex lock; @@ -2671,6 +2675,21 @@ void netif_queue_set_napi(struct net_device *dev, unsigned int queue_index, enum netdev_queue_type type, struct napi_struct *napi); +static inline void netdev_lock(struct net_device *dev) +{ + mutex_lock(&dev->lock); +} + +static inline void netdev_unlock(struct net_device *dev) +{ + mutex_unlock(&dev->lock); +} + +static inline void netdev_assert_locked(struct net_device *dev) +{ + lockdep_assert_held(&dev->lock); +} + static inline void netif_napi_set_irq(struct napi_struct *napi, int irq) { napi->irq = irq; diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index 7740f446c73f..ab908d620285 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -1977,7 +1977,7 @@ static void iavf_finish_config(struct work_struct *work) * The dev->lock is needed to update the queue number */ rtnl_lock(); - mutex_lock(&adapter->netdev->lock); + netdev_lock(adapter->netdev); mutex_lock(&adapter->crit_lock); if ((adapter->flags & IAVF_FLAG_SETUP_NETDEV_FEATURES) && @@ -1997,7 +1997,7 @@ static void iavf_finish_config(struct work_struct *work) netif_set_real_num_tx_queues(adapter->netdev, pairs); if (adapter->netdev->reg_state != NETREG_REGISTERED) { - mutex_unlock(&adapter->netdev->lock); + netdev_unlock(adapter->netdev); netdev_released = true; err = register_netdevice(adapter->netdev); if (err) { @@ -2027,7 +2027,7 @@ static void iavf_finish_config(struct work_struct *work) out: mutex_unlock(&adapter->crit_lock); if (!netdev_released) - mutex_unlock(&adapter->netdev->lock); + netdev_unlock(adapter->netdev); rtnl_unlock(); } @@ -2724,10 +2724,10 @@ static void iavf_watchdog_task(struct work_struct *work) struct iavf_hw *hw = &adapter->hw; u32 reg_val; - mutex_lock(&netdev->lock); + netdev_lock(netdev); if (!mutex_trylock(&adapter->crit_lock)) { if (adapter->state == __IAVF_REMOVE) { - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return; } @@ -2741,35 +2741,35 @@ static void iavf_watchdog_task(struct work_struct *work) case __IAVF_STARTUP: iavf_startup(adapter); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); queue_delayed_work(adapter->wq, &adapter->watchdog_task, msecs_to_jiffies(30)); return; case __IAVF_INIT_VERSION_CHECK: iavf_init_version_check(adapter); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); queue_delayed_work(adapter->wq, &adapter->watchdog_task, msecs_to_jiffies(30)); return; case __IAVF_INIT_GET_RESOURCES: iavf_init_get_resources(adapter); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); queue_delayed_work(adapter->wq, &adapter->watchdog_task, msecs_to_jiffies(1)); return; case __IAVF_INIT_EXTENDED_CAPS: iavf_init_process_extended_caps(adapter); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); queue_delayed_work(adapter->wq, &adapter->watchdog_task, msecs_to_jiffies(1)); return; case __IAVF_INIT_CONFIG_ADAPTER: iavf_init_config_adapter(adapter); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); queue_delayed_work(adapter->wq, &adapter->watchdog_task, msecs_to_jiffies(1)); return; @@ -2781,7 +2781,7 @@ static void iavf_watchdog_task(struct work_struct *work) * as it can loop forever */ mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return; } if (++adapter->aq_wait_count > IAVF_AQ_MAX_ERR) { @@ -2790,7 +2790,7 @@ static void iavf_watchdog_task(struct work_struct *work) adapter->flags |= IAVF_FLAG_PF_COMMS_FAILED; iavf_shutdown_adminq(hw); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); queue_delayed_work(adapter->wq, &adapter->watchdog_task, (5 * HZ)); return; @@ -2798,7 +2798,7 @@ static void iavf_watchdog_task(struct work_struct *work) /* Try again from failed step*/ iavf_change_state(adapter, adapter->last_state); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); queue_delayed_work(adapter->wq, &adapter->watchdog_task, HZ); return; case __IAVF_COMM_FAILED: @@ -2811,7 +2811,7 @@ static void iavf_watchdog_task(struct work_struct *work) iavf_change_state(adapter, __IAVF_INIT_FAILED); adapter->flags &= ~IAVF_FLAG_PF_COMMS_FAILED; mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return; } reg_val = rd32(hw, IAVF_VFGEN_RSTAT) & @@ -2831,14 +2831,14 @@ static void iavf_watchdog_task(struct work_struct *work) adapter->aq_required = 0; adapter->current_op = VIRTCHNL_OP_UNKNOWN; mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); queue_delayed_work(adapter->wq, &adapter->watchdog_task, msecs_to_jiffies(10)); return; case __IAVF_RESETTING: mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); queue_delayed_work(adapter->wq, &adapter->watchdog_task, HZ * 2); return; @@ -2869,7 +2869,7 @@ static void iavf_watchdog_task(struct work_struct *work) case __IAVF_REMOVE: default: mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return; } @@ -2881,14 +2881,14 @@ static void iavf_watchdog_task(struct work_struct *work) dev_err(&adapter->pdev->dev, "Hardware reset detected\n"); iavf_schedule_reset(adapter, IAVF_FLAG_RESET_PENDING); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); queue_delayed_work(adapter->wq, &adapter->watchdog_task, HZ * 2); return; } mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); restart_watchdog: if (adapter->state >= __IAVF_DOWN) queue_work(adapter->wq, &adapter->adminq_task); @@ -3015,12 +3015,12 @@ static void iavf_reset_task(struct work_struct *work) /* When device is being removed it doesn't make sense to run the reset * task, just return in such a case. */ - mutex_lock(&netdev->lock); + netdev_lock(netdev); if (!mutex_trylock(&adapter->crit_lock)) { if (adapter->state != __IAVF_REMOVE) queue_work(adapter->wq, &adapter->reset_task); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return; } @@ -3068,7 +3068,7 @@ static void iavf_reset_task(struct work_struct *work) reg_val); iavf_disable_vf(adapter); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return; /* Do not attempt to reinit. It's dead, Jim. */ } @@ -3209,7 +3209,7 @@ static void iavf_reset_task(struct work_struct *work) wake_up(&adapter->reset_waitqueue); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return; reset_err: @@ -3220,7 +3220,7 @@ static void iavf_reset_task(struct work_struct *work) iavf_disable_vf(adapter); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); dev_err(&adapter->pdev->dev, "failed to allocate resources during reinit\n"); } @@ -3692,10 +3692,10 @@ static int __iavf_setup_tc(struct net_device *netdev, void *type_data) if (test_bit(__IAVF_IN_REMOVE_TASK, &adapter->crit_section)) return 0; - mutex_lock(&netdev->lock); + netdev_lock(netdev); netif_set_real_num_rx_queues(netdev, total_qps); netif_set_real_num_tx_queues(netdev, total_qps); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return ret; } @@ -4365,7 +4365,7 @@ static int iavf_open(struct net_device *netdev) return -EIO; } - mutex_lock(&netdev->lock); + netdev_lock(netdev); while (!mutex_trylock(&adapter->crit_lock)) { /* If we are in __IAVF_INIT_CONFIG_ADAPTER state the crit_lock * is already taken and iavf_open is called from an upper @@ -4373,7 +4373,7 @@ static int iavf_open(struct net_device *netdev) * We have to leave here to avoid dead lock. */ if (adapter->state == __IAVF_INIT_CONFIG_ADAPTER) { - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return -EBUSY; } @@ -4424,7 +4424,7 @@ static int iavf_open(struct net_device *netdev) iavf_irq_enable(adapter, true); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return 0; @@ -4437,7 +4437,7 @@ static int iavf_open(struct net_device *netdev) iavf_free_all_tx_resources(adapter); err_unlock: mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return err; } @@ -4459,12 +4459,12 @@ static int iavf_close(struct net_device *netdev) u64 aq_to_restore; int status; - mutex_lock(&netdev->lock); + netdev_lock(netdev); mutex_lock(&adapter->crit_lock); if (adapter->state <= __IAVF_DOWN_PENDING) { mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return 0; } @@ -4498,7 +4498,7 @@ static int iavf_close(struct net_device *netdev) iavf_free_traffic_irqs(adapter); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); /* We explicitly don't free resources here because the hardware is * still active and can DMA into memory. Resources are cleared in @@ -5375,7 +5375,7 @@ static int iavf_suspend(struct device *dev_d) netif_device_detach(netdev); - mutex_lock(&netdev->lock); + netdev_lock(netdev); mutex_lock(&adapter->crit_lock); if (netif_running(netdev)) { @@ -5387,7 +5387,7 @@ static int iavf_suspend(struct device *dev_d) iavf_reset_interrupt_capability(adapter); mutex_unlock(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); return 0; } @@ -5486,7 +5486,7 @@ static void iavf_remove(struct pci_dev *pdev) if (netdev->reg_state == NETREG_REGISTERED) unregister_netdev(netdev); - mutex_lock(&netdev->lock); + netdev_lock(netdev); mutex_lock(&adapter->crit_lock); dev_info(&adapter->pdev->dev, "Removing device\n"); iavf_change_state(adapter, __IAVF_REMOVE); @@ -5523,7 +5523,7 @@ static void iavf_remove(struct pci_dev *pdev) mutex_destroy(&hw->aq.asq_mutex); mutex_unlock(&adapter->crit_lock); mutex_destroy(&adapter->crit_lock); - mutex_unlock(&netdev->lock); + netdev_unlock(netdev); iounmap(hw->hw_addr); pci_release_regions(pdev); diff --git a/drivers/net/netdevsim/ethtool.c b/drivers/net/netdevsim/ethtool.c index 5fe1eaef99b5..3f44a11aec83 100644 --- a/drivers/net/netdevsim/ethtool.c +++ b/drivers/net/netdevsim/ethtool.c @@ -103,10 +103,10 @@ nsim_set_channels(struct net_device *dev, struct ethtool_channels *ch) struct netdevsim *ns = netdev_priv(dev); int err; - mutex_lock(&dev->lock); + netdev_lock(dev); err = netif_set_real_num_queues(dev, ch->combined_count, ch->combined_count); - mutex_unlock(&dev->lock); + netdev_unlock(dev); if (err) return err; diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c index 15463062fe7b..7101a48bce54 100644 --- a/net/shaper/shaper.c +++ b/net/shaper/shaper.c @@ -40,7 +40,7 @@ static void net_shaper_lock(struct net_shaper_binding *binding) { switch (binding->type) { case NET_SHAPER_BINDING_TYPE_NETDEV: - mutex_lock(&binding->netdev->lock); + netdev_lock(binding->netdev); break; } } @@ -49,7 +49,7 @@ static void net_shaper_unlock(struct net_shaper_binding *binding) { switch (binding->type) { case NET_SHAPER_BINDING_TYPE_NETDEV: - mutex_unlock(&binding->netdev->lock); + netdev_unlock(binding->netdev); break; } } @@ -1398,7 +1398,7 @@ void net_shaper_set_real_num_tx_queues(struct net_device *dev, /* Only drivers implementing shapers support ensure * the lock is acquired in advance. */ - lockdep_assert_held(&dev->lock); + netdev_assert_locked(dev); /* Take action only when decreasing the tx queue number. */ for (i = txq; i < dev->real_num_tx_queues; ++i) { From patchwork Tue Jan 14 03:51:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938383 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD585230994 for ; Tue, 14 Jan 2025 03:51:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826691; cv=none; b=cCoMzjG1UhDPJc0vA/HIP6v38dTaF95eOqEM0vAhUWr+F96oDn5uOTuwnrfhfZBObh6xUZb3sjpu+bbwQtsj2wbjdv1VJCpApq5GkP1TVMVWw96B2xaRGZiHERWC4MbpeTPK7c9oex9zDq+Ck7bEeGNryFR3GbJcbLfoGKzJRhE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826691; c=relaxed/simple; bh=bXObn60U9vvKO8uzSWg7MT8NGVC5GW0T/9pxSVX4fk8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eNDgE8Ot5i6EduGLNKGGb8YIqw6mcKUAFTevU/nxvyNsWUSh49Oks+m+8Ut7l9FkJRXWijaN74d3Q7uCwNE/bwxkX8DuSRJSza4VV3T5Okhm5LAtXvsfzn3OC4Q7A3N3U3Xxp03OPNLGGGslrAhdWg+XcSQitOYfU9OwDEzH1UU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fWpKu/9V; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fWpKu/9V" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0806BC4CEE8; Tue, 14 Jan 2025 03:51:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826691; bh=bXObn60U9vvKO8uzSWg7MT8NGVC5GW0T/9pxSVX4fk8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fWpKu/9VigTE0FeTq+XW4Vfkb4HhHxZKNPEaT6f+UeVCmciMhs/NxRG9zHbyBuPgb /gWpNOSjp30Rmel33eqKPk2xaMuHIu/2ZMO0yntCiRqXmM23AqVaurWyZTUsdQ2lam H5TYJDgFXRYTFrQvLHj/TDPKE7a5D4NFmsrsyKUQTFJn8ZrUS3MxEt07fiB53X1chZ ptIP3nk2NtKvFxS86ZJmyVToomJMS6wN6y/qD2jMfNi3IvOIlyE3EGOrOQubMf5h68 9rtNDK89cYXK8/Amy2l7m1bb0MrcVkq1W4vVNXEL35IMeZkTw/qV/Jdb6QMUDXZQhx aSA4PzlwFJGyA== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski Subject: [PATCH net-next 02/11] net: add helpers for lookup and walking netdevs under netdev_lock() Date: Mon, 13 Jan 2025 19:51:08 -0800 Message-ID: <20250114035118.110297-3-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Add helpers for accessing netdevs under netdev_lock(). There's some careful handling needed to find the device and lock it safely, without it getting unregistered, and without taking rtnl_lock (the latter being the whole point of the new locking, after all). Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet --- net/core/dev.h | 16 +++++++ net/core/dev.c | 110 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 126 insertions(+) diff --git a/net/core/dev.h b/net/core/dev.h index d8966847794c..25ae732c0775 100644 --- a/net/core/dev.h +++ b/net/core/dev.h @@ -2,6 +2,7 @@ #ifndef _NET_CORE_DEV_H #define _NET_CORE_DEV_H +#include #include #include #include @@ -23,8 +24,23 @@ struct sd_flow_limit { extern int netdev_flow_limit_table_len; struct napi_struct *netdev_napi_by_id(struct net *net, unsigned int napi_id); +struct napi_struct * +netdev_napi_by_id_lock(struct net *net, unsigned int napi_id); struct net_device *dev_get_by_napi_id(unsigned int napi_id); +struct net_device *netdev_get_by_index_lock(struct net *net, int ifindex); +struct net_device *__netdev_put_lock(struct net_device *dev); +struct net_device * +netdev_xa_find_lock(struct net *net, struct net_device *dev, + unsigned long *index); + +DEFINE_FREE(netdev_unlock, struct net_device *, if (_T) netdev_unlock(_T)); + +#define for_each_netdev_lock_scoped(net, var_name, ifindex) \ + for (struct net_device *var_name __free(netdev_unlock) = NULL; \ + (var_name = netdev_xa_find_lock(net, var_name, &ifindex)); \ + ifindex++) + #ifdef CONFIG_PROC_FS int __init dev_proc_init(void); #else diff --git a/net/core/dev.c b/net/core/dev.c index fda4e1039bf0..5c1e71afbe1c 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -783,6 +783,49 @@ struct napi_struct *netdev_napi_by_id(struct net *net, unsigned int napi_id) return napi; } +/** + * netdev_napi_by_id_lock() - find a device by NAPI ID and lock it + * @net: the applicable net namespace + * @napi_id: ID of a NAPI of a target device + * + * Find a NAPI instance with @napi_id. Lock its device. + * The device must be in %NETREG_REGISTERED state for lookup to succeed. + * netdev_unlock() must be called to release it. + * + * Return: pointer to NAPI, its device with lock held, NULL if not found. + */ +struct napi_struct * +netdev_napi_by_id_lock(struct net *net, unsigned int napi_id) +{ + struct napi_struct *napi; + struct net_device *dev; + + rcu_read_lock(); + napi = netdev_napi_by_id(net, napi_id); + if (!napi || napi->dev->reg_state != NETREG_REGISTERED) { + rcu_read_unlock(); + return NULL; + } + + dev = napi->dev; + dev_hold(dev); + rcu_read_unlock(); + + dev = __netdev_put_lock(dev); + if (!dev) + return NULL; + + rcu_read_lock(); + napi = netdev_napi_by_id(net, napi_id); + if (napi && napi->dev != dev) + napi = NULL; + rcu_read_unlock(); + + if (!napi) + netdev_unlock(dev); + return napi; +} + /** * __dev_get_by_name - find a device by its name * @net: the applicable net namespace @@ -971,6 +1014,73 @@ struct net_device *dev_get_by_napi_id(unsigned int napi_id) return napi ? napi->dev : NULL; } +/* Release the held reference on the net_device, and if the net_device + * is still registered try to lock the instance lock. If device is being + * unregistered NULL will be returned (but the reference has been released, + * either way!) + * + * This helper is intended for locking net_device after it has been looked up + * using a lockless lookup helper. Lock prevents the instance from going away. + */ +struct net_device *__netdev_put_lock(struct net_device *dev) +{ + netdev_lock(dev); + if (dev->reg_state > NETREG_REGISTERED) { + netdev_unlock(dev); + dev_put(dev); + return NULL; + } + dev_put(dev); + return dev; +} + +/** + * netdev_get_by_index_lock() - find a device by its ifindex + * @net: the applicable net namespace + * @ifindex: index of device + * + * Search for an interface by index. If a valid device + * with @ifindex is found it will be returned with netdev->lock held. + * netdev_unlock() must be called to release it. + * + * Return: pointer to a device with lock held, NULL if not found. + */ +struct net_device *netdev_get_by_index_lock(struct net *net, int ifindex) +{ + struct net_device *dev; + + dev = dev_get_by_index(net, ifindex); + if (!dev) + return NULL; + + return __netdev_put_lock(dev); +} + +struct net_device * +netdev_xa_find_lock(struct net *net, struct net_device *dev, + unsigned long *index) +{ + if (dev) + netdev_unlock(dev); + + do { + rcu_read_lock(); + dev = xa_find(&net->dev_by_index, index, ULONG_MAX, XA_PRESENT); + if (!dev) { + rcu_read_unlock(); + return NULL; + } + dev_hold(dev); + rcu_read_unlock(); + + dev = __netdev_put_lock(dev); + if (dev) + return dev; + + (*index)++; + } while (true); +} + static DEFINE_SEQLOCK(netdev_rename_lock); void netdev_copy_name(struct net_device *dev, char *name) From patchwork Tue Jan 14 03:51:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938384 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF0FE232398 for ; Tue, 14 Jan 2025 03:51:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826692; cv=none; b=W+EehN6zjki4wRbGFp5P4Zpup253rKcF0V6IswOXE1oNeEw6vTy0AAlioyEf+7xPrxjaSwLq4wl76cSNWBQIB+DGkIMOyyiu9bHOXEJSI+mJvhpktsgDH1EGxZfa0MzKEA7E2zFxosHAUzsyCWlaMWBioMh9c5ojMtpMECqIQ5Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826692; c=relaxed/simple; bh=5vpu9qlahrUJSP6WRQw6I7jtoO5NnZf0ODNBYcg1qy8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NMm3mQeT62NBnqFG2TyA0LFQxK/RMur1lrBzrfRL7HSRTMDg/KNFaYnfuYHrAO878o/lbPl8IyD/p/JV6YIIVuyvKJJyEPeh8KTR1Yp/GP1ue1Gig5VZT4dr1MEZL/N+IrXfCjESCdGnjtKY8Uho08iM2evXdpaVauWp4notIWk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BtQg1U2Q; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BtQg1U2Q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AF361C4CEDF; Tue, 14 Jan 2025 03:51:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826692; bh=5vpu9qlahrUJSP6WRQw6I7jtoO5NnZf0ODNBYcg1qy8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BtQg1U2Q0jl/0AUPDHFYu3wZK2izZACg6XRd9SUTKAbeGFTlkD+f/jOvTFRMXj+1h xbyIqQ95l2ESuoDuAlSEm6LdJPncz+SEtWAfpnnxRXp1musetS1JXZJtsP4HfGOSWK QN6u7ChWfAzs4dXjd87KviRknfue94LHRZ43upMcoNeDI5mWTKkOCems4wfk6Gqh7S HJnMPQPdVuofi4RL9HzkEza8Ofb8w7SGehIB/Axjqvlintyt8O5FarYhuicHmgkZDZ 27imBOn7MMQj33Ics9uY/O+mgG5vQoft6liwYjdEdkbO+BUfENQsBtAte15R5/ESKE 3kfaBrwyjk8PQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski Subject: [PATCH net-next 03/11] net: make netdev_lock() protect netdev->reg_state Date: Mon, 13 Jan 2025 19:51:09 -0800 Message-ID: <20250114035118.110297-4-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Protect writes to netdev->reg_state with netdev_lock(). From now on holding netdev_lock() is sufficient to prevent the net_device from getting unregistered, so code which wants to hold just a single netdev around no longer needs to hold rtnl_lock. We do not protect the NETREG_UNREGISTERED -> NETREG_RELEASED transition. We'd need to move mutex_destroy(netdev->lock) to .release, but the real reason is that trying to stop the unregistration process mid-way would be unsafe / crazy. Taking references on such devices is not safe, either. So the intended semantics are to lock REGISTERED devices. Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet Reviewed-by: Joe Damato --- include/linux/netdevice.h | 2 +- net/core/dev.c | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 0e008ce9d5ee..bdbc5849469c 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2448,7 +2448,7 @@ struct net_device { * Should always be taken using netdev_lock() / netdev_unlock() helpers. * Drivers are free to use it for other protection. * - * Protects: @net_shaper_hierarchy. + * Protects: @reg_state, @net_shaper_hierarchy. * Ordering: take after rtnl_lock. */ struct mutex lock; diff --git a/net/core/dev.c b/net/core/dev.c index 5c1e71afbe1c..2ded6eedb4cc 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -10778,7 +10778,9 @@ int register_netdevice(struct net_device *dev) ret = netdev_register_kobject(dev); + netdev_lock(dev); WRITE_ONCE(dev->reg_state, ret ? NETREG_UNREGISTERED : NETREG_REGISTERED); + netdev_unlock(dev); if (ret) goto err_uninit_notify; @@ -11052,7 +11054,9 @@ void netdev_run_todo(void) continue; } + netdev_lock(dev); WRITE_ONCE(dev->reg_state, NETREG_UNREGISTERED); + netdev_unlock(dev); linkwatch_sync_dev(dev); } @@ -11658,7 +11662,9 @@ void unregister_netdevice_many_notify(struct list_head *head, list_for_each_entry(dev, head, unreg_list) { /* And unlink it from device chain. */ unlist_netdevice(dev); + netdev_lock(dev); WRITE_ONCE(dev->reg_state, NETREG_UNREGISTERING); + netdev_unlock(dev); } flush_all_backlogs(); From patchwork Tue Jan 14 03:51:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938385 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 677CE23353C for ; Tue, 14 Jan 2025 03:51:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826693; cv=none; b=ApUtL1scGXJfNgXFce78eBqzOw4bKYeu9TsgEjPfVNslu89AwGIH60vWmKnyzdsQ8D7tw/UCDI2gzGUvHi0b80r77HHbweQHuzQI9QpTZcpp1aGo1QWpWrkQsE7nt6BIh8sX0swOTJpe+0T0PQOOSuIu+8wb9IimSL+e92COUXw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826693; c=relaxed/simple; bh=o9GEh7JWbKmRqO6sblRDoE/qu3xXdfDzYkpbWQC0itI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lYFLG03MbZiNCUiApEgimH2ebqR9Dt0FmCg/4ehB5+RVd4sf8e8DFJo752U6JW+YpHO3Flf5j8LgNClBm6vb5iWPYGvVOZuGNPgqJHsef2VqtncLAqRnvrhk9L7mj3OFz8x/5HRo2PJ4w1SktF1ce6HJdzG1KhjnZB0b6BtyNdQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uNmC0tSu; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uNmC0tSu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 66459C4CEE1; Tue, 14 Jan 2025 03:51:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826692; bh=o9GEh7JWbKmRqO6sblRDoE/qu3xXdfDzYkpbWQC0itI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uNmC0tSugOSFYtTJkdGCHTsj2CKV5Y11g+FjE7q31w6a2z1oB82/yu+T9AbQZaTKd h7Wp2OdH6jeYdKF4TMAaxZStUfi1To5R8MU1hnn/QBpNHF+9nQA1NeoUuLMwA/9gMO Ej0cLRAyoc8jvD4hFf1ze5CEfo1OGH4juOGt03/b4z1rvBNfM+YCd8Er5izD2PnEXF cLMfOlNdOi04tPZaXbnZIp4SpU8GdxKy/+uoFzgSwQn7sKIxVIcrVwIZ0gG78Mevz7 WqiOUalYpLGWgpy9aMoEyjlQNGemC5Jn/y3LMP0UWXzq+0MVrOtL1X2UHamOqWFzL2 6NTleg1v0mUhQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski Subject: [PATCH net-next 04/11] net: add netdev->up protected by netdev_lock() Date: Mon, 13 Jan 2025 19:51:10 -0800 Message-ID: <20250114035118.110297-5-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Some uAPI (netdev netlink) hide net_device's sub-objects while the interface is down to ensure uniform behavior across drivers. To remove the rtnl_lock dependency from those uAPIs we need a way to safely tell if the device is down or up. Add an indication of whether device is open or closed, protected by netdev->lock. The semantics are the same as IFF_UP, but taking netdev_lock around every write to ->flags would be a lot of code churn. We don't want to blanket the entire open / close path by netdev_lock, because it will prevent us from applying it to specific structures - core helpers won't be able to take that lock from any function called by the drivers on open/close paths. So the state of the flag is "pessimistic", as in it may report false negatives, but never false positives. Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet Reviewed-by: Joe Damato --- include/linux/netdevice.h | 13 ++++++++++++- net/core/dev.h | 12 ++++++++++++ net/core/dev.c | 4 ++-- 3 files changed, 26 insertions(+), 3 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index bdbc5849469c..565dfeb78774 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2443,12 +2443,23 @@ struct net_device { unsigned long gro_flush_timeout; u32 napi_defer_hard_irqs; + /** + * @up: copy of @state's IFF_UP, but safe to read with just @lock. + * May report false negatives while the device is being opened + * or closed (@lock does not protect .ndo_open, or .ndo_close). + */ + bool up; + /** * @lock: netdev-scope lock, protects a small selection of fields. * Should always be taken using netdev_lock() / netdev_unlock() helpers. * Drivers are free to use it for other protection. * - * Protects: @reg_state, @net_shaper_hierarchy. + * Protects: + * @net_shaper_hierarchy, @reg_state + * Partially protects (readers hold either @lock or rtnl_lock, + * writers must hold both for registered devices): + * @up * Ordering: take after rtnl_lock. */ struct mutex lock; diff --git a/net/core/dev.h b/net/core/dev.h index 25ae732c0775..ef37e2dd44f4 100644 --- a/net/core/dev.h +++ b/net/core/dev.h @@ -128,6 +128,18 @@ void __dev_notify_flags(struct net_device *dev, unsigned int old_flags, void unregister_netdevice_many_notify(struct list_head *head, u32 portid, const struct nlmsghdr *nlh); +static inline void netif_set_up(struct net_device *dev, bool value) +{ + if (value) + dev->flags |= IFF_UP; + else + dev->flags &= ~IFF_UP; + + netdev_lock(dev); + dev->up = value; + netdev_unlock(dev); +} + static inline void netif_set_gso_max_size(struct net_device *dev, unsigned int size) { diff --git a/net/core/dev.c b/net/core/dev.c index 2ded6eedb4cc..1a05ad60b89f 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -1618,7 +1618,7 @@ static int __dev_open(struct net_device *dev, struct netlink_ext_ack *extack) if (ret) clear_bit(__LINK_STATE_START, &dev->state); else { - dev->flags |= IFF_UP; + netif_set_up(dev, true); dev_set_rx_mode(dev); dev_activate(dev); add_device_randomness(dev->dev_addr, dev->addr_len); @@ -1697,7 +1697,7 @@ static void __dev_close_many(struct list_head *head) if (ops->ndo_stop) ops->ndo_stop(dev); - dev->flags &= ~IFF_UP; + netif_set_up(dev, false); netpoll_poll_enable(dev); } } From patchwork Tue Jan 14 03:51:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938386 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1E172343C1 for ; Tue, 14 Jan 2025 03:51:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826695; cv=none; b=jiMNDcl0X2Ka3SvClak8NhiV55L34rIH1RH3DApnSLPLM8PNsypWOEoWGnUz31HOuqfWlRosDaSjUVMhfa7FwV8WoTe9P0WnmkgpXwVVYv0mau8ppIhewmq567Zz9+EQLWIsJY4miD47UEjoYS0Hj6sFwX3zhKvbm+GOhSYfiZ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826695; c=relaxed/simple; bh=oixeFpHbXSKSe4sdtgCaYKBsu8BSYJszShdwIYht2+U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TbZ0MAquLMAEHbnPBnGRIvbfFD2sAY6x0u8sqhUDPj8sZi6ooTPvmHjrcZDJ4Ln8FgUnytnCig2BfqUVS81o3fwI/AZRnIvJNsaspWj1yaB/z3xkP8kiMNFZrE7DZFUJyLVLe3T5s8LSu89cOEFzGaGkk+hYkOn6mR+KK77/wkg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mIfGwVMX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mIfGwVMX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1E06FC4CEEA; Tue, 14 Jan 2025 03:51:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826693; bh=oixeFpHbXSKSe4sdtgCaYKBsu8BSYJszShdwIYht2+U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mIfGwVMXYxmpeKLsasPFJKxVJi+Vt+Z0NXOp6INqfjbaDRodWlplKcInfGCj+HWxi nNMX+nG9I1ONz8qQuWDdcZzINxWiLhfa6O/A1gEH8A2SsefNefauttarQ29BCh0F4j bcSD9ZsP6pAmXWF5qb0aK/Z1oDMbzng49d0ODL4FSxiRvDL2jv5Ll1SKPmkwjGrpJ7 S91J4X89xxuO7cvPeyZKi9RgkWkAAg/09SX748d1syplaVmwN2YVgXKAs9D0JLOMw8 XbywUXp6nBKbkCNdN9JyLReW0rRE96Yyb0HPHJUz8h1R2y/2HG1TY6UQOa36RR1TE4 thmLRqm9D0n2w== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski Subject: [PATCH net-next 05/11] net: protect netdev->napi_list with netdev_lock() Date: Mon, 13 Jan 2025 19:51:11 -0800 Message-ID: <20250114035118.110297-6-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Hold netdev->lock when NAPIs are getting added or removed. This will allow safe access to NAPI instances of a net_device without rtnl_lock. Create a family of helpers which assume the lock is already taken. Switch iavf to them, as it makes extensive use of netdev->lock, already. Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet Reviewed-by: Joe Damato --- include/linux/netdevice.h | 54 ++++++++++++++++++--- drivers/net/ethernet/intel/iavf/iavf_main.c | 6 +-- net/core/dev.c | 15 ++++-- 3 files changed, 60 insertions(+), 15 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 565dfeb78774..9d0585a656b3 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2456,7 +2456,7 @@ struct net_device { * Drivers are free to use it for other protection. * * Protects: - * @net_shaper_hierarchy, @reg_state + * @napi_list, @net_shaper_hierarchy, @reg_state * Partially protects (readers hold either @lock or rtnl_lock, * writers must hold both for registered devices): * @up @@ -2711,8 +2711,19 @@ static inline void netif_napi_set_irq(struct napi_struct *napi, int irq) */ #define NAPI_POLL_WEIGHT 64 -void netif_napi_add_weight(struct net_device *dev, struct napi_struct *napi, - int (*poll)(struct napi_struct *, int), int weight); +void netif_napi_add_weight_locked(struct net_device *dev, + struct napi_struct *napi, + int (*poll)(struct napi_struct *, int), + int weight); + +static inline void +netif_napi_add_weight(struct net_device *dev, struct napi_struct *napi, + int (*poll)(struct napi_struct *, int), int weight) +{ + netdev_lock(dev); + netif_napi_add_weight_locked(dev, napi, poll, weight); + netdev_unlock(dev); +} /** * netif_napi_add() - initialize a NAPI context @@ -2730,6 +2741,13 @@ netif_napi_add(struct net_device *dev, struct napi_struct *napi, netif_napi_add_weight(dev, napi, poll, NAPI_POLL_WEIGHT); } +static inline void +netif_napi_add_locked(struct net_device *dev, struct napi_struct *napi, + int (*poll)(struct napi_struct *, int)) +{ + netif_napi_add_weight_locked(dev, napi, poll, NAPI_POLL_WEIGHT); +} + static inline void netif_napi_add_tx_weight(struct net_device *dev, struct napi_struct *napi, @@ -2740,6 +2758,15 @@ netif_napi_add_tx_weight(struct net_device *dev, netif_napi_add_weight(dev, napi, poll, weight); } +static inline void +netif_napi_add_config_locked(struct net_device *dev, struct napi_struct *napi, + int (*poll)(struct napi_struct *, int), int index) +{ + napi->index = index; + napi->config = &dev->napi_config[index]; + netif_napi_add_weight_locked(dev, napi, poll, NAPI_POLL_WEIGHT); +} + /** * netif_napi_add_config - initialize a NAPI context with persistent config * @dev: network device @@ -2751,9 +2778,9 @@ static inline void netif_napi_add_config(struct net_device *dev, struct napi_struct *napi, int (*poll)(struct napi_struct *, int), int index) { - napi->index = index; - napi->config = &dev->napi_config[index]; - netif_napi_add_weight(dev, napi, poll, NAPI_POLL_WEIGHT); + netdev_lock(dev); + netif_napi_add_config_locked(dev, napi, poll, index); + netdev_unlock(dev); } /** @@ -2773,6 +2800,8 @@ static inline void netif_napi_add_tx(struct net_device *dev, netif_napi_add_tx_weight(dev, napi, poll, NAPI_POLL_WEIGHT); } +void __netif_napi_del_locked(struct napi_struct *napi); + /** * __netif_napi_del - remove a NAPI context * @napi: NAPI context @@ -2781,7 +2810,18 @@ static inline void netif_napi_add_tx(struct net_device *dev, * containing @napi. Drivers might want to call this helper to combine * all the needed RCU grace periods into a single one. */ -void __netif_napi_del(struct napi_struct *napi); +static inline void __netif_napi_del(struct napi_struct *napi) +{ + netdev_lock(napi->dev); + __netif_napi_del_locked(napi); + netdev_unlock(napi->dev); +} + +static inline void netif_napi_del_locked(struct napi_struct *napi) +{ + __netif_napi_del_locked(napi); + synchronize_net(); +} /** * netif_napi_del - remove a NAPI context diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index ab908d620285..2db97c5d9f9e 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -1800,8 +1800,8 @@ static int iavf_alloc_q_vectors(struct iavf_adapter *adapter) q_vector->v_idx = q_idx; q_vector->reg_idx = q_idx; cpumask_copy(&q_vector->affinity_mask, cpu_possible_mask); - netif_napi_add(adapter->netdev, &q_vector->napi, - iavf_napi_poll); + netif_napi_add_locked(adapter->netdev, &q_vector->napi, + iavf_napi_poll); } return 0; @@ -1827,7 +1827,7 @@ static void iavf_free_q_vectors(struct iavf_adapter *adapter) for (q_idx = 0; q_idx < num_q_vectors; q_idx++) { struct iavf_q_vector *q_vector = &adapter->q_vectors[q_idx]; - netif_napi_del(&q_vector->napi); + netif_napi_del_locked(&q_vector->napi); } kfree(adapter->q_vectors); adapter->q_vectors = NULL; diff --git a/net/core/dev.c b/net/core/dev.c index 1a05ad60b89f..6d9cb1b80a17 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6909,9 +6909,12 @@ netif_napi_dev_list_add(struct net_device *dev, struct napi_struct *napi) list_add_rcu(&napi->dev_list, higher); /* adds after higher */ } -void netif_napi_add_weight(struct net_device *dev, struct napi_struct *napi, - int (*poll)(struct napi_struct *, int), int weight) +void netif_napi_add_weight_locked(struct net_device *dev, + struct napi_struct *napi, + int (*poll)(struct napi_struct *, int), + int weight) { + netdev_assert_locked(dev); if (WARN_ON(test_and_set_bit(NAPI_STATE_LISTED, &napi->state))) return; @@ -6952,7 +6955,7 @@ void netif_napi_add_weight(struct net_device *dev, struct napi_struct *napi, dev->threaded = false; netif_napi_set_irq(napi, -1); } -EXPORT_SYMBOL(netif_napi_add_weight); +EXPORT_SYMBOL(netif_napi_add_weight_locked); void napi_disable(struct napi_struct *n) { @@ -7023,8 +7026,10 @@ static void flush_gro_hash(struct napi_struct *napi) } /* Must be called in process context */ -void __netif_napi_del(struct napi_struct *napi) +void __netif_napi_del_locked(struct napi_struct *napi) { + netdev_assert_locked(napi->dev); + if (!test_and_clear_bit(NAPI_STATE_LISTED, &napi->state)) return; @@ -7044,7 +7049,7 @@ void __netif_napi_del(struct napi_struct *napi) napi->thread = NULL; } } -EXPORT_SYMBOL(__netif_napi_del); +EXPORT_SYMBOL(__netif_napi_del_locked); static int __napi_poll(struct napi_struct *n, bool *repoll) { From patchwork Tue Jan 14 03:51:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938387 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1DAD2343BE for ; Tue, 14 Jan 2025 03:51:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826695; cv=none; b=WByk01HUXqGa+BCJMOXVK/Ut7ZDQ4mLPvEHI37ezJFlLWhaxQHIoeXNGoBph/B5MNwZYAXSfZDUu9yeBkZI8oSvJgRUCtY6oKnHxRj1epy9jfGIAN97UQcYUYrOqWQozhRcTdryMPLFFxSnc6+3suBAXVTGn9K7RV719Y+hggpU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826695; c=relaxed/simple; bh=C2wrHybW+WRl2YX7syXw7wdv7ffn6gkFVGH+HAeEkfU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eKIMj/y3V95PsYczVVCxqpWr+6EFujzcPKbBQFZt/nx0DGkU4pk40U0aVSxyy8IaEObsu7jWab8XDwCf0k7zJQ0D4w4TpD11dJHHKyKS2u2LguvTHuU8kxCZ0ej9/oivq02oqPxSKIIq0ECZ9/g+xE5P17WkjmNjMFunNYESNjI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EtUy4j2q; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EtUy4j2q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C32BCC4CEE1; Tue, 14 Jan 2025 03:51:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826694; bh=C2wrHybW+WRl2YX7syXw7wdv7ffn6gkFVGH+HAeEkfU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EtUy4j2q4Q4nmrDEQqKjOYyfmmB5ZtdQI4Tk4fIKWGxmGJYLTH2TlxmmJU7roj8qM ltQyjLJ6b3vGoyvXgFkxvoUOxyeOWscG5jseqrt9oLHXlfBvF2hdnnHYIQxJ7ADH82 JlLVYl4RnW50ditG7mIBIT9IHBtpnkxp0dUHf0yMc7p8ddndqRMsT1w/gpe+5fPe+l gw1+FhmqDDUS8CcmgU8Q9wETTf3p2iyiQ05Mc+2xnMgyy21JAW3/dmdwHA1lwkXD12 CUngCLKYlH2nMCI3utCds0wrRobxnXtznzH3w4qEFi4wrCwPSOHgQfnJccViWdKJU9 xdknUHUM5ixNA== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski , pcnet32@frontier.com, anthony.l.nguyen@intel.com, przemyslaw.kitszel@intel.com, marcin.s.wojtas@gmail.com, romieu@fr.zoreil.com Subject: [PATCH net-next 06/11] net: protect NAPI enablement with netdev_lock() Date: Mon, 13 Jan 2025 19:51:12 -0800 Message-ID: <20250114035118.110297-7-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Wrap napi_enable() / napi_disable() with netdev_lock(). Provide the "already locked" flavor of the API" iavf needs the usual adjustment. A number of drivers call napi_enable() under a spin lock, so they have to be modified to take netdev_lock() first, then spin lock then call napi_enable_locked(). Protecting napi_enable() implies that napi->napi_id is protected by netdev_lock(). Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet Acked-by: Francois Romieu --- CC: pcnet32@frontier.com CC: anthony.l.nguyen@intel.com CC: przemyslaw.kitszel@intel.com CC: marcin.s.wojtas@gmail.com CC: romieu@fr.zoreil.com --- include/linux/netdevice.h | 11 ++---- drivers/net/ethernet/amd/pcnet32.c | 11 +++++- drivers/net/ethernet/intel/iavf/iavf_main.c | 4 +- drivers/net/ethernet/marvell/mvneta.c | 5 ++- drivers/net/ethernet/via/via-velocity.c | 4 +- net/core/dev.c | 41 +++++++++++++++++---- 6 files changed, 55 insertions(+), 21 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 9d0585a656b3..d3108a12e562 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -382,7 +382,7 @@ struct napi_struct { struct sk_buff *skb; struct list_head rx_list; /* Pending GRO_NORMAL skbs */ int rx_count; /* length of rx_list */ - unsigned int napi_id; + unsigned int napi_id; /* protected by netdev_lock */ struct hrtimer timer; struct task_struct *thread; unsigned long gro_flush_timeout; @@ -570,16 +570,11 @@ static inline bool napi_complete(struct napi_struct *n) int dev_set_threaded(struct net_device *dev, bool threaded); -/** - * napi_disable - prevent NAPI from scheduling - * @n: NAPI context - * - * Stop NAPI from being scheduled on this context. - * Waits till any outstanding processing completes. - */ void napi_disable(struct napi_struct *n); +void napi_disable_locked(struct napi_struct *n); void napi_enable(struct napi_struct *n); +void napi_enable_locked(struct napi_struct *n); /** * napi_synchronize - wait until NAPI is not running diff --git a/drivers/net/ethernet/amd/pcnet32.c b/drivers/net/ethernet/amd/pcnet32.c index 72db9f9e7bee..c6bd803f5b0c 100644 --- a/drivers/net/ethernet/amd/pcnet32.c +++ b/drivers/net/ethernet/amd/pcnet32.c @@ -462,7 +462,7 @@ static void pcnet32_netif_start(struct net_device *dev) val = lp->a->read_csr(ioaddr, CSR3); val &= 0x00ff; lp->a->write_csr(ioaddr, CSR3, val); - napi_enable(&lp->napi); + napi_enable_locked(&lp->napi); } /* @@ -889,6 +889,7 @@ static int pcnet32_set_ringparam(struct net_device *dev, if (netif_running(dev)) pcnet32_netif_stop(dev); + netdev_lock(dev); spin_lock_irqsave(&lp->lock, flags); lp->a->write_csr(ioaddr, CSR0, CSR0_STOP); /* stop the chip */ @@ -920,6 +921,7 @@ static int pcnet32_set_ringparam(struct net_device *dev, } spin_unlock_irqrestore(&lp->lock, flags); + netdev_unlock(dev); netif_info(lp, drv, dev, "Ring Param Settings: RX: %d, TX: %d\n", lp->rx_ring_size, lp->tx_ring_size); @@ -985,6 +987,7 @@ static int pcnet32_loopback_test(struct net_device *dev, uint64_t * data1) if (netif_running(dev)) pcnet32_netif_stop(dev); + netdev_lock(dev); spin_lock_irqsave(&lp->lock, flags); lp->a->write_csr(ioaddr, CSR0, CSR0_STOP); /* stop the chip */ @@ -1122,6 +1125,7 @@ static int pcnet32_loopback_test(struct net_device *dev, uint64_t * data1) lp->a->write_bcr(ioaddr, 20, 4); /* return to 16bit mode */ } spin_unlock_irqrestore(&lp->lock, flags); + netdev_unlock(dev); return rc; } /* end pcnet32_loopback_test */ @@ -2101,6 +2105,7 @@ static int pcnet32_open(struct net_device *dev) return -EAGAIN; } + netdev_lock(dev); spin_lock_irqsave(&lp->lock, flags); /* Check for a valid station address */ if (!is_valid_ether_addr(dev->dev_addr)) { @@ -2266,7 +2271,7 @@ static int pcnet32_open(struct net_device *dev) goto err_free_ring; } - napi_enable(&lp->napi); + napi_enable_locked(&lp->napi); /* Re-initialize the PCNET32, and start it when done. */ lp->a->write_csr(ioaddr, 1, (lp->init_dma_addr & 0xffff)); @@ -2300,6 +2305,7 @@ static int pcnet32_open(struct net_device *dev) lp->a->read_csr(ioaddr, CSR0)); spin_unlock_irqrestore(&lp->lock, flags); + netdev_unlock(dev); return 0; /* Always succeed */ @@ -2315,6 +2321,7 @@ static int pcnet32_open(struct net_device *dev) err_free_irq: spin_unlock_irqrestore(&lp->lock, flags); + netdev_unlock(dev); free_irq(dev->irq, dev); return rc; } diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index 2db97c5d9f9e..cbfaaa5b7d02 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -1180,7 +1180,7 @@ static void iavf_napi_enable_all(struct iavf_adapter *adapter) q_vector = &adapter->q_vectors[q_idx]; napi = &q_vector->napi; - napi_enable(napi); + napi_enable_locked(napi); } } @@ -1196,7 +1196,7 @@ static void iavf_napi_disable_all(struct iavf_adapter *adapter) for (q_idx = 0; q_idx < q_vectors; q_idx++) { q_vector = &adapter->q_vectors[q_idx]; - napi_disable(&q_vector->napi); + napi_disable_locked(&q_vector->napi); } } diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c index fe6261b81540..cc97474852ef 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -4392,6 +4392,7 @@ static int mvneta_cpu_online(unsigned int cpu, struct hlist_node *node) if (pp->neta_armada3700) return 0; + netdev_lock(port->napi.dev); spin_lock(&pp->lock); /* * Configuring the driver for a new CPU while the driver is @@ -4418,7 +4419,7 @@ static int mvneta_cpu_online(unsigned int cpu, struct hlist_node *node) /* Mask all ethernet port interrupts */ on_each_cpu(mvneta_percpu_mask_interrupt, pp, true); - napi_enable(&port->napi); + napi_enable_locked(&port->napi); /* * Enable per-CPU interrupts on the CPU that is @@ -4439,6 +4440,8 @@ static int mvneta_cpu_online(unsigned int cpu, struct hlist_node *node) MVNETA_CAUSE_LINK_CHANGE); netif_tx_start_all_queues(pp->dev); spin_unlock(&pp->lock); + netdev_unlock(port->napi.dev); + return 0; } diff --git a/drivers/net/ethernet/via/via-velocity.c b/drivers/net/ethernet/via/via-velocity.c index dd4a07c97eee..892dc6ef3d3a 100644 --- a/drivers/net/ethernet/via/via-velocity.c +++ b/drivers/net/ethernet/via/via-velocity.c @@ -2322,6 +2322,7 @@ static int velocity_change_mtu(struct net_device *dev, int new_mtu) napi_disable(&vptr->napi); + netdev_lock(dev); spin_lock_irqsave(&vptr->lock, flags); netif_stop_queue(dev); @@ -2342,12 +2343,13 @@ static int velocity_change_mtu(struct net_device *dev, int new_mtu) velocity_give_many_rx_descs(vptr); - napi_enable(&vptr->napi); + napi_enable_locked(&vptr->napi); mac_enable_int(vptr->mac_regs); netif_start_queue(dev); spin_unlock_irqrestore(&vptr->lock, flags); + netdev_unlock(dev); velocity_free_rings(tmp_vptr); diff --git a/net/core/dev.c b/net/core/dev.c index 6d9cb1b80a17..cf3bf8a906b6 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6957,11 +6957,13 @@ void netif_napi_add_weight_locked(struct net_device *dev, } EXPORT_SYMBOL(netif_napi_add_weight_locked); -void napi_disable(struct napi_struct *n) +void napi_disable_locked(struct napi_struct *n) { unsigned long val, new; might_sleep(); + netdev_assert_locked(n->dev); + set_bit(NAPI_STATE_DISABLE, &n->state); val = READ_ONCE(n->state); @@ -6984,16 +6986,25 @@ void napi_disable(struct napi_struct *n) clear_bit(NAPI_STATE_DISABLE, &n->state); } -EXPORT_SYMBOL(napi_disable); +EXPORT_SYMBOL(napi_disable_locked); /** - * napi_enable - enable NAPI scheduling - * @n: NAPI context + * napi_disable() - prevent NAPI from scheduling + * @n: NAPI context * - * Resume NAPI from being scheduled on this context. - * Must be paired with napi_disable. + * Stop NAPI from being scheduled on this context. + * Waits till any outstanding processing completes. + * Takes netdev_lock() for associated net_device. */ -void napi_enable(struct napi_struct *n) +void napi_disable(struct napi_struct *n) +{ + netdev_lock(n->dev); + napi_disable_locked(n); + netdev_unlock(n->dev); +} +EXPORT_SYMBOL(napi_disable); + +void napi_enable_locked(struct napi_struct *n) { unsigned long new, val = READ_ONCE(n->state); @@ -7010,6 +7021,22 @@ void napi_enable(struct napi_struct *n) new |= NAPIF_STATE_THREADED; } while (!try_cmpxchg(&n->state, &val, new)); } +EXPORT_SYMBOL(napi_enable_locked); + +/** + * napi_enable() - enable NAPI scheduling + * @n: NAPI context + * + * Enable scheduling of a NAPI instance. + * Must be paired with napi_disable(). + * Takes netdev_lock() for associated net_device. + */ +void napi_enable(struct napi_struct *n) +{ + netdev_lock(n->dev); + napi_enable_locked(n); + netdev_unlock(n->dev); +} EXPORT_SYMBOL(napi_enable); static void flush_gro_hash(struct napi_struct *napi) From patchwork Tue Jan 14 03:51:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938388 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 649CA23497D for ; Tue, 14 Jan 2025 03:51:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826695; cv=none; b=um8dWNJSZlYs5gfnjC0pg/PdZkiiWbNTLY4ggeAtSXOSuCbqCxOko6bnEFJylfH3CshVswGHtKrRZ/JtfSI9cWNhQEl4HH1MSnPa1cXoZkfY3mc0cLwlMlPIFrzv97395codB9zOQvHhI5YpHtNqrSQoy3khTZNSIg1zhW8bVJI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826695; c=relaxed/simple; bh=FXymMtJWUWrK8RFL5bDXGpYpVIubJFFQf+RggbzRgnk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ax0HxMhVbac+moOEBTd2dge4rSxDAU1DhMiyPlf2IVcf+NemCLQZgzNw9FEc5AIce0k4ymEU3Ni3adjgunydp8RuVhGR0nsVfTd4QRmDHXz2E1XFFwhdbDxmDmhBSx7rA89odmBBEQjXsDBTmC6Hg6hToFIJVgsPJMwAcN9B3fo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NfNNcutY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NfNNcutY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B9226C4CEE4; Tue, 14 Jan 2025 03:51:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826695; bh=FXymMtJWUWrK8RFL5bDXGpYpVIubJFFQf+RggbzRgnk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NfNNcutYlyg2Sp72H1wEsHuB9gIYCSupNq9jfkmhhSfRGhWswRceGBoZ1V2wwIGHu JdL/UwZdVHloj56KTh2XouJliLVEP7DedEpLtT9swWDvgPhs0bdzHvVbKyYECw/NId AUFe5sFWvPvxadC5JnsztQvyYY6Tf73jCERdAW8vBhjmj2+Om0gSH6FkAKlyvdnQFi YebRo3/uBULtQKY9CDWXWXP12n3SPp+c3KYQgjW2COAC/kZH0zINELkBcvrxyompGn u1eSk8ZLNhinQNy0NYmyQpKU7k31Qll+88R0MEKdwlx/k77ZATpSh+lhco6xnTcGtU dAeK9npOcidSw== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski Subject: [PATCH net-next 07/11] net: make netdev netlink ops hold netdev_lock() Date: Mon, 13 Jan 2025 19:51:13 -0800 Message-ID: <20250114035118.110297-8-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org In prep for dropping rtnl_lock, start locking netdev->lock in netlink genl ops. We need to be using netdev->up instead of flags & IFF_UP. We can remove the RCU lock protection for the NAPI since NAPI list is protected by netdev->lock already. Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet --- net/core/dev.h | 1 - net/core/dev.c | 3 ++- net/core/netdev-genl.c | 46 +++++++++++++++++++++++------------------- 3 files changed, 27 insertions(+), 23 deletions(-) diff --git a/net/core/dev.h b/net/core/dev.h index ef37e2dd44f4..a5b166bbd169 100644 --- a/net/core/dev.h +++ b/net/core/dev.h @@ -23,7 +23,6 @@ struct sd_flow_limit { extern int netdev_flow_limit_table_len; -struct napi_struct *netdev_napi_by_id(struct net *net, unsigned int napi_id); struct napi_struct * netdev_napi_by_id_lock(struct net *net, unsigned int napi_id); struct net_device *dev_get_by_napi_id(unsigned int napi_id); diff --git a/net/core/dev.c b/net/core/dev.c index cf3bf8a906b6..1151baaedf4d 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -767,7 +767,8 @@ static struct napi_struct *napi_by_id(unsigned int napi_id) } /* must be called under rcu_read_lock(), as we dont take a reference */ -struct napi_struct *netdev_napi_by_id(struct net *net, unsigned int napi_id) +static struct napi_struct * +netdev_napi_by_id(struct net *net, unsigned int napi_id) { struct napi_struct *napi; diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c index c59619a2ec23..810a446ab62c 100644 --- a/net/core/netdev-genl.c +++ b/net/core/netdev-genl.c @@ -167,7 +167,7 @@ netdev_nl_napi_fill_one(struct sk_buff *rsp, struct napi_struct *napi, void *hdr; pid_t pid; - if (!(napi->dev->flags & IFF_UP)) + if (!napi->dev->up) return 0; hdr = genlmsg_iput(rsp, info); @@ -230,17 +230,16 @@ int netdev_nl_napi_get_doit(struct sk_buff *skb, struct genl_info *info) return -ENOMEM; rtnl_lock(); - rcu_read_lock(); - napi = netdev_napi_by_id(genl_info_net(info), napi_id); + napi = netdev_napi_by_id_lock(genl_info_net(info), napi_id); if (napi) { err = netdev_nl_napi_fill_one(rsp, napi, info); + netdev_unlock(napi->dev); } else { NL_SET_BAD_ATTR(info->extack, info->attrs[NETDEV_A_NAPI_ID]); err = -ENOENT; } - rcu_read_unlock(); rtnl_unlock(); if (err) { @@ -266,7 +265,7 @@ netdev_nl_napi_dump_one(struct net_device *netdev, struct sk_buff *rsp, unsigned int prev_id; int err = 0; - if (!(netdev->flags & IFF_UP)) + if (!netdev->up) return err; prev_id = UINT_MAX; @@ -303,13 +302,15 @@ int netdev_nl_napi_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb) rtnl_lock(); if (ifindex) { - netdev = __dev_get_by_index(net, ifindex); - if (netdev) + netdev = netdev_get_by_index_lock(net, ifindex); + if (netdev) { err = netdev_nl_napi_dump_one(netdev, skb, info, ctx); - else + netdev_unlock(netdev); + } else { err = -ENODEV; + } } else { - for_each_netdev_dump(net, netdev, ctx->ifindex) { + for_each_netdev_lock_scoped(net, netdev, ctx->ifindex) { err = netdev_nl_napi_dump_one(netdev, skb, info, ctx); if (err < 0) break; @@ -358,17 +359,16 @@ int netdev_nl_napi_set_doit(struct sk_buff *skb, struct genl_info *info) napi_id = nla_get_u32(info->attrs[NETDEV_A_NAPI_ID]); rtnl_lock(); - rcu_read_lock(); - napi = netdev_napi_by_id(genl_info_net(info), napi_id); + napi = netdev_napi_by_id_lock(genl_info_net(info), napi_id); if (napi) { err = netdev_nl_napi_set_config(napi, info); + netdev_unlock(napi->dev); } else { NL_SET_BAD_ATTR(info->extack, info->attrs[NETDEV_A_NAPI_ID]); err = -ENOENT; } - rcu_read_unlock(); rtnl_unlock(); return err; @@ -442,7 +442,7 @@ netdev_nl_queue_fill(struct sk_buff *rsp, struct net_device *netdev, u32 q_idx, { int err; - if (!(netdev->flags & IFF_UP)) + if (!netdev->up) return -ENOENT; err = netdev_nl_queue_validate(netdev, q_idx, q_type); @@ -474,11 +474,13 @@ int netdev_nl_queue_get_doit(struct sk_buff *skb, struct genl_info *info) rtnl_lock(); - netdev = __dev_get_by_index(genl_info_net(info), ifindex); - if (netdev) + netdev = netdev_get_by_index_lock(genl_info_net(info), ifindex); + if (netdev) { err = netdev_nl_queue_fill(rsp, netdev, q_id, q_type, info); - else + netdev_unlock(netdev); + } else { err = -ENODEV; + } rtnl_unlock(); @@ -499,7 +501,7 @@ netdev_nl_queue_dump_one(struct net_device *netdev, struct sk_buff *rsp, { int err = 0; - if (!(netdev->flags & IFF_UP)) + if (!netdev->up) return err; for (; ctx->rxq_idx < netdev->real_num_rx_queues; ctx->rxq_idx++) { @@ -532,13 +534,15 @@ int netdev_nl_queue_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb) rtnl_lock(); if (ifindex) { - netdev = __dev_get_by_index(net, ifindex); - if (netdev) + netdev = netdev_get_by_index_lock(net, ifindex); + if (netdev) { err = netdev_nl_queue_dump_one(netdev, skb, info, ctx); - else + netdev_unlock(netdev); + } else { err = -ENODEV; + } } else { - for_each_netdev_dump(net, netdev, ctx->ifindex) { + for_each_netdev_lock_scoped(net, netdev, ctx->ifindex) { err = netdev_nl_queue_dump_one(netdev, skb, info, ctx); if (err < 0) break; From patchwork Tue Jan 14 03:51:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938389 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68FE1237A42 for ; Tue, 14 Jan 2025 03:51:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826696; cv=none; b=FR05QoMSIExFAqxTU+Ckvwbmai3av+25N1W66K9ldEkf9WLznbcQeB5HGffu1Rx3cYdpHy7EbYfpkTicEoKxKgGP91DSOdSWE39wjg1Ra+KxBiXtofd1WjxjTDO5WeptuEZq9XPk2PHrljMqc2ESNzRE1jD9wnKUFWFh84xNWBc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826696; c=relaxed/simple; bh=/4XrkvlKOJPZxij//0PUUE4HevuFjRXGOD+YZA5nW9w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kSdld032S5X60IA5pzwSvyxUKfC/N+C99LzX5eKqZ4FE4gcr4cURwzKFGOQLzZvS7ffyF+fW6iNzHAMpzHoTlb2Xm7nYg+WgDHLKCs5ySdfuIMD7ZjcFdK9xvaQH4oHLL5kAdryf+70M5ByBXRVLve/KsSRoh7WklrOUBL0zf3U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CIcfVaCV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CIcfVaCV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 75E47C4CEE3; Tue, 14 Jan 2025 03:51:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826696; bh=/4XrkvlKOJPZxij//0PUUE4HevuFjRXGOD+YZA5nW9w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CIcfVaCVjmVH2DY16Vy1jubQ7lLAHXT9K73Lb1PeBGnWgKjdcFjjsfXcd62OesNO4 +99fR1bQLGQq78OzAzNBYM9P8MYNwPEhuxt2NsXNmS+MtaPnZxHW+Raxi8rZow3Zmy MRlzCkPigvQbIqnU5kizWPF5o9wnYYZ60qQsfw8IQxHUQGN5HfKqXGTi0iKlAGm1IH iMf57vVZQxT6zfb26nRr4VNZnJOTM9HsRvsmU8kn5L9oY5MmhWlW7KD4rY08fh44qN C0YscvvUlO9hgRgGEOgD7gKuyHSz588C048nv02Bl0gdvIztrJWgz0uuR4SR8BWLrp qzkHQCxc0sqiw== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski , leitao@debian.org Subject: [PATCH net-next 08/11] net: protect threaded status of NAPI with netdev_lock() Date: Mon, 13 Jan 2025 19:51:14 -0800 Message-ID: <20250114035118.110297-9-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Now that NAPI instances can't come and go without holding netdev->lock we can trivially switch from rtnl_lock() to netdev_lock() for setting netdev->threaded via sysfs. Note that since we do not lock netdev_lock around sysfs calls in the core we don't have to "trylock" like we do with rtnl_lock. Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet --- CC: leitao@debian.org --- include/linux/netdevice.h | 13 +++++++++++-- net/core/dev.c | 2 ++ net/core/net-sysfs.c | 32 +++++++++++++++++++++++++++++++- 3 files changed, 44 insertions(+), 3 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index d3108a12e562..75c30404657b 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -384,7 +384,7 @@ struct napi_struct { int rx_count; /* length of rx_list */ unsigned int napi_id; /* protected by netdev_lock */ struct hrtimer timer; - struct task_struct *thread; + struct task_struct *thread; /* protected by netdev_lock */ unsigned long gro_flush_timeout; unsigned long irq_suspend_timeout; u32 defer_hard_irqs; @@ -2451,10 +2451,12 @@ struct net_device { * Drivers are free to use it for other protection. * * Protects: - * @napi_list, @net_shaper_hierarchy, @reg_state + * @napi_list, @net_shaper_hierarchy, @reg_state, @threaded * Partially protects (readers hold either @lock or rtnl_lock, * writers must hold both for registered devices): * @up + * Also protects some fields in struct napi_struct. + * * Ordering: take after rtnl_lock. */ struct mutex lock; @@ -2696,6 +2698,13 @@ static inline void netdev_assert_locked(struct net_device *dev) lockdep_assert_held(&dev->lock); } +static inline void netdev_assert_locked_or_invisible(struct net_device *dev) +{ + if (dev->reg_state == NETREG_REGISTERED || + dev->reg_state == NETREG_UNREGISTERING) + netdev_assert_locked(dev); +} + static inline void netif_napi_set_irq(struct napi_struct *napi, int irq) { napi->irq = irq; diff --git a/net/core/dev.c b/net/core/dev.c index 1151baaedf4d..5872f0797cc3 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6784,6 +6784,8 @@ int dev_set_threaded(struct net_device *dev, bool threaded) struct napi_struct *napi; int err = 0; + netdev_assert_locked_or_invisible(dev); + if (dev->threaded == threaded) return 0; diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 2d9afc6e2161..5602a3c12e9a 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -108,6 +108,36 @@ static ssize_t netdev_store(struct device *dev, struct device_attribute *attr, return ret; } +/* Same as netdev_store() but takes netdev_lock() instead of rtnl_lock() */ +static ssize_t +netdev_lock_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t len, + int (*set)(struct net_device *, unsigned long)) +{ + struct net_device *netdev = to_net_dev(dev); + struct net *net = dev_net(netdev); + unsigned long new; + int ret; + + if (!ns_capable(net->user_ns, CAP_NET_ADMIN)) + return -EPERM; + + ret = kstrtoul(buf, 0, &new); + if (ret) + return ret; + + netdev_lock(netdev); + + if (dev_isalive(netdev)) { + ret = (*set)(netdev, new); + if (ret == 0) + ret = len; + } + netdev_unlock(netdev); + + return ret; +} + NETDEVICE_SHOW_RO(dev_id, fmt_hex); NETDEVICE_SHOW_RO(dev_port, fmt_dec); NETDEVICE_SHOW_RO(addr_assign_type, fmt_dec); @@ -638,7 +668,7 @@ static ssize_t threaded_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { - return netdev_store(dev, attr, buf, len, modify_napi_threaded); + return netdev_lock_store(dev, attr, buf, len, modify_napi_threaded); } static DEVICE_ATTR_RW(threaded); From patchwork Tue Jan 14 03:51:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938390 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4615A23A0E1 for ; Tue, 14 Jan 2025 03:51:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826697; cv=none; b=aniwzJ+9nfbnIJPf+HfBoZQhtNgpXM4/cD1Lw+T8vrylmYQofYb8uqcChosdPAC/OZzCwM2DtE/nq1Zo48EUwcGmGz2PXlxnPniYxQBY23Qgdk6Q1OjMLsVwCwFv5A48ASd9wWMmUB5Pmlb4XT8lXEW3Riz7S5APCBGzz1kSsSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826697; c=relaxed/simple; bh=T2hHdMnwlJjzPSQy/h1ZlyaDftN2OztTzYmg3F2yVRc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KOUc5Spsfl9L6WYpvsM6S9q1HKe3vo6SEmnoNooYx8CKhkN/DHeqLHa8Ct4r9YMph7SRnlsM6QF7o5FABeO2IgrdovwR3/QUgUt7lG3vNkC078/JTlT9E81Vm+HEpKxrqszmaId8EvAkCv+r1ani0hd+Q4xyLjKJIHSnnNYXtmU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rJvpqpFy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rJvpqpFy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 32B0EC4CEDF; Tue, 14 Jan 2025 03:51:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826696; bh=T2hHdMnwlJjzPSQy/h1ZlyaDftN2OztTzYmg3F2yVRc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rJvpqpFySSU/JiFGR2TOETOaBiW5snOxagGZ0+OiWl+Ln+AhhiYFeetzr1LLHVKC9 04kLiqDKQHPYaHJ/L0UAFsKoSkN5+TVLuFEvKMLlLNDwosL/1DP7CWLARqn9ft9A8Y Zq8HmyjPIBq8I9hgBUWroskOTP2EPFP4AgRqq+M4wkR3iU+3jYbUrJ/V+ThlbcISVr jqjZs3gDq4J5huEg1DnPfXUZCjeCO/tnI6J+ZSVHttjiWdcIohlIU2ph6X5y3dyecp IWSW+i+IyvZPuf+5I9jee53I55NL7JqMrX0i5qG4Tz+8kmN0g2MiTmHstRdEXspW31 FxtPalgLG+34Q== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski Subject: [PATCH net-next 09/11] net: protect napi->irq with netdev_lock() Date: Mon, 13 Jan 2025 19:51:15 -0800 Message-ID: <20250114035118.110297-10-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Take netdev_lock() in netif_napi_set_irq(). All NAPI "control fields" are now protected by that lock (most of the other ones are set during napi add/del). The napi_hash_node is fully protected by the hash spin lock, but close enough for the kdoc... Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet Reviewed-by: Joe Damato --- include/linux/netdevice.h | 10 +++++++++- net/core/dev.c | 2 +- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 75c30404657b..03eeeac7dbdf 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -388,6 +388,7 @@ struct napi_struct { unsigned long gro_flush_timeout; unsigned long irq_suspend_timeout; u32 defer_hard_irqs; + /* all fields past this point are write-protected by netdev_lock */ /* control-path-only fields follow */ struct list_head dev_list; struct hlist_node napi_hash_node; @@ -2705,11 +2706,18 @@ static inline void netdev_assert_locked_or_invisible(struct net_device *dev) netdev_assert_locked(dev); } -static inline void netif_napi_set_irq(struct napi_struct *napi, int irq) +static inline void netif_napi_set_irq_locked(struct napi_struct *napi, int irq) { napi->irq = irq; } +static inline void netif_napi_set_irq(struct napi_struct *napi, int irq) +{ + netdev_lock(napi->dev); + netif_napi_set_irq_locked(napi, irq); + netdev_unlock(napi->dev); +} + /* Default NAPI poll() weight * Device drivers are strongly advised to not use bigger value */ diff --git a/net/core/dev.c b/net/core/dev.c index 5872f0797cc3..df2a8b54a9f2 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6956,7 +6956,7 @@ void netif_napi_add_weight_locked(struct net_device *dev, */ if (dev->threaded && napi_kthread_create(napi)) dev->threaded = false; - netif_napi_set_irq(napi, -1); + netif_napi_set_irq_locked(napi, -1); } EXPORT_SYMBOL(netif_napi_add_weight_locked); From patchwork Tue Jan 14 03:51:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938391 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2A2523A0EC for ; Tue, 14 Jan 2025 03:51:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826698; cv=none; b=t2Z6T4YifVjE67gLztUbatJAqAQqIOr9aa3pjd1rgP/tnX4J1bGqvXgybJVTIUjazCXXax2bxtkHBJg3iNV9U02q79LlI7v29GrCgDeJtyS1DbckV8AuamL2AgGJ5qflkPcwRy7ApvYS/003MQ0Hn2JcMXO4zP9AuJTpzN/P0zQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826698; c=relaxed/simple; bh=9B6DbiO4ZSRe+NwJ9j1gzriRNizmstFv55TgpwU6CXA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pq2baniTANix0vxJl9xeGNWClM4qfsHNCDZv2EB3VRgqE2ezn+ZWF6+iq1miIznHQcfVSUpTayEpeW4t5YbdSK4pAT54ez1XrY2OlKfr+XrM6AsNsxCFV8gnnYj+delBU5IsaotFILzbbRCOTCZ3VV0jj9unsUWq+GSNtIg1O/Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Utr38S1L; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Utr38S1L" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E903FC4CEE7; Tue, 14 Jan 2025 03:51:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826697; bh=9B6DbiO4ZSRe+NwJ9j1gzriRNizmstFv55TgpwU6CXA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Utr38S1LY8tm1ApDni7odvNhE5aid9OKjZtgb5lQKNVjUE6MrsL28VABgw7qJI7fl QEBTVQMfLtEQwwqK6TnbNhus/JaUhuXECSEtnQhYuGC8/kDyJ62q9RDii38/mBDRAs 51069vHMAin/Krp8XqzuBL3guUAiLegDt0iBQLsCLh4/+HSAHtBkOE+9mOgHIad1Ya YmryQByNC7Of+YYfbhjYwT+3ypKXj+BjXNtnH9r5h1qtmfVyUEDd3xgHZgikb7us1n MpTY/9Uy+TLeN3Zs2Qn2RjyHvsVQHA5k3XXAPD5zmoZqbB8SPpE73/822F8P8Qg2uv UmdcVjK08HZXg== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski Subject: [PATCH net-next 10/11] net: protect NAPI config fields with netdev_lock() Date: Mon, 13 Jan 2025 19:51:16 -0800 Message-ID: <20250114035118.110297-11-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Protect the following members of netdev and napi by netdev_lock: - defer_hard_irqs, - gro_flush_timeout, - irq_suspend_timeout. The first two are written via sysfs (which this patch switches to new lock), and netdev genl which holds both netdev and rtnl locks. irq_suspend_timeout is only written by netdev genl. Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet Reviewed-by: Joe Damato --- include/linux/netdevice.h | 7 ++++--- net/core/net-sysfs.c | 5 +++-- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 03eeeac7dbdf..e16c32be0681 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -384,11 +384,11 @@ struct napi_struct { int rx_count; /* length of rx_list */ unsigned int napi_id; /* protected by netdev_lock */ struct hrtimer timer; - struct task_struct *thread; /* protected by netdev_lock */ + /* all fields past this point are write-protected by netdev_lock */ + struct task_struct *thread; unsigned long gro_flush_timeout; unsigned long irq_suspend_timeout; u32 defer_hard_irqs; - /* all fields past this point are write-protected by netdev_lock */ /* control-path-only fields follow */ struct list_head dev_list; struct hlist_node napi_hash_node; @@ -2452,7 +2452,8 @@ struct net_device { * Drivers are free to use it for other protection. * * Protects: - * @napi_list, @net_shaper_hierarchy, @reg_state, @threaded + * @gro_flush_timeout, @napi_defer_hard_irqs, @napi_list, + * @net_shaper_hierarchy, @reg_state, @threaded * Partially protects (readers hold either @lock or rtnl_lock, * writers must hold both for registered devices): * @up diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 5602a3c12e9a..173688663464 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -450,7 +450,7 @@ static ssize_t gro_flush_timeout_store(struct device *dev, if (!capable(CAP_NET_ADMIN)) return -EPERM; - return netdev_store(dev, attr, buf, len, change_gro_flush_timeout); + return netdev_lock_store(dev, attr, buf, len, change_gro_flush_timeout); } NETDEVICE_SHOW_RW(gro_flush_timeout, fmt_ulong); @@ -470,7 +470,8 @@ static ssize_t napi_defer_hard_irqs_store(struct device *dev, if (!capable(CAP_NET_ADMIN)) return -EPERM; - return netdev_store(dev, attr, buf, len, change_napi_defer_hard_irqs); + return netdev_lock_store(dev, attr, buf, len, + change_napi_defer_hard_irqs); } NETDEVICE_SHOW_RW(napi_defer_hard_irqs, fmt_uint); From patchwork Tue Jan 14 03:51:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13938392 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BDD323A102 for ; Tue, 14 Jan 2025 03:51:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826698; cv=none; b=tqvWBzltGRyk1IPPEuI73UIGT8E7gyjJTfnZrOqWz3e/qMAHAN0u4mVVTzUNJr/1dnOibnzjy2Q8NdWrnA10618zfYt6RcuN2AjidI2qGixqAzaFg0LCrsy9LlqqgiHMtm/CUA0GVNlWBpiSOAOU8Po13G33BUeBvS9x8EcTnlo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736826698; c=relaxed/simple; bh=btVdnTT5aSt7o7J4iNkD7SgWWI0lL/mUTDjzB3Vomk4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nwz6BrSPlbedAq+Am7LFWpnZWDKk5QhsQ2DBRd20wa1VooXqYhNugBXMBB+4UGtBHU0qOdtynQvu+TwcgVOd3vFgfSccgjL6kR+GTTheKXvgInzHYYA1vyrhB0OnroEN7f9cl6HvVz9mwNjIViDzKpLNoue3ma+4fuzA1Ty0Xmw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lKxBBben; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lKxBBben" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A93A4C4CEE4; Tue, 14 Jan 2025 03:51:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736826698; bh=btVdnTT5aSt7o7J4iNkD7SgWWI0lL/mUTDjzB3Vomk4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lKxBBbendXhtNHUCHhd80ljZ6slJrV/AnbWczRRh28w59G4E0S9ht7npiVJ9Os5r1 vPdAeMJ9sSJnRCXn41mEKTo2NHX3seMCuwL39vy/grkg8Spu6XTSKv2OaS3fWN1QW6 7QJIOEt/DVU9MtOKG+ayRGpOl/NN+O1EuHqLopxgui4YQHV1s8FOrvo4wTpr+C93wN 5whr5ziMuNoeqhZrgxqywQTokF0p14PKUPYQQlOWH5E/lzR6+eO5bR819JcIQUUKhr r4njPrBUFzPwS3PDAbWRhRPL4ro0KLBQDLxbInihtgBiyw3zsB2kr8hl6RArIKfJdJ bGK6Eso4hB4sQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org, jdamato@fastly.com, Jakub Kicinski Subject: [PATCH net-next 11/11] netdev-genl: remove rtnl_lock protection from NAPI ops Date: Mon, 13 Jan 2025 19:51:17 -0800 Message-ID: <20250114035118.110297-12-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250114035118.110297-1-kuba@kernel.org> References: <20250114035118.110297-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org NAPI lifetime, visibility and config are all fully under netdev_lock protection now. Signed-off-by: Jakub Kicinski Reviewed-by: Eric Dumazet Reviewed-by: Joe Damato --- net/core/netdev-genl.c | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c index 810a446ab62c..715f85c6b62e 100644 --- a/net/core/netdev-genl.c +++ b/net/core/netdev-genl.c @@ -229,8 +229,6 @@ int netdev_nl_napi_get_doit(struct sk_buff *skb, struct genl_info *info) if (!rsp) return -ENOMEM; - rtnl_lock(); - napi = netdev_napi_by_id_lock(genl_info_net(info), napi_id); if (napi) { err = netdev_nl_napi_fill_one(rsp, napi, info); @@ -240,8 +238,6 @@ int netdev_nl_napi_get_doit(struct sk_buff *skb, struct genl_info *info) err = -ENOENT; } - rtnl_unlock(); - if (err) { goto err_free_msg; } else if (!rsp->len) { @@ -300,7 +296,6 @@ int netdev_nl_napi_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb) if (info->attrs[NETDEV_A_NAPI_IFINDEX]) ifindex = nla_get_u32(info->attrs[NETDEV_A_NAPI_IFINDEX]); - rtnl_lock(); if (ifindex) { netdev = netdev_get_by_index_lock(net, ifindex); if (netdev) { @@ -317,7 +312,6 @@ int netdev_nl_napi_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb) ctx->napi_id = 0; } } - rtnl_unlock(); return err; } @@ -358,8 +352,6 @@ int netdev_nl_napi_set_doit(struct sk_buff *skb, struct genl_info *info) napi_id = nla_get_u32(info->attrs[NETDEV_A_NAPI_ID]); - rtnl_lock(); - napi = netdev_napi_by_id_lock(genl_info_net(info), napi_id); if (napi) { err = netdev_nl_napi_set_config(napi, info); @@ -369,8 +361,6 @@ int netdev_nl_napi_set_doit(struct sk_buff *skb, struct genl_info *info) err = -ENOENT; } - rtnl_unlock(); - return err; }