From patchwork Mon Feb 5 13:03:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wojciech Drewek X-Patchwork-Id: 13545561 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80F961CA9F for ; Mon, 5 Feb 2024 13:05:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707138360; cv=none; b=nUS4h4TjwDavQP4WCLj2FJflgLKgGldKJYLZrlVhy11DIkIDLa2vggatvoZCO5nkT0tnZr4bZ0/PvOCytnBGhFMFpKo6/gwOJb8ziUJyGYd8pG8D0NWLU4TZdbJsUlpCOAf1RwoFlTPXAB0vX9eX0+Q7H6XbohEu4uEs4Is+zLA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707138360; c=relaxed/simple; bh=fZJPZe5qe/7Wio8r76osPMYdSXcrSQOPifU2To16zD4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R6DTerA3Im8zVk8pX14oC4vUyLhnrhCDtVYPFz/lPrEI9ZlcKeDGhPTlqb/6yNH2FT2cd6HD7xryrnX46IwmKx8uPaxkNf2hSlPbm0jUvUKGm1zX5hTHmMIYQf8JX1vZUzDRu9j91OpDlEyo4fbCZ3KWZhG8dKcji5fcXiowS6Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CDTKreKb; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CDTKreKb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1707138358; x=1738674358; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fZJPZe5qe/7Wio8r76osPMYdSXcrSQOPifU2To16zD4=; b=CDTKreKbVxTAhlTaGpDe5fgkJkSh87rOhp4W5jo+cCmbIsrUMNjJXS5x 57cs5R3+ux6bxpyUzF9Ttpn3VC7bWGLW/e2vqeqX2+yzfbAHyxDw9GvJq OcU3nkcAVt7WUn+DVtBvv53nHHegZyJ7EYpqnTRJXa46GniWCthAHDe1W 2K7IFvNtxlkQci0YhLp2zwmZkL7bbbGSmvSpAQ2zB4JBjUwyRB4BUTwKw Dm4SnaPHkAoLaLJWs9onKDswj0D1k0KYYkbOcYummREZptQutzQpGvbU/ feUT2lxFb4Pyn7zM4kgFsp5KZWETtzRlIv98WkHjq6S8ndW7m4yMs3t1Y g==; X-IronPort-AV: E=McAfee;i="6600,9927,10974"; a="407542" X-IronPort-AV: E=Sophos;i="6.05,245,1701158400"; d="scan'208";a="407542" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2024 05:05:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10974"; a="933151692" X-IronPort-AV: E=Sophos;i="6.05,245,1701158400"; d="scan'208";a="933151692" Received: from irvmail002.ir.intel.com ([10.43.11.120]) by fmsmga001.fm.intel.com with ESMTP; 05 Feb 2024 05:05:53 -0800 Received: from rozewie.igk.intel.com (unknown [10.211.8.69]) by irvmail002.ir.intel.com (Postfix) with ESMTP id 8F5F327BB4; Mon, 5 Feb 2024 13:05:52 +0000 (GMT) From: Wojciech Drewek To: intel-wired-lan@lists.osuosl.org Cc: netdev@vger.kernel.org, jiri@resnulli.us, przemyslaw.kitszel@intel.com, vadim.fedorenko@linux.dev, paul.m.stillwell.jr@intel.com, bcreeley@amd.com Subject: [PATCH iwl-next v5 1/2] ice: Remove and readd netdev during devlink reload Date: Mon, 5 Feb 2024 14:03:56 +0100 Message-Id: <20240205130357.106665-2-wojciech.drewek@intel.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240205130357.106665-1-wojciech.drewek@intel.com> References: <20240205130357.106665-1-wojciech.drewek@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Recent changes to the devlink reload (commit 9b2348e2d6c9 ("devlink: warn about existing entities during reload-reinit")) force the drivers to destroy devlink ports during reinit. Adjust ice driver to this requirement, unregister netdvice, destroy devlink port. ice_init_eth() was removed and all the common code between probe and reload was moved to ice_load(). During devlink reload we can't take devl_lock (it's already taken) and in ice_probe() we have to lock it. Use devl_* variant of the API which does not acquire and release devl_lock. Guard ice_load() with devl_lock only in case of probe. Suggested-by: Jiri Pirko Reviewed-by: Przemek Kitszel Reviewed-by: Vadim Fedorenko Reviewed-by: Simon Horman Reviewed-by: Brett Creeley Signed-off-by: Wojciech Drewek Tested-by: Pucha Himasekhar Reddy (A Contingent worker at Intel) --- v2: empty init removed in ice_devlink_reinit_up v3: refactor locking pattern as Brett suggested v4: fix wrong function name in commit message v5: move debugfs changes to separate commit --- drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_devlink.c | 68 ++++++- drivers/net/ethernet/intel/ice/ice_main.c | 186 ++++++------------- 3 files changed, 125 insertions(+), 131 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index a4ba60e17d0b..118e84835720 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -983,6 +983,8 @@ void ice_service_task_schedule(struct ice_pf *pf); int ice_load(struct ice_pf *pf); void ice_unload(struct ice_pf *pf); void ice_adv_lnk_speed_maps_init(void); +int ice_init_dev(struct ice_pf *pf); +void ice_deinit_dev(struct ice_pf *pf); /** * ice_set_rdma_cap - enable RDMA support diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.c b/drivers/net/ethernet/intel/ice/ice_devlink.c index 97182bacafa3..cc717175178b 100644 --- a/drivers/net/ethernet/intel/ice/ice_devlink.c +++ b/drivers/net/ethernet/intel/ice/ice_devlink.c @@ -444,6 +444,20 @@ ice_devlink_reload_empr_start(struct ice_pf *pf, return 0; } +/** + * ice_devlink_reinit_down - unload given PF + * @pf: pointer to the PF struct + */ +static void ice_devlink_reinit_down(struct ice_pf *pf) +{ + /* No need to take devl_lock, it's already taken by devlink API */ + ice_unload(pf); + rtnl_lock(); + ice_vsi_decfg(ice_get_main_vsi(pf)); + rtnl_unlock(); + ice_deinit_dev(pf); +} + /** * ice_devlink_reload_down - prepare for reload * @devlink: pointer to the devlink instance to reload @@ -477,7 +491,7 @@ ice_devlink_reload_down(struct devlink *devlink, bool netns_change, "Remove all VFs before doing reinit\n"); return -EOPNOTSUPP; } - ice_unload(pf); + ice_devlink_reinit_down(pf); return 0; case DEVLINK_RELOAD_ACTION_FW_ACTIVATE: return ice_devlink_reload_empr_start(pf, extack); @@ -1269,6 +1283,45 @@ static int ice_devlink_set_parent(struct devlink_rate *devlink_rate, return status; } +/** + * ice_devlink_reinit_up - do reinit of the given PF + * @pf: pointer to the PF struct + */ +static int ice_devlink_reinit_up(struct ice_pf *pf) +{ + struct ice_vsi *vsi = ice_get_main_vsi(pf); + struct ice_vsi_cfg_params params; + int err; + + err = ice_init_dev(pf); + if (err) + return err; + + params = ice_vsi_to_params(vsi); + params.flags = ICE_VSI_FLAG_INIT; + + rtnl_lock(); + err = ice_vsi_cfg(vsi, ¶ms); + rtnl_unlock(); + if (err) + goto err_vsi_cfg; + + /* No need to take devl_lock, it's already taken by devlink API */ + err = ice_load(pf); + if (err) + goto err_load; + + return 0; + +err_load: + rtnl_lock(); + ice_vsi_decfg(vsi); + rtnl_unlock(); +err_vsi_cfg: + ice_deinit_dev(pf); + return err; +} + /** * ice_devlink_reload_up - do reload up after reinit * @devlink: pointer to the devlink instance reloading @@ -1289,7 +1342,7 @@ ice_devlink_reload_up(struct devlink *devlink, switch (action) { case DEVLINK_RELOAD_ACTION_DRIVER_REINIT: *actions_performed = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT); - return ice_load(pf); + return ice_devlink_reinit_up(pf); case DEVLINK_RELOAD_ACTION_FW_ACTIVATE: *actions_performed = BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE); return ice_devlink_reload_empr_finish(pf, extack); @@ -1695,6 +1748,7 @@ static const struct devlink_port_ops ice_devlink_port_ops = { * @pf: the PF to create a devlink port for * * Create and register a devlink_port for this PF. + * This function has to be called under devl_lock. * * Return: zero on success or an error code on failure. */ @@ -1707,6 +1761,8 @@ int ice_devlink_create_pf_port(struct ice_pf *pf) struct device *dev; int err; + devlink = priv_to_devlink(pf); + dev = ice_pf_to_dev(pf); devlink_port = &pf->devlink_port; @@ -1727,10 +1783,9 @@ int ice_devlink_create_pf_port(struct ice_pf *pf) ice_devlink_set_switch_id(pf, &attrs.switch_id); devlink_port_attrs_set(devlink_port, &attrs); - devlink = priv_to_devlink(pf); - err = devlink_port_register_with_ops(devlink, devlink_port, vsi->idx, - &ice_devlink_port_ops); + err = devl_port_register_with_ops(devlink, devlink_port, vsi->idx, + &ice_devlink_port_ops); if (err) { dev_err(dev, "Failed to create devlink port for PF %d, error %d\n", pf->hw.pf_id, err); @@ -1745,10 +1800,11 @@ int ice_devlink_create_pf_port(struct ice_pf *pf) * @pf: the PF to cleanup * * Unregisters the devlink_port structure associated with this PF. + * This function has to be called under devl_lock. */ void ice_devlink_destroy_pf_port(struct ice_pf *pf) { - devlink_port_unregister(&pf->devlink_port); + devl_port_unregister(&pf->devlink_port); } /** diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index 1fa3f40743f5..85a996ad2c1f 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -4574,90 +4574,6 @@ static void ice_decfg_netdev(struct ice_vsi *vsi) vsi->netdev = NULL; } -static int ice_start_eth(struct ice_vsi *vsi) -{ - int err; - - err = ice_init_mac_fltr(vsi->back); - if (err) - return err; - - err = ice_vsi_open(vsi); - if (err) - ice_fltr_remove_all(vsi); - - return err; -} - -static void ice_stop_eth(struct ice_vsi *vsi) -{ - ice_fltr_remove_all(vsi); - ice_vsi_close(vsi); -} - -static int ice_init_eth(struct ice_pf *pf) -{ - struct ice_vsi *vsi = ice_get_main_vsi(pf); - int err; - - if (!vsi) - return -EINVAL; - - /* init channel list */ - INIT_LIST_HEAD(&vsi->ch_list); - - err = ice_cfg_netdev(vsi); - if (err) - return err; - /* Setup DCB netlink interface */ - ice_dcbnl_setup(vsi); - - err = ice_init_mac_fltr(pf); - if (err) - goto err_init_mac_fltr; - - err = ice_devlink_create_pf_port(pf); - if (err) - goto err_devlink_create_pf_port; - - SET_NETDEV_DEVLINK_PORT(vsi->netdev, &pf->devlink_port); - - err = ice_register_netdev(vsi); - if (err) - goto err_register_netdev; - - err = ice_tc_indir_block_register(vsi); - if (err) - goto err_tc_indir_block_register; - - ice_napi_add(vsi); - - return 0; - -err_tc_indir_block_register: - ice_unregister_netdev(vsi); -err_register_netdev: - ice_devlink_destroy_pf_port(pf); -err_devlink_create_pf_port: -err_init_mac_fltr: - ice_decfg_netdev(vsi); - return err; -} - -static void ice_deinit_eth(struct ice_pf *pf) -{ - struct ice_vsi *vsi = ice_get_main_vsi(pf); - - if (!vsi) - return; - - ice_vsi_close(vsi); - ice_unregister_netdev(vsi); - ice_devlink_destroy_pf_port(pf); - ice_tc_indir_block_unregister(vsi); - ice_decfg_netdev(vsi); -} - /** * ice_wait_for_fw - wait for full FW readiness * @hw: pointer to the hardware structure @@ -4683,7 +4599,7 @@ static int ice_wait_for_fw(struct ice_hw *hw, u32 timeout) return -ETIMEDOUT; } -static int ice_init_dev(struct ice_pf *pf) +int ice_init_dev(struct ice_pf *pf) { struct device *dev = ice_pf_to_dev(pf); struct ice_hw *hw = &pf->hw; @@ -4776,7 +4692,7 @@ static int ice_init_dev(struct ice_pf *pf) return err; } -static void ice_deinit_dev(struct ice_pf *pf) +void ice_deinit_dev(struct ice_pf *pf) { ice_free_irq_msix_misc(pf); ice_deinit_pf(pf); @@ -5081,31 +4997,47 @@ static void ice_deinit(struct ice_pf *pf) /** * ice_load - load pf by init hw and starting VSI * @pf: pointer to the pf instance + * + * This function has to be called under devl_lock. */ int ice_load(struct ice_pf *pf) { - struct ice_vsi_cfg_params params = {}; struct ice_vsi *vsi; int err; - err = ice_init_dev(pf); + devl_assert_locked(priv_to_devlink(pf)); + + vsi = ice_get_main_vsi(pf); + + /* init channel list */ + INIT_LIST_HEAD(&vsi->ch_list); + + err = ice_cfg_netdev(vsi); if (err) return err; - vsi = ice_get_main_vsi(pf); + /* Setup DCB netlink interface */ + ice_dcbnl_setup(vsi); - params = ice_vsi_to_params(vsi); - params.flags = ICE_VSI_FLAG_INIT; + err = ice_init_mac_fltr(pf); + if (err) + goto err_init_mac_fltr; - rtnl_lock(); - err = ice_vsi_cfg(vsi, ¶ms); + err = ice_devlink_create_pf_port(pf); if (err) - goto err_vsi_cfg; + goto err_devlink_create_pf_port; - err = ice_start_eth(ice_get_main_vsi(pf)); + SET_NETDEV_DEVLINK_PORT(vsi->netdev, &pf->devlink_port); + + err = ice_register_netdev(vsi); + if (err) + goto err_register_netdev; + + err = ice_tc_indir_block_register(vsi); if (err) - goto err_start_eth; - rtnl_unlock(); + goto err_tc_indir_block_register; + + ice_napi_add(vsi); err = ice_init_rdma(pf); if (err) @@ -5119,29 +5051,35 @@ int ice_load(struct ice_pf *pf) return 0; err_init_rdma: - ice_vsi_close(ice_get_main_vsi(pf)); - rtnl_lock(); -err_start_eth: - ice_vsi_decfg(ice_get_main_vsi(pf)); -err_vsi_cfg: - rtnl_unlock(); - ice_deinit_dev(pf); + ice_tc_indir_block_unregister(vsi); +err_tc_indir_block_register: + ice_unregister_netdev(vsi); +err_register_netdev: + ice_devlink_destroy_pf_port(pf); +err_devlink_create_pf_port: +err_init_mac_fltr: + ice_decfg_netdev(vsi); return err; } /** * ice_unload - unload pf by stopping VSI and deinit hw * @pf: pointer to the pf instance + * + * This function has to be called under devl_lock. */ void ice_unload(struct ice_pf *pf) { + struct ice_vsi *vsi = ice_get_main_vsi(pf); + + devl_assert_locked(priv_to_devlink(pf)); + ice_deinit_features(pf); ice_deinit_rdma(pf); - rtnl_lock(); - ice_stop_eth(ice_get_main_vsi(pf)); - ice_vsi_decfg(ice_get_main_vsi(pf)); - rtnl_unlock(); - ice_deinit_dev(pf); + ice_tc_indir_block_unregister(vsi); + ice_unregister_netdev(vsi); + ice_devlink_destroy_pf_port(pf); + ice_decfg_netdev(vsi); } /** @@ -5239,27 +5177,23 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) if (err) goto err_init; - err = ice_init_eth(pf); + devl_lock(priv_to_devlink(pf)); + err = ice_load(pf); + devl_unlock(priv_to_devlink(pf)); if (err) - goto err_init_eth; - - err = ice_init_rdma(pf); - if (err) - goto err_init_rdma; + goto err_load; err = ice_init_devlink(pf); if (err) goto err_init_devlink; - ice_init_features(pf); - return 0; err_init_devlink: - ice_deinit_rdma(pf); -err_init_rdma: - ice_deinit_eth(pf); -err_init_eth: + devl_lock(priv_to_devlink(pf)); + ice_unload(pf); + devl_unlock(priv_to_devlink(pf)); +err_load: ice_deinit(pf); err_init: pci_disable_device(pdev); @@ -5357,12 +5291,14 @@ static void ice_remove(struct pci_dev *pdev) if (!ice_is_safe_mode(pf)) ice_remove_arfs(pf); - ice_deinit_features(pf); + ice_deinit_devlink(pf); - ice_deinit_rdma(pf); - ice_deinit_eth(pf); - ice_deinit(pf); + devl_lock(priv_to_devlink(pf)); + ice_unload(pf); + devl_unlock(priv_to_devlink(pf)); + + ice_deinit(pf); ice_vsi_release_all(pf); ice_setup_mc_magic_wake(pf);