From patchwork Tue Aug 10 13:37:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 12428971 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C716C4320A for ; Tue, 10 Aug 2021 13:37:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3BF0D61052 for ; Tue, 10 Aug 2021 13:37:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241638AbhHJNiO (ORCPT ); Tue, 10 Aug 2021 09:38:14 -0400 Received: from mail.kernel.org ([198.145.29.99]:48672 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241584AbhHJNiG (ORCPT ); Tue, 10 Aug 2021 09:38:06 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id B113360FDA; Tue, 10 Aug 2021 13:37:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1628602664; bh=M7YYC86V3snE++Hi6R8B+ecf16khmRjxEgwHZa+z11c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=k3kgXJCoVnKNB9I7tS2aPTM3rtoAWdciNOGImREzupykz2gaoIyIibSPySBOc4o58 V+VHZbGINFzGVO2mpOkOq5X0WeTyOF1/rDIFVBEL50lmuaSiJBtWrKS1X99hA/Q4zY auXe7P0mt9RnNrhf1rnbvh8RqzdCKkeax4OGrvnpBZUqHYfjWedLgmtPf1rzyKxq/8 PSMANImC+0lmao5bMa9XF1t7PSP06wfYn53xNRJ+kotlFb4CkA2339WvAAGi7WlVdU cqxgOuwXtAjAs3tSnYca3MV1WgK5xMdreEkxwtza8mO0vQ2lEERAcY7wqmLbJwXCPM DDGHA3NJxRP0A== From: Leon Romanovsky To: "David S . Miller" , Jakub Kicinski Cc: Leon Romanovsky , Guangbin Huang , Ido Schimmel , Jiri Pirko , linux-kernel@vger.kernel.org, Michael Guralnik , netdev@vger.kernel.org, Saeed Mahameed , Salil Mehta , Tariq Toukan , Yisen Zhuang , Yufeng Mo Subject: [PATCH net-next 1/5] net: hns3: remove always exist devlink pointer check Date: Tue, 10 Aug 2021 16:37:31 +0300 Message-Id: <2340cee7b177bf7c8825f61b9798034c77baec32.1628599239.git.leonro@nvidia.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Leon Romanovsky The devlink pointer always exists after hclge_devlink_init() succeed. Remove that check together with NULL setting after release and ensure that devlink_register is last command prior to call to devlink_reload_enable(). Fixes: b741269b2759 ("net: hns3: add support for registering devlink for PF") Signed-off-by: Leon Romanovsky --- .../net/ethernet/hisilicon/hns3/hns3pf/hclge_devlink.c | 8 +------- .../net/ethernet/hisilicon/hns3/hns3vf/hclgevf_devlink.c | 8 +------- 2 files changed, 2 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_devlink.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_devlink.c index 448f29aa4e6b..e4aad695abcc 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_devlink.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_devlink.c @@ -118,6 +118,7 @@ int hclge_devlink_init(struct hclge_dev *hdev) priv = devlink_priv(devlink); priv->hdev = hdev; + hdev->devlink = devlink; ret = devlink_register(devlink); if (ret) { @@ -126,8 +127,6 @@ int hclge_devlink_init(struct hclge_dev *hdev) goto out_reg_fail; } - hdev->devlink = devlink; - devlink_reload_enable(devlink); return 0; @@ -141,14 +140,9 @@ void hclge_devlink_uninit(struct hclge_dev *hdev) { struct devlink *devlink = hdev->devlink; - if (!devlink) - return; - devlink_reload_disable(devlink); devlink_unregister(devlink); devlink_free(devlink); - - hdev->devlink = NULL; } diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_devlink.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_devlink.c index 1e6061fb8ed4..f478770299c6 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_devlink.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_devlink.c @@ -120,6 +120,7 @@ int hclgevf_devlink_init(struct hclgevf_dev *hdev) priv = devlink_priv(devlink); priv->hdev = hdev; + hdev->devlink = devlink; ret = devlink_register(devlink); if (ret) { @@ -128,8 +129,6 @@ int hclgevf_devlink_init(struct hclgevf_dev *hdev) goto out_reg_fail; } - hdev->devlink = devlink; - devlink_reload_enable(devlink); return 0; @@ -143,14 +142,9 @@ void hclgevf_devlink_uninit(struct hclgevf_dev *hdev) { struct devlink *devlink = hdev->devlink; - if (!devlink) - return; - devlink_reload_disable(devlink); devlink_unregister(devlink); devlink_free(devlink); - - hdev->devlink = NULL; } From patchwork Tue Aug 10 13:37:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 12428973 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05C17C4320A for ; Tue, 10 Aug 2021 13:38:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DE13360F41 for ; Tue, 10 Aug 2021 13:38:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237711AbhHJNia (ORCPT ); Tue, 10 Aug 2021 09:38:30 -0400 Received: from mail.kernel.org ([198.145.29.99]:48724 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241610AbhHJNiJ (ORCPT ); Tue, 10 Aug 2021 09:38:09 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1521360EE9; Tue, 10 Aug 2021 13:37:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1628602667; bh=3F30XzjiCuima9za1pGO/IygpzGxwpDwxQ6IcD1Gfqg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d54uKdcqBwcVwNJbDlLrnz2TlsCXGwlG8wR4ZEm2SsHEPFrNjJrAV85HQO1ZXwQL0 OQe1UZ9LMq4XIBhyot87XoiSlofZayKJIBbt2ZczO97lIJiO7/Gp1NGRFO0DfpVMIH QSTt0jQKq0E+HdZAlGIHX6uyv2ebM2G2UJijsgCq77edwgtKEsCs93uin6hna/HSQC gS3c60cHuxKepUazPDccTAtfhFyHhE6RGDzbDq/GZvynZ2irVLJob/QMp6DQej3k7V PlhuWK0P1e5e3tC8PnbaWMK4jeIOak50QhFi405pn5NXy4Kr0EWA3GzNbxZlcAUGnY T1wpFedrQ4eoQ== From: Leon Romanovsky To: "David S . Miller" , Jakub Kicinski Cc: Leon Romanovsky , Guangbin Huang , Ido Schimmel , Jiri Pirko , linux-kernel@vger.kernel.org, Michael Guralnik , netdev@vger.kernel.org, Saeed Mahameed , Salil Mehta , Tariq Toukan , Yisen Zhuang , Yufeng Mo Subject: [PATCH net-next 2/5] net/mlx4: Move devlink_register to be the last initialization command Date: Tue, 10 Aug 2021 16:37:32 +0300 Message-Id: <9fe3b30cc86f80881e4a1fa161dc58d4a6a2a9f1.1628599239.git.leonro@nvidia.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Leon Romanovsky Refactor the code to make sure that devlink_register() is the last command during initialization stage. Signed-off-by: Leon Romanovsky --- drivers/net/ethernet/mellanox/mlx4/main.c | 38 ++++++++++++++++------- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c index 7267c6c6d2e2..7005c32195a3 100644 --- a/drivers/net/ethernet/mellanox/mlx4/main.c +++ b/drivers/net/ethernet/mellanox/mlx4/main.c @@ -3996,6 +3996,8 @@ static const struct devlink_ops mlx4_devlink_ops = { .reload_up = mlx4_devlink_reload_up, }; +static void _mlx4_remove_one(struct pci_dev *pdev); + static int mlx4_init_one(struct pci_dev *pdev, const struct pci_device_id *id) { struct devlink *devlink; @@ -4024,28 +4026,29 @@ static int mlx4_init_one(struct pci_dev *pdev, const struct pci_device_id *id) mutex_init(&dev->persist->interface_state_mutex); mutex_init(&dev->persist->pci_status_mutex); - ret = devlink_register(devlink); - if (ret) - goto err_persist_free; ret = devlink_params_register(devlink, mlx4_devlink_params, ARRAY_SIZE(mlx4_devlink_params)); if (ret) - goto err_devlink_unregister; + goto err_persist_free; mlx4_devlink_set_params_init_values(devlink); ret = __mlx4_init_one(pdev, id->driver_data, priv); if (ret) goto err_params_unregister; devlink_params_publish(devlink); - devlink_reload_enable(devlink); pci_save_state(pdev); + + ret = devlink_register(devlink); + if (ret) { + _mlx4_remove_one(pdev); + return ret; + } + devlink_reload_enable(devlink); return 0; err_params_unregister: devlink_params_unregister(devlink, mlx4_devlink_params, ARRAY_SIZE(mlx4_devlink_params)); -err_devlink_unregister: - devlink_unregister(devlink); err_persist_free: kfree(dev->persist); err_devlink_free: @@ -4141,7 +4144,7 @@ static void mlx4_unload_one(struct pci_dev *pdev) priv->removed = 1; } -static void mlx4_remove_one(struct pci_dev *pdev) +static void _mlx4_remove_one(struct pci_dev *pdev) { struct mlx4_dev_persistent *persist = pci_get_drvdata(pdev); struct mlx4_dev *dev = persist->dev; @@ -4149,8 +4152,6 @@ static void mlx4_remove_one(struct pci_dev *pdev) struct devlink *devlink = priv_to_devlink(priv); int active_vfs = 0; - devlink_reload_disable(devlink); - if (mlx4_is_slave(dev)) persist->interface_state |= MLX4_INTERFACE_STATE_NOWAIT; @@ -4185,11 +4186,26 @@ static void mlx4_remove_one(struct pci_dev *pdev) mlx4_pci_disable_device(dev); devlink_params_unregister(devlink, mlx4_devlink_params, ARRAY_SIZE(mlx4_devlink_params)); - devlink_unregister(devlink); kfree(dev->persist); devlink_free(devlink); } +static void mlx4_remove_one(struct pci_dev *pdev) +{ + struct mlx4_dev_persistent *persist = pci_get_drvdata(pdev); + struct devlink *devlink; + struct mlx4_priv *priv; + struct mlx4_dev *dev; + + dev = persist->dev; + priv = mlx4_priv(dev); + devlink = priv_to_devlink(priv); + + devlink_reload_disable(devlink); + devlink_unregister(devlink); + _mlx4_remove_one(pdev); +} + static int restore_current_port_types(struct mlx4_dev *dev, enum mlx4_port_type *types, enum mlx4_port_type *poss_types) From patchwork Tue Aug 10 13:37:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 12428975 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D796BC432BE for ; Tue, 10 Aug 2021 13:38:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BE76461052 for ; Tue, 10 Aug 2021 13:38:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241735AbhHJNil (ORCPT ); Tue, 10 Aug 2021 09:38:41 -0400 Received: from mail.kernel.org ([198.145.29.99]:48910 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241672AbhHJNiU (ORCPT ); Tue, 10 Aug 2021 09:38:20 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 0F2A060EE9; Tue, 10 Aug 2021 13:37:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1628602677; bh=Db1MP3xIgL/VeLq8W65kNJw4ggeLpjP4aKbSFa7JTUs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kXcdyu2Jp93JuSZYXbeiL7lCn0uWqIbEHlYIMp1WCmDEbgptqn4S5VuFqTNQJHlIU HtFM3Qzt+mMVCznZIxsgnAetooDXmggle2qnwlbkMDQ+c1cIdf4yr3Zv7rTobZViu9 8bY5ZGmrY8VsXr12wnSrgJtauakA/ns2GDuOWD+wze8ki282F4PFwRc9GlUGfTDsZD wqsM0IUXRS5vfx1hP5NNhjWQRxSlPuds6rU1SJr6jM/r1ib21FPQUm8Wy3US3rnH97 4OO9i8xILHN/95LNuo+yXyjLYZ8WcyeDF0PCMs/Y16R5frsd8EFLqMBj/h/wjy7MH/ eA0TMeIMFNTuA== From: Leon Romanovsky To: "David S . Miller" , Jakub Kicinski Cc: Leon Romanovsky , Guangbin Huang , Ido Schimmel , Jiri Pirko , linux-kernel@vger.kernel.org, Michael Guralnik , netdev@vger.kernel.org, Saeed Mahameed , Salil Mehta , Tariq Toukan , Yisen Zhuang , Yufeng Mo Subject: [PATCH net-next 3/5] mlxsw: core: Refactor code to publish devlink ops when device is ready Date: Tue, 10 Aug 2021 16:37:33 +0300 Message-Id: X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Leon Romanovsky Move devlink_register() to be last command after device is fully initialized. Signed-off-by: Leon Romanovsky --- drivers/net/ethernet/mellanox/mlxsw/core.c | 27 +++++++++++----------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c index f080fab3de2b..a8a989070aaf 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core.c @@ -1974,12 +1974,6 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, if (err) goto err_emad_init; - if (!reload) { - err = devlink_register(devlink); - if (err) - goto err_devlink_register; - } - if (!reload) { err = mlxsw_core_params_register(mlxsw_core); if (err) @@ -2017,11 +2011,20 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, mlxsw_core->is_initialized = true; devlink_params_publish(devlink); - if (!reload) + if (!reload) { + err = devlink_register(devlink); + if (err) + goto err_devlink_register; + devlink_reload_enable(devlink); + } return 0; +err_devlink_register: + devlink_params_unpublish(devlink); + mlxsw_core->is_initialized = false; + mlxsw_env_fini(mlxsw_core->env); err_env_init: mlxsw_thermal_fini(mlxsw_core->thermal); err_thermal_init: @@ -2036,9 +2039,6 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, if (!reload) mlxsw_core_params_unregister(mlxsw_core); err_register_params: - if (!reload) - devlink_unregister(devlink); -err_devlink_register: mlxsw_emad_fini(mlxsw_core); err_emad_init: kfree(mlxsw_core->lag.mapping); @@ -2087,8 +2087,10 @@ void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core, { struct devlink *devlink = priv_to_devlink(mlxsw_core); - if (!reload) + if (!reload) { devlink_reload_disable(devlink); + devlink_unregister(devlink); + } if (devlink_is_reload_failed(devlink)) { if (!reload) /* Only the parts that were not de-initialized in the @@ -2109,8 +2111,6 @@ void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core, mlxsw_core_health_fini(mlxsw_core); if (!reload) mlxsw_core_params_unregister(mlxsw_core); - if (!reload) - devlink_unregister(devlink); mlxsw_emad_fini(mlxsw_core); kfree(mlxsw_core->lag.mapping); mlxsw_ports_fini(mlxsw_core, reload); @@ -2124,7 +2124,6 @@ void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core, reload_fail_deinit: mlxsw_core_params_unregister(mlxsw_core); - devlink_unregister(devlink); devlink_resources_unregister(devlink, NULL); devlink_free(devlink); } From patchwork Tue Aug 10 13:37:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 12428979 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B8C2C4338F for ; Tue, 10 Aug 2021 13:38:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 647A061076 for ; Tue, 10 Aug 2021 13:38:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241727AbhHJNiw (ORCPT ); Tue, 10 Aug 2021 09:38:52 -0400 Received: from mail.kernel.org ([198.145.29.99]:48792 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241627AbhHJNiN (ORCPT ); Tue, 10 Aug 2021 09:38:13 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 6164C60F41; Tue, 10 Aug 2021 13:37:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1628602671; bh=5iOodNNf45LFVCbt+fmoQm3wJTc3Bn41NbczBUOICdE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EGgVcfxf3mfkzE5QcBNSUZANrCaArrN3n44pxkAHiyC7gJlODQ79mFlJjVGSYih6I 7x3Z8O0GVF7FjRTagQctw8YZqMifXLZyl7EWqGIXaB6xrtawSiYru+UfJHDhfVMMjw rB4tp6b6U9CiAFwmnfHls6y9cdKMVNtYDj8DrkATStOc5yMw96Nl8q7TAy7nvhIvLs WMDjQ+BN/ew4Dxzd5weLaCL2CeQihYAe+/B6PM5KZmuWiQLhdNcgDCHh2Ln5Bt8jN7 Tklxk5CoHs3sNV9H9cIhy85lTczhdc2D6nfWbjeJRKsDyLm7BbaNqYOwjJ6SA5fefm rDnZnJxn4tkeg== From: Leon Romanovsky To: "David S . Miller" , Jakub Kicinski Cc: Leon Romanovsky , Guangbin Huang , Ido Schimmel , Jiri Pirko , linux-kernel@vger.kernel.org, Michael Guralnik , netdev@vger.kernel.org, Saeed Mahameed , Salil Mehta , Tariq Toukan , Yisen Zhuang , Yufeng Mo Subject: [PATCH net-next 4/5] net/mlx5: Accept devlink user input after driver initialization complete Date: Tue, 10 Aug 2021 16:37:34 +0300 Message-Id: <1d6c2ebf166ed742dc53a2b3c25bb5b8afdcf7ec.1628599239.git.leonro@nvidia.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Leon Romanovsky The change of devlink_alloc() to accept device makes sure that device is fully initialized and device_register() does nothing except allowing users to use that devlink instance. Such change ensures that no user input will be usable till that point and it eliminates the need to worry about internal locking as long as devlink_register is called last since all accesses to the devlink are during initialization. This change fixes the following lockdep warning. ====================================================== WARNING: possible circular locking dependency detected 5.14.0-rc2+ #27 Not tainted ------------------------------------------------------ devlink/265 is trying to acquire lock: ffff8880133c2bc0 (&dev->intf_state_mutex){+.+.}-{3:3}, at: mlx5_unload_one+0x1e/0xa0 [mlx5_core] but task is already holding lock: ffffffff8362b468 (devlink_mutex){+.+.}-{3:3}, at: devlink_nl_pre_doit+0x2b/0x8d0 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 (devlink_mutex){+.+.}-{3:3}: __mutex_lock+0x149/0x1310 devlink_register+0xe7/0x280 mlx5_devlink_register+0x118/0x480 [mlx5_core] mlx5_init_one+0x34b/0x440 [mlx5_core] probe_one+0x480/0x6e0 [mlx5_core] pci_device_probe+0x2a0/0x4a0 really_probe+0x1cb/0xba0 __driver_probe_device+0x18f/0x470 driver_probe_device+0x49/0x120 __driver_attach+0x1ce/0x400 bus_for_each_dev+0x11e/0x1a0 bus_add_driver+0x309/0x570 driver_register+0x20f/0x390 0xffffffffa04a0062 do_one_initcall+0xd5/0x400 do_init_module+0x1c8/0x760 load_module+0x7d9d/0xa4b0 __do_sys_finit_module+0x118/0x1a0 do_syscall_64+0x3d/0x90 entry_SYSCALL_64_after_hwframe+0x44/0xae -> #0 (&dev->intf_state_mutex){+.+.}-{3:3}: __lock_acquire+0x2999/0x5a40 lock_acquire+0x1a9/0x4a0 __mutex_lock+0x149/0x1310 mlx5_unload_one+0x1e/0xa0 [mlx5_core] mlx5_devlink_reload_down+0x185/0x2b0 [mlx5_core] devlink_reload+0x1f2/0x640 devlink_nl_cmd_reload+0x6c3/0x10d0 genl_family_rcv_msg_doit+0x1e9/0x2f0 genl_rcv_msg+0x27f/0x4a0 netlink_rcv_skb+0x11e/0x340 genl_rcv+0x24/0x40 netlink_unicast+0x433/0x700 netlink_sendmsg+0x6fb/0xbe0 sock_sendmsg+0xb0/0xe0 __sys_sendto+0x192/0x240 __x64_sys_sendto+0xdc/0x1b0 do_syscall_64+0x3d/0x90 entry_SYSCALL_64_after_hwframe+0x44/0xae other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(devlink_mutex); lock(&dev->intf_state_mutex); lock(devlink_mutex); lock(&dev->intf_state_mutex); *** DEADLOCK *** 3 locks held by devlink/265: #0: ffffffff836371d0 (cb_lock){++++}-{3:3}, at: genl_rcv+0x15/0x40 #1: ffffffff83637288 (genl_mutex){+.+.}-{3:3}, at: genl_rcv_msg+0x31a/0x4a0 #2: ffffffff8362b468 (devlink_mutex){+.+.}-{3:3}, at: devlink_nl_pre_doit+0x2b/0x8d0 stack backtrace: CPU: 0 PID: 265 Comm: devlink Not tainted 5.14.0-rc2+ #27 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 Call Trace: dump_stack_lvl+0x45/0x59 check_noncircular+0x268/0x310 ? print_circular_bug+0x460/0x460 ? __kernel_text_address+0xe/0x30 ? alloc_chain_hlocks+0x1e6/0x5a0 __lock_acquire+0x2999/0x5a40 ? lockdep_hardirqs_on_prepare+0x3e0/0x3e0 ? add_lock_to_list.constprop.0+0x6c/0x530 lock_acquire+0x1a9/0x4a0 ? mlx5_unload_one+0x1e/0xa0 [mlx5_core] ? lock_release+0x6c0/0x6c0 ? lockdep_hardirqs_on_prepare+0x3e0/0x3e0 ? lock_is_held_type+0x98/0x110 __mutex_lock+0x149/0x1310 ? mlx5_unload_one+0x1e/0xa0 [mlx5_core] ? lock_is_held_type+0x98/0x110 ? mlx5_unload_one+0x1e/0xa0 [mlx5_core] ? find_held_lock+0x2d/0x110 ? mutex_lock_io_nested+0x1160/0x1160 ? mlx5_lag_is_active+0x72/0x90 [mlx5_core] ? lock_downgrade+0x6d0/0x6d0 ? do_raw_spin_lock+0x12e/0x270 ? rwlock_bug.part.0+0x90/0x90 ? mlx5_unload_one+0x1e/0xa0 [mlx5_core] mlx5_unload_one+0x1e/0xa0 [mlx5_core] mlx5_devlink_reload_down+0x185/0x2b0 [mlx5_core] ? netlink_broadcast_filtered+0x308/0xac0 ? mlx5_devlink_info_get+0x1f0/0x1f0 [mlx5_core] ? __build_skb_around+0x110/0x2b0 ? __alloc_skb+0x113/0x2b0 devlink_reload+0x1f2/0x640 ? devlink_unregister+0x1e0/0x1e0 ? security_capable+0x51/0x90 devlink_nl_cmd_reload+0x6c3/0x10d0 ? devlink_nl_cmd_get_doit+0x1e0/0x1e0 ? devlink_nl_pre_doit+0x72/0x8d0 genl_family_rcv_msg_doit+0x1e9/0x2f0 ? __lock_acquire+0x15e2/0x5a40 ? genl_family_rcv_msg_attrs_parse.constprop.0+0x240/0x240 ? mutex_lock_io_nested+0x1160/0x1160 ? security_capable+0x51/0x90 genl_rcv_msg+0x27f/0x4a0 ? genl_get_cmd+0x3c0/0x3c0 ? lock_acquire+0x1a9/0x4a0 ? devlink_nl_cmd_get_doit+0x1e0/0x1e0 ? lock_release+0x6c0/0x6c0 netlink_rcv_skb+0x11e/0x340 ? genl_get_cmd+0x3c0/0x3c0 ? netlink_ack+0x930/0x930 genl_rcv+0x24/0x40 netlink_unicast+0x433/0x700 ? netlink_attachskb+0x750/0x750 ? __alloc_skb+0x113/0x2b0 netlink_sendmsg+0x6fb/0xbe0 ? netlink_unicast+0x700/0x700 ? netlink_unicast+0x700/0x700 sock_sendmsg+0xb0/0xe0 __sys_sendto+0x192/0x240 ? __x64_sys_getpeername+0xb0/0xb0 ? do_sys_openat2+0x10a/0x370 ? down_write_nested+0x150/0x150 ? do_user_addr_fault+0x215/0xd50 ? __x64_sys_openat+0x11f/0x1d0 ? __x64_sys_open+0x1a0/0x1a0 __x64_sys_sendto+0xdc/0x1b0 ? syscall_enter_from_user_mode+0x1d/0x50 do_syscall_64+0x3d/0x90 entry_SYSCALL_64_after_hwframe+0x44/0xae RIP: 0033:0x7f50b50b6b3a Code: d8 64 89 02 48 c7 c0 ff ff ff ff eb b8 0f 1f 00 f3 0f 1e fa 41 89 ca 64 8b 04 25 18 00 00 00 85 c0 75 15 b8 2c 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 76 c3 0f 1f 44 00 00 55 48 83 ec 30 44 89 4c RSP: 002b:00007fff6c0d3f38 EFLAGS: 00000246 ORIG_RAX: 000000000000002c RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00007f50b50b6b3a RDX: 0000000000000038 RSI: 000055763ac08440 RDI: 0000000000000003 RBP: 000055763ac08410 R08: 00007f50b5192200 R09: 000000000000000c R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 0000000000000000 R14: 000055763ac08410 R15: 000055763ac08440 mlx5_core 0000:00:09.0: firmware version: 4.8.9999 mlx5_core 0000:00:09.0: 0.000 Gb/s available PCIe bandwidth (8.0 GT/s PCIe x255 link) mlx5_core 0000:00:09.0 eth1: Link up Fixes: a6f3b62386a0 ("net/mlx5: Move devlink registration before interfaces load") Signed-off-by: Leon Romanovsky --- drivers/net/ethernet/mellanox/mlx5/core/devlink.c | 10 ++-------- drivers/net/ethernet/mellanox/mlx5/core/main.c | 13 ++++++++++++- .../net/ethernet/mellanox/mlx5/core/sf/dev/driver.c | 12 ++++++++++-- 3 files changed, 24 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index f38553ff538b..9b058f97c8fd 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -643,14 +643,11 @@ int mlx5_devlink_register(struct devlink *devlink) { int err; - err = devlink_register(devlink); - if (err) - return err; - err = devlink_params_register(devlink, mlx5_devlink_params, ARRAY_SIZE(mlx5_devlink_params)); if (err) - goto params_reg_err; + return err; + mlx5_devlink_set_params_init_values(devlink); devlink_params_publish(devlink); @@ -663,8 +660,6 @@ int mlx5_devlink_register(struct devlink *devlink) traps_reg_err: devlink_params_unregister(devlink, mlx5_devlink_params, ARRAY_SIZE(mlx5_devlink_params)); -params_reg_err: - devlink_unregister(devlink); return err; } @@ -673,5 +668,4 @@ void mlx5_devlink_unregister(struct devlink *devlink) mlx5_devlink_traps_unregister(devlink); devlink_params_unregister(devlink, mlx5_devlink_params, ARRAY_SIZE(mlx5_devlink_params)); - devlink_unregister(devlink); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index a8efd9f1af4c..9f10049a63f9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -1494,10 +1494,20 @@ static int probe_one(struct pci_dev *pdev, const struct pci_device_id *id) dev_err(&pdev->dev, "mlx5_crdump_enable failed with error code %d\n", err); pci_save_state(pdev); + err = devlink_register(devlink); + if (err) { + mlx5_core_err(dev, + "devlink_register failed with error code %d\n", + err); + goto devlink_reg_err; + } if (!mlx5_core_is_mp_slave(dev)) devlink_reload_enable(devlink); return 0; - +devlink_reg_err: + mlx5_crdump_disable(dev); + mlx5_drain_health_wq(dev); + mlx5_uninit_one(dev); err_init_one: mlx5_pci_close(dev); pci_init_err: @@ -1516,6 +1526,7 @@ static void remove_one(struct pci_dev *pdev) struct devlink *devlink = priv_to_devlink(dev); devlink_reload_disable(devlink); + devlink_unregister(devlink); mlx5_crdump_disable(dev); mlx5_drain_health_wq(dev); mlx5_uninit_one(dev); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c index 052f48068dc1..b0f2b9db6d85 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c @@ -46,9 +46,17 @@ static int mlx5_sf_dev_probe(struct auxiliary_device *adev, const struct auxilia mlx5_core_warn(mdev, "mlx5_init_one err=%d\n", err); goto init_one_err; } + + err = devlink_register(devlink); + if (err) { + mlx5_core_warn(mdev, "devlink_register err=%d\n", err); + goto devlink_reg_err; + } devlink_reload_enable(devlink); return 0; +devlink_reg_err: + mlx5_uninit_one(mdev); init_one_err: iounmap(mdev->iseg); remap_err: @@ -61,10 +69,10 @@ static int mlx5_sf_dev_probe(struct auxiliary_device *adev, const struct auxilia static void mlx5_sf_dev_remove(struct auxiliary_device *adev) { struct mlx5_sf_dev *sf_dev = container_of(adev, struct mlx5_sf_dev, adev); - struct devlink *devlink; + struct devlink *devlink = priv_to_devlink(sf_dev->mdev); - devlink = priv_to_devlink(sf_dev->mdev); devlink_reload_disable(devlink); + devlink_unregister(devlink); mlx5_uninit_one(sf_dev->mdev); iounmap(sf_dev->mdev->iseg); mlx5_mdev_uninit(sf_dev->mdev); From patchwork Tue Aug 10 13:37:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 12428977 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06E63C4338F for ; Tue, 10 Aug 2021 13:38:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DB90B60FDA for ; Tue, 10 Aug 2021 13:38:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241660AbhHJNiq (ORCPT ); Tue, 10 Aug 2021 09:38:46 -0400 Received: from mail.kernel.org ([198.145.29.99]:48854 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241647AbhHJNiQ (ORCPT ); Tue, 10 Aug 2021 09:38:16 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id B20C36101E; Tue, 10 Aug 2021 13:37:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1628602674; bh=TvoQojqsY59TCrvQ8l2JACkOs5SoYXc6V5jqqVTGCQc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tDXm4IgUlJoezrUCuAzKv0mLdZjMaHw3eemrJSton4hIxatOqfX/HztjxyHagWG7G Vx4yuhqYBegEORxsgnnxjW7fJ8jvLfwdtxNaTg8ayepFGtau3zB7UVfyl1JJymtSHf 208YurRn9eFewDC1axQRco32NLl1gbc2ply5FHSIkxWoUijmWrEMDgsE2BJ52BM5Rs 5JcVsRpNkogkQ9umAiflG7n3ZHBfIHGEinH3x//myCUvGMsBNiky3Mxw68HTpw1XrD KeBjsYRV0tuHd/b2UTvYDyqmsZX7gXog2GVirYWB8xTgOujg6eOlAoeikySJdJtkHs keFvigv/0CFuQ== From: Leon Romanovsky To: "David S . Miller" , Jakub Kicinski Cc: Leon Romanovsky , Guangbin Huang , Ido Schimmel , Jiri Pirko , linux-kernel@vger.kernel.org, Michael Guralnik , netdev@vger.kernel.org, Saeed Mahameed , Salil Mehta , Tariq Toukan , Yisen Zhuang , Yufeng Mo Subject: [PATCH net-next 5/5] netdevsim: Delay user access till probe is finished Date: Tue, 10 Aug 2021 16:37:35 +0300 Message-Id: X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Leon Romanovsky Don't publish supported user space accessible ops till probe is finished. Signed-off-by: Leon Romanovsky --- drivers/net/netdevsim/dev.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c index 54313bd57797..181258bd72f2 100644 --- a/drivers/net/netdevsim/dev.c +++ b/drivers/net/netdevsim/dev.c @@ -1470,14 +1470,10 @@ int nsim_dev_probe(struct nsim_bus_dev *nsim_bus_dev) if (err) goto err_devlink_free; - err = devlink_register(devlink); - if (err) - goto err_resources_unregister; - err = devlink_params_register(devlink, nsim_devlink_params, ARRAY_SIZE(nsim_devlink_params)); if (err) - goto err_dl_unregister; + goto err_resources_unregister; nsim_devlink_set_params_init_values(nsim_dev, devlink); err = nsim_dev_dummy_region_init(nsim_dev, devlink); @@ -1515,10 +1511,17 @@ int nsim_dev_probe(struct nsim_bus_dev *nsim_bus_dev) goto err_psample_exit; devlink_params_publish(devlink); - devlink_reload_enable(devlink); nsim_dev->esw_mode = DEVLINK_ESWITCH_MODE_LEGACY; + err = devlink_register(devlink); + if (err) + goto err_port_del_all; + + devlink_reload_enable(devlink); return 0; +err_port_del_all: + devlink_params_unpublish(devlink); + nsim_dev_port_del_all(nsim_dev); err_psample_exit: nsim_dev_psample_exit(nsim_dev); err_bpf_dev_exit: @@ -1536,8 +1539,6 @@ int nsim_dev_probe(struct nsim_bus_dev *nsim_bus_dev) err_params_unregister: devlink_params_unregister(devlink, nsim_devlink_params, ARRAY_SIZE(nsim_devlink_params)); -err_dl_unregister: - devlink_unregister(devlink); err_resources_unregister: devlink_resources_unregister(devlink, NULL); err_devlink_free: @@ -1573,6 +1574,7 @@ void nsim_dev_remove(struct nsim_bus_dev *nsim_bus_dev) struct devlink *devlink = priv_to_devlink(nsim_dev); devlink_reload_disable(devlink); + devlink_unregister(devlink); nsim_dev_reload_destroy(nsim_dev); @@ -1580,7 +1582,6 @@ void nsim_dev_remove(struct nsim_bus_dev *nsim_bus_dev) nsim_dev_debugfs_exit(nsim_dev); devlink_params_unregister(devlink, nsim_devlink_params, ARRAY_SIZE(nsim_devlink_params)); - devlink_unregister(devlink); devlink_resources_unregister(devlink, NULL); devlink_free(devlink); }