From patchwork Fri Mar 8 22:07:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 10845497 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CD891803 for ; Fri, 8 Mar 2019 22:08:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2950B2D1F0 for ; Fri, 8 Mar 2019 22:08:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D17E2FC0A; Fri, 8 Mar 2019 22:08:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D9BE2D1F0 for ; Fri, 8 Mar 2019 22:08:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726388AbfCHWIN (ORCPT ); Fri, 8 Mar 2019 17:08:13 -0500 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42803 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726302AbfCHWIN (ORCPT ); Fri, 8 Mar 2019 17:08:13 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from parav@mellanox.com) with ESMTPS (AES256-SHA encrypted); 9 Mar 2019 00:08:09 +0200 Received: from sw-mtx-036.mtx.labs.mlnx (sw-mtx-036.mtx.labs.mlnx [10.12.150.149]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x28M83SV002296; Sat, 9 Mar 2019 00:08:07 +0200 From: Parav Pandit To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, michal.lkml@markovi.net, davem@davemloft.net, gregkh@linuxfoundation.org, jiri@mellanox.com, kwankhede@nvidia.com Cc: parav@mellanox.com, alex.williamson@redhat.com, vuhuong@mellanox.com, yuvalav@mellanox.com, jakub.kicinski@netronome.com, kvm@vger.kernel.org Subject: [RFC net-next v1 1/3] vfio/mdev: Inherit dma masks of parent device Date: Fri, 8 Mar 2019 16:07:54 -0600 Message-Id: <1552082876-60228-2-git-send-email-parav@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1552082876-60228-1-git-send-email-parav@mellanox.com> References: <1552082876-60228-1-git-send-email-parav@mellanox.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Inherit dma mask of parent device in child mdev devices, so that protocol stack can use right dma mask while doing dma mappings. Signed-off-by: Parav Pandit --- drivers/vfio/mdev/mdev_core.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c index 0212f0e..9b8bdc9 100644 --- a/drivers/vfio/mdev/mdev_core.c +++ b/drivers/vfio/mdev/mdev_core.c @@ -315,6 +315,10 @@ int mdev_device_create(struct kobject *kobj, struct device *dev, uuid_le uuid) mdev->dev.parent = dev; mdev->dev.bus = &mdev_bus_type; mdev->dev.release = mdev_device_release; + mdev->dev.dma_mask = dev->dma_mask; + mdev->dev.dma_parms = dev->dma_parms; + mdev->dev.coherent_dma_mask = dev->coherent_dma_mask; + dev_set_name(&mdev->dev, "%pUl", uuid.b); ret = device_register(&mdev->dev); From patchwork Fri Mar 8 22:07:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 10845495 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E9E891803 for ; Fri, 8 Mar 2019 22:08:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D6ABF22B27 for ; Fri, 8 Mar 2019 22:08:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C7E242FC0A; Fri, 8 Mar 2019 22:08:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0AB0E22B27 for ; Fri, 8 Mar 2019 22:08:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726585AbfCHWIV (ORCPT ); Fri, 8 Mar 2019 17:08:21 -0500 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42844 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726512AbfCHWIT (ORCPT ); Fri, 8 Mar 2019 17:08:19 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from parav@mellanox.com) with ESMTPS (AES256-SHA encrypted); 9 Mar 2019 00:08:12 +0200 Received: from sw-mtx-036.mtx.labs.mlnx (sw-mtx-036.mtx.labs.mlnx [10.12.150.149]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x28M83SW002296; Sat, 9 Mar 2019 00:08:10 +0200 From: Parav Pandit To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, michal.lkml@markovi.net, davem@davemloft.net, gregkh@linuxfoundation.org, jiri@mellanox.com, kwankhede@nvidia.com Cc: parav@mellanox.com, alex.williamson@redhat.com, vuhuong@mellanox.com, yuvalav@mellanox.com, jakub.kicinski@netronome.com, kvm@vger.kernel.org Subject: [RFC net-next v1 2/3] net/mlx5: Add mdev sub device life cycle command support Date: Fri, 8 Mar 2019 16:07:55 -0600 Message-Id: <1552082876-60228-3-git-send-email-parav@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1552082876-60228-1-git-send-email-parav@mellanox.com> References: <1552082876-60228-1-git-send-email-parav@mellanox.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Implement mdev hooks to to create mediated devices using mdev driver. Actual mlx5_core driver in the host is expected to bind to these devices using standard device driver model. mdev devices are created using sysfs file as below example. $ uuidgen 49d0e9ac-61b8-4c91-957e-6f6dbc42557d $ echo 49d0e9ac-61b8-4c91-957e-6f6dbc42557d > \ ./bus/pci/devices/0000:05:00.0/mdev_supported_types/mlx5_core-mgmt/create $ echo 49d0e9ac-61b8-4c91-957e-6f6dbc42557d > /sys/bus/mdev/drivers/vfio_mdev/unbind Once mlx5 core driver is registered as mdev driver, mdev can be attached to mlx5_core driver as below. $ echo 49d0e9ac-61b8-4c91-957e-6f6dbc42557d > /sys/bus/mdev/drivers/mlx5_core/bind devlink output: $ devlink dev show pci/0000:05:00.0 mdev/69ea1551-d054-46e9-974d-8edae8f0aefe Signed-off-by: Parav Pandit --- drivers/net/ethernet/mellanox/mlx5/core/Kconfig | 9 ++ drivers/net/ethernet/mellanox/mlx5/core/Makefile | 5 + drivers/net/ethernet/mellanox/mlx5/core/main.c | 9 ++ drivers/net/ethernet/mellanox/mlx5/core/mdev.c | 120 +++++++++++++++++++++ .../net/ethernet/mellanox/mlx5/core/mlx5_core.h | 15 +++ include/linux/mlx5/driver.h | 5 + 6 files changed, 163 insertions(+) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/mdev.c diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig index 37a5514..881ae1a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig +++ b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig @@ -117,3 +117,12 @@ config MLX5_EN_TLS Build support for TLS cryptography-offload accelaration in the NIC. Note: Support for hardware with this capability needs to be selected for this option to become available. + +config MLX5_MDEV + bool "Mellanox Technologies Mediated device support" + depends on MLX5_CORE + depends on VFIO_MDEV + default y + help + Build support for mediated devices. Mediated devices allow creating + multiple netdev and/or rdma device(s) on single PCI function. diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index 82d636b..e5c0822c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -58,4 +58,9 @@ mlx5_core-$(CONFIG_MLX5_EN_IPSEC) += en_accel/ipsec.o en_accel/ipsec_rxtx.o \ mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/tls.o en_accel/tls_rxtx.o en_accel/tls_stats.o +# +# Mdev basic +# +mlx5_core-$(CONFIG_MLX5_MDEV) += mdev.o + CFLAGS_tracepoint.o := -I$(src) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index 40d591c..72b0072 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -851,10 +851,18 @@ static int mlx5_init_once(struct mlx5_core_dev *dev, struct mlx5_priv *priv) goto err_sriov_cleanup; } + err = mlx5_mdev_init(dev); + if (err) { + dev_err(&pdev->dev, "Failed to init mdev device %d\n", err); + goto err_fpga_cleanup; + } + dev->tracer = mlx5_fw_tracer_create(dev); return 0; +err_fpga_cleanup: + mlx5_fpga_cleanup(dev); err_sriov_cleanup: mlx5_sriov_cleanup(dev); err_eswitch_cleanup: @@ -881,6 +889,7 @@ static int mlx5_init_once(struct mlx5_core_dev *dev, struct mlx5_priv *priv) static void mlx5_cleanup_once(struct mlx5_core_dev *dev) { mlx5_fw_tracer_destroy(dev->tracer); + mlx5_mdev_cleanup(dev); mlx5_fpga_cleanup(dev); mlx5_sriov_cleanup(dev); mlx5_eswitch_cleanup(dev->priv.eswitch); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mdev.c b/drivers/net/ethernet/mellanox/mlx5/core/mdev.c new file mode 100644 index 0000000..e8e4aac --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/mdev.c @@ -0,0 +1,120 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) 2018-19 Mellanox Technologies + +#include +#include + +#include "mlx5_core.h" + +#define MLX5_MAX_MDEVS 1 + +struct mlx5_mdev { + struct mlx5_core_dev *dev; +}; + +static int mlx5_mdev_create(struct kobject *kobj, struct mdev_device *mdev) +{ + struct mlx5_core_dev *mlx5_dev; + struct device *parent_dev; + struct mlx5_mdev *mmdev; + struct devlink *devlink; + bool added; + int err; + + parent_dev = mdev_parent_dev(mdev); + mlx5_dev = pci_get_drvdata(to_pci_dev(parent_dev)); + + added = atomic_add_unless(&mlx5_dev->mdev_info.cnt, 1, MLX5_MAX_MDEVS); + if (!added) + return -ENOSPC; + + devlink = devlink_alloc(NULL, sizeof(*mmdev)); + if (!devlink) { + atomic_dec(&mlx5_dev->mdev_info.cnt); + return -ENOMEM; + } + mmdev = devlink_priv(devlink); + mmdev->dev = mlx5_dev; + mdev_set_drvdata(mdev, mmdev); + + /* TODO: create a mediated device hw object that driver + * can work on later on to enable interrupts, create queues etc. + */ + err = devlink_register(devlink, mdev_dev(mdev)); + if (err) { + devlink_free(devlink); + atomic_dec(&mlx5_dev->mdev_info.cnt); + } + return err; +} + +static int mlx5_mdev_remove(struct mdev_device *mdev) +{ + struct mlx5_core_dev *mlx5_dev; + struct device *parent_dev; + struct mlx5_mdev *mmdev; + struct devlink *devlink; + + parent_dev = mdev_parent_dev(mdev); + mlx5_dev = pci_get_drvdata(to_pci_dev(parent_dev)); + + mmdev = mdev_get_drvdata(mdev); + devlink = priv_to_devlink(mmdev); + + devlink_unregister(devlink); + devlink_free(devlink); + atomic_dec(&mlx5_dev->mdev_info.cnt); + return 0; +} + +static ssize_t +num_mdevs_show(struct kobject *kobj, struct device *dev, char *buf) +{ + struct pci_dev *pdev = to_pci_dev(dev); + struct mlx5_core_dev *mlx5_dev; + + mlx5_dev = pci_get_drvdata(pdev); + + return sprintf(buf, "%d\n", atomic_read(&mlx5_dev->mdev_info.cnt)); +} +MDEV_TYPE_ATTR_RO(num_mdevs); + +static ssize_t +available_instances_show(struct kobject *kobj, struct device *dev, char *buf) +{ + return sprintf(buf, "%d\n", MLX5_MAX_MDEVS); +} +MDEV_TYPE_ATTR_RO(available_instances); + +static struct attribute *mdev_dev_attrs[] = { + &mdev_type_attr_num_mdevs.attr, + &mdev_type_attr_available_instances.attr, + NULL, +}; + +static struct attribute_group mdev_mgmt_group = { + .name = "mgmt", + .attrs = mdev_dev_attrs, +}; + +struct attribute_group *mlx5_mdev_groups[] = { + &mdev_mgmt_group, + NULL, +}; + +const struct mdev_parent_ops mlx5_mdev_ops = { + .create = mlx5_mdev_create, + .remove = mlx5_mdev_remove, + .supported_type_groups = mlx5_mdev_groups, +}; + +int mlx5_mdev_init(struct mlx5_core_dev *dev) +{ + return mdev_register_device(&dev->pdev->dev, &mlx5_mdev_ops); +} + +void mlx5_mdev_cleanup(struct mlx5_core_dev *dev) +{ + mdev_unregister_device(&dev->pdev->dev); +} + diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index 9529cf9..0605a63 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -202,4 +202,19 @@ enum { u8 mlx5_get_nic_state(struct mlx5_core_dev *dev); void mlx5_set_nic_state(struct mlx5_core_dev *dev, u8 state); + +#ifdef CONFIG_MLX5_MDEV +int mlx5_mdev_init(struct mlx5_core_dev *mdev); +void mlx5_mdev_cleanup(struct mlx5_core_dev *mdev); +#else +static inline int mlx5_mdev_init(struct mlx5_core_dev *mdev) +{ + return 0; +} + +static inline void mlx5_mdev_cleanup(struct mlx5_core_dev *mdev) +{ +} +#endif + #endif /* __MLX5_CORE_H__ */ diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index c2de50f..6836ab9 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -637,6 +637,10 @@ struct mlx5_clock { struct mlx5_fw_tracer; struct mlx5_vxlan; +struct mlx5_mdevs_info { + atomic_t cnt; +}; + struct mlx5_core_dev { struct pci_dev *pdev; /* sync pci state */ @@ -679,6 +683,7 @@ struct mlx5_core_dev { struct mlx5_ib_clock_info *clock_info; struct page *clock_info_page; struct mlx5_fw_tracer *tracer; + struct mlx5_mdevs_info mdev_info; }; struct mlx5_db { From patchwork Fri Mar 8 22:07:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 10845493 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D2D611669 for ; Fri, 8 Mar 2019 22:08:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD0D422B27 for ; Fri, 8 Mar 2019 22:08:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AF8FE2DEE2; Fri, 8 Mar 2019 22:08:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1847B22B27 for ; Fri, 8 Mar 2019 22:08:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726616AbfCHWIW (ORCPT ); Fri, 8 Mar 2019 17:08:22 -0500 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42846 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726520AbfCHWIS (ORCPT ); Fri, 8 Mar 2019 17:08:18 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from parav@mellanox.com) with ESMTPS (AES256-SHA encrypted); 9 Mar 2019 00:08:15 +0200 Received: from sw-mtx-036.mtx.labs.mlnx (sw-mtx-036.mtx.labs.mlnx [10.12.150.149]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x28M83SX002296; Sat, 9 Mar 2019 00:08:12 +0200 From: Parav Pandit To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, michal.lkml@markovi.net, davem@davemloft.net, gregkh@linuxfoundation.org, jiri@mellanox.com, kwankhede@nvidia.com Cc: parav@mellanox.com, alex.williamson@redhat.com, vuhuong@mellanox.com, yuvalav@mellanox.com, jakub.kicinski@netronome.com, kvm@vger.kernel.org Subject: [RFC net-next v1 3/3] net/mlx5: Add mdev driver to bind to mdev devices Date: Fri, 8 Mar 2019 16:07:56 -0600 Message-Id: <1552082876-60228-4-git-send-email-parav@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1552082876-60228-1-git-send-email-parav@mellanox.com> References: <1552082876-60228-1-git-send-email-parav@mellanox.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a mdev driver to probe the mdev devices and create fake netdevice for it. Similar to pci driver, when new mdev are created/removed or when user triggers binding a mdev to mlx5_core driver by writing mdev device id to /sys/bus/mdev/drivers/mlx5_core/bind,unbind files, mlx5_core driver's probe(), remove() are invokes to handle life cycle of netdev and rdma device associated with the mdev. Current RFC patch only creates one fake netdev, but in subsequent non RFC patch, it will create related hw objects and netdev. Signed-off-by: Parav Pandit --- drivers/net/ethernet/mellanox/mlx5/core/Makefile | 2 +- drivers/net/ethernet/mellanox/mlx5/core/dev.c | 18 ++++ drivers/net/ethernet/mellanox/mlx5/core/main.c | 13 +++ .../net/ethernet/mellanox/mlx5/core/mdev_driver.c | 106 +++++++++++++++++++++ .../net/ethernet/mellanox/mlx5/core/mlx5_core.h | 4 + 5 files changed, 142 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/mdev_driver.c diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index e5c0822c..bded136a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -61,6 +61,6 @@ mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/tls.o en_accel/tls_rxtx.o en_accel/t # # Mdev basic # -mlx5_core-$(CONFIG_MLX5_MDEV) += mdev.o +mlx5_core-$(CONFIG_MLX5_MDEV) += mdev.o mdev_driver.o CFLAGS_tracepoint.o := -I$(src) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/dev.c index ebc046f..91b8d8ba 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/dev.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/dev.c @@ -321,3 +321,21 @@ int mlx5_dev_list_trylock(void) { return mutex_trylock(&mlx5_intf_mutex); } + +struct mlx5_core_dev *mlx5_get_core_dev(const struct device *dev) +{ + struct mlx5_core_dev *found = NULL; + struct mlx5_core_dev *tmp_dev; + struct mlx5_priv *priv; + + mutex_lock(&mlx5_intf_mutex); + list_for_each_entry(priv, &mlx5_dev_list, dev_list) { + tmp_dev = container_of(priv, struct mlx5_core_dev, priv); + if (&tmp_dev->pdev->dev == dev) { + found = tmp_dev; + break; + } + } + mutex_unlock(&mlx5_intf_mutex); + return found; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index 72b0072..c1fc0f4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -1553,7 +1554,15 @@ static int __init init(void) mlx5e_init(); #endif +#if IS_ENABLED(CONFIG_VFIO_MDEV) + err = mdev_register_driver(&mlx5_mdev_driver, THIS_MODULE); + if (err) { + pci_unregister_driver(&mlx5_core_driver); + goto err_debug; + } +#else return 0; +#endif err_debug: mlx5_unregister_debugfs(); @@ -1562,6 +1571,10 @@ static int __init init(void) static void __exit cleanup(void) { +#if IS_ENABLED(CONFIG_VFIO_MDEV) + mdev_unregister_driver(&mlx5_mdev_driver); +#endif + #ifdef CONFIG_MLX5_CORE_EN mlx5e_cleanup(); #endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mdev_driver.c b/drivers/net/ethernet/mellanox/mlx5/core/mdev_driver.c new file mode 100644 index 0000000..7618c5e --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/mdev_driver.c @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) 2018-19 Mellanox Technologies + +#include +#include +#include +#include +#include + +#include "mlx5_core.h" + +struct mlx5_subdev_ndev { + struct net_device ndev; +}; + +static void mlx5_dma_test(struct device *dev) +{ + dma_addr_t pa; + void *va; + + va = dma_alloc_coherent(dev, 4096, &pa, GFP_KERNEL); + if (va) + dma_free_coherent(dev, 4096, va, pa); +} + +static struct net_device *ndev; + +static int mlx5e_mdev_open(struct net_device *netdev) +{ + return 0; +} + +static int mlx5e_mdev_close(struct net_device *netdev) +{ + return 0; +} + +static netdev_tx_t +mlx5e_mdev_xmit(struct sk_buff *skb, struct net_device *netdev) +{ + return NETDEV_TX_BUSY; +} + +const struct net_device_ops mlx5e_mdev_netdev_ops = { + .ndo_open = mlx5e_mdev_open, + .ndo_stop = mlx5e_mdev_close, + .ndo_start_xmit = mlx5e_mdev_xmit, +}; + +static int mlx5_mdev_probe(struct device *dev) +{ + struct mdev_device *mdev = mdev_from_dev(dev); + struct device *parent_dev = mdev_parent_dev(mdev); + struct mlx5_core_dev *mlx5_dev; + int err; + + mlx5_dev = mlx5_get_core_dev(parent_dev); + if (!mlx5_dev) + return -ENODEV; + + mlx5_dma_test(dev); + + ndev = alloc_etherdev_mqs(sizeof(struct mlx5_subdev_ndev), 1, 1); + if (!ndev) + return -ENOMEM; + + SET_NETDEV_DEV(ndev, dev); + ndev->netdev_ops = &mlx5e_mdev_netdev_ops; + + /* TODO: + * init_hca(). + * create eqs, cqs. + * attach to irq of parent pci device. + */ + err = register_netdev(ndev); + if (err) { + free_netdev(ndev); + ndev = NULL; + } + return err; +} + +static void mlx5_mdev_remove(struct device *dev) +{ + if (ndev) { + unregister_netdev(ndev); + free_netdev(ndev); + ndev = NULL; + } +} + +struct mdev_driver mlx5_mdev_driver = { + .name = KBUILD_MODNAME, + .probe = mlx5_mdev_probe, + .remove = mlx5_mdev_remove, +}; + +int __init mlx5_mdev_driver_init(void) +{ + return mdev_register_driver(&mlx5_mdev_driver, THIS_MODULE); +} + +void __exit mlx5_mdev_exit(void) +{ + mdev_unregister_driver(&mlx5_mdev_driver); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index 0605a63..2e2b8b5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -47,6 +47,8 @@ extern uint mlx5_core_debug_mask; +extern struct mdev_driver mlx5_mdev_driver; + #define mlx5_core_dbg(__dev, format, ...) \ dev_dbg(&(__dev)->pdev->dev, "%s:%d:(pid %d): " format, \ __func__, __LINE__, current->pid, \ @@ -217,4 +219,6 @@ static inline void mlx5_mdev_cleanup(struct mlx5_core_dev *mdev) } #endif +struct mlx5_core_dev *mlx5_get_core_dev(const struct device *dev); + #endif /* __MLX5_CORE_H__ */