From patchwork Sat Aug 24 03:19:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nikolova, Tatyana E" X-Patchwork-Id: 13776194 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A09F33997; Sat, 24 Aug 2024 03:20:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.16 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724469645; cv=none; b=kLA5Pu+qZ5/mRNPgk8SkbF4qOnv3bOC11vUqgj/g9ihTvJ7qp1LakUp0zrSWggDpKd7CwkYICYZAnUwdK8TlrF3Pbu4xgtHtmyS/AC3Ous2rBuDnXiANnOi+F6mtSMA6+tJ04NYfhc/QnIg8pt/jJVv9bdFDcuUp5vts9IyigsQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724469645; c=relaxed/simple; bh=3oE/r81zH5g6H80Ri2NlRsYmiUn98l3XjsZlE/m06lQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NuvPbH2QDm2qzmfVfbNuE9pwRNNym0MeCvCP+WkRBsnal6WQTh47ZvpPhyW8sS5nVbcHb80h/BcKXXO+ecd5tZIqLeXuA+BcqZdmX0PKm7ocCIRMYiw3Yy/BgR99Vw3v0SVqDKxH/yD9YnaSbViZLiUJAs4bmEeN3jM/TLTxKLI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Ct6v1aCn; arc=none smtp.client-ip=192.198.163.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Ct6v1aCn" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724469643; x=1756005643; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3oE/r81zH5g6H80Ri2NlRsYmiUn98l3XjsZlE/m06lQ=; b=Ct6v1aCnO+rG5RAB9P8jod40ufryKdmT2r3mKXwrc9vqfQHybi6mPaRK W9wDMCQjpA7Jjxx1HRu1SUEaBD9cJVjYIoV5BPF/dIjdZ/ip9vDdrb7QO YehMqa6FPXC9lcQqut9Yp27H9J1F3niQxFeJIkik9GFrqrn9LRmzMbeuZ SOf64zABIqmd+7QEGPKHYE/BYfAqTgoevWTwb4fnYB1dT2zHtaczUpQ1O F1WfPeOwO0rMO9n43Pip6yS7VaMDAYfUm09WVqYFEvYWfYfD2dzy+Hc5Y /10WK6UXYbS6b5DAdStrbVr1P/79frJgMvVwLUrI6H/lABJpbz5ez//Hp A==; X-CSE-ConnectionGUID: hzqyEf2vRM+XQFUGqorS4A== X-CSE-MsgGUID: +Y+ZceVbQ+WOX0gNBbH2LQ== X-IronPort-AV: E=McAfee;i="6700,10204,11173"; a="13187779" X-IronPort-AV: E=Sophos;i="6.10,172,1719903600"; d="scan'208";a="13187779" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa110.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Aug 2024 20:20:41 -0700 X-CSE-ConnectionGUID: 0bmpC/KPSBKAryUiK5s4Nw== X-CSE-MsgGUID: yyk/Yu8+R76LDR1++bKS2A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,172,1719903600"; d="scan'208";a="99492084" Received: from tenikolo-mobl1.amr.corp.intel.com ([10.124.36.66]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Aug 2024 20:20:40 -0700 From: Tatyana Nikolova To: jgg@nvidia.com, leon@kernel.org Cc: linux-rdma@vger.kernel.org, netdev@vger.kernel.org, Joshua Hay , Tatyana Nikolova Subject: [RFC v2 05/25] idpf: implement rdma vport auxiliary dev create, init, and destroy Date: Fri, 23 Aug 2024 22:19:04 -0500 Message-Id: <20240824031924.421-6-tatyana.e.nikolova@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20240824031924.421-1-tatyana.e.nikolova@intel.com> References: <20240824031924.421-1-tatyana.e.nikolova@intel.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Joshua Hay Implement the functions to create, initialize, and destroy an rdma vport auxiliary device. The vport aux dev creation is dependent on the core aux device to call idpf_idc_vport_dev_ctrl to signal that it is ready for vport aux devices. Implement that core callback to either create and initialize the vport aux dev or deinitialize. Rdma vport aux dev creation is also dependent on the control plane to tell us the vport is rdma enabled. Add a flag in the create vport message to signal individual vport rdma capabilities. v2: Guard against unplugging vport aux dev twice. This is possible if irdma is unloaded and then idpf is unloaded. irdma calls idpf_idc_vport_dev_down during its unload which calls unplug. Set the adev to NULL in dev_down, so that the following call to deinit_vport_aux_device during idpf unload will return early from unplug. Signed-off-by: Joshua Hay Signed-off-by: Tatyana Nikolova --- drivers/net/ethernet/intel/idpf/idpf.h | 3 + drivers/net/ethernet/intel/idpf/idpf_idc.c | 174 +++++++++++++++++++- drivers/net/ethernet/intel/idpf/idpf_lib.c | 2 + drivers/net/ethernet/intel/idpf/virtchnl2.h | 13 +- 4 files changed, 189 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/intel/idpf/idpf.h b/drivers/net/ethernet/intel/idpf/idpf.h index cb9344596bfb..2299be4aee4b 100644 --- a/drivers/net/ethernet/intel/idpf/idpf.h +++ b/drivers/net/ethernet/intel/idpf/idpf.h @@ -315,6 +315,8 @@ struct idpf_vport { u32 rxq_model; struct libeth_rx_pt *rx_ptype_lkup; + struct idc_rdma_vport_dev_info *vdev_info; + struct idpf_adapter *adapter; struct net_device *netdev; DECLARE_BITMAP(flags, IDPF_VPORT_FLAGS_NBITS); @@ -862,5 +864,6 @@ int idpf_idc_init(struct idpf_adapter *adapter); int idpf_idc_init_aux_core_dev(struct idpf_adapter *adapter, enum idc_function_type ftype); void idpf_idc_deinit_core_aux_device(struct idc_rdma_core_dev_info *cdev_info); +void idpf_idc_deinit_vport_aux_device(struct idc_rdma_vport_dev_info *vdev_info); #endif /* !_IDPF_H_ */ diff --git a/drivers/net/ethernet/intel/idpf/idpf_idc.c b/drivers/net/ethernet/intel/idpf/idpf_idc.c index 9eb0e0cfc61f..1f22868b8d75 100644 --- a/drivers/net/ethernet/intel/idpf/idpf_idc.c +++ b/drivers/net/ethernet/intel/idpf/idpf_idc.c @@ -28,6 +28,111 @@ int idpf_idc_init(struct idpf_adapter *adapter) return err; } +/** + * idpf_vport_adev_release - function to be mapped to aux dev's release op + * @dev: pointer to device to free + */ +static void idpf_vport_adev_release(struct device *dev) +{ + struct idc_rdma_vport_auxiliary_dev *iadev; + + iadev = container_of(dev, struct idc_rdma_vport_auxiliary_dev, adev.dev); + kfree(iadev); + iadev = NULL; +} + +/* idpf_plug_vport_aux_dev - allocate and register a vport Auxiliary device + * @cdev_info: idc core device info pointer + * @vdev_info: idc vport device info pointer + */ +static int idpf_plug_vport_aux_dev(struct idc_rdma_core_dev_info *cdev_info, + struct idc_rdma_vport_dev_info *vdev_info) +{ + struct idc_rdma_vport_auxiliary_dev *iadev; + char name[IDPF_IDC_MAX_ADEV_NAME_LEN]; + struct auxiliary_device *adev; + int err; + + iadev = (struct idc_rdma_vport_auxiliary_dev *) + kzalloc(sizeof(*iadev), GFP_KERNEL); + if (!iadev) + return -ENOMEM; + + adev = &iadev->adev; + vdev_info->adev = &iadev->adev; + iadev->vdev_info = vdev_info; + + adev->id = ida_alloc(&idpf_idc_ida, GFP_KERNEL); + if (adev->id < 0) { + pr_err("failed to allocate unique device ID for Auxiliary driver\n"); + err = -ENOMEM; + goto err_ida_alloc; + } + adev->dev.release = idpf_vport_adev_release; + adev->dev.parent = &cdev_info->pdev->dev; + sprintf(name, "%04x.rdma.vdev", cdev_info->pdev->vendor); + adev->name = name; + + err = auxiliary_device_init(adev); + if (err) + goto err_aux_dev_init; + + err = auxiliary_device_add(adev); + if (err) + goto err_aux_dev_add; + + return 0; + +err_aux_dev_add: + vdev_info->adev = NULL; + auxiliary_device_uninit(adev); +err_aux_dev_init: + ida_free(&idpf_idc_ida, adev->id); +err_ida_alloc: + kfree(iadev); + + return err; +} + +/** + * idpf_idc_init_aux_vport_dev - initialize vport Auxiliary Device(s) + * @vport: virtual port data struct + */ +static int idpf_idc_init_aux_vport_dev(struct idpf_vport *vport) +{ + struct idpf_adapter *adapter = vport->adapter; + struct idc_rdma_vport_dev_info *vdev_info; + struct idc_rdma_core_dev_info *cdev_info; + struct virtchnl2_create_vport *vport_msg; + int err; + + vport_msg = (struct virtchnl2_create_vport *) + adapter->vport_params_recvd[vport->idx]; + + if (!(le16_to_cpu(vport_msg->vport_flags) & VIRTCHNL2_VPORT_ENABLE_RDMA)) + return 0; + + vport->vdev_info = (struct idc_rdma_vport_dev_info *) + kzalloc(sizeof(*vdev_info), GFP_KERNEL); + if (!vport->vdev_info) + return -ENOMEM; + + cdev_info = vport->adapter->cdev_info; + + vdev_info = vport->vdev_info; + vdev_info->vport_id = vport->vport_id; + vdev_info->netdev = vport->netdev; + vdev_info->core_adev = cdev_info->adev; + + err = idpf_plug_vport_aux_dev(cdev_info, vdev_info); + if (err) { + kfree(vdev_info); + return err; + } + + return 0; +} + /** * idpf_core_adev_release - function to be mapped to aux dev's release op * @dev: pointer to device to free @@ -97,12 +202,58 @@ static int idpf_plug_core_aux_dev(struct idc_rdma_core_dev_info *cdev_info) */ static void idpf_unplug_aux_dev(struct auxiliary_device *adev) { + if (!adev) + return; + auxiliary_device_delete(adev); auxiliary_device_uninit(adev); ida_free(&idpf_idc_ida, adev->id); } +/** + * idpf_idc_vport_dev_up - called when CORE is ready for vport aux devs + * @adapter: private data struct + */ +static int idpf_idc_vport_dev_up(struct idpf_adapter *adapter) +{ + int i, err = 0; + + for (i = 0; i < adapter->num_alloc_vports; i++) { + struct idpf_vport *vport = adapter->vports[i]; + + if (!vport) + continue; + + if (!vport->vdev_info) + err = idpf_idc_init_aux_vport_dev(vport); + else + err = idpf_plug_vport_aux_dev(vport->adapter->cdev_info, + vport->vdev_info); + } + + return err; +} + +/** + * idpf_idc_vport_dev_down - called CORE is leaving vport aux dev support state + * @adapter: private data struct + */ +static void idpf_idc_vport_dev_down(struct idpf_adapter *adapter) +{ + int i; + + for (i = 0; i < adapter->num_alloc_vports; i++) { + struct idpf_vport *vport = adapter->vports[i]; + + if (!vport) + continue; + + idpf_unplug_aux_dev(vport->vdev_info->adev); + vport->vdev_info->adev = NULL; + } +} + /** * idpf_idc_vport_dev_ctrl - Called by an Auxiliary Driver * @cdev_info: idc core device info pointer @@ -116,7 +267,14 @@ static int idpf_idc_vport_dev_ctrl(struct idc_rdma_core_dev_info *cdev_info, bool up) { - return -EOPNOTSUPP; + struct idpf_adapter *adapter = pci_get_drvdata(cdev_info->pdev); + + if (up) + return idpf_idc_vport_dev_up(adapter); + + idpf_idc_vport_dev_down(adapter); + + return 0; } /** @@ -210,3 +368,17 @@ void idpf_idc_deinit_core_aux_device(struct idc_rdma_core_dev_info *cdev_info) kfree(cdev_info->mapped_mem_regions); kfree(cdev_info); } + +/** + * idpf_idc_deinit_vport_aux_device - de-initialize Auxiliary Device(s) + * @vdev_info: idc vport device info pointer + */ +void idpf_idc_deinit_vport_aux_device(struct idc_rdma_vport_dev_info *vdev_info) +{ + if (!vdev_info) + return; + + idpf_unplug_aux_dev(vdev_info->adev); + + kfree(vdev_info); +} diff --git a/drivers/net/ethernet/intel/idpf/idpf_lib.c b/drivers/net/ethernet/intel/idpf/idpf_lib.c index ff7dcbced76c..d4fb8b1652ae 100644 --- a/drivers/net/ethernet/intel/idpf/idpf_lib.c +++ b/drivers/net/ethernet/intel/idpf/idpf_lib.c @@ -1058,6 +1058,8 @@ static void idpf_vport_dealloc(struct idpf_vport *vport) struct idpf_adapter *adapter = vport->adapter; unsigned int i = vport->idx; + idpf_idc_deinit_vport_aux_device(vport->vdev_info); + idpf_deinit_mac_addr(vport); idpf_vport_stop(vport); diff --git a/drivers/net/ethernet/intel/idpf/virtchnl2.h b/drivers/net/ethernet/intel/idpf/virtchnl2.h index 80c17e4a394e..673a39e6698d 100644 --- a/drivers/net/ethernet/intel/idpf/virtchnl2.h +++ b/drivers/net/ethernet/intel/idpf/virtchnl2.h @@ -562,6 +562,15 @@ struct virtchnl2_queue_reg_chunks { }; VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_queue_reg_chunks); +/** + * enum virtchnl2_vport_flags - Vport flags + * @VIRTCHNL2_VPORT_ENABLE_RDMA: RDMA is enabled for this vport + */ +enum virtchnl2_vport_flags { + /* VIRTCHNL2_VPORT_* bits [0:3] rsvd */ + VIRTCHNL2_VPORT_ENABLE_RDMA = BIT(4), +}; + /** * struct virtchnl2_create_vport - Create vport config info. * @vport_type: See enum virtchnl2_vport_type. @@ -580,7 +589,7 @@ VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_queue_reg_chunks); * @max_mtu: Max MTU. CP populates this field on response. * @vport_id: Vport id. CP populates this field on response. * @default_mac_addr: Default MAC address. - * @pad: Padding. + * @vport_flags: See enum virtchnl2_vport_flags * @rx_desc_ids: See VIRTCHNL2_RX_DESC_IDS definitions. * @tx_desc_ids: See VIRTCHNL2_TX_DESC_IDS definitions. * @pad1: Padding. @@ -613,7 +622,7 @@ struct virtchnl2_create_vport { __le16 max_mtu; __le32 vport_id; u8 default_mac_addr[ETH_ALEN]; - __le16 pad; + __le16 vport_flags; __le64 rx_desc_ids; __le64 tx_desc_ids; u8 pad1[72];