diff mbox series

[v4,for-next] RDMA/core: Get IB width and speed from netdev

Message ID 20230721092052.2090449-1-huangjunxian6@hisilicon.com (mailing list archive)
State Accepted
Headers show
Series [v4,for-next] RDMA/core: Get IB width and speed from netdev | expand

Commit Message

Junxian Huang July 21, 2023, 9:20 a.m. UTC 
From: Haoyue Xu <xuhaoyue1@hisilicon.com>

Previously, there was no way to query the number of lanes for a network
card, so the same netdev_speed would result in a fixed pair of width and
speed. As network card specifications become more diverse, such fixed
mode is no longer suitable, so a method is needed to obtain the correct
width and speed based on the number of lanes.

This patch retrieves netdev lanes and speed from net_device and
translates them to IB width and speed.

Signed-off-by: Haoyue Xu <xuhaoyue1@hisilicon.com>
Signed-off-by: Luoyouming <luoyouming@huawei.com>
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
---
 drivers/infiniband/core/verbs.c | 100 +++++++++++++++++++++++++-------
 1 file changed, 79 insertions(+), 21 deletions(-)

Comments

Leon Romanovsky July 24, 2023, 11:19 a.m. UTC | #1 
On Fri, Jul 21, 2023 at 05:20:52PM +0800, Junxian Huang wrote:
> From: Haoyue Xu <xuhaoyue1@hisilicon.com>
> 
> Previously, there was no way to query the number of lanes for a network
> card, so the same netdev_speed would result in a fixed pair of width and
> speed. As network card specifications become more diverse, such fixed
> mode is no longer suitable, so a method is needed to obtain the correct
> width and speed based on the number of lanes.
> 
> This patch retrieves netdev lanes and speed from net_device and
> translates them to IB width and speed.
> 
> Signed-off-by: Haoyue Xu <xuhaoyue1@hisilicon.com>
> Signed-off-by: Luoyouming <luoyouming@huawei.com>
> Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
> ---
>  drivers/infiniband/core/verbs.c | 100 +++++++++++++++++++++++++-------
>  1 file changed, 79 insertions(+), 21 deletions(-)
> 
> diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
> index b99b3cc283b6..25367bd6dd97 100644
> --- a/drivers/infiniband/core/verbs.c
> +++ b/drivers/infiniband/core/verbs.c
> @@ -1880,6 +1880,80 @@ int ib_modify_qp_with_udata(struct ib_qp *ib_qp, struct ib_qp_attr *attr,
>  }
>  EXPORT_SYMBOL(ib_modify_qp_with_udata);
>  
> +static void ib_get_width_and_speed(u32 netdev_speed, u32 lanes,
> +				   u16 *speed, u8 *width)

<...>

> +	switch (netdev_speed / lanes) {
> +	case SPEED_2500:
> +		*speed = IB_SPEED_SDR;
> +		break;
> +	case SPEED_5000:
> +		*speed = IB_SPEED_DDR;
> +		break;
> +	case SPEED_10000:
> +		*speed = IB_SPEED_FDR10;
> +		break;
> +	case SPEED_14000:
> +		*speed = IB_SPEED_FDR;
> +		break;
> +	case SPEED_25000:
> +		*speed = IB_SPEED_EDR;
> +		break;
> +	case SPEED_50000:
> +		*speed = IB_SPEED_HDR;
> +		break;
> +	case SPEED_100000:
> +		*speed = IB_SPEED_NDR;
> +		break;
> +	default:
> +		*speed = IB_SPEED_SDR;
> +	}

How did you come to these translation values?

Thanks
Junxian Huang July 27, 2023, 3:44 a.m. UTC | #2 
On 2023/7/24 19:19, Leon Romanovsky wrote:
> On Fri, Jul 21, 2023 at 05:20:52PM +0800, Junxian Huang wrote:
>> From: Haoyue Xu <xuhaoyue1@hisilicon.com>
>>
>> Previously, there was no way to query the number of lanes for a network
>> card, so the same netdev_speed would result in a fixed pair of width and
>> speed. As network card specifications become more diverse, such fixed
>> mode is no longer suitable, so a method is needed to obtain the correct
>> width and speed based on the number of lanes.
>>
>> This patch retrieves netdev lanes and speed from net_device and
>> translates them to IB width and speed.
>>
>> Signed-off-by: Haoyue Xu <xuhaoyue1@hisilicon.com>
>> Signed-off-by: Luoyouming <luoyouming@huawei.com>
>> Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
>> ---
>>  drivers/infiniband/core/verbs.c | 100 +++++++++++++++++++++++++-------
>>  1 file changed, 79 insertions(+), 21 deletions(-)
>>
>> diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
>> index b99b3cc283b6..25367bd6dd97 100644
>> --- a/drivers/infiniband/core/verbs.c
>> +++ b/drivers/infiniband/core/verbs.c
>> @@ -1880,6 +1880,80 @@ int ib_modify_qp_with_udata(struct ib_qp *ib_qp, struct ib_qp_attr *attr,
>>  }
>>  EXPORT_SYMBOL(ib_modify_qp_with_udata);
>>  
>> +static void ib_get_width_and_speed(u32 netdev_speed, u32 lanes,
>> +				   u16 *speed, u8 *width)
> 
> <...>
> 
>> +	switch (netdev_speed / lanes) {
>> +	case SPEED_2500:
>> +		*speed = IB_SPEED_SDR;
>> +		break;
>> +	case SPEED_5000:
>> +		*speed = IB_SPEED_DDR;
>> +		break;
>> +	case SPEED_10000:
>> +		*speed = IB_SPEED_FDR10;
>> +		break;
>> +	case SPEED_14000:
>> +		*speed = IB_SPEED_FDR;
>> +		break;
>> +	case SPEED_25000:
>> +		*speed = IB_SPEED_EDR;
>> +		break;
>> +	case SPEED_50000:
>> +		*speed = IB_SPEED_HDR;
>> +		break;
>> +	case SPEED_100000:
>> +		*speed = IB_SPEED_NDR;
>> +		break;
>> +	default:
>> +		*speed = IB_SPEED_SDR;
>> +	}
> 
> How did you come to these translation values?
> 
> Thanks

The IB spec defines the mapping relationship between IB speed and transfer
rate. For example, if the transfer rate of is 2.5Gbps(SPEED_2500), the IB
speed will be set to IB_SPEED_SDR.

Junxian
Leon Romanovsky July 27, 2023, 6:58 a.m. UTC | #3 
On Thu, Jul 27, 2023 at 11:44:50AM +0800, Junxian Huang wrote:
> 
> 
> On 2023/7/24 19:19, Leon Romanovsky wrote:
> > On Fri, Jul 21, 2023 at 05:20:52PM +0800, Junxian Huang wrote:
> >> From: Haoyue Xu <xuhaoyue1@hisilicon.com>
> >>
> >> Previously, there was no way to query the number of lanes for a network
> >> card, so the same netdev_speed would result in a fixed pair of width and
> >> speed. As network card specifications become more diverse, such fixed
> >> mode is no longer suitable, so a method is needed to obtain the correct
> >> width and speed based on the number of lanes.
> >>
> >> This patch retrieves netdev lanes and speed from net_device and
> >> translates them to IB width and speed.
> >>
> >> Signed-off-by: Haoyue Xu <xuhaoyue1@hisilicon.com>
> >> Signed-off-by: Luoyouming <luoyouming@huawei.com>
> >> Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
> >> ---
> >>  drivers/infiniband/core/verbs.c | 100 +++++++++++++++++++++++++-------
> >>  1 file changed, 79 insertions(+), 21 deletions(-)
> >>
> >> diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
> >> index b99b3cc283b6..25367bd6dd97 100644
> >> --- a/drivers/infiniband/core/verbs.c
> >> +++ b/drivers/infiniband/core/verbs.c
> >> @@ -1880,6 +1880,80 @@ int ib_modify_qp_with_udata(struct ib_qp *ib_qp, struct ib_qp_attr *attr,
> >>  }
> >>  EXPORT_SYMBOL(ib_modify_qp_with_udata);
> >>  
> >> +static void ib_get_width_and_speed(u32 netdev_speed, u32 lanes,
> >> +				   u16 *speed, u8 *width)
> > 
> > <...>
> > 
> >> +	switch (netdev_speed / lanes) {
> >> +	case SPEED_2500:
> >> +		*speed = IB_SPEED_SDR;
> >> +		break;
> >> +	case SPEED_5000:
> >> +		*speed = IB_SPEED_DDR;
> >> +		break;
> >> +	case SPEED_10000:
> >> +		*speed = IB_SPEED_FDR10;
> >> +		break;
> >> +	case SPEED_14000:
> >> +		*speed = IB_SPEED_FDR;
> >> +		break;
> >> +	case SPEED_25000:
> >> +		*speed = IB_SPEED_EDR;
> >> +		break;
> >> +	case SPEED_50000:
> >> +		*speed = IB_SPEED_HDR;
> >> +		break;
> >> +	case SPEED_100000:
> >> +		*speed = IB_SPEED_NDR;
> >> +		break;
> >> +	default:
> >> +		*speed = IB_SPEED_SDR;
> >> +	}
> > 
> > How did you come to these translation values?
> > 
> > Thanks
> 
> The IB spec defines the mapping relationship between IB speed and transfer
> rate. For example, if the transfer rate of is 2.5Gbps(SPEED_2500), the IB
> speed will be set to IB_SPEED_SDR.

Are you referring to "Table 250 - Enumeration of the Rate"?

Thanks

> 
> Junxian
Junxian Huang July 27, 2023, 7:53 a.m. UTC | #4 
On 2023/7/27 14:58, Leon Romanovsky wrote:
> On Thu, Jul 27, 2023 at 11:44:50AM +0800, Junxian Huang wrote:
>>
>>
>> On 2023/7/24 19:19, Leon Romanovsky wrote:
>>> On Fri, Jul 21, 2023 at 05:20:52PM +0800, Junxian Huang wrote:
>>>> From: Haoyue Xu <xuhaoyue1@hisilicon.com>
>>>>
>>>> Previously, there was no way to query the number of lanes for a network
>>>> card, so the same netdev_speed would result in a fixed pair of width and
>>>> speed. As network card specifications become more diverse, such fixed
>>>> mode is no longer suitable, so a method is needed to obtain the correct
>>>> width and speed based on the number of lanes.
>>>>
>>>> This patch retrieves netdev lanes and speed from net_device and
>>>> translates them to IB width and speed.
>>>>
>>>> Signed-off-by: Haoyue Xu <xuhaoyue1@hisilicon.com>
>>>> Signed-off-by: Luoyouming <luoyouming@huawei.com>
>>>> Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
>>>> ---
>>>>  drivers/infiniband/core/verbs.c | 100 +++++++++++++++++++++++++-------
>>>>  1 file changed, 79 insertions(+), 21 deletions(-)
>>>>
>>>> diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
>>>> index b99b3cc283b6..25367bd6dd97 100644
>>>> --- a/drivers/infiniband/core/verbs.c
>>>> +++ b/drivers/infiniband/core/verbs.c
>>>> @@ -1880,6 +1880,80 @@ int ib_modify_qp_with_udata(struct ib_qp *ib_qp, struct ib_qp_attr *attr,
>>>>  }
>>>>  EXPORT_SYMBOL(ib_modify_qp_with_udata);
>>>>  
>>>> +static void ib_get_width_and_speed(u32 netdev_speed, u32 lanes,
>>>> +				   u16 *speed, u8 *width)
>>>
>>> <...>
>>>
>>>> +	switch (netdev_speed / lanes) {
>>>> +	case SPEED_2500:
>>>> +		*speed = IB_SPEED_SDR;
>>>> +		break;
>>>> +	case SPEED_5000:
>>>> +		*speed = IB_SPEED_DDR;
>>>> +		break;
>>>> +	case SPEED_10000:
>>>> +		*speed = IB_SPEED_FDR10;
>>>> +		break;
>>>> +	case SPEED_14000:
>>>> +		*speed = IB_SPEED_FDR;
>>>> +		break;
>>>> +	case SPEED_25000:
>>>> +		*speed = IB_SPEED_EDR;
>>>> +		break;
>>>> +	case SPEED_50000:
>>>> +		*speed = IB_SPEED_HDR;
>>>> +		break;
>>>> +	case SPEED_100000:
>>>> +		*speed = IB_SPEED_NDR;
>>>> +		break;
>>>> +	default:
>>>> +		*speed = IB_SPEED_SDR;
>>>> +	}
>>>
>>> How did you come to these translation values?
>>>
>>> Thanks
>>
>> The IB spec defines the mapping relationship between IB speed and transfer
>> rate. For example, if the transfer rate of is 2.5Gbps(SPEED_2500), the IB
>> speed will be set to IB_SPEED_SDR.
> 
> Are you referring to "Table 250 - Enumeration of the Rate"?
> 
> Thanks
> 
>>
>> Junxian

Yes.

Junxian
Leon Romanovsky July 30, 2023, 12:06 p.m. UTC | #5 
On Fri, 21 Jul 2023 17:20:52 +0800, Junxian Huang wrote:
> Previously, there was no way to query the number of lanes for a network
> card, so the same netdev_speed would result in a fixed pair of width and
> speed. As network card specifications become more diverse, such fixed
> mode is no longer suitable, so a method is needed to obtain the correct
> width and speed based on the number of lanes.
> 
> This patch retrieves netdev lanes and speed from net_device and
> translates them to IB width and speed.
> 
> [...]

Applied, thanks!

[1/1] RDMA/core: Get IB width and speed from netdev
      https://git.kernel.org/rdma/rdma/c/cb06b6b3f6cbc5

Best regards,
diff mbox series

Patch

diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index b99b3cc283b6..25367bd6dd97 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -1880,6 +1880,80 @@  int ib_modify_qp_with_udata(struct ib_qp *ib_qp, struct ib_qp_attr *attr,
 }
 EXPORT_SYMBOL(ib_modify_qp_with_udata);
 
+static void ib_get_width_and_speed(u32 netdev_speed, u32 lanes,
+				   u16 *speed, u8 *width)
+{
+	if (!lanes) {
+		if (netdev_speed <= SPEED_1000) {
+			*width = IB_WIDTH_1X;
+			*speed = IB_SPEED_SDR;
+		} else if (netdev_speed <= SPEED_10000) {
+			*width = IB_WIDTH_1X;
+			*speed = IB_SPEED_FDR10;
+		} else if (netdev_speed <= SPEED_20000) {
+			*width = IB_WIDTH_4X;
+			*speed = IB_SPEED_DDR;
+		} else if (netdev_speed <= SPEED_25000) {
+			*width = IB_WIDTH_1X;
+			*speed = IB_SPEED_EDR;
+		} else if (netdev_speed <= SPEED_40000) {
+			*width = IB_WIDTH_4X;
+			*speed = IB_SPEED_FDR10;
+		} else {
+			*width = IB_WIDTH_4X;
+			*speed = IB_SPEED_EDR;
+		}
+
+		return;
+	}
+
+	switch (lanes) {
+	case 1:
+		*width = IB_WIDTH_1X;
+		break;
+	case 2:
+		*width = IB_WIDTH_2X;
+		break;
+	case 4:
+		*width = IB_WIDTH_4X;
+		break;
+	case 8:
+		*width = IB_WIDTH_8X;
+		break;
+	case 12:
+		*width = IB_WIDTH_12X;
+		break;
+	default:
+		*width = IB_WIDTH_1X;
+	}
+
+	switch (netdev_speed / lanes) {
+	case SPEED_2500:
+		*speed = IB_SPEED_SDR;
+		break;
+	case SPEED_5000:
+		*speed = IB_SPEED_DDR;
+		break;
+	case SPEED_10000:
+		*speed = IB_SPEED_FDR10;
+		break;
+	case SPEED_14000:
+		*speed = IB_SPEED_FDR;
+		break;
+	case SPEED_25000:
+		*speed = IB_SPEED_EDR;
+		break;
+	case SPEED_50000:
+		*speed = IB_SPEED_HDR;
+		break;
+	case SPEED_100000:
+		*speed = IB_SPEED_NDR;
+		break;
+	default:
+		*speed = IB_SPEED_SDR;
+	}
+}
+
 int ib_get_eth_speed(struct ib_device *dev, u32 port_num, u16 *speed, u8 *width)
 {
 	int rc;
@@ -1904,29 +1978,13 @@  int ib_get_eth_speed(struct ib_device *dev, u32 port_num, u16 *speed, u8 *width)
 		netdev_speed = lksettings.base.speed;
 	} else {
 		netdev_speed = SPEED_1000;
-		pr_warn("%s speed is unknown, defaulting to %u\n", netdev->name,
-			netdev_speed);
+		if (rc)
+			pr_warn("%s speed is unknown, defaulting to %u\n",
+				netdev->name, netdev_speed);
 	}
 
-	if (netdev_speed <= SPEED_1000) {
-		*width = IB_WIDTH_1X;
-		*speed = IB_SPEED_SDR;
-	} else if (netdev_speed <= SPEED_10000) {
-		*width = IB_WIDTH_1X;
-		*speed = IB_SPEED_FDR10;
-	} else if (netdev_speed <= SPEED_20000) {
-		*width = IB_WIDTH_4X;
-		*speed = IB_SPEED_DDR;
-	} else if (netdev_speed <= SPEED_25000) {
-		*width = IB_WIDTH_1X;
-		*speed = IB_SPEED_EDR;
-	} else if (netdev_speed <= SPEED_40000) {
-		*width = IB_WIDTH_4X;
-		*speed = IB_SPEED_FDR10;
-	} else {
-		*width = IB_WIDTH_4X;
-		*speed = IB_SPEED_EDR;
-	}
+	ib_get_width_and_speed(netdev_speed, lksettings.lanes,
+			       speed, width);
 
 	return 0;
 }