diff mbox series

[for-next,4/8] RDMA/hns: Add check for the validity of sl configuration in UD SQ WQE

Message ID 1604057975-23388-5-git-send-email-liweihang@huawei.com (mailing list archive)
State Superseded
Delegated to: Jason Gunthorpe
Headers show
Series RDMA/hns: Support UD for HIP09 | expand

Commit Message

Weihang Li Oct. 30, 2020, 11:39 a.m. UTC 
From: Jiaran Zhang <zhangjiaran@huawei.com>

According to the RoCE v1 specification, the sl (service level) 0-7 are
mapped directly to priorities 0-7 respectively, sl 8-15 are reserved. The
driver should verify whether the value of sl is larger than 7, if so, an
exception should be returned.

Fixes: d6a3627e311c ("RDMA/hns: Optimize wqe buffer set flow for post send")
Signed-off-by: Jiaran Zhang <zhangjiaran@huawei.com>
Signed-off-by: Weihang Li <liweihang@huawei.com>
---
 drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

Comments

Jason Gunthorpe Nov. 12, 2020, 6:32 p.m. UTC | #1 
On Fri, Oct 30, 2020 at 07:39:31PM +0800, Weihang Li wrote:
> From: Jiaran Zhang <zhangjiaran@huawei.com>
> 
> According to the RoCE v1 specification, the sl (service level) 0-7 are
> mapped directly to priorities 0-7 respectively, sl 8-15 are reserved. The
> driver should verify whether the value of sl is larger than 7, if so, an
> exception should be returned.
> 
> Fixes: d6a3627e311c ("RDMA/hns: Optimize wqe buffer set flow for post send")
> Signed-off-by: Jiaran Zhang <zhangjiaran@huawei.com>
> Signed-off-by: Weihang Li <liweihang@huawei.com>
>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> index 7a1d30f..69386a5 100644
> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> @@ -427,9 +427,10 @@ static inline int set_ud_wqe(struct hns_roce_qp *qp,
>  			     void *wqe, unsigned int *sge_idx,
>  			     unsigned int owner_bit)
>  {
> -	struct hns_roce_dev *hr_dev = to_hr_dev(qp->ibqp.device);
>  	struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
>  	struct hns_roce_v2_ud_send_wqe *ud_sq_wqe = wqe;
> +	struct ib_device *ib_dev = qp->ibqp.device;
> +	struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
>  	unsigned int curr_idx = *sge_idx;
>  	int valid_num_sge;
>  	u32 msg_len = 0;
> @@ -489,6 +490,13 @@ static inline int set_ud_wqe(struct hns_roce_qp *qp,
>  		       V2_UD_SEND_WQE_BYTE_36_TCLASS_S, ah->av.tclass);
>  	roce_set_field(ud_sq_wqe->byte_40, V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
>  		       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, ah->av.flowlabel);
> +
> +	if (unlikely(ah->av.sl > MAX_SERVICE_LEVEL)) {
> +		ibdev_err(ib_dev,
> +			  "failed to fill ud av, ud sl (%d) shouldn't be larger than %d.\n",
> +			  ah->av.sl, MAX_SERVICE_LEVEL);
> +		return -EINVAL;
> +	}

We should not print for things like this, IIRC userspace can cause the
ah's sl to become set out of bounds

Jason
Weihang Li Nov. 14, 2020, 3:46 a.m. UTC | #2 
On 2020/11/13 2:33, Jason Gunthorpe wrote:
> On Fri, Oct 30, 2020 at 07:39:31PM +0800, Weihang Li wrote:
>> From: Jiaran Zhang <zhangjiaran@huawei.com>
>>
>> According to the RoCE v1 specification, the sl (service level) 0-7 are
>> mapped directly to priorities 0-7 respectively, sl 8-15 are reserved. The
>> driver should verify whether the value of sl is larger than 7, if so, an
>> exception should be returned.
>>
>> Fixes: d6a3627e311c ("RDMA/hns: Optimize wqe buffer set flow for post send")
>> Signed-off-by: Jiaran Zhang <zhangjiaran@huawei.com>
>> Signed-off-by: Weihang Li <liweihang@huawei.com>
>>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 10 +++++++++-
>>  1 file changed, 9 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>> index 7a1d30f..69386a5 100644
>> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>> @@ -427,9 +427,10 @@ static inline int set_ud_wqe(struct hns_roce_qp *qp,
>>  			     void *wqe, unsigned int *sge_idx,
>>  			     unsigned int owner_bit)
>>  {
>> -	struct hns_roce_dev *hr_dev = to_hr_dev(qp->ibqp.device);
>>  	struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
>>  	struct hns_roce_v2_ud_send_wqe *ud_sq_wqe = wqe;
>> +	struct ib_device *ib_dev = qp->ibqp.device;
>> +	struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
>>  	unsigned int curr_idx = *sge_idx;
>>  	int valid_num_sge;
>>  	u32 msg_len = 0;
>> @@ -489,6 +490,13 @@ static inline int set_ud_wqe(struct hns_roce_qp *qp,
>>  		       V2_UD_SEND_WQE_BYTE_36_TCLASS_S, ah->av.tclass);
>>  	roce_set_field(ud_sq_wqe->byte_40, V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
>>  		       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, ah->av.flowlabel);
>> +
>> +	if (unlikely(ah->av.sl > MAX_SERVICE_LEVEL)) {
>> +		ibdev_err(ib_dev,
>> +			  "failed to fill ud av, ud sl (%d) shouldn't be larger than %d.\n",
>> +			  ah->av.sl, MAX_SERVICE_LEVEL);
>> +		return -EINVAL;
>> +	}
> 
> We should not print for things like this, IIRC userspace can cause the
> ah's sl to become set out of bounds
> 
> Jason
> 

Hi Jason,

In "Annex A 16: RoCE", I found the following description:

	SL 0-7 are mapped directly to Priorities 0-7, respectively

	SL 8-15 are reserved.

	CA16-18: An attempt to use an Address Vector for a RoCE port containing
	a reserved SL value shall result in the Invalid Address Vector verb result.

So what should we do if the user wants to use the reserved sl? Should I just let it
do mask with 0x7 when creating AH?

Thanks
Weihang
Jason Gunthorpe Nov. 16, 2020, 4:09 p.m. UTC | #3 
On Sat, Nov 14, 2020 at 03:46:58AM +0000, liweihang wrote:
> On 2020/11/13 2:33, Jason Gunthorpe wrote:
> > On Fri, Oct 30, 2020 at 07:39:31PM +0800, Weihang Li wrote:
> >> From: Jiaran Zhang <zhangjiaran@huawei.com>
> >>
> >> According to the RoCE v1 specification, the sl (service level) 0-7 are
> >> mapped directly to priorities 0-7 respectively, sl 8-15 are reserved. The
> >> driver should verify whether the value of sl is larger than 7, if so, an
> >> exception should be returned.
> >>
> >> Fixes: d6a3627e311c ("RDMA/hns: Optimize wqe buffer set flow for post send")
> >> Signed-off-by: Jiaran Zhang <zhangjiaran@huawei.com>
> >> Signed-off-by: Weihang Li <liweihang@huawei.com>
> >>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 10 +++++++++-
> >>  1 file changed, 9 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> index 7a1d30f..69386a5 100644
> >> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> @@ -427,9 +427,10 @@ static inline int set_ud_wqe(struct hns_roce_qp *qp,
> >>  			     void *wqe, unsigned int *sge_idx,
> >>  			     unsigned int owner_bit)
> >>  {
> >> -	struct hns_roce_dev *hr_dev = to_hr_dev(qp->ibqp.device);
> >>  	struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
> >>  	struct hns_roce_v2_ud_send_wqe *ud_sq_wqe = wqe;
> >> +	struct ib_device *ib_dev = qp->ibqp.device;
> >> +	struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
> >>  	unsigned int curr_idx = *sge_idx;
> >>  	int valid_num_sge;
> >>  	u32 msg_len = 0;
> >> @@ -489,6 +490,13 @@ static inline int set_ud_wqe(struct hns_roce_qp *qp,
> >>  		       V2_UD_SEND_WQE_BYTE_36_TCLASS_S, ah->av.tclass);
> >>  	roce_set_field(ud_sq_wqe->byte_40, V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
> >>  		       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, ah->av.flowlabel);
> >> +
> >> +	if (unlikely(ah->av.sl > MAX_SERVICE_LEVEL)) {
> >> +		ibdev_err(ib_dev,
> >> +			  "failed to fill ud av, ud sl (%d) shouldn't be larger than %d.\n",
> >> +			  ah->av.sl, MAX_SERVICE_LEVEL);
> >> +		return -EINVAL;
> >> +	}
> > 
> > We should not print for things like this, IIRC userspace can cause the
> > ah's sl to become set out of bounds

 
> In "Annex A 16: RoCE", I found the following description:
> 
> 	SL 0-7 are mapped directly to Priorities 0-7, respectively
> 
> 	SL 8-15 are reserved.
> 
> 	CA16-18: An attempt to use an Address Vector for a RoCE port containing
> 	a reserved SL value shall result in the Invalid Address Vector verb result.
> 
> So what should we do if the user wants to use the reserved sl? Should I just let it
> do mask with 0x7 when creating AH?

Fail and don't print anything

Jason
Weihang Li Nov. 17, 2020, 6:42 a.m. UTC | #4 
On 2020/11/17 0:09, Jason Gunthorpe wrote:
> On Sat, Nov 14, 2020 at 03:46:58AM +0000, liweihang wrote:
>> On 2020/11/13 2:33, Jason Gunthorpe wrote:
>>> On Fri, Oct 30, 2020 at 07:39:31PM +0800, Weihang Li wrote:
>>>> From: Jiaran Zhang <zhangjiaran@huawei.com>
>>>>
>>>> According to the RoCE v1 specification, the sl (service level) 0-7 are
>>>> mapped directly to priorities 0-7 respectively, sl 8-15 are reserved. The
>>>> driver should verify whether the value of sl is larger than 7, if so, an
>>>> exception should be returned.
>>>>
>>>> Fixes: d6a3627e311c ("RDMA/hns: Optimize wqe buffer set flow for post send")
>>>> Signed-off-by: Jiaran Zhang <zhangjiaran@huawei.com>
>>>> Signed-off-by: Weihang Li <liweihang@huawei.com>
>>>>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 10 +++++++++-
>>>>  1 file changed, 9 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>>>> index 7a1d30f..69386a5 100644
>>>> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>>>> @@ -427,9 +427,10 @@ static inline int set_ud_wqe(struct hns_roce_qp *qp,
>>>>  			     void *wqe, unsigned int *sge_idx,
>>>>  			     unsigned int owner_bit)
>>>>  {
>>>> -	struct hns_roce_dev *hr_dev = to_hr_dev(qp->ibqp.device);
>>>>  	struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
>>>>  	struct hns_roce_v2_ud_send_wqe *ud_sq_wqe = wqe;
>>>> +	struct ib_device *ib_dev = qp->ibqp.device;
>>>> +	struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
>>>>  	unsigned int curr_idx = *sge_idx;
>>>>  	int valid_num_sge;
>>>>  	u32 msg_len = 0;
>>>> @@ -489,6 +490,13 @@ static inline int set_ud_wqe(struct hns_roce_qp *qp,
>>>>  		       V2_UD_SEND_WQE_BYTE_36_TCLASS_S, ah->av.tclass);
>>>>  	roce_set_field(ud_sq_wqe->byte_40, V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
>>>>  		       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, ah->av.flowlabel);
>>>> +
>>>> +	if (unlikely(ah->av.sl > MAX_SERVICE_LEVEL)) {
>>>> +		ibdev_err(ib_dev,
>>>> +			  "failed to fill ud av, ud sl (%d) shouldn't be larger than %d.\n",
>>>> +			  ah->av.sl, MAX_SERVICE_LEVEL);
>>>> +		return -EINVAL;
>>>> +	}
>>>
>>> We should not print for things like this, IIRC userspace can cause the
>>> ah's sl to become set out of bounds
> 
>  
>> In "Annex A 16: RoCE", I found the following description:
>>
>> 	SL 0-7 are mapped directly to Priorities 0-7, respectively
>>
>> 	SL 8-15 are reserved.
>>
>> 	CA16-18: An attempt to use an Address Vector for a RoCE port containing
>> 	a reserved SL value shall result in the Invalid Address Vector verb result.
>>
>> So what should we do if the user wants to use the reserved sl? Should I just let it
>> do mask with 0x7 when creating AH?
> 
> Fail and don't print anything
> 
> Jason
> 

OK, thank you.

Weihang
diff mbox series

Patch

diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 7a1d30f..69386a5 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -427,9 +427,10 @@  static inline int set_ud_wqe(struct hns_roce_qp *qp,
 			     void *wqe, unsigned int *sge_idx,
 			     unsigned int owner_bit)
 {
-	struct hns_roce_dev *hr_dev = to_hr_dev(qp->ibqp.device);
 	struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
 	struct hns_roce_v2_ud_send_wqe *ud_sq_wqe = wqe;
+	struct ib_device *ib_dev = qp->ibqp.device;
+	struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
 	unsigned int curr_idx = *sge_idx;
 	int valid_num_sge;
 	u32 msg_len = 0;
@@ -489,6 +490,13 @@  static inline int set_ud_wqe(struct hns_roce_qp *qp,
 		       V2_UD_SEND_WQE_BYTE_36_TCLASS_S, ah->av.tclass);
 	roce_set_field(ud_sq_wqe->byte_40, V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
 		       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, ah->av.flowlabel);
+
+	if (unlikely(ah->av.sl > MAX_SERVICE_LEVEL)) {
+		ibdev_err(ib_dev,
+			  "failed to fill ud av, ud sl (%d) shouldn't be larger than %d.\n",
+			  ah->av.sl, MAX_SERVICE_LEVEL);
+		return -EINVAL;
+	}
 	roce_set_field(ud_sq_wqe->byte_40, V2_UD_SEND_WQE_BYTE_40_SL_M,
 		       V2_UD_SEND_WQE_BYTE_40_SL_S, ah->av.sl);
 	roce_set_field(ud_sq_wqe->byte_40, V2_UD_SEND_WQE_BYTE_40_PORTN_M,