From patchwork Wed Jan 5 08:07:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12703609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DBF1C433EF for ; Tue, 4 Jan 2022 15:44:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233955AbiADPor (ORCPT ); Tue, 4 Jan 2022 10:44:47 -0500 Received: from mga18.intel.com ([134.134.136.126]:35449 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234998AbiADPok (ORCPT ); Tue, 4 Jan 2022 10:44:40 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10216"; a="229062515" X-IronPort-AV: E=Sophos;i="5.88,261,1635231600"; d="scan'208";a="229062515" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 07:44:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,261,1635231600"; d="scan'208";a="526089390" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga008.jf.intel.com with ESMTP; 04 Jan 2022 07:44:37 -0800 From: yanjun.zhu@linux.dev To: liangwenpeng@huawei.com, liweihang@huawei.com, jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev Subject: [PATCH 1/4] RDMA/core: Calculate UDP source port based on flow label or lqpn/rqpn Date: Wed, 5 Jan 2022 03:07:24 -0500 Message-Id: <20220105080727.2143737-2-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220105080727.2143737-1-yanjun.zhu@linux.dev> References: <20220105080727.2143737-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Calculate and set UDP source port based on the flow label. If flow label is not defined in GRH then calculate it based on lqpn/rqpn. Signed-off-by: Zhu Yanjun --- include/rdma/ib_verbs.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 6e9ad656ecb7..2f122aa81f0f 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -4749,6 +4749,23 @@ static inline u32 rdma_calc_flow_label(u32 lqpn, u32 rqpn) return (u32)(v & IB_GRH_FLOWLABEL_MASK); } +/** + * rdma_get_udp_sport - Calculate and set UDP source port based on the flow + * label. If flow label is not defined in GRH then + * calculate it based on lqpn/rqpn. + * + * @fl: flow label from GRH + * @lqpn: local qp number + * @rqpn: remote qp number + */ +static inline u16 rdma_get_udp_sport(u32 fl, u32 lqpn, u32 rqpn) +{ + if (!fl) + fl = rdma_calc_flow_label(lqpn, rqpn); + + return rdma_flow_label_to_udp_sport(fl); +} + const struct ib_port_immutable* ib_port_immutable_read(struct ib_device *dev, unsigned int port); #endif /* IB_VERBS_H */ From patchwork Wed Jan 5 08:07:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12703612 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72976C433FE for ; Tue, 4 Jan 2022 15:44:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232657AbiADPor (ORCPT ); Tue, 4 Jan 2022 10:44:47 -0500 Received: from mga18.intel.com ([134.134.136.126]:35449 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235053AbiADPom (ORCPT ); Tue, 4 Jan 2022 10:44:42 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10216"; a="229062518" X-IronPort-AV: E=Sophos;i="5.88,261,1635231600"; d="scan'208";a="229062518" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 07:44:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,261,1635231600"; d="scan'208";a="526089395" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga008.jf.intel.com with ESMTP; 04 Jan 2022 07:44:39 -0800 From: yanjun.zhu@linux.dev To: liangwenpeng@huawei.com, liweihang@huawei.com, jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev Subject: [PATCH 2/4] RDMA/hns: Replace get_udp_sport with rdma_get_udp_sport Date: Wed, 5 Jan 2022 03:07:25 -0500 Message-Id: <20220105080727.2143737-3-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220105080727.2143737-1-yanjun.zhu@linux.dev> References: <20220105080727.2143737-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Several drivers have the same function xxx_get_udp_sport. So this function is moved to ib_verbs.h. Signed-off-by: Zhu Yanjun --- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index eb0defa80d0d..cb795663b813 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -4488,14 +4488,6 @@ static int modify_qp_rtr_to_rts(struct ib_qp *ibqp, return 0; } -static inline u16 get_udp_sport(u32 fl, u32 lqpn, u32 rqpn) -{ - if (!fl) - fl = rdma_calc_flow_label(lqpn, rqpn); - - return rdma_flow_label_to_udp_sport(fl); -} - static int get_dip_ctx_idx(struct ib_qp *ibqp, const struct ib_qp_attr *attr, u32 *dip_idx) { @@ -4712,8 +4704,8 @@ static int hns_roce_v2_set_path(struct ib_qp *ibqp, } hr_reg_write(context, QPC_UDPSPN, - is_udp ? get_udp_sport(grh->flow_label, ibqp->qp_num, - attr->dest_qp_num) : 0); + is_udp ? rdma_get_udp_sport(grh->flow_label, ibqp->qp_num, + attr->dest_qp_num) : 0); hr_reg_clear(qpc_mask, QPC_UDPSPN); From patchwork Wed Jan 5 08:07:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12703610 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C762C4332F for ; Tue, 4 Jan 2022 15:44:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229778AbiADPor (ORCPT ); Tue, 4 Jan 2022 10:44:47 -0500 Received: from mga18.intel.com ([134.134.136.126]:35449 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235064AbiADPoo (ORCPT ); Tue, 4 Jan 2022 10:44:44 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10216"; a="229062525" X-IronPort-AV: E=Sophos;i="5.88,261,1635231600"; d="scan'208";a="229062525" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 07:44:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,261,1635231600"; d="scan'208";a="526089404" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga008.jf.intel.com with ESMTP; 04 Jan 2022 07:44:42 -0800 From: yanjun.zhu@linux.dev To: liangwenpeng@huawei.com, liweihang@huawei.com, jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev Subject: [PATCH 3/4] RDMA/irdma: Make the source udp port vary Date: Wed, 5 Jan 2022 03:07:26 -0500 Message-Id: <20220105080727.2143737-4-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220105080727.2143737-1-yanjun.zhu@linux.dev> References: <20220105080727.2143737-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues. Signed-off-by: Zhu Yanjun --- drivers/infiniband/hw/irdma/verbs.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c index 8cd5f9261692..09dba7ed5ab9 100644 --- a/drivers/infiniband/hw/irdma/verbs.c +++ b/drivers/infiniband/hw/irdma/verbs.c @@ -1167,6 +1167,11 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr, memset(&iwqp->roce_ah, 0, sizeof(iwqp->roce_ah)); if (attr->ah_attr.ah_flags & IB_AH_GRH) { + u32 fl = attr->ah_attr.grh.flow_label; + u32 lqp = ibqp->qp_num; + u32 rqp = roce_info->dest_qp; + + udp_info->src_port = rdma_get_udp_sport(fl, lqp, rqp); udp_info->ttl = attr->ah_attr.grh.hop_limit; udp_info->flow_label = attr->ah_attr.grh.flow_label; udp_info->tos = attr->ah_attr.grh.traffic_class; From patchwork Wed Jan 5 08:07:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12703611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97302C43219 for ; Tue, 4 Jan 2022 15:44:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234737AbiADPot (ORCPT ); Tue, 4 Jan 2022 10:44:49 -0500 Received: from mga18.intel.com ([134.134.136.126]:35449 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233957AbiADPos (ORCPT ); Tue, 4 Jan 2022 10:44:48 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10216"; a="229062539" X-IronPort-AV: E=Sophos;i="5.88,261,1635231600"; d="scan'208";a="229062539" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 07:44:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,261,1635231600"; d="scan'208";a="526089413" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga008.jf.intel.com with ESMTP; 04 Jan 2022 07:44:44 -0800 From: yanjun.zhu@linux.dev To: liangwenpeng@huawei.com, liweihang@huawei.com, jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev Subject: [PATCH 4/4] RDMA/rxe: Use the standard method to produce udp source port Date: Wed, 5 Jan 2022 03:07:27 -0500 Message-Id: <20220105080727.2143737-5-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220105080727.2143737-1-yanjun.zhu@linux.dev> References: <20220105080727.2143737-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Use the standard method to produce udp source port. Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe_verbs.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 0aa0d7e52773..f30d98ad13cd 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -469,6 +469,16 @@ static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, if (err) goto err1; + if (mask & IB_QP_AV) { + if (attr->ah_attr.ah_flags & IB_AH_GRH) { + u32 fl = attr->ah_attr.grh.flow_label; + u32 lqp = qp->ibqp.qp_num; + u32 rqp = qp->attr.dest_qp_num; + + qp->src_port = rdma_get_udp_sport(fl, lqp, rqp); + } + } + return 0; err1: