From patchwork Thu Jan 6 18:03:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12704989 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C0ABC433F5 for ; Thu, 6 Jan 2022 01:41:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344417AbiAFBlQ (ORCPT ); Wed, 5 Jan 2022 20:41:16 -0500 Received: from mga01.intel.com ([192.55.52.88]:62226 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344412AbiAFBlP (ORCPT ); Wed, 5 Jan 2022 20:41:15 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10217"; a="266855319" X-IronPort-AV: E=Sophos;i="5.88,265,1635231600"; d="scan'208";a="266855319" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jan 2022 17:41:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,265,1635231600"; d="scan'208";a="513226692" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga007.jf.intel.com with ESMTP; 05 Jan 2022 17:41:12 -0800 From: yanjun.zhu@linux.dev To: jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev, leonro@nvidia.com Subject: [PATCH v3 1/4] RDMA/core: Calculate UDP source port based on flow label or lqpn/rqpn Date: Thu, 6 Jan 2022 13:03:56 -0500 Message-Id: <20220106180359.2915060-2-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220106180359.2915060-1-yanjun.zhu@linux.dev> References: <20220106180359.2915060-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Calculate and set UDP source port based on the flow label. If flow label is not defined in GRH then calculate it based on lqpn/rqpn. Signed-off-by: Zhu Yanjun Reviewed-by: Leon Romanovsky --- include/rdma/ib_verbs.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 6e9ad656ecb7..69d883f7fb41 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -4749,6 +4749,23 @@ static inline u32 rdma_calc_flow_label(u32 lqpn, u32 rqpn) return (u32)(v & IB_GRH_FLOWLABEL_MASK); } +/** + * rdma_get_udp_sport - Calculate and set UDP source port based on the flow + * label. If flow label is not defined in GRH then + * calculate it based on lqpn/rqpn. + * + * @fl: flow label from GRH + * @lqpn: local qp number + * @rqpn: remote qp number + */ +static inline u16 rdma_get_udp_sport(u32 fl, u32 lqpn, u32 rqpn) +{ + if (!fl) + fl = rdma_calc_flow_label(lqpn, rqpn); + + return rdma_flow_label_to_udp_sport(fl); +} + const struct ib_port_immutable* ib_port_immutable_read(struct ib_device *dev, unsigned int port); #endif /* IB_VERBS_H */ From patchwork Thu Jan 6 18:03:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12704990 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2546C433FE for ; Thu, 6 Jan 2022 01:41:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344412AbiAFBlS (ORCPT ); Wed, 5 Jan 2022 20:41:18 -0500 Received: from mga01.intel.com ([192.55.52.88]:62230 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344425AbiAFBlR (ORCPT ); Wed, 5 Jan 2022 20:41:17 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10217"; a="266855324" X-IronPort-AV: E=Sophos;i="5.88,265,1635231600"; d="scan'208";a="266855324" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jan 2022 17:41:16 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,265,1635231600"; d="scan'208";a="513226702" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga007.jf.intel.com with ESMTP; 05 Jan 2022 17:41:14 -0800 From: yanjun.zhu@linux.dev To: jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev, leonro@nvidia.com Subject: [PATCH v3 2/4] RDMA/hns: Replace get_udp_sport with rdma_get_udp_sport Date: Thu, 6 Jan 2022 13:03:57 -0500 Message-Id: <20220106180359.2915060-3-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220106180359.2915060-1-yanjun.zhu@linux.dev> References: <20220106180359.2915060-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Several drivers have the same function xxx_get_udp_sport. So this function is moved to ib_verbs.h. Signed-off-by: Zhu Yanjun Reviewed-by: Leon Romanovsky Acked-by: Wenpeng Liang --- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index eb0defa80d0d..cb795663b813 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -4488,14 +4488,6 @@ static int modify_qp_rtr_to_rts(struct ib_qp *ibqp, return 0; } -static inline u16 get_udp_sport(u32 fl, u32 lqpn, u32 rqpn) -{ - if (!fl) - fl = rdma_calc_flow_label(lqpn, rqpn); - - return rdma_flow_label_to_udp_sport(fl); -} - static int get_dip_ctx_idx(struct ib_qp *ibqp, const struct ib_qp_attr *attr, u32 *dip_idx) { @@ -4712,8 +4704,8 @@ static int hns_roce_v2_set_path(struct ib_qp *ibqp, } hr_reg_write(context, QPC_UDPSPN, - is_udp ? get_udp_sport(grh->flow_label, ibqp->qp_num, - attr->dest_qp_num) : 0); + is_udp ? rdma_get_udp_sport(grh->flow_label, ibqp->qp_num, + attr->dest_qp_num) : 0); hr_reg_clear(qpc_mask, QPC_UDPSPN); From patchwork Thu Jan 6 18:03:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12704991 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E69FBC433F5 for ; Thu, 6 Jan 2022 01:41:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344426AbiAFBlU (ORCPT ); Wed, 5 Jan 2022 20:41:20 -0500 Received: from mga01.intel.com ([192.55.52.88]:62230 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344428AbiAFBlT (ORCPT ); Wed, 5 Jan 2022 20:41:19 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10217"; a="266855327" X-IronPort-AV: E=Sophos;i="5.88,265,1635231600"; d="scan'208";a="266855327" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jan 2022 17:41:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,265,1635231600"; d="scan'208";a="513226713" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga007.jf.intel.com with ESMTP; 05 Jan 2022 17:41:16 -0800 From: yanjun.zhu@linux.dev To: jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev, leonro@nvidia.com Subject: [PATCH v3 3/4] RDMA/irdma: Make the source udp port vary Date: Thu, 6 Jan 2022 13:03:58 -0500 Message-Id: <20220106180359.2915060-4-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220106180359.2915060-1-yanjun.zhu@linux.dev> References: <20220106180359.2915060-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues. Signed-off-by: Zhu Yanjun Reviewed-by: Leon Romanovsky Acked-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/verbs.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c index 8cd5f9261692..8234a101e752 100644 --- a/drivers/infiniband/hw/irdma/verbs.c +++ b/drivers/infiniband/hw/irdma/verbs.c @@ -1170,6 +1170,10 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr, udp_info->ttl = attr->ah_attr.grh.hop_limit; udp_info->flow_label = attr->ah_attr.grh.flow_label; udp_info->tos = attr->ah_attr.grh.traffic_class; + udp_info->src_port = rdma_get_udp_sport( + udp_info->flow_label, + ibqp->qp_num, + roce_info->dest_qp); irdma_qp_rem_qos(&iwqp->sc_qp); dev->ws_remove(iwqp->sc_qp.vsi, ctx_info->user_pri); ctx_info->user_pri = rt_tos2priority(udp_info->tos); From patchwork Thu Jan 6 18:03:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12704992 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00C7AC433F5 for ; Thu, 6 Jan 2022 01:41:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344434AbiAFBlZ (ORCPT ); Wed, 5 Jan 2022 20:41:25 -0500 Received: from mga01.intel.com ([192.55.52.88]:62230 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344425AbiAFBlX (ORCPT ); Wed, 5 Jan 2022 20:41:23 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10217"; a="266855330" X-IronPort-AV: E=Sophos;i="5.88,265,1635231600"; d="scan'208";a="266855330" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jan 2022 17:41:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,265,1635231600"; d="scan'208";a="513226721" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga007.jf.intel.com with ESMTP; 05 Jan 2022 17:41:19 -0800 From: yanjun.zhu@linux.dev To: jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev, leonro@nvidia.com Subject: [PATCH v3 4/4] RDMA/rxe: Use the standard method to produce udp source port Date: Thu, 6 Jan 2022 13:03:59 -0500 Message-Id: <20220106180359.2915060-5-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220106180359.2915060-1-yanjun.zhu@linux.dev> References: <20220106180359.2915060-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Use the standard method to produce udp source port. Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 0aa0d7e52773..42fa81b455de 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -469,6 +469,12 @@ static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, if (err) goto err1; + if ((mask & IB_QP_AV) && (attr->ah_attr.ah_flags & IB_AH_GRH)) + qp->src_port = rdma_get_udp_sport(attr->ah_attr.grh.flow_label, + qp->ibqp.qp_num, + qp->attr.dest_qp_num); + + return 0; err1: