From patchwork Tue Nov 22 19:38:50 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dasaratharaman Chandramouli X-Patchwork-Id: 9442027 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 627A0605EE for ; Tue, 22 Nov 2016 19:42:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 511341FF0B for ; Tue, 22 Nov 2016 19:42:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 45FA01FF13; Tue, 22 Nov 2016 19:42:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE1641FF0B for ; Tue, 22 Nov 2016 19:42:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933972AbcKVTmp (ORCPT ); Tue, 22 Nov 2016 14:42:45 -0500 Received: from mga06.intel.com ([134.134.136.31]:62859 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933968AbcKVTmo (ORCPT ); Tue, 22 Nov 2016 14:42:44 -0500 Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga104.jf.intel.com with ESMTP; 22 Nov 2016 11:42:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,682,1473145200"; d="scan'208";a="33543022" Received: from phwtpriv01.ph.intel.com ([10.228.208.76]) by orsmga004.jf.intel.com with ESMTP; 22 Nov 2016 11:42:43 -0800 From: Dasaratharaman Chandramouli To: Dasaratharaman Chandramouli , Ira Weiny , Don Hiatt , linux-rdma , Doug Ledford Subject: [PATCH v2 09/11] IB/IPoIB: Retrieve 32 bit LIDs from path records when running on OPA devices Date: Tue, 22 Nov 2016 14:38:50 -0500 Message-Id: <1479843532-47496-10-git-send-email-dasaratharaman.chandramouli@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1479843532-47496-1-git-send-email-dasaratharaman.chandramouli@intel.com> References: <1479843532-47496-1-git-send-email-dasaratharaman.chandramouli@intel.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Path record responses will contain the 32 bit LID information in the SGID and DGID field of the responses. Modify IPoIB to use these extended LIDs in datagram and connected mode communication. Reviewed-by: Ira Weiny Signed-off-by: Dasaratharaman Chandramouli Signed-off-by: Don Hiatt --- drivers/infiniband/ulp/ipoib/ipoib.h | 4 +++- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 11 +++++++++++ drivers/infiniband/ulp/ipoib/ipoib_main.c | 26 ++++++++++++++++++++++++++ drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 2 +- include/rdma/opa_addr.h | 12 ++++++++++++ 5 files changed, 53 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 7b8d2d9..fad4560 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -352,7 +352,7 @@ struct ipoib_dev_priv { u32 qkey; union ib_gid local_gid; - u16 local_lid; + u32 local_lid; unsigned int admin_mtu; unsigned int mcast_mtu; @@ -421,6 +421,8 @@ struct ipoib_path { struct rb_node rb_node; struct list_head list; int valid; + u32 dlid; + u32 slid; }; struct ipoib_neigh { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 4ad297d..c27df76 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -38,6 +38,7 @@ #include #include #include +#include #include "ipoib.h" @@ -1356,6 +1357,16 @@ static void ipoib_cm_tx_start(struct work_struct *work) } memcpy(&pathrec, &p->path->pathrec, sizeof pathrec); + if (rdma_cap_opa_ah(priv->ca, priv->port)) { + if (p->path->dlid >= be16_to_cpu(IB_MULTICAST_LID_BASE)) + pathrec.dgid.global.interface_id = + OPA_MAKE_ID(p->path->dlid); + + if (p->path->slid >= be16_to_cpu(IB_MULTICAST_LID_BASE)) + pathrec.sgid.global.interface_id = + OPA_MAKE_ID(p->path->slid); + } + spin_unlock_irqrestore(&priv->lock, flags); netif_tx_unlock_bh(dev); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 5636fc3..474c3bf 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -52,6 +52,7 @@ #include #include #include +#include #define DRV_VERSION "1.0.0" @@ -766,6 +767,31 @@ static void path_rec_completion(int status, spin_lock_irqsave(&priv->lock, flags); if (!IS_ERR_OR_NULL(ah)) { + /* + * Extended LIDs might get programmed into GIDs in the + * case of OPA devices. Since we have created the ah + * above which would have made use of the lids, now is + * a good time to change them back to regular GIDs after + * saving the extended LIDs. + */ + if (rdma_cap_opa_ah(priv->ca, priv->port) && + ib_is_opa_gid(&pathrec->sgid)) { + path->slid = opa_get_lid_from_gid(&pathrec->sgid); + pathrec->sgid = path->pathrec.sgid; + } else { + path->slid = be16_to_cpu(pathrec->slid); + } + + if (rdma_cap_opa_ah(priv->ca, priv->port) && + ib_is_opa_gid(&pathrec->dgid)) { + path->dlid = opa_get_lid_from_gid(&pathrec->dgid); + pathrec->dgid = path->pathrec.dgid; + } else { + path->dlid = be16_to_cpu(pathrec->dlid); + } + ipoib_dbg(priv, "PathRec SGID %pI6 DGID %pI6\n", + pathrec->sgid.raw, pathrec->dgid.raw); + path->pathrec = *pathrec; old_ah = path->ah; diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index bff73b5..d3394b6 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -581,7 +581,7 @@ void ipoib_mcast_join_task(struct work_struct *work) port_attr.state); return; } - priv->local_lid = (u16)port_attr.lid; + priv->local_lid = port_attr.lid; netif_addr_lock_bh(dev); if (!test_bit(IPOIB_FLAG_DEV_ADDR_SET, &priv->flags)) { diff --git a/include/rdma/opa_addr.h b/include/rdma/opa_addr.h index 5c713bc..d0a37d0 100644 --- a/include/rdma/opa_addr.h +++ b/include/rdma/opa_addr.h @@ -53,4 +53,16 @@ static inline bool ib_is_opa_gid(union ib_gid *gid) return ((be64_to_cpu(gid->global.interface_id) >> 40) == OPA_SPECIAL_OUI); } + +/** + * opa_get_lid_from_gid: Returns the last 32 bits of the gid. + * OPA devices use one of the gids in the gid table to also + * store the lid. + * + * @gid: The Global identifier + */ +static inline u32 opa_get_lid_from_gid(union ib_gid *gid) +{ + return be64_to_cpu(gid->global.interface_id) & 0xFFFFFFFF; +} #endif /* OPA_ADDR_H */