| Message ID | 1474652674-13110-4-git-send-email-ira.weiny@intel.com (mailing list archive) |
|---|---|
| State | RFC |
| Headers | show |
On Fri, Sep 23, 2016 at 01:44:26PM -0400, ira.weiny@intel.com wrote: > From: Don Hiatt <don.hiatt@intel.com> > > When either the port lid or the sm lid is above the ib unicast lid > space, the SMI creates an address handle with the revelant GRH > information. Woah, I though you were only using that horrible GID hack for the uABI. Don't do such crazy things in the kernel. Fix the internal kernel to support your larger lid. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
The reason for doing this in the kernel was that we wanted a more uniform approach in specifying extended LIDs across the user and the kernel space. This also meant we could leave the ah_attr.dlid field in the kernel to be 16 bits. We will extend ah_attr.dlid to 32 bits in the kernel per your suggestion. This should make specifying extended LID information in the GRH of the address handle unnecessary. Thanks, Dasa On 9/23/2016 11:32 AM, Jason Gunthorpe wrote: > On Fri, Sep 23, 2016 at 01:44:26PM -0400, ira.weiny@intel.com wrote: >> From: Don Hiatt <don.hiatt@intel.com> >> >> When either the port lid or the sm lid is above the ib unicast lid >> space, the SMI creates an address handle with the revelant GRH >> information. > > Woah, I though you were only using that horrible GID hack for the > uABI. > > Don't do such crazy things in the kernel. Fix the internal kernel to > support your larger lid. > > Jason > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c index b9bf7aa055e7..0f5fdfb787fb 100644 --- a/drivers/infiniband/core/sa_query.c +++ b/drivers/infiniband/core/sa_query.c @@ -958,7 +958,6 @@ static void update_sm_ah(struct work_struct *work) pr_err("Couldn't find index for default PKey\n"); memset(&ah_attr, 0, sizeof ah_attr); - ah_attr.dlid = port_attr.sm_lid; ah_attr.sl = port_attr.sm_sl; ah_attr.port_num = port->port_num; if (port_attr.grh_required) { @@ -967,6 +966,25 @@ static void update_sm_ah(struct work_struct *work) ah_attr.grh.dgid.global.interface_id = cpu_to_be64(IB_SA_WELL_KNOWN_GUID); } + if (rdma_cap_opa_ah(port->agent->device, port->port_num)) { + ah_attr.dlid = OPA_TO_IB_UCAST_LID(port_attr.sm_lid); + if ((port_attr.sm_lid >= be16_to_cpu(IB_MULTICAST_LID_BASE)) || + (port_attr.lid >= be16_to_cpu(IB_MULTICAST_LID_BASE))) { + if (port_attr.grh_required) + pr_warn("Overwriting GRH specified in SM AH\n"); + ah_attr.ah_flags = IB_AH_GRH; + ah_attr.grh.hop_limit = 1; + ah_attr.grh.sgid_index = 0; + /* Same prefix as sgid */ + ah_attr.grh.dgid.global.subnet_prefix = + cpu_to_be64(port_attr.subnet_prefix); + ah_attr.grh.dgid.global.interface_id = + OPA_MAKE_ID(port_attr.sm_lid); + } + } else { + ah_attr.dlid = (u16)port_attr.sm_lid; + } + new_ah->ah = ib_create_ah(port->agent->qp->pd, &ah_attr); if (IS_ERR(new_ah->ah)) { pr_warn("Couldn't create new SM AH\n"); diff --git a/include/rdma/ib_addr.h b/include/rdma/ib_addr.h index 04074d7e0763..e6293338dfb1 100644 --- a/include/rdma/ib_addr.h +++ b/include/rdma/ib_addr.h @@ -51,6 +51,8 @@ #define OPA_TO_IB_UCAST_LID(x) (((x) >= be16_to_cpu(IB_MULTICAST_LID_BASE)) \ ? 0 : x) +#define OPA_STL_OUI (0x00066AULL) +#define OPA_MAKE_ID(x) (cpu_to_be64(OPA_STL_OUI << 40 | (x))) struct rdma_addr_client { atomic_t refcount;