From patchwork Tue Jan 5 10:34:24 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eli Cohen X-Patchwork-Id: 70973 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter.kernel.org (8.14.3/8.14.2) with ESMTP id o05AWnXI031852 for ; Tue, 5 Jan 2010 10:32:50 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754292Ab0AEKce (ORCPT ); Tue, 5 Jan 2010 05:32:34 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1754296Ab0AEKce (ORCPT ); Tue, 5 Jan 2010 05:32:34 -0500 Received: from mail.mellanox.co.il ([194.90.237.43]:37002 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754283Ab0AEKcd (ORCPT ); Tue, 5 Jan 2010 05:32:33 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from eli@mellanox.co.il) with SMTP; 5 Jan 2010 12:32:30 +0200 Received: from localhost ([10.4.1.30]) by mtlexch01.mtl.com with Microsoft SMTPSVC(6.0.3790.3959); Tue, 5 Jan 2010 12:32:30 +0200 Date: Tue, 5 Jan 2010 12:34:24 +0200 From: Eli Cohen To: Roland Dreier Cc: Linux RDMA list Subject: [PATCHv7 1/2] libmlx4: Add RoCEE support Message-ID: <20100105103424.GP31480@mtls03> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) X-OriginalArrivalTime: 05 Jan 2010 10:32:30.0599 (UTC) FILETIME=[6266D970:01CA8DF2] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-6.000.1038-17112.006 X-TM-AS-Result: No--5.205600-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org diff --git a/src/mlx4.h b/src/mlx4.h index 4445998..661255b 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -236,11 +236,14 @@ struct mlx4_av { uint8_t hop_limit; uint32_t sl_tclass_flowlabel; uint8_t dgid[16]; + uint8_t mac[8]; }; struct mlx4_ah { struct ibv_ah ibv_ah; struct mlx4_av av; + uint16_t vlan; + uint8_t mac[6]; }; static inline unsigned long align(unsigned long val, unsigned long align) diff --git a/src/qp.c b/src/qp.c index d194ae3..cd8fab0 100644 --- a/src/qp.c +++ b/src/qp.c @@ -143,6 +143,8 @@ static void set_datagram_seg(struct mlx4_wqe_datagram_seg *dseg, memcpy(dseg->av, &to_mah(wr->wr.ud.ah)->av, sizeof (struct mlx4_av)); dseg->dqpn = htonl(wr->wr.ud.remote_qpn); dseg->qkey = htonl(wr->wr.ud.remote_qkey); + dseg->vlan = htons(to_mah(wr->wr.ud.ah)->vlan); + memcpy(dseg->mac, to_mah(wr->wr.ud.ah)->mac, 6); } static void __set_data_seg(struct mlx4_wqe_data_seg *dseg, struct ibv_sge *sg) diff --git a/src/verbs.c b/src/verbs.c index 1ac1362..667ef68 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -614,9 +614,21 @@ int mlx4_destroy_qp(struct ibv_qp *ibqp) return 0; } +static int mcast_mac(uint8_t *mac) +{ + int i; + uint8_t val = 0xff; + + for (i = 0; i < 6; ++i) + val &= mac[i]; + + return val == 0xff; +} + struct ibv_ah *mlx4_create_ah(struct ibv_pd *pd, struct ibv_ah_attr *attr) { struct mlx4_ah *ah; + struct ibv_port_attr port_attr; ah = malloc(sizeof *ah); if (!ah) @@ -642,7 +654,24 @@ struct ibv_ah *mlx4_create_ah(struct ibv_pd *pd, struct ibv_ah_attr *attr) memcpy(ah->av.dgid, attr->grh.dgid.raw, 16); } + if (ibv_query_port(pd->context, attr->port_num, &port_attr)) + goto err; + + if (port_attr.link_layer == IBV_LINK_LAYER_ETHERNET) { + if (ibv_cmd_get_mac(pd, attr->port_num, ah->av.dgid, ah->mac)) + goto err; + + ah->vlan = 0; + if (mcast_mac(ah->mac)) + ah->av.dlid = htons(0xc000); + + } + + return &ah->ibv_ah; +err: + free(ah); + return NULL; } int mlx4_destroy_ah(struct ibv_ah *ah) diff --git a/src/wqe.h b/src/wqe.h index 6f7f309..bbd22ba 100644 --- a/src/wqe.h +++ b/src/wqe.h @@ -78,7 +78,8 @@ struct mlx4_wqe_datagram_seg { uint32_t av[8]; uint32_t dqpn; uint32_t qkey; - uint32_t reserved[2]; + uint16_t vlan; + uint8_t mac[6]; }; struct mlx4_wqe_data_seg {