From patchwork Sun May 13 15:37:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10396275 X-Patchwork-Delegate: leon@leon.nu Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6E52D602B1 for ; Sun, 13 May 2018 15:38:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 51E282869A for ; Sun, 13 May 2018 15:38:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 46536286D5; Sun, 13 May 2018 15:38:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E4C0F2869A for ; Sun, 13 May 2018 15:38:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751489AbeEMPiT (ORCPT ); Sun, 13 May 2018 11:38:19 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42818 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751634AbeEMPiS (ORCPT ); Sun, 13 May 2018 11:38:18 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 13 May 2018 18:40:02 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id w4DFcFwL029794; Sun, 13 May 2018 18:38:15 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id w4DFcFBa002146; Sun, 13 May 2018 18:38:15 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id w4DFcFFT002144; Sun, 13 May 2018 18:38:15 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, artemyko@mellanox.com, Alexr@mellanox.com Subject: [PATCH rdma-core 2/2] Enforce is_global if GRH required Date: Sun, 13 May 2018 18:37:57 +0300 Message-Id: <1526225877-2050-3-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1526225877-2050-1-git-send-email-yishaih@mellanox.com> References: <1526225877-2050-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Artemy Kovalyov When IBV_PORT_GRH_REQUIRED port capability flag is set, the applications must create all AH with GRH configured. This patch enforces the above requirement for providers which implement the ibv_create_ah() verb in user space, for others the kernel enforces that. Signed-off-by: Artemy Kovalyov Signed-off-by: Yishai Hadas --- providers/mlx4/verbs.c | 4 ++++ providers/mlx5/mlx5.c | 5 ++++- providers/mlx5/mlx5.h | 1 + providers/mlx5/verbs.c | 6 +++++- providers/mthca/verbs.c | 8 ++++++++ 5 files changed, 22 insertions(+), 2 deletions(-) diff --git a/providers/mlx4/verbs.c b/providers/mlx4/verbs.c index 495f84e..7f0c0ab 100644 --- a/providers/mlx4/verbs.c +++ b/providers/mlx4/verbs.c @@ -1328,6 +1328,10 @@ struct ibv_ah *mlx4_create_ah(struct ibv_pd *pd, struct ibv_ah_attr *attr) if (query_port_cache(pd->context, attr->port_num, &port_attr)) return NULL; + if (port_attr.port_cap_flags & IBV_PORT_GRH_REQUIRED && + !attr->is_global) + return NULL; + ah = malloc(sizeof *ah); if (!ah) return NULL; diff --git a/providers/mlx5/mlx5.c b/providers/mlx5/mlx5.c index 5590241..5079dcd 100644 --- a/providers/mlx5/mlx5.c +++ b/providers/mlx5/mlx5.c @@ -1176,8 +1176,11 @@ static struct verbs_context *mlx5_alloc_context(struct ibv_device *ibdev, for (j = 0; j < min(MLX5_MAX_PORTS_NUM, context->num_ports); ++j) { memset(&port_attr, 0, sizeof(port_attr)); - if (!mlx5_query_port(&v_ctx->context, j + 1, &port_attr)) + if (!mlx5_query_port(&v_ctx->context, j + 1, &port_attr)) { context->cached_link_layer[j] = port_attr.link_layer; + context->cached_port_flags[j] = + port_attr.port_cap_flags; + } } return v_ctx; diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h index d3f0829..546d3a6 100644 --- a/providers/mlx5/mlx5.h +++ b/providers/mlx5/mlx5.h @@ -269,6 +269,7 @@ struct mlx5_context { struct list_head hugetlb_list; int cqe_version; uint8_t cached_link_layer[MLX5_MAX_PORTS_NUM]; + uint32_t cached_port_flags[MLX5_MAX_PORTS_NUM]; unsigned int cached_device_cap_flags; enum ibv_atomic_cap atomic_cap; struct { diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index 6ed2c35..70c6af9 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -2211,6 +2211,7 @@ struct ibv_ah *mlx5_create_ah(struct ibv_pd *pd, struct ibv_ah_attr *attr) __be32 tmp; uint8_t grh; int is_eth; + int grh_req; if (attr->port_num < 1 || attr->port_num > ctx->num_ports) return NULL; @@ -2218,14 +2219,17 @@ struct ibv_ah *mlx5_create_ah(struct ibv_pd *pd, struct ibv_ah_attr *attr) if (ctx->cached_link_layer[attr->port_num - 1]) { is_eth = ctx->cached_link_layer[attr->port_num - 1] == IBV_LINK_LAYER_ETHERNET; + grh_req = ctx->cached_port_flags[attr->port_num - 1] & + IBV_PORT_GRH_REQUIRED; } else { if (ibv_query_port(pd->context, attr->port_num, &port_attr)) return NULL; is_eth = (port_attr.link_layer == IBV_LINK_LAYER_ETHERNET); + grh_req = (port_attr.port_cap_flags & IBV_PORT_GRH_REQUIRED); } - if (unlikely((!attr->is_global) && is_eth)) { + if (unlikely((!attr->is_global) && (is_eth || grh_req))) { errno = EINVAL; return NULL; } diff --git a/providers/mthca/verbs.c b/providers/mthca/verbs.c index ebe804a..014733f 100644 --- a/providers/mthca/verbs.c +++ b/providers/mthca/verbs.c @@ -709,6 +709,14 @@ int mthca_destroy_qp(struct ibv_qp *qp) struct ibv_ah *mthca_create_ah(struct ibv_pd *pd, struct ibv_ah_attr *attr) { struct mthca_ah *ah; + struct ibv_port_attr port_attr; + + if (mthca_query_port(pd->context, attr->port_num, &port_attr)) + return NULL; + + if (port_attr.port_cap_flags & IBV_PORT_GRH_REQUIRED && + !attr->is_global) + return NULL; ah = malloc(sizeof *ah); if (!ah)