From patchwork Tue May 8 14:25:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 10386245 X-Patchwork-Delegate: dledford@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 85428602C2 for ; Tue, 8 May 2018 14:27:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6CF9727F85 for ; Tue, 8 May 2018 14:27:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 608542856F; Tue, 8 May 2018 14:27:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F24F285FB for ; Tue, 8 May 2018 14:25:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932344AbeEHOZw (ORCPT ); Tue, 8 May 2018 10:25:52 -0400 Received: from mail.kernel.org ([198.145.29.99]:45402 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932358AbeEHOZQ (ORCPT ); Tue, 8 May 2018 10:25:16 -0400 Received: from localhost (unknown [193.47.165.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0F4B421747; Tue, 8 May 2018 14:25:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1525789515; bh=lFCszGeBWNDUOprMYXIhQJr7bTpgtaztSKaiQVI1ScY=; h=From:To:Cc:Subject:Date:From; b=GEwlrEdn05jkhNrsdeSKjI8AhAiAKCQ17aH+b/RCAE3y0eiJVAsN51D3vqCQMQffO cSVxBE6/9xsUA8/vwttI/bsgnVXQpLMAXZyWtTCExE1Lo0hRQ5SB+0qXRdTUrH4LV3 IuX3B5IPgO8idp7AoJUB4Mb+BOITuoFHtP9BQ/Qw= From: Leon Romanovsky To: Doug Ledford , Jason Gunthorpe Cc: Artemy Kovalyov , RDMA mailing list , Yossi Itigin , Leon Romanovsky Subject: [PATCH rdma-next] IB/uverbs: Pass IB_PORT_GRH_RQUIRED to user space Date: Tue, 8 May 2018 17:25:09 +0300 Message-Id: <20180508142509.17858-1-leon@kernel.org> X-Mailer: git-send-email 2.17.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Artemy Kovalyov Add port_cap_flags value IB_PORT_GRH_REQUIRED instead of field in ib_uverbs_query_port_resp so user space may know whether port configured to work with GRH only. Reviewed-by: Yossi Itigin Signed-off-by: Artemy Kovalyov Signed-off-by: Leon Romanovsky --- drivers/infiniband/core/sa_query.c | 2 +- drivers/infiniband/core/uverbs_cmd.c | 15 +++++++-- drivers/infiniband/hw/hfi1/verbs.c | 4 +-- drivers/infiniband/hw/mlx4/main.c | 29 +++++++++++++++++ drivers/infiniband/hw/mlx5/main.c | 48 ++++++++++++++++++++++++---- drivers/infiniband/hw/mthca/mthca_provider.c | 29 +++++++++++++++++ drivers/infiniband/hw/vmw_pvrdma/pvrdma.h | 29 ++++++++++++++++- include/rdma/ib_verbs.h | 10 +++++- 8 files changed, 151 insertions(+), 15 deletions(-) diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c index a61ec7e33613..6fd1018a6c1b 100644 --- a/drivers/infiniband/core/sa_query.c +++ b/drivers/infiniband/core/sa_query.c @@ -2314,7 +2314,7 @@ static void update_sm_ah(struct work_struct *work) rdma_ah_set_dlid(&ah_attr, port_attr.sm_lid); rdma_ah_set_sl(&ah_attr, port_attr.sm_sl); rdma_ah_set_port_num(&ah_attr, port->port_num); - if (port_attr.grh_required) { + if (port_attr.port_cap_flags & IB_PORT_GRH_REQUIRED) { if (ah_attr.type == RDMA_AH_ATTR_TYPE_OPA) { rdma_ah_set_make_grd(&ah_attr, true); } else { diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 126d3efb976f..868f7549182d 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1979,14 +1979,20 @@ static int modify_qp(struct ib_uverbs_file *file, } if ((cmd->base.attr_mask & IB_QP_AV) && - !rdma_is_port_valid(qp->device, cmd->base.dest.port_num)) { + !(rdma_is_port_valid(qp->device, cmd->base.dest.port_num) && + rdma_validate_av(qp->device, cmd->base.dest.port_num, + cmd->base.dest.is_global))) { ret = -EINVAL; goto release_qp; } if ((cmd->base.attr_mask & IB_QP_ALT_PATH) && - (!rdma_is_port_valid(qp->device, cmd->base.alt_port_num) || - !rdma_is_port_valid(qp->device, cmd->base.alt_dest.port_num))) { + !(rdma_is_port_valid(qp->device, cmd->base.alt_port_num) && + rdma_validate_av(qp->device, cmd->base.alt_port_num, + cmd->base.alt_dest.is_global) && + rdma_is_port_valid(qp->device, cmd->base.alt_dest.port_num) && + rdma_validate_av(qp->device, cmd->base.alt_dest.port_num, + cmd->base.alt_dest.is_global))) { ret = -EINVAL; goto release_qp; } @@ -2559,6 +2565,9 @@ ssize_t ib_uverbs_create_ah(struct ib_uverbs_file *file, if (!rdma_is_port_valid(ib_dev, cmd.attr.port_num)) return -EINVAL; + if (!rdma_validate_av(ib_dev, cmd.attr.port_num, cmd.attr.is_global)) + return -EINVAL; + ib_uverbs_init_udata(&udata, buf + sizeof(cmd), u64_to_user_ptr(cmd.response) + sizeof(resp), in_len - sizeof(cmd) - sizeof(struct ib_uverbs_cmd_hdr), diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c index c8cf4d4984d3..f00eeb636c7e 100644 --- a/drivers/infiniband/hw/hfi1/verbs.c +++ b/drivers/infiniband/hw/hfi1/verbs.c @@ -1491,11 +1491,11 @@ static int query_port(struct rvt_dev_info *rdi, u8 port_num, /* * sm_lid of 0xFFFF needs special handling so that it can * be differentiated from a permissve LID of 0xFFFF. - * We set the grh_required flag here so the SA can program + * We set the IB_PORT_GRH_REQUIRED flag here so the SA can program * the DGID in the address handle appropriately */ if (props->sm_lid == be16_to_cpu(IB_LID_PERMISSIVE)) - props->grh_required = true; + props->port_cap_flags |= IB_PORT_GRH_REQUIRED; return 0; } diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 5b70744f414a..385f28a16bc2 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -85,6 +85,34 @@ static enum rdma_link_layer mlx4_ib_port_link_layer(struct ib_device *device, static struct workqueue_struct *wq; +enum { + MLX4_SUPPORTED_IB_SPEC_PORT_CAP_FLAGS = + IB_PORT_SM | + IB_PORT_NOTICE_SUP | + IB_PORT_TRAP_SUP | + IB_PORT_OPT_IPD_SUP | + IB_PORT_AUTO_MIGR_SUP | + IB_PORT_SL_MAP_SUP | + IB_PORT_MKEY_NVRAM | + IB_PORT_PKEY_NVRAM | + IB_PORT_LED_INFO_SUP | + IB_PORT_SM_DISABLED | + IB_PORT_SYS_IMAGE_GUID_SUP | + IB_PORT_PKEY_SW_EXT_PORT_TRAP_SUP | + IB_PORT_EXTENDED_SPEEDS_SUP | + IB_PORT_CM_SUP | + IB_PORT_SNMP_TUNNEL_SUP | + IB_PORT_REINIT_SUP | + IB_PORT_DEVICE_MGMT_SUP | + IB_PORT_VENDOR_CLASS_SUP | + IB_PORT_DR_NOTICE_SUP | + IB_PORT_CAP_MASK_NOTICE_SUP | + IB_PORT_BOOT_MGMT_SUP | + IB_PORT_LINK_LATENCY_SUP | + IB_PORT_CLIENT_REG_SUP +}; + + static void init_query_mad(struct ib_smp *mad) { mad->base_version = 1; @@ -694,6 +722,7 @@ static int ib_link_query_port(struct ib_device *ibdev, u8 port, props->subnet_timeout = out_mad->data[51] & 0x1f; props->max_vl_num = out_mad->data[37] >> 4; props->init_type_reply = out_mad->data[41] >> 4; + props->port_cap_flags &= MLX4_SUPPORTED_IB_SPEC_PORT_CAP_FLAGS; /* Check if extended speeds (EDR/FDR/...) are supported */ if (props->port_cap_flags & IB_PORT_EXTENDED_SPEEDS_SUP) { diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index 169f32a0726b..a4817049b06b 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -93,6 +93,33 @@ enum { MLX5_ATOMIC_SIZE_QP_8BYTES = 1 << 3, }; +enum { + MLX5_SUPPORTED_IB_SPEC_PORT_CAP_FLAGS = + IB_PORT_SM | + IB_PORT_NOTICE_SUP | + IB_PORT_TRAP_SUP | + IB_PORT_OPT_IPD_SUP | + IB_PORT_AUTO_MIGR_SUP | + IB_PORT_SL_MAP_SUP | + IB_PORT_MKEY_NVRAM | + IB_PORT_PKEY_NVRAM | + IB_PORT_LED_INFO_SUP | + IB_PORT_SM_DISABLED | + IB_PORT_SYS_IMAGE_GUID_SUP | + IB_PORT_PKEY_SW_EXT_PORT_TRAP_SUP | + IB_PORT_EXTENDED_SPEEDS_SUP | + IB_PORT_CM_SUP | + IB_PORT_SNMP_TUNNEL_SUP | + IB_PORT_REINIT_SUP | + IB_PORT_DEVICE_MGMT_SUP | + IB_PORT_VENDOR_CLASS_SUP | + IB_PORT_DR_NOTICE_SUP | + IB_PORT_CAP_MASK_NOTICE_SUP | + IB_PORT_BOOT_MGMT_SUP | + IB_PORT_LINK_LATENCY_SUP | + IB_PORT_CLIENT_REG_SUP +}; + static struct workqueue_struct *mlx5_ib_event_wq; static LIST_HEAD(mlx5_ib_unaffiliated_port_list); static LIST_HEAD(mlx5_ib_dev_list); @@ -1239,7 +1266,9 @@ static int mlx5_query_hca_port(struct ib_device *ibdev, u8 port, props->qkey_viol_cntr = rep->qkey_violation_counter; props->subnet_timeout = rep->subnet_timeout; props->init_type_reply = rep->init_type_reply; - props->grh_required = rep->grh_required; + props->port_cap_flags &= MLX5_SUPPORTED_IB_SPEC_PORT_CAP_FLAGS; + if (rep->grh_required) + props->port_cap_flags |= IB_PORT_GRH_REQUIRED; err = mlx5_query_port_link_width_oper(mdev, &ib_link_width_oper, port); if (err) @@ -4302,7 +4331,8 @@ static void destroy_dev_resources(struct mlx5_ib_resources *devr) cancel_work_sync(&devr->ports[port].pkey_change_work); } -static u32 get_core_cap_flags(struct ib_device *ibdev) +static u32 get_core_cap_flags(struct ib_device *ibdev, + struct ib_port_attr *port_attr) { struct mlx5_ib_dev *dev = to_mdev(ibdev); enum rdma_link_layer ll = mlx5_ib_port_link_layer(ibdev, 1); @@ -4311,8 +4341,14 @@ static u32 get_core_cap_flags(struct ib_device *ibdev) bool raw_support = !mlx5_core_mp_enabled(dev->mdev); u32 ret = 0; - if (ll == IB_LINK_LAYER_INFINIBAND) - return RDMA_CORE_PORT_IBA_IB; + if (ll == IB_LINK_LAYER_INFINIBAND) { + ret = RDMA_CORE_PORT_IBA_IB; + + if (port_attr->port_cap_flags & IB_PORT_GRH_REQUIRED) + ret |= RDMA_CORE_CAP_IB_GRH_REQUIRED; + + return ret; + } if (raw_support) ret = RDMA_CORE_PORT_RAW_PACKET; @@ -4340,15 +4376,13 @@ static int mlx5_port_immutable(struct ib_device *ibdev, u8 port_num, enum rdma_link_layer ll = mlx5_ib_port_link_layer(ibdev, port_num); int err; - immutable->core_cap_flags = get_core_cap_flags(ibdev); - err = ib_query_port(ibdev, port_num, &attr); if (err) return err; immutable->pkey_tbl_len = attr.pkey_tbl_len; immutable->gid_tbl_len = attr.gid_tbl_len; - immutable->core_cap_flags = get_core_cap_flags(ibdev); + immutable->core_cap_flags = get_core_cap_flags(ibdev, &attr); if ((ll == IB_LINK_LAYER_INFINIBAND) || MLX5_CAP_GEN(dev->mdev, roce)) immutable->max_mad_size = IB_MGMT_MAD_SIZE; diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c index 541f237965c7..8917a29ff194 100644 --- a/drivers/infiniband/hw/mthca/mthca_provider.c +++ b/drivers/infiniband/hw/mthca/mthca_provider.c @@ -49,6 +49,34 @@ #include #include "mthca_memfree.h" + +enum { + MTHCA_SUPPORTED_IB_SPEC_PORT_CAP_FLAGS = + IB_PORT_SM | + IB_PORT_NOTICE_SUP | + IB_PORT_TRAP_SUP | + IB_PORT_OPT_IPD_SUP | + IB_PORT_AUTO_MIGR_SUP | + IB_PORT_SL_MAP_SUP | + IB_PORT_MKEY_NVRAM | + IB_PORT_PKEY_NVRAM | + IB_PORT_LED_INFO_SUP | + IB_PORT_SM_DISABLED | + IB_PORT_SYS_IMAGE_GUID_SUP | + IB_PORT_PKEY_SW_EXT_PORT_TRAP_SUP | + IB_PORT_EXTENDED_SPEEDS_SUP | + IB_PORT_CM_SUP | + IB_PORT_SNMP_TUNNEL_SUP | + IB_PORT_REINIT_SUP | + IB_PORT_DEVICE_MGMT_SUP | + IB_PORT_VENDOR_CLASS_SUP | + IB_PORT_DR_NOTICE_SUP | + IB_PORT_CAP_MASK_NOTICE_SUP | + IB_PORT_BOOT_MGMT_SUP | + IB_PORT_LINK_LATENCY_SUP | + IB_PORT_CLIENT_REG_SUP +}; + static void init_query_mad(struct ib_smp *mad) { mad->base_version = 1; @@ -176,6 +204,7 @@ static int mthca_query_port(struct ib_device *ibdev, props->subnet_timeout = out_mad->data[51] & 0x1f; props->max_vl_num = out_mad->data[37] >> 4; props->init_type_reply = out_mad->data[41] >> 4; + props->port_cap_flags &= MTHCA_SUPPORTED_IB_SPEC_PORT_CAP_FLAGS; out: kfree(in_mad); diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h b/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h index 44cb1cfba417..aa15af435362 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h @@ -72,6 +72,33 @@ #define PVRDMA_NUM_RING_PAGES 4 #define PVRDMA_QP_NUM_HEADER_PAGES 1 +enum { + PVRDMA_SUPPORTED_IB_SPEC_PORT_CAP_FLAGS = + IB_PORT_SM | + IB_PORT_NOTICE_SUP | + IB_PORT_TRAP_SUP | + IB_PORT_OPT_IPD_SUP | + IB_PORT_AUTO_MIGR_SUP | + IB_PORT_SL_MAP_SUP | + IB_PORT_MKEY_NVRAM | + IB_PORT_PKEY_NVRAM | + IB_PORT_LED_INFO_SUP | + IB_PORT_SM_DISABLED | + IB_PORT_SYS_IMAGE_GUID_SUP | + IB_PORT_PKEY_SW_EXT_PORT_TRAP_SUP | + IB_PORT_EXTENDED_SPEEDS_SUP | + IB_PORT_CM_SUP | + IB_PORT_SNMP_TUNNEL_SUP | + IB_PORT_REINIT_SUP | + IB_PORT_DEVICE_MGMT_SUP | + IB_PORT_VENDOR_CLASS_SUP | + IB_PORT_DR_NOTICE_SUP | + IB_PORT_CAP_MASK_NOTICE_SUP | + IB_PORT_BOOT_MGMT_SUP | + IB_PORT_LINK_LATENCY_SUP | + IB_PORT_CLIENT_REG_SUP +}; + struct pvrdma_dev; struct pvrdma_page_dir { @@ -351,7 +378,7 @@ static inline int ib_port_cap_flags_to_pvrdma(int flags) static inline int pvrdma_port_cap_flags_to_ib(int flags) { - return flags; + return flags & PVRDMA_SUPPORTED_IB_SPEC_PORT_CAP_FLAGS; } static inline enum pvrdma_port_width ib_port_width_to_pvrdma( diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index d84e6b6a8a56..2323a5624161 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -455,6 +455,7 @@ enum ib_port_cap_flags { IB_PORT_LINK_LATENCY_SUP = 1 << 24, IB_PORT_CLIENT_REG_SUP = 1 << 25, IB_PORT_IP_BASED_GIDS = 1 << 26, + IB_PORT_GRH_REQUIRED = 1 << 27, }; enum ib_port_width { @@ -554,6 +555,7 @@ static inline struct rdma_hw_stats *rdma_alloc_hw_stats_struct( #define RDMA_CORE_CAP_AF_IB 0x00001000 #define RDMA_CORE_CAP_ETH_AH 0x00002000 #define RDMA_CORE_CAP_OPA_AH 0x00004000 +#define RDMA_CORE_CAP_IB_GRH_REQUIRED 0x00008000 /* Protocol 0xFFF00000 */ #define RDMA_CORE_CAP_PROT_IB 0x00100000 @@ -610,7 +612,6 @@ struct ib_port_attr { u8 active_width; u8 active_speed; u8 phys_state; - bool grh_required; }; enum ib_device_modify_flags { @@ -2706,6 +2707,13 @@ static inline int rdma_is_port_valid(const struct ib_device *device, port <= rdma_end_port(device)); } +static inline bool rdma_validate_av(const struct ib_device *device, + u8 port_num, u8 is_global) +{ + return is_global || !(device->port_immutable[port_num].core_cap_flags & + RDMA_CORE_CAP_IB_GRH_REQUIRED); +} + static inline bool rdma_protocol_ib(const struct ib_device *device, u8 port_num) { return device->port_immutable[port_num].core_cap_flags & RDMA_CORE_CAP_PROT_IB;