From patchwork Wed Aug 20 06:52:08 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matan Barak X-Patchwork-Id: 4747731 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 24E3CC0338 for ; Wed, 20 Aug 2014 06:52:20 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 51AC72013A for ; Wed, 20 Aug 2014 06:52:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5EF6F2016C for ; Wed, 20 Aug 2014 06:52:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751467AbaHTGwR (ORCPT ); Wed, 20 Aug 2014 02:52:17 -0400 Received: from mailp.voltaire.com ([193.47.165.129]:41175 "EHLO mellanox.co.il" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1750916AbaHTGwR (ORCPT ); Wed, 20 Aug 2014 02:52:17 -0400 Received: from Internal Mail-Server by MTLPINE2 (envelope-from matanb@mellanox.com) with SMTP; 20 Aug 2014 09:52:11 +0300 Received: from r-vnc06.mtr.labs.mlnx (r-vnc06.mtr.labs.mlnx [10.208.0.117]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id s7K6qB1e014122; Wed, 20 Aug 2014 09:52:11 +0300 From: Matan Barak To: Yishai Hadas , Jason Gunthorpe Cc: Roland Dreier , linux-rdma@vger.kernel.org, dledford@redhat.com, Or Gerlitz , Matan Barak Subject: [PATCH libmlx4 V4 1/2] Add ibv_query_port caching support Date: Wed, 20 Aug 2014 09:52:08 +0300 Message-Id: <1408517529-20631-2-git-send-email-matanb@mellanox.com> X-Mailer: git-send-email 1.7.6.4 In-Reply-To: <1408517529-20631-1-git-send-email-matanb@mellanox.com> References: <1408517529-20631-1-git-send-email-matanb@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Return fields according to the port cache rather than calling the kernel. Signed-off-by: Matan Barak Signed-off-by: Or Gerlitz --- src/mlx4.c | 2 ++ src/mlx4.h | 7 +++++++ src/verbs.c | 40 ++++++++++++++++++++++++++++++++++++++-- 3 files changed, 47 insertions(+), 2 deletions(-) diff --git a/src/mlx4.c b/src/mlx4.c index 1ee0338..ee57a84 100644 --- a/src/mlx4.c +++ b/src/mlx4.c @@ -157,6 +157,8 @@ static int mlx4_init_context(struct verbs_device *v_device, context->qp_table_shift = ffs(context->num_qps) - 1 - MLX4_QP_TABLE_BITS; context->qp_table_mask = (1 << context->qp_table_shift) - 1; + for (i = 0; i < MLX4_PORTS_NUM; ++i) + context->port_query_cache[i].valid = 0; pthread_mutex_init(&context->qp_table_mutex, NULL); for (i = 0; i < MLX4_QP_TABLE_SIZE; ++i) diff --git a/src/mlx4.h b/src/mlx4.h index d71450f..05ec70c 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -40,6 +40,8 @@ #include #include +#define MLX4_PORTS_NUM 2 + #ifdef HAVE_VALGRIND_MEMCHECK_H # include @@ -189,6 +191,11 @@ struct mlx4_context { pthread_mutex_t db_list_mutex; int cqe_size; struct mlx4_xsrq_table xsrq_table; + struct { + uint8_t valid; + uint8_t link_layer; + enum ibv_port_cap_flags caps; + } port_query_cache[MLX4_PORTS_NUM]; }; struct mlx4_buf { diff --git a/src/verbs.c b/src/verbs.c index 623d576..5c069ea 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -70,8 +70,44 @@ int mlx4_query_port(struct ibv_context *context, uint8_t port, struct ibv_port_attr *attr) { struct ibv_query_port cmd; + int err; + + err = ibv_cmd_query_port(context, port, attr, &cmd, sizeof(cmd)); + if (!err && port <= MLX4_PORTS_NUM && port > 0) { + struct mlx4_context *mctx = to_mctx(context); + if (!mctx->port_query_cache[port - 1].valid) { + mctx->port_query_cache[port - 1].link_layer = + attr->link_layer; + mctx->port_query_cache[port - 1].caps = + attr->port_cap_flags; + mctx->port_query_cache[port - 1].valid = 1; + } + } + + return err; +} + +/* Only the fields in the port cache will be valid */ +static int query_port_cache(struct ibv_context *context, uint8_t port_num, + struct ibv_port_attr *port_attr) +{ + struct mlx4_context *mctx = to_mctx(context); + if (port_num <= 0 || port_num > MLX4_PORTS_NUM) + return -EINVAL; + if (mctx->port_query_cache[port_num - 1].valid) { + port_attr->link_layer = + mctx-> + port_query_cache[port_num - 1]. + link_layer; + port_attr->port_cap_flags = + mctx-> + port_query_cache[port_num - 1]. + caps; + return 0; + } + return mlx4_query_port(context, port_num, + (struct ibv_port_attr *)port_attr); - return ibv_cmd_query_port(context, port, attr, &cmd, sizeof cmd); } struct ibv_pd *mlx4_alloc_pd(struct ibv_context *context) @@ -788,7 +824,7 @@ struct ibv_ah *mlx4_create_ah(struct ibv_pd *pd, struct ibv_ah_attr *attr) struct mlx4_ah *ah; struct ibv_port_attr port_attr; - if (ibv_query_port(pd->context, attr->port_num, &port_attr)) + if (query_port_cache(pd->context, attr->port_num, &port_attr)) return NULL; ah = malloc(sizeof *ah);