From patchwork Thu Sep 3 14:56:39 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haggai Eran X-Patchwork-Id: 7117101 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id C7CEDBEEC1 for ; Thu, 3 Sep 2015 14:57:11 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id D957C207DE for ; Thu, 3 Sep 2015 14:57:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E2593207D5 for ; Thu, 3 Sep 2015 14:57:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755572AbbICO5I (ORCPT ); Thu, 3 Sep 2015 10:57:08 -0400 Received: from [193.47.165.129] ([193.47.165.129]:44594 "EHLO mellanox.co.il" rhost-flags-FAIL-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1754570AbbICO5H (ORCPT ); Thu, 3 Sep 2015 10:57:07 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from haggaie@mellanox.com) with ESMTPS (AES256-SHA encrypted); 3 Sep 2015 17:56:48 +0300 Received: from arch003.mtl.labs.mlnx (arch003.mtl.labs.mlnx [10.137.35.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id t83Eue2q000360; Thu, 3 Sep 2015 17:56:42 +0300 From: Haggai Eran To: Doug Ledford Cc: linux-rdma@vger.kernel.org, Eli Cohen , Matan Barak , Yevgeny Petrilin , Eran Ben Elisha , Moshe Lazer , Majd Dibbiny , Haggai Eran Subject: [PATCH v1 3/3] libibverbs/examples: Support odp in rc_pingpong Date: Thu, 3 Sep 2015 17:56:39 +0300 Message-Id: <1441292199-8371-4-git-send-email-haggaie@mellanox.com> X-Mailer: git-send-email 1.7.11.2 In-Reply-To: <1441292199-8371-1-git-send-email-haggaie@mellanox.com> References: <1441292199-8371-1-git-send-email-haggaie@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Majd Dibbiny Signed-off-by: Majd Dibbiny Signed-off-by: Haggai Eran --- examples/rc_pingpong.c | 31 +++++++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git a/examples/rc_pingpong.c b/examples/rc_pingpong.c index ddfe8d007e1a..90a8320121b9 100644 --- a/examples/rc_pingpong.c +++ b/examples/rc_pingpong.c @@ -55,6 +55,7 @@ enum { }; static int page_size; +static int use_odp; struct pingpong_context { struct ibv_context *context; @@ -315,6 +316,7 @@ static struct pingpong_context *pp_init_ctx(struct ibv_device *ib_dev, int size, int use_event) { struct pingpong_context *ctx; + int access_flags = IBV_ACCESS_LOCAL_WRITE; ctx = calloc(1, sizeof *ctx); if (!ctx) @@ -355,7 +357,25 @@ static struct pingpong_context *pp_init_ctx(struct ibv_device *ib_dev, int size, goto clean_comp_channel; } - ctx->mr = ibv_reg_mr(ctx->pd, ctx->buf, size, IBV_ACCESS_LOCAL_WRITE); + if (use_odp) { + const uint32_t rc_caps_mask = IBV_ODP_SUPPORT_SEND | + IBV_ODP_SUPPORT_RECV; + struct ibv_device_attr_ex attrx; + + if (ibv_query_device_ex(ctx->context, NULL, &attrx)) { + fprintf(stderr, "Couldn't query device for its features\n"); + goto clean_comp_channel; + } + + if (!(attrx.odp_caps.general_caps & IBV_ODP_SUPPORT) || + (attrx.odp_caps.per_transport_caps.rc_odp_caps & rc_caps_mask) != rc_caps_mask) { + fprintf(stderr, "The device isn't ODP capable or does not support RC send and receive with ODP\n"); + goto clean_comp_channel; + } + access_flags |= IBV_ACCESS_ON_DEMAND; + } + ctx->mr = ibv_reg_mr(ctx->pd, ctx->buf, size, access_flags); + if (!ctx->mr) { fprintf(stderr, "Couldn't register MR\n"); goto clean_pd; @@ -540,6 +560,7 @@ static void usage(const char *argv0) printf(" -l, --sl= service level value\n"); printf(" -e, --events sleep on CQ events (default poll)\n"); printf(" -g, --gid-idx= local port gid index\n"); + printf(" -o, --odp use on demand paging\n"); } int main(int argc, char *argv[]) @@ -582,11 +603,13 @@ int main(int argc, char *argv[]) { .name = "sl", .has_arg = 1, .val = 'l' }, { .name = "events", .has_arg = 0, .val = 'e' }, { .name = "gid-idx", .has_arg = 1, .val = 'g' }, + { .name = "odp", .has_arg = 0, .val = 'o' }, { 0 } }; - c = getopt_long(argc, argv, "p:d:i:s:m:r:n:l:eg:", + c = getopt_long(argc, argv, "p:d:i:s:m:r:n:l:eg:o", long_options, NULL); + if (c == -1) break; @@ -643,6 +666,10 @@ int main(int argc, char *argv[]) gidx = strtol(optarg, NULL, 0); break; + case 'o': + use_odp = 1; + break; + default: usage(argv[0]); return 1;