From patchwork Thu Aug 27 15:22:35 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haggai Eran X-Patchwork-Id: 7085661 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id D17A39F358 for ; Thu, 27 Aug 2015 15:23:00 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id EBF11209C3 for ; Thu, 27 Aug 2015 15:22:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 03F2220993 for ; Thu, 27 Aug 2015 15:22:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752555AbbH0PW5 (ORCPT ); Thu, 27 Aug 2015 11:22:57 -0400 Received: from [193.47.165.129] ([193.47.165.129]:40208 "EHLO mellanox.co.il" rhost-flags-FAIL-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1751936AbbH0PW4 (ORCPT ); Thu, 27 Aug 2015 11:22:56 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from haggaie@mellanox.com) with ESMTPS (AES256-SHA encrypted); 27 Aug 2015 18:22:35 +0300 Received: from arch003.mtl.labs.mlnx (arch003.mtl.labs.mlnx [10.137.35.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id t7RFMVO3032664; Thu, 27 Aug 2015 18:22:31 +0300 From: Haggai Eran To: Doug Ledford Cc: linux-rdma@vger.kernel.org, Eli Cohen , Matan Barak , Yevgeny Petrilin , Eran Ben Elisha , Moshe Lazer , Majd Dibbiny , Haggai Eran Subject: [PATCH 3/3] libibverbs/examples: Support odp in rc_pingpong Date: Thu, 27 Aug 2015 18:22:35 +0300 Message-Id: <1440688955-7709-4-git-send-email-haggaie@mellanox.com> X-Mailer: git-send-email 1.7.11.2 In-Reply-To: <1440688955-7709-1-git-send-email-haggaie@mellanox.com> References: <1440688955-7709-1-git-send-email-haggaie@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-8.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Majd Dibbiny Signed-off-by: Majd Dibbiny Signed-off-by: Haggai Eran --- examples/rc_pingpong.c | 31 +++++++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git a/examples/rc_pingpong.c b/examples/rc_pingpong.c index ddfe8d007e1a..904ec83a633f 100644 --- a/examples/rc_pingpong.c +++ b/examples/rc_pingpong.c @@ -55,6 +55,7 @@ enum { }; static int page_size; +static int use_odp; struct pingpong_context { struct ibv_context *context; @@ -315,6 +316,7 @@ static struct pingpong_context *pp_init_ctx(struct ibv_device *ib_dev, int size, int use_event) { struct pingpong_context *ctx; + int access_flags = IBV_ACCESS_LOCAL_WRITE; ctx = calloc(1, sizeof *ctx); if (!ctx) @@ -355,7 +357,25 @@ static struct pingpong_context *pp_init_ctx(struct ibv_device *ib_dev, int size, goto clean_comp_channel; } - ctx->mr = ibv_reg_mr(ctx->pd, ctx->buf, size, IBV_ACCESS_LOCAL_WRITE); + if (use_odp) { + const uint32_t rc_caps_mask = IBV_ODP_SUPPORT_SEND | + IBV_ODP_SUPPORT_RECV; + struct ibv_device_attr_ex attrx = {}; + + if (ibv_query_device_ex(ctx->context, &attrx)) { + fprintf(stderr, "Couldn't query device for its features\n"); + goto clean_comp_channel; + } + + if (!(attrx.odp_caps.general_caps & IBV_ODP_SUPPORT) || + (attrx.odp_caps.per_transport_caps.rc_odp_caps & rc_caps_mask) != rc_caps_mask) { + fprintf(stderr, "The device isn't ODP capable or does not support RC send and receive with ODP\n"); + goto clean_comp_channel; + } + access_flags |= IBV_ACCESS_ON_DEMAND; + } + ctx->mr = ibv_reg_mr(ctx->pd, ctx->buf, size, access_flags); + if (!ctx->mr) { fprintf(stderr, "Couldn't register MR\n"); goto clean_pd; @@ -540,6 +560,7 @@ static void usage(const char *argv0) printf(" -l, --sl= service level value\n"); printf(" -e, --events sleep on CQ events (default poll)\n"); printf(" -g, --gid-idx= local port gid index\n"); + printf(" -o, --odp use on demand paging\n"); } int main(int argc, char *argv[]) @@ -582,11 +603,13 @@ int main(int argc, char *argv[]) { .name = "sl", .has_arg = 1, .val = 'l' }, { .name = "events", .has_arg = 0, .val = 'e' }, { .name = "gid-idx", .has_arg = 1, .val = 'g' }, + { .name = "odp", .has_arg = 0, .val = 'o' }, { 0 } }; - c = getopt_long(argc, argv, "p:d:i:s:m:r:n:l:eg:", + c = getopt_long(argc, argv, "p:d:i:s:m:r:n:l:eg:o", long_options, NULL); + if (c == -1) break; @@ -643,6 +666,10 @@ int main(int argc, char *argv[]) gidx = strtol(optarg, NULL, 0); break; + case 'o': + use_odp = 1; + break; + default: usage(argv[0]); return 1;