From patchwork Fri Aug 2 22:20:20 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Hefty, Sean" X-Patchwork-Id: 2838199 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 544E6BF53E for ; Fri, 2 Aug 2013 22:20:35 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 75A202035C for ; Fri, 2 Aug 2013 22:20:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 29FCE20352 for ; Fri, 2 Aug 2013 22:20:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752800Ab3HBWUa (ORCPT ); Fri, 2 Aug 2013 18:20:30 -0400 Received: from mga01.intel.com ([192.55.52.88]:20958 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753962Ab3HBWU3 (ORCPT ); Fri, 2 Aug 2013 18:20:29 -0400 Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP; 02 Aug 2013 15:20:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.89,804,1367996400"; d="scan'208";a="375385375" Received: from cst-linux.jf.intel.com ([10.23.221.72]) by fmsmga001.fm.intel.com with ESMTP; 02 Aug 2013 15:20:28 -0700 From: sean.hefty@intel.com To: linux-rdma@vger.kernel.org Cc: Sean Hefty Subject: [PATCH librdmacm 3/5] rsockets: Add ability to set the IB route directly Date: Fri, 2 Aug 2013 15:20:20 -0700 Message-Id: <1375482022-5953-4-git-send-email-sean.hefty@intel.com> X-Mailer: git-send-email 1.7.3 In-Reply-To: <1375482022-5953-1-git-send-email-sean.hefty@intel.com> References: <1375482022-5953-1-git-send-email-sean.hefty@intel.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-8.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sean Hefty Add an RDMA specific rsocket option that allows the user to program the RDMA route directly. This is useful for apps that have path record data available, e.g. from ibacm. Signed-off-by: Sean Hefty --- include/rdma/rsocket.h | 3 ++- src/rsocket.c | 30 +++++++++++++++++++++++++++--- 2 files changed, 29 insertions(+), 4 deletions(-) diff --git a/include/rdma/rsocket.h b/include/rdma/rsocket.h index f220c13..efd0db5 100644 --- a/include/rdma/rsocket.h +++ b/include/rdma/rsocket.h @@ -78,7 +78,8 @@ enum { RDMA_SQSIZE, RDMA_RQSIZE, RDMA_INLINE, - RDMA_IOMAPSIZE + RDMA_IOMAPSIZE, + RDMA_ROUTE }; int rsetsockopt(int socket, int level, int optname, diff --git a/src/rsocket.c b/src/rsocket.c index e2b1cc7..57df290 100644 --- a/src/rsocket.c +++ b/src/rsocket.c @@ -329,6 +329,8 @@ struct rsocket { long fd_flags; uint64_t so_opts; uint64_t ipv6_opts; + void *optval; + size_t optlen; int state; int cq_armed; int retries; @@ -1243,13 +1245,26 @@ resolve_addr: rs->retries = 0; resolve_route: to = 1000 << rs->retries++; - ret = rdma_resolve_route(rs->cm_id, to); - if (!ret) - goto do_connect; + if (rs->optval) { + ret = rdma_set_option(rs->cm_id, RDMA_OPTION_IB, + RDMA_OPTION_IB_PATH, rs->optval, + rs->optlen); + free(rs->optval); + rs->optval = NULL; + if (!ret) { + rs->state = rs_resolving_route; + goto resolving_route; + } + } else { + ret = rdma_resolve_route(rs->cm_id, to); + if (!ret) + goto do_connect; + } if (errno == EAGAIN || errno == EWOULDBLOCK) rs->state = rs_resolving_route; break; case rs_resolving_route: +resolving_route: ret = ucma_complete(rs->cm_id); if (ret) { if (errno == ETIMEDOUT && rs->retries <= RS_CONN_RETRIES) @@ -3312,6 +3327,15 @@ int rsetsockopt(int socket, int level, int optname, rs->target_iomap_size = (uint16_t) rs_scale_to_value( (uint8_t) rs_value_to_scale(*(int *) optval, 8), 8); break; + case RDMA_ROUTE: + if ((rs->optval = calloc(optlen, 1))) { + memcpy(rs->optval, optval, optlen); + rs->optlen = optlen; + ret = 0; + } else { + ret = ERR(ENOMEM); + } + break; default: break; }