From patchwork Sun Jan 27 15:28:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10782915 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B80691E for ; Sun, 27 Jan 2019 15:29:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 192202AA33 for ; Sun, 27 Jan 2019 15:29:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0D8332AA34; Sun, 27 Jan 2019 15:29:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 706252AA35 for ; Sun, 27 Jan 2019 15:29:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726423AbfA0P3i (ORCPT ); Sun, 27 Jan 2019 10:29:38 -0500 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:49264 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726354AbfA0P3h (ORCPT ); Sun, 27 Jan 2019 10:29:37 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 27 Jan 2019 17:29:33 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x0RFTWp3014749; Sun, 27 Jan 2019 17:29:32 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id x0RFTW0k011525; Sun, 27 Jan 2019 17:29:32 +0200 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id x0RFTWTf011524; Sun, 27 Jan 2019 17:29:32 +0200 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, danitg@mellanox.com, yosefe@mellanox.com, maxg@mellanox.com, jgg@mellanox.com, majd@mellanox.com Subject: [PATCH rdma-core 2/3] rdmacm: Expose CONNECT_RESPONSE event and rdma_establish() Date: Sun, 27 Jan 2019 17:28:56 +0200 Message-Id: <1548602938-11370-3-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1548602938-11370-1-git-send-email-yishaih@mellanox.com> References: <1548602938-11370-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Yossi Itigin Applications which do not create a QP through rdma_create_qp() may want to postpone the ESTABLISHED event on the passive side, to let the active side complete an application-specific connection establishment phase. For example, modify a QP created by the application to RTR, or make some preparations for receiving messages from the passive side. This patch will return a new event on the active side: CONNECT_RESPONSE, instead of ESTABLISHED, if id->qp==NULL. This gives a chance to the application to perform the extra connection setup. In addition, this patch adds a new rdma_establish() API, which should be called afterwards to complete the connection and generate an ESTABLISHED event on the passive side. Signed-off-by: Yossi Itigin Signed-off-by: Danit Goldberg Signed-off-by: Yishai Hadas --- debian/librdmacm-dev.install | 1 + debian/librdmacm1.symbols | 1 + librdmacm/cma.c | 29 +++++++++++++++---- librdmacm/librdmacm.map | 1 + librdmacm/man/CMakeLists.txt | 1 + librdmacm/man/rdma_establish.3.md | 59 +++++++++++++++++++++++++++++++++++++++ librdmacm/rdma_cma.h | 18 ++++++++++++ 7 files changed, 104 insertions(+), 6 deletions(-) create mode 100644 librdmacm/man/rdma_establish.3.md diff --git a/debian/librdmacm-dev.install b/debian/librdmacm-dev.install index fe1b755..e12c300 100644 --- a/debian/librdmacm-dev.install +++ b/debian/librdmacm-dev.install @@ -22,6 +22,7 @@ usr/share/man/man3/rdma_destroy_id.3 usr/share/man/man3/rdma_destroy_qp.3 usr/share/man/man3/rdma_destroy_srq.3 usr/share/man/man3/rdma_disconnect.3 +usr/share/man/man3/rdma_establish.3 usr/share/man/man3/rdma_event_str.3 usr/share/man/man3/rdma_free_devices.3 usr/share/man/man3/rdma_get_cm_event.3 diff --git a/debian/librdmacm1.symbols b/debian/librdmacm1.symbols index 0d1a54f..996122f 100644 --- a/debian/librdmacm1.symbols +++ b/debian/librdmacm1.symbols @@ -25,6 +25,7 @@ librdmacm.so.1 librdmacm1 #MINVER# rdma_destroy_srq@RDMACM_1.0 1.0.15 rdma_disconnect@RDMACM_1.0 1.0.15 rdma_event_str@RDMACM_1.0 1.0.15 + rdma_establish@RDMACM_1.2 23 rdma_free_devices@RDMACM_1.0 1.0.15 rdma_freeaddrinfo@RDMACM_1.0 1.0.15 rdma_get_cm_event@RDMACM_1.0 1.0.15 diff --git a/librdmacm/cma.c b/librdmacm/cma.c index 687ffd7..5fbbcf5 100644 --- a/librdmacm/cma.c +++ b/librdmacm/cma.c @@ -2077,6 +2077,18 @@ static void ucma_copy_ud_event(struct cma_event *event, dst->qkey = src->qkey; } +int rdma_establish(struct rdma_cm_id *id) +{ + if (id->qp) + return ERR(EINVAL); + + /* id->qp is NULL, so ucma_process_conn_resp() will only send ACCEPT to + * the passive side, and will not attempt to modify the QP. + */ + return ucma_process_conn_resp(container_of(id, struct cma_id_private, + id)); +} + int rdma_get_cm_event(struct rdma_event_channel *channel, struct rdma_cm_event **event) { @@ -2153,12 +2165,17 @@ retry: break; case RDMA_CM_EVENT_CONNECT_RESPONSE: ucma_copy_conn_event(evt, &resp.param.conn); - evt->event.status = ucma_process_conn_resp(evt->id_priv); - if (!evt->event.status) - evt->event.event = RDMA_CM_EVENT_ESTABLISHED; - else { - evt->event.event = RDMA_CM_EVENT_CONNECT_ERROR; - evt->id_priv->connect_error = 1; + if (!evt->id_priv->id.qp) { + evt->event.event = RDMA_CM_EVENT_CONNECT_RESPONSE; + } else { + evt->event.status = + ucma_process_conn_resp(evt->id_priv); + if (!evt->event.status) + evt->event.event = RDMA_CM_EVENT_ESTABLISHED; + else { + evt->event.event = RDMA_CM_EVENT_CONNECT_ERROR; + evt->id_priv->connect_error = 1; + } } break; case RDMA_CM_EVENT_ESTABLISHED: diff --git a/librdmacm/librdmacm.map b/librdmacm/librdmacm.map index bde1bc4..7f55e84 100644 --- a/librdmacm/librdmacm.map +++ b/librdmacm/librdmacm.map @@ -79,5 +79,6 @@ RDMACM_1.1 { RDMACM_1.2 { global: + rdma_establish; rdma_init_qp_attr; } RDMACM_1.1; diff --git a/librdmacm/man/CMakeLists.txt b/librdmacm/man/CMakeLists.txt index aec3b6b..2d1efbf 100644 --- a/librdmacm/man/CMakeLists.txt +++ b/librdmacm/man/CMakeLists.txt @@ -20,6 +20,7 @@ rdma_man_pages( rdma_destroy_qp.3 rdma_destroy_srq.3 rdma_disconnect.3 + rdma_establish.3.md rdma_event_str.3 rdma_free_devices.3 rdma_get_cm_event.3 diff --git a/librdmacm/man/rdma_establish.3.md b/librdmacm/man/rdma_establish.3.md new file mode 100644 index 0000000..91f390d --- /dev/null +++ b/librdmacm/man/rdma_establish.3.md @@ -0,0 +1,59 @@ +--- +date: 2019-01-16 +footer: librdmacm +header: "Librdmacm Programmer's Manual" +layout: page +license: 'Licensed under the OpenIB.org BSD license (FreeBSD Variant) - See COPYING.md' +section: 3 +title: RDMA_ESTABLISH +--- + +# NAME + +rdma_establish - Complete an active connection request. + +# SYNOPSIS + +```c +#include + +int rdma_establish(struct rdma_cm_id *id); +``` + +# DESCRIPTION + +**rdma_establish()** Acknowledge an incoming connection response event and complete the connection establishment. + +Notes: + +If a QP has not been created on the rdma_cm_id, this function should be called by the active side to complete the connection, + +after getting connect response event. + +This will trigger a connection established event on the passive side. + +This function should not be used on an rdma_cm_id on which a QP has been created. + +# ARGUMENTS + +*id* +: RDMA identifier. + +# RETURN VALUE + +**rdma_establish()** returns 0 on success, or -1 on error. If an error occurs, errno will be set to indicate the failure reason. + +# SEE ALSO + +**rdma_connect**(3), +**rdma_disconnect**(3) +**rdma_get_cm_event**(3) + +# AUTHORS + +Danit Goldberg + +Yossi Itigin + + + diff --git a/librdmacm/rdma_cma.h b/librdmacm/rdma_cma.h index cf4a683..758a3f7 100644 --- a/librdmacm/rdma_cma.h +++ b/librdmacm/rdma_cma.h @@ -442,6 +442,24 @@ void rdma_destroy_qp(struct rdma_cm_id *id); int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param); /** + * rdma_establish - Complete an active connection request. + * @id: RDMA identifier. + * Description: + * Acknowledge an incoming connection response event and complete the + * connection establishment. + * Notes: + * If a QP has not been created on the rdma_cm_id, this function should be + * called by the active side to complete the connection, after getting connect + * response event. This will trigger a connection established event on the + * passive side. + * This function should not be used on an rdma_cm_id on which a QP has been + * created. + * See also: + * rdma_connect, rdma_disconnect, rdma_get_cm_event + */ +int rdma_establish(struct rdma_cm_id *id); + +/** * rdma_listen - Listen for incoming connection requests. * @id: RDMA identifier. * @backlog: backlog of incoming connection requests.