From patchwork Mon Aug 22 18:58:15 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 9294249 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 541A160574 for ; Mon, 22 Aug 2016 19:14:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 45F8D28A64 for ; Mon, 22 Aug 2016 19:14:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A58728A6C; Mon, 22 Aug 2016 19:14:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B22CD28A64 for ; Mon, 22 Aug 2016 19:14:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754130AbcHVTOf (ORCPT ); Mon, 22 Aug 2016 15:14:35 -0400 Received: from mail-it0-f66.google.com ([209.85.214.66]:35655 "EHLO mail-it0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753017AbcHVTOe (ORCPT ); Mon, 22 Aug 2016 15:14:34 -0400 Received: by mail-it0-f66.google.com with SMTP id f6so5585738ith.2; Mon, 22 Aug 2016 12:14:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=pDdjsWYKXHIzlZeLAcvRNd2L98ZPi3S+RxPLChuTDjc=; b=gh3cKBfGf50VrdEvhRAKFs9JKzRwkKrVv7nYacNi0hhIBXFIOdpRKIJW9RFPgOGdxI il0MNrRt4sk9A+Js85Ni0b45h2XNtE6yxfPHp3ZHUVKnSqEceQJgwSbUTOP9SAFQykMC U1Q1r7+m8NtkxEi2lyn6GSm0BAyBpkn1obNp2iJCazEcO50VkI0XYrDxkQLhPRQszsXt dIwfVNfPBDVX5gz4y34jR/QWY6PM2csjAZMPKLFZ6Cc9aG5VYndJlYTRL8RgKXLpjUm0 /xdrxmdilvgiUZUThlBqKFj+b52aP1Y1v3R+GKt1C1EyKvg6JbDUo/1zhWApygkKPnc5 1ufw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=pDdjsWYKXHIzlZeLAcvRNd2L98ZPi3S+RxPLChuTDjc=; b=KIuvdYaBbu87tHg0xaDkb/zOKz0KdZ72rXxX0X5+MSL8ny0lzomzBlqjrjrRkm57E4 MYRE/iSiSLQ5mPQsfbc6yGb2IOz8kpSJInqfLOn1kJ+KjdkmZ/dvJfi7Kukpa0l8lG+6 KNheUadquR5MvXFI73w51RuRnWWRsFMTmzl73pY8gWzkSDZXOgVb1g1ZJQH9+n6q1hPB yttq2jwqVYmZzUup1Bsx8Ozas/BZHRHIQTO2zRuyTzB7MENoqBVPAJjw1Im8qIGCFJBK AYSI8oGcGKGgkTiDNilB0/cwX7zQNRdegBQqLUmmKqGBpsnJNJzZj9hmxS26qUiUe/Ki thyg== X-Gm-Message-State: AEkoousWW6qAUWLKqOCnOsEEEePdukgQVdXXSIiymADtFqICA1g7dJFFENbFv3Z8eehvgQ== X-Received: by 10.107.34.72 with SMTP id i69mr25304430ioi.13.1471892296113; Mon, 22 Aug 2016 11:58:16 -0700 (PDT) Received: from klimt.1015granger.net ([2604:8800:100:81fc:ec4:7aff:fe6c:6aa0]) by smtp.gmail.com with ESMTPSA id u75sm8217097ita.16.2016.08.22.11.58.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Aug 2016 11:58:15 -0700 (PDT) Subject: [PATCH v2 5/6] svcrdma: Server-side support for rpcrdma_connect_private From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Mon, 22 Aug 2016 14:58:15 -0400 Message-ID: <20160822185815.12076.77878.stgit@klimt.1015granger.net> In-Reply-To: <20160822185459.12076.43516.stgit@klimt.1015granger.net> References: <20160822185459.12076.43516.stgit@klimt.1015granger.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Prepare to receive an RDMA-CM private message when handling a new connection attempt, and send a similar message as part of connection acceptance. Both sides can communicate their various implementation limits. Implementations that don't support this sideband protocol ignore it. Signed-off-by: Chuck Lever Reviewed-by: Sagi Grimberg --- net/sunrpc/xprtrdma/svc_rdma_transport.c | 34 ++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index 924271c..f51e98a 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -648,6 +648,21 @@ int svc_rdma_repost_recv(struct svcxprt_rdma *xprt, gfp_t flags) return ret; } +static void +svc_rdma_parse_connect_private(struct svcxprt_rdma *newxprt, + struct rdma_conn_param *param) +{ + const struct rpcrdma_connect_private *pmsg = param->private_data; + + if (pmsg && + pmsg->cp_magic == rpcrdma_cmp_magic && + pmsg->cp_version == RPCRDMA_CMP_VERSION) { + dprintk("svcrdma: client send_size %u, recv_size %u\n", + rpcrdma_decode_buffer_size(pmsg->cp_send_size), + rpcrdma_decode_buffer_size(pmsg->cp_recv_size)); + } +} + /* * This function handles the CONNECT_REQUEST event on a listening * endpoint. It is passed the cma_id for the _new_ connection. The context in @@ -659,7 +674,8 @@ int svc_rdma_repost_recv(struct svcxprt_rdma *xprt, gfp_t flags) * will call the recvfrom method on the listen xprt which will accept the new * connection. */ -static void handle_connect_req(struct rdma_cm_id *new_cma_id, size_t client_ird) +static void handle_connect_req(struct rdma_cm_id *new_cma_id, + struct rdma_conn_param *param) { struct svcxprt_rdma *listen_xprt = new_cma_id->context; struct svcxprt_rdma *newxprt; @@ -675,9 +691,10 @@ static void handle_connect_req(struct rdma_cm_id *new_cma_id, size_t client_ird) new_cma_id->context = newxprt; dprintk("svcrdma: Creating newxprt=%p, cm_id=%p, listenxprt=%p\n", newxprt, newxprt->sc_cm_id, listen_xprt); + svc_rdma_parse_connect_private(newxprt, param); /* Save client advertised inbound read limit for use later in accept. */ - newxprt->sc_ord = client_ird; + newxprt->sc_ord = param->initiator_depth; /* Set the local and remote addresses in the transport */ sa = (struct sockaddr *)&newxprt->sc_cm_id->route.addr.dst_addr; @@ -712,8 +729,7 @@ static int rdma_listen_handler(struct rdma_cm_id *cma_id, dprintk("svcrdma: Connect request on cma_id=%p, xprt = %p, " "event = %s (%d)\n", cma_id, cma_id->context, rdma_event_msg(event->event), event->event); - handle_connect_req(cma_id, - event->param.conn.initiator_depth); + handle_connect_req(cma_id, &event->param.conn); break; case RDMA_CM_EVENT_ESTABLISHED: @@ -947,6 +963,7 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) struct svcxprt_rdma *listen_rdma; struct svcxprt_rdma *newxprt = NULL; struct rdma_conn_param conn_param; + struct rpcrdma_connect_private pmsg; struct ib_qp_init_attr qp_attr; struct ib_device *dev; unsigned int i; @@ -1100,11 +1117,20 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) /* Swap out the handler */ newxprt->sc_cm_id->event_handler = rdma_cma_handler; + /* Construct RDMA-CM private message */ + pmsg.cp_magic = rpcrdma_cmp_magic; + pmsg.cp_version = RPCRDMA_CMP_VERSION; + pmsg.cp_flags = 0; + pmsg.cp_send_size = pmsg.cp_recv_size = + rpcrdma_encode_buffer_size(newxprt->sc_max_req_size); + /* Accept Connection */ set_bit(RDMAXPRT_CONN_PENDING, &newxprt->sc_flags); memset(&conn_param, 0, sizeof conn_param); conn_param.responder_resources = 0; conn_param.initiator_depth = newxprt->sc_ord; + conn_param.private_data = &pmsg; + conn_param.private_data_len = sizeof(pmsg); ret = rdma_accept(newxprt->sc_cm_id, &conn_param); if (ret) { dprintk("svcrdma: failed to accept new connection, ret=%d\n",