From patchwork Sun Jul 26 20:59:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11686145 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16A80913 for ; Sun, 26 Jul 2020 20:59:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EAE58206E3 for ; Sun, 26 Jul 2020 20:59:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gCp8QjNM" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727097AbgGZU7t (ORCPT ); Sun, 26 Jul 2020 16:59:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45500 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726719AbgGZU7s (ORCPT ); Sun, 26 Jul 2020 16:59:48 -0400 Received: from mail-il1-x144.google.com (mail-il1-x144.google.com [IPv6:2607:f8b0:4864:20::144]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BAF31C0619D2; Sun, 26 Jul 2020 13:59:48 -0700 (PDT) Received: by mail-il1-x144.google.com with SMTP id q3so11412011ilt.8; Sun, 26 Jul 2020 13:59:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:user-agent:mime-version :content-transfer-encoding; bh=6s+NFWVjIrnSt/dV3KbX/od4mms9sT5SPrs9wEmSudY=; b=gCp8QjNM8YTU7wCggq2okWIB38vMiOd2uinDWHqwfiJ1mACCTI3V2b2jWRVcWOsdyR lhzWUhDRWN63MWKdtUn6pubhPOmgTlQR6L7euuqcWiW2amTxDY102XArf2xwbPcOJGxo jO+waCpDDYfVTKY3q63iBKgxOGQvrw9VBNg2mFs+FpF5nScGoNnKihKoIUVtuMM+aA+L 3Z5yegv6+bl99WZhlYjYbUZuTV9XBA3yItFEFdYEBFEZgIKTJwwkOIVHyrKN0IgRlzHo EZnY4WPgW5Idtw/aj3ja7cNuotnxP7NpEbq2r2wWhFPc5B4nOJ7cDJSMVhQydHOXvHRK UMJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :user-agent:mime-version:content-transfer-encoding; bh=6s+NFWVjIrnSt/dV3KbX/od4mms9sT5SPrs9wEmSudY=; b=qPAEJv5UpjWEhpRdlLOo7Ru4bhiaym4uoB96Hd8LG0T0GZPTWsVcWUuFM+IVWFhVol McuzDwuf47soiBwu+pSGZNnyQepM4bNO3fji0YXPTYAs6laKXQ9N/xLxIFt7PQvzs3xZ uOwkXW3UuFCwyDD418WDQznZhQwHTLrA0UCSFn8n0z2Yruy6AgIx3HlXkpE42+OKxUuY zFO31KtxIgjSnRIcVr0NdPcJrvv0ptpcbV+IAaBV9m4d65+qWX2pU2w97vz4fcsAL1+O ahzeyZw/Dkgi2PBu/tlaEVKPex0/tADtcvDHH+qoarA7FfRsxMAPEGYhkNUmiMM3J7/v qrvg== X-Gm-Message-State: AOAM533EC9PIAnoKZ/+Ljm6N+S6R1Czg76II+uzuNjqigqRChHCbFwkO 5khBb2sLV55E/wfoFUP6upoG3zF4D+k= X-Google-Smtp-Source: ABdhPJyrrpd/Jq592uGVt2AQEExQKsTLDh3PSBbiHlwavzGYTySiMd3eTLw7LhpKGaSzBxCgjNlfDA== X-Received: by 2002:a05:6e02:14c2:: with SMTP id o2mr20586396ilk.54.1595797187517; Sun, 26 Jul 2020 13:59:47 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id l17sm7027564ilm.70.2020.07.26.13.59.46 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 26 Jul 2020 13:59:46 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 06QKxjeW013834; Sun, 26 Jul 2020 20:59:45 GMT Subject: [PATCH v1 1/3] svcrdma: Fix another Receive buffer leak From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Sun, 26 Jul 2020 16:59:45 -0400 Message-ID: <159579718507.2004.16208139278801479272.stgit@klimt.1015granger.net> User-Agent: StGit/0.23 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org During a connection tear down, the Receive queue is flushed before the device resources are freed. Typically, all the Receives flush with IB_WR_FLUSH_ERR. However, any pending successful Receives flush with IB_WR_SUCCESS, and the server automatically posts a fresh Receive to replace the completing one. This happens even after the connection has closed and the RQ is drained. Receives that are posted after the RQ is drained appear never to complete, causing a Receive resource leak. The leaked Receive buffer is left DMA-mapped. To prevent these late-posted recv_ctxt's from leaking, block new Receive posting after XPT_CLOSE is set. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index d5ec85cb652c..5bb97b5f4606 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -275,6 +275,8 @@ static int svc_rdma_post_recv(struct svcxprt_rdma *rdma) { struct svc_rdma_recv_ctxt *ctxt; + if (test_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags)) + return 0; ctxt = svc_rdma_recv_ctxt_get(rdma); if (!ctxt) return -ENOMEM; From patchwork Sun Jul 26 20:59:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11686151 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A5BCC14E3 for ; Sun, 26 Jul 2020 20:59:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8D6EB206E3 for ; Sun, 26 Jul 2020 20:59:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FwaQ8KkB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727775AbgGZU7y (ORCPT ); Sun, 26 Jul 2020 16:59:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726719AbgGZU7x (ORCPT ); Sun, 26 Jul 2020 16:59:53 -0400 Received: from mail-il1-x143.google.com (mail-il1-x143.google.com [IPv6:2607:f8b0:4864:20::143]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AB6FC0619D2; Sun, 26 Jul 2020 13:59:53 -0700 (PDT) Received: by mail-il1-x143.google.com with SMTP id r12so11407822ilh.4; Sun, 26 Jul 2020 13:59:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=OzYL794bdANxKqvBKFYfm3GV6VDCfi36xHGz4dBMwac=; b=FwaQ8KkB4g7gsy5Cr9ZSde8OJ3p1TPnSVFBI7q0cVM6rgSuqdxhsiEgE09nOttNV16 D8onFx1Zv8aYo/umAWL9CC6KNcRev3uZ7fTl8ifuSkHJWskUTeT03CXKSRuHuWK1nTEV +chNxUxiR2lW2FssgXRB4scwQ5Mu3UceXlZmEbgoN7/ytTT1Rkl2LBGfJwZjdYZQ7gY5 z0ebtLgIARLEMQ6UTPatciu56/YdEbbNgQDppUYuWufNIYqjEboWieiqzB576Vj9Jyk9 auHx5NvL3PNlKxROqmtgwXko3CVKihzLT2zE65EiXYXXJYDisLwn7b4G1A5nmahiVNg6 /8hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=OzYL794bdANxKqvBKFYfm3GV6VDCfi36xHGz4dBMwac=; b=sD393KIO9T8SjUv9jMBPDriRnzxz1EyFiJLPmUd0anA2tJAXueMbgAvXLDRpxK2fkO N9SNLDvKxP/CkNYgQuESDuaua/VM2WsFjLmv/9NKY62lYDJ22N9GvLGZl6cOxaVyLNuj j6HWVzklKIUUaQyzSIn7lz83/1yil4O+ubaUrQuXQW+lJuTCpxwoDgdkrSM0XkOm2nj2 dLpi3bBUoebpCTxSC/y5FGelUtM0iszhV+2+TfFS7fWXvVrHsJxlA/1umTpFINrl4FFc sjchYEHaE8fmWynGU2elmJteHsD2bjQ51C+e7pKjPl52drzknNC50nazocHBCN8Iv+ip KpgA== X-Gm-Message-State: AOAM532yYuV+Y0FuT01a0nDCxC/2IWPTK+3hPret/HBeUPpdeKYT9Tn0 sEdrLIssrwhWZLLJfLXa3x9rx2vkQ4k= X-Google-Smtp-Source: ABdhPJy0QJf6Qi8WhwUT+XsCRgnl0Rdl7i1wfzF4B0sKCwIizv2wOjJHWlyvzOhnsVdsxTP1DDVZCQ== X-Received: by 2002:a92:cb06:: with SMTP id s6mr82716ilo.13.1595797192414; Sun, 26 Jul 2020 13:59:52 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id v5sm7474029ios.54.2020.07.26.13.59.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 26 Jul 2020 13:59:51 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 06QKxo58013837; Sun, 26 Jul 2020 20:59:50 GMT Subject: [PATCH v1 2/3] svcrdma: Remove transport reference counting From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Sun, 26 Jul 2020 16:59:50 -0400 Message-ID: <159579719076.2004.14162309967032149756.stgit@klimt.1015granger.net> In-Reply-To: <159579718507.2004.16208139278801479272.stgit@klimt.1015granger.net> References: <159579718507.2004.16208139278801479272.stgit@klimt.1015granger.net> User-Agent: StGit/0.23 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Jason tells me that a ULP cannot rely on getting an ESTABLISHED and DISCONNECTED event pair for each connection, so transport reference counting in the CM event handler will never be reliable. Now that we have ib_drain_qp(), svcrdma should no longer need to hold transport references while Sends and Receives are posted. So remove the get/put call sites in the CM event handlers. This eliminates a significant source of locked memory bus traffic. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 6 +----- net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 -- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 4 ---- net/sunrpc/xprtrdma/svc_rdma_transport.c | 17 +---------------- 4 files changed, 2 insertions(+), 27 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 5bb97b5f4606..c6ea2903c21a 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -257,7 +257,6 @@ static int __svc_rdma_post_recv(struct svcxprt_rdma *rdma, { int ret; - svc_xprt_get(&rdma->sc_xprt); trace_svcrdma_post_recv(ctxt); ret = ib_post_recv(rdma->sc_qp, &ctxt->rc_recv_wr, NULL); if (ret) @@ -267,7 +266,6 @@ static int __svc_rdma_post_recv(struct svcxprt_rdma *rdma, err_post: trace_svcrdma_rq_post_err(rdma, ret); svc_rdma_recv_ctxt_put(rdma, ctxt); - svc_xprt_put(&rdma->sc_xprt); return ret; } @@ -344,15 +342,13 @@ static void svc_rdma_wc_receive(struct ib_cq *cq, struct ib_wc *wc) spin_unlock(&rdma->sc_rq_dto_lock); if (!test_bit(RDMAXPRT_CONN_PENDING, &rdma->sc_flags)) svc_xprt_enqueue(&rdma->sc_xprt); - goto out; + return; flushed: post_err: svc_rdma_recv_ctxt_put(rdma, ctxt); set_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags); svc_xprt_enqueue(&rdma->sc_xprt); -out: - svc_xprt_put(&rdma->sc_xprt); } /** diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index c16d10601d65..fe54cbe97a46 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -164,7 +164,6 @@ static void svc_rdma_cc_init(struct svcxprt_rdma *rdma, { svc_rdma_cc_cid_init(rdma, &cc->cc_cid); cc->cc_rdma = rdma; - svc_xprt_get(&rdma->sc_xprt); INIT_LIST_HEAD(&cc->cc_rwctxts); cc->cc_sqecount = 0; @@ -184,7 +183,6 @@ static void svc_rdma_cc_release(struct svc_rdma_chunk_ctxt *cc, ctxt->rw_nents, dir); svc_rdma_put_rw_ctxt(rdma, ctxt); } - svc_xprt_put(&rdma->sc_xprt); } /* State for sending a Write or Reply chunk. diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 73d46e8cdc16..7b94d971feb3 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -291,8 +291,6 @@ static void svc_rdma_wc_send(struct ib_cq *cq, struct ib_wc *wc) set_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags); svc_xprt_enqueue(&rdma->sc_xprt); } - - svc_xprt_put(&rdma->sc_xprt); } /** @@ -330,7 +328,6 @@ int svc_rdma_send(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt) continue; } - svc_xprt_get(&rdma->sc_xprt); trace_svcrdma_post_send(ctxt); ret = ib_post_send(rdma->sc_qp, wr, NULL); if (ret) @@ -340,7 +337,6 @@ int svc_rdma_send(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt) trace_svcrdma_sq_post_err(rdma, ret); set_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags); - svc_xprt_put(&rdma->sc_xprt); wake_up(&rdma->sc_send_wait); return ret; } diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index 3da7901a49e6..aa60f75c8c1d 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -271,7 +271,6 @@ static int rdma_cma_handler(struct rdma_cm_id *cma_id, switch (event->event) { case RDMA_CM_EVENT_ESTABLISHED: /* Accept complete */ - svc_xprt_get(xprt); dprintk("svcrdma: Connection completed on DTO xprt=%p, " "cm_id=%p\n", xprt, cma_id); clear_bit(RDMAXPRT_CONN_PENDING, &rdma->sc_flags); @@ -282,7 +281,6 @@ static int rdma_cma_handler(struct rdma_cm_id *cma_id, xprt, cma_id); set_bit(XPT_CLOSE, &xprt->xpt_flags); svc_xprt_enqueue(xprt); - svc_xprt_put(xprt); break; case RDMA_CM_EVENT_DEVICE_REMOVAL: dprintk("svcrdma: Device removal cma_id=%p, xprt = %p, " @@ -290,7 +288,6 @@ static int rdma_cma_handler(struct rdma_cm_id *cma_id, rdma_event_msg(event->event), event->event); set_bit(XPT_CLOSE, &xprt->xpt_flags); svc_xprt_enqueue(xprt); - svc_xprt_put(xprt); break; default: dprintk("svcrdma: Unexpected event on DTO endpoint %p, " @@ -539,24 +536,11 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) return NULL; } -/* - * When connected, an svc_xprt has at least two references: - * - * - A reference held by the cm_id between the ESTABLISHED and - * DISCONNECTED events. If the remote peer disconnected first, this - * reference could be gone. - * - * - A reference held by the svc_recv code that called this function - * as part of close processing. - * - * At a minimum one references should still be held. - */ static void svc_rdma_detach(struct svc_xprt *xprt) { struct svcxprt_rdma *rdma = container_of(xprt, struct svcxprt_rdma, sc_xprt); - /* Disconnect and flush posted WQE */ rdma_disconnect(rdma->sc_cm_id); } @@ -566,6 +550,7 @@ static void __svc_rdma_free(struct work_struct *work) container_of(work, struct svcxprt_rdma, sc_work); struct svc_xprt *xprt = &rdma->sc_xprt; + /* This blocks until the Completion Queues are empty */ if (rdma->sc_qp && !IS_ERR(rdma->sc_qp)) ib_drain_qp(rdma->sc_qp); From patchwork Sun Jul 26 20:59:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11686155 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2401B14E3 for ; Sun, 26 Jul 2020 21:00:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0C2F4206E3 for ; Sun, 26 Jul 2020 21:00:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HLJ/Bt7j" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727780AbgGZU77 (ORCPT ); Sun, 26 Jul 2020 16:59:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726719AbgGZU76 (ORCPT ); Sun, 26 Jul 2020 16:59:58 -0400 Received: from mail-il1-x144.google.com (mail-il1-x144.google.com [IPv6:2607:f8b0:4864:20::144]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E4CEC0619D2; Sun, 26 Jul 2020 13:59:58 -0700 (PDT) Received: by mail-il1-x144.google.com with SMTP id p16so644241ile.0; Sun, 26 Jul 2020 13:59:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=PRYgMVIXtgqm6LIgBOqkl74Fauj/p2bxSG5MktH/UF4=; b=HLJ/Bt7jxv3jFoI+XlAv/PSQCG92OMBLFuQWOo6Twa9CHnZM+1xHLIlubpR3pwIrQb 36+u9OEKlsIP0zqBhA0bU/S4avaJLKnHhriAljLcGXp632Rpic4yZmgb+hNGrU7FBdyT V5BWicP4Cw29tT6u2f8/MORQxCQ38b+xM9JsxfrLhsdaEGK/09QmuaH1ufbqIwrJ2f5E eqWab309CaOiSSO/Ukya4i0JtTsIHp5AXzzFzbB7apoTzK6/q+u6x03U4SF0yYDEnpOd AMwpccyzgqJSxTqvzvXVwUa4iOnzTGgtUkGjnk+Sq0dgVGfXZpqsRUcc4m3QqmCnEozj 3qGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=PRYgMVIXtgqm6LIgBOqkl74Fauj/p2bxSG5MktH/UF4=; b=VZeHiJkQEX7+rn9AV6yfiUHVEx9h6mE7UkINhmKBbI0LjGwCIawbj+BAk/lJfZg5NM T2lVq7zSpDuTRh+3Ee/grNgnzLJKWzAfClOp3VOVKWvz9/Ziw+MAYksakuWoQIb5+uyG iV097F+AQxRW0Ue1VtDVewon0ycqUAbDsK3S7odS2cvX2uo6thcuJwSuRlAjmm5k7Myu 3Y5tShdNT0oieApiUqdWKxXB1472qmK5vdk2B6tI9uRYiMMESxDj/Yi/DeoHrsf0qpES NsljFaHTCAcYfSFNor9hQDXhrSe7OvypCnlOtenGiDMVuPDJUTVTAJoJoZcLSb7kDOqN BJPg== X-Gm-Message-State: AOAM530GVysTnQqoAPGNc/wN5dtArXeQg1SYVnP3kh+9G5+C15Oz6Q1E zjzWgHQFZFsE6kP4fRyTEEY+OJo5RXo= X-Google-Smtp-Source: ABdhPJxBRTHj1FavNSusPZYQ3qcVL6yjKNR97OJeBvXCIjOLcuw085lfkGXHuD/hf43s8pyTFjXiwQ== X-Received: by 2002:a92:dac8:: with SMTP id o8mr20116061ilq.152.1595797197465; Sun, 26 Jul 2020 13:59:57 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id i8sm7089140ilq.67.2020.07.26.13.59.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 26 Jul 2020 13:59:56 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 06QKxuAx013841; Sun, 26 Jul 2020 20:59:56 GMT Subject: [PATCH v1 3/3] svcrdma: CM event handler clean up From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Sun, 26 Jul 2020 16:59:56 -0400 Message-ID: <159579719607.2004.1280396481349688427.stgit@klimt.1015granger.net> In-Reply-To: <159579718507.2004.16208139278801479272.stgit@klimt.1015granger.net> References: <159579718507.2004.16208139278801479272.stgit@klimt.1015granger.net> User-Agent: StGit/0.23 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Now that there's a core tracepoint that reports these events, there's no need to maintain dprintk() call sites in each arm of the switch statements. We also refresh the documenting comments. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_transport.c | 56 +++++++++++++----------------- 1 file changed, 25 insertions(+), 31 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index aa60f75c8c1d..fb044792b571 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -237,62 +237,56 @@ static void handle_connect_req(struct rdma_cm_id *new_cma_id, svc_xprt_enqueue(&listen_xprt->sc_xprt); } -/* - * Handles events generated on the listening endpoint. These events will be - * either be incoming connect requests or adapter removal events. +/** + * svc_rdma_listen_handler - Handle CM events generated on a listening endpoint + * @cma_id: the server's listener rdma_cm_id + * @event: details of the event + * + * Return values: + * %0: Do not destroy @cma_id + * %1: Destroy @cma_id (never returned here) + * + * NB: There is never a DEVICE_REMOVAL event for INADDR_ANY listeners. */ -static int rdma_listen_handler(struct rdma_cm_id *cma_id, - struct rdma_cm_event *event) +static int svc_rdma_listen_handler(struct rdma_cm_id *cma_id, + struct rdma_cm_event *event) { switch (event->event) { case RDMA_CM_EVENT_CONNECT_REQUEST: - dprintk("svcrdma: Connect request on cma_id=%p, xprt = %p, " - "event = %s (%d)\n", cma_id, cma_id->context, - rdma_event_msg(event->event), event->event); handle_connect_req(cma_id, &event->param.conn); break; default: - /* NB: No device removal upcall for INADDR_ANY listeners */ - dprintk("svcrdma: Unexpected event on listening endpoint %p, " - "event = %s (%d)\n", cma_id, - rdma_event_msg(event->event), event->event); break; } - return 0; } -static int rdma_cma_handler(struct rdma_cm_id *cma_id, - struct rdma_cm_event *event) +/** + * svc_rdma_cma_handler - Handle CM events on client connections + * @cma_id: the server's listener rdma_cm_id + * @event: details of the event + * + * Return values: + * %0: Do not destroy @cma_id + * %1: Destroy @cma_id (never returned here) + */ +static int svc_rdma_cma_handler(struct rdma_cm_id *cma_id, + struct rdma_cm_event *event) { struct svcxprt_rdma *rdma = cma_id->context; struct svc_xprt *xprt = &rdma->sc_xprt; switch (event->event) { case RDMA_CM_EVENT_ESTABLISHED: - /* Accept complete */ - dprintk("svcrdma: Connection completed on DTO xprt=%p, " - "cm_id=%p\n", xprt, cma_id); clear_bit(RDMAXPRT_CONN_PENDING, &rdma->sc_flags); svc_xprt_enqueue(xprt); break; case RDMA_CM_EVENT_DISCONNECTED: - dprintk("svcrdma: Disconnect on DTO xprt=%p, cm_id=%p\n", - xprt, cma_id); - set_bit(XPT_CLOSE, &xprt->xpt_flags); - svc_xprt_enqueue(xprt); - break; case RDMA_CM_EVENT_DEVICE_REMOVAL: - dprintk("svcrdma: Device removal cma_id=%p, xprt = %p, " - "event = %s (%d)\n", cma_id, xprt, - rdma_event_msg(event->event), event->event); set_bit(XPT_CLOSE, &xprt->xpt_flags); svc_xprt_enqueue(xprt); break; default: - dprintk("svcrdma: Unexpected event on DTO endpoint %p, " - "event = %s (%d)\n", cma_id, - rdma_event_msg(event->event), event->event); break; } return 0; @@ -318,7 +312,7 @@ static struct svc_xprt *svc_rdma_create(struct svc_serv *serv, set_bit(XPT_LISTENER, &cma_xprt->sc_xprt.xpt_flags); strcpy(cma_xprt->sc_xprt.xpt_remotebuf, "listener"); - listen_id = rdma_create_id(net, rdma_listen_handler, cma_xprt, + listen_id = rdma_create_id(net, svc_rdma_listen_handler, cma_xprt, RDMA_PS_TCP, IB_QPT_RC); if (IS_ERR(listen_id)) { ret = PTR_ERR(listen_id); @@ -482,7 +476,7 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) goto errout; /* Swap out the handler */ - newxprt->sc_cm_id->event_handler = rdma_cma_handler; + newxprt->sc_cm_id->event_handler = svc_rdma_cma_handler; /* Construct RDMA-CM private message */ pmsg.cp_magic = rpcrdma_cmp_magic;