From patchwork Sat Jun 27 16:35:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11629757 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4F4301392 for ; Sat, 27 Jun 2020 16:35:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 367E920724 for ; Sat, 27 Jun 2020 16:35:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="J6PMz3wh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726489AbgF0QfH (ORCPT ); Sat, 27 Jun 2020 12:35:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725882AbgF0QfH (ORCPT ); Sat, 27 Jun 2020 12:35:07 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1EFEC061794; Sat, 27 Jun 2020 09:35:06 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id l6so11675115qkc.6; Sat, 27 Jun 2020 09:35:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=hqrnM2P86aSZRrC5NtmU271qkiumzYYR3Yt7LqqAEUc=; b=J6PMz3whRLurBm6qpccrsfC2D0yZUvlOLgOI3Sg7viz+70uV9xC6f0/Y7kzD5PJ6+i RCjWEEtpfPWK930xqPstd6AZ+ap49b+NPMFPw7fA0/bxbbIpwLDQWpkLqgIbnXx89crs SmwZKMLBTUGx5O9+pYuSUSpjix4erRB9P6r3f3VIzWfOFvH5HIHT0q5ZJXbVDAhwvv++ JqZqCA9Jso7Lky/xvylI8jiwMxknJ80muBD6qBE/DtrRZMdU/C3pPEhsu7lo3NdDk/Dm 0Oxq5wUzJliNIUyYjHU2FtNKJZtUhvr3D+sQ0SykEAGTWBuE05j83gJgz827liZwaZMc 3nxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=hqrnM2P86aSZRrC5NtmU271qkiumzYYR3Yt7LqqAEUc=; b=KvLVURGQr9whVWL6IAJ2Ti+jvtWjzM2PmmUgQQGvMmq8k5VEsCFpIJ0+60q4DIsgzQ Mxar9ShXgKsb/hoXeT4A+N5J9tprb369f5kI2dzGygPRbH3kiBeaoBDdVDzElB7Uxb9p Icz8nF2o7GeA1+Ge04G8JbK44VI+va8157seGNsX0uyHSYmIKdUkk2R3tCTVFgwFrwbV HTfB7E2cnkQPuSYCwl+FJIVG6VIsyx7p3FEYhoHDyXWdNbmc2G/1UMmhwrWrROTR5c/Q fWApRCBjcDGVUssG0W0+tz+aoD41r08d7FJBSOzyLba1oAN1vLLgdjhQ3JzNaL3/Dstf UBmw== X-Gm-Message-State: AOAM532WhC2+pxe6rqZEESsItz0XLNnNeisnl/NSZYobRbJQ3mbisjnF eNrnHuXjkV4vz9QKCBeKY/w= X-Google-Smtp-Source: ABdhPJz/dTTvjuIk8kgihzZIy6g9s7Ai2OSL8ruUdz0cIhq2GSMYvxxbD6eUts2WiamQ/UfOi6b5Ag== X-Received: by 2002:a37:4c0d:: with SMTP id z13mr7813682qka.170.1593275706212; Sat, 27 Jun 2020 09:35:06 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id z17sm9998959qth.24.2020.06.27.09.35.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 27 Jun 2020 09:35:05 -0700 (PDT) Received: from manet.1015granger.net (manet.1015granger.net [192.168.1.51]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 05RGZ49O003758; Sat, 27 Jun 2020 16:35:04 GMT Subject: [PATCH v1 1/4] xprtrdma: Fix double-free in rpcrdma_ep_create() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Cc: dan@kernelim.com Date: Sat, 27 Jun 2020 12:35:04 -0400 Message-ID: <20200627163504.22826.98966.stgit@manet.1015granger.net> In-Reply-To: <20200627162911.22826.34426.stgit@manet.1015granger.net> References: <20200627162911.22826.34426.stgit@manet.1015granger.net> User-Agent: StGit/0.22-38-gfb18 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org In the error paths, there's no need to call kfree(ep) after calling rpcrdma_ep_put(ep). Fixes: e28ce90083f0 ("xprtrdma: kmalloc rpcrdma_ep separate from rpcrdma_xprt") Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/verbs.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 2198c8ec8dff..e4c0df7c7d78 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -406,8 +406,8 @@ static int rpcrdma_ep_create(struct rpcrdma_xprt *r_xprt) id = rpcrdma_create_id(r_xprt, ep); if (IS_ERR(id)) { - rc = PTR_ERR(id); - goto out_free; + kfree(ep); + return PTR_ERR(id); } __module_get(THIS_MODULE); device = id->device; @@ -506,9 +506,6 @@ static int rpcrdma_ep_create(struct rpcrdma_xprt *r_xprt) out_destroy: rpcrdma_ep_put(ep); rdma_destroy_id(id); -out_free: - kfree(ep); - r_xprt->rx_ep = NULL; return rc; } From patchwork Sat Jun 27 16:35:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11629761 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 79ABE138C for ; Sat, 27 Jun 2020 16:35:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6116F20707 for ; Sat, 27 Jun 2020 16:35:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mO3Ez2f5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726506AbgF0QfM (ORCPT ); Sat, 27 Jun 2020 12:35:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57206 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725882AbgF0QfM (ORCPT ); Sat, 27 Jun 2020 12:35:12 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 217F1C061794; Sat, 27 Jun 2020 09:35:12 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id e13so11676778qkg.5; Sat, 27 Jun 2020 09:35:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=RUw34TT2rSwOKq3vM9E+XLzokUjgR20CFjUZD3sc9Fc=; b=mO3Ez2f5Ap/EL5DGjuHDBZmpv5YtmMQs7dkrchSG06WLT3PfqpT3miCSdXf/PyoNRj zppWCTbECpKWom7pggv/7O4G13EWt5V3a1xEtBqc+QTjZj99BgnkDKP02ak4E4/R4yd7 tMVMLVD7/TNrwjutDTFGahYBcjOu5V5K4pwFjuA8QulAcSOhx02yJQNtCh9erJn/jK99 yQIQpmnEUUPM3Q+yT5/irOvHTPbnXwd0qlEkYx2BdsiqpBIhdFGBHNPxFkBiV0w9B4qf yGri/cEt/+GrmmBjByt3tGLdvu1P0QK6rPt4cEXwK9f4mpHYThyInC2e3FSVBgO4Mr+Q okfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=RUw34TT2rSwOKq3vM9E+XLzokUjgR20CFjUZD3sc9Fc=; b=JETrUg9yifFrboSO3NeS86hWyY6V/1C14Q3FKzG65+bA9djseXF323isdjjFgFHvf1 Y7y1KKjGn3lLCfCYUE3T9UMN2JZvrFpv8RYKN/nKow0N5rGd9Dz/2ZsbUSPr9I+PbMHp jss4LS1FyyEDZB50guvkSEQ2HVt2sZQYOsctcm9SkRaMHUuOXyfd+CvSBkkLeIuUL8mM P+G0G9gW1OdPtLSFyjRI0z+hkf0HN2BntnyFnLmXsEffydwYSnW5tGexDhFNrG8Qaqhm oqtS81SQAJ3p7tgTS+c6VxK18yNrwqfImuEVkpRw55ugdekNWYUt8uhNJj9xLJ5fZ4fe EkJA== X-Gm-Message-State: AOAM531We9+Y6JRHcbnHQFtkwHCbR/SIZ30PWugeJ543TN8RdJ9zdJIN gw9FgN1ocIdchW7ri4rhKsE= X-Google-Smtp-Source: ABdhPJwLi52XM1ZdJoOYcGT26RGrluigH/nGqa5dr5cx01Foi2QrhjwXcsPs269C195fVroT/If4+Q== X-Received: by 2002:a37:62c6:: with SMTP id w189mr7880519qkb.67.1593275711323; Sat, 27 Jun 2020 09:35:11 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id e129sm11243842qkf.132.2020.06.27.09.35.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 27 Jun 2020 09:35:10 -0700 (PDT) Received: from manet.1015granger.net (manet.1015granger.net [192.168.1.51]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 05RGZ90c003761; Sat, 27 Jun 2020 16:35:09 GMT Subject: [PATCH v1 2/4] xprtrdma: Fix recursion into rpcrdma_xprt_disconnect() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Cc: dan@kernelim.com Date: Sat, 27 Jun 2020 12:35:09 -0400 Message-ID: <20200627163509.22826.24171.stgit@manet.1015granger.net> In-Reply-To: <20200627162911.22826.34426.stgit@manet.1015granger.net> References: <20200627162911.22826.34426.stgit@manet.1015granger.net> User-Agent: StGit/0.22-38-gfb18 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Both Dan and I have observed two processes invoking rpcrdma_xprt_disconnect() concurrently. In my case: 1. The connect worker invokes rpcrdma_xprt_disconnect(), which drains the QP and waits for the final completion 2. This causes the newly posted Receive to flush and invoke xprt_force_disconnect() 3. xprt_force_disconnect() sets CLOSE_WAIT and wakes up the RPC task that is holding the transport lock 4. The RPC task invokes xprt_connect(), which calls ->ops->close 5. xprt_rdma_close() invokes rpcrdma_xprt_disconnect(), which tries to destroy the QP. Deadlock. To prevent xprt_force_disconnect() from waking anything, handle the clean up after a failed connection attempt in the xprt's sndtask. The retry loop is removed from rpcrdma_xprt_connect() to ensure that the newly allocated ep and id are properly released before a REJECTED connection attempt can be retried. Reported-by: Dan Aloni Fixes: e28ce90083f0 ("xprtrdma: kmalloc rpcrdma_ep separate from rpcrdma_xprt") Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/transport.c | 5 +++++ net/sunrpc/xprtrdma/verbs.c | 10 ++-------- 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index 14165b673b20..053c8ab1265a 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -249,6 +249,11 @@ xprt_rdma_connect_worker(struct work_struct *work) xprt->stat.connect_start; xprt_set_connected(xprt); rc = -EAGAIN; + } else { + /* Force a call to xprt_rdma_close to clean up */ + spin_lock(&xprt->transport_lock); + set_bit(XPRT_CLOSE_WAIT, &xprt->state); + spin_unlock(&xprt->transport_lock); } xprt_wake_pending_tasks(xprt, rc); } diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index e4c0df7c7d78..641a3ca0fc8f 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -290,7 +290,7 @@ rpcrdma_cm_event_handler(struct rdma_cm_id *id, struct rdma_cm_event *event) sap, rdma_reject_msg(id, event->status)); ep->re_connect_status = -ECONNREFUSED; if (event->status == IB_CM_REJ_STALE_CONN) - ep->re_connect_status = -EAGAIN; + ep->re_connect_status = -ENOTCONN; goto disconnected; case RDMA_CM_EVENT_DISCONNECTED: ep->re_connect_status = -ECONNABORTED; @@ -521,8 +521,6 @@ int rpcrdma_xprt_connect(struct rpcrdma_xprt *r_xprt) struct rpcrdma_ep *ep; int rc; -retry: - rpcrdma_xprt_disconnect(r_xprt); rc = rpcrdma_ep_create(r_xprt); if (rc) return rc; @@ -550,17 +548,13 @@ int rpcrdma_xprt_connect(struct rpcrdma_xprt *r_xprt) wait_event_interruptible(ep->re_connect_wait, ep->re_connect_status != 0); if (ep->re_connect_status <= 0) { - if (ep->re_connect_status == -EAGAIN) - goto retry; rc = ep->re_connect_status; goto out; } rc = rpcrdma_reqs_setup(r_xprt); - if (rc) { - rpcrdma_xprt_disconnect(r_xprt); + if (rc) goto out; - } rpcrdma_mrs_create(r_xprt); out: From patchwork Sat Jun 27 16:35:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11629765 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B1E1D1392 for ; Sat, 27 Jun 2020 16:35:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9A84220724 for ; Sat, 27 Jun 2020 16:35:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IWZ9wXK0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726553AbgF0QfS (ORCPT ); Sat, 27 Jun 2020 12:35:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57220 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726525AbgF0QfR (ORCPT ); Sat, 27 Jun 2020 12:35:17 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84342C061794; Sat, 27 Jun 2020 09:35:17 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id h23so9885124qtr.0; Sat, 27 Jun 2020 09:35:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=3wdZ6UFKpvgBlLdEtBtXrh1crlptSTbcovoH+7aLRMQ=; b=IWZ9wXK0xt1DQCazFK5TkC/xkX6HVWi94XkTfLgCuptFPieec+s+nqh9yX+OEfrHak VkYqZFc0p6waWrvlU4GXe1ZImP+6o0QrrAbI/vtmxkGmf++MIXMsHZvJGY5nsWEAi8dP WGe2/+lsSJhDvCSzov484W+5yFiUc/c4mDTzkfH0XiCcORqqg604fAchGtWKu9soxpAF 5sukU78DE1O/vQeD8pXdiMrWNoR8s6eqFz6FphAtVtiJDvVPqwEGwmEDImeO4tB85wBh TVKxTV7XjestqWu2eMMopmuBJuGo9J0O18zn3CxLlOjq03wRIH3guVliYUnCa7OsxvKk oRxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=3wdZ6UFKpvgBlLdEtBtXrh1crlptSTbcovoH+7aLRMQ=; b=XyBBfG3EL8A2XURU3Uyfb5UIKOfTT3EeqSxP4cFDWbAkC42Wvf6Vx4ymc0Y9h59XNp dn059f2KRR2qMTT0L2Pit/kqGCdtTEqEWnYUEq/B56Z8CWQ3avzKZTNHlIrasUy+8tq9 lRRrTlvT31zptWwWt9mMkuOTNY7p8HP4lRkd8Zyj89m6ucLSajfDje0jpK0UcD3zOFDn LOl0A2nCENlh7ApT+A+ZAsfBoA3R4slOwumhoapFKbXm2URU4Rz/XkJLrvHsf4OCIAUr WYUcBPKp0YW3qjjCaDCIjZcdMj1qCUvpo+K+Gt51NxiQF0UjZKOhnLMWcE0GuAdTu7ge cnyw== X-Gm-Message-State: AOAM533z/+JVP6y8VcX7BPyia2RNwZWVgs87S244CFc8CxKiCEWJShx8 YpIijK7Co1SBPOP3Ysigs7o= X-Google-Smtp-Source: ABdhPJxHHuM5dJaEuYe14CzJzCyWWEob3EpVuMrH+ade7fGprocCZAq+hItkuMroCNIxoAyEmvP6iQ== X-Received: by 2002:aed:2846:: with SMTP id r64mr8646791qtd.55.1593275716809; Sat, 27 Jun 2020 09:35:16 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id s71sm11470036qke.0.2020.06.27.09.35.15 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 27 Jun 2020 09:35:16 -0700 (PDT) Received: from manet.1015granger.net (manet.1015granger.net [192.168.1.51]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 05RGZFhA003764; Sat, 27 Jun 2020 16:35:15 GMT Subject: [PATCH v1 3/4] xprtrdma: Fix return code from rpcrdma_xprt_connect() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Cc: dan@kernelim.com Date: Sat, 27 Jun 2020 12:35:15 -0400 Message-ID: <20200627163515.22826.65783.stgit@manet.1015granger.net> In-Reply-To: <20200627162911.22826.34426.stgit@manet.1015granger.net> References: <20200627162911.22826.34426.stgit@manet.1015granger.net> User-Agent: StGit/0.22-38-gfb18 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org I noticed that when rpcrdma_xprt_connect() returns -ENOMEM, instead of retrying the connect, the RPC client kills the RPC task that requested the connection. We want a retry here. Fixes: cb586decbb88 ("xprtrdma: Make sendctx queue lifetime the same as connection lifetime") Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/verbs.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 641a3ca0fc8f..13d671dccfd8 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -400,7 +400,7 @@ static int rpcrdma_ep_create(struct rpcrdma_xprt *r_xprt) ep = kzalloc(sizeof(*ep), GFP_NOFS); if (!ep) - return -EAGAIN; + return -ENOTCONN; ep->re_xprt = &r_xprt->rx_xprt; kref_init(&ep->re_kref); @@ -535,10 +535,6 @@ int rpcrdma_xprt_connect(struct rpcrdma_xprt *r_xprt) rpcrdma_ep_get(ep); rpcrdma_post_recvs(r_xprt, true); - rc = rpcrdma_sendctxs_create(r_xprt); - if (rc) - goto out; - rc = rdma_connect(ep->re_id, &ep->re_remote_cma); if (rc) goto out; @@ -552,9 +548,17 @@ int rpcrdma_xprt_connect(struct rpcrdma_xprt *r_xprt) goto out; } + rc = rpcrdma_sendctxs_create(r_xprt); + if (rc) { + rc = -ENOTCONN; + goto out; + } + rc = rpcrdma_reqs_setup(r_xprt); - if (rc) + if (rc) { + rc = -ENOTCONN; goto out; + } rpcrdma_mrs_create(r_xprt); out: From patchwork Sat Jun 27 16:35:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11629769 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 027C1138C for ; Sat, 27 Jun 2020 16:35:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DAE4A20720 for ; Sat, 27 Jun 2020 16:35:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mSRcYDsg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725932AbgF0QfX (ORCPT ); Sat, 27 Jun 2020 12:35:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725882AbgF0QfW (ORCPT ); Sat, 27 Jun 2020 12:35:22 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB3EEC061794; Sat, 27 Jun 2020 09:35:22 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id b4so11651597qkn.11; Sat, 27 Jun 2020 09:35:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=vxYvbaNdmWPDmCOW0aLfaDFq5b3bewpbHDAgyiKlBzg=; b=mSRcYDsgUwpiljSYk1KFxXZEPwwgRvmH1SwnRI66mimp0+9j1A6LVk/GqwZem1hZ/u 5QBL3KnNP/B5T9jbYiXrFX0PaaynrONSnK+lM4NWPZQdFEnYwJ2nQLAbD/qbACdF1IzW wkS0HL6yi5k5twQEEIVNooqo/F3i3khmuvTz1KveaqTCCzgKvkuPsXhp/Cijxrml+Wws F6svvjj1maDSbB/e4GUYvwTIILK7WZ+Tq5eAdDZFUHl8yOVUGqZlYYO67WqCdlGzB/2W //A6T5d0USJpFRy7CtueF2mDfPxqPj21vIdUrGu004TNPsoiEBpR1nMnq8mOUyuhtXEX pABw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=vxYvbaNdmWPDmCOW0aLfaDFq5b3bewpbHDAgyiKlBzg=; b=H8LazghC2zdoUhE7/dh8M2tmqBX2OW/3kKf/KnDo7qfogXjiJE9T49sfQtKaWEq+1U W4vd09eyZo4LLmXXBkqA//si9YH9GAcxriN/iBUTrbMNxLCqmxdzDWEoIPwzlPR2zehV 3Ua+NyOJKot0AP3B2SctAewLGbEr5mcpyCIhilcb7yFIVjwJaX7Z3mkpIN87j2G28MWx 2/1EXCL4GSk8zWWym0dW2ySvEHhdd616YHZeIn3yUxXcjy2INTqHkri6TUx3odwpVULY 6DSzOXWjsRjjyFziAzfh+x2fIM588srF/CBQJvzQIrSzVQCQ5fOKvXyYblz9cKua9rHj LRxQ== X-Gm-Message-State: AOAM531ch1s4wwD58xOVf42M2UUyw1KdO4WvshUvqy+/mMNstmrjBZ3d iPGx0noe2UZyFe/pRYTxQadtJOaZ X-Google-Smtp-Source: ABdhPJzZpKD40ROEsrFi6O3S/5KO0T02JNye6USsQSTGCnJBIUGa41uioaKRDzhPp4j65B5miZPA6g== X-Received: by 2002:a37:6cc1:: with SMTP id h184mr8019775qkc.259.1593275721922; Sat, 27 Jun 2020 09:35:21 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id f3sm12760119qta.44.2020.06.27.09.35.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 27 Jun 2020 09:35:21 -0700 (PDT) Received: from manet.1015granger.net (manet.1015granger.net [192.168.1.51]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 05RGZKx1003767; Sat, 27 Jun 2020 16:35:20 GMT Subject: [PATCH v1 4/4] xprtrdma: Fix handling of connect errors From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Cc: dan@kernelim.com Date: Sat, 27 Jun 2020 12:35:20 -0400 Message-ID: <20200627163520.22826.73749.stgit@manet.1015granger.net> In-Reply-To: <20200627162911.22826.34426.stgit@manet.1015granger.net> References: <20200627162911.22826.34426.stgit@manet.1015granger.net> User-Agent: StGit/0.22-38-gfb18 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Ensure that the connect worker is awoken if an attempt to establish a connection is unsuccessful. Otherwise the worker waits forever and the transport workload hangs. Connect errors should not attempt to destroy the ep, since the connect worker continues to use it after the handler runs, so these errors are now handled independently of DISCONNECTED events. Reported-by: Dan Aloni Fixes: e28ce90083f0 ("xprtrdma: kmalloc rpcrdma_ep separate from rpcrdma_xprt") Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/verbs.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 13d671dccfd8..75c646743df3 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -281,17 +281,19 @@ rpcrdma_cm_event_handler(struct rdma_cm_id *id, struct rdma_cm_event *event) break; case RDMA_CM_EVENT_CONNECT_ERROR: ep->re_connect_status = -ENOTCONN; - goto disconnected; + goto wake_connect_worker; case RDMA_CM_EVENT_UNREACHABLE: ep->re_connect_status = -ENETUNREACH; - goto disconnected; + goto wake_connect_worker; case RDMA_CM_EVENT_REJECTED: dprintk("rpcrdma: connection to %pISpc rejected: %s\n", sap, rdma_reject_msg(id, event->status)); ep->re_connect_status = -ECONNREFUSED; if (event->status == IB_CM_REJ_STALE_CONN) ep->re_connect_status = -ENOTCONN; - goto disconnected; +wake_connect_worker: + wake_up_all(&ep->re_connect_wait); + return 0; case RDMA_CM_EVENT_DISCONNECTED: ep->re_connect_status = -ECONNABORTED; disconnected: