From patchwork Tue Sep 6 15:22:49 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 9317351 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 009E2601C0 for ; Tue, 6 Sep 2016 15:33:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 14FD4289B3 for ; Tue, 6 Sep 2016 15:33:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 093C828DE5; Tue, 6 Sep 2016 15:33:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.4 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA60028D23 for ; Tue, 6 Sep 2016 15:33:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933872AbcIFPc7 (ORCPT ); Tue, 6 Sep 2016 11:32:59 -0400 Received: from mail-oi0-f67.google.com ([209.85.218.67]:33920 "EHLO mail-oi0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S935259AbcIFPW5 (ORCPT ); Tue, 6 Sep 2016 11:22:57 -0400 Received: by mail-oi0-f67.google.com with SMTP id v82so8461451oie.1 for ; Tue, 06 Sep 2016 08:22:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=AE+O7RfrXN5UExXmcHRpOhUcx3ZfDl31HgCTLJeKC4Q=; b=EwXg0UJ3mAwSBVjbZGqxcxWLkW+RdjCbgnR/ikyDSWnzO5LfxfLOJ3YSg1/d7Y6DIx e9ASjZiMtCniQnA+CxIa3GIdKQBM8nmsWKoSFiJSPLQ6Vbqq0PePoqimcYmKGzR7DElf uXD5UJSRnsrWiBwGNZARycE/dXezKsdNhGD/VQ9EKWGy3eQ8WF9H1pJEf8A0mmBuKLAJ vhL6sK7X8Ysmk/AIL54XWt/hu75QVhlFZFNq2dx4XNrilwrebJ8EqPJNoSYs7u++IqrN iNVhtwSCld7duCvyzVrdrAZORclAlmoJQqy6vCNVmTb6HLyb27e9y5hh75AYSFvIxe8H VSIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=AE+O7RfrXN5UExXmcHRpOhUcx3ZfDl31HgCTLJeKC4Q=; b=DH+du6JEQSPKFCXHtpgEOiXX/7KEtPe086bNos7bmgUWDVhWMNJs/WdfVMrhC/AA2K sF2oVYVlk9u7u/6NbkhukpStD7lomhC/S/4l0onNlbbMuxSTb2EkmuSwuxUoP0oQMoFI ZKQO9SdmaeWc3zJfMCoZo063tWdKiZDfxttB6V9TW4KZD+3cEvFNPtAXQdjS4BibJs2X lohUPhTgaMTXfrZTh8k2ambrdU/2UkNjE9ej+Vja9S8xaK/TU+EVufEjmLTPZDN4SPdx v+1iYyAtlytbuK7yZNbZ4eitT0FpWnTuUvyMM4aQbdBWIppndsDvWZ3LzlVZbzn5yx+b thiA== X-Gm-Message-State: AE9vXwOc/kRYg89cxD6etJ07HA1dfcnSO9o/cmHNFktKQgWbgCSeTHvIBzgw672/HakYwA== X-Received: by 10.107.205.197 with SMTP id d188mr5286304iog.46.1473175371295; Tue, 06 Sep 2016 08:22:51 -0700 (PDT) Received: from manet.1015granger.net ([2604:8800:100:81fc:ec4:7aff:fe6c:1dce]) by smtp.gmail.com with ESMTPSA id w132sm12266848ita.5.2016.09.06.08.22.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Sep 2016 08:22:50 -0700 (PDT) Subject: [PATCH 1/2] xprtrdma: Revert 3d4cf35bd4fa ("xprtrdma: Reply buffer exhaustion...") From: Chuck Lever To: trond.myklebust@primarydata.com, anna.schumaker@netapp.com Cc: linux-nfs@vger.kernel.org Date: Tue, 06 Sep 2016 11:22:49 -0400 Message-ID: <20160906152249.4958.55165.stgit@manet.1015granger.net> In-Reply-To: <20160906151843.4958.84689.stgit@manet.1015granger.net> References: <20160906151843.4958.84689.stgit@manet.1015granger.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Receive buffer exhaustion, if it were to actually occur, would be catastrophic. However, when there are no reply buffers to post, that means all of them have already been posted and are waiting for incoming replies. By design, there can never be more RPCs in flight than there are available receive buffers. A receive buffer can be left posted after an RPC exits without a received reply; say, due to a credential problem or a soft timeout. This does not result in fewer posted receive buffers than there are pending RPCs, and there is already logic in xprtrdma to deal appropriately with this case. It also looks like the "+ 2" that was removed was accidentally accommodating the number of extra receive buffers needed for receiving backchannel requests. That will need to be addressed by another patch. Fixes: 3d4cf35bd4fa ("xprtrdma: Reply buffer exhaustion can be...") Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/verbs.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index a74d79d..10edfca 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -923,7 +923,7 @@ rpcrdma_buffer_create(struct rpcrdma_xprt *r_xprt) } INIT_LIST_HEAD(&buf->rb_recv_bufs); - for (i = 0; i < buf->rb_max_requests; i++) { + for (i = 0; i < buf->rb_max_requests + 2; i++) { struct rpcrdma_rep *rep; rep = rpcrdma_create_rep(r_xprt); @@ -1076,6 +1076,8 @@ rpcrdma_put_mw(struct rpcrdma_xprt *r_xprt, struct rpcrdma_mw *mw) /* * Get a set of request/reply buffers. + * + * Reply buffer (if available) is attached to send buffer upon return. */ struct rpcrdma_req * rpcrdma_buffer_get(struct rpcrdma_buffer *buffers) @@ -1094,13 +1096,13 @@ rpcrdma_buffer_get(struct rpcrdma_buffer *buffers) out_reqbuf: spin_unlock(&buffers->rb_lock); - pr_warn("rpcrdma: out of request buffers (%p)\n", buffers); + pr_warn("RPC: %s: out of request buffers\n", __func__); return NULL; out_repbuf: - list_add(&req->rl_free, &buffers->rb_send_bufs); spin_unlock(&buffers->rb_lock); - pr_warn("rpcrdma: out of reply buffers (%p)\n", buffers); - return NULL; + pr_warn("RPC: %s: out of reply buffers\n", __func__); + req->rl_reply = NULL; + return req; } /*