From patchwork Mon Aug 12 15:47:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13760817 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 258461474C3 for ; Mon, 12 Aug 2024 15:48:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723477689; cv=none; b=iZ5akZsP74NmWvbzJKIcxC7cGVJ07MJ8zYlQXBDZB8VE8k+FUtp78pNIlJTxNW7EhED6/YMandI8YiUGDZ30vYk1IdAQa30DeENwHtI8yJKLw+ds3BxJzJOF17HLDExOugXJjDLOl5kNgGaNlilGyFitOyKtyEoKx97rg/sHi1c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723477689; c=relaxed/simple; bh=k0m32zStcWiR9oAdVE1hhSQ7NggAdOm/k9S8q9Ko1B0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T2tf7zuO6C5TVmb5elearROIEmPESJQ58dWQcqvYIarySblp28PuWz7JmZ2XMKip1NcIYQREZjGSaS2styjANKlx0o8lyKF90+xNZqC2T4QV5gO1pdkNX1bU1BJcSdPlgQEgj0zmB8agiu84uiyPX9Zr4wlVqjPLj6mxcnLXrJY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Q9AsJPqH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Q9AsJPqH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3ACABC4AF09; Mon, 12 Aug 2024 15:48:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723477688; bh=k0m32zStcWiR9oAdVE1hhSQ7NggAdOm/k9S8q9Ko1B0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q9AsJPqHjUMfu3xar6CM7m2/NwliXDhFMzGeg9cHSsmJ3+D+LwfpY3ydrMWjtsCr0 XVXmgh1rQTa/4b2iYV5SgE21lVMj7AkFNDlhkMYvEDhrY8XfnmRSqezgzhswuBQPFF 0LH8zHJbU4pUFDxtChL1Zfq9yHd0VrE7KOE57waobCrL2zMckIGiWxWXVGmqArdZKj 6rCgvVFWvxiITPcaeJVVUAOkRk/mq9+r5VAvT6LOlMBHyjBLLyCncSKrFtS6x7m8if Ap9DGAu9WYPd64bJw0PA9UrgxgSJTXgUw8wdG4zTmzHEcRa8eWaomCibFHBS6oe3If /DaSRV8x9WLNA== From: cel@kernel.org To: Anna Schumaker Cc: Sagi Grimberg , , Chuck Lever Subject: [PATCH 1/3] rpcrdma: Device kref is over-incremented on error from xa_alloc Date: Mon, 12 Aug 2024 11:47:57 -0400 Message-ID: <20240812154759.29870-2-cel@kernel.org> X-Mailer: git-send-email 2.45.1 In-Reply-To: <20240812154759.29870-1-cel@kernel.org> References: <20240812154759.29870-1-cel@kernel.org> Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Chuck Lever If the device's reference count is too high, the device completion callback never fires. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/ib_client.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/sunrpc/xprtrdma/ib_client.c b/net/sunrpc/xprtrdma/ib_client.c index a938c19c3490..4d1e9fa89573 100644 --- a/net/sunrpc/xprtrdma/ib_client.c +++ b/net/sunrpc/xprtrdma/ib_client.c @@ -62,9 +62,9 @@ int rpcrdma_rn_register(struct ib_device *device, if (!rd || test_bit(RPCRDMA_RD_F_REMOVING, &rd->rd_flags)) return -ENETUNREACH; - kref_get(&rd->rd_kref); if (xa_alloc(&rd->rd_xa, &rn->rn_index, rn, xa_limit_32b, GFP_KERNEL) < 0) return -ENOMEM; + kref_get(&rd->rd_kref); rn->rn_done = done; return 0; } From patchwork Mon Aug 12 15:47:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13760818 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEBAF1474C3 for ; Mon, 12 Aug 2024 15:48:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723477691; cv=none; b=T0ZU6fFIz0ub2sMfUC8K57Wupxbi9ND9Ew40NSVB1aqBFUu6ugJA3pI2B94uASeef7x/iw3I3G1O88jqa3LW1+18kVwsZjGyaOlOf1r03nQugXnVhEwR4oQyrsWO0cUBdiHPQBn/wTSS0LZSwZcBNSXsYBjyDOx42zEDs3aMILk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723477691; c=relaxed/simple; bh=c2LidYo9gBGy1+JOSYGpE7X2oUTBq/HLdC73inBXOOg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gij6XgAg3vB/1z/7AOR7tfnL+2lZ4/UGITq+KiTL/cEtZtS5MPphkakJL7/CQdlHpOC4qHxLeY8pqG5ER2+M5IITjfTOaIDZE5fK5g5lT8nXuDdwpXeXm7a7eNtHOhxV98mxgWZAPRbrTe4YEINHwOqMylcXXM0egAd1yeQA7Us= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KY0YkykE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KY0YkykE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 090E6C4AF10; Mon, 12 Aug 2024 15:48:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723477690; bh=c2LidYo9gBGy1+JOSYGpE7X2oUTBq/HLdC73inBXOOg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KY0YkykEcXbvUZyD+NSBkOtNnCtMmqChKaMcz2H3aJiBY9u+2yk/CXBiWfX9eZ88Z NXSGEOCrUuVegf5m/ox34tkByiNcAX53L+Hh6QFDC+G9BtEzQKjp4wBw0c/97WIIOh KK0mkeembxId73KxWJ6EfnBfOegcqXrkXO9T5CcnUangESzibeQPZ0dSb6Su28dujM 9SQr9cw6BSZ1zrKoUUx/+xCj7994nVr7f2DETWp1LFAlNFr9zrtvSPcAOdJXuw4pov xy9bWdQRAQiRM2MQ1cSHShqlv/7mNbPHiWYeFU36PH8ON5vNdbjtE7DL99QV0DxyWG ihzKsSFLmADGQ== From: cel@kernel.org To: Anna Schumaker Cc: Sagi Grimberg , , Chuck Lever Subject: [PATCH 2/3] rpcrdma: Use XA_FLAGS_ALLOC instead of XA_FLAGS_ALLOC1 Date: Mon, 12 Aug 2024 11:47:58 -0400 Message-ID: <20240812154759.29870-3-cel@kernel.org> X-Mailer: git-send-email 2.45.1 In-Reply-To: <20240812154759.29870-1-cel@kernel.org> References: <20240812154759.29870-1-cel@kernel.org> Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Chuck Lever Nit: The built-in xa_limit_32b range starts at 0, but XA_FLAGS_ALLOC1 configures the xarray's allocator to start at 1. Adopt the more conventional XA_FLAGS_ALLOC because there's no mechanical reason to skip 0. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/ib_client.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/sunrpc/xprtrdma/ib_client.c b/net/sunrpc/xprtrdma/ib_client.c index 4d1e9fa89573..7913d7bad23d 100644 --- a/net/sunrpc/xprtrdma/ib_client.c +++ b/net/sunrpc/xprtrdma/ib_client.c @@ -111,7 +111,7 @@ static int rpcrdma_add_one(struct ib_device *device) return -ENOMEM; kref_init(&rd->rd_kref); - xa_init_flags(&rd->rd_xa, XA_FLAGS_ALLOC1); + xa_init_flags(&rd->rd_xa, XA_FLAGS_ALLOC); rd->rd_device = device; init_completion(&rd->rd_done); ib_set_client_data(device, &rpcrdma_ib_client, rd); From patchwork Mon Aug 12 15:47:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13760819 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 477731474C3 for ; Mon, 12 Aug 2024 15:48:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723477692; cv=none; b=bQ+rXx1kV9cHEkIloytV0eIcxtVDJMcWxyjeX02IY9hmOABL0t2K1nlcXjggG4op6SxYsH08USG0YrnHskAQ3kEvk5sI8rITZIav0KjqqYmFULj20A+pKKJV687PuVZAIC1pj0B0xSprg/2LBEeMUJrv1qSPvb8Uan+cskJoJAs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723477692; c=relaxed/simple; bh=v2TCkm8mpwdhAyyFfDqJPr8AA4Zb/E5f18OoYCAd0QE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fdDjkErbdT+IQ2c3emoUXyoVL+skiKSdxuZRHsp/oFYyH7DsOhFlpYnAPkuktuDhVRM4lgVCmVSMbKJ7wcIyIr1D+kKgFo/2SfyMUS033gyxSnd1ZMkydLeDJ7EnKl03JezbKR+s3pzNlPmMFJghh78/evENTErOf9+mbn841OE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EyaQ665T; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EyaQ665T" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5DEDFC32782; Mon, 12 Aug 2024 15:48:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723477691; bh=v2TCkm8mpwdhAyyFfDqJPr8AA4Zb/E5f18OoYCAd0QE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EyaQ665TWw0NUvmkaetB9l/VsxuAhpYRO9nBGP97tsv6zAX6BXldwh5fv4MPZnVmI 11wyk8CGEUPCB8Pz0qB6/chbz4+8ThKwvADJnp5j03uY64Z04mtT6aPtj4oUl1JgTh 89H7aoqYLFwcUcegDecV22OZOf4pKTMGLXPXo2VywBmXSJ2Tjqtt/N9/b4MKglXZd5 6nWeqRQAJpZns0xSpoHHptbHA2Paj12yqEX6eWpjbvyQi0z0sPjJc2MqO0dUMxPi2n edaY/+v+LcDHfo8uOBrYTuWyQhZj8P4cZBHa+tUc3G7gwsDBJP1OXNsbjd+lNEWt9l tc5wJO9980hIQ== From: cel@kernel.org To: Anna Schumaker Cc: Sagi Grimberg , , Chuck Lever Subject: [PATCH 3/3] rpcrdma: Trace connection registration and unregistration Date: Mon, 12 Aug 2024 11:47:59 -0400 Message-ID: <20240812154759.29870-4-cel@kernel.org> X-Mailer: git-send-email 2.45.1 In-Reply-To: <20240812154759.29870-1-cel@kernel.org> References: <20240812154759.29870-1-cel@kernel.org> Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Chuck Lever These new trace points record xarray indices and the time of endpoint registration and unregistration, to co-ordinate with device removal events. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 36 +++++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/ib_client.c | 2 ++ 2 files changed, 38 insertions(+) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 9141398591e0..e6a72646c507 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -2300,6 +2300,42 @@ DEFINE_CLIENT_DEVICE_EVENT(rpcrdma_client_remove_one); DEFINE_CLIENT_DEVICE_EVENT(rpcrdma_client_wait_on); DEFINE_CLIENT_DEVICE_EVENT(rpcrdma_client_remove_one_done); +DECLARE_EVENT_CLASS(rpcrdma_client_register_class, + TP_PROTO( + const struct ib_device *device, + const struct rpcrdma_notification *rn + ), + + TP_ARGS(device, rn), + + TP_STRUCT__entry( + __string(name, device->name) + __field(void *, callback) + __field(u32, index) + ), + + TP_fast_assign( + __assign_str(name); + __entry->callback = rn->rn_done; + __entry->index = rn->rn_index; + ), + + TP_printk("device=%s index=%u done callback=%pS\n", + __get_str(name), __entry->index, __entry->callback + ) +); + +#define DEFINE_CLIENT_REGISTER_EVENT(name) \ + DEFINE_EVENT(rpcrdma_client_register_class, name, \ + TP_PROTO( \ + const struct ib_device *device, \ + const struct rpcrdma_notification *rn \ + ), \ + TP_ARGS(device, rn)) + +DEFINE_CLIENT_REGISTER_EVENT(rpcrdma_client_register); +DEFINE_CLIENT_REGISTER_EVENT(rpcrdma_client_unregister); + #endif /* _TRACE_RPCRDMA_H */ #include diff --git a/net/sunrpc/xprtrdma/ib_client.c b/net/sunrpc/xprtrdma/ib_client.c index 7913d7bad23d..8507cd4d8921 100644 --- a/net/sunrpc/xprtrdma/ib_client.c +++ b/net/sunrpc/xprtrdma/ib_client.c @@ -66,6 +66,7 @@ int rpcrdma_rn_register(struct ib_device *device, return -ENOMEM; kref_get(&rd->rd_kref); rn->rn_done = done; + trace_rpcrdma_client_register(device, rn); return 0; } @@ -91,6 +92,7 @@ void rpcrdma_rn_unregister(struct ib_device *device, if (!rd) return; + trace_rpcrdma_client_unregister(device, rn); xa_erase(&rd->rd_xa, rn->rn_index); kref_put(&rd->rd_kref, rpcrdma_rn_release); }