diff mbox series

[RFC] SUNRPC: Fix a slow server-side memory leak with RPC-over-TCP

Message ID 171208672277.1654.1052289246945629541.stgit@klimt.1015granger.net (mailing list archive)
State New
Headers show
Series [RFC] SUNRPC: Fix a slow server-side memory leak with RPC-over-TCP | expand

Commit Message

Chuck Lever April 2, 2024, 7:38 p.m. UTC 
From: Chuck Lever <chuck.lever@oracle.com>

Jan Schunk reports that his small NFS servers suffer from memory
exhaustion after just a few days. A bisect shows that commit
e18e157bb5c8 ("SUNRPC: Send RPC message on TCP with a single
sock_sendmsg() call") is the first bad commit.

That commit assumed that sock_sendmsg() releases all the pages in
the underlying bio_vec array, but the reality is that it doesn't.
svc_xprt_release() releases the rqst's response pages, but the
record marker page fragment isn't one of those, so it was never
released.

This is a narrow fix that can be applied to stable kernels. A
more extensive fix is in the works.

Reported-by: Jan Schunk <scpcom@gmx.de>
Closes: https://bugzilla.kernel.org/show_bug.cgi?id=218671
Fixes: e18e157bb5c8 ("SUNRPC: Send RPC message on TCP with a single sock_sendmsg() call")
Cc: Alexander Duyck <alexander.duyck@gmail.com>
Cc: Jakub Kacinski <kuba@kernel.org>
Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 net/sunrpc/svcsock.c |    3 +++
 1 file changed, 3 insertions(+)

Comments

Cedric Blancher April 3, 2024, 5:29 a.m. UTC | #1 
On Tue, 2 Apr 2024 at 21:38, Chuck Lever <cel@kernel.org> wrote:
>
> From: Chuck Lever <chuck.lever@oracle.com>
>
> Jan Schunk reports that his small NFS servers suffer from memory
> exhaustion after just a few days. A bisect shows that commit
> e18e157bb5c8 ("SUNRPC: Send RPC message on TCP with a single
> sock_sendmsg() call") is the first bad commit.
>
> That commit assumed that sock_sendmsg() releases all the pages in
> the underlying bio_vec array, but the reality is that it doesn't.
> svc_xprt_release() releases the rqst's response pages, but the
> record marker page fragment isn't one of those, so it was never
> released.
>
> This is a narrow fix that can be applied to stable kernels. A
> more extensive fix is in the works.
>
> Reported-by: Jan Schunk <scpcom@gmx.de>
> Closes: https://bugzilla.kernel.org/show_bug.cgi?id=218671
> Fixes: e18e157bb5c8 ("SUNRPC: Send RPC message on TCP with a single sock_sendmsg() call")

Is this bug present in 6.6 LTS?

Ced
Chuck Lever April 3, 2024, 1:24 p.m. UTC | #2 
On Wed, Apr 03, 2024 at 07:29:00AM +0200, Cedric Blancher wrote:
> On Tue, 2 Apr 2024 at 21:38, Chuck Lever <cel@kernel.org> wrote:
> >
> > From: Chuck Lever <chuck.lever@oracle.com>
> >
> > Jan Schunk reports that his small NFS servers suffer from memory
> > exhaustion after just a few days. A bisect shows that commit
> > e18e157bb5c8 ("SUNRPC: Send RPC message on TCP with a single
> > sock_sendmsg() call") is the first bad commit.
> >
> > That commit assumed that sock_sendmsg() releases all the pages in
> > the underlying bio_vec array, but the reality is that it doesn't.
> > svc_xprt_release() releases the rqst's response pages, but the
> > record marker page fragment isn't one of those, so it was never
> > released.
> >
> > This is a narrow fix that can be applied to stable kernels. A
> > more extensive fix is in the works.
> >
> > Reported-by: Jan Schunk <scpcom@gmx.de>
> > Closes: https://bugzilla.kernel.org/show_bug.cgi?id=218671
> > Fixes: e18e157bb5c8 ("SUNRPC: Send RPC message on TCP with a single sock_sendmsg() call")
> 
> Is this bug present in 6.6 LTS?

It was introduced in v6.6, so yes.

~/linux $ git describe --contains e18e157bb5c8
v6.6-rc1~108^2~27
~/linux $

Once this fix is merged upstream, the LTS maintainers will notice
the Fixes: tag and backport it to v6.6 automatically.
diff mbox series

Patch

diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index 545017a3daa4..be6c6ee85c8f 100644
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -130,6 +130,8 @@  static void svc_reclassify_socket(struct socket *sock)
  */
 static void svc_tcp_release_ctxt(struct svc_xprt *xprt, void *ctxt)
 {
+	if (ctxt)
+		page_frag_free(ctxt);
 }
 
 /**
@@ -1237,6 +1239,7 @@  static int svc_tcp_sendmsg(struct svc_sock *svsk, struct svc_rqst *rqstp,
 		return -ENOMEM;
 	memcpy(buf, &marker, sizeof(marker));
 	bvec_set_virt(rqstp->rq_bvec, buf, sizeof(marker));
+	rqstp->rq_xprt_ctxt = buf;
 
 	count = xdr_buf_to_bvec(rqstp->rq_bvec + 1,
 				ARRAY_SIZE(rqstp->rq_bvec) - 1, &rqstp->rq_res);