| Message ID | 20230713173511.24651-1-olga.kornievskaia@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [1/1] NFSv4.1: fix pnfs MDS=DS session trunking | expand |
Hi Olga, On Thu, Jul 13, 2023 at 1:39 PM Olga Kornievskaia <olga.kornievskaia@gmail.com> wrote: > > From: Olga Kornievskaia <kolga@netapp.com> > > Currently, when GETDEVICEINFO returns multiple locations where each > is a different IP but the server's identity is same as MDS, then > nfs4_set_ds_client() finds the existing nfs_client structure which > has the MDS's max_connect value (and if it's 1), then the 1st IP > on the DS's list will get dropped due to MDS trunking rules. Other > IPs would be added as they fall under the pnfs trunking rules. > > Instead, this patch prposed to treat MDS=DS as DS trunking and > make sure that MDS's max_connect limit does not apply to the > 1st IP returned in the GETDEVICEINFO list. > > Signed-off-by: Olga Kornievskaia <kolga@netapp.com> > --- > fs/nfs/nfs4client.c | 7 ++++++- > net/sunrpc/clnt.c | 7 +++++-- > 2 files changed, 11 insertions(+), 3 deletions(-) > > diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c > index 27fb25567ce7..b35acd79b895 100644 > --- a/fs/nfs/nfs4client.c > +++ b/fs/nfs/nfs4client.c > @@ -417,6 +417,7 @@ static void nfs4_add_trunk(struct nfs_client *clp, struct nfs_client *old) > .net = old->cl_net, > .servername = old->cl_hostname, > }; > + int max_connect = old->cl_max_connect; > > if (clp->cl_proto != old->cl_proto) > return; > @@ -428,9 +429,12 @@ static void nfs4_add_trunk(struct nfs_client *clp, struct nfs_client *old) > > xprt_args.dstaddr = clp_sap; > xprt_args.addrlen = clp_salen; > + if (clp->cl_max_connect != old->cl_max_connect && > + test_bit(NFS_CS_DS, &clp->cl_flags)) > + max_connect = clp->cl_max_connect; > > rpc_clnt_add_xprt(old->cl_rpcclient, &xprt_args, > - rpc_clnt_test_and_add_xprt, NULL); > + rpc_clnt_test_and_add_xprt, &max_connect); > } > > /** > @@ -1010,6 +1014,7 @@ struct nfs_client *nfs4_set_ds_client(struct nfs_server *mds_srv, > __set_bit(NFS_CS_NORESVPORT, &cl_init.init_flags); > > __set_bit(NFS_CS_DS, &cl_init.init_flags); > + cl_init.max_connect = NFS_MAX_TRANSPORTS; > /* > * Set an authflavor equual to the MDS value. Use the MDS nfs_client > * cl_ipaddr so as to use the same EXCHANGE_ID co_ownerid as the MDS > diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c > index d7c697af3762..dfdb4bc96367 100644 > --- a/net/sunrpc/clnt.c > +++ b/net/sunrpc/clnt.c > @@ -2908,12 +2908,15 @@ static const struct rpc_call_ops rpc_cb_add_xprt_call_ops = { > */ > int rpc_clnt_test_and_add_xprt(struct rpc_clnt *clnt, > struct rpc_xprt_switch *xps, struct rpc_xprt *xprt, > - void *dummy) > + void *in_max_connect) I was wondering if you could fix up the documentation comment above this function for the new parameter? I'm seeing this when I compile with W=1: net/sunrpc/clnt.c:2913: warning: Function parameter or member 'in_max_connect' not described in 'rpc_clnt_test_and_add_xprt' net/sunrpc/clnt.c:2913: warning: Excess function parameter 'dummy' description in 'rpc_clnt_test_and_add_xprt' Thanks, Anna > { > struct rpc_cb_add_xprt_calldata *data; > struct rpc_task *task; > + int max_connect = clnt->cl_max_connect; > > - if (xps->xps_nunique_destaddr_xprts + 1 > clnt->cl_max_connect) { > + if (in_max_connect) > + max_connect = *(int *)in_max_connect; > + if (xps->xps_nunique_destaddr_xprts + 1 > max_connect) { > rcu_read_lock(); > pr_warn("SUNRPC: reached max allowed number (%d) did not add " > "transport to server: %s\n", clnt->cl_max_connect, > -- > 2.39.1 >
diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c index 27fb25567ce7..b35acd79b895 100644 --- a/fs/nfs/nfs4client.c +++ b/fs/nfs/nfs4client.c @@ -417,6 +417,7 @@ static void nfs4_add_trunk(struct nfs_client *clp, struct nfs_client *old) .net = old->cl_net, .servername = old->cl_hostname, }; + int max_connect = old->cl_max_connect; if (clp->cl_proto != old->cl_proto) return; @@ -428,9 +429,12 @@ static void nfs4_add_trunk(struct nfs_client *clp, struct nfs_client *old) xprt_args.dstaddr = clp_sap; xprt_args.addrlen = clp_salen; + if (clp->cl_max_connect != old->cl_max_connect && + test_bit(NFS_CS_DS, &clp->cl_flags)) + max_connect = clp->cl_max_connect; rpc_clnt_add_xprt(old->cl_rpcclient, &xprt_args, - rpc_clnt_test_and_add_xprt, NULL); + rpc_clnt_test_and_add_xprt, &max_connect); } /** @@ -1010,6 +1014,7 @@ struct nfs_client *nfs4_set_ds_client(struct nfs_server *mds_srv, __set_bit(NFS_CS_NORESVPORT, &cl_init.init_flags); __set_bit(NFS_CS_DS, &cl_init.init_flags); + cl_init.max_connect = NFS_MAX_TRANSPORTS; /* * Set an authflavor equual to the MDS value. Use the MDS nfs_client * cl_ipaddr so as to use the same EXCHANGE_ID co_ownerid as the MDS diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index d7c697af3762..dfdb4bc96367 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2908,12 +2908,15 @@ static const struct rpc_call_ops rpc_cb_add_xprt_call_ops = { */ int rpc_clnt_test_and_add_xprt(struct rpc_clnt *clnt, struct rpc_xprt_switch *xps, struct rpc_xprt *xprt, - void *dummy) + void *in_max_connect) { struct rpc_cb_add_xprt_calldata *data; struct rpc_task *task; + int max_connect = clnt->cl_max_connect; - if (xps->xps_nunique_destaddr_xprts + 1 > clnt->cl_max_connect) { + if (in_max_connect) + max_connect = *(int *)in_max_connect; + if (xps->xps_nunique_destaddr_xprts + 1 > max_connect) { rcu_read_lock(); pr_warn("SUNRPC: reached max allowed number (%d) did not add " "transport to server: %s\n", clnt->cl_max_connect,