[v5,0/3] Make nfs stats visible in network NS

Message

Josef Bacik Feb. 15, 2024, 7:57 p.m. UTC

v1: https://lore.kernel.org/linux-nfs/[e-mail]/
v2: https://lore.kernel.org/linux-nfs/[e-mail]/
v3: https://lore.kernel.org/linux-nfs/[e-mail]/
v4: https://lore.kernel.org/linux-nfs/[e-mail]/

v4->v5:
- If we clone a client pass through their ->cl_stats so it's populated properly.

v3->v4:
- Fix a weird formatting thing that snuck into 1/3.

v2->v3:
- Split out the nfs and nfsd related changes into their own patches.
- Dropped the change adding sv_stats throuch svc_create()
- Changed the th_cnt to be global, re-arranged it's location.

v1->v2:
- rework the sunprc service creation to take a pointer to the sv_stats.
- dropped ->pg_stats from the svc_program.
- converted all of the nfsd global stats to per-network namespace.
- added the ability to point at a specific rpc_stat for rpc program creation.
- converted the rpc stats for nfs to per-network namespace.

-- Original email --
Hello,

We're currently deploying NFS internally and have run into some oddities with
our usage of containers.  All of the services that mount and export NFS volumes
run inside of containers, specifically all the namespaces including network
namespaces.  Our monitoring is done on a per-container basis, so we need access
to the nfs and nfsd stats that are under /proc/net/sunrpc.  However these are
only tied to the init_net, which makes them invisible to containers in a
different network namespace.

Fix this so that these files are tied to the network namespace.  This allows us
to avoid the hack of bind mounting the hosts /proc into the container in order
to do proper monitoring.  Thanks,

Josef

Josef Bacik (3):
  sunrpc: add a struct rpc_stats arg to rpc_create_args
  nfs: expose /proc/net/sunrpc/nfs in net namespaces
  nfs: make the rpc_stat per net namespace

 fs/nfs/client.c             | 5 ++++-
 fs/nfs/inode.c              | 8 ++++----
 fs/nfs/internal.h           | 2 --
 fs/nfs/netns.h              | 2 ++
 include/linux/sunrpc/clnt.h | 1 +
 net/sunrpc/clnt.c           | 5 ++++-
 6 files changed, 15 insertions(+), 8 deletions(-)

Comments

Petr Vorel June 20, 2024, 11:22 a.m. UTC | #1

From: Josef Bacik <[e-mail]>

> Now that we're exposing the rpc stats on a per-network namespace basis,
> move this struct into struct nfs_net and use that to make sure only the
> per-network namespace stats are exposed.

Hi Josef, all,

I suppose this or previous commit caused global /proc/net/rpc/nfs does not have
rpc statistics. Therefore I send a patch [1] [2] to update LTP test when running
on LTP namespaces. Hope the change was intentional, please let us know if not.

Kind regards,
Petr

[1] https://lore.kernel.org/ltp/[e-mail]/
[2] https://patchwork.ozlabs.org/project/ltp/patch/[e-mail]/

> Signed-off-by: Josef Bacik <[e-mail]>
> ---
>  fs/nfs/client.c   | 5 ++++-
>  fs/nfs/inode.c    | 4 +++-
>  fs/nfs/internal.h | 2 --
>  fs/nfs/netns.h    | 2 ++
>  4 files changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/nfs/client.c b/fs/nfs/client.c
> index 44eca51b2808..4d9249c99989 100644
> --- a/fs/nfs/client.c
> +++ b/fs/nfs/client.c
> @@ -73,7 +73,6 @@ const struct rpc_program nfs_program = {
>  	.number			= NFS_PROGRAM,
>  	.nrvers			= ARRAY_SIZE(nfs_version),
>  	.version		= nfs_version,
> -	.stats			= &nfs_rpcstat,
>  	.pipe_dir_name		= NFS_PIPE_DIRNAME,
>  };
>  
> @@ -502,6 +501,7 @@ int nfs_create_rpc_client(struct nfs_client *clp,
>  			  const struct nfs_client_initdata *cl_init,
>  			  rpc_authflavor_t flavor)
>  {
> +	struct nfs_net		*nn = net_generic(clp->cl_net, nfs_net_id);
>  	struct rpc_clnt		*clnt = NULL;
>  	struct rpc_create_args args = {
>  		.net		= clp->cl_net,
> @@ -513,6 +513,7 @@ int nfs_create_rpc_client(struct nfs_client *clp,
>  		.servername	= clp->cl_hostname,
>  		.nodename	= cl_init->nodename,
>  		.program	= &nfs_program,
> +		.stats		= &nn->rpcstats,
>  		.version	= clp->rpc_ops->version,
>  		.authflavor	= flavor,
>  		.cred		= cl_init->cred,
> @@ -1175,6 +1176,8 @@ void nfs_clients_init(struct net *net)
>  #endif
>  	spin_lock_init(&nn->nfs_client_lock);
>  	nn->boot_time = ktime_get_real();
> +	memset(&nn->rpcstats, 0, sizeof(nn->rpcstats));
> +	nn->rpcstats.program = &nfs_program;
>  
>  	nfs_netns_sysfs_setup(nn, net);
>  }
> diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
> index e11e9c34aa56..91b4d811958a 100644
> --- a/fs/nfs/inode.c
> +++ b/fs/nfs/inode.c
> @@ -2426,8 +2426,10 @@ EXPORT_SYMBOL_GPL(nfs_net_id);
>  
>  static int nfs_net_init(struct net *net)
>  {
> +	struct nfs_net *nn = net_generic(net, nfs_net_id);
> +
>  	nfs_clients_init(net);
> -	rpc_proc_register(net, &nfs_rpcstat);
> +	rpc_proc_register(net, &nn->rpcstats);
>  	return nfs_fs_proc_net_init(net);
>  }
>  
> diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
> index e3722ce6722e..06253695fe53 100644
> --- a/fs/nfs/internal.h
> +++ b/fs/nfs/internal.h
> @@ -449,8 +449,6 @@ int nfs_try_get_tree(struct fs_context *);
>  int nfs_get_tree_common(struct fs_context *);
>  void nfs_kill_super(struct super_block *);
>  
> -extern struct rpc_stat nfs_rpcstat;
> -
>  extern int __init register_nfs_fs(void);
>  extern void __exit unregister_nfs_fs(void);
>  extern bool nfs_sb_active(struct super_block *sb);
> diff --git a/fs/nfs/netns.h b/fs/nfs/netns.h
> index c8374f74dce1..a68b21603ea9 100644
> --- a/fs/nfs/netns.h
> +++ b/fs/nfs/netns.h
> @@ -9,6 +9,7 @@
>  #include <linux/nfs4.h>
>  #include <net/net_namespace.h>
>  #include <net/netns/generic.h>
> +#include <linux/sunrpc/stats.h>
>  
>  struct bl_dev_msg {
>  	int32_t status;
> @@ -34,6 +35,7 @@ struct nfs_net {
>  	struct nfs_netns_client *nfs_client;
>  	spinlock_t nfs_client_lock;
>  	ktime_t boot_time;
> +	struct rpc_stat rpcstats;
>  #ifdef CONFIG_PROC_FS
>  	struct proc_dir_entry *proc_nfsfs;
>  #endif
> -- 
> 2.43.0
> 
>