| Message ID | 168891733570.3964.15456501153247760888.stgit@manet.1015granger.net (mailing list archive) |
|---|---|
| Headers | show |
| Series | Fix some lock contention in the NFS server's DRC | expand |
On Sun, 2023-07-09 at 11:45 -0400, Chuck Lever wrote: > This series optimizes DRC scalability by freeing cache objects only > once the hash bucket lock is no longer held. There are a couple of > related clean-ups to go along with this optimization. > The conventional wisdom that I've always heard is that a kfree under spinlock is generally no big deal. It can't block and is usually quite fast. Are you able to measure any performance delta from this set? > --- > > Chuck Lever (6): > NFSD: Refactor nfsd_reply_cache_free_locked() > NFSD: Rename nfsd_reply_cache_alloc() > NFSD: Replace nfsd_prune_bucket() > NFSD: Refactor the duplicate reply cache shrinker > NFSD: Remove svc_rqst::rq_cacherep > NFSD: Rename struct svc_cacherep > > > fs/nfsd/cache.h | 8 +- > fs/nfsd/nfscache.c | 203 ++++++++++++++++++++++++------------- > fs/nfsd/nfssvc.c | 10 +- > fs/nfsd/trace.h | 26 ++++- > include/linux/sunrpc/svc.h | 1 - > 5 files changed, 165 insertions(+), 83 deletions(-) > > -- > Chuck Lever >
On Sun, 2023-07-09 at 11:45 -0400, Chuck Lever wrote: > This series optimizes DRC scalability by freeing cache objects only > once the hash bucket lock is no longer held. There are a couple of > related clean-ups to go along with this optimization. > > --- > > Chuck Lever (6): > NFSD: Refactor nfsd_reply_cache_free_locked() > NFSD: Rename nfsd_reply_cache_alloc() > NFSD: Replace nfsd_prune_bucket() > NFSD: Refactor the duplicate reply cache shrinker > NFSD: Remove svc_rqst::rq_cacherep > NFSD: Rename struct svc_cacherep > > > fs/nfsd/cache.h | 8 +- > fs/nfsd/nfscache.c | 203 ++++++++++++++++++++++++------------- > fs/nfsd/nfssvc.c | 10 +- > fs/nfsd/trace.h | 26 ++++- > include/linux/sunrpc/svc.h | 1 - > 5 files changed, 165 insertions(+), 83 deletions(-) > > -- > Chuck Lever > This all looks like reasonable cleanup to me, regardless of whether it's produces measurable optimization. Reviewed-by: Jeff Layton <jlayton@kernel.org>
> On Jul 10, 2023, at 9:03 AM, Jeff Layton <jlayton@kernel.org> wrote: > > On Sun, 2023-07-09 at 11:45 -0400, Chuck Lever wrote: >> This series optimizes DRC scalability by freeing cache objects only >> once the hash bucket lock is no longer held. There are a couple of >> related clean-ups to go along with this optimization. >> > > > The conventional wisdom that I've always heard is that a kfree under > spinlock is generally no big deal. It can't block and is usually quite > fast. Are you able to measure any performance delta from this set? Yes, a couple of percent better throughput with a tmpfs export and a fast transport, which is not a common use case, granted. I think the difference is that, after this change, we're holding each bucket lock for a shorter period of time. That's enough to reduce some lock contention. But I also like the shrinker improvements. >> --- >> >> Chuck Lever (6): >> NFSD: Refactor nfsd_reply_cache_free_locked() >> NFSD: Rename nfsd_reply_cache_alloc() >> NFSD: Replace nfsd_prune_bucket() >> NFSD: Refactor the duplicate reply cache shrinker >> NFSD: Remove svc_rqst::rq_cacherep >> NFSD: Rename struct svc_cacherep >> >> >> fs/nfsd/cache.h | 8 +- >> fs/nfsd/nfscache.c | 203 ++++++++++++++++++++++++------------- >> fs/nfsd/nfssvc.c | 10 +- >> fs/nfsd/trace.h | 26 ++++- >> include/linux/sunrpc/svc.h | 1 - >> 5 files changed, 165 insertions(+), 83 deletions(-) >> >> -- >> Chuck Lever >> > > -- > Jeff Layton <jlayton@kernel.org> -- Chuck Lever
This series optimizes DRC scalability by freeing cache objects only once the hash bucket lock is no longer held. There are a couple of related clean-ups to go along with this optimization. --- Chuck Lever (6): NFSD: Refactor nfsd_reply_cache_free_locked() NFSD: Rename nfsd_reply_cache_alloc() NFSD: Replace nfsd_prune_bucket() NFSD: Refactor the duplicate reply cache shrinker NFSD: Remove svc_rqst::rq_cacherep NFSD: Rename struct svc_cacherep fs/nfsd/cache.h | 8 +- fs/nfsd/nfscache.c | 203 ++++++++++++++++++++++++------------- fs/nfsd/nfssvc.c | 10 +- fs/nfsd/trace.h | 26 ++++- include/linux/sunrpc/svc.h | 1 - 5 files changed, 165 insertions(+), 83 deletions(-) -- Chuck Lever