mbox series

[V2,0/2] percpu_ref & block: reduce memory footprint of percpu_ref in fast path

Message ID 20200902122643.634143-1-ming.lei@redhat.com (mailing list archive)
Headers show
Series percpu_ref & block: reduce memory footprint of percpu_ref in fast path | expand

Message

Ming Lei Sept. 2, 2020, 12:26 p.m. UTC 
Hi,

The 1st patch removes memory footprint of percpu_ref in fast path
from 7 words to 2 words, since it is often used in fast path and
embedded in user struct.

The 2nd patch moves .q_usage_counter to 1st cacheline of
'request_queue'.

Simple test on null_blk shows ~2% IOPS boost on one 16cores(two threads
per core) machine, dual socket/numa.

V2:
	- pass 'gfp' to kzalloc() for fixing block/027 failure reported by
	kernel test robot
	- protect percpu_ref_is_zero() with destroying percpu-refcount by
	spin lock  

Ming Lei (2):
  percpu_ref: reduce memory footprint of percpu_ref in fast path
  block: move 'q_usage_counter' into front of 'request_queue'

 drivers/infiniband/sw/rdmavt/mr.c |   2 +-
 include/linux/blkdev.h            |   3 +-
 include/linux/percpu-refcount.h   |  45 ++++-------
 lib/percpu-refcount.c             | 128 ++++++++++++++++++++++--------
 4 files changed, 115 insertions(+), 63 deletions(-)

Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: Tejun Heo <tj@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Bart Van Assche <bvanassche@acm.org>

Comments

Jens Axboe Sept. 2, 2020, 2:02 p.m. UTC | #1 
On 9/2/20 6:26 AM, Ming Lei wrote:
> Hi,
> 
> The 1st patch removes memory footprint of percpu_ref in fast path
> from 7 words to 2 words, since it is often used in fast path and
> embedded in user struct.
> 
> The 2nd patch moves .q_usage_counter to 1st cacheline of
> 'request_queue'.
> 
> Simple test on null_blk shows ~2% IOPS boost on one 16cores(two threads
> per core) machine, dual socket/numa.
> 
> V2:
> 	- pass 'gfp' to kzalloc() for fixing block/027 failure reported by
> 	kernel test robot
> 	- protect percpu_ref_is_zero() with destroying percpu-refcount by
> 	spin lock  
> 
> Ming Lei (2):
>   percpu_ref: reduce memory footprint of percpu_ref in fast path
>   block: move 'q_usage_counter' into front of 'request_queue'

Applied, thanks.