mbox series

[0/5] target: fixes and perf improvements

Message ID 20210930020422.92578-1-michael.christie@oracle.com (mailing list archive)
Headers show
Series target: fixes and perf improvements | expand

Message

Mike Christie Sept. 30, 2021, 2:04 a.m. UTC 
The following patches apply to Martin's staging tree or Linus's tree.
The patches main goal is to take the locks out of the main IO path but
for the case of ordered cmds they also fix a handfull of bugs.

For the locks we currently have:

1. lun_tg_pt_gp_lock
2. delayed_cmd_lock
3. dev_reservation_lock

and this set takes out 1 and 2. With them removed a simple fio:

fio --filename=/dev/sdb  --direct=1 --rw=randrw --bs=4k \
--ioengine=libaio --iodepth=64  --numjobs=$NUM_QUEUES

can increase IOPs by up to 30% (from a max of 1.4M to 2M) when using
multiple queues and vhost-scsi with the multiple vhost thread patches or
tcm loop with nr_hw_queues set.

Note: I normally hit a ceiling of 1.4M IOPs with around 8 queues but with
the patches I hit a ceiling at around 16 queues and 2M IOPs.

If I cheat and set emulate_pr=0 so the reservation lock is removed then
it scales nicely and you can continue to add a job and queue per CPU (at
least up to 20 CPUs which is when I run out of CPUs).

Comments

Martin K. Petersen Oct. 17, 2021, 3:08 a.m. UTC | #1 
Mike,

> The following patches apply to Martin's staging tree or Linus's tree.
> The patches main goal is to take the locks out of the main IO path but
> for the case of ordered cmds they also fix a handfull of bugs.

Applied to 5.16/scsi-staging, thanks!
Martin K. Petersen Oct. 21, 2021, 3:42 a.m. UTC | #2 
On Wed, 29 Sep 2021 21:04:17 -0500, Mike Christie wrote:

> The following patches apply to Martin's staging tree or Linus's tree.
> The patches main goal is to take the locks out of the main IO path but
> for the case of ordered cmds they also fix a handfull of bugs.
> 
> For the locks we currently have:
> 
> 1. lun_tg_pt_gp_lock
> 2. delayed_cmd_lock
> 3. dev_reservation_lock
> 
> [...]

Applied to 5.16/scsi-queue, thanks!

[1/5] target: fix ordered CMD_T_SENT handling
      https://git.kernel.org/mkp/scsi/c/945a160794a9
[2/5] target: fix ordered tag handling
      https://git.kernel.org/mkp/scsi/c/ed1227e08099
[3/5] target: fix alua_tg_pt_gps_count tracking
      https://git.kernel.org/mkp/scsi/c/1283c0d1a32b
[4/5] target: replace lun_tg_pt_gp_lock with rcu in IO path
      https://git.kernel.org/mkp/scsi/c/7324f47d4293
[5/5] target: perform alua group changes in one step
      https://git.kernel.org/mkp/scsi/c/f9793d649c29