[0/8] mutipath: handle nvme:tcp paths better

Message

Benjamin Marzinski Oct. 7, 2022, 5:35 p.m. UTC

nvme:tcp devices set BLK_MQ_F_BLOCKING (they are the only block devices
which multipath supports that do so), meaning that block_mq expects that
they can block at certain points while servicing a request.  However,
due to the way device-mapper sets up its queue, it is not able to set
BLK_MQ_F_BLOCKING when it includes paths that set this flag.  Patches
were written to address this issue but they were rejected upstream

https://lore.kernel.org/linux-block/[e-mail]/T/#t

The proposed solution was to have multipath use the bio queue_mode for
multipath devices that include nvme:tcp paths.

Multipathd currently doesn't handle queue_mode as well as it could. Once
a multipath device is created, queue_mode cannot be changed, but
multipath doesn't enforce this. This patchset improves multipath's
handling of the queue_mode feature, and then makes sure that if a
multipath device includes a nvme:tcp path, it will have queue_mode
set to bio.

Benjamin Marzinski (8):
  libmultipath: cleanup remove_feature
  libmultipath: cleanup add_feature
  multipath tests: tests for adding and removing features
  libmultipath: fix queue_mode feature handling
  multipath tests: tests for reconcile_features_with_queue_mode
  libmultipath: prepare proto_id for use by non-scsi devivces
  libmultipath: get nvme path transport protocol
  libmultipath: enforce queue_mode bio for nmve:tcp paths

 libmultipath/configure.c   |  28 +-
 libmultipath/discovery.c   |  31 ++-
 libmultipath/dmparser.c    |   2 +
 libmultipath/print.c       |   6 +-
 libmultipath/propsel.c     |  55 ++++
 libmultipath/structs.c     | 155 +++++------
 libmultipath/structs.h     |  44 ++-
 libmultipath/structs_vec.c |   7 +
 multipath/multipath.conf.5 |  17 +-
 multipathd/fpin_handlers.c |   2 +-
 tests/Makefile             |   4 +-
 tests/features.c           | 549 +++++++++++++++++++++++++++++++++++++
 12 files changed, 784 insertions(+), 116 deletions(-)
 create mode 100644 tests/features.c

Comments

Martin Wilck Oct. 20, 2022, 2:58 p.m. UTC | #1

On Fri, 2022-10-07 at 12:35 -0500, Benjamin Marzinski wrote:
> nvme:tcp devices set BLK_MQ_F_BLOCKING (they are the only block
> devices
> which multipath supports that do so), meaning that block_mq expects
> that
> they can block at certain points while servicing a request.  However,
> due to the way device-mapper sets up its queue, it is not able to set
> BLK_MQ_F_BLOCKING when it includes paths that set this flag.  Patches
> were written to address this issue but they were rejected upstream
> 
> https://lore.kernel.org/linux-block/[e-mail]/T/#t
> 
> The proposed solution was to have multipath use the bio queue_mode
> for
> multipath devices that include nvme:tcp paths.

Request-based dm-multipath is superior to bio-based in multiple ways. I
thought that was consensus. It's sad that we need to make this clumsy
workaround in user space while a much neater kernel-side approach would
be available.

> Multipathd currently doesn't handle queue_mode as well as it could.
> Once
> a multipath device is created, queue_mode cannot be changed, but
> multipath doesn't enforce this. This patchset improves multipath's
> handling of the queue_mode feature, and then makes sure that if a
> multipath device includes a nvme:tcp path, it will have queue_mode
> set to bio.
> 
> Benjamin Marzinski (8):
>   libmultipath: cleanup remove_feature
>   libmultipath: cleanup add_feature
>   multipath tests: tests for adding and removing features
>   libmultipath: fix queue_mode feature handling
>   multipath tests: tests for reconcile_features_with_queue_mode
>   libmultipath: prepare proto_id for use by non-scsi devivces
>   libmultipath: get nvme path transport protocol
>   libmultipath: enforce queue_mode bio for nmve:tcp paths
> 
>  libmultipath/configure.c   |  28 +-
>  libmultipath/discovery.c   |  31 ++-
>  libmultipath/dmparser.c    |   2 +
>  libmultipath/print.c       |   6 +-
>  libmultipath/propsel.c     |  55 ++++
>  libmultipath/structs.c     | 155 +++++------
>  libmultipath/structs.h     |  44 ++-
>  libmultipath/structs_vec.c |   7 +
>  multipath/multipath.conf.5 |  17 +-
>  multipathd/fpin_handlers.c |   2 +-
>  tests/Makefile             |   4 +-
>  tests/features.c           | 549
> +++++++++++++++++++++++++++++++++++++
>  12 files changed, 784 insertions(+), 116 deletions(-)
>  create mode 100644 tests/features.c
> 

For the series: 

Reviewed-by: Martin Wilck <[e-mail]>

I have a few remarks and suggestions for additional improvement which
I'll send as replies to the individual patches.

--
dm-devel mailing list
[e-mail]
https://listman.redhat.com/mailman/listinfo/dm-devel