mbox series

[v6,00/12] Introduce Microsoft Azure Network Adapter (MANA) RDMA driver

Message ID 1663723352-598-1-git-send-email-longli@linuxonhyperv.com (mailing list archive)
Headers show
Series Introduce Microsoft Azure Network Adapter (MANA) RDMA driver | expand

Message

Long Li Sept. 21, 2022, 1:22 a.m. UTC
From: Long Li <longli@microsoft.com>

This patchset implements a RDMA driver for Microsoft Azure Network
Adapter (MANA). In MANA, the RDMA device is modeled as an auxiliary device
to the Ethernet device.

The first 11 patches modify the MANA Ethernet driver to support RDMA driver.
The last patch implementes the RDMA driver.

The user-mode of the driver is being reviewed at:
https://github.com/linux-rdma/rdma-core/pull/1177


Ajay Sharma (3):
  net: mana: Set the DMA device max segment size
  net: mana: Define and process GDMA response code
    GDMA_STATUS_MORE_ENTRIES
  net: mana: Define data structures for protection domain and memory
    registration

Long Li (9):
  net: mana: Add support for auxiliary device
  net: mana: Record the physical address for doorbell page region
  net: mana: Handle vport sharing between devices
  net: mana: Add functions for allocating doorbell page from GDMA
  net: mana: Export Work Queue functions for use by RDMA driver
  net: mana: Record port number in netdev
  net: mana: Move header files to a common location
  net: mana: Define max values for SGL entries
  RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter

 MAINTAINERS                                   |   4 +
 drivers/infiniband/Kconfig                    |   1 +
 drivers/infiniband/hw/Makefile                |   1 +
 drivers/infiniband/hw/mana/Kconfig            |   7 +
 drivers/infiniband/hw/mana/Makefile           |   4 +
 drivers/infiniband/hw/mana/cq.c               |  80 +++
 drivers/infiniband/hw/mana/device.c           | 129 ++++
 drivers/infiniband/hw/mana/main.c             | 555 ++++++++++++++++++
 drivers/infiniband/hw/mana/mana_ib.h          | 165 ++++++
 drivers/infiniband/hw/mana/mr.c               | 133 +++++
 drivers/infiniband/hw/mana/qp.c               | 501 ++++++++++++++++
 drivers/infiniband/hw/mana/wq.c               | 114 ++++
 drivers/net/ethernet/microsoft/Kconfig        |   1 +
 .../net/ethernet/microsoft/mana/gdma_main.c   |  96 ++-
 .../net/ethernet/microsoft/mana/hw_channel.c  |   6 +-
 .../net/ethernet/microsoft/mana/mana_bpf.c    |   2 +-
 drivers/net/ethernet/microsoft/mana/mana_en.c | 176 +++++-
 .../ethernet/microsoft/mana/mana_ethtool.c    |   2 +-
 .../net/ethernet/microsoft/mana/shm_channel.c |   2 +-
 .../microsoft => include/net}/mana/gdma.h     | 162 ++++-
 .../net}/mana/hw_channel.h                    |   0
 .../microsoft => include/net}/mana/mana.h     |  23 +-
 include/net/mana/mana_auxiliary.h             |  10 +
 .../net}/mana/shm_channel.h                   |   0
 include/uapi/rdma/ib_user_ioctl_verbs.h       |   1 +
 include/uapi/rdma/mana-abi.h                  |  66 +++
 26 files changed, 2196 insertions(+), 45 deletions(-)
 create mode 100644 drivers/infiniband/hw/mana/Kconfig
 create mode 100644 drivers/infiniband/hw/mana/Makefile
 create mode 100644 drivers/infiniband/hw/mana/cq.c
 create mode 100644 drivers/infiniband/hw/mana/device.c
 create mode 100644 drivers/infiniband/hw/mana/main.c
 create mode 100644 drivers/infiniband/hw/mana/mana_ib.h
 create mode 100644 drivers/infiniband/hw/mana/mr.c
 create mode 100644 drivers/infiniband/hw/mana/qp.c
 create mode 100644 drivers/infiniband/hw/mana/wq.c
 rename {drivers/net/ethernet/microsoft => include/net}/mana/gdma.h (80%)
 rename {drivers/net/ethernet/microsoft => include/net}/mana/hw_channel.h (100%)
 rename {drivers/net/ethernet/microsoft => include/net}/mana/mana.h (95%)
 create mode 100644 include/net/mana/mana_auxiliary.h
 rename {drivers/net/ethernet/microsoft => include/net}/mana/shm_channel.h (100%)
 create mode 100644 include/uapi/rdma/mana-abi.h

Comments

Jason Gunthorpe Sept. 21, 2022, 4:25 p.m. UTC | #1
On Tue, Sep 20, 2022 at 06:22:20PM -0700, longli@linuxonhyperv.com wrote:
> From: Long Li <longli@microsoft.com>
> 
> This patchset implements a RDMA driver for Microsoft Azure Network
> Adapter (MANA). In MANA, the RDMA device is modeled as an auxiliary device
> to the Ethernet device.
> 
> The first 11 patches modify the MANA Ethernet driver to support RDMA driver.
> The last patch implementes the RDMA driver.
> 
> The user-mode of the driver is being reviewed at:
> https://github.com/linux-rdma/rdma-core/pull/1177
> 
> 
> Ajay Sharma (3):
>   net: mana: Set the DMA device max segment size
>   net: mana: Define and process GDMA response code
>     GDMA_STATUS_MORE_ENTRIES
>   net: mana: Define data structures for protection domain and memory
>     registration
> 
> Long Li (9):
>   net: mana: Add support for auxiliary device
>   net: mana: Record the physical address for doorbell page region
>   net: mana: Handle vport sharing between devices
>   net: mana: Add functions for allocating doorbell page from GDMA
>   net: mana: Export Work Queue functions for use by RDMA driver
>   net: mana: Record port number in netdev
>   net: mana: Move header files to a common location
>   net: mana: Define max values for SGL entries
>   RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter

Still some basic checkpatchy stuff:

/tmp/tmpm2fsg47h/0012-RDMA-mana_ib-Add-a-driver-for-Microsoft-Azure-Network-Adapter.patch:412: WARNING: quoted string split across lines
#412: FILE: drivers/infiniband/hw/mana/main.c:70:
+		  "vport handle %llx pdid %x doorbell_id %x "
+		  "tx_shortform_allowed %d tx_vp_offset %u\n",

/tmp/tmpm2fsg47h/0012-RDMA-mana_ib-Add-a-driver-for-Microsoft-Azure-Network-Adapter.patch:540: WARNING: quoted string split across lines
#540: FILE: drivers/infiniband/hw/mana/main.c:198:
+		  "size_dma_region %lu num_pages_total %lu, "
+		  "page_sz 0x%llx offset_in_page %u\n",

And it thinks you should write more for the kconfig symbol, eg why
would someone want to turn it on (hint, to use dpkd on some Azure
instances)

/tmp/tmpm2fsg47h/0012-RDMA-mana_ib-Add-a-driver-for-Microsoft-Azure-Network-Adapter.patch:100: WARNING: please write a help paragraph that fully describes the config symbol
#100: FILE: drivers/infiniband/hw/mana/Kconfig:2:
+config MANA_INFINIBAND
+	tristate "Microsoft Azure Network Adapter support"
+	depends on NETDEVICES && ETHERNET && PCI && MICROSOFT_MANA
+	help
+	  This driver provides low-level RDMA support for
+	  Microsoft Azure Network Adapter (MANA).

I'll put it in linux-next and you can fix the rest of the stuff bots
will usually find.

Jason
Long Li Sept. 21, 2022, 6:53 p.m. UTC | #2
> Subject: Re: [Patch v6 00/12] Introduce Microsoft Azure Network Adapter
> (MANA) RDMA driver
> 
> On Tue, Sep 20, 2022 at 06:22:20PM -0700, longli@linuxonhyperv.com wrote:
> > From: Long Li <longli@microsoft.com>
> >
> > This patchset implements a RDMA driver for Microsoft Azure Network
> > Adapter (MANA). In MANA, the RDMA device is modeled as an auxiliary
> > device to the Ethernet device.
> >
> > The first 11 patches modify the MANA Ethernet driver to support RDMA
> driver.
> > The last patch implementes the RDMA driver.
> >
> > The user-mode of the driver is being reviewed at:
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> > ub.com%2Flinux-rdma%2Frdma-
> core%2Fpull%2F1177&amp;data=05%7C01%7Clongl
> >
> i%40microsoft.com%7C1d30ce9d49bc411b0fb808da9bede4bd%7C72f988bf86
> f141a
> >
> f91ab2d7cd011db47%7C1%7C0%7C637993743320198387%7CUnknown%7CT
> WFpbGZsb3d
> >
> 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> 3D%7C
> >
> 3000%7C%7C%7C&amp;sdata=4cT9PFnKqb2yHRLkKNrbVVMAGwO6Ig91eoQ
> zomlm6lY%3D
> > &amp;reserved=0
> >
> >
> > Ajay Sharma (3):
> >   net: mana: Set the DMA device max segment size
> >   net: mana: Define and process GDMA response code
> >     GDMA_STATUS_MORE_ENTRIES
> >   net: mana: Define data structures for protection domain and memory
> >     registration
> >
> > Long Li (9):
> >   net: mana: Add support for auxiliary device
> >   net: mana: Record the physical address for doorbell page region
> >   net: mana: Handle vport sharing between devices
> >   net: mana: Add functions for allocating doorbell page from GDMA
> >   net: mana: Export Work Queue functions for use by RDMA driver
> >   net: mana: Record port number in netdev
> >   net: mana: Move header files to a common location
> >   net: mana: Define max values for SGL entries
> >   RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter
> 
> Still some basic checkpatchy stuff:
> 
> /tmp/tmpm2fsg47h/0012-RDMA-mana_ib-Add-a-driver-for-Microsoft-
> Azure-Network-Adapter.patch:412: WARNING: quoted string split across
> lines
> #412: FILE: drivers/infiniband/hw/mana/main.c:70:
> +		  "vport handle %llx pdid %x doorbell_id %x "
> +		  "tx_shortform_allowed %d tx_vp_offset %u\n",
> 
> /tmp/tmpm2fsg47h/0012-RDMA-mana_ib-Add-a-driver-for-Microsoft-
> Azure-Network-Adapter.patch:540: WARNING: quoted string split across
> lines
> #540: FILE: drivers/infiniband/hw/mana/main.c:198:
> +		  "size_dma_region %lu num_pages_total %lu, "
> +		  "page_sz 0x%llx offset_in_page %u\n",
> 
> And it thinks you should write more for the kconfig symbol, eg why would
> someone want to turn it on (hint, to use dpkd on some Azure
> instances)

Will fix those.

> 
> /tmp/tmpm2fsg47h/0012-RDMA-mana_ib-Add-a-driver-for-Microsoft-
> Azure-Network-Adapter.patch:100: WARNING: please write a help
> paragraph that fully describes the config symbol
> #100: FILE: drivers/infiniband/hw/mana/Kconfig:2:
> +config MANA_INFINIBAND
> +	tristate "Microsoft Azure Network Adapter support"
> +	depends on NETDEVICES && ETHERNET && PCI &&
> MICROSOFT_MANA
> +	help
> +	  This driver provides low-level RDMA support for
> +	  Microsoft Azure Network Adapter (MANA).
>