mbox series

[v2,0/3] UIO_MEM_DMA_COHERENT for cnic/bnx2/bnx2x

Message ID 20240103091137.27142-1-njavali@marvell.com (mailing list archive)
Headers show
Series UIO_MEM_DMA_COHERENT for cnic/bnx2/bnx2x | expand

Message

Nilesh Javali Jan. 3, 2024, 9:11 a.m. UTC 
During bnx2i iSCSI testing we ran into page refcounting issues in the
uio mmaps exported from cnic to the iscsiuio process, and bisected back
to the removal of the __GFP_COMP flag from dma_alloc_coherent calls.

In order to fix these drivers to be able to mmap dma coherent memory via
a uio device, without resorting to hacks and working with an iommu
enabled, introduce a new uio mmap type backed by dma_mmap_coherent.

While converting the uio interface, I also noticed that not all of these
allocations were PAGE_SIZE aligned. Particularly the bnx2/bnx2x status
block mapping was much smaller than any architecture page size, and I
was concerned that it could be unintentionally exposing kernel memory.

v2:
- expose only the dma_addr within uio and cnic.
- Cleanup newly added unions comprising virtual_addr
  and struct device

Chris Leech (3):
  uio: introduce UIO_MEM_DMA_COHERENT type
  cnic,bnx2,bnx2x: use UIO_MEM_DMA_COHERENT
  cnic,bnx2,bnx2x: page align uio mmap allocations

 drivers/net/ethernet/broadcom/bnx2.c          |  2 +
 .../net/ethernet/broadcom/bnx2x/bnx2x_main.c  | 10 +++--
 drivers/net/ethernet/broadcom/cnic.c          | 26 ++++++++-----
 drivers/net/ethernet/broadcom/cnic.h          |  1 +
 drivers/net/ethernet/broadcom/cnic_if.h       |  1 +
 drivers/uio/uio.c                             | 38 +++++++++++++++++++
 include/linux/uio_driver.h                    |  2 +
 7 files changed, 67 insertions(+), 13 deletions(-)

Comments

John Meneghini Jan. 3, 2024, 3:31 p.m. UTC | #1 
Nilesh,

Please explain what has been done to test these patches on bnx2i. Was testing done with iommu enabled/disabled?  Was testing 
done on any/all platforms, or was only x86_64 tested?

Thanks,

/John

On 1/3/24 04:11, Nilesh Javali wrote:
> During bnx2i iSCSI testing we ran into page refcounting issues in the
> uio mmaps exported from cnic to the iscsiuio process, and bisected back
> to the removal of the __GFP_COMP flag from dma_alloc_coherent calls.
> 
> In order to fix these drivers to be able to mmap dma coherent memory via
> a uio device, without resorting to hacks and working with an iommu
> enabled, introduce a new uio mmap type backed by dma_mmap_coherent.
> 
> While converting the uio interface, I also noticed that not all of these
> allocations were PAGE_SIZE aligned. Particularly the bnx2/bnx2x status
> block mapping was much smaller than any architecture page size, and I
> was concerned that it could be unintentionally exposing kernel memory.
> 
> v2:
> - expose only the dma_addr within uio and cnic.
> - Cleanup newly added unions comprising virtual_addr
>    and struct device
> 
> Chris Leech (3):
>    uio: introduce UIO_MEM_DMA_COHERENT type
>    cnic,bnx2,bnx2x: use UIO_MEM_DMA_COHERENT
>    cnic,bnx2,bnx2x: page align uio mmap allocations
> 
>   drivers/net/ethernet/broadcom/bnx2.c          |  2 +
>   .../net/ethernet/broadcom/bnx2x/bnx2x_main.c  | 10 +++--
>   drivers/net/ethernet/broadcom/cnic.c          | 26 ++++++++-----
>   drivers/net/ethernet/broadcom/cnic.h          |  1 +
>   drivers/net/ethernet/broadcom/cnic_if.h       |  1 +
>   drivers/uio/uio.c                             | 38 +++++++++++++++++++
>   include/linux/uio_driver.h                    |  2 +
>   7 files changed, 67 insertions(+), 13 deletions(-)
>
Nilesh Javali Jan. 3, 2024, 4 p.m. UTC | #2 
John,

> -----Original Message-----
> From: John Meneghini <jmeneghi@redhat.com>
> Sent: Wednesday, January 3, 2024 9:01 PM
> To: Nilesh Javali <njavali@marvell.com>; martin.petersen@oracle.com;
> lduncan@suse.com; cleech@redhat.com
> Cc: linux-scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic-
> Storage-Upstream@marvell.com>; gregkh@linuxfoundation.org; Hannes
> Reinecke <hare@suse.de>; Jerry Snitselaar <jsnitsel@redhat.com>; Ming Lei
> <ming.lei@redhat.com>; Ewan Milne <emilne@redhat.com>
> Subject: [EXT] Re: [PATCH v2 0/3] UIO_MEM_DMA_COHERENT for
> cnic/bnx2/bnx2x
> 
> External Email
> 
> ----------------------------------------------------------------------
> Nilesh,
> 
> Please explain what has been done to test these patches on bnx2i. Was
> testing done with iommu enabled/disabled?  Was testing
> done on any/all platforms, or was only x86_64 tested?
> 

I have verified bnx2i/cnic/bnx2x driver with iommu disabled by default on my x86_64 setup with 6.7.0-rc1+ kernel.
At a high level the tests comprised of iSCSI target logins/logouts, IOs, multiple iterations of iscsid.service and iscsiuio.service restarts,
port perturbation, sg resets, ping test.

Thanks,
Nilesh