[v2,0/8] vfio: Split the container code into a clean layer and dedicated file

Message ID	0-v2-d7744ee9cf4f+33d-vfio_container_split_jgg@nvidia.com (mailing list archive)
Headers	show Return-Path: <kvm-owner@kernel.org> From: Jason Gunthorpe <jgg@nvidia.com> To: Alex Williamson <alex.williamson@redhat.com>, Cornelia Huck <cohuck@redhat.com>, kvm@vger.kernel.org Cc: Kevin Tian <kevin.tian@intel.com> Subject: [PATCH v2 0/8] vfio: Split the container code into a clean layer and dedicated file Date: Tue, 20 Sep 2022 21:42:28 -0300 Message-Id: <0-v2-d7744ee9cf4f+33d-vfio_container_split_jgg@nvidia.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain MIME-Version: 1.0 Precedence: bulk
Series	vfio: Split the container code into a clean layer and dedicated file \| expand [v2,0/8] vfio: Split the container code into a clean layer and dedicated file [v2,1/8] vfio: Add header guards and includes to drivers/vfio/vfio.h [v2,2/8] vfio: Rename __vfio_group_unset_container() [v2,3/8] vfio: Split the container logic into vfio_container_attach_group() [v2,4/8] vfio: Remove #ifdefs around CONFIG_VFIO_NOIOMMU [v2,5/8] vfio: Split out container code from the init/cleanup functions [v2,6/8] vfio: Rename vfio_ioctl_check_extension() [v2,7/8] vfio: Split the register_device ops call into functions [v2,8/8] vfio: Move container code into drivers/vfio/container.c

Jason Gunthorpe Sept. 21, 2022, 12:42 a.m. UTC

This creates an isolated layer around the container FD code and everything
under it, including the VFIO iommu drivers. All this code is placed into
container.c, along with the "struct vfio_container" to compartmentalize
it.

Future patches will provide an iommufd based layer that gives the same API
as the container layer and choose which layer to go to based on how
userspace operates.

The patches continue to split up existing functions and finally the last
patch just moves every function that is a "container" function to the new
file and creates the global symbols to link them together.

Cross-file container functions are prefixed with vfio_container_* for
clarity.

The last patch can be defered and queued during the merge window to manage
conflicts. The earlier patches should be fine immediately conflicts wise.

This is the last big series I have to enable basic iommufd functionality.
As part of the iommufd series the entire container.c becomes conditionally
compiled:

https://github.com/jgunthorpe/linux/commits/vfio_iommufd

v2:
 - Rename s/vfio_container_detatch_group/vfio_group_detach_container/
          s/vfio_container_register_device/vfio_device_container_register/
          s/vfio_container_unregister_device/vfio_device_container_unregister/
 - Change argument order of vfio_container_attach_group()
 - Rebased onto merged patches
v1: https://lore.kernel.org/r/0-v1-a805b607f1fb+17b-vfio_container_split_jgg@nvidia.com

Jason Gunthorpe (8):
  vfio: Add header guards and includes to drivers/vfio/vfio.h
  vfio: Rename __vfio_group_unset_container()
  vfio: Split the container logic into vfio_container_attach_group()
  vfio: Remove #ifdefs around CONFIG_VFIO_NOIOMMU
  vfio: Split out container code from the init/cleanup functions
  vfio: Rename vfio_ioctl_check_extension()
  vfio: Split the register_device ops call into functions
  vfio: Move container code into drivers/vfio/container.c

 drivers/vfio/Makefile    |   1 +
 drivers/vfio/container.c | 680 +++++++++++++++++++++++++++++++++++++
 drivers/vfio/vfio.h      |  56 ++++
 drivers/vfio/vfio_main.c | 708 ++-------------------------------------
 4 files changed, 765 insertions(+), 680 deletions(-)
 create mode 100644 drivers/vfio/container.c


base-commit: 245898eb9275ce31942cff95d0bdc7412ad3d589

Tian, Kevin Sept. 21, 2022, 8:07 a.m. UTC | #1

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, September 21, 2022 8:42 AM
>  drivers/vfio/Makefile    |   1 +
>  drivers/vfio/container.c | 680 +++++++++++++++++++++++++++++++++++++
>  drivers/vfio/vfio.h      |  56 ++++
>  drivers/vfio/vfio_main.c | 708 ++-------------------------------------
>  4 files changed, 765 insertions(+), 680 deletions(-)
>  create mode 100644 drivers/vfio/container.c
> 
> 
> base-commit: 245898eb9275ce31942cff95d0bdc7412ad3d589

it's not the latest vfio/next:

commit f39856aacb078c1c93acef011a37121b17d54fe0 (vfio/next)
Author: Yishai Hadas <yishaih@nvidia.com>
Date:   Thu Sep 8 21:34:48 2022 +0300

    vfio/mlx5: Set the driver DMA logging callbacks
    
    Now that everything is ready set the driver DMA logging callbacks if
    supported by the device.
    
    Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
    Link: https://lore.kernel.org/r/20220908183448.195262-11-yishaih@nvidia.com
    Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

Jason Gunthorpe Sept. 22, 2022, 1 a.m. UTC | #2

On Wed, Sep 21, 2022 at 08:07:42AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Wednesday, September 21, 2022 8:42 AM
> >  drivers/vfio/Makefile    |   1 +
> >  drivers/vfio/container.c | 680 +++++++++++++++++++++++++++++++++++++
> >  drivers/vfio/vfio.h      |  56 ++++
> >  drivers/vfio/vfio_main.c | 708 ++-------------------------------------
> >  4 files changed, 765 insertions(+), 680 deletions(-)
> >  create mode 100644 drivers/vfio/container.c
> > 
> > 
> > base-commit: 245898eb9275ce31942cff95d0bdc7412ad3d589
> 
> it's not the latest vfio/next:

Ah, I did the rebase before I left for lpc..

There is a minor merge conflict with the stuff from the last week:

diff --cc drivers/vfio/Makefile
index d67c604d0407ef,d5ae6921eb4ece..00000000000000
--- a/drivers/vfio/Makefile
+++ b/drivers/vfio/Makefile
@@@ -1,11 -1,10 +1,12 @@@
  # SPDX-License-Identifier: GPL-2.0
  vfio_virqfd-y := virqfd.o
  
 -vfio-y += container.o
 -vfio-y += vfio_main.o
 -
  obj-$(CONFIG_VFIO) += vfio.o
 +
 +vfio-y += vfio_main.o \
 +        iova_bitmap.o \
++        container.o
 +
  obj-$(CONFIG_VFIO_VIRQFD) += vfio_virqfd.o
  obj-$(CONFIG_VFIO_IOMMU_TYPE1) += vfio_iommu_type1.o
  obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o

Alex, let me know if you want me to respin it

Thanks,
Jason

Alex Williamson Sept. 22, 2022, 5:09 p.m. UTC | #3

On Wed, 21 Sep 2022 22:00:30 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:

> On Wed, Sep 21, 2022 at 08:07:42AM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > Sent: Wednesday, September 21, 2022 8:42 AM
> > >  drivers/vfio/Makefile    |   1 +
> > >  drivers/vfio/container.c | 680 +++++++++++++++++++++++++++++++++++++
> > >  drivers/vfio/vfio.h      |  56 ++++
> > >  drivers/vfio/vfio_main.c | 708 ++-------------------------------------
> > >  4 files changed, 765 insertions(+), 680 deletions(-)
> > >  create mode 100644 drivers/vfio/container.c
> > > 
> > > 
> > > base-commit: 245898eb9275ce31942cff95d0bdc7412ad3d589  
> > 
> > it's not the latest vfio/next:  
> 
> Ah, I did the rebase before I left for lpc..
> 
> There is a minor merge conflict with the stuff from the last week:
> 
> diff --cc drivers/vfio/Makefile
> index d67c604d0407ef,d5ae6921eb4ece..00000000000000
> --- a/drivers/vfio/Makefile
> +++ b/drivers/vfio/Makefile
> @@@ -1,11 -1,10 +1,12 @@@
>   # SPDX-License-Identifier: GPL-2.0
>   vfio_virqfd-y := virqfd.o
>   
>  -vfio-y += container.o
>  -vfio-y += vfio_main.o
>  -
>   obj-$(CONFIG_VFIO) += vfio.o
>  +
>  +vfio-y += vfio_main.o \
>  +        iova_bitmap.o \
> ++        container.o
>  +
>   obj-$(CONFIG_VFIO_VIRQFD) += vfio_virqfd.o
>   obj-$(CONFIG_VFIO_IOMMU_TYPE1) += vfio_iommu_type1.o
>   obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o
> 
> Alex, let me know if you want me to respin it

That's trivial, but you also have conflicts with Kevin's 'Tidy up
vfio_device life cycle' series, which gets uglier than I'd like to
fixup on commit.  Could one of you volunteer to rebase on the other?
Thanks,

Alex

Jason Gunthorpe Sept. 22, 2022, 5:22 p.m. UTC | #4

On Thu, Sep 22, 2022 at 11:09:30AM -0600, Alex Williamson wrote:
> On Wed, 21 Sep 2022 22:00:30 -0300
> Jason Gunthorpe <jgg@nvidia.com> wrote:
> 
> > On Wed, Sep 21, 2022 at 08:07:42AM +0000, Tian, Kevin wrote:
> > > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > > Sent: Wednesday, September 21, 2022 8:42 AM
> > > >  drivers/vfio/Makefile    |   1 +
> > > >  drivers/vfio/container.c | 680 +++++++++++++++++++++++++++++++++++++
> > > >  drivers/vfio/vfio.h      |  56 ++++
> > > >  drivers/vfio/vfio_main.c | 708 ++-------------------------------------
> > > >  4 files changed, 765 insertions(+), 680 deletions(-)
> > > >  create mode 100644 drivers/vfio/container.c
> > > > 
> > > > 
> > > > base-commit: 245898eb9275ce31942cff95d0bdc7412ad3d589  
> > > 
> > > it's not the latest vfio/next:  
> > 
> > Ah, I did the rebase before I left for lpc..
> > 
> > There is a minor merge conflict with the stuff from the last week:
> > 
> > diff --cc drivers/vfio/Makefile
> > index d67c604d0407ef,d5ae6921eb4ece..00000000000000
> > --- a/drivers/vfio/Makefile
> > +++ b/drivers/vfio/Makefile
> > @@@ -1,11 -1,10 +1,12 @@@
> >   # SPDX-License-Identifier: GPL-2.0
> >   vfio_virqfd-y := virqfd.o
> >   
> >  -vfio-y += container.o
> >  -vfio-y += vfio_main.o
> >  -
> >   obj-$(CONFIG_VFIO) += vfio.o
> >  +
> >  +vfio-y += vfio_main.o \
> >  +        iova_bitmap.o \
> > ++        container.o
> >  +
> >   obj-$(CONFIG_VFIO_VIRQFD) += vfio_virqfd.o
> >   obj-$(CONFIG_VFIO_IOMMU_TYPE1) += vfio_iommu_type1.o
> >   obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o
> > 
> > Alex, let me know if you want me to respin it
> 
> That's trivial, but you also have conflicts with Kevin's 'Tidy up
> vfio_device life cycle' series, which gets uglier than I'd like to
> fixup on commit.  Could one of you volunteer to rebase on the other?

Sure, I'll rebase this one, can you pick up Kevin's so I have a stable target?

Thanks,
Jason

Alex Williamson Sept. 22, 2022, 5:58 p.m. UTC | #5

On Thu, 22 Sep 2022 14:22:03 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:

> On Thu, Sep 22, 2022 at 11:09:30AM -0600, Alex Williamson wrote:
> > On Wed, 21 Sep 2022 22:00:30 -0300
> > Jason Gunthorpe <jgg@nvidia.com> wrote:
> >   
> > > On Wed, Sep 21, 2022 at 08:07:42AM +0000, Tian, Kevin wrote:  
> > > > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > > > Sent: Wednesday, September 21, 2022 8:42 AM
> > > > >  drivers/vfio/Makefile    |   1 +
> > > > >  drivers/vfio/container.c | 680 +++++++++++++++++++++++++++++++++++++
> > > > >  drivers/vfio/vfio.h      |  56 ++++
> > > > >  drivers/vfio/vfio_main.c | 708 ++-------------------------------------
> > > > >  4 files changed, 765 insertions(+), 680 deletions(-)
> > > > >  create mode 100644 drivers/vfio/container.c
> > > > > 
> > > > > 
> > > > > base-commit: 245898eb9275ce31942cff95d0bdc7412ad3d589    
> > > > 
> > > > it's not the latest vfio/next:    
> > > 
> > > Ah, I did the rebase before I left for lpc..
> > > 
> > > There is a minor merge conflict with the stuff from the last week:
> > > 
> > > diff --cc drivers/vfio/Makefile
> > > index d67c604d0407ef,d5ae6921eb4ece..00000000000000
> > > --- a/drivers/vfio/Makefile
> > > +++ b/drivers/vfio/Makefile
> > > @@@ -1,11 -1,10 +1,12 @@@
> > >   # SPDX-License-Identifier: GPL-2.0
> > >   vfio_virqfd-y := virqfd.o
> > >   
> > >  -vfio-y += container.o
> > >  -vfio-y += vfio_main.o
> > >  -
> > >   obj-$(CONFIG_VFIO) += vfio.o
> > >  +
> > >  +vfio-y += vfio_main.o \
> > >  +        iova_bitmap.o \
> > > ++        container.o
> > >  +
> > >   obj-$(CONFIG_VFIO_VIRQFD) += vfio_virqfd.o
> > >   obj-$(CONFIG_VFIO_IOMMU_TYPE1) += vfio_iommu_type1.o
> > >   obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o
> > > 
> > > Alex, let me know if you want me to respin it  
> > 
> > That's trivial, but you also have conflicts with Kevin's 'Tidy up
> > vfio_device life cycle' series, which gets uglier than I'd like to
> > fixup on commit.  Could one of you volunteer to rebase on the other?  
> 
> Sure, I'll rebase this one, can you pick up Kevin's so I have a stable target?


Yup, you should see it in my next branch now.  Thanks,

Alex

Tian, Kevin Sept. 22, 2022, 10:49 p.m. UTC | #6

> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Friday, September 23, 2022 1:59 AM
> 
> On Thu, 22 Sep 2022 14:22:03 -0300
> Jason Gunthorpe <jgg@nvidia.com> wrote:
> 
> > On Thu, Sep 22, 2022 at 11:09:30AM -0600, Alex Williamson wrote:
> > > On Wed, 21 Sep 2022 22:00:30 -0300
> > > Jason Gunthorpe <jgg@nvidia.com> wrote:
> > >
> > > > On Wed, Sep 21, 2022 at 08:07:42AM +0000, Tian, Kevin wrote:
> > > > > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > > > > Sent: Wednesday, September 21, 2022 8:42 AM
> > > > > >  drivers/vfio/Makefile    |   1 +
> > > > > >  drivers/vfio/container.c | 680
> +++++++++++++++++++++++++++++++++++++
> > > > > >  drivers/vfio/vfio.h      |  56 ++++
> > > > > >  drivers/vfio/vfio_main.c | 708 ++-------------------------------------
> > > > > >  4 files changed, 765 insertions(+), 680 deletions(-)
> > > > > >  create mode 100644 drivers/vfio/container.c
> > > > > >
> > > > > >
> > > > > > base-commit: 245898eb9275ce31942cff95d0bdc7412ad3d589
> > > > >
> > > > > it's not the latest vfio/next:
> > > >
> > > > Ah, I did the rebase before I left for lpc..
> > > >
> > > > There is a minor merge conflict with the stuff from the last week:
> > > >
> > > > diff --cc drivers/vfio/Makefile
> > > > index d67c604d0407ef,d5ae6921eb4ece..00000000000000
> > > > --- a/drivers/vfio/Makefile
> > > > +++ b/drivers/vfio/Makefile
> > > > @@@ -1,11 -1,10 +1,12 @@@
> > > >   # SPDX-License-Identifier: GPL-2.0
> > > >   vfio_virqfd-y := virqfd.o
> > > >
> > > >  -vfio-y += container.o
> > > >  -vfio-y += vfio_main.o
> > > >  -
> > > >   obj-$(CONFIG_VFIO) += vfio.o
> > > >  +
> > > >  +vfio-y += vfio_main.o \
> > > >  +        iova_bitmap.o \
> > > > ++        container.o
> > > >  +
> > > >   obj-$(CONFIG_VFIO_VIRQFD) += vfio_virqfd.o
> > > >   obj-$(CONFIG_VFIO_IOMMU_TYPE1) += vfio_iommu_type1.o
> > > >   obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o
> > > >
> > > > Alex, let me know if you want me to respin it
> > >
> > > That's trivial, but you also have conflicts with Kevin's 'Tidy up
> > > vfio_device life cycle' series, which gets uglier than I'd like to
> > > fixup on commit.  Could one of you volunteer to rebase on the other?
> >
> > Sure, I'll rebase this one, can you pick up Kevin's so I have a stable target?
> 
> 
> Yup, you should see it in my next branch now.  Thanks,
> 

Thank you both!

[v2,0/8] vfio: Split the container code into a clean layer and dedicated file

Message

Comments