mbox series

[00/15] Tidy up vfio_device life cycle

Message ID 20220827171037.30297-1-kevin.tian@intel.com (mailing list archive)
Headers show
Series Tidy up vfio_device life cycle | expand

Message

Tian, Kevin Aug. 27, 2022, 5:10 p.m. UTC
The idea is to let vfio core manage the vfio_device life cycle instead
of duplicating the logic cross drivers. Besides cleaner code in driver
side this also allows adding struct device to vfio_device as the first
step toward adding cdev uAPI in the future. Another benefit is that
user can now look at sysfs to decide whether a device is bound to
vfio [1], e.g.:

	/sys/devices/pci0000\:6f/0000\:6f\:01.0/vfio-dev/vfio0

Though most drivers can fit the new model naturally:

 - vfio_alloc_device() to allocate and initialize vfio_device
 - vfio_put_device() to release vfio_device
 - dev_ops->init() for driver private initialization
 - dev_ops->release() for driver private cleanup

vfio-ccw is the only exception due to a life cycle mess that its private
structure mixes both parent and mdev info hence must be alloc/free'ed
outside of the life cycle of vfio device.

Per prior discussions this won't be fixed in short term by IBM folks [2].

Instead of waiting this series introduces a few tricks to move forward:

 - vfio_init_device() to initialize a pre-allocated device structure;

 - require *EVERY* driver to implement @release and free vfio_device
   inside. Then vfio-ccw can use a completion mechanism to delay the
   free to css driver;

The second trick is not a real burden to other drivers because they
all require a @release for private cleanup anyay. Later once the ccw
mess is fixed a simple cleanup can be done by moving free from @release
to vfio core.

Thanks
Kevin

[1] https://listman.redhat.com/archives/libvir-list/2022-August/233482.html
[2] https://lore.kernel.org/all/0ee29bd6583f17f0ee4ec0769fa50e8ea6703623.camel@linux.ibm.com/

Kevin Tian (6):
  vfio: Add helpers for unifying vfio_device life cycle
  drm/i915/gvt: Use the new device life cycle helpers
  vfio/platform: Use the new device life cycle helpers
  vfio/amba: Use the new device life cycle helpers
  vfio/ccw: Use the new device life cycle helpers
  vfio: Rename vfio_device_put() and vfio_device_try_get()

Yi Liu (9):
  vfio/pci: Use the new device life cycle helpers
  vfio/mlx5: Use the new device life cycle helpers
  vfio/hisi_acc: Use the new device life cycle helpers
  vfio/mdpy: Use the new device life cycle helpers
  vfio/mtty: Use the new device life cycle helpers
  vfio/mbochs: Use the new device life cycle helpers
  vfio/ap: Use the new device life cycle helpers
  vfio/fsl-mc: Use the new device life cycle helpers
  vfio: Add struct device to vfio_device

 drivers/gpu/drm/i915/gvt/gvt.h                |   5 +-
 drivers/gpu/drm/i915/gvt/kvmgt.c              |  52 ++++--
 drivers/gpu/drm/i915/gvt/vgpu.c               |  31 ++--
 drivers/s390/cio/vfio_ccw_ops.c               |  52 +++++-
 drivers/s390/cio/vfio_ccw_private.h           |   3 +
 drivers/s390/crypto/vfio_ap_ops.c             |  50 +++---
 drivers/vfio/fsl-mc/vfio_fsl_mc.c             |  87 +++++----
 .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c    |  80 ++++-----
 drivers/vfio/pci/mlx5/main.c                  |  49 ++++--
 drivers/vfio/pci/vfio_pci.c                   |  20 +--
 drivers/vfio/pci/vfio_pci_core.c              |  23 ++-
 drivers/vfio/platform/vfio_amba.c             |  72 ++++++--
 drivers/vfio/platform/vfio_platform.c         |  66 +++++--
 drivers/vfio/platform/vfio_platform_common.c  |  61 +++----
 drivers/vfio/platform/vfio_platform_private.h |  18 +-
 drivers/vfio/vfio_main.c                      | 165 +++++++++++++++---
 include/linux/vfio.h                          |  29 ++-
 include/linux/vfio_pci_core.h                 |   6 +-
 samples/vfio-mdev/mbochs.c                    |  73 +++++---
 samples/vfio-mdev/mdpy.c                      |  81 +++++----
 samples/vfio-mdev/mtty.c                      |  67 ++++---
 21 files changed, 724 insertions(+), 366 deletions(-)


base-commit: 1c23f9e627a7b412978b4e852793c5e3c3efc555

Comments

Tian, Kevin Aug. 27, 2022, 10:41 p.m. UTC | #1
lkp reported some warnings below. I'll got them fixed in the next
version. Not sure why they are not captured in my builds though
I did turn on treating warnings as error.

drivers/vfio/fsl-mc/vfio_fsl_mc.c:570:6: warning: no previous prototype for 'vfio_fsl_mc_release_dev' [-Wmissing-prototypes]
drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c:1216:5: warning: no previous prototype for 'hisi_acc_vfio_pci_migrn_init_dev' [-Wmissing-prototypes]
drivers/vfio/platform/vfio_amba.c:110:37: warning: unused variable 'vfio_platform_ops' [-Wunused-const-variable]
drivers/vfio/platform/vfio_platform_common.c:608:37: warning: unused variable 'vfio_platform_ops' [-Wunused-const-variable]


> From: Tian, Kevin <kevin.tian@intel.com>
> Sent: Sunday, August 28, 2022 1:10 AM
> 
> The idea is to let vfio core manage the vfio_device life cycle instead
> of duplicating the logic cross drivers. Besides cleaner code in driver
> side this also allows adding struct device to vfio_device as the first
> step toward adding cdev uAPI in the future. Another benefit is that
> user can now look at sysfs to decide whether a device is bound to
> vfio [1], e.g.:
> 
> 	/sys/devices/pci0000\:6f/0000\:6f\:01.0/vfio-dev/vfio0
> 
> Though most drivers can fit the new model naturally:
> 
>  - vfio_alloc_device() to allocate and initialize vfio_device
>  - vfio_put_device() to release vfio_device
>  - dev_ops->init() for driver private initialization
>  - dev_ops->release() for driver private cleanup
> 
> vfio-ccw is the only exception due to a life cycle mess that its private
> structure mixes both parent and mdev info hence must be alloc/free'ed
> outside of the life cycle of vfio device.
> 
> Per prior discussions this won't be fixed in short term by IBM folks [2].
> 
> Instead of waiting this series introduces a few tricks to move forward:
> 
>  - vfio_init_device() to initialize a pre-allocated device structure;
> 
>  - require *EVERY* driver to implement @release and free vfio_device
>    inside. Then vfio-ccw can use a completion mechanism to delay the
>    free to css driver;
> 
> The second trick is not a real burden to other drivers because they
> all require a @release for private cleanup anyay. Later once the ccw
> mess is fixed a simple cleanup can be done by moving free from @release
> to vfio core.
> 
> Thanks
> Kevin
> 
> [1] https://listman.redhat.com/archives/libvir-list/2022-August/233482.html
> [2]
> https://lore.kernel.org/all/0ee29bd6583f17f0ee4ec0769fa50e8ea6703623.ca
> mel@linux.ibm.com/
> 
> Kevin Tian (6):
>   vfio: Add helpers for unifying vfio_device life cycle
>   drm/i915/gvt: Use the new device life cycle helpers
>   vfio/platform: Use the new device life cycle helpers
>   vfio/amba: Use the new device life cycle helpers
>   vfio/ccw: Use the new device life cycle helpers
>   vfio: Rename vfio_device_put() and vfio_device_try_get()
> 
> Yi Liu (9):
>   vfio/pci: Use the new device life cycle helpers
>   vfio/mlx5: Use the new device life cycle helpers
>   vfio/hisi_acc: Use the new device life cycle helpers
>   vfio/mdpy: Use the new device life cycle helpers
>   vfio/mtty: Use the new device life cycle helpers
>   vfio/mbochs: Use the new device life cycle helpers
>   vfio/ap: Use the new device life cycle helpers
>   vfio/fsl-mc: Use the new device life cycle helpers
>   vfio: Add struct device to vfio_device
> 
>  drivers/gpu/drm/i915/gvt/gvt.h                |   5 +-
>  drivers/gpu/drm/i915/gvt/kvmgt.c              |  52 ++++--
>  drivers/gpu/drm/i915/gvt/vgpu.c               |  31 ++--
>  drivers/s390/cio/vfio_ccw_ops.c               |  52 +++++-
>  drivers/s390/cio/vfio_ccw_private.h           |   3 +
>  drivers/s390/crypto/vfio_ap_ops.c             |  50 +++---
>  drivers/vfio/fsl-mc/vfio_fsl_mc.c             |  87 +++++----
>  .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c    |  80 ++++-----
>  drivers/vfio/pci/mlx5/main.c                  |  49 ++++--
>  drivers/vfio/pci/vfio_pci.c                   |  20 +--
>  drivers/vfio/pci/vfio_pci_core.c              |  23 ++-
>  drivers/vfio/platform/vfio_amba.c             |  72 ++++++--
>  drivers/vfio/platform/vfio_platform.c         |  66 +++++--
>  drivers/vfio/platform/vfio_platform_common.c  |  61 +++----
>  drivers/vfio/platform/vfio_platform_private.h |  18 +-
>  drivers/vfio/vfio_main.c                      | 165 +++++++++++++++---
>  include/linux/vfio.h                          |  29 ++-
>  include/linux/vfio_pci_core.h                 |   6 +-
>  samples/vfio-mdev/mbochs.c                    |  73 +++++---
>  samples/vfio-mdev/mdpy.c                      |  81 +++++----
>  samples/vfio-mdev/mtty.c                      |  67 ++++---
>  21 files changed, 724 insertions(+), 366 deletions(-)
> 
> 
> base-commit: 1c23f9e627a7b412978b4e852793c5e3c3efc555
> --
> 2.21.3
Jason Gunthorpe Aug. 31, 2022, 12:34 a.m. UTC | #2
On Sun, Aug 28, 2022 at 01:10:22AM +0800, Kevin Tian wrote:

> Kevin Tian (6):
>   vfio: Add helpers for unifying vfio_device life cycle
>   drm/i915/gvt: Use the new device life cycle helpers
>   vfio/platform: Use the new device life cycle helpers
>   vfio/amba: Use the new device life cycle helpers
>   vfio/ccw: Use the new device life cycle helpers
>   vfio: Rename vfio_device_put() and vfio_device_try_get()
> 
> Yi Liu (9):
>   vfio/pci: Use the new device life cycle helpers
>   vfio/mlx5: Use the new device life cycle helpers
>   vfio/hisi_acc: Use the new device life cycle helpers
>   vfio/mdpy: Use the new device life cycle helpers
>   vfio/mtty: Use the new device life cycle helpers
>   vfio/mbochs: Use the new device life cycle helpers
>   vfio/ap: Use the new device life cycle helpers
>   vfio/fsl-mc: Use the new device life cycle helpers
>   vfio: Add struct device to vfio_device

Other than my small remarks this all looked good to me - for every patch:

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>

Thanks,
Jason
Tian, Kevin Aug. 31, 2022, 6:14 a.m. UTC | #3
> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, August 31, 2022 8:35 AM
> 
> On Sun, Aug 28, 2022 at 01:10:22AM +0800, Kevin Tian wrote:
> 
> > Kevin Tian (6):
> >   vfio: Add helpers for unifying vfio_device life cycle
> >   drm/i915/gvt: Use the new device life cycle helpers
> >   vfio/platform: Use the new device life cycle helpers
> >   vfio/amba: Use the new device life cycle helpers
> >   vfio/ccw: Use the new device life cycle helpers
> >   vfio: Rename vfio_device_put() and vfio_device_try_get()
> >
> > Yi Liu (9):
> >   vfio/pci: Use the new device life cycle helpers
> >   vfio/mlx5: Use the new device life cycle helpers
> >   vfio/hisi_acc: Use the new device life cycle helpers
> >   vfio/mdpy: Use the new device life cycle helpers
> >   vfio/mtty: Use the new device life cycle helpers
> >   vfio/mbochs: Use the new device life cycle helpers
> >   vfio/ap: Use the new device life cycle helpers
> >   vfio/fsl-mc: Use the new device life cycle helpers
> >   vfio: Add struct device to vfio_device
> 
> Other than my small remarks this all looked good to me - for every patch:
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> 

Thanks for the review!