| Message ID | 20220308184902.2242-1-shameerali.kolothum.thodi@huawei.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | vfio/hisilicon: add ACC live migration driver | expand |
Hi Alex, > -----Original Message----- > From: Shameerali Kolothum Thodi > Sent: 08 March 2022 18:49 > To: kvm@vger.kernel.org; linux-kernel@vger.kernel.org; > linux-crypto@vger.kernel.org > Cc: linux-pci@vger.kernel.org; alex.williamson@redhat.com; jgg@nvidia.com; > cohuck@redhat.com; mgurtovoy@nvidia.com; yishaih@nvidia.com; > kevin.tian@intel.com; Linuxarm <linuxarm@huawei.com>; liulongfang > <liulongfang@huawei.com>; Zengtao (B) <prime.zeng@hisilicon.com>; > Jonathan Cameron <jonathan.cameron@huawei.com>; Wangzhou (B) > <wangzhou1@hisilicon.com> > Subject: [PATCH v9 0/9] vfio/hisilicon: add ACC live migration driver > > Hi, > > This series attempts to add vfio live migration support for HiSilicon > ACC VF devices based on the new v2 migration protocol definition and > mlx5 v9 series discussed here[0]. > > v8 --> v9 > - Added acks by Wangzhou/Longfang/Yekai > - Added R-by tags by Jason. > - Addressed comments by Alex on v8. > - Fixed the pf_queue pointer assignment error in patch #8. > - Addressed comments from Kevin, > -Updated patch #5 commit log msg with a clarification that VF > migration BAR assignment is fine if migration support is not there. > -Added QM description to patch #8 commit msg. Hope there is nothing pending for this series now to make it to next. I know ack from Bjorn is still pending for patch #3, and I have sent a ping last week and also CCd him on that patch. Please let me know if there is anything I missed. Thanks, Shameer > > This is sanity tested on a HiSilicon platform using the Qemu branch > provided here[1]. > > Please take a look and let me know your feedback. > > Thanks, > Shameer > [0] > https://lore.kernel.org/kvm/20220224142024.147653-1-yishaih@nvidia.com/ > [1] https://github.com/jgunthorpe/qemu/commits/vfio_migration_v2 > > v7 --> v8 > - Dropped PRE_COPY support and early compatibility checking based on > the discussion here[1]. > - Addressed comments from John, Jason & Alex (Thanks!). > > v6 --> v7 > -Renamed MIG_PRECOPY ioctl name and struct name. Updated ioctl > descriptions > regarding ioctl validity (patch #7). > - Adressed comments from Jason and Alex on PRE_COPY read() and ioctl() fns > (patch #9). > - Moved only VF PCI ids to pci_ids.h(patch #3). > > v5 --> v6 > -Report PRE_COPY support and use that for early compatibility check > between src and dst devices. > -For generic PRE_COPY support, included patch #7 from Jason(Thanks!). > -Addressed comments from Alex(Thanks!). > -Added the QM state register update to QM driver(patch #8) since that > is being used in migration driver to decide whether the device is > ready to save the state. > > RFCv4 --> v5 > - Dropped RFC tag as v2 migration APIs are more stable now. > - Addressed review comments from Jason and Alex (Thanks!). > > v3 --> RFCv4 > -Based on migration v2 protocol and mlx5 v7 series. > -Added RFC tag again as migration v2 protocol is still under discussion. > -Added new patch #6 to retrieve the PF QM data. > -PRE_COPY compatibility check is now done after the migration data > transfer. This is not ideal and needs discussion. > > RFC v2 --> v3 > -Dropped RFC tag as the vfio_pci_core subsystem framework is now > part of 5.15-rc1. > -Added override methods for vfio_device_ops read/write/mmap calls > to limit the access within the functional register space. > -Patches 1 to 3 are code refactoring to move the common ACC QM > definitions and header around. > > RFCv1 --> RFCv2 > > -Adds a new vendor-specific vfio_pci driver(hisi-acc-vfio-pci) > for HiSilicon ACC VF devices based on the new vfio-pci-core > framework proposal. > > -Since HiSilicon ACC VF device MMIO space contains both the > functional register space and migration control register space, > override the vfio_device_ops ioctl method to report only the > functional space to VMs. > > -For a successful migration, we still need access to VF dev > functional register space mainly to read the status registers. > But accessing these while the Guest vCPUs are running may leave > a security hole. To avoid any potential security issues, we > map/unmap the MMIO regions on a need basis and is safe to do so. > (Please see hisi_acc_vf_ioremap/unmap() fns in patch #4). > > -Dropped debugfs support for now. > -Uses common QM functions for mailbox access(patch #3). > > Longfang Liu (3): > crypto: hisilicon/qm: Move few definitions to common header > crypto: hisilicon/qm: Set the VF QM state register > hisi_acc_vfio_pci: Add support for VFIO live migration > > Shameer Kolothum (6): > crypto: hisilicon/qm: Move the QM header to include/linux > hisi_acc_qm: Move VF PCI device IDs to common header > hisi_acc_vfio_pci: add new vfio_pci driver for HiSilicon ACC devices > hisi_acc_vfio_pci: Restrict access to VF dev BAR2 migration region > hisi_acc_vfio_pci: Add helper to retrieve the struct pci_driver > hisi_acc_vfio_pci: Use its own PCI reset_done error handler > > MAINTAINERS | 7 + > drivers/crypto/hisilicon/hpre/hpre.h | 2 +- > drivers/crypto/hisilicon/hpre/hpre_main.c | 19 +- > drivers/crypto/hisilicon/qm.c | 68 +- > drivers/crypto/hisilicon/sec2/sec.h | 2 +- > drivers/crypto/hisilicon/sec2/sec_main.c | 21 +- > drivers/crypto/hisilicon/sgl.c | 2 +- > drivers/crypto/hisilicon/zip/zip.h | 2 +- > drivers/crypto/hisilicon/zip/zip_main.c | 17 +- > drivers/vfio/pci/Kconfig | 2 + > drivers/vfio/pci/Makefile | 2 + > drivers/vfio/pci/hisilicon/Kconfig | 15 + > drivers/vfio/pci/hisilicon/Makefile | 4 + > .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c | 1326 +++++++++++++++++ > .../vfio/pci/hisilicon/hisi_acc_vfio_pci.h | 116 ++ > .../qm.h => include/linux/hisi_acc_qm.h | 49 + > include/linux/pci_ids.h | 3 + > 17 files changed, 1591 insertions(+), 66 deletions(-) > create mode 100644 drivers/vfio/pci/hisilicon/Kconfig > create mode 100644 drivers/vfio/pci/hisilicon/Makefile > create mode 100644 drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c > create mode 100644 drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h > rename drivers/crypto/hisilicon/qm.h => include/linux/hisi_acc_qm.h (87%) > > -- > 2.25.1
On Mon, 14 Mar 2022 17:26:59 +0000 Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com> wrote: > Hi Alex, > > > -----Original Message----- > > From: Shameerali Kolothum Thodi > > Sent: 08 March 2022 18:49 > > To: kvm@vger.kernel.org; linux-kernel@vger.kernel.org; > > linux-crypto@vger.kernel.org > > Cc: linux-pci@vger.kernel.org; alex.williamson@redhat.com; jgg@nvidia.com; > > cohuck@redhat.com; mgurtovoy@nvidia.com; yishaih@nvidia.com; > > kevin.tian@intel.com; Linuxarm <linuxarm@huawei.com>; liulongfang > > <liulongfang@huawei.com>; Zengtao (B) <prime.zeng@hisilicon.com>; > > Jonathan Cameron <jonathan.cameron@huawei.com>; Wangzhou (B) > > <wangzhou1@hisilicon.com> > > Subject: [PATCH v9 0/9] vfio/hisilicon: add ACC live migration driver > > > > Hi, > > > > This series attempts to add vfio live migration support for HiSilicon > > ACC VF devices based on the new v2 migration protocol definition and > > mlx5 v9 series discussed here[0]. > > > > v8 --> v9 > > - Added acks by Wangzhou/Longfang/Yekai > > - Added R-by tags by Jason. > > - Addressed comments by Alex on v8. > > - Fixed the pf_queue pointer assignment error in patch #8. > > - Addressed comments from Kevin, > > -Updated patch #5 commit log msg with a clarification that VF > > migration BAR assignment is fine if migration support is not there. > > -Added QM description to patch #8 commit msg. > > Hope there is nothing pending for this series now to make it to next. > I know ack from Bjorn is still pending for patch #3, and I have > sent a ping last week and also CCd him on that patch. > > Please let me know if there is anything I missed. Hi Shameer, Nothing still pending from my end, as soon as we can get an ack on patch #3 and barring issues in the interim, I'll merge it. Thanks, Alex
On Tue, 8 Mar 2022 18:48:53 +0000 Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> wrote: > Hi, > > This series attempts to add vfio live migration support for HiSilicon > ACC VF devices based on the new v2 migration protocol definition and > mlx5 v9 series discussed here[0]. > > v8 --> v9 > - Added acks by Wangzhou/Longfang/Yekai > - Added R-by tags by Jason. > - Addressed comments by Alex on v8. > - Fixed the pf_queue pointer assignment error in patch #8. > - Addressed comments from Kevin, > -Updated patch #5 commit log msg with a clarification that VF > migration BAR assignment is fine if migration support is not there. > -Added QM description to patch #8 commit msg. > > This is sanity tested on a HiSilicon platform using the Qemu branch > provided here[1]. > > Please take a look and let me know your feedback. > > Thanks, > Shameer > [0] https://lore.kernel.org/kvm/20220224142024.147653-1-yishaih@nvidia.com/ > [1] https://github.com/jgunthorpe/qemu/commits/vfio_migration_v2 > > v7 --> v8 > - Dropped PRE_COPY support and early compatibility checking based on > the discussion here[1]. > - Addressed comments from John, Jason & Alex (Thanks!). > > v6 --> v7 > -Renamed MIG_PRECOPY ioctl name and struct name. Updated ioctl descriptions > regarding ioctl validity (patch #7). > - Adressed comments from Jason and Alex on PRE_COPY read() and ioctl() fns > (patch #9). > - Moved only VF PCI ids to pci_ids.h(patch #3). > > v5 --> v6 > -Report PRE_COPY support and use that for early compatibility check > between src and dst devices. > -For generic PRE_COPY support, included patch #7 from Jason(Thanks!). > -Addressed comments from Alex(Thanks!). > -Added the QM state register update to QM driver(patch #8) since that > is being used in migration driver to decide whether the device is > ready to save the state. > > RFCv4 --> v5 > - Dropped RFC tag as v2 migration APIs are more stable now. > - Addressed review comments from Jason and Alex (Thanks!). > > v3 --> RFCv4 > -Based on migration v2 protocol and mlx5 v7 series. > -Added RFC tag again as migration v2 protocol is still under discussion. > -Added new patch #6 to retrieve the PF QM data. > -PRE_COPY compatibility check is now done after the migration data > transfer. This is not ideal and needs discussion. > > RFC v2 --> v3 > -Dropped RFC tag as the vfio_pci_core subsystem framework is now > part of 5.15-rc1. > -Added override methods for vfio_device_ops read/write/mmap calls > to limit the access within the functional register space. > -Patches 1 to 3 are code refactoring to move the common ACC QM > definitions and header around. > > RFCv1 --> RFCv2 > > -Adds a new vendor-specific vfio_pci driver(hisi-acc-vfio-pci) > for HiSilicon ACC VF devices based on the new vfio-pci-core > framework proposal. > > -Since HiSilicon ACC VF device MMIO space contains both the > functional register space and migration control register space, > override the vfio_device_ops ioctl method to report only the > functional space to VMs. > > -For a successful migration, we still need access to VF dev > functional register space mainly to read the status registers. > But accessing these while the Guest vCPUs are running may leave > a security hole. To avoid any potential security issues, we > map/unmap the MMIO regions on a need basis and is safe to do so. > (Please see hisi_acc_vf_ioremap/unmap() fns in patch #4). > > -Dropped debugfs support for now. > -Uses common QM functions for mailbox access(patch #3). > > Longfang Liu (3): > crypto: hisilicon/qm: Move few definitions to common header > crypto: hisilicon/qm: Set the VF QM state register > hisi_acc_vfio_pci: Add support for VFIO live migration > > Shameer Kolothum (6): > crypto: hisilicon/qm: Move the QM header to include/linux > hisi_acc_qm: Move VF PCI device IDs to common header > hisi_acc_vfio_pci: add new vfio_pci driver for HiSilicon ACC devices > hisi_acc_vfio_pci: Restrict access to VF dev BAR2 migration region > hisi_acc_vfio_pci: Add helper to retrieve the struct pci_driver > hisi_acc_vfio_pci: Use its own PCI reset_done error handler > > MAINTAINERS | 7 + > drivers/crypto/hisilicon/hpre/hpre.h | 2 +- > drivers/crypto/hisilicon/hpre/hpre_main.c | 19 +- > drivers/crypto/hisilicon/qm.c | 68 +- > drivers/crypto/hisilicon/sec2/sec.h | 2 +- > drivers/crypto/hisilicon/sec2/sec_main.c | 21 +- > drivers/crypto/hisilicon/sgl.c | 2 +- > drivers/crypto/hisilicon/zip/zip.h | 2 +- > drivers/crypto/hisilicon/zip/zip_main.c | 17 +- > drivers/vfio/pci/Kconfig | 2 + > drivers/vfio/pci/Makefile | 2 + > drivers/vfio/pci/hisilicon/Kconfig | 15 + > drivers/vfio/pci/hisilicon/Makefile | 4 + > .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c | 1326 +++++++++++++++++ > .../vfio/pci/hisilicon/hisi_acc_vfio_pci.h | 116 ++ > .../qm.h => include/linux/hisi_acc_qm.h | 49 + > include/linux/pci_ids.h | 3 + > 17 files changed, 1591 insertions(+), 66 deletions(-) > create mode 100644 drivers/vfio/pci/hisilicon/Kconfig > create mode 100644 drivers/vfio/pci/hisilicon/Makefile > create mode 100644 drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c > create mode 100644 drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h > rename drivers/crypto/hisilicon/qm.h => include/linux/hisi_acc_qm.h (87%) > Applied to vfio next branch for v5.18 with reviews/acks from Kevin and Bjorn. Thanks, Alex