| Message ID | 20230928164550.980832-2-dtatulea@nvidia.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | vdpa: Add support for vq descriptor mappings | expand |
On Thu, Sep 28, 2023 at 07:45:11PM +0300, Dragos Tatulea wrote: > This patch series adds support for vq descriptor table mappings which > are used to improve vdpa live migration downtime. The improvement comes > from using smaller mappings which take less time to create and destroy > in hw. > > The first part adds the vdpa core changes from Si-Wei [0]. > > The second part adds support in mlx5_vdpa: > - Refactor the mr code to be able to cleanly add descriptor mappings. > - Add hardware descriptor mr support. > - Properly update iotlb for cvq during ASID switch. > > Changes in v2: > > - The "vdpa/mlx5: Enable hw support for vq descriptor mapping" change > was split off into two patches to avoid merge conflicts into the tree > of Linus. > > The first patch contains only changes for mlx5_ifc.h. This must be > applied into the mlx5-next tree [1] first. Once this patch is applied > on mlx5-next, the change has to be pulled fom mlx5-next into the vhost > tree and only then the remaining patches can be applied. > > [0] https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com > [1] https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next > > Dragos Tatulea (13): > vdpa/mlx5: Expose descriptor group mkey hw capability I prepared shared branch with this patch. https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-vhost Thanks
On Thu, 28 Sep 2023 19:45:11 +0300, Dragos Tatulea wrote: > This patch series adds support for vq descriptor table mappings which > are used to improve vdpa live migration downtime. The improvement comes > from using smaller mappings which take less time to create and destroy > in hw. > > The first part adds the vdpa core changes from Si-Wei [0]. > > [...] Applied, thanks! [01/16] vdpa/mlx5: Expose descriptor group mkey hw capability https://git.kernel.org/rdma/rdma/c/d424348b060d87 Best regards,
On Thu, Sep 28, 2023 at 07:45:11PM +0300, Dragos Tatulea wrote: > This patch series adds support for vq descriptor table mappings which > are used to improve vdpa live migration downtime. The improvement comes > from using smaller mappings which take less time to create and destroy > in hw. > > The first part adds the vdpa core changes from Si-Wei [0]. > > The second part adds support in mlx5_vdpa: > - Refactor the mr code to be able to cleanly add descriptor mappings. > - Add hardware descriptor mr support. > - Properly update iotlb for cvq during ASID switch. > > Changes in v2: > > - The "vdpa/mlx5: Enable hw support for vq descriptor mapping" change > was split off into two patches to avoid merge conflicts into the tree > of Linus. > > The first patch contains only changes for mlx5_ifc.h. This must be > applied into the mlx5-next tree [1] first. Once this patch is applied > on mlx5-next, the change has to be pulled fom mlx5-next into the vhost > tree and only then the remaining patches can be applied. I get it you plan v3? > [0] https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com > [1] https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next > > Dragos Tatulea (13): > vdpa/mlx5: Expose descriptor group mkey hw capability > vdpa/mlx5: Create helper function for dma mappings > vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code > vdpa/mlx5: Take cvq iotlb lock during refresh > vdpa/mlx5: Collapse "dvq" mr add/delete functions > vdpa/mlx5: Rename mr destroy functions > vdpa/mlx5: Allow creation/deletion of any given mr struct > vdpa/mlx5: Move mr mutex out of mr struct > vdpa/mlx5: Improve mr update flow > vdpa/mlx5: Introduce mr for vq descriptor > vdpa/mlx5: Enable hw support for vq descriptor mapping > vdpa/mlx5: Make iotlb helper functions more generic > vdpa/mlx5: Update cvq iotlb mapping on ASID change > > Si-Wei Liu (3): > vdpa: introduce dedicated descriptor group for virtqueue > vhost-vdpa: introduce descriptor group backend feature > vhost-vdpa: uAPI to get dedicated descriptor group id > > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++-- > drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++------------- > drivers/vdpa/mlx5/core/resources.c | 6 +- > drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++----- > drivers/vhost/vdpa.c | 27 ++++ > include/linux/mlx5/mlx5_ifc.h | 8 +- > include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +- > include/linux/vdpa.h | 11 ++ > include/uapi/linux/vhost.h | 8 ++ > include/uapi/linux/vhost_types.h | 5 + > 10 files changed, 264 insertions(+), 130 deletions(-) > > -- > 2.41.0
On Thu, 2023-10-05 at 13:31 -0400, Michael S. Tsirkin wrote: > On Thu, Sep 28, 2023 at 07:45:11PM +0300, Dragos Tatulea wrote: > > This patch series adds support for vq descriptor table mappings which > > are used to improve vdpa live migration downtime. The improvement comes > > from using smaller mappings which take less time to create and destroy > > in hw. > > > > The first part adds the vdpa core changes from Si-Wei [0]. > > > > The second part adds support in mlx5_vdpa: > > - Refactor the mr code to be able to cleanly add descriptor mappings. > > - Add hardware descriptor mr support. > > - Properly update iotlb for cvq during ASID switch. > > > > Changes in v2: > > > > - The "vdpa/mlx5: Enable hw support for vq descriptor mapping" change > > was split off into two patches to avoid merge conflicts into the tree > > of Linus. > > > > The first patch contains only changes for mlx5_ifc.h. This must be > > applied into the mlx5-next tree [1] first. Once this patch is applied > > on mlx5-next, the change has to be pulled fom mlx5-next into the vhost > > tree and only then the remaining patches can be applied. > > > I get it you plan v3? There are some very small improvements (commit message in 13/16 and fix in 16/16) that could make a v3. The latter can be addressed as a separate patch when moving dup_iotlb to vhost/iotlb. What do you think? > > > [0] > > https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com > > [1] > > https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next > > > > Dragos Tatulea (13): > > vdpa/mlx5: Expose descriptor group mkey hw capability > > vdpa/mlx5: Create helper function for dma mappings > > vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code > > vdpa/mlx5: Take cvq iotlb lock during refresh > > vdpa/mlx5: Collapse "dvq" mr add/delete functions > > vdpa/mlx5: Rename mr destroy functions > > vdpa/mlx5: Allow creation/deletion of any given mr struct > > vdpa/mlx5: Move mr mutex out of mr struct > > vdpa/mlx5: Improve mr update flow > > vdpa/mlx5: Introduce mr for vq descriptor > > vdpa/mlx5: Enable hw support for vq descriptor mapping > > vdpa/mlx5: Make iotlb helper functions more generic > > vdpa/mlx5: Update cvq iotlb mapping on ASID change > > > > Si-Wei Liu (3): > > vdpa: introduce dedicated descriptor group for virtqueue > > vhost-vdpa: introduce descriptor group backend feature > > vhost-vdpa: uAPI to get dedicated descriptor group id > > > > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++-- > > drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++------------- > > drivers/vdpa/mlx5/core/resources.c | 6 +- > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++----- > > drivers/vhost/vdpa.c | 27 ++++ > > include/linux/mlx5/mlx5_ifc.h | 8 +- > > include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +- > > include/linux/vdpa.h | 11 ++ > > include/uapi/linux/vhost.h | 8 ++ > > include/uapi/linux/vhost_types.h | 5 + > > 10 files changed, 264 insertions(+), 130 deletions(-) > > > > -- > > 2.41.0 >
On Thu, Oct 05, 2023 at 05:44:01PM +0000, Dragos Tatulea wrote: > On Thu, 2023-10-05 at 13:31 -0400, Michael S. Tsirkin wrote: > > On Thu, Sep 28, 2023 at 07:45:11PM +0300, Dragos Tatulea wrote: > > > This patch series adds support for vq descriptor table mappings which > > > are used to improve vdpa live migration downtime. The improvement comes > > > from using smaller mappings which take less time to create and destroy > > > in hw. > > > > > > The first part adds the vdpa core changes from Si-Wei [0]. > > > > > > The second part adds support in mlx5_vdpa: > > > - Refactor the mr code to be able to cleanly add descriptor mappings. > > > - Add hardware descriptor mr support. > > > - Properly update iotlb for cvq during ASID switch. > > > > > > Changes in v2: > > > > > > - The "vdpa/mlx5: Enable hw support for vq descriptor mapping" change > > > was split off into two patches to avoid merge conflicts into the tree > > > of Linus. > > > > > > The first patch contains only changes for mlx5_ifc.h. This must be > > > applied into the mlx5-next tree [1] first. Once this patch is applied > > > on mlx5-next, the change has to be pulled fom mlx5-next into the vhost > > > tree and only then the remaining patches can be applied. > > > > > > I get it you plan v3? > There are some very small improvements (commit message in 13/16 and fix in > 16/16) that could make a v3. The latter can be addressed as a separate patch > when moving dup_iotlb to vhost/iotlb. What do you think? if there's a fix by all means post v3. > > > > > [0] > > > https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com > > > [1] > > > https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next > > > > > > Dragos Tatulea (13): > > > vdpa/mlx5: Expose descriptor group mkey hw capability > > > vdpa/mlx5: Create helper function for dma mappings > > > vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code > > > vdpa/mlx5: Take cvq iotlb lock during refresh > > > vdpa/mlx5: Collapse "dvq" mr add/delete functions > > > vdpa/mlx5: Rename mr destroy functions > > > vdpa/mlx5: Allow creation/deletion of any given mr struct > > > vdpa/mlx5: Move mr mutex out of mr struct > > > vdpa/mlx5: Improve mr update flow > > > vdpa/mlx5: Introduce mr for vq descriptor > > > vdpa/mlx5: Enable hw support for vq descriptor mapping > > > vdpa/mlx5: Make iotlb helper functions more generic > > > vdpa/mlx5: Update cvq iotlb mapping on ASID change > > > > > > Si-Wei Liu (3): > > > vdpa: introduce dedicated descriptor group for virtqueue > > > vhost-vdpa: introduce descriptor group backend feature > > > vhost-vdpa: uAPI to get dedicated descriptor group id > > > > > > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++-- > > > drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++------------- > > > drivers/vdpa/mlx5/core/resources.c | 6 +- > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++----- > > > drivers/vhost/vdpa.c | 27 ++++ > > > include/linux/mlx5/mlx5_ifc.h | 8 +- > > > include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +- > > > include/linux/vdpa.h | 11 ++ > > > include/uapi/linux/vhost.h | 8 ++ > > > include/uapi/linux/vhost_types.h | 5 + > > > 10 files changed, 264 insertions(+), 130 deletions(-) > > > > > > -- > > > 2.41.0 > > >