| Message ID | 20230605235005.20649-5-ankita@nvidia.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Expose GPU memory as coherently CPU accessible | expand |
On 6/6/23 01:50, ankita@nvidia.com wrote: > From: Ankit Agrawal <ankita@nvidia.com> > > The GPU device memory is reported to the VM as a BAR. The device memory > may not be aligned to the power-of-2, but the QEMU expects the PCI BAR to > be. Align the reported device memory size to the next power-of-2 before > QEMU does an mmap. > > Signed-off-by: Ankit Agrawal <ankita@nvidia.com> > --- > hw/vfio/common.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/hw/vfio/common.c b/hw/vfio/common.c > index 4d01ea3515..bb49200458 100644 > --- a/hw/vfio/common.c > +++ b/hw/vfio/common.c > @@ -2061,7 +2061,7 @@ int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *region, > > region->vbasedev = vbasedev; > region->flags = info->flags; > - region->size = info->size; > + region->size = info->size ? pow2ceil(info->size) : info->size; region->size = [REAL_]HOST_PAGE_ALIGN(info->size)? > region->fd_offset = info->offset; > region->nr = index; >
On Mon, 5 Jun 2023 16:50:05 -0700 <ankita@nvidia.com> wrote: > From: Ankit Agrawal <ankita@nvidia.com> > > The GPU device memory is reported to the VM as a BAR. The device memory > may not be aligned to the power-of-2, but the QEMU expects the PCI BAR to > be. Align the reported device memory size to the next power-of-2 before > QEMU does an mmap. > > Signed-off-by: Ankit Agrawal <ankita@nvidia.com> > --- > hw/vfio/common.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/hw/vfio/common.c b/hw/vfio/common.c > index 4d01ea3515..bb49200458 100644 > --- a/hw/vfio/common.c > +++ b/hw/vfio/common.c > @@ -2061,7 +2061,7 @@ int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *region, > > region->vbasedev = vbasedev; > region->flags = info->flags; > - region->size = info->size; > + region->size = info->size ? pow2ceil(info->size) : info->size; > region->fd_offset = info->offset; > region->nr = index; > Nak, this means our kernel emulation of the BAR is broken, a BAR that is not naturally aligned is not a PCI BAR. PCI BAR sizing through the BAR register still needs to work via the kernel interface alone. It's clear now how the kernel resizing the vma on mmap was a hack around userspace mangling the region size. Maybe this needs to be exposed as a device specific region, which then userspace emulates as a BAR for the VM facing device rather than the kernel emulating it as a BAR. Thanks, Alex
On 6/6/23 14:54, Alex Williamson wrote: > On Mon, 5 Jun 2023 16:50:05 -0700 > <ankita@nvidia.com> wrote: > >> From: Ankit Agrawal <ankita@nvidia.com> >> >> The GPU device memory is reported to the VM as a BAR. The device memory >> may not be aligned to the power-of-2, but the QEMU expects the PCI BAR to >> be. Align the reported device memory size to the next power-of-2 before >> QEMU does an mmap. >> >> Signed-off-by: Ankit Agrawal <ankita@nvidia.com> >> --- >> hw/vfio/common.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/hw/vfio/common.c b/hw/vfio/common.c >> index 4d01ea3515..bb49200458 100644 >> --- a/hw/vfio/common.c >> +++ b/hw/vfio/common.c >> @@ -2061,7 +2061,7 @@ int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *region, >> >> region->vbasedev = vbasedev; >> region->flags = info->flags; >> - region->size = info->size; >> + region->size = info->size ? pow2ceil(info->size) : info->size; >> region->fd_offset = info->offset; >> region->nr = index; >> > > Nak, this means our kernel emulation of the BAR is broken, a BAR that > is not naturally aligned is not a PCI BAR. Right. So the common code could check this value is correct, like: assert(is_power_of_2(->size)); Or less violet using error_report :)
diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 4d01ea3515..bb49200458 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -2061,7 +2061,7 @@ int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *region, region->vbasedev = vbasedev; region->flags = info->flags; - region->size = info->size; + region->size = info->size ? pow2ceil(info->size) : info->size; region->fd_offset = info->offset; region->nr = index;