| Message ID | 20241203133548.38252-1-tomitamoeko@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | vfio/igd: Enable legacy mode on more devices | expand |
On Tue, 3 Dec 2024 21:35:39 +0800 Tomita Moeko <tomitamoeko@gmail.com> wrote: > This patchset extends the support of legacy mode igd passthrough to > all Intel Gen 11 and 12 devices (including Ice Lake, Jasper Lake, > Rocket Lake, Alder Lake and Raptor Lake), and emulates GGC register > in MMIO BAR0 for better compatibiltiy (It is tested Windows and GOP > driver will read this MMIO register). > > It also replaces magic numbers with macros to improve readability, > and aligns behavior (BDSM registor mirroring and GGMS calculation for > gen7) with i915 driver to avoid possible issues. > > The x-igd-gms option removed in 971ca22f041b ("vfio/igd: don't set > stolen memory size to zero") is also added back so that data stolen > memory size can be specified for guest. It is tested that GMS may > related to framebuffer size, a small GMS value may cause display issues > like blackscreen. It can be changed by DVMT Pre-allocated option in > host BIOS, but not all BIOS comes with this option. Having it in QEMU > helps resolves such issues. > > This patchset was verified on Intel i9-12900K CPU(UHD 770, 8086:4680) > with custom OVMF firmware [1] and IntelGopDriver extracted from host > bios. IGD device works well in both Windows and Linux guests, and > scored 726 in 3DMark Time Spy Graphics on Windows guest. > > [1] https://github.com/tomitamoeko/edk2/commits/igd-pt-adl/ > > Btw, IO BAR4 seems never be used by guest, and it the IO BAR itself > is not working on Gen11+ devices in my experiments. There is no hints > about that in old commit message and mailing list. It would be greatly > appreciated if someone shares the background. The quirks related to BAR4 access are generally for the vBIOS, we wouldn't expect guest OS level drivers to use them. IIRC this is handling moving the stolen memory from the HPA to the GPA when the vBIOS is writing the GTT. Maybe that brings up an interesting topic. Traditionally "legacy mode" IGD assignment has been only for 440fx machines with SeaBIOS and last I was aware edk2 wasn't willing to accept the same hack for the BDSM as we had put into SeaBIOS, instead indicating that it should be implemented in the device ROM. Your branch in [1] above seems to indicate edk2 does now have assigned IGD specific code. Are these patches developing full stack support of these new devices, from BIOS hand-off, through pre-boot environments, and through to guest OS drivers, or are we only concerned that the guest OS level driver lights up a display? If you're using q35 and OVMF then you must be operating in the realm of the mythical "Universal Pass-through" mode that I thought Intel had abandoned. It seems like we need an update to docs/igd-assign.txt as it's likely very out of date based on recent improvements here and by Corvin. Also, are you proposing the noted edk2 change upstream? It seems like edk2 would need some sort of device version detection to know whether to use a 32 or 64-bit BDSM value. Thanks, Alex > Changelog: > v2: > * Droped "vfio/igd: fix GTT stolen memory size calculation for gen 7". > * Fixed conditions when calculating GGMS size. > * Added Gemini Lake and Comet Lake device ids. > * Splited mirroring register declaration macro into a new patch. > * Minor fixes. > Link: https://lore.kernel.org/qemu-devel/20241201160938.44355-1-tomitamoeko@gmail.com/ > > Tomita Moeko (9): > vfio/igd: remove unsupported device ids > vfio/igd: align generation with i915 kernel driver > vfio/igd: canonicalize memory size calculations > vfio/igd: add Gemini Lake and Comet Lake device ids > vfio/igd: add Alder/Raptor/Rocket/Ice/Jasper Lake device ids > vfio/igd: add macro for declaring mirrored registers > vfio/igd: emulate GGC register in mmio bar0 > vfio/igd: emulate BDSM in mmio bar0 for gen 6-10 devices > vfio/igd: add x-igd-gms option back to set DSM region size for guest > > hw/vfio/igd.c | 248 ++++++++++++++++++++++++++++++-------------------- > 1 file changed, 151 insertions(+), 97 deletions(-) >
On 12/4/24 04:12, Alex Williamson wrote: > On Tue, 3 Dec 2024 21:35:39 +0800 > Tomita Moeko <tomitamoeko@gmail.com> wrote: > >> This patchset extends the support of legacy mode igd passthrough to >> all Intel Gen 11 and 12 devices (including Ice Lake, Jasper Lake, >> Rocket Lake, Alder Lake and Raptor Lake), and emulates GGC register >> in MMIO BAR0 for better compatibiltiy (It is tested Windows and GOP >> driver will read this MMIO register). >> >> It also replaces magic numbers with macros to improve readability, >> and aligns behavior (BDSM registor mirroring and GGMS calculation for >> gen7) with i915 driver to avoid possible issues. >> >> The x-igd-gms option removed in 971ca22f041b ("vfio/igd: don't set >> stolen memory size to zero") is also added back so that data stolen >> memory size can be specified for guest. It is tested that GMS may >> related to framebuffer size, a small GMS value may cause display issues >> like blackscreen. It can be changed by DVMT Pre-allocated option in >> host BIOS, but not all BIOS comes with this option. Having it in QEMU >> helps resolves such issues. >> >> This patchset was verified on Intel i9-12900K CPU(UHD 770, 8086:4680) >> with custom OVMF firmware [1] and IntelGopDriver extracted from host >> bios. IGD device works well in both Windows and Linux guests, and >> scored 726 in 3DMark Time Spy Graphics on Windows guest. >> >> [1] https://github.com/tomitamoeko/edk2/commits/igd-pt-adl/ >> >> Btw, IO BAR4 seems never be used by guest, and it the IO BAR itself >> is not working on Gen11+ devices in my experiments. There is no hints >> about that in old commit message and mailing list. It would be greatly >> appreciated if someone shares the background. > > The quirks related to BAR4 access are generally for the vBIOS, we > wouldn't expect guest OS level drivers to use them. IIRC this is > handling moving the stolen memory from the HPA to the GPA when the > vBIOS is writing the GTT. Got it. I'm wondering why vBIOS still writes HPA instead of GPA when it's in virtual machine, maybe the address is hardcoded? > Maybe that brings up an interesting topic. Traditionally "legacy mode" > IGD assignment has been only for 440fx machines with SeaBIOS and last I > was aware edk2 wasn't willing to accept the same hack for the BDSM as > we had put into SeaBIOS, instead indicating that it should be > implemented in the device ROM. Your branch in [1] above seems to > indicate edk2 does now have assigned IGD specific code. > > Are these patches developing full stack support of these new devices, > from BIOS hand-off, through pre-boot environments, and through to guest > OS drivers, or are we only concerned that the guest OS level driver > lights up a display? Yes these patches provide a complete legacy mode passthrough solution, from EFI DXE phase to guest OS, but the EFI part requires specific changes in edk2. > If you're using q35 and OVMF then you must be operating in the realm of > the mythical "Universal Pass-through" mode that I thought Intel had > abandoned. It seems like we need an update to docs/igd-assign.txt as > it's likely very out of date based on recent improvements here and by > Corvin. Actually the only machine supports legacy mode is i440fx, windows driver checks the vendor and device id of LPC bridge device at 00:1f.0, if it doesn't match, display driver won't work [1]. On q35 machine, there is already a emulated ICH9 LPC at 00.1f.0. Previous there was a try in modifying the id, but it breaks functionality [2]. [1] https://github.com/projectacrn/acrn-hypervisor/blob/master/devicemodel/hw/pci/lpc.c#L519 [2] https://lore.kernel.org/all/1457080913-30018-1-git-send-email-kraxel@redhat.com/ I had a try "Univerisal Pass-through" mode as igd-assign.txt with my UHD 770. On linux guest, it works just as a normal gpu, except there is no display output before i915 driver loaded. Even the device's vbdf is not 00:02.0, and i915 log shows the DSM is 0M, it works perfectly. intel_gpu_top shows igpu is working when I am playing a youtube video. I also tried setting primary gpu in bios to discrete gpu, which makes the pci class code of igpu changed from 0x030000 (VGA compatible controller) to 0x308000 (Display controller), it can still output to monitor connected to it on Linux guest. All with a simple -device vfio-pci,host=00:02.0,id=hostdev0 But for windows guest, I never had any luck. I attached a virtio-gpu to it, with intel grahics drivers installed in guest. If igd is not at 00:02.0 or gop driver is not provided, windows BSOD immediately on boot > Also, are you proposing the noted edk2 change upstream? It seems like > edk2 would need some sort of device version detection to know whether > to use a 32 or 64-bit BDSM value. Thanks, > > Alex I'm afraid the answer is no, these edk2 changes are not fully open source as they were taken from inten directly [3], except the last patch. (It seems intel uses a modified qemu as "etc/igd-dsm-base" used in patch 5 doesn't exist in qemu, probably they are creating a identical GPA->HPA mapping for igd?). One of these intel edk2 patches is also included in a edk2 bug [4] and markd as hack. Besides these edk changes, GOP driver is also needed. Intel never released them to public. Even acrn hypervisor developed by intel says "Fetch the VBT and GOP drivers from the board manufacturer". The only way for users to get it seems to be using tools like "UEFI BIOS Updater" to extract driver from host bios image, or try the luck by downloading the gop driver shared on internet. After having the GOP driver, a virtual rom for igd device in qemu can be created by EfiRom -f 0x8086 -i <device_id> -e IgdAssignmentDxe.efi \ PlatformGOPPolicy.efi IntelGopDriver.efi IgdAssignementDxe.efi and PlatformGOPPolicy.efi are built with the edk2 changes. I'm not sure whether this can be added to official document. [3] https://eci.intel.com/docs/3.0/components/kvm-hypervisor.html#build-ovmf-fd-for-kvm [4] https://bugzilla.tianocore.org/show_bug.cgi?id=935 >> Changelog: >> v2: >> * Droped "vfio/igd: fix GTT stolen memory size calculation for gen 7". >> * Fixed conditions when calculating GGMS size. >> * Added Gemini Lake and Comet Lake device ids. >> * Splited mirroring register declaration macro into a new patch. >> * Minor fixes. >> Link: https://lore.kernel.org/qemu-devel/20241201160938.44355-1-tomitamoeko@gmail.com/ >> >> Tomita Moeko (9): >> vfio/igd: remove unsupported device ids >> vfio/igd: align generation with i915 kernel driver >> vfio/igd: canonicalize memory size calculations >> vfio/igd: add Gemini Lake and Comet Lake device ids >> vfio/igd: add Alder/Raptor/Rocket/Ice/Jasper Lake device ids >> vfio/igd: add macro for declaring mirrored registers >> vfio/igd: emulate GGC register in mmio bar0 >> vfio/igd: emulate BDSM in mmio bar0 for gen 6-10 devices >> vfio/igd: add x-igd-gms option back to set DSM region size for guest >> >> hw/vfio/igd.c | 248 ++++++++++++++++++++++++++++++-------------------- >> 1 file changed, 151 insertions(+), 97 deletions(-) >> >
This patchset extends the support of legacy mode igd passthrough to all Intel Gen 11 and 12 devices (including Ice Lake, Jasper Lake, Rocket Lake, Alder Lake and Raptor Lake), and emulates GGC register in MMIO BAR0 for better compatibiltiy (It is tested Windows and GOP driver will read this MMIO register). It also replaces magic numbers with macros to improve readability, and aligns behavior (BDSM registor mirroring and GGMS calculation for gen7) with i915 driver to avoid possible issues. The x-igd-gms option removed in 971ca22f041b ("vfio/igd: don't set stolen memory size to zero") is also added back so that data stolen memory size can be specified for guest. It is tested that GMS may related to framebuffer size, a small GMS value may cause display issues like blackscreen. It can be changed by DVMT Pre-allocated option in host BIOS, but not all BIOS comes with this option. Having it in QEMU helps resolves such issues. This patchset was verified on Intel i9-12900K CPU(UHD 770, 8086:4680) with custom OVMF firmware [1] and IntelGopDriver extracted from host bios. IGD device works well in both Windows and Linux guests, and scored 726 in 3DMark Time Spy Graphics on Windows guest. [1] https://github.com/tomitamoeko/edk2/commits/igd-pt-adl/ Btw, IO BAR4 seems never be used by guest, and it the IO BAR itself is not working on Gen11+ devices in my experiments. There is no hints about that in old commit message and mailing list. It would be greatly appreciated if someone shares the background. Changelog: v2: * Droped "vfio/igd: fix GTT stolen memory size calculation for gen 7". * Fixed conditions when calculating GGMS size. * Added Gemini Lake and Comet Lake device ids. * Splited mirroring register declaration macro into a new patch. * Minor fixes. Link: https://lore.kernel.org/qemu-devel/20241201160938.44355-1-tomitamoeko@gmail.com/ Tomita Moeko (9): vfio/igd: remove unsupported device ids vfio/igd: align generation with i915 kernel driver vfio/igd: canonicalize memory size calculations vfio/igd: add Gemini Lake and Comet Lake device ids vfio/igd: add Alder/Raptor/Rocket/Ice/Jasper Lake device ids vfio/igd: add macro for declaring mirrored registers vfio/igd: emulate GGC register in mmio bar0 vfio/igd: emulate BDSM in mmio bar0 for gen 6-10 devices vfio/igd: add x-igd-gms option back to set DSM region size for guest hw/vfio/igd.c | 248 ++++++++++++++++++++++++++++++-------------------- 1 file changed, 151 insertions(+), 97 deletions(-)