From patchwork Mon May 9 15:48:11 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 769622 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter2.kernel.org (8.14.4/8.14.3) with ESMTP id p49FmcK2021854 for ; Mon, 9 May 2011 15:48:38 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752873Ab1EIPsR (ORCPT ); Mon, 9 May 2011 11:48:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:62970 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751758Ab1EIPsQ (ORCPT ); Mon, 9 May 2011 11:48:16 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p49FmDlD020207 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 9 May 2011 11:48:13 -0400 Received: from [10.3.113.10] ([10.3.113.10]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id p49FmCRj008272; Mon, 9 May 2011 11:48:12 -0400 Subject: Re: Graphics pass-through From: Alex Williamson To: Jan Kiszka Cc: Prasad Joshi , =?ISO-8859-1?Q?Andr=E9?= Weidemann , "kvm@vger.kernel.org" , Oswaldo Cadenas , Maxim Nikolaev In-Reply-To: <4DC807E3.4080706@siemens.com> References: <4D415D70.8070105@web.de> <1296175554.2891.29.camel@x201> <4DC264CD.9080700@siemens.com> <1304608642.3081.35.camel@x201> <4DC7CCB3.8030607@siemens.com> <4DC807E3.4080706@siemens.com> Date: Mon, 09 May 2011 09:48:11 -0600 Message-ID: <1304956091.26106.39.camel@x201> Mime-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter2.kernel.org [140.211.167.43]); Mon, 09 May 2011 15:48:38 +0000 (UTC) On Mon, 2011-05-09 at 17:27 +0200, Jan Kiszka wrote: > On 2011-05-09 16:55, Prasad Joshi wrote: > > On Mon, May 9, 2011 at 12:14 PM, Jan Kiszka wrote: > >> On 2011-05-05 17:17, Alex Williamson wrote: > >>>> And what about the host? When does Linux release the legacy range? > >>>> Always or only when a specific (!=vga/vesa) framebuffer driver is loaded? > >>> > >>> Well, that's where it'd be nice if the vga arbiter was actually in more > >>> widespread use. It currently seems to be nothing more than a shared > >>> mutex, but it would actually be useful if it included backends to do the > >>> chipset vga routing changes. I think when I was testing this, I was > >>> externally poking PCI bridge chipset to toggle the VGA_EN bit. > >> > >> Right, we had to drop the approach to pass through the secondary card > >> for now, the arbiter was not switching properly. Haven't checked yet if > >> VGA_EN was properly set, though the kernel code looks like it should > >> take care of this. > >> > >> Even with handing out the primary adapter, we had only mixed success so > >> far. The onboard adapter worked well (in VESA mode), but the NVIDIA is > >> not displaying early boot messages at all. Maybe a vgabios issue. > >> Windows was booting nevertheless - until we installed the NVIDIA > >> drivers. Than it ran into a blue screen. > >> > >> BTW, what ATI adapter did you use precisely, and what did work, what not? > > > > Not hijacking the mail thread. Just wanted to provide some inputs. > > Much appreciated in fact! > > > > > Few days back I had tried passing through the secondary graphics card. > > I could pass-through two graphics cards to virtual machine. > > > > 02:00.0 VGA compatible controller: ATI Technologies Inc Redwood > > [Radeon HD 5670] (prog-if 00 [VGA controller]) > > Subsystem: PC Partner Limited Device e151 > > Flags: bus master, fast devsel, latency 0, IRQ 87 > > Memory at d0000000 (64-bit, prefetchable) [size=256M] > > Memory at fe6e0000 (64-bit, non-prefetchable) [size=128K] > > I/O ports at b000 [size=256] > > Expansion ROM at fe6c0000 [disabled] [size=128K] > > Capabilities: > > Kernel driver in use: radeon > > Kernel modules: radeon > > > > 07:00.0 VGA compatible controller: nVidia Corporation G86 [Quadro NVS > > 290] (rev a1) (prog-if 00 [VGA controller]) > > Subsystem: nVidia Corporation Device 0492 > > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > > ParErr-Stepping- SERR+ FastB2B- DisINTx- > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast > >> TAbort-SERR- > Latency: 0, Cache Line Size: 64 bytes > > Interrupt: pin A routed to IRQ 24 > > Region 0: Memory at fd000000 (32-bit, non-prefetchable) [size=16M] > > Region 1: Memory at d0000000 (64-bit, prefetchable) [size=256M] > > Region 3: Memory at fa000000 (64-bit, non-prefetchable) [size=32M] > > Region 5: I/O ports at ec00 [size=128] > > Expansion ROM at fe9e0000 [disabled] [size=128K] > > Capabilities: > > Kernel driver in use: nouveau > > Kernel modules: nouveau, nvidiafb > > > > Both of them are PCIe cards. I have one more ATI card and another > > NVIDIA card which does not work. > > Interesting. That may rule out missing PCIe capabilities as source for > the NVIDIA driver indisposition. > > Did you passed those cards each as primary to the guest, or was the > guest seeing multiple adapters? I presume you only got output after > early boot was completed, right? > > To avoid having to deal with legacy I/O forwarding, we started with a > dual adapter setup in the hope to leave the primary guest adapter at > know-to-work cirrus-vga. But already in a native setup with on-board > primary + NVIDIA secondary, the NVIDIA Windows drivers refused to talk > to its hardware in this constellation. > > > > > One of the reason the pass-through did not work is because of the > > limit on amount of pci configuration memory by SeaBIOS. SeaBIOS places > > a hard limit of 256MB or so on the amount of PCI memory space. Thus, > > for some of the VGA device that need more memory never worked for me. > > > > SeaBIOS allows this memory region to be extended to some value near > > 512MB, but even then the range is not enough. > > > > Another problem with SeaBIOS which limits the amount of memory space > > is: SeaBIOS allocates the BAR regions as they are encountered. As far > > as I know, the BAR regions should be naturally aligned. Thus the > > simple strategy of the SeaBIOS results in large fragmentation. > > Therefore, even after increasing the PCI memory space to 512MB the BAR > > regions were unallocated. > > That's an interesting trace! We'll check this here, but I bet it > contributes to the problems. Our FX 3800 has 1G memory... Yes, qemu leaves far too little MMIO space to think about assigning graphics cards. Both of my cards have 512MB and I hacked qemu to leave a bigger gap via something like: That's all the tricks I remember. Thanks, Alex --- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/hw/pc.c b/hw/pc.c index 0ea6d10..a6376f8 100644 --- a/hw/pc.c +++ b/hw/pc.c @@ -879,6 +879,8 @@ void pc_cpus_init(const char *cpu_model) } } +#define PC_MAX_LOW_RAM 0xc0000000 + void pc_memory_init(ram_addr_t ram_size, const char *kernel_filename, const char *kernel_cmdline, @@ -893,9 +895,9 @@ void pc_memory_init(ram_addr_t ram_size, int bios_size, isa_bios_size; void *fw_cfg; - if (ram_size >= 0xe0000000 ) { - above_4g_mem_size = ram_size - 0xe0000000; - below_4g_mem_size = 0xe0000000; + if (ram_size >= PC_MAX_LOW_RAM ) { + above_4g_mem_size = ram_size - PC_MAX_LOW_RAM; + below_4g_mem_size = PC_MAX_LOW_RAM; } else { below_4g_mem_size = ram_size; } There's also a #define that needs to be changed in seabios config.h and and acpi dsdt update, but I can't seem to find patches for those. Also pay attention to the cpu_register_physical_memory calls in i440fx_update_memory_mappings(), those can steal the legacy VGA MMIO range from you. I just commented them out: diff --git a/hw/piix_pci.c b/hw/piix_pci.c index b5589b9..1327563 100644 --- a/hw/piix_pci.c +++ b/hw/piix_pci.c @@ -106,11 +106,11 @@ static void i440fx_update_memory_mappings(PCII440FXState *d) } smram = d->dev.config[I440FX_SMRAM]; if ((d->smm_enabled && (smram & 0x08)) || (smram & 0x40)) { - cpu_register_physical_memory(0xa0000, 0x20000, 0xa0000); + //cpu_register_physical_memory(0xa0000, 0x20000, 0xa0000); } else { for(addr = 0xa0000; addr < 0xc0000; addr += 4096) { - cpu_register_physical_memory(addr, 4096, - d->isa_page_descs[(addr - 0xa0000) >> 12]); + //cpu_register_physical_memory(addr, 4096, + // d->isa_page_descs[(addr - 0xa0000) >> 12]); } } }