From patchwork Sat Sep 26 09:44:56 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cong Zhang X-Patchwork-Id: 7269121 Return-Path: X-Original-To: patchwork-intel-gfx@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 9B56D9F380 for ; Sat, 26 Sep 2015 09:45:04 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id D343A20641 for ; Sat, 26 Sep 2015 09:45:02 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) by mail.kernel.org (Postfix) with ESMTP id E7DAC20619 for ; Sat, 26 Sep 2015 09:45:00 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0F50C6E1AF; Sat, 26 Sep 2015 02:44:59 -0700 (PDT) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mail-qk0-f195.google.com (mail-qk0-f195.google.com [209.85.220.195]) by gabe.freedesktop.org (Postfix) with ESMTPS id 94D416E1AF for ; Sat, 26 Sep 2015 02:44:57 -0700 (PDT) Received: by qkfr4 with SMTP id r4so5283613qkf.2 for ; Sat, 26 Sep 2015 02:44:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=0+4VTwu+oA2Q2ZPXYtYcVtIcF2ugWe81lCWwpa8p6pg=; b=xtY/+b2rsjAgZjde87K9Zz8OzIP03Eee/WI7e9WYwNtNcZKWR+TfHy3BoiBrVW9yA1 rASf2Zt7p8X1cSoLnOLG4SPaVCVlneEjXgdyc1oAgM7/+Am2qUn/PBSw3ozMA6KgGOEL 4wl2jbcrh3xSqvpDEeH1tFbsugo3yO/uKGRwgT8wC84v7Mo7AfO0ml/s/NEIKazitlLM rpx5VKJnKPbhO+fppxOb8EcR39IVu0hGWRpPAzpmY4es6SOiqVQHpyo7+urCQbe6UXed jaq0GQc5NCuCbTB1xVLG+HApbUs+BDe1F+o4m6pPWm+m/XnnCdc+vp6Lqbz3Dnj0rLWf zCzQ== MIME-Version: 1.0 X-Received: by 10.55.53.142 with SMTP id c136mr326917qka.89.1443260696606; Sat, 26 Sep 2015 02:44:56 -0700 (PDT) Received: by 10.55.45.3 with HTTP; Sat, 26 Sep 2015 02:44:56 -0700 (PDT) Date: Sat, 26 Sep 2015 17:44:56 +0800 Message-ID: From: Cong Zhang To: intel-gfx@lists.freedesktop.org, xorg-devel@lists.x.org, vfio-users , debian-kernel@lists.debian.org Subject: [Intel-gfx] any one try to make vga passthrough work with new kernel(4.1.6 but not 4.3) under igfx+nvidia? X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, HTML_MESSAGE, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hello all, I try to make VGA pass through works, but always failed. I try to manually apply the patch from https://lkml.org/lkml/2014/5/9/517 , it seems the code struct has changed, the the patch does not work as expect! I can make KVM+OVFM work, but for legacy mode, my Nvidia card never successful pass through to vm. something out of expect: 1. As expect, drm should disable, but glxinfo show direct rendering is yes 2. disable_vga_mem was called, but no effect dome dmesg output: *dmesg|grep vgaarb* [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.1.6 root=UUID=728f9f82-2c79-4054-8b66-df6b92623d52 ro quiet intel_iommu=on,igfx_off i915.enable_hd_vgaarb=1 enable_hd_vgaarb=1 kvm.allow_unsafe_assigned_interrupts=1 allow_unsafe_assigned_interrupts=1 blacklist.nouveau=1 [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.1.6 root=UUID=728f9f82-2c79-4054-8b66-df6b92623d52 ro quiet intel_iommu=on,igfx_off i915.enable_hd_vgaarb=1 enable_hd_vgaarb=1 kvm.allow_unsafe_assigned_interrupts=1 allow_unsafe_assigned_interrupts=1 blacklist.nouveau=1 [ 0.168279] vgaarb: setting as boot device: PCI:0000:00:02.0 [ 0.168280] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none [ 0.168283] vgaarb: device added: PCI:0000:01:00.0,decodes=io+mem,owns=io+mem,locks=none [ 0.168284] vgaarb: loaded [ 0.168284] vgaarb: bridge control possible 0000:01:00.0 [ 0.168285] vgaarb: no bridge control possible 0000:00:02.0 [ 18.470036] vgaarb: device changed decodes: PCI:0000:00:02.0,olddecodes=io+mem,decodes=io:owns=io+mem *dmesg|grep drm* [ 17.640797] [drm] Initialized drm 1.1.0 20060810 [ 18.457550] [drm] Memory usable by graphics device = 2048M [ 18.457555] fb: switching to inteldrmfb from simple [ 18.457659] [drm] Replacing VGA console driver [ 18.464174] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013). [ 18.464175] [drm] Driver supports precise vblank timestamp query. [ 18.471443] [drm] Initialized i915 1.6.0 20150327 for 0000:00:02.0 on minor 0 [ 18.494301] fbcon: inteldrmfb (fb0) is primary device [ 18.537278] i915 0000:00:02.0: fb0: inteldrmfb frame buffer device *dmesg|grep iommu* [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.1.6 root=UUID=728f9f82-2c79-4054-8b66-df6b92623d52 ro quiet intel_iommu=on,igfx_off i915.enable_hd_vgaarb=1 enable_hd_vgaarb=1 kvm.allow_unsafe_assigned_interrupts=1 allow_unsafe_assigned_interrupts=1 blacklist.nouveau=1 [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.1.6 root=UUID=728f9f82-2c79-4054-8b66-df6b92623d52 ro quiet intel_iommu=on,igfx_off i915.enable_hd_vgaarb=1 enable_hd_vgaarb=1 kvm.allow_unsafe_assigned_interrupts=1 allow_unsafe_assigned_interrupts=1 blacklist.nouveau=1 congzhang@debian:~/projects/linux-4.1.6$ dmesg|grep -i iommu [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.1.6 root=UUID=728f9f82-2c79-4054-8b66-df6b92623d52 ro quiet intel_iommu=on,igfx_off i915.enable_hd_vgaarb=1 enable_hd_vgaarb=1 kvm.allow_unsafe_assigned_interrupts=1 allow_unsafe_assigned_interrupts=1 blacklist.nouveau=1 [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.1.6 root=UUID=728f9f82-2c79-4054-8b66-df6b92623d52 ro quiet intel_iommu=on,igfx_off i915.enable_hd_vgaarb=1 enable_hd_vgaarb=1 kvm.allow_unsafe_assigned_interrupts=1 allow_unsafe_assigned_interrupts=1 blacklist.nouveau=1 [ 0.000000] Intel-IOMMU: enabled [ 0.000000] Intel-IOMMU: disable GFX device mapping [ 0.022861] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a [ 0.022866] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008020660462 ecap f010da [ 0.022869] IOAPIC id 8 under DRHD base 0xfed91000 IOMMU 1 [ 0.453609] IOMMU: dmar1 using Queued invalidation [ 0.453611] IOMMU: Setting RMRR: [ 0.453622] IOMMU: Setting identity map for device 0000:00:14.0 [0x8de7b000 - 0x8de89fff] [ 0.453647] IOMMU: Setting identity map for device 0000:00:1a.0 [0x8de7b000 - 0x8de89fff] [ 0.453667] IOMMU: Setting identity map for device 0000:00:1d.0 [0x8de7b000 - 0x8de89fff] [ 0.453680] IOMMU: Prepare 0-16MiB unity mapping for LPC [ 0.453687] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff] [ 0.463405] AMD IOMMUv2 driver by Joerg Roedel [ 0.463406] AMD IOMMUv2 functionality not available on this system * glxinfo|head* name of display: :0 display: :0 screen: 0 direct rendering: Yes server glx vendor string: SGI server glx version string: 1.4 server glx extensions: GLX_ARB_multisample, GLX_EXT_import_context, GLX_EXT_texture_from_pixmap, GLX_EXT_visual_info, GLX_EXT_visual_rating, GLX_MESA_copy_sub_buffer, GLX_OML_swap_method, GLX_SGIS_multisample, GLX_SGIX_fbconfig, GLX_SGIX_pbuffer, GLX_SGI_make_current_read *sudo cat /dev/vga_arbiter |head -2* count:2,PCI:0000:00:02.0,decodes=io,owns=io+mem,locks=none(0:0) count:2,PCI:0000:00:02.0,decodes=io,owns=io+mem,locks=none(0:0) *lspci -vnn* 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK106 [GeForce GTX 660] [10de:11c0] (rev a1) (prog-if 00 [VGA controller]) Subsystem: Gigabyte Technology Co., Ltd Device [1458:354e] Flags: bus master, fast devsel, latency 0, IRQ 255 Memory at f6000000 (32-bit, non-prefetchable) [size=16M] Memory at e0000000 (64-bit, prefetchable) [size=128M] Memory at e8000000 (64-bit, prefetchable) [size=32M] I/O ports at e000 [size=128] Expansion ROM at f7000000 [disabled] [size=512K] Capabilities: 01:00.1 Audio device [0403]: NVIDIA Corporation GK106 HDMI Audio Controller [10de:0e0b] (rev a1) Subsystem: Gigabyte Technology Co., Ltd Device [1458:354e] Flags: bus master, fast devsel, latency 0, IRQ 17 Memory at f7080000 (32-bit, non-prefetchable) [size=16K] Capabilities: Kernel driver in use: snd_hda_intel *dmesg|grep monkey(myself log)* [ 18.470027] ---monkey---ret = intel_fbdev_init(dev);---monkey---begin--i915_disable_vga_mem(dev);---monkey--- /* Disable VGA memory on Intel HD */ [ 18.470038] ---monkey--- run/* Disable VGA memory on Intel HD */---monkey---end--i915_disable_vga_mem(dev); *my version(which not work as expect and has some mor log) of vgaarb(for debian testing kernel 4.1.6) was attatched!* *any help or advice will be very appreciate,* Thanks, Cong Index: linux-4.1.6/drivers/gpu/drm/i915/i915_dma.c =================================================================== --- linux-4.1.6.orig/drivers/gpu/drm/i915/i915_dma.c +++ linux-4.1.6/drivers/gpu/drm/i915/i915_dma.c @@ -400,11 +400,21 @@ static int i915_load_modeset_init(struct * If we are a secondary display controller (!PCI_DISPLAY_CLASS_VGA), * then we do not take part in VGA arbitration and the * vga_client_register() fails with -ENODEV. + * + * NB. The set_decode callback here actually only works on GMCH + * devices, on newer HD devices we can only disable VGA MMIO space. + * Disabling VGA I/O space requires disabling I/O in the PCI command + * register. Nonetheless, we like to pretend that we participate in + * VGA arbitration and can dynamically disable VGA I/O space because + * this makes X happy, even though it's a complete lie. */ - ret = vga_client_register(dev->pdev, dev, NULL, i915_vga_set_decode); - if (ret && ret != -ENODEV) - goto out; - + if (!i915.enable_hd_vgaarb || !HAS_PCH_SPLIT(dev)) { + ret = vga_client_register(dev->pdev, dev, NULL, + i915_vga_set_decode); + if (ret && ret != -ENODEV) + goto out; + } + intel_register_dsm_handler(); ret = vga_switcheroo_register_client(dev->pdev, &i915_switcheroo_ops, false); @@ -440,6 +450,7 @@ static int i915_load_modeset_init(struct if (INTEL_INFO(dev)->num_pipes == 0) return 0; + printk("---monkey---ret = intel_fbdev_init(dev);"); ret = intel_fbdev_init(dev); if (ret) goto cleanup_gem; @@ -457,8 +468,17 @@ static int i915_load_modeset_init(struct * scanning against hotplug events. Hence do this first and ignore the * tiny window where we will loose hotplug notifactions. */ - async_schedule(intel_fbdev_initial_config, dev_priv); + //async_schedule(intel_fbdev_initial_config, dev_priv); + intel_fbdev_initial_config(dev_priv,0); + /* + * Must do this after fbcon init so that + * vgacon_save_screen() works during the handover. + */ + printk("---monkey---begin--i915_disable_vga_mem(dev);"); + i915_disable_vga_mem(dev); + printk("---monkey---end--i915_disable_vga_mem(dev);"); + drm_kms_helper_poll_init(dev); return 0; Index: linux-4.1.6/drivers/gpu/drm/i915/i915_drv.h =================================================================== --- linux-4.1.6.orig/drivers/gpu/drm/i915/i915_drv.h +++ linux-4.1.6/drivers/gpu/drm/i915/i915_drv.h @@ -2465,12 +2465,13 @@ struct i915_params { bool prefault_disable; bool load_detect_test; bool reset; - bool disable_display; + bool disable_display; bool disable_vtd_wa; int use_mmio_flip; int mmio_debug; bool verbose_state_checks; bool nuclear_pageflip; + bool enable_hd_vgaarb; }; extern struct i915_params i915 __read_mostly; Index: linux-4.1.6/drivers/gpu/drm/i915/i915_params.c =================================================================== --- linux-4.1.6.orig/drivers/gpu/drm/i915/i915_params.c +++ linux-4.1.6/drivers/gpu/drm/i915/i915_params.c @@ -53,6 +53,7 @@ struct i915_params i915 __read_mostly = .mmio_debug = 0, .verbose_state_checks = 1, .nuclear_pageflip = 0, + .enable_hd_vgaarb = 1, }; module_param_named(modeset, i915.modeset, int, 0400); @@ -168,6 +169,10 @@ module_param_named(enable_cmd_parser, i9 MODULE_PARM_DESC(enable_cmd_parser, "Enable command parsing (1=enabled [default], 0=disabled)"); +module_param_named(enable_hd_vgaarb, i915.enable_hd_vgaarb, bool, 0444); +MODULE_PARM_DESC(enable_hd_vgaarb, + "Enable support for VGA arbitration on Intel HD IGD. (default: false)"); + module_param_named(use_mmio_flip, i915.use_mmio_flip, int, 0600); MODULE_PARM_DESC(use_mmio_flip, "use MMIO flips (-1=never, 0=driver discretion [default], 1=always)"); Index: linux-4.1.6/drivers/gpu/drm/i915/intel_display.c =================================================================== --- linux-4.1.6.orig/drivers/gpu/drm/i915/intel_display.c +++ linux-4.1.6/drivers/gpu/drm/i915/intel_display.c @@ -13743,6 +13743,36 @@ static void i915_disable_vga(struct drm_ POSTING_READ(vga_reg); } +static void i915_enable_vga_mem(struct drm_device *dev) +{ + /* Enable VGA memory on Intel HD */ + if (i915.enable_hd_vgaarb && HAS_PCH_SPLIT(dev)) { + vga_get_uninterruptible(dev->pdev, VGA_RSRC_LEGACY_IO); + outb(inb(VGA_MSR_READ) | VGA_MSR_MEM_EN, VGA_MSR_WRITE); + vga_set_legacy_decoding(dev->pdev, VGA_RSRC_LEGACY_IO | + VGA_RSRC_LEGACY_MEM | + VGA_RSRC_NORMAL_IO | + VGA_RSRC_NORMAL_MEM); + vga_put(dev->pdev, VGA_RSRC_LEGACY_IO); + } +} + +void i915_disable_vga_mem(struct drm_device *dev) +{ + printk("---monkey--- /* Disable VGA memory on Intel HD */"); + /* Disable VGA memory on Intel HD */ + if (i915.enable_hd_vgaarb && HAS_PCH_SPLIT(dev)) { + vga_get_uninterruptible(dev->pdev, VGA_RSRC_LEGACY_IO); + outb(inb(VGA_MSR_READ) & ~VGA_MSR_MEM_EN, VGA_MSR_WRITE); + vga_set_legacy_decoding(dev->pdev, VGA_RSRC_LEGACY_IO | + VGA_RSRC_NORMAL_IO | + VGA_RSRC_NORMAL_MEM); + vga_put(dev->pdev, VGA_RSRC_LEGACY_IO); + printk("---monkey--- run/* Disable VGA memory on Intel HD */"); + } +} + + void intel_modeset_init_hw(struct drm_device *dev) { intel_prepare_ddi(dev); @@ -14070,9 +14100,14 @@ void i915_redisable_vga_power_on(struct struct drm_i915_private *dev_priv = dev->dev_private; u32 vga_reg = i915_vgacntrl_reg(dev); + + printk("---monkey---i915_redisable_vga_power_on"); + if (!(I915_READ(vga_reg) & VGA_DISP_DISABLE)) { DRM_DEBUG_KMS("Something enabled VGA plane, disabling it\n"); i915_disable_vga(dev); + i915_disable_vga_mem(dev); + printk("---monkey---i915_redisable_vga_power_on in if"); } } @@ -14353,6 +14388,8 @@ void intel_modeset_cleanup(struct drm_de intel_fbc_disable(dev); + i915_enable_vga_mem(dev); + mutex_unlock(&dev->struct_mutex); /* flush any delayed tasks or pending work */ Index: linux-4.1.6/drivers/gpu/drm/i915/intel_drv.h =================================================================== --- linux-4.1.6.orig/drivers/gpu/drm/i915/intel_drv.h +++ linux-4.1.6/drivers/gpu/drm/i915/intel_drv.h @@ -1291,6 +1291,8 @@ void intel_pre_disable_primary(struct dr /* intel_tv.c */ void intel_tv_init(struct drm_device *dev); +extern void i915_disable_vga_mem(struct drm_device *dev); + /* intel_atomic.c */ int intel_atomic_check(struct drm_device *dev, struct drm_atomic_state *state);