drm/i915: Disable lazy PPGTT page table optimization for vGPU

Commit Message

Joonas Lahtinen Oct. 18, 2017, 10:25 a.m. UTC

When running under virtualization (vGPU active), we must disable
the azy PPGTT page table initialization optimization introduced by:

14826673247e ("drm/i915: Only initialize partially filled pagetables")

We must do this because GVT-g makes unduly assumptions about guest
behaviour, which this optimization breaks. This results in following
looking errors in the host:

ERROR gvt: guest page write error -22, gfn 0x7ada8, pa 0x7ada89a8, var 0x6, len 1

The real fix is to not to depend on i915 driver behaviour, but instead
either rely on only the contracts that i915 has with the hardware, or
add some paravirtualization. While the real fix is en route, it won't
be finished in time for 4.15, so the best option is to disable the
optimization for now when vGPU is active to avoid breaking 4.15 guests
in existing VM environments.

Fixes: 14826673247e ("drm/i915: Only initialize partially filled pagetables")
Suggested-by: Xiaolin Zhang <[e-mail]>
Signed-off-by: Xiaolin Zhang <[e-mail]>
[Joonas: Rewrote the commit message and added tags.]
Signed-off-by: Joonas Lahtinen <[e-mail]>
Cc: Zhenyu Wang <[e-mail]>
Cc: Zhi Wang <[e-mail]>
Cc: Chris Wilson <[e-mail]>
Cc: Matthew Auld <[e-mail]>
Cc: Joonas Lahtinen <[e-mail]>
Cc: Jani Nikula <[e-mail]>
Cc: Rodrigo Vivi <[e-mail]>
---
 drivers/gpu/drm/i915/i915_gem_gtt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Chris Wilson Oct. 18, 2017, 10:29 a.m. UTC | #1

Quoting Joonas Lahtinen (2017-10-18 11:25:43)
> When running under virtualization (vGPU active), we must disable
> the azy PPGTT page table initialization optimization introduced by:
> 
> 14826673247e ("drm/i915: Only initialize partially filled pagetables")
> 
> We must do this because GVT-g makes unduly assumptions about guest
> behaviour, which this optimization breaks. This results in following
> looking errors in the host:
> 
> ERROR gvt: guest page write error -22, gfn 0x7ada8, pa 0x7ada89a8, var 0x6, len 1
> 
> The real fix is to not to depend on i915 driver behaviour, but instead
> either rely on only the contracts that i915 has with the hardware, or
> add some paravirtualization. While the real fix is en route, it won't
> be finished in time for 4.15, so the best option is to disable the
> optimization for now when vGPU is active to avoid breaking 4.15 guests
> in existing VM environments.
> 
> Fixes: 14826673247e ("drm/i915: Only initialize partially filled pagetables")
> Suggested-by: Xiaolin Zhang <[e-mail]>
> Signed-off-by: Xiaolin Zhang <[e-mail]>
> [Joonas: Rewrote the commit message and added tags.]
> Signed-off-by: Joonas Lahtinen <[e-mail]>
> Cc: Zhenyu Wang <[e-mail]>
> Cc: Zhi Wang <[e-mail]>
> Cc: Chris Wilson <[e-mail]>
> Cc: Matthew Auld <[e-mail]>
> Cc: Joonas Lahtinen <[e-mail]>
> Cc: Jani Nikula <[e-mail]>
> Cc: Rodrigo Vivi <[e-mail]>

Acked-by: Chris Wilson <[e-mail]>

I do not claim to understand why, even after reading the explanation.
-Chris

Zhenyu Wang Oct. 19, 2017, 2:48 a.m. UTC | #2

On 2017.10.18 13:25:43 +0300, Joonas Lahtinen wrote:
> When running under virtualization (vGPU active), we must disable
> the azy PPGTT page table initialization optimization introduced by:
> 
> 14826673247e ("drm/i915: Only initialize partially filled pagetables")
> 
> We must do this because GVT-g makes unduly assumptions about guest
> behaviour, which this optimization breaks. This results in following
> looking errors in the host:
> 
> ERROR gvt: guest page write error -22, gfn 0x7ada8, pa 0x7ada89a8, var 0x6, len 1
> 
> The real fix is to not to depend on i915 driver behaviour, but instead
> either rely on only the contracts that i915 has with the hardware, or
> add some paravirtualization. While the real fix is en route, it won't
> be finished in time for 4.15, so the best option is to disable the
> optimization for now when vGPU is active to avoid breaking 4.15 guests
> in existing VM environments.
> 
> Fixes: 14826673247e ("drm/i915: Only initialize partially filled pagetables")
> Suggested-by: Xiaolin Zhang <[e-mail]>
> Signed-off-by: Xiaolin Zhang <[e-mail]>
> [Joonas: Rewrote the commit message and added tags.]
> Signed-off-by: Joonas Lahtinen <[e-mail]>
> Cc: Zhenyu Wang <[e-mail]>
> Cc: Zhi Wang <[e-mail]>
> Cc: Chris Wilson <[e-mail]>
> Cc: Matthew Auld <[e-mail]>
> Cc: Joonas Lahtinen <[e-mail]>
> Cc: Jani Nikula <[e-mail]>
> Cc: Rodrigo Vivi <[e-mail]>
> ---
>  drivers/gpu/drm/i915/i915_gem_gtt.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
> index 527a2d2d6281..5eaa6893daaa 100644
> --- a/drivers/gpu/drm/i915/i915_gem_gtt.c
> +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
> @@ -1341,7 +1341,7 @@ static int gen8_ppgtt_alloc_pd(struct i915_address_space *vm,
>  			if (IS_ERR(pt))
>  				goto unwind;
>  
> -			if (count < GEN8_PTES)
> +			if (count < GEN8_PTES || intel_vgpu_active(vm->i915))
>  				gen8_initialize_pt(vm, pt);
>  
>  			gen8_ppgtt_set_pde(vm, pd, pt, pde);
> -- 

Acked-by: Zhenyu Wang <[e-mail]>

As this optimization would be broken on older kernel for GVT-g, final fix
need to do compat check to really enable this. So ok to fallback for now.

thanks

Joonas Lahtinen Oct. 20, 2017, 9:03 a.m. UTC | #3

Yesterday, Zhi was still looking if this could be fixed by a small
modification in the GVT-g side, which would be much preferred.

Zhi, What's the status there?

Regards, Joonas

On Thu, 2017-10-19 at 10:48 +0800, Zhenyu Wang wrote:
> On 2017.10.18 13:25:43 +0300, Joonas Lahtinen wrote:
> > When running under virtualization (vGPU active), we must disable
> > the azy PPGTT page table initialization optimization introduced by:
> > 
> > 14826673247e ("drm/i915: Only initialize partially filled pagetables")
> > 
> > We must do this because GVT-g makes unduly assumptions about guest
> > behaviour, which this optimization breaks. This results in following
> > looking errors in the host:
> > 
> > ERROR gvt: guest page write error -22, gfn 0x7ada8, pa 0x7ada89a8, var 0x6, len 1
> > 
> > The real fix is to not to depend on i915 driver behaviour, but instead
> > either rely on only the contracts that i915 has with the hardware, or
> > add some paravirtualization. While the real fix is en route, it won't
> > be finished in time for 4.15, so the best option is to disable the
> > optimization for now when vGPU is active to avoid breaking 4.15 guests
> > in existing VM environments.
> > 
> > Fixes: 14826673247e ("drm/i915: Only initialize partially filled pagetables")
> > Suggested-by: Xiaolin Zhang <[e-mail]>
> > Signed-off-by: Xiaolin Zhang <[e-mail]>
> > [Joonas: Rewrote the commit message and added tags.]
> > Signed-off-by: Joonas Lahtinen <[e-mail]>
> > Cc: Zhenyu Wang <[e-mail]>
> > Cc: Zhi Wang <[e-mail]>
> > Cc: Chris Wilson <[e-mail]>
> > Cc: Matthew Auld <[e-mail]>
> > Cc: Joonas Lahtinen <[e-mail]>
> > Cc: Jani Nikula <[e-mail]>
> > Cc: Rodrigo Vivi <[e-mail]>

<SNIP>

Wang, Zhi A Oct. 20, 2017, 9:06 a.m. UTC | #4

Hi Joonas
    Better you can take this patch first, since I was still stuck in floods of creating invalid entries in page table.

Thanks,
Zhi.

-----Original Message-----
From: Joonas Lahtinen [mailto:[e-mail]] 

Sent: Friday, October 20, 2017 12:04 PM
To: Zhenyu Wang <[e-mail]>; Wang, Zhi A <[e-mail]>
Cc: Intel graphics driver community testing & development <[e-mail]>; Zhang, Xiaolin <[e-mail]>; Chris Wilson <[e-mail]>; Auld, Matthew <[e-mail]>; Jani Nikula <[e-mail]>; Vivi, Rodrigo <[e-mail]>
Subject: Re: [PATCH] drm/i915: Disable lazy PPGTT page table optimization for vGPU

Yesterday, Zhi was still looking if this could be fixed by a small modification in the GVT-g side, which would be much preferred.

Zhi, What's the status there?

Regards, Joonas

On Thu, 2017-10-19 at 10:48 +0800, Zhenyu Wang wrote:
> On 2017.10.18 13:25:43 +0300, Joonas Lahtinen wrote:

> > When running under virtualization (vGPU active), we must disable the 

> > azy PPGTT page table initialization optimization introduced by:

> > 

> > 14826673247e ("drm/i915: Only initialize partially filled 

> > pagetables")

> > 

> > We must do this because GVT-g makes unduly assumptions about guest 

> > behaviour, which this optimization breaks. This results in following 

> > looking errors in the host:

> > 

> > ERROR gvt: guest page write error -22, gfn 0x7ada8, pa 0x7ada89a8, 

> > var 0x6, len 1

> > 

> > The real fix is to not to depend on i915 driver behaviour, but 

> > instead either rely on only the contracts that i915 has with the 

> > hardware, or add some paravirtualization. While the real fix is en 

> > route, it won't be finished in time for 4.15, so the best option is 

> > to disable the optimization for now when vGPU is active to avoid 

> > breaking 4.15 guests in existing VM environments.

> > 

> > Fixes: 14826673247e ("drm/i915: Only initialize partially filled 

> > pagetables")

> > Suggested-by: Xiaolin Zhang <[e-mail]>

> > Signed-off-by: Xiaolin Zhang <[e-mail]>

> > [Joonas: Rewrote the commit message and added tags.]

> > Signed-off-by: Joonas Lahtinen <[e-mail]>

> > Cc: Zhenyu Wang <[e-mail]>

> > Cc: Zhi Wang <[e-mail]>

> > Cc: Chris Wilson <[e-mail]>

> > Cc: Matthew Auld <[e-mail]>

> > Cc: Joonas Lahtinen <[e-mail]>

> > Cc: Jani Nikula <[e-mail]>

> > Cc: Rodrigo Vivi <[e-mail]>


<SNIP>
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation

Patch

diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 527a2d2d6281..5eaa6893daaa 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -1341,7 +1341,7 @@  static int gen8_ppgtt_alloc_pd(struct i915_address_space *vm,
 			if (IS_ERR(pt))
 				goto unwind;
 
-			if (count < GEN8_PTES)
+			if (count < GEN8_PTES || intel_vgpu_active(vm->i915))
 				gen8_initialize_pt(vm, pt);
 
 			gen8_ppgtt_set_pde(vm, pd, pt, pde);