diff mbox

drm/amdgpu: Fix default page access routing

Message ID 1446750375-9967-1-git-send-email-jay@jcornwall.me (mailing list archive)
State New, archived
Headers show

Commit Message

Jay Cornwall Nov. 5, 2015, 7:06 p.m. UTC
The VM default page (used when a VM translation fails) is allocated in
system memory. The VM is misconfigured to interpret the physical address
as referencing a VRAM physical page.

Route default page accesses to system memory.

Signed-off-by: Jay Cornwall <jay@jcornwall.me>
Cc: <stable@vger.kernel.org> # v4.2+
---
 drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 1 +
 drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 1 +
 2 files changed, 2 insertions(+)

Comments

Christian König Nov. 6, 2015, 8:28 a.m. UTC | #1
On 05.11.2015 20:06, Jay Cornwall wrote:
> The VM default page (used when a VM translation fails) is allocated in
> system memory. The VM is misconfigured to interpret the physical address
> as referencing a VRAM physical page.
>
> Route default page accesses to system memory.
>
> Signed-off-by: Jay Cornwall <jay@jcornwall.me>
> Cc: <stable@vger.kernel.org> # v4.2+

Nice catch, patch is Reviewed-by: Christian König <christian.koenig@amd.com>

Do we also need this for Radeon?

Regards,
Christian.

> ---
>   drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 1 +
>   drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 1 +
>   2 files changed, 2 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> index fab5471..b9836f6 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> @@ -474,6 +474,7 @@ static int gmc_v7_0_gart_enable(struct amdgpu_device *adev)
>   	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
>   	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
>   	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
> +	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
>   	WREG32(mmVM_L2_CNTL, tmp);
>   	tmp = REG_SET_FIELD(0, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
>   	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_L2_CACHE, 1);
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> index 7bc9e9f..cb4e2bb 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> @@ -588,6 +588,7 @@ static int gmc_v8_0_gart_enable(struct amdgpu_device *adev)
>   	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
>   	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
>   	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
> +	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
>   	WREG32(mmVM_L2_CNTL, tmp);
>   	tmp = RREG32(mmVM_L2_CNTL2);
>   	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
Jay Cornwall Nov. 6, 2015, 2:40 p.m. UTC | #2
On 2015-11-06 02:28, Christian König wrote:
> On 05.11.2015 20:06, Jay Cornwall wrote:
>> The VM default page (used when a VM translation fails) is allocated in
>> system memory. The VM is misconfigured to interpret the physical 
>> address
>> as referencing a VRAM physical page.
>> 
>> Route default page accesses to system memory.
>> 
>> Signed-off-by: Jay Cornwall <jay@jcornwall.me>
>> Cc: <stable@vger.kernel.org> # v4.2+
> 
> Nice catch, patch is Reviewed-by: Christian König 
> <christian.koenig@amd.com>
> 
> Do we also need this for Radeon?

It looks like this field was introduced in CIK. This change could be 
replicated in radeon on those parts, but I haven't got one to test with.

I'm not sure if this misconfiguration is fatal. The default page should 
be used for loads only. I haven't been able to get anything besides 0 
from a faulting load on my Fiji with a system or VRAM default page. A 
physical address > VRAM doesn't seem to cause problems, either.

My primary motivation for this change is to allow HSA to handle faulting 
atomic operations. The KFD exposes a MTYPE=UC aperture to the user which 
allows atomics to be forwarded to system memory, via MC/BIF/PCI. When 
one of these atomic operations faults the MC is unable to handle the 
VRAM atomic (atomics are implemented by TC) and the TC hangs waiting for 
a response.

If the default page is directed towards BIF then the atomic is correctly 
discarded.
Alex Deucher Nov. 10, 2015, 6:05 p.m. UTC | #3
On Thu, Nov 5, 2015 at 2:06 PM, Jay Cornwall <jay@jcornwall.me> wrote:
> The VM default page (used when a VM translation fails) is allocated in
> system memory. The VM is misconfigured to interpret the physical address
> as referencing a VRAM physical page.
>
> Route default page accesses to system memory.
>
> Signed-off-by: Jay Cornwall <jay@jcornwall.me>
> Cc: <stable@vger.kernel.org> # v4.2+

Applied.  thanks!

Alex

> ---
>  drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 1 +
>  drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 1 +
>  2 files changed, 2 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> index fab5471..b9836f6 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> @@ -474,6 +474,7 @@ static int gmc_v7_0_gart_enable(struct amdgpu_device *adev)
>         tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
>         tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
>         tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
> +       tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
>         WREG32(mmVM_L2_CNTL, tmp);
>         tmp = REG_SET_FIELD(0, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
>         tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_L2_CACHE, 1);
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> index 7bc9e9f..cb4e2bb 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> @@ -588,6 +588,7 @@ static int gmc_v8_0_gart_enable(struct amdgpu_device *adev)
>         tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
>         tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
>         tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
> +       tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
>         WREG32(mmVM_L2_CNTL, tmp);
>         tmp = RREG32(mmVM_L2_CNTL2);
>         tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
> --
> 1.9.1
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel
diff mbox

Patch

diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
index fab5471..b9836f6 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
@@ -474,6 +474,7 @@  static int gmc_v7_0_gart_enable(struct amdgpu_device *adev)
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
+	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
 	WREG32(mmVM_L2_CNTL, tmp);
 	tmp = REG_SET_FIELD(0, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_L2_CACHE, 1);
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
index 7bc9e9f..cb4e2bb 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
@@ -588,6 +588,7 @@  static int gmc_v8_0_gart_enable(struct amdgpu_device *adev)
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
+	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
 	WREG32(mmVM_L2_CNTL, tmp);
 	tmp = RREG32(mmVM_L2_CNTL2);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);