From patchwork Fri Nov 11 01:02:55 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Russell Currey X-Patchwork-Id: 9425241 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6307860233 for ; Mon, 14 Nov 2016 01:35:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5547826419 for ; Mon, 14 Nov 2016 01:35:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4A33D28868; Mon, 14 Nov 2016 01:35:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E466A26419 for ; Mon, 14 Nov 2016 01:35:29 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 22A9A6E2E2; Mon, 14 Nov 2016 01:33:37 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org X-Greylist: delayed 398 seconds by postgrey-1.35 at gabe; Fri, 11 Nov 2016 01:09:47 UTC Received: from russell.cc (russell.cc [IPv6:2404:9400:2:0:216:3eff:fee0:3370]) by gabe.freedesktop.org (Postfix) with ESMTPS id A67306E2CF; Fri, 11 Nov 2016 01:09:47 +0000 (UTC) Received: from snap.ozlabs.ibm.com (static-82-10.transact.net.au [122.99.82.10]) by russell.cc (OpenSMTPD) with ESMTPSA id ac662670 (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128:NO); Fri, 11 Nov 2016 01:03:07 +0000 (UTC) From: Russell Currey To: amd-gfx@lists.freedesktop.org Subject: [PATCH] drm/amdgpu: Restrict GART size to half of the GPU pages Date: Fri, 11 Nov 2016 12:02:55 +1100 Message-Id: <20161111010255.20542-1-ruscur@russell.cc> X-Mailer: git-send-email 2.10.2 X-Mailman-Approved-At: Mon, 14 Nov 2016 01:31:55 +0000 Cc: dri-devel@lists.freedesktop.org, Russell Currey X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Using the kernel zone memory size to determine the size of the GART table introduced a regression on systems with a large amount of system memory. On a system with 512GB RAM, driver initialisation would fail as there are more GART entries than there are GPU pages to map to them. With 256GB RAM, the table would barely fit, but no other allocations are possible. Restrict the size of the GART such that it will occupy at maximum half of the pages available to the GPU. Fixes: a693e050 ("drm/amdgpu: use the kernel zone memory size as the max remote memory in amdgpu") Signed-off-by: Russell Currey --- I don't know much about GPUs, restricting this to less than half might make more sense. --- drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c | 21 +++++++++++++++------ drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 21 +++++++++++++++------ drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 21 +++++++++++++++------ 3 files changed, 45 insertions(+), 18 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c index b13c8aa..f8bbb17 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c @@ -264,6 +264,7 @@ static int gmc_v6_0_mc_init(struct amdgpu_device *adev) u32 tmp; int chansize, numchan; + u64 max_gtt_entries; tmp = RREG32(MC_ARB_RAMCFG); if (tmp & CHANSIZE_OVERRIDE) { @@ -313,13 +314,21 @@ static int gmc_v6_0_mc_init(struct amdgpu_device *adev) adev->mc.real_vram_size = RREG32(CONFIG_MEMSIZE) * 1024ULL * 1024ULL; adev->mc.visible_vram_size = adev->mc.aper_size; - /* unless the user had overridden it, set the gart - * size equal to the 1024 or vram, whichever is larger. - */ - if (amdgpu_gart_size == -1) - adev->mc.gtt_size = amdgpu_ttm_get_gtt_mem_size(adev); - else + /* unless the user had overridden it, set the GART size proportional + * to system memory, but no more than half of the GPU pages. */ + if (amdgpu_gart_size == -1) { + /* a table entry is 8 GPU pages */ + max_gtt_entries = amdgpu_ttm_get_gtt_mem_size(adev) / + (AMDGPU_GPU_PAGE_SIZE / 8); + + if (max_gtt_entries > adev->mc.visible_vram_size / 2) + adev->mc.gtt_size = (adev->mc.visible_vram_size / 2) * + AMDGPU_GPU_PAGE_SIZE / 8; + else + adev->mc.gtt_size = amdgpu_ttm_get_gtt_mem_size(adev); + } else { adev->mc.gtt_size = (uint64_t)amdgpu_gart_size << 20; + } gmc_v6_0_vram_gtt_location(adev, &adev->mc); diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c index aa0c4b9..276a662 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c @@ -329,6 +329,7 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) { u32 tmp; int chansize, numchan; + u64 max_gtt_entries; /* Get VRAM informations */ tmp = RREG32(mmMC_ARB_RAMCFG); @@ -381,13 +382,21 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) if (adev->mc.visible_vram_size > adev->mc.real_vram_size) adev->mc.visible_vram_size = adev->mc.real_vram_size; - /* unless the user had overridden it, set the gart - * size equal to the 1024 or vram, whichever is larger. - */ - if (amdgpu_gart_size == -1) - adev->mc.gtt_size = amdgpu_ttm_get_gtt_mem_size(adev); - else + /* unless the user had overridden it, set the GART size proportional + * to system memory, but no more than half of the GPU pages. */ + if (amdgpu_gart_size == -1) { + /* a table entry is 8 GPU pages */ + max_gtt_entries = amdgpu_ttm_get_gtt_mem_size(adev) / + (AMDGPU_GPU_PAGE_SIZE / 8); + + if (max_gtt_entries > adev->mc.visible_vram_size / 2) + adev->mc.gtt_size = (adev->mc.visible_vram_size / 2) * + AMDGPU_GPU_PAGE_SIZE / 8; + else + adev->mc.gtt_size = amdgpu_ttm_get_gtt_mem_size(adev); + } else { adev->mc.gtt_size = (uint64_t)amdgpu_gart_size << 20; + } gmc_v7_0_vram_gtt_location(adev, &adev->mc); diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c index a16b220..460100a 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c @@ -416,6 +416,7 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) { u32 tmp; int chansize, numchan; + u64 max_gtt_entries; /* Get VRAM informations */ tmp = RREG32(mmMC_ARB_RAMCFG); @@ -468,13 +469,21 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) if (adev->mc.visible_vram_size > adev->mc.real_vram_size) adev->mc.visible_vram_size = adev->mc.real_vram_size; - /* unless the user had overridden it, set the gart - * size equal to the 1024 or vram, whichever is larger. - */ - if (amdgpu_gart_size == -1) - adev->mc.gtt_size = amdgpu_ttm_get_gtt_mem_size(adev); - else + /* unless the user had overridden it, set the GART size proportional + * to system memory, but no more than half of the GPU pages. */ + if (amdgpu_gart_size == -1) { + /* a table entry is 8 GPU pages */ + max_gtt_entries = amdgpu_ttm_get_gtt_mem_size(adev) / + (AMDGPU_GPU_PAGE_SIZE / 8); + + if (max_gtt_entries > adev->mc.visible_vram_size / 2) + adev->mc.gtt_size = (adev->mc.visible_vram_size / 2) * + AMDGPU_GPU_PAGE_SIZE / 8; + else + adev->mc.gtt_size = amdgpu_ttm_get_gtt_mem_size(adev); + } else { adev->mc.gtt_size = (uint64_t)amdgpu_gart_size << 20; + } gmc_v8_0_vram_gtt_location(adev, &adev->mc);