From patchwork Wed Apr 8 11:13:27 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michel Thierry X-Patchwork-Id: 6178301 Return-Path: X-Original-To: patchwork-intel-gfx@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 79FC29F440 for ; Wed, 8 Apr 2015 11:13:46 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 674CC2034F for ; Wed, 8 Apr 2015 11:13:45 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) by mail.kernel.org (Postfix) with ESMTP id 81D6420376 for ; Wed, 8 Apr 2015 11:13:39 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 17C5F89E75; Wed, 8 Apr 2015 04:13:39 -0700 (PDT) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by gabe.freedesktop.org (Postfix) with ESMTP id DA1696E691 for ; Wed, 8 Apr 2015 04:13:37 -0700 (PDT) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP; 08 Apr 2015 04:13:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.11,544,1422950400"; d="scan'208";a="705395838" Received: from michelth-linux.isw.intel.com ([10.102.226.66]) by fmsmga002.fm.intel.com with ESMTP; 08 Apr 2015 04:13:36 -0700 From: Michel Thierry To: intel-gfx@lists.freedesktop.org Date: Wed, 8 Apr 2015 12:13:27 +0100 Message-Id: <1428491615-27577-6-git-send-email-michel.thierry@intel.com> X-Mailer: git-send-email 2.1.1 In-Reply-To: <1428491615-27577-1-git-send-email-michel.thierry@intel.com> References: <1428491615-27577-1-git-send-email-michel.thierry@intel.com> Subject: [Intel-gfx] [PATCH 05/13] drm/i915/gen8: page directories rework allocation X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Start using gen8_for_each_pdpe macro to allocate the page directories. Similar to PTs, while setting up a page directory, make all entries of the pd point to the scratch pd before mapping (and make all its entries point to the scratch page); this is to be safe in case of out of bound access or proactive prefetch. Systems without LLC require an explicit flush. v2: Rebased after s/free_pt_*/unmap_and_free_pt/ change. v3: Rebased after teardown va range logic was removed. v4: Keep setting up all page directories for systems with less than 4GB of memory. v5: Initialize PDs. (Mika) v6: Initialize also the extra PDs from systems with less than 4GB of memory. (Mika) Cc: Mika Kuoppala Signed-off-by: Ben Widawsky Signed-off-by: Michel Thierry (v2+) --- drivers/gpu/drm/i915/i915_gem_gtt.c | 100 ++++++++++++++++++++++++++++++------ 1 file changed, 85 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c index cd28e343..ac82ccb 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.c +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c @@ -606,6 +606,36 @@ static void gen8_ppgtt_insert_entries(struct i915_address_space *vm, } } +static void __gen8_do_map_pt(gen8_pde_t * const pde, + struct i915_page_table *pt, + struct drm_device *dev) +{ + gen8_pde_t entry = + gen8_pde_encode(dev, pt->daddr, I915_CACHE_LLC); + *pde = entry; +} + +static void gen8_initialize_pd(struct i915_address_space *vm, + struct i915_page_directory *pd) +{ + struct i915_hw_ppgtt *ppgtt = + container_of(vm, struct i915_hw_ppgtt, base); + gen8_pde_t *page_directory; + struct i915_page_table *pt; + int i; + + page_directory = kmap_atomic(pd->page); + pt = ppgtt->scratch_pt; + for (i = 0; i < I915_PDES; i++) + /* Map the PDE to the page table */ + __gen8_do_map_pt(page_directory + i, pt, vm->dev); + + if (!HAS_LLC(vm->dev)) + drm_clflush_virt_range(page_directory, PAGE_SIZE); + + kunmap_atomic(page_directory); +} + static void gen8_free_page_tables(struct i915_page_directory *pd, struct drm_device *dev) { int i; @@ -633,6 +663,8 @@ static void gen8_ppgtt_free(struct i915_hw_ppgtt *ppgtt) gen8_free_page_tables(ppgtt->pdp.page_directory[i], ppgtt->base.dev); unmap_and_free_pd(ppgtt->pdp.page_directory[i]); } + + unmap_and_free_pt(ppgtt->scratch_pt, ppgtt->base.dev); } static void gen8_ppgtt_cleanup(struct i915_address_space *vm) @@ -663,25 +695,55 @@ unwind_out: return -ENOMEM; } -static int gen8_ppgtt_allocate_page_directories(struct i915_hw_ppgtt *ppgtt, - const int max_pdp) +static int gen8_ppgtt_alloc_page_directories(struct i915_page_directory_pointer *pdp, + uint64_t start, + uint64_t length) { - int i; + struct i915_hw_ppgtt *ppgtt = + container_of(pdp, struct i915_hw_ppgtt, pdp); + struct i915_page_directory *unused; + uint64_t temp; + uint32_t pdpe; + + /* FIXME: PPGTT container_of won't work for 64b */ + WARN_ON((start + length) > 0x800000000ULL); + + gen8_for_each_pdpe(unused, pdp, start, length, temp, pdpe) { + WARN_ON(unused); + pdp->page_directory[pdpe] = alloc_pd_single(); + if (IS_ERR(ppgtt->pdp.page_directory[pdpe])) + goto unwind_out; + + gen8_initialize_pd(&ppgtt->base, + ppgtt->pdp.page_directory[pdpe]); + ppgtt->num_pd_pages++; + } - for (i = 0; i < max_pdp; i++) { - ppgtt->pdp.page_directory[i] = alloc_pd_single(); - if (IS_ERR(ppgtt->pdp.page_directory[i])) + /* XXX: Still alloc all page directories in systems with less than + * 4GB of memory. This won't be needed after a subsequent patch. + */ + while (ppgtt->num_pd_pages < GEN8_LEGACY_PDPES) { + ppgtt->pdp.page_directory[ppgtt->num_pd_pages] = alloc_pd_single(); + if (IS_ERR(ppgtt->pdp.page_directory[ppgtt->num_pd_pages])) goto unwind_out; + + gen8_initialize_pd(&ppgtt->base, + ppgtt->pdp.page_directory[ppgtt->num_pd_pages]); + pdpe++; + ppgtt->num_pd_pages++; } - ppgtt->num_pd_pages = max_pdp; BUG_ON(ppgtt->num_pd_pages > GEN8_LEGACY_PDPES); return 0; unwind_out: - while (i--) - unmap_and_free_pd(ppgtt->pdp.page_directory[i]); + while (pdpe--) { + unmap_and_free_pd(ppgtt->pdp.page_directory[pdpe]); + ppgtt->num_pd_pages--; + } + + WARN_ON(ppgtt->num_pd_pages); return -ENOMEM; } @@ -691,7 +753,8 @@ static int gen8_ppgtt_alloc(struct i915_hw_ppgtt *ppgtt, { int ret; - ret = gen8_ppgtt_allocate_page_directories(ppgtt, max_pdp); + ret = gen8_ppgtt_alloc_page_directories(&ppgtt->pdp, ppgtt->base.start, + ppgtt->base.total); if (ret) return ret; @@ -769,6 +832,17 @@ static int gen8_ppgtt_init(struct i915_hw_ppgtt *ppgtt, uint64_t size) if (size % (1<<30)) DRM_INFO("Pages will be wasted unless GTT size (%llu) is divisible by 1GB\n", size); + ppgtt->base.start = 0; + /* This is the area that we advertise as usable for the caller */ + ppgtt->base.total = max_pdp * I915_PDES * GEN8_PTES * PAGE_SIZE; + WARN_ON(ppgtt->base.total == 0); + + ppgtt->scratch_pt = alloc_pt_single(ppgtt->base.dev); + if (IS_ERR(ppgtt->scratch_pt)) + return PTR_ERR(ppgtt->scratch_pt); + + gen8_initialize_pt(&ppgtt->base, ppgtt->scratch_pt); + /* 1. Do all our allocations for page directories and page tables. * We allocate more than was asked so that we can point the unused parts * to valid entries that point to scratch page. Dynamic page tables @@ -794,7 +868,7 @@ static int gen8_ppgtt_init(struct i915_hw_ppgtt *ppgtt, uint64_t size) } /* - * 3. Map all the page directory entires to point to the page tables + * 3. Map all the page directory entries to point to the page tables * we've allocated. * * For now, the PPGTT helper functions all require that the PDEs are @@ -820,10 +894,6 @@ static int gen8_ppgtt_init(struct i915_hw_ppgtt *ppgtt, uint64_t size) ppgtt->base.clear_range = gen8_ppgtt_clear_range; ppgtt->base.insert_entries = gen8_ppgtt_insert_entries; ppgtt->base.cleanup = gen8_ppgtt_cleanup; - ppgtt->base.start = 0; - - /* This is the area that we advertise as usable for the caller */ - ppgtt->base.total = max_pdp * I915_PDES * GEN8_PTES * PAGE_SIZE; /* Set all ptes to a valid scratch page. Also above requested space */ ppgtt->base.clear_range(&ppgtt->base, 0,