From patchwork Tue Feb 14 11:04:49 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 9571745 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A87DC60578 for ; Tue, 14 Feb 2017 11:05:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 80A33280CF for ; Tue, 14 Feb 2017 11:05:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7577E28394; Tue, 14 Feb 2017 11:05:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D6A94280CF for ; Tue, 14 Feb 2017 11:05:41 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D414C6E693; Tue, 14 Feb 2017 11:05:20 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mail-wr0-x242.google.com (mail-wr0-x242.google.com [IPv6:2a00:1450:400c:c0c::242]) by gabe.freedesktop.org (Postfix) with ESMTPS id EE3FD6E687 for ; Tue, 14 Feb 2017 11:05:16 +0000 (UTC) Received: by mail-wr0-x242.google.com with SMTP id c4so1637562wrd.1 for ; Tue, 14 Feb 2017 03:05:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=S+My4P3Rb+tXKOyQ8VoukjiIE3hF9Q+zfgVEcq6EoyY=; b=CD/Qzzf7M7XCjylgWj5rZNye/3o/ST4ojgQjivTjsoYvwXDqXUNZDCWpDNOzyFvJAc bbSwTn7NWpZFsbrtEdDJdl+88ZoiybqLKYCt+jP4GLtLjQWgU6YWZWO2xcDNG7N8YfrE 4UXMBZg1Jn3jLvf5gLh2d71gs0ebesZEKHAGCesZh815sW2sHr0gUTQaret3vXdmMGnT O68kcZ7L31VEdBgqYojJnewcPfT8y4vgXhIkjpH/vY6d08FxfjhvcB/MMVPPg+8qE2Uj LPGvf4wiXNVcst3CAPdHOxBtlu2xENgpbhZ5E38DLF06N8hv1bu6geAuXCUFx0r622I+ 1yOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=S+My4P3Rb+tXKOyQ8VoukjiIE3hF9Q+zfgVEcq6EoyY=; b=TEjXrW/lcOr0YbhrFHs640HC2zgJb4LZQle9d3cwIJhi8QeN42I1M/HS2/1x8KEqFp 2AFNaJ/r5W8M4kaXLC/28DozAVmAaJs7HAVFB+TxWR+QUpYdIsq/DRtuKdqmIYEw/gTJ h0nB04xa/ira07S8Du3mDsgcNQ2kGigD9+so5+JjOyMWMjt6X27jNyRch0xA7AkSMxZp DNFQuGMHP7J6HMNaWPChdifptW8GTt9gCD9fpDct5OfjNQ4bOclfSUaYqStTjlqN4sx9 yctDkaXslHtli7QkZlCeh3LVHw+hRxlWD5xuTLYEpsk7AnAI900Z1mn6nJegwnGBN0qV tZqA== X-Gm-Message-State: AMke39lRJX5T3fMfFPcVWS+xMafkDKRrcbhop+B9cUzVWu6T9wX+RMWIzGufkGjMoDj7cQ== X-Received: by 10.223.174.199 with SMTP id y65mr27662165wrc.19.1487070315555; Tue, 14 Feb 2017 03:05:15 -0800 (PST) Received: from haswell.alporthouse.com ([78.156.65.138]) by smtp.gmail.com with ESMTPSA id s26sm323512wra.26.2017.02.14.03.05.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Feb 2017 03:05:15 -0800 (PST) From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Tue, 14 Feb 2017 11:04:49 +0000 Message-Id: <20170214110457.9316-16-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170214110457.9316-1-chris@chris-wilson.co.uk> References: <20170214110457.9316-1-chris@chris-wilson.co.uk> Cc: matthew.auld@intel.com, mika.kuoppala@intel.com Subject: [Intel-gfx] [PATCH v3 15/23] drm/i915: Remove bitmap tracking for used-pml4 X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP We only operate on known extents (both for alloc/clear) and so we can use both the knowledge of the bind/unbind range along with the knowledge of the existing pagetable to avoid having to allocate temporary and auxiliary bitmaps. Signed-off-by: Chris Wilson Reviewed-by: Matthew Auld --- drivers/gpu/drm/i915/i915_gem_gtt.c | 132 +++++++++++------------------------- drivers/gpu/drm/i915/i915_gem_gtt.h | 2 - 2 files changed, 38 insertions(+), 96 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c index 885f0ee013e5..1cb3f486a170 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.c +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c @@ -807,7 +807,6 @@ static void gen8_ppgtt_clear_4lvl(struct i915_address_space *vm, continue; gen8_ppgtt_set_pml4e(pml4, vm->scratch_pdp, pml4e); - __clear_bit(pml4e, pml4->used_pml4es); free_pdp(vm, pdp); } @@ -1026,8 +1025,8 @@ static void gen8_ppgtt_cleanup_4lvl(struct i915_hw_ppgtt *ppgtt) { int i; - for_each_set_bit(i, ppgtt->pml4.used_pml4es, GEN8_PML4ES_PER_PML4) { - if (WARN_ON(!ppgtt->pml4.pdps[i])) + for (i = 0; i < GEN8_PML4ES_PER_PML4; i++) { + if (ppgtt->pml4.pdps[i] == ppgtt->base.scratch_pdp) continue; gen8_ppgtt_cleanup_3lvl(&ppgtt->base, ppgtt->pml4.pdps[i]); @@ -1081,53 +1080,9 @@ static int gen8_ppgtt_alloc_pd(struct i915_address_space *vm, return -ENOMEM; } -/** - * gen8_ppgtt_alloc_page_dirpointers() - Allocate pdps for VA range. - * @vm: Master vm structure. - * @pml4: Page map level 4 for this address range. - * @start: Starting virtual address to begin allocations. - * @length: Size of the allocations. - * @new_pdps: Bitmap set by function with new allocations. Likely used by the - * caller to free on error. - * - * Allocate the required number of page directory pointers. Extremely similar to - * gen8_ppgtt_alloc_page_directories() and gen8_ppgtt_alloc_pd(). - * The main difference is here we are limited by the pml4 boundary (instead of - * the page directory pointer). - * - * Return: 0 if success; negative error code otherwise. - */ -static int -gen8_ppgtt_alloc_page_dirpointers(struct i915_address_space *vm, - struct i915_pml4 *pml4, - uint64_t start, - uint64_t length) -{ - struct i915_page_directory_pointer *pdp; - uint32_t pml4e; - - gen8_for_each_pml4e(pdp, pml4, start, length, pml4e) { - if (!test_bit(pml4e, pml4->used_pml4es)) { - pdp = alloc_pdp(vm); - if (IS_ERR(pdp)) - return PTR_ERR(pdp); - - gen8_initialize_pdp(vm, pdp); - pml4->pdps[pml4e] = pdp; - trace_i915_page_directory_pointer_entry_alloc(vm, - pml4e, - start, - GEN8_PML4E_SHIFT); - } - } - - return 0; -} - -static int gen8_alloc_va_range_3lvl(struct i915_address_space *vm, - struct i915_page_directory_pointer *pdp, - uint64_t start, - uint64_t length) +static int gen8_ppgtt_alloc_pdp(struct i915_address_space *vm, + struct i915_page_directory_pointer *pdp, + u64 start, u64 length) { struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm); struct i915_page_directory *pd; @@ -1163,58 +1118,46 @@ static int gen8_alloc_va_range_3lvl(struct i915_address_space *vm, return -ENOMEM; } -static int gen8_alloc_va_range_4lvl(struct i915_address_space *vm, - struct i915_pml4 *pml4, - uint64_t start, - uint64_t length) +static int gen8_ppgtt_alloc_3lvl(struct i915_address_space *vm, + u64 start, u64 length) { - DECLARE_BITMAP(new_pdps, GEN8_PML4ES_PER_PML4); - struct i915_page_directory_pointer *pdp; - uint64_t pml4e; - int ret = 0; - - /* Do the pml4 allocations first, so we don't need to track the newly - * allocated tables below the pdp */ - bitmap_zero(new_pdps, GEN8_PML4ES_PER_PML4); + return gen8_ppgtt_alloc_pdp(vm, + &i915_vm_to_ppgtt(vm)->pdp, start, length); +} - /* The pagedirectory and pagetable allocations are done in the shared 3 - * and 4 level code. Just allocate the pdps. - */ - ret = gen8_ppgtt_alloc_page_dirpointers(vm, pml4, start, length); - if (ret) - return ret; +static int gen8_ppgtt_alloc_4lvl(struct i915_address_space *vm, + u64 start, u64 length) +{ + struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm); + struct i915_pml4 *pml4 = &ppgtt->pml4; + struct i915_page_directory_pointer *pdp; + u64 from = start; + u32 pml4e; + int ret; gen8_for_each_pml4e(pdp, pml4, start, length, pml4e) { - WARN_ON(!pdp); + if (pml4->pdps[pml4e] == vm->scratch_pdp) { + pdp = alloc_pdp(vm); + if (IS_ERR(pdp)) + goto unwind; - ret = gen8_alloc_va_range_3lvl(vm, pdp, start, length); - if (ret) - goto err_out; + gen8_initialize_pdp(vm, pdp); + gen8_ppgtt_set_pml4e(pml4, pdp, pml4e); + } - gen8_ppgtt_set_pml4e(pml4, pdp, pml4e); + ret = gen8_ppgtt_alloc_pdp(vm, pdp, start, length); + if (unlikely(ret)) { + gen8_ppgtt_set_pml4e(pml4, vm->scratch_pdp, pml4e); + free_pdp(vm, pdp); + goto unwind; + } } - bitmap_or(pml4->used_pml4es, new_pdps, pml4->used_pml4es, - GEN8_PML4ES_PER_PML4); - return 0; -err_out: - for_each_set_bit(pml4e, new_pdps, GEN8_PML4ES_PER_PML4) - gen8_ppgtt_cleanup_3lvl(vm, pml4->pdps[pml4e]); - - return ret; -} - -static int gen8_alloc_va_range(struct i915_address_space *vm, - uint64_t start, uint64_t length) -{ - struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm); - - if (USES_FULL_48BIT_PPGTT(vm->i915)) - return gen8_alloc_va_range_4lvl(vm, &ppgtt->pml4, start, length); - else - return gen8_alloc_va_range_3lvl(vm, &ppgtt->pdp, start, length); +unwind: + gen8_ppgtt_clear_4lvl(vm, from, start - from); + return -ENOMEM; } static void gen8_dump_pdp(struct i915_hw_ppgtt *ppgtt, @@ -1288,7 +1231,7 @@ static void gen8_dump_ppgtt(struct i915_hw_ppgtt *ppgtt, struct seq_file *m) struct i915_page_directory_pointer *pdp; gen8_for_each_pml4e(pdp, pml4, start, length, pml4e) { - if (!test_bit(pml4e, pml4->used_pml4es)) + if (pml4->pdps[pml4e] == ppgtt->base.scratch_pdp) continue; seq_printf(m, " PML4E #%llu\n", pml4e); @@ -1347,7 +1290,6 @@ static int gen8_ppgtt_init(struct i915_hw_ppgtt *ppgtt) ppgtt->base.start = 0; ppgtt->base.cleanup = gen8_ppgtt_cleanup; - ppgtt->base.allocate_va_range = gen8_alloc_va_range; ppgtt->base.unbind_vma = ppgtt_unbind_vma; ppgtt->base.bind_vma = ppgtt_bind_vma; ppgtt->debug_dump = gen8_dump_ppgtt; @@ -1368,6 +1310,7 @@ static int gen8_ppgtt_init(struct i915_hw_ppgtt *ppgtt) ppgtt->base.total = 1ULL << 48; ppgtt->switch_mm = gen8_48b_mm_switch; + ppgtt->base.allocate_va_range = gen8_ppgtt_alloc_4lvl; ppgtt->base.insert_entries = gen8_ppgtt_insert_4lvl; ppgtt->base.clear_range = gen8_ppgtt_clear_4lvl; } else { @@ -1389,6 +1332,7 @@ static int gen8_ppgtt_init(struct i915_hw_ppgtt *ppgtt) } } + ppgtt->base.allocate_va_range = gen8_ppgtt_alloc_3lvl; ppgtt->base.insert_entries = gen8_ppgtt_insert_3lvl; ppgtt->base.clear_range = gen8_ppgtt_clear_3lvl; } diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h index 3628c7c564ae..933888725368 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.h +++ b/drivers/gpu/drm/i915/i915_gem_gtt.h @@ -238,8 +238,6 @@ struct i915_page_directory_pointer { struct i915_pml4 { struct i915_page_dma base; - - DECLARE_BITMAP(used_pml4es, GEN8_PML4ES_PER_PML4); struct i915_page_directory_pointer *pdps[GEN8_PML4ES_PER_PML4]; };