From patchwork Fri Jun 3 16:37:24 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 9153601 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8412B6074E for ; Fri, 3 Jun 2016 16:38:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 751D526C9B for ; Fri, 3 Jun 2016 16:38:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A3F628309; Fri, 3 Jun 2016 16:38:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC99D26C9B for ; Fri, 3 Jun 2016 16:38:53 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4B6DF6EE58; Fri, 3 Jun 2016 16:38:44 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) by gabe.freedesktop.org (Postfix) with ESMTPS id A64056EE31 for ; Fri, 3 Jun 2016 16:38:30 +0000 (UTC) Received: by mail-wm0-x241.google.com with SMTP id e3so612273wme.2 for ; Fri, 03 Jun 2016 09:38:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=9uM3yCVW3R0IRvmTm1JgPKtSK7P1lPaF04OYpNT5i7M=; b=OgY23E8MmZM04pI3b5ScoJJERJ3/FvOubIjV4i31WtgokJoTpPAmf8BGTXMiUpJ2am zyBQczgC+bMvoqVP1aDzJc9dX9WR9JIYuzhXaNPK3hXb6KREmm9ArRu7oJxAzVJ1ZMSW ASzoG9L31C5X5kAsqeJ8SElqSEhxhNQQxstDQd9jUct+hZGmIJFMexkRH3h65r6ral2x JXdVZbG5YaOMYjIF+0dRrd/oukSTRd45kGguim8yswkSWrxo0fVISKkUpZrLejrO/fPF efjv1W6D/YlRRDGoUpMsgB6lENH6VosFiDjgQP+xAUhb20sSArd1OdnYUrX/o5Ne+/zA ztDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=9uM3yCVW3R0IRvmTm1JgPKtSK7P1lPaF04OYpNT5i7M=; b=DwuFKN4psA6zPHzKyLQMm15gQSMRpcaGPawoeMwT+wVnJrKsdl9Vp2lI2M5CFhFbTB cafRxm2gZ9FeeRjLcyCq6DqOh9VU6Rsm9/7zahb0AGSrivIJGQjpmUKzmGRqcGCgF57P CqPwSmoMmj0b+mEhYm92MqPdGpaoZj0abh7iJqqNEmzmu2kuuIAw1dhrssnCPpQmrqcR D0nSoH58PSMGaNVlyWr0YufK6R27H4XWeH1Pq6+tc77xbK+GPQAO5dEsmfp5I0vF9wGs N792jiAEhyUg16WGMZPKM5rYUDMdFcplTqAATBzzriFWHKlrzsQZgBJqerEXghy0YxUr P31g== X-Gm-Message-State: ALyK8tIGw101GjPLiFsuxRhy05AXqvuexKBjc/SsDPut/c/fY0v7EByQlPbRHnBqh+fjlw== X-Received: by 10.28.35.67 with SMTP id j64mr379173wmj.77.1464971908663; Fri, 03 Jun 2016 09:38:28 -0700 (PDT) Received: from haswell.alporthouse.com ([78.156.65.138]) by smtp.gmail.com with ESMTPSA id l9sm6565607wjm.0.2016.06.03.09.38.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jun 2016 09:38:28 -0700 (PDT) From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Fri, 3 Jun 2016 17:37:24 +0100 Message-Id: <1464971847-15809-60-git-send-email-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1464971847-15809-1-git-send-email-chris@chris-wilson.co.uk> References: <1464971847-15809-1-git-send-email-chris@chris-wilson.co.uk> Subject: [Intel-gfx] [PATCH 59/62] drm/i915: Track active vma requests X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP Hook the vma itself into the i915_gem_request_retire() so that we can accurately track when a solitary vma is inactive (as opposed to having to wait for the entire object to be idle). This improves the interaction when using multiple contexts (with full-ppgtt) and eliminates some frequent list walking when retiring objects after a completed request. A side-effect is that we get an active vma reference for free. The consequence of this is shown in the next patch... Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/i915_debugfs.c | 2 +- drivers/gpu/drm/i915/i915_gem.c | 20 +++++++------------- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 10 +++++++++- drivers/gpu/drm/i915/i915_gem_gtt.c | 20 ++++++++++++++++++++ drivers/gpu/drm/i915/i915_gem_gtt.h | 26 ++++++++++++++++++++++++++ 5 files changed, 63 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c index 51f84dd37675..99857ee0bb8b 100644 --- a/drivers/gpu/drm/i915/i915_debugfs.c +++ b/drivers/gpu/drm/i915/i915_debugfs.c @@ -367,7 +367,7 @@ static int per_file_stats(int id, void *ptr, void *data) continue; } - if (obj->active) /* XXX per-vma statistic */ + if (i915_vma_is_active(vma)) stats->active += vma->node.size; else stats->inactive += vma->node.size; diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index a39d767d8137..ef68a9183d7d 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -2100,7 +2100,6 @@ i915_gem_object_retire__read(struct i915_gem_active *active, int ring = request->engine->id; struct drm_i915_gem_object *obj = container_of(active, struct drm_i915_gem_object, last_read[ring]); - struct i915_vma *vma; GEM_BUG_ON((obj->active & (1 << ring)) == 0); @@ -2112,12 +2111,9 @@ i915_gem_object_retire__read(struct i915_gem_active *active, * so that we don't steal from recently used but inactive objects * (unless we are forced to ofc!) */ - list_move_tail(&obj->global_list, &request->i915->mm.bound_list); - - list_for_each_entry(vma, &obj->vma_list, obj_link) { - if (!list_empty(&vma->vm_link)) - list_move_tail(&vma->vm_link, &vma->vm->inactive_list); - } + if (obj->bind_count) + list_move_tail(&obj->global_list, + &request->i915->mm.bound_list); i915_gem_object_put(obj); } @@ -2915,9 +2911,6 @@ i915_gem_object_flush_cpu_write_domain(struct drm_i915_gem_object *obj) int i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write) { - struct drm_device *dev = obj->base.dev; - struct drm_i915_private *dev_priv = to_i915(dev); - struct i915_ggtt *ggtt = &dev_priv->ggtt; uint32_t old_write_domain, old_read_domains; struct i915_vma *vma; int ret; @@ -2970,9 +2963,10 @@ i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write) /* And bump the LRU for this access */ vma = i915_gem_obj_to_ggtt(obj); - if (vma && drm_mm_node_allocated(&vma->node) && !obj->active) - list_move_tail(&vma->vm_link, - &ggtt->base.inactive_list); + if (vma && + drm_mm_node_allocated(&vma->node) && + !i915_vma_is_active(vma)) + list_move_tail(&vma->vm_link, &vma->vm->inactive_list); return 0; } diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c index e099080b3b5b..7b381358512e 100644 --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c @@ -1121,7 +1121,13 @@ void i915_vma_move_to_active(struct i915_vma *vma, obj->dirty = 1; /* be paranoid */ - /* Add a reference if we're newly entering the active list. */ + /* Add a reference if we're newly entering the active list. + * The order in which we add operations to the retirement queue is + * vital here: mark_active adds to the start of the callback list, + * such that subsequent callbacks are called first. Therefore we + * add the active reference first and queue for it to be dropped + * *last*. + */ if (obj->active == 0) i915_gem_object_get(obj); obj->active |= 1 << idx; @@ -1145,6 +1151,8 @@ void i915_vma_move_to_active(struct i915_vma *vma, } } + i915_vma_set_active(vma, idx); + i915_gem_active_set(&vma->last_read[idx], req); list_move_tail(&vma->vm_link, &vma->vm->active_list); } diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c index 57fc84b9b633..4d3179e15b94 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.c +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c @@ -3281,12 +3281,30 @@ void i915_gem_restore_gtt_mappings(struct drm_device *dev) i915_ggtt_flush(dev_priv); } +static void +i915_vma_retire(struct i915_gem_active *active, + struct drm_i915_gem_request *rq) +{ + const unsigned idx = rq->engine->id; + struct i915_vma *vma = + container_of(active, struct i915_vma, last_read[idx]); + + GEM_BUG_ON(!i915_vma_has_active_engine(vma, idx)); + + i915_vma_unset_active(vma, idx); + if (i915_vma_is_active(vma)) + return; + + list_move_tail(&vma->vm_link, &vma->vm->inactive_list); +} + static struct i915_vma * __i915_gem_vma_create(struct drm_i915_gem_object *obj, struct i915_address_space *vm, const struct i915_ggtt_view *ggtt_view) { struct i915_vma *vma; + int i; if (WARN_ON(i915_is_ggtt(vm) != !!ggtt_view)) return ERR_PTR(-EINVAL); @@ -3298,6 +3316,8 @@ __i915_gem_vma_create(struct drm_i915_gem_object *obj, INIT_LIST_HEAD(&vma->vm_link); INIT_LIST_HEAD(&vma->obj_link); INIT_LIST_HEAD(&vma->exec_list); + for (i = 0; i < ARRAY_SIZE(vma->last_read); i++) + init_request_active(&vma->last_read[i], i915_vma_retire); vma->vm = vm; vma->obj = obj; vma->is_ggtt = i915_is_ggtt(vm); diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h index 4cabf891fd1d..d86b3e4777a7 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.h +++ b/drivers/gpu/drm/i915/i915_gem_gtt.h @@ -36,6 +36,8 @@ #include +#include "i915_gem_request.h" + struct drm_i915_file_private; typedef uint32_t gen6_pte_t; @@ -179,10 +181,13 @@ struct i915_vma { struct i915_address_space *vm; void __iomem *iomap; + struct i915_gem_active last_read[I915_NUM_ENGINES]; + /** Flags and address space this VMA is bound to */ #define GLOBAL_BIND (1<<0) #define LOCAL_BIND (1<<1) unsigned int bound : 4; + unsigned int active : I915_NUM_ENGINES; bool is_ggtt : 1; /** @@ -222,6 +227,27 @@ struct i915_vma { #define DRM_I915_GEM_OBJECT_MAX_PIN_COUNT 0xf }; +static inline bool i915_vma_is_active(const struct i915_vma *vma) +{ + return vma->active; +} + +static inline void i915_vma_set_active(struct i915_vma *vma, unsigned engine) +{ + vma->active |= 1 << engine; +} + +static inline void i915_vma_unset_active(struct i915_vma *vma, unsigned engine) +{ + vma->active &= ~(1 << engine); +} + +static inline bool i915_vma_has_active_engine(const struct i915_vma *vma, + unsigned engine) +{ + return vma->active & (1 << engine); +} + struct i915_page_dma { struct page *page; union {