From patchwork Wed Jul 4 19:13:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 10507545 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 49627601D7 for ; Wed, 4 Jul 2018 19:13:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 57B0328D70 for ; Wed, 4 Jul 2018 19:13:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 49DF528D7E; Wed, 4 Jul 2018 19:13:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B481428D70 for ; Wed, 4 Jul 2018 19:13:52 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 10F4B6EC1B; Wed, 4 Jul 2018 19:13:52 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1A5F96EC1B for ; Wed, 4 Jul 2018 19:13:49 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 12254083-1500050 for multiple; Wed, 04 Jul 2018 20:13:41 +0100 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Wed, 4 Jul 2018 20:13:42 +0100 Message-Id: <20180704191342.18067-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180702114452.21098-1-chris@chris-wilson.co.uk> References: <20180702114452.21098-1-chris@chris-wilson.co.uk> Subject: [Intel-gfx] [PATCH] drm/i915: Show who pinned the pages when a leak is hit X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP Currently, we emit a warning when freeing an object if we do so with the pages still pinned (presumably as they are still in use somewhere). This only tells us that there is a problem, but doesn't tell us anything about the object or who might be pinning them and so provides no clue as to track down the problem. Let's try tracking everyone who pins the pages to see if we strike it lucky and catch the culprit red handed. v2: Suppress oom warnings, they will happen given sufficient abuse. References: https://bugs.freedesktop.org/show_bug.cgi?id=105360 Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/Kconfig.debug | 1 + drivers/gpu/drm/i915/i915_drv.h | 2 + drivers/gpu/drm/i915/i915_gem.c | 6 +- drivers/gpu/drm/i915/i915_gem_object.c | 109 +++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_gem_object.h | 34 ++++++++ 5 files changed, 151 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/Kconfig.debug b/drivers/gpu/drm/i915/Kconfig.debug index 9de8b1c51a5c..84052becb279 100644 --- a/drivers/gpu/drm/i915/Kconfig.debug +++ b/drivers/gpu/drm/i915/Kconfig.debug @@ -43,6 +43,7 @@ config DRM_I915_DEBUG_GEM bool "Insert extra checks into the GEM internals" default n depends on DRM_I915_WERROR + select STACKDEPOT help Enable extra sanity checks (including BUGs) along the GEM driver paths that may slow the system down and if hit hang the machine. diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index e5a0a65ec2e9..50dfc114ba01 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -2990,6 +2990,8 @@ i915_gem_object_pin_pages(struct drm_i915_gem_object *obj) { might_lock(&obj->mm.lock); + track_i915_gem_object_pin_pages(obj); + if (atomic_inc_not_zero(&obj->mm.pages_pin_count)) return 0; diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 048b722cf27c..50e5d35a5d9d 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -2465,6 +2465,7 @@ void __i915_gem_object_put_pages(struct drm_i915_gem_object *obj, if (!IS_ERR(pages)) obj->ops->put_pages(obj, pages); + untrack_i915_gem_object_pin_pages(obj); unlock: mutex_unlock(&obj->mm.lock); } @@ -4681,6 +4682,7 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj, const struct drm_i915_gem_object_ops *ops) { mutex_init(&obj->mm.lock); + spin_lock_init(&obj->mm.debug_lock); INIT_LIST_HEAD(&obj->vma_list); INIT_LIST_HEAD(&obj->lut_list); @@ -4876,8 +4878,10 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915, if (obj->ops->release) obj->ops->release(obj); - if (WARN_ON(i915_gem_object_has_pinned_pages(obj))) + if (WARN_ON(i915_gem_object_has_pinned_pages(obj))) { + show_i915_gem_object_pin_pages(obj); atomic_set(&obj->mm.pages_pin_count, 0); + } __i915_gem_object_put_pages(obj, I915_MM_NORMAL); GEM_BUG_ON(i915_gem_object_has_pages(obj)); diff --git a/drivers/gpu/drm/i915/i915_gem_object.c b/drivers/gpu/drm/i915/i915_gem_object.c index aab8cdd80e6d..39b1e84bf494 100644 --- a/drivers/gpu/drm/i915/i915_gem_object.c +++ b/drivers/gpu/drm/i915/i915_gem_object.c @@ -25,6 +25,115 @@ #include "i915_drv.h" #include "i915_gem_object.h" +#if IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) + +#include + +#define STACKDEPTH 12 + +void track_i915_gem_object_pin_pages(struct drm_i915_gem_object *obj) +{ + unsigned long entries[STACKDEPTH]; + struct stack_trace trace = { + .entries = entries, + .max_entries = ARRAY_SIZE(entries), + .skip = 1 + }; + unsigned long flags; + depot_stack_handle_t stack, *stacks; + + save_stack_trace(&trace); + if (trace.nr_entries && + trace.entries[trace.nr_entries - 1] == ULONG_MAX) + trace.nr_entries--; + + stack = depot_save_stack(&trace, GFP_KERNEL | __GFP_NOWARN); + if (!stack) + return; + + spin_lock_irqsave(&obj->mm.debug_lock, flags); + stacks = krealloc(obj->mm.debug_owners, + (obj->mm.debug_count + 1) * sizeof(*stacks), + GFP_NOWAIT | __GFP_NOWARN); + if (stacks) { + stacks[obj->mm.debug_count++] = stack; + obj->mm.debug_owners = stacks; + } + spin_unlock_irqrestore(&obj->mm.debug_lock, flags); +} + +void untrack_i915_gem_object_pin_pages(struct drm_i915_gem_object *obj) +{ + depot_stack_handle_t *stacks; + unsigned long flags; + + spin_lock_irqsave(&obj->mm.debug_lock, flags); + stacks = fetch_and_zero(&obj->mm.debug_owners); + obj->mm.debug_count = 0; + spin_unlock_irqrestore(&obj->mm.debug_lock, flags); + + kfree(stacks); +} + +static int cmphandle(const void *_a, const void *_b) +{ + const depot_stack_handle_t * const a = _a, * const b = _b; + + if (*a < *b) + return -1; + else if (*a > *b) + return 1; + else + return 0; +} + +void show_i915_gem_object_pin_pages(struct drm_i915_gem_object *obj) +{ + unsigned long entries[STACKDEPTH]; + depot_stack_handle_t *stacks; + unsigned long flags, count, i; + char *buf; + + spin_lock_irqsave(&obj->mm.debug_lock, flags); + stacks = fetch_and_zero(&obj->mm.debug_owners); + count = fetch_and_zero(&obj->mm.debug_count); + spin_unlock_irqrestore(&obj->mm.debug_lock, flags); + if (!count) + return; + + DRM_DEBUG_DRIVER("obj %p leaked pages, pinned %lu\n", obj, count); + + buf = kmalloc(PAGE_SIZE, GFP_KERNEL); + if (!buf) + goto out_stacks; + + sort(stacks, count, sizeof(*stacks), cmphandle, NULL); + + for (i = 0; i < count; i++) { + struct stack_trace trace = { + .entries = entries, + .max_entries = ARRAY_SIZE(entries), + }; + depot_stack_handle_t stack = stacks[i]; + unsigned long rep; + + rep = 1; + while (i + 1 < count && stacks[i + 1] == stack) + rep++, i++; + + depot_fetch_stack(stack, &trace); + snprint_stack_trace(buf, PAGE_SIZE, &trace, 0); + DRM_DEBUG_DRIVER("obj %p pages pinned x%lu at\n%s", + obj, rep, buf); + } + + kfree(buf); +out_stacks: + kfree(stacks); +} + +#endif + /** * Mark up the object's coherency levels for a given cache_level * @obj: #drm_i915_gem_object diff --git a/drivers/gpu/drm/i915/i915_gem_object.h b/drivers/gpu/drm/i915/i915_gem_object.h index c3c6f2e588fb..ebc2ee17426f 100644 --- a/drivers/gpu/drm/i915/i915_gem_object.h +++ b/drivers/gpu/drm/i915/i915_gem_object.h @@ -26,6 +26,9 @@ #define __I915_GEM_OBJECT_H__ #include +#if IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) +#include +#endif #include #include @@ -245,6 +248,12 @@ struct drm_i915_gem_object { * swizzling. */ bool quirked:1; + +#if IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) + spinlock_t debug_lock; + unsigned long debug_count; + depot_stack_handle_t *debug_owners; +#endif } mm; /** Breadcrumb of last rendering to the buffer. @@ -451,6 +460,31 @@ i915_gem_object_get_tile_row_size(struct drm_i915_gem_object *obj) int i915_gem_object_set_tiling(struct drm_i915_gem_object *obj, unsigned int tiling, unsigned int stride); +#if IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) + +void track_i915_gem_object_pin_pages(struct drm_i915_gem_object *obj); +void untrack_i915_gem_object_pin_pages(struct drm_i915_gem_object *obj); +void show_i915_gem_object_pin_pages(struct drm_i915_gem_object *obj); + +#else + +static inline void +track_i915_gem_object_pin_pages(struct drm_i915_gem_object *obj) +{ +} + +static inline void +untrack_i915_gem_object_pin_pages(struct drm_i915_gem_object *obj) +{ +} + +static inline void +show_i915_gem_object_pin_pages(struct drm_i915_gem_object *obj) +{ +} + +#endif + static inline struct intel_engine_cs * i915_gem_object_last_write_engine(struct drm_i915_gem_object *obj) {