From patchwork Fri Dec 23 14:57:56 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 9487503 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 61517601D7 for ; Fri, 23 Dec 2016 14:59:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5345523B3D for ; Fri, 23 Dec 2016 14:59:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 45CF625D9E; Fri, 23 Dec 2016 14:59:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BFBE923B3D for ; Fri, 23 Dec 2016 14:59:41 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A02CC6E071; Fri, 23 Dec 2016 14:59:40 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mail-wj0-x241.google.com (mail-wj0-x241.google.com [IPv6:2a00:1450:400c:c01::241]) by gabe.freedesktop.org (Postfix) with ESMTPS id DF37B6E06A for ; Fri, 23 Dec 2016 14:58:12 +0000 (UTC) Received: by mail-wj0-x241.google.com with SMTP id j10so40601655wjb.3 for ; Fri, 23 Dec 2016 06:58:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=TvT99sYEXJIPAOxxJWaNDIcU9CTYnjFn30gEXdh0Mow=; b=YE+DBU2VtA99kLh3RQ2WsqYO+S94nlNdvwsvuxpyM7aPFm0wkiYsPxIO5cZ72gR7uL m3TdgWG2XqMmQAtBKBIzVM8HUY9xNVQOWfpLprQJ38B2EKspU0BWXO5qFCN2codrmtRh 38uk9w6n+1jvjF5KIfWbI+mz91Hw6tf0ZI0wamFTula+vgoziYPfStXwDx/s9BdwVyDp AvXrVhruEvBSfZWYFM6vpUCdzmoVXIc9DCC0Jyjz2BMB6pcRPvnZh0EFoT0WY97LDLBF +IBa94qa6sy72BrBLYkb8ee4/3AOylX3U+OFsisL4eDEpQrAXRj3FPezpH+Ifob1HTfM HSig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=TvT99sYEXJIPAOxxJWaNDIcU9CTYnjFn30gEXdh0Mow=; b=qEUF3yD2lFqZVAEoey9SKUPJ3Hpgzx6sx8TZNjT7HoweLncsVTwNnU7/uv3qwVyl81 hW1qPZQronaYAQgVRuJ5AhwPKDAB1++ckrK0keTzXu7Pw2XeSzxy3ZgUTvaRwjMSF6Nc z6Cs5D36rgfwERULMoB7CFgKobXxJITXtZb6Yg3bMZ8q+YDBIf+gQlkqiRSX9vAyxSEq 5V0jX7o/LgD0BnoWf53pBX0ObzG/OGdFd9IIUVVS61lnOCfqn0o2EIaoiPc/opnPjmym MLOuNBKhD6SFY1c5TiMgNoJ2uCd27By/ciOh/fQ6bt6IpfFeUo/8jZVbub34vdMZMiya H89A== X-Gm-Message-State: AIkVDXJgG7NPxKmxFoUTOmj/IYfSDUIh1d89gNOJahBnfyQljBNs70HpC1v8v9SkU19DVQ== X-Received: by 10.194.249.72 with SMTP id ys8mr15588741wjc.92.1482505090659; Fri, 23 Dec 2016 06:58:10 -0800 (PST) Received: from haswell.alporthouse.com ([78.156.65.138]) by smtp.gmail.com with ESMTPSA id i15sm40786462wjs.16.2016.12.23.06.58.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Dec 2016 06:58:10 -0800 (PST) From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Fri, 23 Dec 2016 14:57:56 +0000 Message-Id: <20161223145804.6605-2-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20161223145804.6605-1-chris@chris-wilson.co.uk> References: <20161223145804.6605-1-chris@chris-wilson.co.uk> Subject: [Intel-gfx] [PATCH 02/10] drm/i915: Repeat flush of idle work during suspend X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP The idle work handler is self-arming - if it detects that it needs to run again it will queue itself from its work handler. Take greater care when trying to drain the idle work, and double check that it is flushed. The free worker has a similar issue where it is armed by an RCU task which may be running concurrently with us. This should hopefully help with the sporadic WARN_ON(dev_priv->gt.awake) from i915_gem_suspend. v2: Reuse drain_freed_objects. v3: Don't try to flush the freed objects from the shrinker, as it may be underneath the struct_mutex already. v4: do while and comment upon the excess rcu_barrier in drain_freed_objects Signed-off-by: Chris Wilson Reviewed-by: Joonas Lahtinen --- drivers/gpu/drm/i915/i915_debugfs.c | 2 +- drivers/gpu/drm/i915/i915_drv.c | 3 +-- drivers/gpu/drm/i915/i915_drv.h | 13 +++++++++++++ drivers/gpu/drm/i915/i915_gem.c | 10 ++++++++-- 4 files changed, 23 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c index a5552a1ca01f..655b6719c3da 100644 --- a/drivers/gpu/drm/i915/i915_debugfs.c +++ b/drivers/gpu/drm/i915/i915_debugfs.c @@ -4124,7 +4124,7 @@ i915_drop_caches_set(void *data, u64 val) if (val & DROP_FREED) { synchronize_rcu(); - flush_work(&dev_priv->mm.free_work); + i915_gem_drain_freed_objects(dev_priv); } return ret; diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index 6428588518aa..2c020eafada6 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -545,8 +545,7 @@ static void i915_gem_fini(struct drm_i915_private *dev_priv) i915_gem_context_fini(dev_priv); mutex_unlock(&dev_priv->drm.struct_mutex); - rcu_barrier(); - flush_work(&dev_priv->mm.free_work); + i915_gem_drain_freed_objects(dev_priv); WARN_ON(!list_empty(&dev_priv->context_list)); } diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index 1a914095efdd..3794b0933531 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -3208,6 +3208,19 @@ i915_gem_object_create_from_data(struct drm_i915_private *dev_priv, void i915_gem_close_object(struct drm_gem_object *gem, struct drm_file *file); void i915_gem_free_object(struct drm_gem_object *obj); +static inline void i915_gem_drain_freed_objects(struct drm_i915_private *i915) +{ + /* A single pass should suffice to release all the freed objects (along + * most call paths) , but be a little more paranoid in that freeing + * the objects does take a little amount of time, during which the rcu + * callbacks could have added new objects into the freed list, and + * armed the work again. + */ + do { + rcu_barrier(); + } while (flush_work(&i915->mm.free_work)); +} + struct i915_vma * __must_check i915_gem_object_ggtt_pin(struct drm_i915_gem_object *obj, const struct i915_ggtt_view *view, diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 5a08e8c187e8..bdb37bd3813b 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -4263,8 +4263,14 @@ int i915_gem_suspend(struct drm_i915_private *dev_priv) cancel_delayed_work_sync(&dev_priv->gpu_error.hangcheck_work); cancel_delayed_work_sync(&dev_priv->gt.retire_work); - flush_delayed_work(&dev_priv->gt.idle_work); - flush_work(&dev_priv->mm.free_work); + + /* As the idle_work is rearming if it detects a race, play safe and + * repeat the flush until it is definitely idle. + */ + while (flush_delayed_work(&dev_priv->gt.idle_work)) + ; + + i915_gem_drain_freed_objects(dev_priv); /* Assert that we sucessfully flushed all the work and * reset the GPU back to its idle, low power state.