From patchwork Wed Oct 12 11:48:26 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 9372407 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6404560772 for ; Wed, 12 Oct 2016 11:48:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 536D12949C for ; Wed, 12 Oct 2016 11:48:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 47A882950C; Wed, 12 Oct 2016 11:48:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 641802949C for ; Wed, 12 Oct 2016 11:48:35 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DA68E6E824; Wed, 12 Oct 2016 11:48:34 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mail-lf0-x243.google.com (mail-lf0-x243.google.com [IPv6:2a00:1450:4010:c07::243]) by gabe.freedesktop.org (Postfix) with ESMTPS id A7AFB6E398 for ; Wed, 12 Oct 2016 11:48:32 +0000 (UTC) Received: by mail-lf0-x243.google.com with SMTP id x79so6707614lff.2 for ; Wed, 12 Oct 2016 04:48:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=bYnOdnaHIAjMir5jOfVltvZ6QvZDD2Ipb6n9wDcQuHQ=; b=V0FxMpjOxis6gPiAIr7QVIWF1bOMtCGiyjZweFOPNp/wlUYNA4dh+y6FwQYxRkHebO B0w3BRGOo9GmM2p4MYHXgfdN6ciyQd73W8AZgG2o8kXmYYd22Xmm3F2kVmLpZr+GlXWd peO/vkBagvaqVGaKXAldN88+xYl2MlMA0OfWX9sLXfflOztDlD+4SrV+4tWL0KL0/hTw rr4kod++XHz36acvhrHgrMQ54krdgZlumS09fnBPVLWK13hv8XFXoeW3Bhfz1f/vsDhH SZ7hyP/RpF0P4kOEDDqIXUkyVqh+OTLRIWlfh9IQx9LPwZNL+fiApTQtjlE+inrav9BM BT5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:subject:date:message-id :mime-version:content-transfer-encoding; bh=bYnOdnaHIAjMir5jOfVltvZ6QvZDD2Ipb6n9wDcQuHQ=; b=XJV0m4XiNxz7JqO8vooGobEEdc6gy09cJcG6qUzEaFssXG/F8JB9d/u3CJ4J98Gd5m 0u1T7U/eefHW/Hr8wPmcm4nygFC08phsaHDwg+gxwmnEASAbyxZ1Ab6bO68dhEj0UzPJ wFfMaq3K+C8iQRIF4OVKxjaH33yV25vYVSMjtgadiQHSELC8pBP+SgbMWCLHrjtMHYP6 7X7fr72cpUjs9tu1w0r2NMLNNNqjl5dk4ZlKuWPJyR0msFpq2jsebJDn3oPZliUC8R9Y urkVkr/zR2IavVqPLRWXwCz8SEHncwKrOUHcHHWpfDkfBO25+dqI0kiUpy1l6Im44Vs8 EePg== X-Gm-Message-State: AA6/9RnADtPzaBn1zF7G4CDviRdpUwhTVJYJBA2lZzzKHrb1mo39+hmZvcuRDoHvFOkyAw== X-Received: by 10.194.78.130 with SMTP id b2mr835880wjx.29.1476272910754; Wed, 12 Oct 2016 04:48:30 -0700 (PDT) Received: from haswell.alporthouse.com ([78.156.65.138]) by smtp.gmail.com with ESMTPSA id ma5sm12179819wjb.47.2016.10.12.04.48.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 12 Oct 2016 04:48:30 -0700 (PDT) From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Wed, 12 Oct 2016 12:48:26 +0100 Message-Id: <20161012114827.17031-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.9.3 MIME-Version: 1.0 Subject: [Intel-gfx] [CI 1/2] drm/i915: Use fence_write() from rpm resume X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP During rpm resume we restore the fences, but we do not have the protection of struct_mutex. This rules out updating the activity tracking on the fences, and requires us to rely on the rpm as the serialisation barrier instead. [ 350.298052] [drm:intel_runtime_resume [i915]] Resuming device [ 350.308606] [ 350.310520] =============================== [ 350.315560] [ INFO: suspicious RCU usage. ] [ 350.320554] 4.8.0-rc8-bsw-rapl+ #3133 Tainted: G U W [ 350.327208] ------------------------------- [ 350.331977] ../drivers/gpu/drm/i915/i915_gem_request.h:371 suspicious rcu_dereference_protected() usage! [ 350.342619] [ 350.342619] other info that might help us debug this: [ 350.342619] [ 350.351593] [ 350.351593] rcu_scheduler_active = 1, debug_locks = 0 [ 350.358952] 3 locks held by Xorg/320: [ 350.363077] #0: (&dev->mode_config.mutex){+.+.+.}, at: [] drm_modeset_lock_all+0x3c/0xd0 [drm] [ 350.375162] #1: (crtc_ww_class_acquire){+.+.+.}, at: [] drm_modeset_lock_all+0x46/0xd0 [drm] [ 350.387022] #2: (crtc_ww_class_mutex){+.+.+.}, at: [] drm_modeset_lock+0x36/0x110 [drm] [ 350.398236] [ 350.398236] stack backtrace: [ 350.403196] CPU: 1 PID: 320 Comm: Xorg Tainted: G U W 4.8.0-rc8-bsw-rapl+ #3133 [ 350.412457] Hardware name: Intel Corporation CHERRYVIEW C0 PLATFORM/Braswell CRB, BIOS BRAS.X64.X088.R00.1510270350 10/27/2015 [ 350.425212] 0000000000000000 ffff8801680a78c8 ffffffff81332187 ffff88016c5c5000 [ 350.433611] 0000000000000001 ffff8801680a78f8 ffffffff810ca6da ffff88016cc8b0f0 [ 350.442012] ffff88016cc80000 ffff88016cc80000 ffff880177ad0000 ffff8801680a7948 [ 350.450409] Call Trace: [ 350.453165] [] dump_stack+0x67/0x90 [ 350.458931] [] lockdep_rcu_suspicious+0xea/0x120 [ 350.466002] [] fence_update+0xbd/0x670 [i915] [ 350.472766] [] i915_gem_restore_fences+0x52/0x70 [i915] [ 350.480496] [] vlv_resume_prepare+0x72/0x570 [i915] [ 350.487839] [] intel_runtime_resume+0x102/0x210 [i915] [ 350.495442] [] pci_pm_runtime_resume+0x7f/0xb0 [ 350.502274] [] ? pci_restore_standard_config+0x40/0x40 [ 350.509883] [] __rpm_callback+0x35/0x70 [ 350.516037] [] ? pci_restore_standard_config+0x40/0x40 [ 350.523646] [] rpm_callback+0x24/0x80 [ 350.529604] [] ? pci_restore_standard_config+0x40/0x40 [ 350.537212] [] rpm_resume+0x4ad/0x740 [ 350.543161] [] __pm_runtime_resume+0x51/0x80 [ 350.549824] [] intel_runtime_pm_get+0x28/0x90 [i915] [ 350.557265] [] intel_display_power_get+0x23/0x50 [i915] [ 350.565001] [] intel_atomic_commit_tail+0xdfd/0x10b0 [i915] [ 350.573106] [] ? drm_atomic_helper_swap_state+0x159/0x300 [drm_kms_helper] [ 350.582659] [] ? _raw_spin_unlock+0x31/0x50 [ 350.589205] [] ? drm_atomic_helper_swap_state+0x159/0x300 [drm_kms_helper] [ 350.598787] [] intel_atomic_commit+0x3b5/0x500 [i915] [ 350.606319] [] ? drm_atomic_set_crtc_for_connector+0xcc/0x100 [drm] [ 350.615209] [] drm_atomic_commit+0x49/0x50 [drm] [ 350.622242] [] drm_atomic_helper_set_config+0x88/0xc0 [drm_kms_helper] [ 350.631419] [] drm_mode_set_config_internal+0x6c/0x120 [drm] [ 350.639623] [] drm_mode_setcrtc+0x22c/0x4d0 [drm] [ 350.646760] [] drm_ioctl+0x209/0x460 [drm] [ 350.653217] [] ? drm_mode_getcrtc+0x150/0x150 [drm] [ 350.660536] [] ? __lock_is_held+0x4a/0x70 [ 350.666885] [] do_vfs_ioctl+0x93/0x6b0 [ 350.672939] [] ? __fget+0x113/0x200 [ 350.678797] [] ? __fget+0x5/0x200 [ 350.684361] [] SyS_ioctl+0x44/0x80 [ 350.690030] [] do_syscall_64+0x5b/0x120 [ 350.696184] [] entry_SYSCALL64_slow_path+0x25/0x25 Note we also have to remember the lesson from commit 4fc788f5ee3d ("drm/i915: Flush delayed fence releases after reset") where we have to flush any changes to the fence on restore. v2: Replace call to release user mmaps with an assertion that they have already been zapped. Fixes: 49ef5294cda2 ("drm/i915: Move fence tracking from object to vma") Reported-by: Ville Syrjälä Signed-off-by: Chris Wilson Cc: Ville Syrjälä Cc: Joonas Lahtinen Cc: Mika Kuoppala Reviewed-by: Joonas Lahtinen --- drivers/gpu/drm/i915/i915_gem_fence.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_gem_fence.c b/drivers/gpu/drm/i915/i915_gem_fence.c index 8df1fa7234e8..2c7ba0ee127c 100644 --- a/drivers/gpu/drm/i915/i915_gem_fence.c +++ b/drivers/gpu/drm/i915/i915_gem_fence.c @@ -290,6 +290,8 @@ i915_vma_put_fence(struct i915_vma *vma) { struct drm_i915_fence_reg *fence = vma->fence; + assert_rpm_wakelock_held(to_i915(vma->vm->dev)); + if (!fence) return 0; @@ -341,6 +343,8 @@ i915_vma_get_fence(struct i915_vma *vma) struct drm_i915_fence_reg *fence; struct i915_vma *set = i915_gem_object_is_tiled(vma->obj) ? vma : NULL; + assert_rpm_wakelock_held(to_i915(vma->vm->dev)); + /* Just update our place in the LRU if our fence is getting reused. */ if (vma->fence) { fence = vma->fence; @@ -371,6 +375,12 @@ void i915_gem_restore_fences(struct drm_device *dev) struct drm_i915_private *dev_priv = to_i915(dev); int i; + /* Note that this may be called outside of struct_mutex, by + * runtime suspend/resume. The barrier we require is enforced by + * rpm itself - all access to fences/GTT are only within an rpm + * wakeref, and to acquire that wakeref you must pass through here. + */ + for (i = 0; i < dev_priv->num_fence_regs; i++) { struct drm_i915_fence_reg *reg = &dev_priv->fence_regs[i]; struct i915_vma *vma = reg->vma; @@ -379,10 +389,17 @@ void i915_gem_restore_fences(struct drm_device *dev) * Commit delayed tiling changes if we have an object still * attached to the fence, otherwise just clear the fence. */ - if (vma && !i915_gem_object_is_tiled(vma->obj)) + if (vma && !i915_gem_object_is_tiled(vma->obj)) { + GEM_BUG_ON(!reg->dirty); + GEM_BUG_ON(vma->obj->fault_mappable); + + list_move(®->link, &dev_priv->mm.fence_list); + vma->fence = NULL; vma = NULL; + } - fence_update(reg, vma); + fence_write(reg, vma); + reg->vma = vma; } }