From patchwork Sat Nov 18 00:30:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michel Thierry X-Patchwork-Id: 10064235 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 082676037E for ; Sat, 18 Nov 2017 00:30:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E668B2AE7E for ; Sat, 18 Nov 2017 00:30:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DB53C2AE80; Sat, 18 Nov 2017 00:30:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 633B32AE7E for ; Sat, 18 Nov 2017 00:30:43 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 277EC6E321; Sat, 18 Nov 2017 00:30:42 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6E3146E321 for ; Sat, 18 Nov 2017 00:30:39 +0000 (UTC) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Nov 2017 16:30:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.44,411,1505804400"; d="scan'208";a="177516526" Received: from relo-linux-11.sc.intel.com ([10.3.160.161]) by fmsmga006.fm.intel.com with ESMTP; 17 Nov 2017 16:30:39 -0800 From: Michel Thierry To: intel-gfx@lists.freedesktop.org Date: Fri, 17 Nov 2017 16:30:38 -0800 Message-Id: <20171118003038.7935-1-michel.thierry@intel.com> X-Mailer: git-send-email 2.15.0 Subject: [Intel-gfx] [PATCH] drm/i915/lrc: Stop writing to ELSP until HW has processed the previous write X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP The hardware needs some time to process the information received in the ExecList Submission Port, and expects us to don't write anything new until it has 'acknowledged' this new execlist by sending an IDLE_ACTIVE or PREEMPTED CSB event. If we do not follow this, the driver could write new data into the ELSP before HW had finishing fetching the previous one, putting us in 'undefined behaviour' space. This seems to be the problem causing the spurious PREEMPTED & COMPLETE events after a COMPLETE like the one below: [] vcs0: sw rd pointer = 2, hw wr pointer = 0, current 'head' = 3. [] vcs0: Execlist CSB[0]: 0x00000018 _ 0x00000007 [] vcs0: Execlist CSB[1]: 0x00000001 _ 0x00000000 [] vcs0: Execlist CSB[2]: 0x00000018 _ 0x00000007 <<< COMPLETE [] vcs0: Execlist CSB[3]: 0x00000012 _ 0x00000007 <<< PREEMPTED & COMPLETE [] vcs0: Execlist CSB[4]: 0x00008002 _ 0x00000006 [] vcs0: Execlist CSB[5]: 0x00000014 _ 0x00000006 The ELSP writes that lead to this CSB sequence show that the HW hadn't started executing the previous execlist (the one with only ctx 0x6) by the time the new one was submitted; this is a bit more clear in the data show in the EXECLIST_STATUS register at the time of the ELSP write. [] vcs0: ELSP[0] = 0x0_0 [execlist1] - status_reg = 0x0_302 [] vcs0: ELSP[1] = 0x6_fedb2119 [execlist0] - status_reg = 0x0_8302 [] vcs0: ELSP[2] = 0x7_fedaf119 [execlist1] - status_reg = 0x0_8308 [] vcs0: ELSP[3] = 0x6_fedb2119 [execlist0] - status_reg = 0x7_8308 Note that having to wait for this ack does not disable lite-restores, although it may reduce their numbers. And take this as a RFC, since there are probably better ways to still respect this HW requirement. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=102035 Signed-off-by: Michel Thierry Cc: Chris Wilson --- drivers/gpu/drm/i915/intel_lrc.c | 16 +++++++++++++++- drivers/gpu/drm/i915/intel_ringbuffer.h | 6 ++++++ 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c index af41165e3da4..10b7eb64f169 100644 --- a/drivers/gpu/drm/i915/intel_lrc.c +++ b/drivers/gpu/drm/i915/intel_lrc.c @@ -449,11 +449,16 @@ static inline void elsp_write(u64 desc, u32 __iomem *elsp) static void execlists_submit_ports(struct intel_engine_cs *engine) { + struct intel_engine_execlists * const execlists = &engine->execlists; struct execlist_port *port = engine->execlists.port; u32 __iomem *elsp = engine->i915->regs + i915_mmio_reg_offset(RING_ELSP(engine)); unsigned int n; + if (wait_for_atomic(!READ_ONCE(execlists->outstanding_submission), 10)) + GEM_TRACE("%s outstanding submission stuck\n", + engine->name); + for (n = execlists_num_ports(&engine->execlists); n--; ) { struct drm_i915_gem_request *rq; unsigned int count; @@ -479,6 +484,8 @@ static void execlists_submit_ports(struct intel_engine_cs *engine) elsp_write(desc, elsp); } + + WRITE_ONCE(execlists->outstanding_submission, 1); } static bool ctx_single_port_submission(const struct i915_gem_context *ctx) @@ -889,6 +896,11 @@ static void execlists_submission_tasklet(unsigned long data) GEM_TRACE("%s csb[%dd]: status=0x%08x:0x%08x\n", engine->name, head, status, buf[2*head + 1]); + + if ((status & GEN8_CTX_STATUS_IDLE_ACTIVE) || + (status & GEN8_CTX_STATUS_PREEMPTED)) + WRITE_ONCE(execlists->outstanding_submission, 0); + if (!(status & GEN8_CTX_STATUS_COMPLETED_MASK)) continue; @@ -944,9 +956,11 @@ static void execlists_submission_tasklet(unsigned long data) /* After the final element, the hw should be idle */ GEM_BUG_ON(port_count(port) == 0 && !(status & GEN8_CTX_STATUS_ACTIVE_IDLE)); - if (port_count(port) == 0) + if (port_count(port) == 0) { execlists_clear_active(execlists, EXECLISTS_ACTIVE_USER); + WRITE_ONCE(execlists->outstanding_submission, 0); + } } if (head != execlists->csb_head) { diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h index e5c62f8ef0da..2c8e1a74c266 100644 --- a/drivers/gpu/drm/i915/intel_ringbuffer.h +++ b/drivers/gpu/drm/i915/intel_ringbuffer.h @@ -288,6 +288,12 @@ struct intel_engine_execlists { * @csb_use_mmio: access csb through mmio, instead of hwsp */ bool csb_use_mmio; + + /** + * @outstanding_submission: prevent further ELSP writes until HW + * has ack'd one (with IDLE_ACTIVE or PREEMPT CSB events) + */ + bool outstanding_submission; }; #define INTEL_ENGINE_CS_MAX_NAME 8