From patchwork Thu Dec 12 01:46:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11286721 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6F14F14B7 for ; Thu, 12 Dec 2019 01:46:47 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5746422527 for ; Thu, 12 Dec 2019 01:46:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5746422527 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A3F726EC29; Thu, 12 Dec 2019 01:46:45 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5D5B36EC29 for ; Thu, 12 Dec 2019 01:46:44 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19548544-1500050 for multiple; Thu, 12 Dec 2019 01:46:28 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Thu, 12 Dec 2019 01:46:27 +0000 Message-Id: <20191212014629.854076-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 1/3] drm/i915: Use EAGAIN for trylock failures X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" While not good behaviour, it is, however, established behaviour that we can punt EAGAIN to userspace if we need to retry the ioctl. When trying to acquire a mutex, prefer to use EAGAIN to propagate losing the race so that if it does end up back in userspace, we try again. Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin Reviewed-by: Joonas Lahtinen --- drivers/gpu/drm/i915/gt/intel_timeline.c | 2 +- drivers/gpu/drm/i915/i915_request.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_timeline.c b/drivers/gpu/drm/i915/gt/intel_timeline.c index 038e05a6336c..d71aafb66d6e 100644 --- a/drivers/gpu/drm/i915/gt/intel_timeline.c +++ b/drivers/gpu/drm/i915/gt/intel_timeline.c @@ -527,7 +527,7 @@ int intel_timeline_read_hwsp(struct i915_request *from, GEM_BUG_ON(rcu_access_pointer(to->timeline) == tl); - err = -EBUSY; + err = -EAGAIN; if (mutex_trylock(&tl->mutex)) { struct intel_timeline_cacheline *cl = from->hwsp_cacheline; diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index 51bb8a0812a1..fb8738987aeb 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -783,7 +783,7 @@ i915_request_await_start(struct i915_request *rq, struct i915_request *signal) if (!tl) /* already started or maybe even completed */ return 0; - fence = ERR_PTR(-EBUSY); + fence = ERR_PTR(-EAGAIN); if (mutex_trylock(&tl->mutex)) { fence = NULL; if (!i915_request_started(signal) && From patchwork Thu Dec 12 01:46:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11286723 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E82315AB for ; Thu, 12 Dec 2019 01:46:49 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 074BA222C4 for ; Thu, 12 Dec 2019 01:46:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 074BA222C4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 094B96EC2C; Thu, 12 Dec 2019 01:46:46 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6B88C6EC2B for ; Thu, 12 Dec 2019 01:46:44 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19548545-1500050 for multiple; Thu, 12 Dec 2019 01:46:28 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Thu, 12 Dec 2019 01:46:28 +0000 Message-Id: <20191212014629.854076-2-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191212014629.854076-1-chris@chris-wilson.co.uk> References: <20191212014629.854076-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 2/3] drm/i915/gt: Pull intel_timeline.requests list under a spinlock X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Currently we use the intel_timeline.mutex to guard constructing and retiring requests in the timeline, including the adding and removing of the request from the list of requests (intel_timeline.requests). However, we want to peek at neighbouring elements in the request list while constructing a request on another timeline (see i915_request_await_start) and this implies nesting timeline mutexes. To avoid the nested mutex, we currently use a mutex_trylock() but this is fraught with a potential race causing an -EBUSY. We can eliminate the nested mutex here with a spinlock guarding list operations within the broader mutex, so callers can choose between locking everything with the mutex or just the list with the spinlock. (The mutex caters for virtually all of the current users, but maybe being able to easily peek at the request list, we will do so more often in the future.) Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin Reviewed-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/gt/intel_timeline.c | 1 + .../gpu/drm/i915/gt/intel_timeline_types.h | 1 + .../gpu/drm/i915/gt/selftests/mock_timeline.c | 1 + drivers/gpu/drm/i915/i915_request.c | 58 +++++++++++-------- 4 files changed, 36 insertions(+), 25 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_timeline.c b/drivers/gpu/drm/i915/gt/intel_timeline.c index d71aafb66d6e..728da39e8ace 100644 --- a/drivers/gpu/drm/i915/gt/intel_timeline.c +++ b/drivers/gpu/drm/i915/gt/intel_timeline.c @@ -256,6 +256,7 @@ int intel_timeline_init(struct intel_timeline *timeline, INIT_ACTIVE_FENCE(&timeline->last_request); INIT_LIST_HEAD(&timeline->requests); + spin_lock_init(&timeline->request_lock); i915_syncmap_init(&timeline->sync); diff --git a/drivers/gpu/drm/i915/gt/intel_timeline_types.h b/drivers/gpu/drm/i915/gt/intel_timeline_types.h index aaf15cbe1ce1..7c9f49f46626 100644 --- a/drivers/gpu/drm/i915/gt/intel_timeline_types.h +++ b/drivers/gpu/drm/i915/gt/intel_timeline_types.h @@ -57,6 +57,7 @@ struct intel_timeline { * outstanding. */ struct list_head requests; + spinlock_t request_lock; /* * Contains an RCU guarded pointer to the last request. No reference is diff --git a/drivers/gpu/drm/i915/gt/selftests/mock_timeline.c b/drivers/gpu/drm/i915/gt/selftests/mock_timeline.c index aeb1d1f616e8..540729250fef 100644 --- a/drivers/gpu/drm/i915/gt/selftests/mock_timeline.c +++ b/drivers/gpu/drm/i915/gt/selftests/mock_timeline.c @@ -17,6 +17,7 @@ void mock_timeline_init(struct intel_timeline *timeline, u64 context) INIT_ACTIVE_FENCE(&timeline->last_request); INIT_LIST_HEAD(&timeline->requests); + spin_lock_init(&timeline->request_lock); i915_syncmap_init(&timeline->sync); diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index fb8738987aeb..5d8b11e64373 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -184,6 +184,27 @@ remove_from_client(struct i915_request *request) rcu_read_unlock(); } +static inline void remove_from_timeline(struct i915_request *rq) +{ + struct intel_timeline *tl = i915_request_timeline(rq); + + /* + * We know the GPU must have read the request to have + * sent us the seqno + interrupt, so use the position + * of tail of the request to update the last known position + * of the GPU head. + * + * Note this requires that we are always called in request + * completion order. + */ + GEM_BUG_ON(!list_is_first(&rq->link, &tl->requests)); + rq->ring->head = rq->postfix; + + spin_lock(&tl->request_lock); + list_del(&rq->link); + spin_unlock(&tl->request_lock); +} + static void free_capture_list(struct i915_request *request) { struct i915_capture_list *capture; @@ -231,19 +252,6 @@ bool i915_request_retire(struct i915_request *rq) GEM_BUG_ON(!i915_sw_fence_signaled(&rq->submit)); trace_i915_request_retire(rq); - /* - * We know the GPU must have read the request to have - * sent us the seqno + interrupt, so use the position - * of tail of the request to update the last known position - * of the GPU head. - * - * Note this requires that we are always called in request - * completion order. - */ - GEM_BUG_ON(!list_is_first(&rq->link, - &i915_request_timeline(rq)->requests)); - rq->ring->head = rq->postfix; - /* * We only loosely track inflight requests across preemption, * and so we may find ourselves attempting to retire a _completed_ @@ -270,7 +278,7 @@ bool i915_request_retire(struct i915_request *rq) spin_unlock_irq(&rq->lock); remove_from_client(rq); - list_del(&rq->link); + remove_from_timeline(rq); intel_context_exit(rq->hw_context); intel_context_unpin(rq->hw_context); @@ -783,19 +791,17 @@ i915_request_await_start(struct i915_request *rq, struct i915_request *signal) if (!tl) /* already started or maybe even completed */ return 0; - fence = ERR_PTR(-EAGAIN); - if (mutex_trylock(&tl->mutex)) { - fence = NULL; - if (!i915_request_started(signal) && - !list_is_first(&signal->link, &tl->requests)) { - signal = list_prev_entry(signal, link); - fence = dma_fence_get(&signal->fence); - } - mutex_unlock(&tl->mutex); + fence = NULL; + spin_lock(&tl->request_lock); + if (!i915_request_started(signal) && + !list_is_first(&signal->link, &tl->requests)) { + signal = list_prev_entry(signal, link); + fence = dma_fence_get(&signal->fence); } + spin_unlock(&tl->request_lock); intel_timeline_put(tl); - if (IS_ERR_OR_NULL(fence)) - return PTR_ERR_OR_ZERO(fence); + if (!fence) + return 0; err = 0; if (intel_timeline_sync_is_later(i915_request_timeline(rq), fence)) @@ -1238,7 +1244,9 @@ __i915_request_add_to_timeline(struct i915_request *rq) 0); } + spin_lock(&timeline->request_lock); list_add_tail(&rq->link, &timeline->requests); + spin_unlock(&timeline->request_lock); /* * Make sure that no request gazumped us - if it was allocated after From patchwork Thu Dec 12 01:46:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11286725 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E5A913B6 for ; Thu, 12 Dec 2019 01:46:50 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 66FA7222C4 for ; Thu, 12 Dec 2019 01:46:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 66FA7222C4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D54BF6EC2A; Thu, 12 Dec 2019 01:46:45 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 648BB6EC2A for ; Thu, 12 Dec 2019 01:46:44 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19548546-1500050 for multiple; Thu, 12 Dec 2019 01:46:29 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Thu, 12 Dec 2019 01:46:29 +0000 Message-Id: <20191212014629.854076-3-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191212014629.854076-1-chris@chris-wilson.co.uk> References: <20191212014629.854076-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 3/3] drm/i915/gt: Eliminate the trylock for reading a timeline's hwsp X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" As we stash a pointer to the HWSP cacheline on the request, when reading it we only need confirm that the cacheline is still valid by checking that the request and timeline are still intact. Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin --- drivers/gpu/drm/i915/gt/intel_timeline.c | 38 ++++++++---------------- 1 file changed, 13 insertions(+), 25 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_timeline.c b/drivers/gpu/drm/i915/gt/intel_timeline.c index 728da39e8ace..566ce19bb0ea 100644 --- a/drivers/gpu/drm/i915/gt/intel_timeline.c +++ b/drivers/gpu/drm/i915/gt/intel_timeline.c @@ -515,6 +515,7 @@ int intel_timeline_read_hwsp(struct i915_request *from, struct i915_request *to, u32 *hwsp) { + struct intel_timeline_cacheline *cl = from->hwsp_cacheline; struct intel_timeline *tl; int err; @@ -527,33 +528,20 @@ int intel_timeline_read_hwsp(struct i915_request *from, return 1; GEM_BUG_ON(rcu_access_pointer(to->timeline) == tl); - - err = -EAGAIN; - if (mutex_trylock(&tl->mutex)) { - struct intel_timeline_cacheline *cl = from->hwsp_cacheline; - - if (i915_request_completed(from)) { - err = 1; - goto unlock; - } - - err = cacheline_ref(cl, to); - if (err) - goto unlock; - - if (likely(cl == tl->hwsp_cacheline)) { - *hwsp = tl->hwsp_offset; - } else { /* across a seqno wrap, recover the original offset */ - *hwsp = i915_ggtt_offset(cl->hwsp->vma) + - ptr_unmask_bits(cl->vaddr, CACHELINE_BITS) * - CACHELINE_BYTES; - } - -unlock: - mutex_unlock(&tl->mutex); + err = cacheline_ref(cl, to); + if (err) + goto out; + + *hwsp = tl->hwsp_offset; + if (unlikely(cl != READ_ONCE(tl->hwsp_cacheline))) { + /* across a seqno wrap, recover the original offset */ + *hwsp = i915_ggtt_offset(cl->hwsp->vma) + + ptr_unmask_bits(cl->vaddr, CACHELINE_BITS) * + CACHELINE_BYTES; } - intel_timeline_put(tl); +out: + intel_timeline_put(tl); return err; }