From patchwork Tue Feb 28 02:11:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andi Shyti X-Patchwork-Id: 13154353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8153BC7EE23 for ; Tue, 28 Feb 2023 02:12:07 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B4D6210E475; Tue, 28 Feb 2023 02:12:06 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9E6F310E475; Tue, 28 Feb 2023 02:12:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677550325; x=1709086325; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FIenfPmcj9m43p8FWplwCkugQtdS7DimsqqZbxg0K1I=; b=NzFJQukKPoGvNURnRI4F8QL8WYJridIedZTOnahH7+48jctQQ+pa/EW1 y+DCULDwq/RzhKltjv0d7cFCbDeqWEiX8dqMhYsywaXqxlUEV9iH/bwNe TB+G2+Q5oWak7rndssRmNwbOFfnzthehfpijjQL9P+dGyvz1SKWPa6pSG 2lAxY4m1lvXR9agNyNPMcqFeMrwzqFJ4Z8XibDx6wIvniNU/a4HOI2MDC K/Un7DeFd0N9bXOrdaa3axaFRaps4oj/I3K3utIjXWsFFBAt6fwjnh8mL i77mGkgoEHiubl8HavbosJCwJ0qob2kHbC1H1y2bZtLZ5GSrv6++SKxfl Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10634"; a="322274010" X-IronPort-AV: E=Sophos;i="5.98,220,1673942400"; d="scan'208";a="322274010" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 18:12:04 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10634"; a="816905928" X-IronPort-AV: E=Sophos;i="5.98,220,1673942400"; d="scan'208";a="816905928" Received: from gsavorni-mobl1.ger.corp.intel.com (HELO intel.com) ([10.249.41.82]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 18:12:00 -0800 From: Andi Shyti To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, stable@vger.kernel.org Subject: [PATCH v3 1/2] drm/i915: Throttle for ringspace prior to taking the timeline mutex Date: Tue, 28 Feb 2023 03:11:41 +0100 Message-Id: <20230228021142.1905349-2-andi.shyti@linux.intel.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230228021142.1905349-1-andi.shyti@linux.intel.com> References: <20230228021142.1905349-1-andi.shyti@linux.intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andi Shyti , Chris Wilson , Matthew Auld , Andi Shyti , Maciej Patelczyk Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Chris Wilson Before taking exclusive ownership of the ring for emitting the request, wait for space in the ring to become available. This allows others to take the timeline->mutex to make forward progresses while userspace is blocked. In particular, this allows regular clients to issue requests on the kernel context, potentially filling the ring, but allow the higher priority heartbeats and pulses to still be submitted without being blocked by the less critical work. Fixes: cf586021642d80 ("drm/i915/gt: Pipelined page migration") Signed-off-by: Chris Wilson Cc: Maciej Patelczyk Cc: stable@vger.kernel.org Signed-off-by: Andi Shyti --- Hi, I'm not sure I need to add the Fixes tag here as this is more preparatory for the next patch. Together, though, patch 1 and 2 make the fix with proper locking mechanism. Andi drivers/gpu/drm/i915/gt/intel_context.c | 41 +++++++++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_context.h | 2 ++ drivers/gpu/drm/i915/i915_request.c | 3 ++ 3 files changed, 46 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c index 2aa63ec521b89..59cd612a23561 100644 --- a/drivers/gpu/drm/i915/gt/intel_context.c +++ b/drivers/gpu/drm/i915/gt/intel_context.c @@ -626,6 +626,47 @@ bool intel_context_revoke(struct intel_context *ce) return ret; } +int intel_context_throttle(const struct intel_context *ce) +{ + const struct intel_ring *ring = ce->ring; + const struct intel_timeline *tl = ce->timeline; + struct i915_request *rq; + int err = 0; + + if (READ_ONCE(ring->space) >= SZ_1K) + return 0; + + rcu_read_lock(); + list_for_each_entry_reverse(rq, &tl->requests, link) { + if (__i915_request_is_complete(rq)) + break; + + if (rq->ring != ring) + continue; + + /* Wait until there will be enough space following that rq */ + if (__intel_ring_space(rq->postfix, + ring->emit, + ring->size) < ring->size / 2) { + if (i915_request_get_rcu(rq)) { + rcu_read_unlock(); + + if (i915_request_wait(rq, + I915_WAIT_INTERRUPTIBLE, + MAX_SCHEDULE_TIMEOUT) < 0) + err = -EINTR; + + rcu_read_lock(); + i915_request_put(rq); + } + break; + } + } + rcu_read_unlock(); + + return err; +} + #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST) #include "selftest_context.c" #endif diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h index 0a8d553da3f43..f919a66cebf5b 100644 --- a/drivers/gpu/drm/i915/gt/intel_context.h +++ b/drivers/gpu/drm/i915/gt/intel_context.h @@ -226,6 +226,8 @@ static inline void intel_context_exit(struct intel_context *ce) ce->ops->exit(ce); } +int intel_context_throttle(const struct intel_context *ce); + static inline struct intel_context *intel_context_get(struct intel_context *ce) { kref_get(&ce->ref); diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index 7503dcb9043bb..a1741c4a8cffd 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -1035,6 +1035,9 @@ i915_request_create(struct intel_context *ce) struct i915_request *rq; struct intel_timeline *tl; + if (intel_context_throttle(ce)) + return ERR_PTR(-EINTR); + tl = intel_context_timeline_lock(ce); if (IS_ERR(tl)) return ERR_CAST(tl); From patchwork Tue Feb 28 02:11:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andi Shyti X-Patchwork-Id: 13154354 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 588C7C64ED6 for ; Tue, 28 Feb 2023 02:12:17 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 755B210E480; Tue, 28 Feb 2023 02:12:16 +0000 (UTC) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by gabe.freedesktop.org (Postfix) with ESMTPS id 964BE10E480; Tue, 28 Feb 2023 02:12:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677550334; x=1709086334; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=u18ddDQOwan1fLFqHOnOS3Odna9gVESGyCeFrwXiY6M=; b=fCh6y/BObvrj6IwaI8WG5GbE9aIC5bYMybH0KbsT4p3wWuOaRVd+sUP6 QJGRGRcRwXOUSesOGAH6tjf8PqMV4NDR4Xv3R+seXR44YBjQ2co0g5g+P JYBkgetoSBoVgRVxnAzMd2wO6khZ7Xak0vzBI2ADpgzzWHtL2PNUz+FJg z4/tt+3ozj2T27rh2KOqvH6n9R8MxlqkQ8GTfaab7fZjQOEHhY4UHEsZs +DmNs0ZhP9XyLlXJLIuPKriadEkQh29gkprinVCQQZejIk0jdJn5mVd5n CLZyvmSj+gMhwL9C8zRQucBaKpyi9nF8lFbulSa2ho2l7ddlPOubN8n5z g==; X-IronPort-AV: E=McAfee;i="6500,9779,10634"; a="396590571" X-IronPort-AV: E=Sophos;i="5.98,220,1673942400"; d="scan'208";a="396590571" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 18:12:14 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10634"; a="742804739" X-IronPort-AV: E=Sophos;i="5.98,220,1673942400"; d="scan'208";a="742804739" Received: from gsavorni-mobl1.ger.corp.intel.com (HELO intel.com) ([10.249.41.82]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 18:12:09 -0800 From: Andi Shyti To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, stable@vger.kernel.org Subject: [PATCH v3 2/2] drm/i915/gt: Make sure that errors are propagated through request chains Date: Tue, 28 Feb 2023 03:11:42 +0100 Message-Id: <20230228021142.1905349-3-andi.shyti@linux.intel.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230228021142.1905349-1-andi.shyti@linux.intel.com> References: <20230228021142.1905349-1-andi.shyti@linux.intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andi Shyti , Chris Wilson , Matthew Auld , Andi Shyti , Maciej Patelczyk Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Currently, when we perform operations such as clearing or copying large blocks of memory, we generate multiple requests that are executed in a chain. However, if one of these requests fails, we may not realize it unless it happens to be the last request in the chain. This is because errors are not properly propagated. For this we need to keep propagating the chain of fence notification in order to always reach the final fence associated to the final request. To address this issue, we need to ensure that the chain of fence notifications is always propagated so that we can reach the final fence associated with the last request. By doing so, we will be able to detect any memory operation failures and determine whether the memory is still invalid. On copy and clear migration signal fences upon completion. On copy and clear migration, signal fences upon request completion to ensure that we have a reliable perpetuation of the operation outcome. Fixes: cf586021642d80 ("drm/i915/gt: Pipelined page migration") Reported-by: Matthew Auld Suggested-by: Chris Wilson Signed-off-by: Andi Shyti Cc: stable@vger.kernel.org Reviewed-by: Matthew Auld --- drivers/gpu/drm/i915/gt/intel_migrate.c | 39 ++++++++++++++++++------- 1 file changed, 29 insertions(+), 10 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.c b/drivers/gpu/drm/i915/gt/intel_migrate.c index 3f638f1987968..6b497640d3a0a 100644 --- a/drivers/gpu/drm/i915/gt/intel_migrate.c +++ b/drivers/gpu/drm/i915/gt/intel_migrate.c @@ -742,13 +742,19 @@ intel_context_migrate_copy(struct intel_context *ce, dst_offset = 2 * CHUNK_SZ; } + /* + * While building the chain of requests, we need to ensure + * that no one can sneak into the timeline unnoticed. + */ + mutex_lock(&ce->timeline->mutex); + do { int len; rq = i915_request_create(ce); if (IS_ERR(rq)) { err = PTR_ERR(rq); - goto out_ce; + break; } if (deps) { @@ -878,10 +884,14 @@ intel_context_migrate_copy(struct intel_context *ce, /* Arbitration is re-enabled between requests. */ out_rq: - if (*out) - i915_request_put(*out); - *out = i915_request_get(rq); + i915_sw_fence_await(&rq->submit); + i915_request_get(rq); i915_request_add(rq); + if (*out) { + i915_sw_fence_complete(&(*out)->submit); + i915_request_put(*out); + } + *out = rq; if (err) break; @@ -905,7 +915,10 @@ intel_context_migrate_copy(struct intel_context *ce, cond_resched(); } while (1); -out_ce: + mutex_unlock(&ce->timeline->mutex); + + if (*out) + i915_sw_fence_complete(&(*out)->submit); return err; } @@ -1005,7 +1018,7 @@ intel_context_migrate_clear(struct intel_context *ce, rq = i915_request_create(ce); if (IS_ERR(rq)) { err = PTR_ERR(rq); - goto out_ce; + break; } if (deps) { @@ -1056,17 +1069,23 @@ intel_context_migrate_clear(struct intel_context *ce, /* Arbitration is re-enabled between requests. */ out_rq: - if (*out) - i915_request_put(*out); - *out = i915_request_get(rq); + i915_sw_fence_await(&rq->submit); + i915_request_get(rq); i915_request_add(rq); + if (*out) { + i915_sw_fence_complete(&(*out)->submit); + i915_request_put(*out); + } + *out = rq; + if (err || !it.sg || !sg_dma_len(it.sg)) break; cond_resched(); } while (1); -out_ce: + if (*out) + i915_sw_fence_complete(&(*out)->submit); return err; }