From patchwork Thu Jul 7 20:02:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910184 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 097D1C43334 for ; Thu, 7 Jul 2022 20:02:48 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 9311D11B553; Thu, 7 Jul 2022 20:02:46 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e5ab]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8613B11B553; Thu, 7 Jul 2022 20:02:45 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id DD9BC66019CB; Thu, 7 Jul 2022 21:02:43 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224164; bh=2wQM6XEEejfV8LYpxGFHtQpBTSKwot13+IyHpuiYCD4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RH+CyuXvddCAWhla5EfcOd2icXwSgTngfMCwZolc719fEAU7HdLoASvbuETZmOlXR pSgDR/qGVzL6a+x3yb/ZQ0H5oRAKIh6fVlDy62/K6XxMZJMM5VqKs+NOJviXns7ZsA HdywLb/+jpUtRfCqTJiwmg5PvhrCHq1+i4+wLDAqpkkcoi5U+P9xmEZfSQJa6oQqAx 6/3ELJyYXsOc8j7x/iUcw7y15MaLMtfWfDOKWFfT3uxXIt4XTDa7T+NKnKKxiLNv2S 4blRyYGDYgH9iyS1hge50yyxh0DtLaFpmxxZCNU5hN40sd217gXql+jFip0PbQtZsV lU0WFQLlc+kDQ== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:19 +0000 Message-Id: <20220707200230.1657555-2-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 01/11] drm/i915/ttm: dont trample cache_level overrides during ttm move X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Various places within the driver override the default chosen cache_level. Before ttm, these overrides were permanent until explicitly changed again or for the lifetime of the buffer. TTM movement code came along and decided that it could make that decision at that time, which is usually well after object creation, so overrode the cache_level decision and reverted it back to its default decision. Add logic to indicate whether the caching mode has been set by anything other than the move logic. If so, assume that the code that overrode the defaults knows best and keep it. Signed-off-by: Robert Beckett Reviewed-by: Thomas Hellström --- drivers/gpu/drm/i915/gem/i915_gem_object.c | 1 + drivers/gpu/drm/i915/gem/i915_gem_object_types.h | 1 + drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 1 + drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 9 ++++++--- 4 files changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c index 06b1b188ce5a..519887769c08 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c @@ -125,6 +125,7 @@ void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj, struct drm_i915_private *i915 = to_i915(obj->base.dev); obj->cache_level = cache_level; + obj->ttm.cache_level_override = true; if (cache_level != I915_CACHE_NONE) obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ | diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h index 2c88bdb8ff7c..6632ed52e919 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h @@ -605,6 +605,7 @@ struct drm_i915_gem_object { struct i915_gem_object_page_iter get_io_page; struct drm_i915_gem_object *backup; bool created:1; + bool cache_level_override:1; } ttm; /* diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 4c25d9b2f138..27d59639177f 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -1241,6 +1241,7 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, i915_gem_object_init_memory_region(obj, mem); i915_ttm_adjust_domains_after_move(obj); i915_ttm_adjust_gem_after_move(obj); + obj->ttm.cache_level_override = false; i915_gem_object_unlock(obj); return 0; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c index a10716f4e717..4c1de0b4a10f 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c @@ -123,9 +123,12 @@ void i915_ttm_adjust_gem_after_move(struct drm_i915_gem_object *obj) obj->mem_flags |= i915_ttm_cpu_maps_iomem(bo->resource) ? I915_BO_FLAG_IOMEM : I915_BO_FLAG_STRUCT_PAGE; - cache_level = i915_ttm_cache_level(to_i915(bo->base.dev), bo->resource, - bo->ttm); - i915_gem_object_set_cache_coherency(obj, cache_level); + if (!obj->ttm.cache_level_override) { + cache_level = i915_ttm_cache_level(to_i915(bo->base.dev), + bo->resource, bo->ttm); + i915_gem_object_set_cache_coherency(obj, cache_level); + obj->ttm.cache_level_override = false; + } } /** From patchwork Thu Jul 7 20:02:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6D44BCCA47F for ; Thu, 7 Jul 2022 20:03:13 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4CBA412B9AC; Thu, 7 Jul 2022 20:02:55 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e5ab]) by gabe.freedesktop.org (Postfix) with ESMTPS id EF38A11B553; Thu, 7 Jul 2022 20:02:45 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id 8066F66019CD; Thu, 7 Jul 2022 21:02:44 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224164; bh=fPrXPZGZmc1m43xW5GU/UJEKX5XpJJEdaAwsrSiitrc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gKvlSMDpk4XbqHd/1UwzTfU9FHqAovhmTDgp80gftMOwmVHGAeZwQFzKuNOeDzcZp JEg+sLqJ3GUrikIFe9CX/11v7N6Qnqne9p24b8G2t9ToubPJuLH6ERRDOMcGXlhkqj NLJ7jDgTd2XPPsWAPtX94rjstE969CQzTvr9PDnnzo7bLObJa8AwqfNUaisIMHDouA nAxXKH8qIys4ZO3s6yKtKq5p11GfPl8QV6yn2RDSiNMVg85X/yNN29hD6FS22qQ9el 2qewM72NNJar0KdPqV1YES5oG9+m04bzm8ErbeMlk1OKravkJDGSLChu2sTFLyRKf2 05tx1LVcp16wg== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:20 +0000 Message-Id: <20220707200230.1657555-3-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 02/11] drm/i915: limit ttm to dma32 for i965G[M] X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" i965G[M] cannot relocate objects above 4GiB. Ensure ttm uses dma32 on these systems. Signed-off-by: Robert Beckett Reviewed-by: Thomas Hellström --- drivers/gpu/drm/i915/intel_region_ttm.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/intel_region_ttm.c b/drivers/gpu/drm/i915/intel_region_ttm.c index 62ff77445b01..fd2ecfdd8fa1 100644 --- a/drivers/gpu/drm/i915/intel_region_ttm.c +++ b/drivers/gpu/drm/i915/intel_region_ttm.c @@ -32,10 +32,15 @@ int intel_region_ttm_device_init(struct drm_i915_private *dev_priv) { struct drm_device *drm = &dev_priv->drm; + bool use_dma32 = false; + + /* i965g[m] cannot relocate objects above 4GiB. */ + if (IS_I965GM(dev_priv) || IS_I965G(dev_priv)) + use_dma32 = true; return ttm_device_init(&dev_priv->bdev, i915_ttm_driver(), drm->dev, drm->anon_inode->i_mapping, - drm->vma_offset_manager, false, false); + drm->vma_offset_manager, false, use_dma32); } /** From patchwork Thu Jul 7 20:02:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1C20C43334 for ; Thu, 7 Jul 2022 20:03:12 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C4DB812B9A4; Thu, 7 Jul 2022 20:02:53 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e5ab]) by gabe.freedesktop.org (Postfix) with ESMTPS id 88A9A11B66D; Thu, 7 Jul 2022 20:02:46 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id 247CD66019D1; Thu, 7 Jul 2022 21:02:45 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224165; bh=x5DY8lqdscLPNZjf5Ukvpn+M8BELAAg44M2twj/KzYc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dnSzpR2Vluyq95hA7h6IbToJQcYmv/0Euqnjfjf+LmuZhruMRFStGs3MVcJxZqS2K GAR2cyWytyFzinh5+mtZFpBM6AzMUNZ0/Sltc8kBnDXGmsACWax1GcVrgEFSC0u8wh D8tuiXBDBlYo3NkbyqF7x/Oo8DWA/Ek7Mq5z8kKxwkH6dqYXCVirAhggobqMFD8/cJ jPk01m14GFg4GdtIMCrXQMqT9FumaiIUpdmtPmyi9M+LaEHijvtQKNYi+J34bBONQ9 BJZfeAWqmZG8yk9OPqae1urnOwPrdbHbH3Z4sTlhITbLBXHNYfccc3NEh/0fE9yNPo 4w2R6LvVZA31g== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:21 +0000 Message-Id: <20220707200230.1657555-4-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 03/11] drm/i915/ttm: only trust snooping for dgfx when deciding default cache_level X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" By default i915_ttm_cache_level() decides I915_CACHE_LLC if HAS_SNOOP. This is divergent from existing backends code which only considers HAS_LLC. Testing shows that trusting snooping on gen5- is unreliable and bsw via ggtt mappings, so limit DGFX for now and maintain previous behaviour. Signed-off-by: Robert Beckett Reviewed-by: Thomas Hellström --- drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c index 4c1de0b4a10f..40249fa28a7a 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c @@ -46,7 +46,9 @@ static enum i915_cache_level i915_ttm_cache_level(struct drm_i915_private *i915, struct ttm_resource *res, struct ttm_tt *ttm) { - return ((HAS_LLC(i915) || HAS_SNOOP(i915)) && + bool can_snoop = HAS_SNOOP(i915) && IS_DGFX(i915); + + return ((HAS_LLC(i915) || can_snoop) && !i915_ttm_gtt_binds_lmem(res) && ttm->caching == ttm_cached) ? I915_CACHE_LLC : I915_CACHE_NONE; From patchwork Thu Jul 7 20:02:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8B299CCA479 for ; Thu, 7 Jul 2022 20:03:08 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 72FD111B6FF; Thu, 7 Jul 2022 20:02:53 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e5ab]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5B38B11B66D; Thu, 7 Jul 2022 20:02:47 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id B746066019D5; Thu, 7 Jul 2022 21:02:45 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224166; bh=qy8+fGCHoC0k4JC8KL3MA8/A+stlKuCvaW0SY+DabgM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XHKITXNbokLUcewLHkw402KhFQh7UWhgyGj5R9Eh9QkF37aVykycR81Mn/seHYjYz JJLhpR764RYcSYoBCjHYemNaGGNUxP62KZTCa4VDU88Vb61vvlVW3sAAd3gSVFL8/v 6ka8foOd1euFPuNFRBTiZnPIJoq/WcXjb2l3VhSfVuu+aqYkOT9Sje+BHh4/vNs/F6 KaMouzWvdWDATsCMPDJq3MBWBoGo5m4IuRhBTmG1g7Oo6oiMtH3bG+cfXZgKzN6DA/ 8f0TZ/NMN+wDasrAHFwdtdmdfJSm335RLPv+cPbUVN9r9URZ0lalwHJzrOKC6bwxe2 NkR1uw1NwXacw== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:22 +0000 Message-Id: <20220707200230.1657555-5-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 04/11] drm/i915/gem: selftest should not attempt mmap of private regions X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" During testing make can_mmap consider whether the region is private. Signed-off-by: Robert Beckett Reviewed-by: Thomas Hellström --- drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c index 5bc93a1ce3e3..76181e28c75e 100644 --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c @@ -869,6 +869,9 @@ static bool can_mmap(struct drm_i915_gem_object *obj, enum i915_mmap_type type) struct drm_i915_private *i915 = to_i915(obj->base.dev); bool no_map; + if (obj->mm.region && obj->mm.region->private) + return false; + if (obj->ops->mmap_offset) return type == I915_MMAP_TYPE_FIXED; else if (type == I915_MMAP_TYPE_FIXED) From patchwork Thu Jul 7 20:02:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2C58C43334 for ; Thu, 7 Jul 2022 20:02:56 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DF79011B6C3; Thu, 7 Jul 2022 20:02:50 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by gabe.freedesktop.org (Postfix) with ESMTPS id E92B511B688; Thu, 7 Jul 2022 20:02:47 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id 6FE0766019D9; Thu, 7 Jul 2022 21:02:46 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224166; bh=4aLhOjNbnMWEd5TE2kdvFPY0DN2hydFT2IvNHlkBa+w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=evQyPjJlvvmk4j5PwsrJFD1QCYTbCuIf/tTRGtPN/q2Q48euAkUZ/mF5pIgVBhdcG hz/Os5AMdrMpuO+y51kCVCwaWfJL/qa/bJev4ksPXcwiv2mSQInsNIibPjlThh2JyL +4IJWlc2lZQEa0sIHedfY86y6aqETQNB6QaPx9zkkke9xGjmejoqO9vk6ADPeCiQyq JPasR3b5z6CHDbleZxYPLl9/+Ki83u+b/nE/jKUkPr7hG6mvl7iovNgYqTQkJkOOjC RJPkCY+EcrvvXbCQIOSZOWM79OYoQL9IQcNXK/jP/Y/G4LYBEVu/BVgNYQFfZsHYQL qaDp9z+Qzcqzw== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:23 +0000 Message-Id: <20220707200230.1657555-6-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 05/11] drm/i915: instantiate ttm ranger manager for stolen memory X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" prepare for ttm based stolen region by using ttm range manager as the resource manager for stolen region. Signed-off-by: Robert Beckett Reviewed-by: Thomas Hellström --- drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 6 ++-- drivers/gpu/drm/i915/intel_region_ttm.c | 31 +++++++++++++++----- 2 files changed, 27 insertions(+), 10 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c index 40249fa28a7a..675e9ab30396 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c @@ -60,11 +60,13 @@ i915_ttm_region(struct ttm_device *bdev, int ttm_mem_type) struct drm_i915_private *i915 = container_of(bdev, typeof(*i915), bdev); /* There's some room for optimization here... */ - GEM_BUG_ON(ttm_mem_type != I915_PL_SYSTEM && - ttm_mem_type < I915_PL_LMEM0); + GEM_BUG_ON(ttm_mem_type == I915_PL_GGTT); + if (ttm_mem_type == I915_PL_SYSTEM) return intel_memory_region_lookup(i915, INTEL_MEMORY_SYSTEM, 0); + if (ttm_mem_type == I915_PL_STOLEN) + return i915->mm.stolen_region; return intel_memory_region_lookup(i915, INTEL_MEMORY_LOCAL, ttm_mem_type - I915_PL_LMEM0); diff --git a/drivers/gpu/drm/i915/intel_region_ttm.c b/drivers/gpu/drm/i915/intel_region_ttm.c index fd2ecfdd8fa1..694e9acb69e2 100644 --- a/drivers/gpu/drm/i915/intel_region_ttm.c +++ b/drivers/gpu/drm/i915/intel_region_ttm.c @@ -54,7 +54,7 @@ void intel_region_ttm_device_fini(struct drm_i915_private *dev_priv) /* * Map the i915 memory regions to TTM memory types. We use the - * driver-private types for now, reserving TTM_PL_VRAM for stolen + * driver-private types for now, reserving I915_PL_STOLEN for stolen * memory and TTM_PL_TT for GGTT use if decided to implement this. */ int intel_region_to_ttm_type(const struct intel_memory_region *mem) @@ -63,11 +63,17 @@ int intel_region_to_ttm_type(const struct intel_memory_region *mem) GEM_BUG_ON(mem->type != INTEL_MEMORY_LOCAL && mem->type != INTEL_MEMORY_MOCK && - mem->type != INTEL_MEMORY_SYSTEM); + mem->type != INTEL_MEMORY_SYSTEM && + mem->type != INTEL_MEMORY_STOLEN_SYSTEM && + mem->type != INTEL_MEMORY_STOLEN_LOCAL); if (mem->type == INTEL_MEMORY_SYSTEM) return TTM_PL_SYSTEM; + if (mem->type == INTEL_MEMORY_STOLEN_SYSTEM || + mem->type == INTEL_MEMORY_STOLEN_LOCAL) + return I915_PL_STOLEN; + type = mem->instance + TTM_PL_PRIV; GEM_BUG_ON(type >= TTM_NUM_MEM_TYPES); @@ -91,10 +97,16 @@ int intel_region_ttm_init(struct intel_memory_region *mem) int mem_type = intel_region_to_ttm_type(mem); int ret; - ret = i915_ttm_buddy_man_init(bdev, mem_type, false, - resource_size(&mem->region), - mem->io_size, - mem->min_page_size, PAGE_SIZE); + if (mem_type == I915_PL_STOLEN) { + ret = ttm_range_man_init(bdev, mem_type, false, + resource_size(&mem->region) >> PAGE_SHIFT); + mem->is_range_manager = true; + } else { + ret = i915_ttm_buddy_man_init(bdev, mem_type, false, + resource_size(&mem->region), + mem->io_size, + mem->min_page_size, PAGE_SIZE); + } if (ret) return ret; @@ -114,6 +126,7 @@ int intel_region_ttm_init(struct intel_memory_region *mem) int intel_region_ttm_fini(struct intel_memory_region *mem) { struct ttm_resource_manager *man = mem->region_private; + int mem_type = intel_region_to_ttm_type(mem); int ret = -EBUSY; int count; @@ -144,8 +157,10 @@ int intel_region_ttm_fini(struct intel_memory_region *mem) if (ret || !man) return ret; - ret = i915_ttm_buddy_man_fini(&mem->i915->bdev, - intel_region_to_ttm_type(mem)); + if (mem_type == I915_PL_STOLEN) + ret = ttm_range_man_fini(&mem->i915->bdev, mem_type); + else + ret = i915_ttm_buddy_man_fini(&mem->i915->bdev, mem_type); GEM_WARN_ON(ret); mem->region_private = NULL; From patchwork Thu Jul 7 20:02:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9AE08C43334 for ; Thu, 7 Jul 2022 20:03:10 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C304511B787; Thu, 7 Jul 2022 20:02:53 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7B2FF11B697; Thu, 7 Jul 2022 20:02:48 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id 0FB3E66019CA; Thu, 7 Jul 2022 21:02:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224167; bh=AgP5Riz/FHb587iBL1vYiW/g4qmDc92BtRCaR0jjFGw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=c3A3YQU43mD8tHoWDu6hmqoZ0rPpWu01rTbGnvF4TSgJEBEs96Y3CPDjSXrDMcPWp 37wFTItIVXgx7iJ7Uu+rr4dNvNaqWjNqGIaHkvjsb7N6Tum3qOAQgo7oRsPruywZdi O/uzPKUQLt4w8PEuN+iIZHlnBxKXtgDdj8JEYd7zjzZgnltAXlLSXKK81era0eVTUz deeUuVlLP6ly6deQeqRczfX2LlfwqLtnwGl3a1RtbHDTmVxFl1TJn8exIgRhq8Jn4n KG9ZiNne4MMdXmVhXXC19GMzBCKqciKV1DRtknRjHvaVVPeyMLYGtUJgqEpNcI6L4H FQ4IGo4dW6zeg== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:24 +0000 Message-Id: <20220707200230.1657555-7-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 06/11] drm/i915: sanitize mem_flags for stolen buffers X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Stolen regions are not page backed or considered iomem. Prevent flags indicating such. This correctly prevents stolen buffers from attempting to directly map them. See i915_gem_object_has_struct_page() and i915_gem_object_has_iomem() usage for where it would break otherwise. Signed-off-by: Robert Beckett Reviewed-by: Thomas Hellström --- drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c index 675e9ab30396..81c67ca9edda 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c @@ -14,6 +14,7 @@ #include "gem/i915_gem_region.h" #include "gem/i915_gem_ttm.h" #include "gem/i915_gem_ttm_move.h" +#include "gem/i915_gem_stolen.h" #include "gt/intel_engine_pm.h" #include "gt/intel_gt.h" @@ -124,8 +125,9 @@ void i915_ttm_adjust_gem_after_move(struct drm_i915_gem_object *obj) obj->mem_flags &= ~(I915_BO_FLAG_STRUCT_PAGE | I915_BO_FLAG_IOMEM); - obj->mem_flags |= i915_ttm_cpu_maps_iomem(bo->resource) ? I915_BO_FLAG_IOMEM : - I915_BO_FLAG_STRUCT_PAGE; + if (!i915_gem_object_is_stolen(obj)) + obj->mem_flags |= i915_ttm_cpu_maps_iomem(bo->resource) ? I915_BO_FLAG_IOMEM : + I915_BO_FLAG_STRUCT_PAGE; if (!obj->ttm.cache_level_override) { cache_level = i915_ttm_cache_level(to_i915(bo->base.dev), From patchwork Thu Jul 7 20:02:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0FA17CCA47F for ; Thu, 7 Jul 2022 20:02:57 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 25CCE11B66D; Thu, 7 Jul 2022 20:02:51 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3C32011B6D0; Thu, 7 Jul 2022 20:02:49 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id A4C9D66019DD; Thu, 7 Jul 2022 21:02:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224167; bh=vcV9d53cSiZS9S6xUBuwTwjcLWVgxyTlm1c7ldt4HDk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DkdaUT9KatkLleaiwgAQXYI5tiMl4pr7LWT3/t9PDKeOVGNBCamBwHfChCGxn6FpO xP4dFE70BvWSXHUzda+yHPqbXv7s0MyYpPfGQM5sdl2Mh0bjDh43mfjst82FRUDgHL Wo4wESi3d3rPIRtRqT8qD263nLVzjRLyeH00YMZrrkf4LMukqm+4vZdL6xrGFaCc/j Qakp0dMbC37zF/3DArOAaGy0CAcJInWyhdWRCnZJoVh6F+lHRQ3bNcIVwUGpAl2xEH pI3qgQEyQve11NAxOk5QJLgygwc++0l8PXrWsEbb/r2ZMILOArx8k+iz5xe5XElcEZ pMyQXDuLsDWMg== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:25 +0000 Message-Id: <20220707200230.1657555-8-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 07/11] drm/i915: ttm move/clear logic fix X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" ttm managed buffers start off with system resource definitions and ttm_tt tracking structures allocated (though unpopulated). currently this prevents clearing of buffers on first move to desired placements. The desired behaviour is to clear user allocated buffers and any kernel buffers that specifically requests it only. Make the logic match the desired behaviour. Signed-off-by: Robert Beckett Reviewed-by: Thomas Hellström --- drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 22 +++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c index 81c67ca9edda..a3f8fc056dbc 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c @@ -3,6 +3,7 @@ * Copyright © 2021 Intel Corporation */ +#include "drm/ttm/ttm_tt.h" #include #include "i915_deps.h" @@ -476,6 +477,25 @@ __i915_ttm_move(struct ttm_buffer_object *bo, return fence; } +static bool +allow_clear(struct drm_i915_gem_object *obj, struct ttm_tt *ttm, struct ttm_resource *dst_mem) +{ + /* never clear stolen */ + if (dst_mem->mem_type == I915_PL_STOLEN) + return false; + /* + * we want to clear user buffers and any kernel buffers + * that specifically request clearing. + */ + if (obj->flags & I915_BO_ALLOC_USER) + return true; + + if (ttm && ttm->page_flags & TTM_TT_FLAG_ZERO_ALLOC) + return true; + + return false; +} + /** * i915_ttm_move - The TTM move callback used by i915. * @bo: The buffer object. @@ -526,7 +546,7 @@ int i915_ttm_move(struct ttm_buffer_object *bo, bool evict, return PTR_ERR(dst_rsgt); clear = !i915_ttm_cpu_maps_iomem(bo->resource) && (!ttm || !ttm_tt_is_populated(ttm)); - if (!(clear && ttm && !(ttm->page_flags & TTM_TT_FLAG_ZERO_ALLOC))) { + if (!clear || allow_clear(obj, ttm, dst_mem)) { struct i915_deps deps; i915_deps_init(&deps, GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN); From patchwork Thu Jul 7 20:02:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 19ABDCCA479 for ; Thu, 7 Jul 2022 20:03:12 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 614DA11B6F8; Thu, 7 Jul 2022 20:02:53 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by gabe.freedesktop.org (Postfix) with ESMTPS id D5BE811B6C3; Thu, 7 Jul 2022 20:02:49 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id 633BC66019E2; Thu, 7 Jul 2022 21:02:48 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224168; bh=BJ87eisXj9Ppx4nwiDdWpEsHsZ0BoBn8nQEON2j1/N8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eMVQAxAA055i7JCbwZV4VOLNSzs/tarHYtDiTlr9njl1uqAHufvYPx23qR5eAwSUW ZVHVSmD8hBjGAJtpVUnnnpRmsnWcu/y7+u5IiiRbJwf+er3koiV+mGcCNEcYGzXr7Y Ff0TJGoWs2BfEOeMXRNkZFqVXyj+JukdEKhLX9iw9P6YIJPP3yvXLwIyMWqyi9n92k JnJV2SYaKFQX8OGfkIZkBNouzRAevtR9wzvQpmdNd/Tvs+PaI12MYc+3/m2YaZsW6s Wft5LOa+oUnQaNdqJ9ViZXisA0oKzW25j0ued7BY53r1g8c4MWU4RDN9sJb2lP+/ab QHkPQV3xvU7NA== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:26 +0000 Message-Id: <20220707200230.1657555-9-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 08/11] drm/i915/ttm: add buffer pin on alloc flag X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" For situations where allocations need to fail on alloc instead of delayed get_pages, add a new alloc flag to pin the ttm bo. This makes sure that the resource has been allocated during buffer creation, allowing it to fail with an error if the placement is exhausted. This allows existing fallback options for stolen backend allocation like create_ring_vma to work as expected. Signed-off-by: Robert Beckett --- .../gpu/drm/i915/gem/i915_gem_object_types.h | 13 ++++++---- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 25 ++++++++++++++++++- 2 files changed, 32 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h index 6632ed52e919..07bc11247a3e 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h @@ -325,17 +325,20 @@ struct drm_i915_gem_object { * dealing with userspace objects the CPU fault handler is free to ignore this. */ #define I915_BO_ALLOC_GPU_ONLY BIT(6) +/* object should be pinned in destination region from allocation */ +#define I915_BO_ALLOC_PINNED BIT(7) #define I915_BO_ALLOC_FLAGS (I915_BO_ALLOC_CONTIGUOUS | \ I915_BO_ALLOC_VOLATILE | \ I915_BO_ALLOC_CPU_CLEAR | \ I915_BO_ALLOC_USER | \ I915_BO_ALLOC_PM_VOLATILE | \ I915_BO_ALLOC_PM_EARLY | \ - I915_BO_ALLOC_GPU_ONLY) -#define I915_BO_READONLY BIT(7) -#define I915_TILING_QUIRK_BIT 8 /* unknown swizzling; do not release! */ -#define I915_BO_PROTECTED BIT(9) -#define I915_BO_WAS_BOUND_BIT 10 + I915_BO_ALLOC_GPU_ONLY | \ + I915_BO_ALLOC_PINNED) +#define I915_BO_READONLY BIT(8) +#define I915_TILING_QUIRK_BIT 9 /* unknown swizzling; do not release! */ +#define I915_BO_PROTECTED BIT(10) +#define I915_BO_WAS_BOUND_BIT 11 /** * @mem_flags - Mutable placement-related flags * diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 27d59639177f..bb988608296d 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -998,6 +998,13 @@ static void i915_ttm_delayed_free(struct drm_i915_gem_object *obj) { GEM_BUG_ON(!obj->ttm.created); + /* stolen objects are pinned for lifetime. Unpin before putting */ + if (obj->flags & I915_BO_ALLOC_PINNED) { + ttm_bo_reserve(i915_gem_to_ttm(obj), true, false, NULL); + ttm_bo_unpin(i915_gem_to_ttm(obj)); + ttm_bo_unreserve(i915_gem_to_ttm(obj)); + } + ttm_bo_put(i915_gem_to_ttm(obj)); } @@ -1193,6 +1200,9 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, .no_wait_gpu = false, }; enum ttm_bo_type bo_type; + struct ttm_place _place; + struct ttm_placement _placement; + struct ttm_placement *placement; int ret; drm_gem_private_object_init(&i915->drm, &obj->base, size); @@ -1222,6 +1232,17 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, */ i915_gem_object_make_unshrinkable(obj); + if (obj->flags & I915_BO_ALLOC_PINNED) { + i915_ttm_place_from_region(mem, &_place, obj->bo_offset, + obj->base.size, obj->flags); + _placement.num_placement = 1; + _placement.placement = &_place; + _placement.num_busy_placement = 0; + _placement.busy_placement = NULL; + placement = &_placement; + } else { + placement = &i915_sys_placement; + } /* * If this function fails, it will call the destructor, but * our caller still owns the object. So no freeing in the @@ -1230,7 +1251,7 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, * until successful initialization. */ ret = ttm_bo_init_reserved(&i915->bdev, i915_gem_to_ttm(obj), size, - bo_type, &i915_sys_placement, + bo_type, placement, page_size >> PAGE_SHIFT, &ctx, NULL, NULL, i915_ttm_bo_destroy); if (ret) @@ -1242,6 +1263,8 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, i915_ttm_adjust_domains_after_move(obj); i915_ttm_adjust_gem_after_move(obj); obj->ttm.cache_level_override = false; + if (obj->flags & I915_BO_ALLOC_PINNED) + ttm_bo_pin(i915_gem_to_ttm(obj)); i915_gem_object_unlock(obj); return 0; From patchwork Thu Jul 7 20:02:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A6EDAC43334 for ; Thu, 7 Jul 2022 20:03:07 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F22D912B9A6; Thu, 7 Jul 2022 20:02:53 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7192B11B66D; Thu, 7 Jul 2022 20:02:50 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id 08AEB66019E7; Thu, 7 Jul 2022 21:02:49 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224169; bh=yyqoiKo1vjbgBxUDvFfH6fIoAaG7ofkIEVjSYkMkda8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jxcCnHRvMsZooefKE1L+riEhT0DTivOOpHuwBE74HcbdbVkGnvIxNHyOmXJQt/b0H yD9IJJ1DlVtNoeY7VQgePaGz3DfgeXPME8mN0WPZ8sfYPXv9PbKp9uppjJ+iI1Al7M DFWZZVf5MANOJ4mF4+6lhwqQxuuq1xaShFE2tq2i6tV4EJT0/jKGKZryH2r1Kz5SRV weyzCxtpTWONDfevbIkhyMBKebe7EO4uaRuOeRXhsfYQDCoryFPJ2Z2j+NVTm1L9Wr gj1CpHJt0y/TM/Iv0mENn83ah7SITLEhAxYL6OMjLaIdVRwD59yR601W51Jo1gpfCL hw++BJC9wbaWA== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:27 +0000 Message-Id: <20220707200230.1657555-10-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 09/11] drm/i915/selftest: don't attempt engine reset of guc submission engines X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" igt_reset_engines_stolen tries to reset engines without checking if it is possible. Engines using GuC submission are not able to be reset from the host. In this scenario, the reset exits early, then on the next iteration of the each engine loop, the async teardown of the spinner request context's ring occurs while the next engine is under test. This is seen as a stolen memory corruption as the ring buffer was busy initially, but free during the confirmation check and had been poisoned during cleanup. Fix this by not testing GuC submission using engines. Signed-off-by: Robert Beckett --- drivers/gpu/drm/i915/gt/selftest_reset.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c index 37c38bdd5f47..55f3b34e5f6e 100644 --- a/drivers/gpu/drm/i915/gt/selftest_reset.c +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c @@ -194,6 +194,8 @@ static int igt_reset_engines_stolen(void *arg) return 0; for_each_engine(engine, gt, id) { + if (intel_engine_uses_guc(engine)) + continue; err = __igt_reset_stolen(gt, engine->mask, engine->name); if (err) return err; From patchwork Thu Jul 7 20:02:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5FBF6CCA47F for ; Thu, 7 Jul 2022 20:03:11 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DAB3312B9AA; Thu, 7 Jul 2022 20:02:54 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e5ab]) by gabe.freedesktop.org (Postfix) with ESMTPS id 072C211B6D7; Thu, 7 Jul 2022 20:02:51 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id A131D66019EE; Thu, 7 Jul 2022 21:02:49 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224169; bh=4Q2MmpoHgvo6IqR+FhRTJWk4TRp24CULBaAdeSzO30A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JcQ8SBZEpPP8EVYE51rSwLw651bQzWdQSRRUwK1M4rWQYn5HR8crPwf6o8cKWODl0 DbR1AXNNhzUfA5JTgnIiRB36fje01CdfpOXc0UKLYFI0VOVJJZiDRqrnfPKDqDNW5s EC+Y6vaNL2T15/5SglZMBCePcuWrgjqmYRlzWOQ5bMEeo15W1EE6f4boLQh1nPFB98 lj0gphq9lFYno2gjqGknMcPlsFBzgTbUp20r6OeN0TxIDVn+6/ipjGXVly5b7g8BsP TldLyJXAQZiExu1gXaj0VUa/JhQM6r3k8mlRBls158d/E1fsqNdIF+g9iA2fyi+DD6 3Z2HFxjwCl8dA== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:28 +0000 Message-Id: <20220707200230.1657555-11-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 10/11] drm/i915/selftest: wait for requests during engine reset selftest X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" While looping around each engine and testing for corrupted solen memory during engine reset, the old requests from the previous engine can still be yet to retire. To prevent false positive corruption tests, wait for the outstanding requests at the end of the test Signed-off-by: Robert Beckett --- drivers/gpu/drm/i915/gt/selftest_reset.c | 35 ++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c index 55f3b34e5f6e..a2558bc31408 100644 --- a/drivers/gpu/drm/i915/gt/selftest_reset.c +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c @@ -6,6 +6,7 @@ #include #include "gem/i915_gem_stolen.h" +#include "gt/intel_gt.h" #include "i915_memcpy.h" #include "i915_selftest.h" @@ -26,6 +27,7 @@ __igt_reset_stolen(struct intel_gt *gt, intel_wakeref_t wakeref; enum intel_engine_id id; struct igt_spinner spin; + struct i915_request *requests[I915_NUM_ENGINES] = {0}; long max, count; void *tmp; u32 *crc; @@ -68,15 +70,16 @@ __igt_reset_stolen(struct intel_gt *gt, ce = intel_context_create(engine); if (IS_ERR(ce)) { err = PTR_ERR(ce); - goto err_spin; + goto err_requests; } rq = igt_spinner_create_request(&spin, ce, MI_ARB_CHECK); intel_context_put(ce); if (IS_ERR(rq)) { err = PTR_ERR(rq); - goto err_spin; + goto err_requests; } i915_request_add(rq); + requests[id] = i915_request_get(rq); } for (page = 0; page < num_pages; page++) { @@ -165,6 +168,34 @@ __igt_reset_stolen(struct intel_gt *gt, err = -EINVAL; } + /* wait for requests and idle, otherwise cleanup can happen on next loop */ + for (id = 0; id < I915_NUM_ENGINES; id++) { + if (!requests[id]) + continue; + err = i915_request_wait(requests[id], I915_WAIT_INTERRUPTIBLE, HZ); + if (err < 0) { + pr_err("%s failed to wait for rq: %d\n", msg, err); + goto err_requests; + } + + i915_request_put(requests[id]); + requests[id] = NULL; + } + + err = intel_gt_wait_for_idle(gt, HZ); + if (err < 0) { + pr_err("%s failed to wait for gt idle: %d\n", msg, err); + goto err_spin; + } + + err = 0; + +err_requests: + for (id = 0; id < I915_NUM_ENGINES; id++) { + if (!requests[id]) + continue; + i915_request_put(requests[id]); + } err_spin: igt_spinner_fini(&spin); From patchwork Thu Jul 7 20:02:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bob Beckett X-Patchwork-Id: 12910194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CEFE4C43334 for ; Thu, 7 Jul 2022 20:03:19 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F245212BC8C; Thu, 7 Jul 2022 20:02:58 +0000 (UTC) Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3894111B6EC; Thu, 7 Jul 2022 20:02:53 +0000 (UTC) Received: from hermes-devbox.fritz.box (82-71-8-225.dsl.in-addr.zen.co.uk [82.71.8.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbeckett) by madras.collabora.co.uk (Postfix) with ESMTPSA id 8B23C66019F1; Thu, 7 Jul 2022 21:02:51 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1657224171; bh=jZeHaHpiG+CfucQ4kIuFIzT+Z4f61SurVqtlXfpCors=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U/KtqxX1Hpu9qpHL7lz11TkIfrXhS7gXw/kTIuMfL2bw30tFKsfHKMYE3Od9QbQ50 FDL+var6jPeWaFqhy4C7SWCPX1x7LwBW7h1xw0R4mSl3L2NVQJUOTxVAaPpXLUKYrY +ty/6H9iz4juKiZcqwhuUoU27n6PjoKVNlXO6plB0dtWcdUIaHi396sqn3JSV2qB9t g7sSVc1wbIV3lKZDVIXuKFOD/70fmLsByHiPeJXcZJY8cLfNcceeU//jTTZKPyB3V0 5n65DByMS9TexSpciTYwZqBVWgJhn1VMW5R4tan4RPwzq9bL3/JRFiFTwcB0AeCdPo Fcn0lJ4/mCSnQ== From: Robert Beckett To: dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter Date: Thu, 7 Jul 2022 20:02:29 +0000 Message-Id: <20220707200230.1657555-12-bob.beckett@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220707200230.1657555-1-bob.beckett@collabora.com> References: <20220707200230.1657555-1-bob.beckett@collabora.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v10 11/11] drm/i915: stolen memory use ttm backend X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , kernel@collabora.com, Matthew Auld , linux-kernel@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" refactor stolen memory region to use ttm. this necessitates using ttm resources to track reserved stolen regions instead of drm_mm_nodes. Signed-off-by: Robert Beckett --- drivers/gpu/drm/i915/display/intel_fbc.c | 78 ++-- .../gpu/drm/i915/gem/i915_gem_object_types.h | 2 - drivers/gpu/drm/i915/gem/i915_gem_stolen.c | 425 ++++++------------ drivers/gpu/drm/i915/gem/i915_gem_stolen.h | 21 +- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 3 +- drivers/gpu/drm/i915/gem/i915_gem_ttm.h | 7 + drivers/gpu/drm/i915/gt/intel_rc6.c | 4 +- drivers/gpu/drm/i915/gt/selftest_reset.c | 16 +- drivers/gpu/drm/i915/i915_debugfs.c | 7 +- drivers/gpu/drm/i915/i915_drv.h | 6 +- drivers/gpu/drm/i915/intel_region_ttm.c | 42 +- drivers/gpu/drm/i915/intel_region_ttm.h | 8 +- drivers/gpu/drm/i915/selftests/mock_region.c | 12 +- 13 files changed, 280 insertions(+), 351 deletions(-) diff --git a/drivers/gpu/drm/i915/display/intel_fbc.c b/drivers/gpu/drm/i915/display/intel_fbc.c index 8b807284cde1..6f3afac5e8c9 100644 --- a/drivers/gpu/drm/i915/display/intel_fbc.c +++ b/drivers/gpu/drm/i915/display/intel_fbc.c @@ -38,6 +38,7 @@ * forcibly disable it to allow proper screen updates. */ +#include "gem/i915_gem_stolen.h" #include #include @@ -51,6 +52,7 @@ #include "intel_display_types.h" #include "intel_fbc.h" #include "intel_frontbuffer.h" +#include "gem/i915_gem_region.h" #define for_each_fbc_id(__dev_priv, __fbc_id) \ for ((__fbc_id) = INTEL_FBC_A; (__fbc_id) < I915_MAX_FBCS; (__fbc_id)++) \ @@ -92,8 +94,8 @@ struct intel_fbc { struct mutex lock; unsigned int busy_bits; - struct drm_mm_node compressed_fb; - struct drm_mm_node compressed_llb; + struct ttm_resource *compressed_fb; + struct ttm_resource *compressed_llb; enum intel_fbc_id id; @@ -331,16 +333,20 @@ static void i8xx_fbc_nuke(struct intel_fbc *fbc) static void i8xx_fbc_program_cfb(struct intel_fbc *fbc) { struct drm_i915_private *i915 = fbc->i915; + u64 fb_offset = i915_gem_stolen_reserve_offset(fbc->compressed_fb); + u64 llb_offset = i915_gem_stolen_reserve_offset(fbc->compressed_llb); + GEM_BUG_ON(fb_offset == I915_BO_INVALID_OFFSET); + GEM_BUG_ON(llb_offset == I915_BO_INVALID_OFFSET); GEM_BUG_ON(range_overflows_end_t(u64, i915->dsm.start, - fbc->compressed_fb.start, U32_MAX)); + fb_offset, U32_MAX)); GEM_BUG_ON(range_overflows_end_t(u64, i915->dsm.start, - fbc->compressed_llb.start, U32_MAX)); + llb_offset, U32_MAX)); intel_de_write(i915, FBC_CFB_BASE, - i915->dsm.start + fbc->compressed_fb.start); + i915->dsm.start + fb_offset); intel_de_write(i915, FBC_LL_BASE, - i915->dsm.start + fbc->compressed_llb.start); + i915->dsm.start + llb_offset); } static const struct intel_fbc_funcs i8xx_fbc_funcs = { @@ -448,8 +454,10 @@ static bool g4x_fbc_is_compressing(struct intel_fbc *fbc) static void g4x_fbc_program_cfb(struct intel_fbc *fbc) { struct drm_i915_private *i915 = fbc->i915; + u64 fb_offset = i915_gem_stolen_reserve_offset(fbc->compressed_fb); - intel_de_write(i915, DPFC_CB_BASE, fbc->compressed_fb.start); + GEM_BUG_ON(fb_offset == I915_BO_INVALID_OFFSET); + intel_de_write(i915, DPFC_CB_BASE, fb_offset); } static const struct intel_fbc_funcs g4x_fbc_funcs = { @@ -499,8 +507,10 @@ static bool ilk_fbc_is_compressing(struct intel_fbc *fbc) static void ilk_fbc_program_cfb(struct intel_fbc *fbc) { struct drm_i915_private *i915 = fbc->i915; + u64 fb_offset = i915_gem_stolen_reserve_offset(fbc->compressed_fb); - intel_de_write(i915, ILK_DPFC_CB_BASE(fbc->id), fbc->compressed_fb.start); + GEM_BUG_ON(fb_offset == I915_BO_INVALID_OFFSET); + intel_de_write(i915, ILK_DPFC_CB_BASE(fbc->id), fb_offset); } static const struct intel_fbc_funcs ilk_fbc_funcs = { @@ -744,21 +754,24 @@ static int find_compression_limit(struct intel_fbc *fbc, { struct drm_i915_private *i915 = fbc->i915; u64 end = intel_fbc_stolen_end(i915); - int ret, limit = min_limit; + int limit = min_limit; + struct ttm_resource *res; size /= limit; /* Try to over-allocate to reduce reallocations and fragmentation. */ - ret = i915_gem_stolen_insert_node_in_range(i915, &fbc->compressed_fb, - size <<= 1, 4096, 0, end); - if (ret == 0) + res = i915_gem_stolen_reserve_range(i915, size <<= 1, 0, end); + if (!IS_ERR(res)) { + fbc->compressed_fb = res; return limit; + } for (; limit <= intel_fbc_max_limit(i915); limit <<= 1) { - ret = i915_gem_stolen_insert_node_in_range(i915, &fbc->compressed_fb, - size >>= 1, 4096, 0, end); - if (ret == 0) + res = i915_gem_stolen_reserve_range(i915, size >>= 1, 0, end); + if (!IS_ERR(res)) { + fbc->compressed_fb = res; return limit; + } } return 0; @@ -769,17 +782,18 @@ static int intel_fbc_alloc_cfb(struct intel_fbc *fbc, { struct drm_i915_private *i915 = fbc->i915; int ret; + struct ttm_resource *res; - drm_WARN_ON(&i915->drm, - drm_mm_node_allocated(&fbc->compressed_fb)); - drm_WARN_ON(&i915->drm, - drm_mm_node_allocated(&fbc->compressed_llb)); + drm_WARN_ON(&i915->drm, fbc->compressed_fb); + drm_WARN_ON(&i915->drm, fbc->compressed_llb); if (DISPLAY_VER(i915) < 5 && !IS_G4X(i915)) { - ret = i915_gem_stolen_insert_node(i915, &fbc->compressed_llb, - 4096, 4096); - if (ret) + res = i915_gem_stolen_reserve_range(i915, 4096, I915_GEM_STOLEN_BIAS, 0); + if (IS_ERR(res)) { + ret = PTR_ERR(res); goto err; + } + fbc->compressed_llb = res; } ret = find_compression_limit(fbc, size, min_limit); @@ -793,15 +807,14 @@ static int intel_fbc_alloc_cfb(struct intel_fbc *fbc, drm_dbg_kms(&i915->drm, "reserved %llu bytes of contiguous stolen space for FBC, limit: %d\n", - fbc->compressed_fb.size, fbc->limit); + i915_gem_stolen_reserve_size(fbc->compressed_fb), fbc->limit); return 0; err_llb: - if (drm_mm_node_allocated(&fbc->compressed_llb)) - i915_gem_stolen_remove_node(i915, &fbc->compressed_llb); + i915_gem_stolen_release_range(i915, fetch_and_zero(&fbc->compressed_llb)); err: - if (drm_mm_initialized(&i915->mm.stolen)) + if (IS_ERR(res) && (PTR_ERR(res) == -ENOMEM || PTR_ERR(res) == -ENXIO)) drm_info_once(&i915->drm, "not enough stolen space for compressed buffer (need %d more bytes), disabling. Hint: you may be able to increase stolen memory size in the BIOS to avoid this.\n", size); return -ENOSPC; } @@ -826,10 +839,10 @@ static void __intel_fbc_cleanup_cfb(struct intel_fbc *fbc) if (WARN_ON(intel_fbc_hw_is_active(fbc))) return; - if (drm_mm_node_allocated(&fbc->compressed_llb)) - i915_gem_stolen_remove_node(i915, &fbc->compressed_llb); - if (drm_mm_node_allocated(&fbc->compressed_fb)) - i915_gem_stolen_remove_node(i915, &fbc->compressed_fb); + if (fbc->compressed_llb) + i915_gem_stolen_release_range(i915, fetch_and_zero(&fbc->compressed_llb)); + if (fbc->compressed_fb) + i915_gem_stolen_release_range(i915, fetch_and_zero(&fbc->compressed_fb)); } void intel_fbc_cleanup(struct drm_i915_private *i915) @@ -1034,9 +1047,10 @@ static bool intel_fbc_is_cfb_ok(const struct intel_plane_state *plane_state) { struct intel_plane *plane = to_intel_plane(plane_state->uapi.plane); struct intel_fbc *fbc = plane->fbc; + u64 fb_size = i915_gem_stolen_reserve_size(fbc->compressed_fb); return intel_fbc_min_limit(plane_state) <= fbc->limit && - intel_fbc_cfb_size(plane_state) <= fbc->compressed_fb.size * fbc->limit; + intel_fbc_cfb_size(plane_state) <= fb_size * fbc->limit; } static bool intel_fbc_is_ok(const struct intel_plane_state *plane_state) @@ -1702,7 +1716,7 @@ void intel_fbc_init(struct drm_i915_private *i915) { enum intel_fbc_id fbc_id; - if (!drm_mm_initialized(&i915->mm.stolen)) + if (!i915->mm.stolen_region) mkwrite_device_info(i915)->display.fbc_mask = 0; if (need_fbc_vtd_wa(i915)) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h index 07bc11247a3e..13466cca7af8 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h @@ -633,8 +633,6 @@ struct drm_i915_gem_object { } userptr; #endif - struct drm_mm_node *stolen; - resource_size_t bo_offset; unsigned long scratch; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c index 47b5e0e342ab..56152db11440 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c @@ -4,75 +4,101 @@ * Copyright © 2008-2012 Intel Corporation */ +#include "drm/ttm/ttm_placement.h" +#include "gem/i915_gem_object_types.h" #include #include -#include #include #include "gem/i915_gem_lmem.h" #include "gem/i915_gem_region.h" #include "gt/intel_gt.h" #include "gt/intel_region_lmem.h" +#include "gem/i915_gem_ttm.h" #include "i915_drv.h" #include "i915_gem_stolen.h" #include "i915_reg.h" #include "i915_utils.h" #include "i915_vgpu.h" #include "intel_mchbar_regs.h" - -/* - * The BIOS typically reserves some of the system's memory for the exclusive - * use of the integrated graphics. This memory is no longer available for - * use by the OS and so the user finds that his system has less memory - * available than he put in. We refer to this memory as stolen. +#include "intel_region_ttm.h" + +/** + * i915_gem_stolen_reserve_range - reserve a region of space in a given range + * @i915: i915 device instance + * @size: size of region to reserve + * @start: start of search area + * @end: end of search area * - * The BIOS will allocate its framebuffer from the stolen memory. Our - * goal is try to reuse that object for our own fbcon which must always - * be available for panics. Anything else we can reuse the stolen memory - * for is a boon. + * Search for @size amount of free space within the region delimeted by @start and @end. + * If found reserve it from future use until later release with @i915_gem_stolen_release_range. + * + * Return: pointer to resource tracking structure on success, ERR_PTR otherwise */ - -int i915_gem_stolen_insert_node_in_range(struct drm_i915_private *i915, - struct drm_mm_node *node, u64 size, - unsigned alignment, u64 start, u64 end) +struct ttm_resource * +i915_gem_stolen_reserve_range(struct drm_i915_private *i915, + resource_size_t size, + u64 start, u64 end) { - int ret; + struct intel_memory_region *mem = i915->mm.stolen_region; - if (!drm_mm_initialized(&i915->mm.stolen)) - return -ENODEV; - - /* WaSkipStolenMemoryFirstPage:bdw+ */ - if (GRAPHICS_VER(i915) >= 8 && start < 4096) - start = 4096; - - mutex_lock(&i915->mm.stolen_lock); - ret = drm_mm_insert_node_in_range(&i915->mm.stolen, node, - size, alignment, 0, - start, end, DRM_MM_INSERT_BEST); - mutex_unlock(&i915->mm.stolen_lock); + if (!mem) + return ERR_PTR(-ENODEV); + return intel_region_ttm_resource_alloc(mem, size, start, end, I915_BO_ALLOC_CONTIGUOUS); +} - return ret; +/** + * i915_gem_stolen_reserve_offset - return the offset of the reserved space + * @res: pointer to resource tracking structure to check + * + * Return: The offset of the reserved resource, or I915_BO_INVALID_OFFSET on error + */ +u64 i915_gem_stolen_reserve_offset(struct ttm_resource *res) +{ + if (!res) + return I915_BO_INVALID_OFFSET; + return PFN_PHYS(res->start); } -int i915_gem_stolen_insert_node(struct drm_i915_private *i915, - struct drm_mm_node *node, u64 size, - unsigned alignment) +/** + * i915_gem_stolen_reserve_size - return the reserved size of the reserved space + * @res: pointer to resource tracking structure to check + * + * Return: The size of the reserved resource, or I915_BO_INVALID_OFFSET on error + */ +u64 i915_gem_stolen_reserve_size(struct ttm_resource *res) { - return i915_gem_stolen_insert_node_in_range(i915, node, - size, alignment, - I915_GEM_STOLEN_BIAS, - U64_MAX); + if (!res) + return I915_BO_INVALID_OFFSET; + return PFN_PHYS(res->num_pages); } -void i915_gem_stolen_remove_node(struct drm_i915_private *i915, - struct drm_mm_node *node) +/** + * i915_gem_stolen_release_range - release the reserved area to be free for allocation again + * @i915: i915 device instance + * @res: pointer to resource tracking structure allocated via @i915_gem_stolen_reserve_range + */ +void i915_gem_stolen_release_range(struct drm_i915_private *i915, + struct ttm_resource *res) { - mutex_lock(&i915->mm.stolen_lock); - drm_mm_remove_node(node); - mutex_unlock(&i915->mm.stolen_lock); + struct intel_memory_region *mem = i915->mm.stolen_region; + + intel_region_ttm_resource_free(mem, res); } +/* + * The BIOS typically reserves some of the system's memory for the exclusive + * use of the integrated graphics. This memory is no longer available for + * use by the OS and so the user finds that his system has less memory + * available than he put in. We refer to this memory as stolen. + * + * The BIOS will allocate its framebuffer from the stolen memory. Our + * goal is try to reuse that object for our own fbcon which must always + * be available for panics. Anything else we can reuse the stolen memory + * for is a boon. + */ + static int i915_adjust_stolen(struct drm_i915_private *i915, struct resource *dsm) { @@ -173,14 +199,6 @@ static int i915_adjust_stolen(struct drm_i915_private *i915, return 0; } -static void i915_gem_cleanup_stolen(struct drm_i915_private *i915) -{ - if (!drm_mm_initialized(&i915->mm.stolen)) - return; - - drm_mm_takedown(&i915->mm.stolen); -} - static void g4x_get_stolen_reserved(struct drm_i915_private *i915, struct intel_uncore *uncore, resource_size_t *base, @@ -394,8 +412,8 @@ static int i915_gem_init_stolen(struct intel_memory_region *mem) struct intel_uncore *uncore = &i915->uncore; resource_size_t reserved_base, stolen_top; resource_size_t reserved_total, reserved_size; - - mutex_init(&i915->mm.stolen_lock); + struct ttm_resource *resvd; + int ret; if (intel_vgpu_active(i915)) { drm_notice(&i915->drm, @@ -513,229 +531,28 @@ static int i915_gem_init_stolen(struct intel_memory_region *mem) return 0; /* Basic memrange allocator for stolen space. */ - drm_mm_init(&i915->mm.stolen, 0, i915->stolen_usable_size); - - return 0; -} - -static void dbg_poison(struct i915_ggtt *ggtt, - dma_addr_t addr, resource_size_t size, - u8 x) -{ -#if IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) - if (!drm_mm_node_allocated(&ggtt->error_capture)) - return; - - if (ggtt->vm.bind_async_flags & I915_VMA_GLOBAL_BIND) - return; /* beware stop_machine() inversion */ - - GEM_BUG_ON(!IS_ALIGNED(size, PAGE_SIZE)); - - mutex_lock(&ggtt->error_mutex); - while (size) { - void __iomem *s; - - ggtt->vm.insert_page(&ggtt->vm, addr, - ggtt->error_capture.start, - I915_CACHE_NONE, 0); - mb(); - - s = io_mapping_map_wc(&ggtt->iomap, - ggtt->error_capture.start, - PAGE_SIZE); - memset_io(s, x, PAGE_SIZE); - io_mapping_unmap(s); - - addr += PAGE_SIZE; - size -= PAGE_SIZE; - } - mb(); - ggtt->vm.clear_range(&ggtt->vm, ggtt->error_capture.start, PAGE_SIZE); - mutex_unlock(&ggtt->error_mutex); -#endif -} - -static struct sg_table * -i915_pages_create_for_stolen(struct drm_device *dev, - resource_size_t offset, resource_size_t size) -{ - struct drm_i915_private *i915 = to_i915(dev); - struct sg_table *st; - struct scatterlist *sg; - - GEM_BUG_ON(range_overflows(offset, size, resource_size(&i915->dsm))); - - /* We hide that we have no struct page backing our stolen object - * by wrapping the contiguous physical allocation with a fake - * dma mapping in a single scatterlist. - */ - - st = kmalloc(sizeof(*st), GFP_KERNEL); - if (st == NULL) - return ERR_PTR(-ENOMEM); - - if (sg_alloc_table(st, 1, GFP_KERNEL)) { - kfree(st); - return ERR_PTR(-ENOMEM); - } - - sg = st->sgl; - sg->offset = 0; - sg->length = size; - - sg_dma_address(sg) = (dma_addr_t)i915->dsm.start + offset; - sg_dma_len(sg) = size; - - return st; -} - -static int i915_gem_object_get_pages_stolen(struct drm_i915_gem_object *obj) -{ - struct drm_i915_private *i915 = to_i915(obj->base.dev); - struct sg_table *pages = - i915_pages_create_for_stolen(obj->base.dev, - obj->stolen->start, - obj->stolen->size); - if (IS_ERR(pages)) - return PTR_ERR(pages); - - dbg_poison(to_gt(i915)->ggtt, - sg_dma_address(pages->sgl), - sg_dma_len(pages->sgl), - POISON_INUSE); - - __i915_gem_object_set_pages(obj, pages, obj->stolen->size); - - return 0; -} - -static void i915_gem_object_put_pages_stolen(struct drm_i915_gem_object *obj, - struct sg_table *pages) -{ - struct drm_i915_private *i915 = to_i915(obj->base.dev); - /* Should only be called from i915_gem_object_release_stolen() */ - - dbg_poison(to_gt(i915)->ggtt, - sg_dma_address(pages->sgl), - sg_dma_len(pages->sgl), - POISON_FREE); - - sg_free_table(pages); - kfree(pages); -} - -static void -i915_gem_object_release_stolen(struct drm_i915_gem_object *obj) -{ - struct drm_i915_private *i915 = to_i915(obj->base.dev); - struct drm_mm_node *stolen = fetch_and_zero(&obj->stolen); - - GEM_BUG_ON(!stolen); - i915_gem_stolen_remove_node(i915, stolen); - kfree(stolen); - - i915_gem_object_release_memory_region(obj); -} - -static const struct drm_i915_gem_object_ops i915_gem_object_stolen_ops = { - .name = "i915_gem_object_stolen", - .get_pages = i915_gem_object_get_pages_stolen, - .put_pages = i915_gem_object_put_pages_stolen, - .release = i915_gem_object_release_stolen, -}; - -static int __i915_gem_object_create_stolen(struct intel_memory_region *mem, - struct drm_i915_gem_object *obj, - struct drm_mm_node *stolen) -{ - static struct lock_class_key lock_class; - unsigned int cache_level; - unsigned int flags; - int err; - - /* - * Stolen objects are always physically contiguous since we just - * allocate one big block underneath using the drm_mm range allocator. - */ - flags = I915_BO_ALLOC_CONTIGUOUS; - - drm_gem_private_object_init(&mem->i915->drm, &obj->base, stolen->size); - i915_gem_object_init(obj, &i915_gem_object_stolen_ops, &lock_class, flags); - - obj->stolen = stolen; - obj->read_domains = I915_GEM_DOMAIN_CPU | I915_GEM_DOMAIN_GTT; - cache_level = HAS_LLC(mem->i915) ? I915_CACHE_LLC : I915_CACHE_NONE; - i915_gem_object_set_cache_coherency(obj, cache_level); - - if (WARN_ON(!i915_gem_object_trylock(obj, NULL))) - return -EBUSY; - - i915_gem_object_init_memory_region(obj, mem); - - err = i915_gem_object_pin_pages(obj); - if (err) - i915_gem_object_release_memory_region(obj); - i915_gem_object_unlock(obj); - - return err; -} - -static int _i915_gem_object_stolen_init(struct intel_memory_region *mem, - struct drm_i915_gem_object *obj, - resource_size_t offset, - resource_size_t size, - resource_size_t page_size, - unsigned int flags) -{ - struct drm_i915_private *i915 = mem->i915; - struct drm_mm_node *stolen; - int ret; - - if (!drm_mm_initialized(&i915->mm.stolen)) - return -ENODEV; - - if (size == 0) - return -EINVAL; + ret = intel_region_ttm_init(mem); + if (ret) + return ret; /* - * With discrete devices, where we lack a mappable aperture there is no - * possible way to ever access this memory on the CPU side. + * Reserve the bias area. Nothing should be allocating that region. + * Also covers WaSkipStolenMemoryFirstPage:bdw+ */ - if (mem->type == INTEL_MEMORY_STOLEN_LOCAL && !mem->io_size && - !(flags & I915_BO_ALLOC_GPU_ONLY)) - return -ENOSPC; - - stolen = kzalloc(sizeof(*stolen), GFP_KERNEL); - if (!stolen) - return -ENOMEM; - - if (offset != I915_BO_INVALID_OFFSET) { - drm_dbg(&i915->drm, - "creating preallocated stolen object: stolen_offset=%pa, size=%pa\n", - &offset, &size); - - stolen->start = offset; - stolen->size = size; - mutex_lock(&i915->mm.stolen_lock); - ret = drm_mm_reserve_node(&i915->mm.stolen, stolen); - mutex_unlock(&i915->mm.stolen_lock); - } else { - ret = i915_gem_stolen_insert_node(i915, stolen, size, - mem->min_page_size); + resvd = intel_region_ttm_resource_alloc(mem, I915_GEM_STOLEN_BIAS, + 0, I915_GEM_STOLEN_BIAS, 0); + if (IS_ERR(resvd)) { + ret = PTR_ERR(resvd); + goto err_no_reserve; } - if (ret) - goto err_free; - ret = __i915_gem_object_create_stolen(mem, obj, stolen); - if (ret) - goto err_remove; + i915->bias_reserved = resvd; - return 0; + return ret; + +err_no_reserve: + intel_region_ttm_fini(mem); -err_remove: - i915_gem_stolen_remove_node(i915, stolen); -err_free: - kfree(stolen); return ret; } @@ -743,7 +560,8 @@ struct drm_i915_gem_object * i915_gem_object_create_stolen(struct drm_i915_private *i915, resource_size_t size) { - return i915_gem_object_create_region(i915->mm.stolen_region, size, 0, 0); + return i915_gem_object_create_region(i915->mm.stolen_region, size, 0, + I915_BO_ALLOC_CONTIGUOUS | I915_BO_ALLOC_PINNED); } static int init_stolen_smem(struct intel_memory_region *mem) @@ -755,16 +573,36 @@ static int init_stolen_smem(struct intel_memory_region *mem) return i915_gem_init_stolen(mem); } -static int release_stolen_smem(struct intel_memory_region *mem) +static int release_stolen(struct intel_memory_region *mem) { - i915_gem_cleanup_stolen(mem->i915); - return 0; + if (mem->i915->bias_reserved) + intel_region_ttm_resource_free(mem, mem->i915->bias_reserved); + + return intel_region_ttm_fini(mem); +} + +static int stolen_object_init(struct intel_memory_region *mem, + struct drm_i915_gem_object *obj, + resource_size_t offset, + resource_size_t size, + resource_size_t page_size, + unsigned int flags) +{ + if (!mem->region_private) + return -ENODEV; + + if (size == 0) + return -EINVAL; + + /* default range manager relies on page_size for alignment */ + page_size = max(page_size, mem->min_page_size); + return __i915_gem_ttm_object_init(mem, obj, offset, size, page_size, flags); } static const struct intel_memory_region_ops i915_region_stolen_smem_ops = { .init = init_stolen_smem, - .release = release_stolen_smem, - .init_object = _i915_gem_object_stolen_init, + .release = release_stolen, + .init_object = stolen_object_init, }; static int init_stolen_lmem(struct intel_memory_region *mem) @@ -793,7 +631,7 @@ static int init_stolen_lmem(struct intel_memory_region *mem) return 0; err_cleanup: - i915_gem_cleanup_stolen(mem->i915); + intel_region_ttm_fini(mem); return err; } @@ -801,14 +639,13 @@ static int release_stolen_lmem(struct intel_memory_region *mem) { if (mem->io_size) io_mapping_fini(&mem->iomap); - i915_gem_cleanup_stolen(mem->i915); - return 0; + return release_stolen(mem); } static const struct intel_memory_region_ops i915_region_stolen_lmem_ops = { .init = init_stolen_lmem, .release = release_stolen_lmem, - .init_object = _i915_gem_object_stolen_init, + .init_object = stolen_object_init, }; struct intel_memory_region * @@ -896,7 +733,37 @@ i915_gem_stolen_smem_setup(struct drm_i915_private *i915, u16 type, return mem; } -bool i915_gem_object_is_stolen(const struct drm_i915_gem_object *obj) +/** + * i915_gem_object_stolen_offset - return offset from start of stolen region + * @obj: the object to return the offset of + * + * Get the offset from stolen region if this object is currently placed in stolen memory. + * + * Return: offset from stolen if successful, I915_BO_INVALID_OFFSET otherwise + */ +u64 i915_gem_object_stolen_offset(struct drm_i915_gem_object *obj) +{ + struct ttm_buffer_object *ttm_obj; + + if (!obj || !i915_gem_object_is_stolen(obj)) + return I915_BO_INVALID_OFFSET; + + ttm_obj = i915_gem_to_ttm(obj); + if (ttm_obj->resource->mem_type != I915_PL_STOLEN) + return I915_BO_INVALID_OFFSET; + + return PFN_PHYS(ttm_obj->resource->start); +} + +bool i915_gem_object_is_stolen(struct drm_i915_gem_object *obj) { - return obj->ops == &i915_gem_object_stolen_ops; + struct intel_memory_region *mr = READ_ONCE(obj->mm.region); + +#ifdef CONFIG_LOCKDEP + if (i915_gem_object_migratable(obj) && + i915_gem_object_evictable(obj)) + assert_object_held(obj); +#endif + return mr && (mr->type == INTEL_MEMORY_STOLEN_SYSTEM || + mr->type == INTEL_MEMORY_STOLEN_LOCAL); } diff --git a/drivers/gpu/drm/i915/gem/i915_gem_stolen.h b/drivers/gpu/drm/i915/gem/i915_gem_stolen.h index d5005a39d130..b39cb6e6c768 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_stolen.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_stolen.h @@ -10,17 +10,9 @@ struct drm_i915_private; struct drm_mm_node; +struct ttm_resource; struct drm_i915_gem_object; -int i915_gem_stolen_insert_node(struct drm_i915_private *dev_priv, - struct drm_mm_node *node, u64 size, - unsigned alignment); -int i915_gem_stolen_insert_node_in_range(struct drm_i915_private *dev_priv, - struct drm_mm_node *node, u64 size, - unsigned alignment, u64 start, - u64 end); -void i915_gem_stolen_remove_node(struct drm_i915_private *dev_priv, - struct drm_mm_node *node); struct intel_memory_region * i915_gem_stolen_smem_setup(struct drm_i915_private *i915, u16 type, u16 instance); @@ -32,7 +24,16 @@ struct drm_i915_gem_object * i915_gem_object_create_stolen(struct drm_i915_private *dev_priv, resource_size_t size); -bool i915_gem_object_is_stolen(const struct drm_i915_gem_object *obj); +u64 i915_gem_object_stolen_offset(struct drm_i915_gem_object *obj); +bool i915_gem_object_is_stolen(struct drm_i915_gem_object *obj); +struct ttm_resource * +i915_gem_stolen_reserve_range(struct drm_i915_private *i915, + resource_size_t size, + u64 start, u64 end); +u64 i915_gem_stolen_reserve_offset(struct ttm_resource *res); +u64 i915_gem_stolen_reserve_size(struct ttm_resource *res); +void i915_gem_stolen_release_range(struct drm_i915_private *i915, + struct ttm_resource *res); #define I915_GEM_STOLEN_BIAS SZ_128K diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index bb988608296d..c7611efddcf7 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -1203,6 +1203,7 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, struct ttm_place _place; struct ttm_placement _placement; struct ttm_placement *placement; + bool is_stolen = intel_region_to_ttm_type(mem) == I915_PL_STOLEN; int ret; drm_gem_private_object_init(&i915->drm, &obj->base, size); @@ -1222,7 +1223,7 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, obj->base.vma_node.driver_private = i915_gem_to_ttm(obj); /* Forcing the page size is kernel internal only */ - GEM_BUG_ON(page_size && obj->mm.n_placements); + GEM_BUG_ON(page_size && obj->mm.n_placements && !is_stolen); /* * Keep an extra shrink pin to prevent the object from being made diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.h b/drivers/gpu/drm/i915/gem/i915_gem_ttm.h index 73e371aa3850..81654a51df51 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.h @@ -49,6 +49,13 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem, resource_size_t size, resource_size_t page_size, unsigned int flags); +int i915_gem_ttm_object_init_in_place(struct intel_memory_region *mem, + struct drm_i915_gem_object *obj, + resource_size_t size, + resource_size_t page_size, + unsigned int flags, + u64 start, + u64 end); /* Internal I915 TTM declarations and definitions below. */ diff --git a/drivers/gpu/drm/i915/gt/intel_rc6.c b/drivers/gpu/drm/i915/gt/intel_rc6.c index f8d0523f4c18..846eef898f7e 100644 --- a/drivers/gpu/drm/i915/gt/intel_rc6.c +++ b/drivers/gpu/drm/i915/gt/intel_rc6.c @@ -355,9 +355,9 @@ static int vlv_rc6_init(struct intel_rc6 *rc6) GEM_BUG_ON(range_overflows_end_t(u64, i915->dsm.start, - pctx->stolen->start, + i915_gem_object_stolen_offset(pctx), U32_MAX)); - pctx_paddr = i915->dsm.start + pctx->stolen->start; + pctx_paddr = i915->dsm.start + i915_gem_object_stolen_offset(pctx); intel_uncore_write(uncore, VLV_PCBR, pctx_paddr); out: diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c index a2558bc31408..1b25e976b584 100644 --- a/drivers/gpu/drm/i915/gt/selftest_reset.c +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c @@ -7,6 +7,7 @@ #include "gem/i915_gem_stolen.h" #include "gt/intel_gt.h" +#include "intel_region_ttm.h" #include "i915_memcpy.h" #include "i915_selftest.h" @@ -86,6 +87,7 @@ __igt_reset_stolen(struct intel_gt *gt, dma_addr_t dma = (dma_addr_t)dsm->start + (page << PAGE_SHIFT); void __iomem *s; void *in; + bool busy; ggtt->vm.insert_page(&ggtt->vm, dma, ggtt->error_capture.start, @@ -96,9 +98,9 @@ __igt_reset_stolen(struct intel_gt *gt, ggtt->error_capture.start, PAGE_SIZE); - if (!__drm_mm_interval_first(>->i915->mm.stolen, - page << PAGE_SHIFT, - ((page + 1) << PAGE_SHIFT) - 1)) + busy = intel_region_ttm_range_busy(gt->i915->mm.stolen_region, + PFN_PHYS(page), PAGE_SIZE); + if (!busy) memset_io(s, STACK_MAGIC, PAGE_SIZE); in = (void __force *)s; @@ -127,6 +129,7 @@ __igt_reset_stolen(struct intel_gt *gt, void __iomem *s; void *in; u32 x; + bool busy; ggtt->vm.insert_page(&ggtt->vm, dma, ggtt->error_capture.start, @@ -142,10 +145,9 @@ __igt_reset_stolen(struct intel_gt *gt, in = tmp; x = crc32_le(0, in, PAGE_SIZE); - if (x != crc[page] && - !__drm_mm_interval_first(>->i915->mm.stolen, - page << PAGE_SHIFT, - ((page + 1) << PAGE_SHIFT) - 1)) { + busy = intel_region_ttm_range_busy(gt->i915->mm.stolen_region, + PFN_PHYS(page), PAGE_SIZE); + if (x != crc[page] && !busy) { pr_debug("unused stolen page %pa modified by GPU reset\n", &page); if (count++ == 0) diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c index 94e5c29d2ee3..e538b8f71dcc 100644 --- a/drivers/gpu/drm/i915/i915_debugfs.c +++ b/drivers/gpu/drm/i915/i915_debugfs.c @@ -32,6 +32,7 @@ #include +#include "gem/i915_gem_region.h" #include "gem/i915_gem_context.h" #include "gt/intel_gt.h" #include "gt/intel_gt_buffer_pool.h" @@ -157,6 +158,7 @@ i915_debugfs_describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj) struct drm_i915_private *dev_priv = to_i915(obj->base.dev); struct i915_vma *vma; int pin_count = 0; + u64 offset; seq_printf(m, "%pK: %c%c%c %8zdKiB %02x %02x %s%s%s", &obj->base, @@ -241,8 +243,9 @@ i915_debugfs_describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj) spin_unlock(&obj->vma.lock); seq_printf(m, " (pinned x %d)", pin_count); - if (i915_gem_object_is_stolen(obj)) - seq_printf(m, " (stolen: %08llx)", obj->stolen->start); + offset = i915_gem_object_stolen_offset(obj); + if (offset != I915_BO_INVALID_OFFSET) + seq_printf(m, " (stolen: %08llx)", offset); if (i915_gem_object_is_framebuffer(obj)) seq_printf(m, " (fb)"); } diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index eba94fa76b18..c3eab85177e4 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -224,11 +224,6 @@ struct i915_gem_mm { * support stolen. */ struct intel_memory_region *stolen_region; - /** Memory allocator for GTT stolen memory */ - struct drm_mm stolen; - /** Protects the usage of the GTT stolen memory allocator. This is - * always the inner lock when overlapping with struct_mutex. */ - struct mutex stolen_lock; /* Protects bound_list/unbound_list and #drm_i915_gem_object.mm.link */ spinlock_t obj_lock; @@ -396,6 +391,7 @@ struct drm_i915_private { * Reseved portion of Data Stolen Memory */ struct resource dsm_reserved; + struct ttm_resource *bias_reserved; /* * Stolen memory is segmented in hardware with different portions diff --git a/drivers/gpu/drm/i915/intel_region_ttm.c b/drivers/gpu/drm/i915/intel_region_ttm.c index 694e9acb69e2..b3850188981b 100644 --- a/drivers/gpu/drm/i915/intel_region_ttm.c +++ b/drivers/gpu/drm/i915/intel_region_ttm.c @@ -194,11 +194,12 @@ intel_region_ttm_resource_to_rsgt(struct intel_memory_region *mem, } } -#ifdef CONFIG_DRM_I915_SELFTEST /** * intel_region_ttm_resource_alloc - Allocate memory resources from a region * @mem: The memory region, * @size: The requested size in bytes + * @start: start of allowed range + * @end: end of allowed range * @flags: Allocation flags * * This functionality is provided only for callers that need to allocate @@ -212,8 +213,9 @@ intel_region_ttm_resource_to_rsgt(struct intel_memory_region *mem, */ struct ttm_resource * intel_region_ttm_resource_alloc(struct intel_memory_region *mem, - resource_size_t offset, resource_size_t size, + u64 start, + u64 end, unsigned int flags) { struct ttm_resource_manager *man = mem->region_private; @@ -222,11 +224,14 @@ intel_region_ttm_resource_alloc(struct intel_memory_region *mem, struct ttm_resource *res; int ret; + if (!man) + return ERR_PTR(-ENODEV); + if (flags & I915_BO_ALLOC_CONTIGUOUS) place.flags |= TTM_PL_FLAG_CONTIGUOUS; - if (offset != I915_BO_INVALID_OFFSET) { - place.fpfn = offset >> PAGE_SHIFT; - place.lpfn = place.fpfn + (size >> PAGE_SHIFT); + if (start || end) { + place.fpfn = PFN_DOWN(start); + place.lpfn = PFN_UP(end); } else if (mem->io_size && mem->io_size < mem->total) { if (flags & I915_BO_ALLOC_GPU_ONLY) { place.flags |= TTM_PL_FLAG_TOPDOWN; @@ -247,8 +252,6 @@ intel_region_ttm_resource_alloc(struct intel_memory_region *mem, return ret ? ERR_PTR(ret) : res; } -#endif - /** * intel_region_ttm_resource_free - Free a resource allocated from a resource manager * @mem: The region the resource was allocated from. @@ -260,9 +263,34 @@ void intel_region_ttm_resource_free(struct intel_memory_region *mem, struct ttm_resource_manager *man = mem->region_private; struct ttm_buffer_object mock_bo = {}; + if (!man) + return; + mock_bo.base.size = res->num_pages << PAGE_SHIFT; mock_bo.bdev = &mem->i915->bdev; res->bo = &mock_bo; man->func->free(man, res); } + +/** + * intel_region_ttm_range_busy - check whether range has any allocations + * @mem: The region to check + * @start: the start of the range to check + * @size: size of the range to check + * + * Return: true if something is alloceted within the region, false otherwise. + */ +bool intel_region_ttm_range_busy(struct intel_memory_region *mem, + u64 start, u64 size) +{ + struct ttm_resource *dummy; + + dummy = intel_region_ttm_resource_alloc(mem, size, start, start + size, + I915_BO_ALLOC_CONTIGUOUS); + if (IS_ERR(dummy)) + return true; + + intel_region_ttm_resource_free(mem, dummy); + return false; +} diff --git a/drivers/gpu/drm/i915/intel_region_ttm.h b/drivers/gpu/drm/i915/intel_region_ttm.h index cf9d86dcf409..1e88472fb2ea 100644 --- a/drivers/gpu/drm/i915/intel_region_ttm.h +++ b/drivers/gpu/drm/i915/intel_region_ttm.h @@ -29,15 +29,17 @@ intel_region_ttm_resource_to_rsgt(struct intel_memory_region *mem, void intel_region_ttm_resource_free(struct intel_memory_region *mem, struct ttm_resource *res); +bool intel_region_ttm_range_busy(struct intel_memory_region *mem, + u64 start, u64 size); + int intel_region_to_ttm_type(const struct intel_memory_region *mem); struct ttm_device_funcs *i915_ttm_driver(void); -#ifdef CONFIG_DRM_I915_SELFTEST struct ttm_resource * intel_region_ttm_resource_alloc(struct intel_memory_region *mem, - resource_size_t offset, resource_size_t size, + u64 start, + u64 end, unsigned int flags); -#endif #endif /* _INTEL_REGION_TTM_H_ */ diff --git a/drivers/gpu/drm/i915/selftests/mock_region.c b/drivers/gpu/drm/i915/selftests/mock_region.c index 670557ce1024..a41d81dc345d 100644 --- a/drivers/gpu/drm/i915/selftests/mock_region.c +++ b/drivers/gpu/drm/i915/selftests/mock_region.c @@ -24,10 +24,20 @@ static int mock_region_get_pages(struct drm_i915_gem_object *obj) { struct sg_table *pages; int err; + u64 start, end; + + if (obj->bo_offset == I915_BO_INVALID_OFFSET) { + start = 0; + end = 0; + } else { + start = obj->bo_offset; + end = obj->bo_offset + obj->base.size; + } obj->mm.res = intel_region_ttm_resource_alloc(obj->mm.region, - obj->bo_offset, obj->base.size, + start, + end, obj->flags); if (IS_ERR(obj->mm.res)) return PTR_ERR(obj->mm.res);