From patchwork Mon Jan 24 12:25:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722020 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 82E90C433F5 for ; Mon, 24 Jan 2022 12:25:23 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 40E3910E7BB; Mon, 24 Jan 2022 12:25:19 +0000 (UTC) Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8E83C10E675 for ; Mon, 24 Jan 2022 12:25:18 +0000 (UTC) Received: by mail-wm1-x330.google.com with SMTP id q141-20020a1ca793000000b00347b48dfb53so27617820wme.0 for ; Mon, 24 Jan 2022 04:25:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=NBshbzgaVx5WjPSxSIkOCrolEfFyFf8tLsKK29eP3YE=; b=qUsSN5A8pDN5le0vGd62RUBS83SU8nE8Lp2+owpdHx5Nua6SWA7+5ME/1H7vat1ZZ+ 5jDGDH1/i9aLoBqvhkqU9rHhdwUHvT5otzRAJARIjizjpSt68XZOhXo9Zd6iAUmJK8rk eyYME8zSXYt590wg+JIXUrpa3ayNCD980J50WwR4fbdkGKEzav7Gcgxyg2QuNqDtbbAZ MYxO6yBloqRQSnn0G35g8Xx/WP6hBQs5YKQnPpOR0tTqWkGV1axFBB1l8KYfzOION/RE rtFbFaK7S5PRVZJKpzJ+yUVrpCZwR/WG3yGkffxSWFhUTKKQyCDVb48xQpuCsAgOPIpG 3KDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NBshbzgaVx5WjPSxSIkOCrolEfFyFf8tLsKK29eP3YE=; b=3C+3rMvPE2rzvt+Nne5PYwpVa4E03AXRM/zZ4SzVklNKHDfGb3TVa1z7+r0aaFQXbJ /W9NEaVseshGHqyMq0ln48hl2pvjOLgpWsL/fggjpr+JnPo2X0agXM0WGjN5z2f7yOPC r4ip2gRKaPvGaKPSzchfXLSfYex6C26FZc7G3gLMXkMOlPihAAgqI3jZGJhnwF5GpyUc yvAvBGG4dG8DhcQOjQGf1zNwUoiHA8FV0SWUhWfhQrf4F9K3b6On1xjuNSRPGQr3ZoIA KFABhevODKnkadJLF/spLlT02d2oVgqfVixlYMMZbCdeNImFE8D1JzaIO6Q+XeXSV52+ uqbg== X-Gm-Message-State: AOAM533yKqrd/vjOQV99mSWmidL3LxgG6N014Y3tkGZJ8ZrOg7peJckh ITQXZq3XI58GL5FDJ5qsFpc= X-Google-Smtp-Source: ABdhPJz83GqZ4G8qVDs65XKaoO4W6b0p/FC1+6KpGwuf3ZYhJjEse5g7rdDsPW6LTBVsGo20mzD+vQ== X-Received: by 2002:a7b:cdfa:: with SMTP id p26mr1628425wmj.132.1643027117128; Mon, 24 Jan 2022 04:25:17 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:16 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 01/12] drm/ttm: add ttm_resource_fini Date: Mon, 24 Jan 2022 13:25:03 +0100 Message-Id: <20220124122514.1832-2-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Make sure we call the common cleanup function in all implementations of the resource manager. Signed-off-by: Christian König Reviewed-by: Daniel Vetter --- drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 2 ++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 1 + drivers/gpu/drm/nouveau/nouveau_mem.c | 3 ++- drivers/gpu/drm/nouveau/nouveau_mem.h | 3 ++- drivers/gpu/drm/nouveau/nouveau_ttm.c | 9 +++++---- drivers/gpu/drm/ttm/ttm_range_manager.c | 2 ++ drivers/gpu/drm/ttm/ttm_resource.c | 6 ++++++ drivers/gpu/drm/ttm/ttm_sys_manager.c | 1 + drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c | 2 ++ drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c | 1 + include/drm/ttm/ttm_resource.h | 3 +++ 13 files changed, 30 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 675a72ef305d..ea5470c8c921 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -169,6 +169,7 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, return 0; err_free: + ttm_resource_fini(man, &node->base.base); kfree(node); err_out: @@ -200,6 +201,7 @@ static void amdgpu_gtt_mgr_del(struct ttm_resource_manager *man, if (!(res->placement & TTM_PL_FLAG_TEMPORARY)) atomic64_sub(res->num_pages, &mgr->used); + ttm_resource_fini(man, res); kfree(node); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c index d02c8637f909..ffddec08e931 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c @@ -95,6 +95,7 @@ static void amdgpu_preempt_mgr_del(struct ttm_resource_manager *man, struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man); atomic64_sub(res->num_pages, &mgr->used); + ttm_resource_fini(man, res); kfree(res); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 7b2b0980ec41..55d68408951d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -476,6 +476,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, while (i--) drm_mm_remove_node(&node->mm_nodes[i]); spin_unlock(&mgr->lock); + ttm_resource_fini(man, &node->base); kvfree(node); error_sub: @@ -515,6 +516,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, atomic64_sub(usage, &mgr->usage); atomic64_sub(vis_usage, &mgr->vis_usage); + ttm_resource_fini(man, res); kvfree(node); } diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index 6ba314f9836a..bf857703c797 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -124,6 +124,7 @@ static void i915_ttm_buddy_man_free(struct ttm_resource_manager *man, drm_buddy_free_list(&bman->mm, &bman_res->blocks); mutex_unlock(&bman->lock); + ttm_resource_fini(man, res); kfree(bman_res); } diff --git a/drivers/gpu/drm/nouveau/nouveau_mem.c b/drivers/gpu/drm/nouveau/nouveau_mem.c index 2ca3207c13fc..2e517cdc24c9 100644 --- a/drivers/gpu/drm/nouveau/nouveau_mem.c +++ b/drivers/gpu/drm/nouveau/nouveau_mem.c @@ -162,11 +162,12 @@ nouveau_mem_vram(struct ttm_resource *reg, bool contig, u8 page) } void -nouveau_mem_del(struct ttm_resource *reg) +nouveau_mem_del(struct ttm_resource_manager *man, struct ttm_resource *reg) { struct nouveau_mem *mem = nouveau_mem(reg); nouveau_mem_fini(mem); + ttm_resource_fini(man, reg); kfree(mem); } diff --git a/drivers/gpu/drm/nouveau/nouveau_mem.h b/drivers/gpu/drm/nouveau/nouveau_mem.h index 2c01166a90f2..325551eba5cd 100644 --- a/drivers/gpu/drm/nouveau/nouveau_mem.h +++ b/drivers/gpu/drm/nouveau/nouveau_mem.h @@ -23,7 +23,8 @@ nouveau_mem(struct ttm_resource *reg) int nouveau_mem_new(struct nouveau_cli *, u8 kind, u8 comp, struct ttm_resource **); -void nouveau_mem_del(struct ttm_resource *); +void nouveau_mem_del(struct ttm_resource_manager *man, + struct ttm_resource *); int nouveau_mem_vram(struct ttm_resource *, bool contig, u8 page); int nouveau_mem_host(struct ttm_resource *, struct ttm_tt *); void nouveau_mem_fini(struct nouveau_mem *); diff --git a/drivers/gpu/drm/nouveau/nouveau_ttm.c b/drivers/gpu/drm/nouveau/nouveau_ttm.c index 2ca9d9a9e5d5..91ef33f8f22c 100644 --- a/drivers/gpu/drm/nouveau/nouveau_ttm.c +++ b/drivers/gpu/drm/nouveau/nouveau_ttm.c @@ -36,9 +36,10 @@ #include static void -nouveau_manager_del(struct ttm_resource_manager *man, struct ttm_resource *reg) +nouveau_manager_del(struct ttm_resource_manager *man, + struct ttm_resource *reg) { - nouveau_mem_del(reg); + nouveau_mem_del(man, reg); } static int @@ -62,7 +63,7 @@ nouveau_vram_manager_new(struct ttm_resource_manager *man, ret = nouveau_mem_vram(*res, nvbo->contig, nvbo->page); if (ret) { - nouveau_mem_del(*res); + nouveau_mem_del(man, *res); return ret; } @@ -118,7 +119,7 @@ nv04_gart_manager_new(struct ttm_resource_manager *man, ret = nvif_vmm_get(&mem->cli->vmm.vmm, PTES, false, 12, 0, (long)(*res)->num_pages << PAGE_SHIFT, &mem->vma[0]); if (ret) { - nouveau_mem_del(*res); + nouveau_mem_del(man, *res); return ret; } diff --git a/drivers/gpu/drm/ttm/ttm_range_manager.c b/drivers/gpu/drm/ttm/ttm_range_manager.c index 072e0baf2ab4..55a41355f684 100644 --- a/drivers/gpu/drm/ttm/ttm_range_manager.c +++ b/drivers/gpu/drm/ttm/ttm_range_manager.c @@ -89,6 +89,7 @@ static int ttm_range_man_alloc(struct ttm_resource_manager *man, spin_unlock(&rman->lock); if (unlikely(ret)) { + ttm_resource_fini(man, *res); kfree(node); return ret; } @@ -108,6 +109,7 @@ static void ttm_range_man_free(struct ttm_resource_manager *man, drm_mm_remove_node(&node->mm_nodes[0]); spin_unlock(&rman->lock); + ttm_resource_fini(man, res); kfree(node); } diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index 035d71332d18..89bcfe22a0ca 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -44,6 +44,12 @@ void ttm_resource_init(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_resource_init); +void ttm_resource_fini(struct ttm_resource_manager *man, + struct ttm_resource *res) +{ +} +EXPORT_SYMBOL(ttm_resource_fini); + int ttm_resource_alloc(struct ttm_buffer_object *bo, const struct ttm_place *place, struct ttm_resource **res_ptr) diff --git a/drivers/gpu/drm/ttm/ttm_sys_manager.c b/drivers/gpu/drm/ttm/ttm_sys_manager.c index 63aca52f75e1..135394dcca95 100644 --- a/drivers/gpu/drm/ttm/ttm_sys_manager.c +++ b/drivers/gpu/drm/ttm/ttm_sys_manager.c @@ -23,6 +23,7 @@ static int ttm_sys_man_alloc(struct ttm_resource_manager *man, static void ttm_sys_man_free(struct ttm_resource_manager *man, struct ttm_resource *res) { + ttm_resource_fini(man, res); kfree(res); } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c b/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c index ebb4505a31a3..99ccf690f8a1 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c @@ -117,6 +117,7 @@ static int vmw_gmrid_man_get_node(struct ttm_resource_manager *man, gman->used_gmr_pages -= (*res)->num_pages; spin_unlock(&gman->lock); ida_free(&gman->gmr_ida, id); + ttm_resource_fini(man, *res); kfree(*res); return -ENOSPC; } @@ -130,6 +131,7 @@ static void vmw_gmrid_man_put_node(struct ttm_resource_manager *man, spin_lock(&gman->lock); gman->used_gmr_pages -= res->num_pages; spin_unlock(&gman->lock); + ttm_resource_fini(man, res); kfree(res); } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c b/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c index b0005b03a617..a64188c7268d 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c @@ -49,6 +49,7 @@ static int vmw_sys_man_alloc(struct ttm_resource_manager *man, static void vmw_sys_man_free(struct ttm_resource_manager *man, struct ttm_resource *res) { + ttm_resource_fini(man, res); kfree(res); } diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index 5952051091cd..df1f06b7b504 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -261,6 +261,9 @@ ttm_resource_manager_cleanup(struct ttm_resource_manager *man) void ttm_resource_init(struct ttm_buffer_object *bo, const struct ttm_place *place, struct ttm_resource *res); +void ttm_resource_fini(struct ttm_resource_manager *man, + struct ttm_resource *res); + int ttm_resource_alloc(struct ttm_buffer_object *bo, const struct ttm_place *place, struct ttm_resource **res); From patchwork Mon Jan 24 12:25:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2E64CC433F5 for ; Mon, 24 Jan 2022 12:25:28 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BC2BA10E8BC; Mon, 24 Jan 2022 12:25:24 +0000 (UTC) Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8073110E96F for ; Mon, 24 Jan 2022 12:25:19 +0000 (UTC) Received: by mail-wr1-x434.google.com with SMTP id r22so5622470wra.8 for ; Mon, 24 Jan 2022 04:25:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=qKzmjg1ffVsriKFNLMHGoofbaZ0iT9YZb5M0jc/qjsw=; b=WzivpKeZGeGbuRGBpqZbvN5yRClVAeKY6RQ0zetgfldeQ+sj196Ok/83rv+Wz9Zo6s PzWeHuAupgH/OR0blwBKk3MdJBgeZKre8Zlb+TfhGPu3Ysb3V84H5qjvnAHE9T4V46/Z L1ZY+K2Kkz4Rq6rn6NIFcvMaZX6e4v7FXrSh04nwT9phQmCgpJSDCfStsDJK/Ixh8V9D e7WMf525sB5tPftreFLYp2D25Pfu72i/+B66Z8wmOZL9fxzA5Wey5fkpMzXKOiDozQ7F tvl+R9OrL4BJ9jtsyruz1q7EZ+t/hCAvQ0QwsNIp9/fumi0f+dWdiHK3Ey0pVkdH6ZTM wc6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qKzmjg1ffVsriKFNLMHGoofbaZ0iT9YZb5M0jc/qjsw=; b=u8plJKFlKVVD/ebEUXPJpQL24B/QKt1w+FpFg2S7zCNudVN/B2nV3zmrL8Tn8h+SWH hB4r6UyAkR85GqaPJw+bTSv8UFoI8cF1GBHaXIyWNwQ+JzrHSpFFkZZWWPlyt+ig0YiK M0ux3/AGyRlTe0/yAdOvyZIL0bgkqbi2sHWcnMIiqApcgJslFK/vEF44O2eZZkP1P3OE aZDn6iHzCEEW+HnXJ0/L1U3svo30Ga8YV9qeA/gwUAHq8y/ZPljFrHGHFsQCLkFQSMgY 6NnoUzAOkwZiHc7aXY5OcpcVar4tXMe9JpZsAf54/IrPz5TbUlmUzjpluR6bd+/l538X anXw== X-Gm-Message-State: AOAM530CqEmorbZp31PvTVtyv5OACXgcRSTWtpDtIpuVde3815pYwbfU Q7aIako8yS5N3T0KrMtXuqFL7n7gc44= X-Google-Smtp-Source: ABdhPJypyAy4EfweenZe6c46m8mFf5ePJSUxvJjFhQztkiETQUMe6ctDWrunkm7qONj1dyv+9LOqLA== X-Received: by 2002:a5d:61d2:: with SMTP id q18mr13800850wrv.547.1643027117878; Mon, 24 Jan 2022 04:25:17 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:17 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 02/12] drm/ttm: add back a reference to the bdev to the res manager Date: Mon, 24 Jan 2022 13:25:04 +0100 Message-Id: <20220124122514.1832-3-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" It is simply a lot cleaner to have this around instead of adding the device throughout the call chain. Signed-off-by: Christian König Reviewed-by: Huang Rui Acked-by: Daniel Vetter --- drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 3 ++- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 2 +- drivers/gpu/drm/nouveau/nouveau_ttm.c | 4 ++-- drivers/gpu/drm/ttm/ttm_range_manager.c | 2 +- drivers/gpu/drm/ttm/ttm_resource.c | 3 +++ drivers/gpu/drm/ttm/ttm_sys_manager.c | 2 +- drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c | 2 +- drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c | 2 +- include/drm/ttm/ttm_resource.h | 16 +++++++++------- 11 files changed, 24 insertions(+), 17 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index ea5470c8c921..9e7685a4878c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -293,7 +293,8 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) man->use_tt = true; man->func = &amdgpu_gtt_mgr_func; - ttm_resource_manager_init(man, gtt_size >> PAGE_SHIFT); + ttm_resource_manager_init(man, &adev->mman.bdev, + gtt_size >> PAGE_SHIFT); start = AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS; size = (adev->gmc.gart_size >> PAGE_SHIFT) - start; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c index ffddec08e931..6f7189d32f0a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c @@ -153,7 +153,7 @@ int amdgpu_preempt_mgr_init(struct amdgpu_device *adev) man->use_tt = true; man->func = &amdgpu_preempt_mgr_func; - ttm_resource_manager_init(man, (1 << 30)); + ttm_resource_manager_init(man, &adev->mman.bdev, (1 << 30)); atomic64_set(&mgr->used, 0); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 55d68408951d..ddd0b6d74070 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -699,7 +699,8 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr; struct ttm_resource_manager *man = &mgr->manager; - ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT); + ttm_resource_manager_init(man, &adev->mman.bdev, + adev->gmc.real_vram_size >> PAGE_SHIFT); man->func = &amdgpu_vram_mgr_func; diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index bf857703c797..79dc8c6f2833 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -204,7 +204,7 @@ int i915_ttm_buddy_man_init(struct ttm_device *bdev, man = &bman->manager; man->use_tt = use_tt; man->func = &i915_ttm_buddy_manager_func; - ttm_resource_manager_init(man, bman->mm.size >> PAGE_SHIFT); + ttm_resource_manager_init(man, bdev, bman->mm.size >> PAGE_SHIFT); ttm_resource_manager_set_used(man, true); ttm_set_driver_manager(bdev, type, man); diff --git a/drivers/gpu/drm/nouveau/nouveau_ttm.c b/drivers/gpu/drm/nouveau/nouveau_ttm.c index 91ef33f8f22c..85f1f5a0fe5d 100644 --- a/drivers/gpu/drm/nouveau/nouveau_ttm.c +++ b/drivers/gpu/drm/nouveau/nouveau_ttm.c @@ -164,7 +164,7 @@ nouveau_ttm_init_vram(struct nouveau_drm *drm) man->func = &nouveau_vram_manager; - ttm_resource_manager_init(man, + ttm_resource_manager_init(man, &drm->ttm.bdev, drm->gem.vram_available >> PAGE_SHIFT); ttm_set_driver_manager(&drm->ttm.bdev, TTM_PL_VRAM, man); ttm_resource_manager_set_used(man, true); @@ -211,7 +211,7 @@ nouveau_ttm_init_gtt(struct nouveau_drm *drm) man->func = func; man->use_tt = true; - ttm_resource_manager_init(man, size_pages); + ttm_resource_manager_init(man, &drm->ttm.bdev, size_pages); ttm_set_driver_manager(&drm->ttm.bdev, TTM_PL_TT, man); ttm_resource_manager_set_used(man, true); return 0; diff --git a/drivers/gpu/drm/ttm/ttm_range_manager.c b/drivers/gpu/drm/ttm/ttm_range_manager.c index 55a41355f684..8cd4f3fb9f79 100644 --- a/drivers/gpu/drm/ttm/ttm_range_manager.c +++ b/drivers/gpu/drm/ttm/ttm_range_manager.c @@ -158,7 +158,7 @@ int ttm_range_man_init_nocheck(struct ttm_device *bdev, man->func = &ttm_range_manager_func; - ttm_resource_manager_init(man, p_size); + ttm_resource_manager_init(man, bdev, p_size); drm_mm_init(&rman->mm, 0, p_size); spin_lock_init(&rman->lock); diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index 89bcfe22a0ca..41e7bf195168 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -126,16 +126,19 @@ EXPORT_SYMBOL(ttm_resource_compat); * ttm_resource_manager_init * * @man: memory manager object to init + * @bdev: ttm device this manager belongs to * @p_size: size managed area in pages. * * Initialise core parts of a manager object. */ void ttm_resource_manager_init(struct ttm_resource_manager *man, + struct ttm_device *bdev, unsigned long p_size) { unsigned i; spin_lock_init(&man->move_lock); + man->bdev = bdev; man->size = p_size; for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) diff --git a/drivers/gpu/drm/ttm/ttm_sys_manager.c b/drivers/gpu/drm/ttm/ttm_sys_manager.c index 135394dcca95..2ced169513cb 100644 --- a/drivers/gpu/drm/ttm/ttm_sys_manager.c +++ b/drivers/gpu/drm/ttm/ttm_sys_manager.c @@ -43,7 +43,7 @@ void ttm_sys_man_init(struct ttm_device *bdev) man->use_tt = true; man->func = &ttm_sys_manager_func; - ttm_resource_manager_init(man, 0); + ttm_resource_manager_init(man, bdev, 0); ttm_set_driver_manager(bdev, TTM_PL_SYSTEM, man); ttm_resource_manager_set_used(man, true); } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c b/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c index 99ccf690f8a1..60e3cc537f36 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c @@ -162,7 +162,7 @@ int vmw_gmrid_man_init(struct vmw_private *dev_priv, int type) man->func = &vmw_gmrid_manager_func; man->use_tt = true; - ttm_resource_manager_init(man, 0); + ttm_resource_manager_init(man, &dev_priv->bdev, 0); spin_lock_init(&gman->lock); gman->used_gmr_pages = 0; ida_init(&gman->gmr_ida); diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c b/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c index a64188c7268d..d3007bf1b8f5 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c @@ -70,7 +70,7 @@ int vmw_sys_man_init(struct vmw_private *dev_priv) man->use_tt = true; man->func = &vmw_sys_manager_func; - ttm_resource_manager_init(man, 0); + ttm_resource_manager_init(man, bdev, 0); ttm_set_driver_manager(bdev, VMW_PL_SYSTEM, man); ttm_resource_manager_set_used(man, true); return 0; diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index df1f06b7b504..6bf37383002b 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -105,11 +105,11 @@ struct ttm_resource_manager_func { * @use_type: The memory type is enabled. * @use_tt: If a TT object should be used for the backing store. * @size: Size of the managed region. + * @bdev: ttm device this manager belongs to * @func: structure pointer implementing the range manager. See above * @move_lock: lock for move fence - * static information. bdev::driver::io_mem_free is never used. - * @lru: The lru list for this memory type. * @move: The fence of the last pipelined move operation. + * @lru: The lru list for this memory type. * * This structure is used to identify and manage memory types for a device. */ @@ -119,20 +119,21 @@ struct ttm_resource_manager { */ bool use_type; bool use_tt; + struct ttm_device *bdev; uint64_t size; const struct ttm_resource_manager_func *func; spinlock_t move_lock; /* - * Protected by the global->lru_lock. + * Protected by @move_lock. */ - - struct list_head lru[TTM_MAX_BO_PRIORITY]; + struct dma_fence *move; /* - * Protected by @move_lock. + * Protected by the global->lru_lock. */ - struct dma_fence *move; + + struct list_head lru[TTM_MAX_BO_PRIORITY]; }; /** @@ -272,6 +273,7 @@ bool ttm_resource_compat(struct ttm_resource *res, struct ttm_placement *placement); void ttm_resource_manager_init(struct ttm_resource_manager *man, + struct ttm_device *bdev, unsigned long p_size); int ttm_resource_manager_evict_all(struct ttm_device *bdev, From patchwork Mon Jan 24 12:25:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722022 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 28553C433FE for ; Mon, 24 Jan 2022 12:25:31 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id EFC6F10E99E; Mon, 24 Jan 2022 12:25:24 +0000 (UTC) Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1FA7410E8BC for ; Mon, 24 Jan 2022 12:25:20 +0000 (UTC) Received: by mail-wr1-x42b.google.com with SMTP id w11so1102374wra.4 for ; Mon, 24 Jan 2022 04:25:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=W/9N9xYoKgiFtiQW+jzimeQXI12uRru2/HMWPIJw1Tg=; b=LEheQgAOuvDUXTvYv2R9Ie9Blh07LSwR6Uxp3gOaxU/OOq6Lzewc3y+X57ECPxpnoS NVcJCoq4oaEJBwmv3XIhdYVAuX2ngzbFwEa57pbK1X4dxydIA0zjEb5raX8AYyI329bC XXXsmtBAbo/pzunRJU3tsRFKt0hnNhAkd9tzRqzCNNxJh9FjuimWznK+8KFDy4B4Gm8J qNka6m57a6VlbJMZNJjSy+O3t0xzZ8C+8tSlJtxXHeXfnbAowDxxnycuRntDsDrLK6bJ LE8t3dBkrk3dOEBf5qXVodLtO5hdi4HHAnyOv71D+ojqZFuTzbR1wof+iugyrJleB3hL eL1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=W/9N9xYoKgiFtiQW+jzimeQXI12uRru2/HMWPIJw1Tg=; b=NtAI6EBMqe6ONvJRzaeD5uwLAclqcldTwNRklUXlQDoiyytYds2CQun5/TgWZkfqgw p1yPxkFK+xKk3cY5hS3lvSKf/6taxRmJEx2RPTlmgK/5wYn5mcOYgr8oJknPAfTCaHCd lyUNZtNSR00hqRQrkvY8TeXjDsc66bcj08AL1yjgw/0W+8DKjL4B5qDMn4FKgS6v2XMm g/KVn2qmcQ9/+VQ8VGGP9812lGSW1MQ99f0HqHP7ZnhSanDjFLJF3QgWKaFVuYHyWukH MKyHmeB3OYGi7W2rr8j6up0xQVp35ptK9MK3l13XIBQSZFI8X+PbEWLqw1lWfwQ08jE4 3jeA== X-Gm-Message-State: AOAM533qcVIgnbviRjSdXCVgAZO+viYO/GR7+AZRPardPEdbjJQ8eDBQ +XRyvRmSPBGkINSElZvB+1Q= X-Google-Smtp-Source: ABdhPJwvpsp+qLJU0cBQyK4b8tB3CQwhdhA9HD9uE8XXnxn1J98iqbqfCCKLPdDpFX1/K0zuJPI9Zw== X-Received: by 2002:adf:80f1:: with SMTP id 104mr14210804wrl.200.1643027118747; Mon, 24 Jan 2022 04:25:18 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:18 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 03/12] drm/ttm: add a weak BO reference to the resource v3 Date: Mon, 24 Jan 2022 13:25:05 +0100 Message-Id: <20220124122514.1832-4-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Keep track for which BO a resource was allocated. This is necessary to move the LRU handling into the resources. A bit problematic is i915 since it tries to use the resource interface without a BO which is illegal from the conceptional point of view. v2: Document that this is a weak reference and add a workaround for i915 v3: further document that this is protected by ttm_device::lru_lock and clarify the i915 workaround Signed-off-by: Christian König Acked-by: Huang Rui --- drivers/gpu/drm/ttm/ttm_bo_util.c | 7 +++++-- drivers/gpu/drm/ttm/ttm_resource.c | 9 +++++++++ include/drm/ttm/ttm_resource.h | 4 ++++ 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c index 72a94301bc95..544a84fa6589 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_util.c +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c @@ -241,6 +241,11 @@ static int ttm_buffer_object_transfer(struct ttm_buffer_object *bo, if (bo->type != ttm_bo_type_sg) fbo->base.base.resv = &fbo->base.base._resv; + if (fbo->base.resource) { + ttm_resource_set_bo(fbo->base.resource, &fbo->base); + bo->resource = NULL; + } + dma_resv_init(&fbo->base.base._resv); fbo->base.base.dev = NULL; ret = dma_resv_trylock(&fbo->base.base._resv); @@ -509,7 +514,6 @@ static int ttm_bo_move_to_ghost(struct ttm_buffer_object *bo, ghost_obj->ttm = NULL; else bo->ttm = NULL; - bo->resource = NULL; dma_resv_unlock(&ghost_obj->base._resv); ttm_bo_put(ghost_obj); @@ -637,7 +641,6 @@ int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo) dma_resv_unlock(&ghost->base._resv); ttm_bo_put(ghost); bo->ttm = ttm; - bo->resource = NULL; ttm_bo_assign_mem(bo, sys_res); return 0; diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index 41e7bf195168..7fdd58b53c61 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -41,6 +41,7 @@ void ttm_resource_init(struct ttm_buffer_object *bo, res->bus.offset = 0; res->bus.is_iomem = false; res->bus.caching = ttm_cached; + res->bo = bo; } EXPORT_SYMBOL(ttm_resource_init); @@ -122,6 +123,14 @@ bool ttm_resource_compat(struct ttm_resource *res, } EXPORT_SYMBOL(ttm_resource_compat); +void ttm_resource_set_bo(struct ttm_resource *res, + struct ttm_buffer_object *bo) +{ + spin_lock(&bo->bdev->lru_lock); + res->bo = bo; + spin_unlock(&bo->bdev->lru_lock); +} + /** * ttm_resource_manager_init * diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index 6bf37383002b..69eea9d6399b 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -161,6 +161,7 @@ struct ttm_bus_placement { * @mem_type: Resource type of the allocation. * @placement: Placement flags. * @bus: Placement on io bus accessible to the CPU + * @bo: weak reference to the BO, protected by ttm_device::lru_lock * * Structure indicating the placement and space resources used by a * buffer object. @@ -171,6 +172,7 @@ struct ttm_resource { uint32_t mem_type; uint32_t placement; struct ttm_bus_placement bus; + struct ttm_buffer_object *bo; }; /** @@ -271,6 +273,8 @@ int ttm_resource_alloc(struct ttm_buffer_object *bo, void ttm_resource_free(struct ttm_buffer_object *bo, struct ttm_resource **res); bool ttm_resource_compat(struct ttm_resource *res, struct ttm_placement *placement); +void ttm_resource_set_bo(struct ttm_resource *res, + struct ttm_buffer_object *bo); void ttm_resource_manager_init(struct ttm_resource_manager *man, struct ttm_device *bdev, From patchwork Mon Jan 24 12:25:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4461CC433EF for ; Mon, 24 Jan 2022 12:25:41 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 499DB10EA4F; Mon, 24 Jan 2022 12:25:40 +0000 (UTC) Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by gabe.freedesktop.org (Postfix) with ESMTPS id E8C0F10E8BC for ; Mon, 24 Jan 2022 12:25:20 +0000 (UTC) Received: by mail-wr1-x42a.google.com with SMTP id u15so13244294wrt.3 for ; Mon, 24 Jan 2022 04:25:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=UXn8v4gDRQLa7yCgiM90E9GBqmcF2E/H65e/pHnK8to=; b=efanpkRoklGuVG0ex3r+HqS+ijixFVlEIG4Ue6nYW5Gfgb0P2XxUWzlAxPSdY6VGcc fgVGFzp7BjmEdZnAPdk8Vv+w7TNMEY+2T53QS/Efv2iS9WQ90zCh2PDd2nCCr2sXn8hd kLL2KKEyq9b6g2ovf7vW+/xQGTykiXPn4PDQUxP6ztsjNVSwMfQWYRgtPH/ab8K6swAT 6BGrWPSH7sAJQ1neCAuxwOseJ6pzbkZyb40BcAFDloAtEcKs1h/l2tLVk0pz9uzBszC+ ViSup5F57dj+62AJPOx7qLvi0xMZRhBumMDyIOl3glzaSBm/1nhIs1EKs15jqyC1TFVP Or8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UXn8v4gDRQLa7yCgiM90E9GBqmcF2E/H65e/pHnK8to=; b=CPfgItBGGX+7HoN76Yv+wJNW3QClTv6JbuBxk7BdnyikHEmGD4cRF94hT2gdmY5aU5 G1to8rDfCWbWsjn+DaW4Wc4QxT1oJ+XbjECpVtR6qVbtPRVUvW8RnDl3kCvzehQ2N8HB nP0jruu/P+L9JXYdeZPBoW5Wo/4VlwwwswvMmWwITdm5Ls55ZB1WIXp/4LWS3l8MIbjW FdvJz5NYfScANcWcylUFmTFdGpblgy++dA+PTjl6vtXy2Xau4Uh/nqYakMGxekLs7KM4 /8FFmGzNgqTAZecP78FOn54kRwB/bBDDLjxBaQzg9LvpcR2O6tFDJg8841r3s8H/iad8 21GA== X-Gm-Message-State: AOAM530tOCxH5RxqrQ0RNeNJVM2nCImK5YZrvxl4hk1twQTU983CfB/i 8BeaZLfKZkDmU9uvP5JayoE= X-Google-Smtp-Source: ABdhPJzOT3SH+PaAeaS1Z/8yDvkoqVOevxa3CC8zAcTsHGNH1kV78HDjdzvZTK/wv3bJ9OxGzZggeQ== X-Received: by 2002:adf:a141:: with SMTP id r1mr13536285wrr.124.1643027119470; Mon, 24 Jan 2022 04:25:19 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:19 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 04/12] drm/ttm: add common accounting to the resource mgr v2 Date: Mon, 24 Jan 2022 13:25:06 +0100 Message-Id: <20220124122514.1832-5-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" It makes sense to have this in the common manager for debugging and accounting of how much resources are used. v2: cleanup kerneldoc a bit Signed-off-by: Christian König Reviewed-by: Huang Rui --- drivers/gpu/drm/ttm/ttm_resource.c | 8 ++++++++ include/drm/ttm/ttm_resource.h | 20 +++++++++++++++++++- 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index 7fdd58b53c61..b8362492980d 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -33,6 +33,8 @@ void ttm_resource_init(struct ttm_buffer_object *bo, const struct ttm_place *place, struct ttm_resource *res) { + struct ttm_resource_manager *man; + res->start = 0; res->num_pages = PFN_UP(bo->base.size); res->mem_type = place->mem_type; @@ -42,12 +44,16 @@ void ttm_resource_init(struct ttm_buffer_object *bo, res->bus.is_iomem = false; res->bus.caching = ttm_cached; res->bo = bo; + + man = ttm_manager_type(bo->bdev, place->mem_type); + atomic64_add(bo->base.size, &man->usage); } EXPORT_SYMBOL(ttm_resource_init); void ttm_resource_fini(struct ttm_resource_manager *man, struct ttm_resource *res) { + atomic64_sub(res->bo->base.size, &man->usage); } EXPORT_SYMBOL(ttm_resource_fini); @@ -149,6 +155,7 @@ void ttm_resource_manager_init(struct ttm_resource_manager *man, spin_lock_init(&man->move_lock); man->bdev = bdev; man->size = p_size; + atomic64_set(&man->usage, 0); for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) INIT_LIST_HEAD(&man->lru[i]); @@ -221,6 +228,7 @@ void ttm_resource_manager_debug(struct ttm_resource_manager *man, drm_printf(p, " use_type: %d\n", man->use_type); drm_printf(p, " use_tt: %d\n", man->use_tt); drm_printf(p, " size: %llu\n", man->size); + drm_printf(p, " usage: %llu\n", atomic64_read(&man->usage)); if (man->func->debug) man->func->debug(man, p); } diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index 69eea9d6399b..3d391279b33f 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -27,6 +27,7 @@ #include #include +#include #include #include #include @@ -132,8 +133,12 @@ struct ttm_resource_manager { /* * Protected by the global->lru_lock. */ - struct list_head lru[TTM_MAX_BO_PRIORITY]; + + /** + * @usage: How much of the region is used, has its own protection. + */ + atomic64_t usage; }; /** @@ -261,6 +266,19 @@ ttm_resource_manager_cleanup(struct ttm_resource_manager *man) man->move = NULL; } +/** + * ttm_resource_manager_usage + * + * @man: A memory manager object. + * + * Return how many resources are currently used. + */ +static inline uint64_t +ttm_resource_manager_usage(struct ttm_resource_manager *man) +{ + return atomic64_read(&man->usage); +} + void ttm_resource_init(struct ttm_buffer_object *bo, const struct ttm_place *place, struct ttm_resource *res); From patchwork Mon Jan 24 12:25:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 90E0FC433EF for ; Mon, 24 Jan 2022 12:25:37 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id EEA9210EA71; Mon, 24 Jan 2022 12:25:25 +0000 (UTC) Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0416510E8BC for ; Mon, 24 Jan 2022 12:25:21 +0000 (UTC) Received: by mail-wm1-x32c.google.com with SMTP id 131so358133wmc.4 for ; Mon, 24 Jan 2022 04:25:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=rKR+9FQSd8Amz8CWDFgOebUn1AKcnSAM+Ggak63Wbss=; b=UcK9xw0fcF1uOKixdlyKRnt4yHXdSpXpSmM38rnzm5bCwat+6pjAi7/lWmoXkhqCvw KozSo4hzGBqp1hZICrmVDwRk4GCWbPntBRfw92kowEYWALHXpQMdUDimydifIGOZZb4S GiAknoNnH1IUAMyR2JjgxVjSz/VzO8TaCH6h5gp5euLQUteoJarCUV/AKmkOheZh7s1L VWIWVn1vlSFENC2UlBKaA2t+7BUwCzdxuSPGZXjH4e5qgfbbkiD2DJhKhS7c2isLp9MZ x9TxI+rAmuMRuKMaymxNyue7pm5OrNQG4WxXzGdf6iAQmjgQH+vQ8Kg5t97xwGxxSkNf TWJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rKR+9FQSd8Amz8CWDFgOebUn1AKcnSAM+Ggak63Wbss=; b=G7DvKmMhx+jmut3CemO10D/0DBh7sPvLnG8WTXZwgW47U02Je2TPVGhV+gapCYTdYr 7pQmC/2QV9lfiEH7mFF8h/jen2K6kY95idpfisuT9WJBi7yGnOdMI7k/JrR7QhMbbD74 Obb/se1odDGNuxer8SQ75QM4PIshzSFx0vWOOSWgUcLrUOr41++cbc6ZNCP/PvYpmqCF BTXwhgrODIVuuuXvX3aeXVPLoEO/Bc28jU87nRu4h4wXROJJW8OOcrA+Wh5QyAoWxBP4 AJfu3sMbtqR6J3xu6tIsZnx1oIYZ5bDkN0tBBrEXhIQUJs/FvuW8raviFmFrCAY7UPSH aQHQ== X-Gm-Message-State: AOAM532Ru8D+FjlWVdM7S1XWyouXvvOvEMxAxOqwASlZZ/MUlWVl/Jn0 nPEiW/ruNDUBRBxZN92LYZ0= X-Google-Smtp-Source: ABdhPJz00yHHC1pS/qF8t3IHufY1tLjcWuHZlcU3o8nGqysWgvsoX5BG3hwp1iZCIdX8SOJTEwSFLg== X-Received: by 2002:a05:600c:1548:: with SMTP id f8mr1622660wmg.80.1643027120422; Mon, 24 Jan 2022 04:25:20 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:20 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 05/12] drm/ttm: move the LRU into resource handling v2 Date: Mon, 24 Jan 2022 13:25:07 +0100 Message-Id: <20220124122514.1832-6-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This way we finally fix the problem that new resource are not immediately evict-able after allocation. That has caused numerous problems including OOM on GDS handling and not being able to use TTM as general resource manager. v2: stop assuming in ttm_resource_fini that res->bo is still valid. Signed-off-by: Christian König Reviewed-by: Daniel Vetter --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 +- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 2 +- drivers/gpu/drm/ttm/ttm_bo.c | 115 ++-------------------- drivers/gpu/drm/ttm/ttm_bo_util.c | 1 - drivers/gpu/drm/ttm/ttm_device.c | 64 ++++++------- drivers/gpu/drm/ttm/ttm_resource.c | 122 +++++++++++++++++++++++- include/drm/ttm/ttm_bo_api.h | 16 ---- include/drm/ttm/ttm_bo_driver.h | 29 +----- include/drm/ttm/ttm_resource.h | 35 +++++++ 9 files changed, 201 insertions(+), 191 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index a96ae4c0e040..cdb489cdf27b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -683,12 +683,12 @@ void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev, if (vm->bulk_moveable) { spin_lock(&adev->mman.bdev.lru_lock); - ttm_bo_bulk_move_lru_tail(&vm->lru_bulk_move); + ttm_lru_bulk_move_tail(&vm->lru_bulk_move); spin_unlock(&adev->mman.bdev.lru_lock); return; } - memset(&vm->lru_bulk_move, 0, sizeof(vm->lru_bulk_move)); + ttm_lru_bulk_move_init(&vm->lru_bulk_move); spin_lock(&adev->mman.bdev.lru_lock); list_for_each_entry(bo_base, &vm->idle, vm_status) { @@ -698,11 +698,9 @@ void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev, if (!bo->parent) continue; - ttm_bo_move_to_lru_tail(&bo->tbo, bo->tbo.resource, - &vm->lru_bulk_move); + ttm_bo_move_to_lru_tail(&bo->tbo, &vm->lru_bulk_move); if (shadow) ttm_bo_move_to_lru_tail(&shadow->tbo, - shadow->tbo.resource, &vm->lru_bulk_move); } spin_unlock(&adev->mman.bdev.lru_lock); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 218a9b3037c7..cdf1ff87c88d 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -835,7 +835,7 @@ void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj) bo->priority = I915_TTM_PRIO_NO_PAGES; } - ttm_bo_move_to_lru_tail(bo, bo->resource, NULL); + ttm_bo_move_to_lru_tail(bo, NULL); spin_unlock(&bo->bdev->lru_lock); } diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index db3dc7ef5382..cb0fa932d495 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -69,108 +69,16 @@ static void ttm_bo_mem_space_debug(struct ttm_buffer_object *bo, } } -static inline void ttm_bo_move_to_pinned(struct ttm_buffer_object *bo) -{ - struct ttm_device *bdev = bo->bdev; - - list_move_tail(&bo->lru, &bdev->pinned); - - if (bdev->funcs->del_from_lru_notify) - bdev->funcs->del_from_lru_notify(bo); -} - -static inline void ttm_bo_del_from_lru(struct ttm_buffer_object *bo) -{ - struct ttm_device *bdev = bo->bdev; - - list_del_init(&bo->lru); - - if (bdev->funcs->del_from_lru_notify) - bdev->funcs->del_from_lru_notify(bo); -} - -static void ttm_bo_bulk_move_set_pos(struct ttm_lru_bulk_move_pos *pos, - struct ttm_buffer_object *bo) -{ - if (!pos->first) - pos->first = bo; - pos->last = bo; -} - void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, - struct ttm_resource *mem, struct ttm_lru_bulk_move *bulk) { - struct ttm_device *bdev = bo->bdev; - struct ttm_resource_manager *man; - - if (!bo->deleted) - dma_resv_assert_held(bo->base.resv); - - if (bo->pin_count) { - ttm_bo_move_to_pinned(bo); - return; - } - - if (!mem) - return; - - man = ttm_manager_type(bdev, mem->mem_type); - list_move_tail(&bo->lru, &man->lru[bo->priority]); - - if (bdev->funcs->del_from_lru_notify) - bdev->funcs->del_from_lru_notify(bo); - - if (bulk && !bo->pin_count) { - switch (bo->resource->mem_type) { - case TTM_PL_TT: - ttm_bo_bulk_move_set_pos(&bulk->tt[bo->priority], bo); - break; + dma_resv_assert_held(bo->base.resv); - case TTM_PL_VRAM: - ttm_bo_bulk_move_set_pos(&bulk->vram[bo->priority], bo); - break; - } - } + if (bo->resource) + ttm_resource_move_to_lru_tail(bo->resource, bulk); } EXPORT_SYMBOL(ttm_bo_move_to_lru_tail); -void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk) -{ - unsigned i; - - for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { - struct ttm_lru_bulk_move_pos *pos = &bulk->tt[i]; - struct ttm_resource_manager *man; - - if (!pos->first) - continue; - - dma_resv_assert_held(pos->first->base.resv); - dma_resv_assert_held(pos->last->base.resv); - - man = ttm_manager_type(pos->first->bdev, TTM_PL_TT); - list_bulk_move_tail(&man->lru[i], &pos->first->lru, - &pos->last->lru); - } - - for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { - struct ttm_lru_bulk_move_pos *pos = &bulk->vram[i]; - struct ttm_resource_manager *man; - - if (!pos->first) - continue; - - dma_resv_assert_held(pos->first->base.resv); - dma_resv_assert_held(pos->last->base.resv); - - man = ttm_manager_type(pos->first->bdev, TTM_PL_VRAM); - list_bulk_move_tail(&man->lru[i], &pos->first->lru, - &pos->last->lru); - } -} -EXPORT_SYMBOL(ttm_bo_bulk_move_lru_tail); - static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, struct ttm_resource *mem, bool evict, struct ttm_operation_ctx *ctx, @@ -344,7 +252,6 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo, return ret; } - ttm_bo_move_to_pinned(bo); list_del_init(&bo->ddestroy); spin_unlock(&bo->bdev->lru_lock); ttm_bo_cleanup_memtype_use(bo); @@ -445,7 +352,7 @@ static void ttm_bo_release(struct kref *kref) */ if (bo->pin_count) { bo->pin_count = 0; - ttm_bo_move_to_lru_tail(bo, bo->resource, NULL); + ttm_resource_move_to_lru_tail(bo->resource, NULL); } kref_init(&bo->kref); @@ -458,7 +365,6 @@ static void ttm_bo_release(struct kref *kref) } spin_lock(&bo->bdev->lru_lock); - ttm_bo_del_from_lru(bo); list_del(&bo->ddestroy); spin_unlock(&bo->bdev->lru_lock); @@ -673,15 +579,17 @@ int ttm_mem_evict_first(struct ttm_device *bdev, struct ww_acquire_ctx *ticket) { struct ttm_buffer_object *bo = NULL, *busy_bo = NULL; + struct ttm_resource *res; bool locked = false; unsigned i; int ret; spin_lock(&bdev->lru_lock); for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { - list_for_each_entry(bo, &man->lru[i], lru) { + list_for_each_entry(res, &man->lru[i], lru) { bool busy; + bo = res->bo; if (!ttm_bo_evict_swapout_allowable(bo, ctx, place, &locked, &busy)) { if (busy && !busy_bo && ticket != @@ -699,7 +607,7 @@ int ttm_mem_evict_first(struct ttm_device *bdev, } /* If the inner loop terminated early, we have our candidate */ - if (&bo->lru != &man->lru[i]) + if (&res->lru != &man->lru[i]) break; bo = NULL; @@ -875,9 +783,6 @@ int ttm_bo_mem_space(struct ttm_buffer_object *bo, } error: - if (bo->resource->mem_type == TTM_PL_SYSTEM && !bo->pin_count) - ttm_bo_move_to_lru_tail_unlocked(bo); - return ret; } EXPORT_SYMBOL(ttm_bo_mem_space); @@ -971,7 +876,6 @@ int ttm_bo_init_reserved(struct ttm_device *bdev, bo->destroy = destroy ? destroy : ttm_bo_default_destroy; kref_init(&bo->kref); - INIT_LIST_HEAD(&bo->lru); INIT_LIST_HEAD(&bo->ddestroy); bo->bdev = bdev; bo->type = type; @@ -1021,8 +925,6 @@ int ttm_bo_init_reserved(struct ttm_device *bdev, return ret; } - ttm_bo_move_to_lru_tail_unlocked(bo); - return ret; } EXPORT_SYMBOL(ttm_bo_init_reserved); @@ -1123,7 +1025,6 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, return ret == -EBUSY ? -ENOSPC : ret; } - ttm_bo_move_to_pinned(bo); /* TODO: Cleanup the locking */ spin_unlock(&bo->bdev->lru_lock); diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c index 544a84fa6589..0163e92b57af 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_util.c +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c @@ -231,7 +231,6 @@ static int ttm_buffer_object_transfer(struct ttm_buffer_object *bo, atomic_inc(&ttm_glob.bo_count); INIT_LIST_HEAD(&fbo->base.ddestroy); - INIT_LIST_HEAD(&fbo->base.lru); fbo->base.moving = NULL; drm_vma_node_reset(&fbo->base.base.vma_node); diff --git a/drivers/gpu/drm/ttm/ttm_device.c b/drivers/gpu/drm/ttm/ttm_device.c index be24bb6cefd0..ba35887147ba 100644 --- a/drivers/gpu/drm/ttm/ttm_device.c +++ b/drivers/gpu/drm/ttm/ttm_device.c @@ -144,6 +144,7 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx, { struct ttm_resource_manager *man; struct ttm_buffer_object *bo; + struct ttm_resource *res; unsigned i, j; int ret; @@ -154,8 +155,11 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx, continue; for (j = 0; j < TTM_MAX_BO_PRIORITY; ++j) { - list_for_each_entry(bo, &man->lru[j], lru) { - uint32_t num_pages = PFN_UP(bo->base.size); + list_for_each_entry(res, &man->lru[j], lru) { + uint32_t num_pages; + + bo = res->bo; + num_pages = PFN_UP(bo->base.size); ret = ttm_bo_swapout(bo, ctx, gfp_flags); /* ttm_bo_swapout has dropped the lru_lock */ @@ -259,49 +263,45 @@ void ttm_device_fini(struct ttm_device *bdev) } EXPORT_SYMBOL(ttm_device_fini); -void ttm_device_clear_dma_mappings(struct ttm_device *bdev) +static void ttm_device_clear_lru_dma_mappings(struct ttm_device *bdev, + struct list_head *list) { - struct ttm_resource_manager *man; - struct ttm_buffer_object *bo; - unsigned int i, j; + struct ttm_resource *res; spin_lock(&bdev->lru_lock); - while (!list_empty(&bdev->pinned)) { - bo = list_first_entry(&bdev->pinned, struct ttm_buffer_object, lru); + while ((res = list_first_entry_or_null(list, typeof(*res), lru))) { + struct ttm_buffer_object *bo = res->bo; + /* Take ref against racing releases once lru_lock is unlocked */ - if (ttm_bo_get_unless_zero(bo)) { - list_del_init(&bo->lru); - spin_unlock(&bdev->lru_lock); + if (!ttm_bo_get_unless_zero(bo)) + continue; - if (bo->ttm) - ttm_tt_unpopulate(bo->bdev, bo->ttm); + list_del_init(&res->lru); + spin_unlock(&bdev->lru_lock); - ttm_bo_put(bo); - spin_lock(&bdev->lru_lock); - } + if (bo->ttm) + ttm_tt_unpopulate(bo->bdev, bo->ttm); + + ttm_bo_put(bo); + spin_lock(&bdev->lru_lock); } + spin_unlock(&bdev->lru_lock); +} + +void ttm_device_clear_dma_mappings(struct ttm_device *bdev) +{ + struct ttm_resource_manager *man; + unsigned int i, j; + + ttm_device_clear_lru_dma_mappings(bdev, &bdev->pinned); for (i = TTM_PL_SYSTEM; i < TTM_NUM_MEM_TYPES; ++i) { man = ttm_manager_type(bdev, i); if (!man || !man->use_tt) continue; - for (j = 0; j < TTM_MAX_BO_PRIORITY; ++j) { - while (!list_empty(&man->lru[j])) { - bo = list_first_entry(&man->lru[j], struct ttm_buffer_object, lru); - if (ttm_bo_get_unless_zero(bo)) { - list_del_init(&bo->lru); - spin_unlock(&bdev->lru_lock); - - if (bo->ttm) - ttm_tt_unpopulate(bo->bdev, bo->ttm); - - ttm_bo_put(bo); - spin_lock(&bdev->lru_lock); - } - } - } + for (j = 0; j < TTM_MAX_BO_PRIORITY; ++j) + ttm_device_clear_lru_dma_mappings(bdev, &man->lru[j]); } - spin_unlock(&bdev->lru_lock); } EXPORT_SYMBOL(ttm_device_clear_dma_mappings); diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index b8362492980d..450e665c357b 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -29,6 +29,106 @@ #include #include +/** + * ttm_lru_bulk_move_init - initialize a bulk move structure + * @bulk: the structure to init + * + * For now just memset the structure to zero. + */ +void ttm_lru_bulk_move_init(struct ttm_lru_bulk_move *bulk) +{ + memset(bulk, 0, sizeof(*bulk)); +} +EXPORT_SYMBOL(ttm_lru_bulk_move_init); + +/** + * ttm_lru_bulk_move_tail + * + * @bulk: bulk move structure + * + * Bulk move BOs to the LRU tail, only valid to use when driver makes sure that + * resource order never changes. Should be called with ttm_device::lru_lock held. + */ +void ttm_lru_bulk_move_tail(struct ttm_lru_bulk_move *bulk) +{ + unsigned i; + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_lru_bulk_move_pos *pos = &bulk->tt[i]; + struct ttm_resource_manager *man; + + if (!pos->first) + continue; + + dma_resv_assert_held(pos->first->bo->base.resv); + dma_resv_assert_held(pos->last->bo->base.resv); + + man = ttm_manager_type(pos->first->bo->bdev, TTM_PL_TT); + list_bulk_move_tail(&man->lru[i], &pos->first->lru, + &pos->last->lru); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_lru_bulk_move_pos *pos = &bulk->vram[i]; + struct ttm_resource_manager *man; + + if (!pos->first) + continue; + + dma_resv_assert_held(pos->first->bo->base.resv); + dma_resv_assert_held(pos->last->bo->base.resv); + + man = ttm_manager_type(pos->first->bo->bdev, TTM_PL_VRAM); + list_bulk_move_tail(&man->lru[i], &pos->first->lru, + &pos->last->lru); + } +} +EXPORT_SYMBOL(ttm_lru_bulk_move_tail); + +/* Record a resource position in a bulk move structure */ +static void ttm_lru_bulk_move_set_pos(struct ttm_lru_bulk_move_pos *pos, + struct ttm_resource *res) +{ + if (!pos->first) + pos->first = res; + pos->last = res; +} + +/* Move a resource to the LRU tail and track the bulk position */ +void ttm_resource_move_to_lru_tail(struct ttm_resource *res, + struct ttm_lru_bulk_move *bulk) +{ + struct ttm_buffer_object *bo = res->bo; + struct ttm_device *bdev = bo->bdev; + struct ttm_resource_manager *man; + + if (bo->pin_count) { + list_move_tail(&res->lru, &bdev->pinned); + if (bdev->funcs->del_from_lru_notify) + bdev->funcs->del_from_lru_notify(res->bo); + return; + } + + man = ttm_manager_type(bdev, res->mem_type); + list_move_tail(&res->lru, &man->lru[bo->priority]); + + if (bdev->funcs->del_from_lru_notify) + bdev->funcs->del_from_lru_notify(bo); + + if (!bulk) + return; + + switch (res->mem_type) { + case TTM_PL_TT: + ttm_lru_bulk_move_set_pos(&bulk->tt[bo->priority], res); + break; + + case TTM_PL_VRAM: + ttm_lru_bulk_move_set_pos(&bulk->vram[bo->priority], res); + break; + } +} + void ttm_resource_init(struct ttm_buffer_object *bo, const struct ttm_place *place, struct ttm_resource *res) @@ -44,16 +144,36 @@ void ttm_resource_init(struct ttm_buffer_object *bo, res->bus.is_iomem = false; res->bus.caching = ttm_cached; res->bo = bo; + INIT_LIST_HEAD(&res->lru); man = ttm_manager_type(bo->bdev, place->mem_type); atomic64_add(bo->base.size, &man->usage); + + spin_lock(&bo->bdev->lru_lock); + ttm_resource_move_to_lru_tail(res, NULL); + spin_unlock(&bo->bdev->lru_lock); } EXPORT_SYMBOL(ttm_resource_init); +/** + * ttm_resource_fini + * + * @res: the resource to clean up + * + * Make sure the resource is removed from the LRU before destruction. + */ void ttm_resource_fini(struct ttm_resource_manager *man, struct ttm_resource *res) { - atomic64_sub(res->bo->base.size, &man->usage); + struct ttm_device *bdev = man->bdev; + + spin_lock(&bdev->lru_lock); + list_del_init(&res->lru); + if (res->bo && bdev->funcs->del_from_lru_notify) + bdev->funcs->del_from_lru_notify(res->bo); + spin_unlock(&bdev->lru_lock); + + atomic64_sub(res->num_pages << PAGE_SHIFT, &man->usage); } EXPORT_SYMBOL(ttm_resource_fini); diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index c17b2df9178b..3da77fc54552 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -55,8 +55,6 @@ struct ttm_placement; struct ttm_place; -struct ttm_lru_bulk_move; - /** * enum ttm_bo_type * @@ -94,7 +92,6 @@ struct ttm_tt; * @ttm: TTM structure holding system pages. * @evicted: Whether the object was evicted without user-space knowing. * @deleted: True if the object is only a zombie and already deleted. - * @lru: List head for the lru list. * @ddestroy: List head for the delayed destroy list. * @swap: List head for swap LRU list. * @moving: Fence set when BO is moving @@ -143,7 +140,6 @@ struct ttm_buffer_object { * Members protected by the bdev::lru_lock. */ - struct list_head lru; struct list_head ddestroy; /** @@ -295,7 +291,6 @@ void ttm_bo_put(struct ttm_buffer_object *bo); * ttm_bo_move_to_lru_tail * * @bo: The buffer object. - * @mem: Resource object. * @bulk: optional bulk move structure to remember BO positions * * Move this BO to the tail of all lru lists used to lookup and reserve an @@ -303,19 +298,8 @@ void ttm_bo_put(struct ttm_buffer_object *bo); * held, and is used to make a BO less likely to be considered for eviction. */ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, - struct ttm_resource *mem, struct ttm_lru_bulk_move *bulk); -/** - * ttm_bo_bulk_move_lru_tail - * - * @bulk: bulk move structure - * - * Bulk move BOs to the LRU tail, only valid to use when driver makes sure that - * BO order never changes. Should be called with ttm_global::lru_lock held. - */ -void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk); - /** * ttm_bo_lock_delayed_workqueue * diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h index 5f087575194b..6c7352e13708 100644 --- a/include/drm/ttm/ttm_bo_driver.h +++ b/include/drm/ttm/ttm_bo_driver.h @@ -45,33 +45,6 @@ #include "ttm_tt.h" #include "ttm_pool.h" -/** - * struct ttm_lru_bulk_move_pos - * - * @first: first BO in the bulk move range - * @last: last BO in the bulk move range - * - * Positions for a lru bulk move. - */ -struct ttm_lru_bulk_move_pos { - struct ttm_buffer_object *first; - struct ttm_buffer_object *last; -}; - -/** - * struct ttm_lru_bulk_move - * - * @tt: first/last lru entry for BOs in the TT domain - * @vram: first/last lru entry for BOs in the VRAM domain - * @swap: first/last lru entry for BOs on the swap list - * - * Helper structure for bulk moves on the LRU list. - */ -struct ttm_lru_bulk_move { - struct ttm_lru_bulk_move_pos tt[TTM_MAX_BO_PRIORITY]; - struct ttm_lru_bulk_move_pos vram[TTM_MAX_BO_PRIORITY]; -}; - /* * ttm_bo.c */ @@ -182,7 +155,7 @@ static inline void ttm_bo_move_to_lru_tail_unlocked(struct ttm_buffer_object *bo) { spin_lock(&bo->bdev->lru_lock); - ttm_bo_move_to_lru_tail(bo, bo->resource, NULL); + ttm_bo_move_to_lru_tail(bo, NULL); spin_unlock(&bo->bdev->lru_lock); } diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index 3d391279b33f..a54d52517a30 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -26,10 +26,12 @@ #define _TTM_RESOURCE_H_ #include +#include #include #include #include #include + #include #include #include @@ -178,6 +180,33 @@ struct ttm_resource { uint32_t placement; struct ttm_bus_placement bus; struct ttm_buffer_object *bo; + struct list_head lru; +}; + +/** + * struct ttm_lru_bulk_move_pos + * + * @first: first res in the bulk move range + * @last: last res in the bulk move range + * + * Positions for a lru bulk move. + */ +struct ttm_lru_bulk_move_pos { + struct ttm_resource *first; + struct ttm_resource *last; +}; + +/** + * struct ttm_lru_bulk_move + * + * @tt: first/last lru entry for resources in the TT domain + * @vram: first/last lru entry for resources in the VRAM domain + * + * Helper structure for bulk moves on the LRU list. + */ +struct ttm_lru_bulk_move { + struct ttm_lru_bulk_move_pos tt[TTM_MAX_BO_PRIORITY]; + struct ttm_lru_bulk_move_pos vram[TTM_MAX_BO_PRIORITY]; }; /** @@ -279,6 +308,12 @@ ttm_resource_manager_usage(struct ttm_resource_manager *man) return atomic64_read(&man->usage); } +void ttm_lru_bulk_move_init(struct ttm_lru_bulk_move *bulk); +void ttm_lru_bulk_move_tail(struct ttm_lru_bulk_move *bulk); + +void ttm_resource_move_to_lru_tail(struct ttm_resource *res, + struct ttm_lru_bulk_move *bulk); + void ttm_resource_init(struct ttm_buffer_object *bo, const struct ttm_place *place, struct ttm_resource *res); From patchwork Mon Jan 24 12:25:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7F264C433F5 for ; Mon, 24 Jan 2022 12:25:33 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C1E5C10E96F; Mon, 24 Jan 2022 12:25:24 +0000 (UTC) Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9021410E8BC for ; Mon, 24 Jan 2022 12:25:22 +0000 (UTC) Received: by mail-wr1-x432.google.com with SMTP id k18so13243982wrg.11 for ; Mon, 24 Jan 2022 04:25:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=q8Sf4Nfn4+tnVtMGVUOZ7yPbmyDp/qlQ6cLWMQPWbVU=; b=ozTuLUU6RwpcDCM9DcjKTQ7+czMsCRTU2n29/Gpq2NqMT+zrAn1J2IP8ItaqvlP8uY 8x1nNNTFi2ZF4qxyTe/2mz5nNFPlojEqBs/IsPIq7DWZHiF53k+TvitSN7uJ/7sfxbge dL4BN/3wp85m26SwqR0/5qCUvcaqm3cgnQdlL5+BG5wbTByprp0GpAMuacbde1coEfMM bZDrLIypboN0OQMI13xzK9nBHf3m7nUsTQBwMalWH6h97ZU69tb/Lxvwr4xPji/iVLX0 Y3W/6lbLU4U+29xsKDLc+ZS6kWjwsmpDthOOEPOqv+couvB+tCOYdeSifUgiOKKZizY9 BByA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q8Sf4Nfn4+tnVtMGVUOZ7yPbmyDp/qlQ6cLWMQPWbVU=; b=R8IjHpV4vMBOeH/E0o6IcN+Dezl039wIgOXIedGsXxfKtagE4OT9xnAGghxbZvyqDK wjjcp8X3BRl1u2Ssa06BfblxHLjWt/azlggxZ7YP2zQsZNlBX6Knv1YwFoT8j679Ukl0 Q9B+tJyQ9VpLmh6l8fKKri1jfBwjJUMZuyo2wgWco3Z7tmfcjgNXM1Vh57F08IYYiC7j DJ9osUSAiD+VXBzIcliq9DuagTsm9/O/km8sspfF22APjrZqAvHeFRYictFESfpVxuDT eUmDETKdDG/iTc0MpOdR9Vv9AvlhZ+zoFjabJmVK9OJtk5B/un6dXu7UlC7C5sny4kaN IOiA== X-Gm-Message-State: AOAM531+HR7X8szV9fHE8LPYSdWkFjlLTcarEn6fdgEE1RoQUu+W0/8w j2U+YwoLp3oYaxO/efrJZCc= X-Google-Smtp-Source: ABdhPJxT9twKoH8AlzmbFNCgXOG6chkM1ZT9wvy3Ondpy7GfUS7XlosuN2VECmry7kxDd/bwdzkG9g== X-Received: by 2002:adf:d1ef:: with SMTP id g15mr7729136wrd.440.1643027121170; Mon, 24 Jan 2022 04:25:21 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:20 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 06/12] drm/ttm: add resource iterator Date: Mon, 24 Jan 2022 13:25:08 +0100 Message-Id: <20220124122514.1832-7-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Instead of duplicating that at different places add an iterator over all the resources in a resource manager. Signed-off-by: Christian König Reviewed-by: Daniel Vetter --- drivers/gpu/drm/ttm/ttm_bo.c | 41 +++++++++++---------------- drivers/gpu/drm/ttm/ttm_device.c | 26 ++++++++--------- drivers/gpu/drm/ttm/ttm_resource.c | 45 ++++++++++++++++++++++++++++++ include/drm/ttm/ttm_resource.h | 23 +++++++++++++++ 4 files changed, 95 insertions(+), 40 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index cb0fa932d495..599be3dda8a9 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -579,38 +579,29 @@ int ttm_mem_evict_first(struct ttm_device *bdev, struct ww_acquire_ctx *ticket) { struct ttm_buffer_object *bo = NULL, *busy_bo = NULL; + struct ttm_resource_cursor cursor; struct ttm_resource *res; bool locked = false; - unsigned i; int ret; spin_lock(&bdev->lru_lock); - for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { - list_for_each_entry(res, &man->lru[i], lru) { - bool busy; - - bo = res->bo; - if (!ttm_bo_evict_swapout_allowable(bo, ctx, place, - &locked, &busy)) { - if (busy && !busy_bo && ticket != - dma_resv_locking_ctx(bo->base.resv)) - busy_bo = bo; - continue; - } - - if (!ttm_bo_get_unless_zero(bo)) { - if (locked) - dma_resv_unlock(bo->base.resv); - continue; - } - break; + ttm_resource_manager_for_each_res(man, &cursor, res) { + bool busy; + + if (!ttm_bo_evict_swapout_allowable(res->bo, ctx, place, + &locked, &busy)) { + if (busy && !busy_bo && ticket != + dma_resv_locking_ctx(bo->base.resv)) + busy_bo = res->bo; + continue; } - /* If the inner loop terminated early, we have our candidate */ - if (&res->lru != &man->lru[i]) - break; - - bo = NULL; + if (!ttm_bo_get_unless_zero(res->bo)) { + if (locked) + dma_resv_unlock(res->bo->base.resv); + continue; + } + bo = res->bo; } if (!bo) { diff --git a/drivers/gpu/drm/ttm/ttm_device.c b/drivers/gpu/drm/ttm/ttm_device.c index ba35887147ba..a0562ab386f5 100644 --- a/drivers/gpu/drm/ttm/ttm_device.c +++ b/drivers/gpu/drm/ttm/ttm_device.c @@ -142,10 +142,10 @@ EXPORT_SYMBOL(ttm_global_swapout); int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx, gfp_t gfp_flags) { + struct ttm_resource_cursor cursor; struct ttm_resource_manager *man; - struct ttm_buffer_object *bo; struct ttm_resource *res; - unsigned i, j; + unsigned i; int ret; spin_lock(&bdev->lru_lock); @@ -154,20 +154,16 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx, if (!man || !man->use_tt) continue; - for (j = 0; j < TTM_MAX_BO_PRIORITY; ++j) { - list_for_each_entry(res, &man->lru[j], lru) { - uint32_t num_pages; - - bo = res->bo; - num_pages = PFN_UP(bo->base.size); + ttm_resource_manager_for_each_res(man, &cursor, res) { + struct ttm_buffer_object *bo = res->bo; + uint32_t num_pages = PFN_UP(bo->base.size); - ret = ttm_bo_swapout(bo, ctx, gfp_flags); - /* ttm_bo_swapout has dropped the lru_lock */ - if (!ret) - return num_pages; - if (ret != -EBUSY) - return ret; - } + ret = ttm_bo_swapout(bo, ctx, gfp_flags); + /* ttm_bo_swapout has dropped the lru_lock */ + if (!ret) + return num_pages; + if (ret != -EBUSY) + return ret; } } spin_unlock(&bdev->lru_lock); diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index 450e665c357b..9e68d36a1546 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -354,6 +354,51 @@ void ttm_resource_manager_debug(struct ttm_resource_manager *man, } EXPORT_SYMBOL(ttm_resource_manager_debug); +/** + * ttm_resource_manager_first + * + * @man: resource manager to iterate over + * @cursor: cursor to record the position + * + * Returns the first resource from the resource manager. + */ +struct ttm_resource * +ttm_resource_manager_first(struct ttm_resource_manager *man, + struct ttm_resource_cursor *cursor) +{ + struct ttm_resource *res; + + for (cursor->priority = 0; cursor->priority < TTM_MAX_BO_PRIORITY; + ++cursor->priority) + list_for_each_entry(res, &man->lru[cursor->priority], lru) + return res; + + return NULL; +} + +/** + * ttm_resource_manager_next + * + * @man: resource manager to iterate over + * @cursor: cursor to record the position + * + * Returns the next resource from the resource manager. + */ +struct ttm_resource * +ttm_resource_manager_next(struct ttm_resource_manager *man, + struct ttm_resource_cursor *cursor, + struct ttm_resource *res) +{ + list_for_each_entry_continue(res, &man->lru[cursor->priority], lru) + return res; + + for (; cursor->priority < TTM_MAX_BO_PRIORITY; ++cursor->priority) + list_for_each_entry(res, &man->lru[cursor->priority], lru) + return res; + + return NULL; +} + static void ttm_kmap_iter_iomap_map_local(struct ttm_kmap_iter *iter, struct dma_buf_map *dmap, pgoff_t i) diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index a54d52517a30..13da5e337350 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -183,6 +183,17 @@ struct ttm_resource { struct list_head lru; }; +/** + * struct ttm_resource_cursor + * + * @priority: the current priority + * + * Cursor to iterate over the resources in a manager. + */ +struct ttm_resource_cursor { + unsigned int priority; +}; + /** * struct ttm_lru_bulk_move_pos * @@ -339,6 +350,18 @@ int ttm_resource_manager_evict_all(struct ttm_device *bdev, void ttm_resource_manager_debug(struct ttm_resource_manager *man, struct drm_printer *p); +struct ttm_resource * +ttm_resource_manager_first(struct ttm_resource_manager *man, + struct ttm_resource_cursor *cursor); +struct ttm_resource * +ttm_resource_manager_next(struct ttm_resource_manager *man, + struct ttm_resource_cursor *cursor, + struct ttm_resource *res); + +#define ttm_resource_manager_for_each_res(man, cursor, res) \ + for (res = ttm_resource_manager_first(man, cursor); res; \ + res = ttm_resource_manager_next(man, cursor, res)) + struct ttm_kmap_iter * ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io, struct io_mapping *iomap, From patchwork Mon Jan 24 12:25:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 18B44C433F5 for ; Mon, 24 Jan 2022 12:25:43 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0734610EA87; Mon, 24 Jan 2022 12:25:41 +0000 (UTC) Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5C4F510E8BC for ; Mon, 24 Jan 2022 12:25:23 +0000 (UTC) Received: by mail-wr1-x42c.google.com with SMTP id k18so13244043wrg.11 for ; Mon, 24 Jan 2022 04:25:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ypeiZpV86bu1C1eyehMZYIostfkHLRM2UAhnnAFqCmI=; b=EkPOdEFEZf0j1qkBs5iBbOBUW46ulxjGaBLHcxCvJ7AKIhbeRPHAcGwirQOzI+LBhs x1RnRNiG3yQB9NrtNbB6Ot4GiT2Af1EBjBUNuHlExX4AnyAsTu6VG055kHDrxCmjF9nC 9CnVADFP8IV5lK+dSbvJyfnnugkrHC+2n3XJS4eEtAOHm00xR4EkFfaaR9SPZEaXbedv vjA7qAqm5sfW6ujEa+Ei8NZYKdoUQ1bxeHzahk9wq/hNtp+vQh7R9VUNuP+U7jwDEUO0 kTXWVLVfqHbXtxL/NtHl/i3xL+p3kpHOd5zjZDD5V2JERdqSlUmR5ELv4+34MoUcVIPd 6byQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ypeiZpV86bu1C1eyehMZYIostfkHLRM2UAhnnAFqCmI=; b=cOkEa/0+3dudZSm8ePYKcchHVa6F7n627hI4+RnvHfMCjJtC0RAOndMlAPyb52D/ON 4cUqn6TYCUH8tmwWsDKPL5Xl1P/O0OogMBJdlWqiigk6WgdHYLu7jLOq/qLTJ2W4gZqb gm7BOXCVzsc9lXdwi1x4OccQhpcsA4Q9sGlj+HADsLQ414LDWzijeQ3QTaw15BEaVaM+ mALIz/x1WNaEYCiHtRaw3Y5VH2N71kslOwc4xfs95ePULIl56QNQnkDXUWljKpcssv5s mACQg3EGpeuTWW1Nq6aLlCLM+ASqDBccQ0c17urkVe+7saJWY8mQ0AG/h60izUkx5LEr UL0Q== X-Gm-Message-State: AOAM5328/vDkz8rDicPxapg2gLoOWHW5hhc3I/TjGeZd5AzCViu6eUVI h2wShWNgLjBrI5y/dfUOdTZA7pmHE9Q= X-Google-Smtp-Source: ABdhPJwfO+nJmIVuz18434aXfdFq8pIwE6CvAC8E1xCpN77kthuX9KY9ogeRHeuM6myB3r5y54XA/w== X-Received: by 2002:a05:6000:15c9:: with SMTP id y9mr9801179wry.121.1643027121914; Mon, 24 Jan 2022 04:25:21 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:21 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 07/12] drm/radeon: remove resource accounting Date: Mon, 24 Jan 2022 13:25:09 +0100 Message-Id: <20220124122514.1832-8-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use the one provided by TTM instead. Signed-off-by: Christian König --- drivers/gpu/drm/radeon/radeon.h | 2 -- drivers/gpu/drm/radeon/radeon_kms.c | 7 ++++-- drivers/gpu/drm/radeon/radeon_object.c | 30 +++----------------------- drivers/gpu/drm/radeon/radeon_object.h | 1 - drivers/gpu/drm/radeon/radeon_ttm.c | 18 ++-------------- 5 files changed, 10 insertions(+), 48 deletions(-) diff --git a/drivers/gpu/drm/radeon/radeon.h b/drivers/gpu/drm/radeon/radeon.h index 895776c421d4..08f83bf2c330 100644 --- a/drivers/gpu/drm/radeon/radeon.h +++ b/drivers/gpu/drm/radeon/radeon.h @@ -2462,8 +2462,6 @@ struct radeon_device { struct radeon_vm_manager vm_manager; struct mutex gpu_clock_mutex; /* memory stats */ - atomic64_t vram_usage; - atomic64_t gtt_usage; atomic64_t num_bytes_moved; atomic_t gpu_reset_counter; /* ACPI interface */ diff --git a/drivers/gpu/drm/radeon/radeon_kms.c b/drivers/gpu/drm/radeon/radeon_kms.c index 7afe28408085..3a9226d0cb05 100644 --- a/drivers/gpu/drm/radeon/radeon_kms.c +++ b/drivers/gpu/drm/radeon/radeon_kms.c @@ -241,6 +241,7 @@ int radeon_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) struct drm_radeon_info *info = data; struct radeon_mode_info *minfo = &rdev->mode_info; uint32_t *value, value_tmp, *value_ptr, value_size; + struct ttm_resource_manager *man; uint64_t value64; struct drm_crtc *crtc; int i, found; @@ -550,12 +551,14 @@ int radeon_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) case RADEON_INFO_VRAM_USAGE: value = (uint32_t*)&value64; value_size = sizeof(uint64_t); - value64 = atomic64_read(&rdev->vram_usage); + man = ttm_manager_type(&rdev->mman.bdev, TTM_PL_VRAM); + value64 = ttm_resource_manager_usage(man); break; case RADEON_INFO_GTT_USAGE: value = (uint32_t*)&value64; value_size = sizeof(uint64_t); - value64 = atomic64_read(&rdev->gtt_usage); + man = ttm_manager_type(&rdev->mman.bdev, TTM_PL_TT); + value64 = ttm_resource_manager_usage(man); break; case RADEON_INFO_ACTIVE_CU_COUNT: if (rdev->family >= CHIP_BONAIRE) diff --git a/drivers/gpu/drm/radeon/radeon_object.c b/drivers/gpu/drm/radeon/radeon_object.c index 56ede9d63b12..c9bbed2a25ad 100644 --- a/drivers/gpu/drm/radeon/radeon_object.c +++ b/drivers/gpu/drm/radeon/radeon_object.c @@ -49,27 +49,6 @@ static void radeon_bo_clear_surface_reg(struct radeon_bo *bo); * function are calling it. */ -static void radeon_update_memory_usage(struct ttm_buffer_object *bo, - unsigned int mem_type, int sign) -{ - struct radeon_device *rdev = radeon_get_rdev(bo->bdev); - - switch (mem_type) { - case TTM_PL_TT: - if (sign > 0) - atomic64_add(bo->base.size, &rdev->gtt_usage); - else - atomic64_sub(bo->base.size, &rdev->gtt_usage); - break; - case TTM_PL_VRAM: - if (sign > 0) - atomic64_add(bo->base.size, &rdev->vram_usage); - else - atomic64_sub(bo->base.size, &rdev->vram_usage); - break; - } -} - static void radeon_ttm_bo_destroy(struct ttm_buffer_object *tbo) { struct radeon_bo *bo; @@ -434,7 +413,9 @@ void radeon_bo_fini(struct radeon_device *rdev) static u64 radeon_bo_get_threshold_for_moves(struct radeon_device *rdev) { u64 real_vram_size = rdev->mc.real_vram_size; - u64 vram_usage = atomic64_read(&rdev->vram_usage); + struct ttm_resource_manager *man = + ttm_manager_type(&rdev->mman.bdev, TTM_PL_VRAM); + u64 vram_usage = ttm_resource_manager_usage(man); /* This function is based on the current VRAM usage. * @@ -725,15 +706,10 @@ int radeon_bo_check_tiling(struct radeon_bo *bo, bool has_moved, } void radeon_bo_move_notify(struct ttm_buffer_object *bo, - unsigned int old_type, struct ttm_resource *new_mem) { struct radeon_bo *rbo; - radeon_update_memory_usage(bo, old_type, -1); - if (new_mem) - radeon_update_memory_usage(bo, new_mem->mem_type, 1); - if (!radeon_ttm_bo_is_radeon_bo(bo)) return; diff --git a/drivers/gpu/drm/radeon/radeon_object.h b/drivers/gpu/drm/radeon/radeon_object.h index 1afc7992ef91..0b64e202577b 100644 --- a/drivers/gpu/drm/radeon/radeon_object.h +++ b/drivers/gpu/drm/radeon/radeon_object.h @@ -161,7 +161,6 @@ extern void radeon_bo_get_tiling_flags(struct radeon_bo *bo, extern int radeon_bo_check_tiling(struct radeon_bo *bo, bool has_moved, bool force_drop); extern void radeon_bo_move_notify(struct ttm_buffer_object *bo, - unsigned int old_type, struct ttm_resource *new_mem); extern vm_fault_t radeon_bo_fault_reserve_notify(struct ttm_buffer_object *bo); extern int radeon_bo_get_surface_reg(struct radeon_bo *bo); diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c index 0d1283cdc8fb..ae09a91a486a 100644 --- a/drivers/gpu/drm/radeon/radeon_ttm.c +++ b/drivers/gpu/drm/radeon/radeon_ttm.c @@ -199,7 +199,7 @@ static int radeon_bo_move(struct ttm_buffer_object *bo, bool evict, struct ttm_resource *old_mem = bo->resource; struct radeon_device *rdev; struct radeon_bo *rbo; - int r, old_type; + int r; if (new_mem->mem_type == TTM_PL_TT) { r = radeon_ttm_tt_bind(bo->bdev, bo->ttm, new_mem); @@ -216,9 +216,6 @@ static int radeon_bo_move(struct ttm_buffer_object *bo, bool evict, if (WARN_ON_ONCE(rbo->tbo.pin_count > 0)) return -EINVAL; - /* Save old type for statistics update */ - old_type = old_mem->mem_type; - rdev = radeon_get_rdev(bo->bdev); if (old_mem->mem_type == TTM_PL_SYSTEM && bo->ttm == NULL) { ttm_bo_move_null(bo, new_mem); @@ -264,7 +261,7 @@ static int radeon_bo_move(struct ttm_buffer_object *bo, bool evict, out: /* update statistics */ atomic64_add(bo->base.size, &rdev->num_bytes_moved); - radeon_bo_move_notify(bo, old_type, new_mem); + radeon_bo_move_notify(bo, new_mem); return 0; } @@ -679,16 +676,6 @@ bool radeon_ttm_tt_is_readonly(struct radeon_device *rdev, return !!(gtt->userflags & RADEON_GEM_USERPTR_READONLY); } -static void -radeon_bo_delete_mem_notify(struct ttm_buffer_object *bo) -{ - unsigned int old_type = TTM_PL_SYSTEM; - - if (bo->resource) - old_type = bo->resource->mem_type; - radeon_bo_move_notify(bo, old_type, NULL); -} - static struct ttm_device_funcs radeon_bo_driver = { .ttm_tt_create = &radeon_ttm_tt_create, .ttm_tt_populate = &radeon_ttm_tt_populate, @@ -697,7 +684,6 @@ static struct ttm_device_funcs radeon_bo_driver = { .eviction_valuable = ttm_bo_eviction_valuable, .evict_flags = &radeon_evict_flags, .move = &radeon_bo_move, - .delete_mem_notify = &radeon_bo_delete_mem_notify, .io_mem_reserve = &radeon_ttm_io_mem_reserve, }; From patchwork Mon Jan 24 12:25:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96E31C433EF for ; Mon, 24 Jan 2022 12:25:35 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BE60910EA39; Mon, 24 Jan 2022 12:25:25 +0000 (UTC) Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by gabe.freedesktop.org (Postfix) with ESMTPS id 226C010E8BC for ; Mon, 24 Jan 2022 12:25:24 +0000 (UTC) Received: by mail-wr1-x42c.google.com with SMTP id h29so10610145wrb.5 for ; Mon, 24 Jan 2022 04:25:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=mL9p2jWNUVXLZ5w9308C92ujxW+8PpClhjMW4UXdDb8=; b=IST1mh4oe//JLZU8n4tlAx+tZfRfUsrWxPGluTJItNymF93m+VEx6/Dp4CUV7LIKCM 1Ga7GFooqUYv4gjNthyQM8QL1Vfs89uiczzv7ZsGU7ZOJXpRdjRJlso3xczNxgdUbEG2 OzimrzArFiJhZcSunDdcW5U5WVJQvChQ59BB3uso0YJZPbjZ/P2YydAdKGzp2wZRAqZn I3vAsm8qezegxhp6+3VCKND2gotdGLtdVW3T77rW+TMlGgVKGoHAhwyPEp3mWpFJ4XWh ZNxP5YPSPI552LZwsopaMcbMSk9bDiY0B6loNaSgj8iSMwObCYrZQlDwvGj5s8vGUdaw GXug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mL9p2jWNUVXLZ5w9308C92ujxW+8PpClhjMW4UXdDb8=; b=JxIK96rPwwRNxpRs26A9aYsJ+mdRqCbNXcCXR7q77GhJ8AB5vbT+jjkaW1kXvC3q0W cZSjqpdf4+ZbCfDjlMprevoEoFjHqkhBFaDM8KS4GOJlX//2yBtaK2wJOT5AmG2U2CLP rMWLz7wiuZqH7m7CVD4MvmlSzaFWkAtBA8+KzqhTsjEWTYxlguzt+i0RVlBAEe68HljY U5g8th26JEJoB88Y496biaoPtH4K64UBdCJQ9ysdAhHyusFbM9ROznT8vSttn+r78vy/ 2jpAHV6I3CssbXXalkcH/i58Pj+p0CAJxbEzxLS+yZQkFcdqkvF5tqk36P5oniPj3EA+ tGiQ== X-Gm-Message-State: AOAM530Xrhc2mq905tGzPB29VGXaQQ0tqb520FTubq1wrj39oX70onWj GHKVHd3u0vqPVYcVTLgELhc= X-Google-Smtp-Source: ABdhPJyPM1v6kNYjDVdpEZ2cFG8gNEuPlzoKcbGzN/9I7sjTg7hQt801HEVrtBkVB8CrbrRDYAmeag== X-Received: by 2002:a05:6000:1543:: with SMTP id 3mr13954286wry.683.1643027122689; Mon, 24 Jan 2022 04:25:22 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:22 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 08/12] drm/amdgpu: remove GTT accounting Date: Mon, 24 Jan 2022 13:25:10 +0100 Message-Id: <20220124122514.1832-9-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This is provided by TTM now. Also switch man->size to bytes instead of pages and fix the double printing of size and usage in debugfs. Signed-off-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 50 +++++---------------- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 5 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 2 - 3 files changed, 12 insertions(+), 45 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 9e7685a4878c..ce5eeb3c1097 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -60,7 +60,7 @@ static ssize_t amdgpu_mem_info_gtt_total_show(struct device *dev, struct ttm_resource_manager *man; man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT); - return sysfs_emit(buf, "%llu\n", man->size * PAGE_SIZE); + return sysfs_emit(buf, "%llu\n", man->size); } /** @@ -80,7 +80,7 @@ static ssize_t amdgpu_mem_info_gtt_used_show(struct device *dev, struct ttm_resource_manager *man; man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT); - return sysfs_emit(buf, "%llu\n", amdgpu_gtt_mgr_usage(man)); + return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man)); } static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, @@ -132,20 +132,17 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, struct amdgpu_gtt_node *node; int r; - if (!(place->flags & TTM_PL_FLAG_TEMPORARY) && - atomic64_add_return(num_pages, &mgr->used) > man->size) { - atomic64_sub(num_pages, &mgr->used); - return -ENOSPC; - } - node = kzalloc(struct_size(node, base.mm_nodes, 1), GFP_KERNEL); - if (!node) { - r = -ENOMEM; - goto err_out; - } + if (!node) + return -ENOMEM; node->tbo = tbo; ttm_resource_init(tbo, place, &node->base.base); + if (!(place->flags & TTM_PL_FLAG_TEMPORARY) && + ttm_resource_manager_usage(man) > man->size) { + r = -ENOSPC; + goto err_free; + } if (place->lpfn) { spin_lock(&mgr->lock); @@ -171,11 +168,6 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, err_free: ttm_resource_fini(man, &node->base.base); kfree(node); - -err_out: - if (!(place->flags & TTM_PL_FLAG_TEMPORARY)) - atomic64_sub(num_pages, &mgr->used); - return r; } @@ -198,27 +190,10 @@ static void amdgpu_gtt_mgr_del(struct ttm_resource_manager *man, drm_mm_remove_node(&node->base.mm_nodes[0]); spin_unlock(&mgr->lock); - if (!(res->placement & TTM_PL_FLAG_TEMPORARY)) - atomic64_sub(res->num_pages, &mgr->used); - ttm_resource_fini(man, res); kfree(node); } -/** - * amdgpu_gtt_mgr_usage - return usage of GTT domain - * - * @man: TTM memory type manager - * - * Return how many bytes are used in the GTT domain - */ -uint64_t amdgpu_gtt_mgr_usage(struct ttm_resource_manager *man) -{ - struct amdgpu_gtt_mgr *mgr = to_gtt_mgr(man); - - return atomic64_read(&mgr->used) * PAGE_SIZE; -} - /** * amdgpu_gtt_mgr_recover - re-init gart * @@ -265,9 +240,6 @@ static void amdgpu_gtt_mgr_debug(struct ttm_resource_manager *man, spin_lock(&mgr->lock); drm_mm_print(&mgr->mm, printer); spin_unlock(&mgr->lock); - - drm_printf(printer, "man size:%llu pages, gtt used:%llu pages\n", - man->size, atomic64_read(&mgr->used)); } static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func = { @@ -293,14 +265,12 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) man->use_tt = true; man->func = &amdgpu_gtt_mgr_func; - ttm_resource_manager_init(man, &adev->mman.bdev, - gtt_size >> PAGE_SHIFT); + ttm_resource_manager_init(man, &adev->mman.bdev, gtt_size); start = AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS; size = (adev->gmc.gart_size >> PAGE_SHIFT) - start; drm_mm_init(&mgr->mm, start, size); spin_lock_init(&mgr->lock); - atomic64_set(&mgr->used, 0); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); ttm_resource_manager_set_used(man, true); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 651c7abfde03..8d5d56d9d26d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -678,7 +678,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) ui64 = amdgpu_vram_mgr_vis_usage(ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM)); return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; case AMDGPU_INFO_GTT_USAGE: - ui64 = amdgpu_gtt_mgr_usage(ttm_manager_type(&adev->mman.bdev, TTM_PL_TT)); + ui64 = ttm_resource_manager_usage(ttm_manager_type(&adev->mman.bdev, TTM_PL_TT)); return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; case AMDGPU_INFO_GDS_CONFIG: { struct drm_amdgpu_info_gds gds_info; @@ -737,8 +737,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) mem.gtt.total_heap_size *= PAGE_SIZE; mem.gtt.usable_heap_size = mem.gtt.total_heap_size - atomic64_read(&adev->gart_pin_size); - mem.gtt.heap_usage = - amdgpu_gtt_mgr_usage(gtt_man); + mem.gtt.heap_usage = ttm_resource_manager_usage(gtt_man); mem.gtt.max_allocation = mem.gtt.usable_heap_size * 3 / 4; return copy_to_user(out, &mem, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h index 7346ecff4438..2ac332cd4f59 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h @@ -52,7 +52,6 @@ struct amdgpu_gtt_mgr { struct ttm_resource_manager manager; struct drm_mm mm; spinlock_t lock; - atomic64_t used; }; struct amdgpu_preempt_mgr { @@ -114,7 +113,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev); void amdgpu_vram_mgr_fini(struct amdgpu_device *adev); bool amdgpu_gtt_mgr_has_gart_addr(struct ttm_resource *mem); -uint64_t amdgpu_gtt_mgr_usage(struct ttm_resource_manager *man); int amdgpu_gtt_mgr_recover(struct ttm_resource_manager *man); uint64_t amdgpu_preempt_mgr_usage(struct ttm_resource_manager *man); From patchwork Mon Jan 24 12:25:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722028 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4FFB5C433FE for ; Mon, 24 Jan 2022 12:25:44 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8905A10EA4B; Mon, 24 Jan 2022 12:25:42 +0000 (UTC) Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by gabe.freedesktop.org (Postfix) with ESMTPS id E768710E997 for ; Mon, 24 Jan 2022 12:25:24 +0000 (UTC) Received: by mail-wr1-x42e.google.com with SMTP id r14so2685153wrc.6 for ; Mon, 24 Jan 2022 04:25:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=JPyPAPdxhdSJD2/USY4sqrmS4Bru8vhGwzHcwPjKAcg=; b=DpHc4o9Ho7lQAdDedIpFgwdYBWVQRbeQTXQqQRdalPjyqduyqbaVjHhBxZHzAH2/pN mYlfSL7Eb3CAvkVZdYUpUMY7SM9rwwoTRbjQ2S0dhkXusDbzXiiEtrgSHGlC7+CjwK5C DyxzJZqoaLLjwn2Yrzra/zPI6pjiE+c53+R22cKpRREIeu1ex8sjSPqy6LMe72Ommo69 d8tvw2zBm8UIdSGmJdYvzRti+fSNSHf5n2+Fsy2vAvvXp+ui0p+or0AXP8CDPezAA1rd qSJzwbTOvJOXMeOx8ntdSeb/3Jvs5nxLJHY0/JwWPxATjL49GF5wSyym4MVOF1RVh06k gn7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JPyPAPdxhdSJD2/USY4sqrmS4Bru8vhGwzHcwPjKAcg=; b=AInZzhRhCBm9UwsvBKaszNNa3YCzFUg8LgU+txGCx5yrlZ6a1nfTFCQBYYM+BoA7ui ao4qlBMd2yaU/XpknGcLfS6op/xFoN48Yi+zdBMPZGHRo/GKYg1zJ98rav4ke54Nr2PB LUsaQNMsCos+AvYDP5OqoSzkTcw8oFasA+Asn3DwrNgsXPOiYSBQBfWUBz0bIUjwfrCc yzAfrnf1eJWm/NCFtRtWlK9EzrblzLrmFhrPn8X/7g2JUGu9b2esdm8ZVO0izrT0io6Q gZUT2yWPfXdGUeDUfqlKzKq1k8MKZusn8If5l4TgZwxgm/06nisghR75a4EWpMjqXJBk Yz3A== X-Gm-Message-State: AOAM533SJbiKvEVKfsBs9tpOrafKdnKnh8DvdJkI9IWsVkwc4NAa1X0u wKt9UaTAVKWVT+KC+E8g68g= X-Google-Smtp-Source: ABdhPJxYbcoMhNFq2R3ABHKrkFgsof8hx+NPizC0NM8MfDZ+tTK4EK57DHxrVfQwxNZzsWyNImspdQ== X-Received: by 2002:a05:6000:2ad:: with SMTP id l13mr13788692wry.374.1643027123506; Mon, 24 Jan 2022 04:25:23 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:23 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 09/12] drm/amdgpu: remove VRAM accounting Date: Mon, 24 Jan 2022 13:25:11 +0100 Message-Id: <20220124122514.1832-10-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This is provided by TTM now. Also switch man->size to bytes instead of pages and fix the double printing of size and usage in debugfs. Signed-off-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 2 - drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 56 +++++++------------- 7 files changed, 26 insertions(+), 48 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 46cf48b3904a..5a3ecdea20eb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -516,9 +516,10 @@ int amdgpu_amdkfd_get_dmabuf_info(struct amdgpu_device *adev, int dma_buf_fd, uint64_t amdgpu_amdkfd_get_vram_usage(struct amdgpu_device *adev) { - struct ttm_resource_manager *vram_man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM); + struct ttm_resource_manager *vram_man = + ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM); - return amdgpu_vram_mgr_usage(vram_man); + return ttm_resource_manager_usage(vram_man); } uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct amdgpu_device *dst, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 53e407ea4c89..936c1a7174f6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -315,7 +315,7 @@ static void amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev, } total_vram = adev->gmc.real_vram_size - atomic64_read(&adev->vram_pin_size); - used_vram = amdgpu_vram_mgr_usage(vram_man); + used_vram = ttm_resource_manager_usage(vram_man); free_vram = used_vram >= total_vram ? 0 : total_vram - used_vram; spin_lock(&adev->mm_stats.lock); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 8d5d56d9d26d..6d21ee430f03 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -672,7 +672,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) ui64 = atomic64_read(&adev->num_vram_cpu_page_faults); return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; case AMDGPU_INFO_VRAM_USAGE: - ui64 = amdgpu_vram_mgr_usage(ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM)); + ui64 = ttm_resource_manager_usage(ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM)); return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; case AMDGPU_INFO_VIS_VRAM_USAGE: ui64 = amdgpu_vram_mgr_vis_usage(ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM)); @@ -718,8 +718,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) mem.vram.usable_heap_size = adev->gmc.real_vram_size - atomic64_read(&adev->vram_pin_size) - AMDGPU_VM_RESERVED_VRAM; - mem.vram.heap_usage = - amdgpu_vram_mgr_usage(vram_man); + mem.vram.heap_usage = ttm_resource_manager_usage(vram_man); mem.vram.max_allocation = mem.vram.usable_heap_size * 3 / 4; mem.cpu_accessible_vram.total_heap_size = diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index eac2ff4647e3..f6b3695979a6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -1873,7 +1873,7 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable) size = adev->gmc.real_vram_size; else size = adev->gmc.visible_vram_size; - man->size = size >> PAGE_SHIFT; + man->size = size; adev->mman.buffer_funcs_enabled = enable; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h index 2ac332cd4f59..f3eb4029bdcf 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h @@ -44,7 +44,6 @@ struct amdgpu_vram_mgr { spinlock_t lock; struct list_head reservations_pending; struct list_head reserved_pages; - atomic64_t usage; atomic64_t vis_usage; }; @@ -127,7 +126,6 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev, void amdgpu_vram_mgr_free_sgt(struct device *dev, enum dma_data_direction dir, struct sg_table *sgt); -uint64_t amdgpu_vram_mgr_usage(struct ttm_resource_manager *man); uint64_t amdgpu_vram_mgr_vis_usage(struct ttm_resource_manager *man); int amdgpu_vram_mgr_reserve_range(struct ttm_resource_manager *man, uint64_t start, uint64_t size); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c index 3fc49823f527..4a23dfe39556 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c @@ -576,7 +576,7 @@ static int amdgpu_virt_write_vf2pf_data(struct amdgpu_device *adev) vf2pf_info->driver_cert = 0; vf2pf_info->os_info.all = 0; - vf2pf_info->fb_usage = amdgpu_vram_mgr_usage(vram_man) >> 20; + vf2pf_info->fb_usage = ttm_resource_manager_usage(vram_man) >> 20; vf2pf_info->fb_vis_usage = amdgpu_vram_mgr_vis_usage(vram_man) >> 20; vf2pf_info->fb_size = adev->gmc.real_vram_size >> 20; vf2pf_info->fb_vis_size = adev->gmc.visible_vram_size >> 20; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index ddd0b6d74070..862c0f3d2329 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -99,7 +99,7 @@ static ssize_t amdgpu_mem_info_vram_used_show(struct device *dev, struct ttm_resource_manager *man; man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM); - return sysfs_emit(buf, "%llu\n", amdgpu_vram_mgr_usage(man)); + return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man)); } /** @@ -255,7 +255,7 @@ static void amdgpu_vram_mgr_do_reserve(struct ttm_resource_manager *man) vis_usage = amdgpu_vram_mgr_vis_size(adev, &rsv->mm_node); atomic64_add(vis_usage, &mgr->vis_usage); - atomic64_add(rsv->mm_node.size << PAGE_SHIFT, &mgr->usage); + atomic64_add(rsv->mm_node.size << PAGE_SHIFT, &man->usage); list_move(&rsv->node, &mgr->reserved_pages); } } @@ -382,19 +382,13 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, lpfn = place->lpfn; if (!lpfn) - lpfn = man->size; + lpfn = man->size >> PAGE_SHIFT; max_bytes = adev->gmc.mc_vram_size; if (tbo->type != ttm_bo_type_kernel) max_bytes -= AMDGPU_VM_RESERVED_VRAM; - /* bail out quickly if there's likely not enough VRAM for this BO */ mem_bytes = tbo->base.size; - if (atomic64_add_return(mem_bytes, &mgr->usage) > max_bytes) { - r = -ENOSPC; - goto error_sub; - } - if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { pages_per_node = ~0ul; num_nodes = 1; @@ -412,13 +406,17 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, node = kvmalloc(struct_size(node, mm_nodes, num_nodes), GFP_KERNEL | __GFP_ZERO); - if (!node) { - r = -ENOMEM; - goto error_sub; - } + if (!node) + return -ENOMEM; ttm_resource_init(tbo, place, &node->base); + /* bail out quickly if there's likely not enough VRAM for this BO */ + if (ttm_resource_manager_usage(man) > max_bytes) { + r = -ENOSPC; + goto error_fini; + } + mode = DRM_MM_INSERT_BEST; if (place->flags & TTM_PL_FLAG_TOPDOWN) mode = DRM_MM_INSERT_HIGH; @@ -476,11 +474,10 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, while (i--) drm_mm_remove_node(&node->mm_nodes[i]); spin_unlock(&mgr->lock); +error_fini: ttm_resource_fini(man, &node->base); kvfree(node); -error_sub: - atomic64_sub(mem_bytes, &mgr->usage); return r; } @@ -498,7 +495,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, struct ttm_range_mgr_node *node = to_ttm_range_mgr_node(res); struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); struct amdgpu_device *adev = to_amdgpu_device(mgr); - uint64_t usage = 0, vis_usage = 0; + uint64_t vis_usage = 0; unsigned i, pages; spin_lock(&mgr->lock); @@ -507,13 +504,11 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, struct drm_mm_node *mm = &node->mm_nodes[i]; drm_mm_remove_node(mm); - usage += mm->size << PAGE_SHIFT; vis_usage += amdgpu_vram_mgr_vis_size(adev, mm); } amdgpu_vram_mgr_do_reserve(man); spin_unlock(&mgr->lock); - atomic64_sub(usage, &mgr->usage); atomic64_sub(vis_usage, &mgr->vis_usage); ttm_resource_fini(man, res); @@ -631,20 +626,6 @@ void amdgpu_vram_mgr_free_sgt(struct device *dev, kfree(sgt); } -/** - * amdgpu_vram_mgr_usage - how many bytes are used in this domain - * - * @man: TTM memory type manager - * - * Returns how many bytes are used in this domain. - */ -uint64_t amdgpu_vram_mgr_usage(struct ttm_resource_manager *man) -{ - struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); - - return atomic64_read(&mgr->usage); -} - /** * amdgpu_vram_mgr_vis_usage - how many bytes are used in the visible part * @@ -672,13 +653,12 @@ static void amdgpu_vram_mgr_debug(struct ttm_resource_manager *man, { struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); + drm_printf(printer, " vis usage:%llu\n", + amdgpu_vram_mgr_vis_usage(man)); + spin_lock(&mgr->lock); drm_mm_print(&mgr->mm, printer); spin_unlock(&mgr->lock); - - drm_printf(printer, "man size:%llu pages, ram usage:%lluMB, vis usage:%lluMB\n", - man->size, amdgpu_vram_mgr_usage(man) >> 20, - amdgpu_vram_mgr_vis_usage(man) >> 20); } static const struct ttm_resource_manager_func amdgpu_vram_mgr_func = { @@ -700,11 +680,11 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) struct ttm_resource_manager *man = &mgr->manager; ttm_resource_manager_init(man, &adev->mman.bdev, - adev->gmc.real_vram_size >> PAGE_SHIFT); + adev->gmc.real_vram_size); man->func = &amdgpu_vram_mgr_func; - drm_mm_init(&mgr->mm, 0, man->size); + drm_mm_init(&mgr->mm, 0, man->size >> PAGE_SHIFT); spin_lock_init(&mgr->lock); INIT_LIST_HEAD(&mgr->reservations_pending); INIT_LIST_HEAD(&mgr->reserved_pages); From patchwork Mon Jan 24 12:25:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722030 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 62535C433EF for ; Mon, 24 Jan 2022 12:25:50 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5E56D10E997; Mon, 24 Jan 2022 12:25:48 +0000 (UTC) Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8F15C10E997 for ; Mon, 24 Jan 2022 12:25:25 +0000 (UTC) Received: by mail-wr1-x432.google.com with SMTP id w11so1102798wra.4 for ; Mon, 24 Jan 2022 04:25:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=s3Oc5Fovoof+St9z2rSoNTT35y0Fxll9qvvgsIYRPS8=; b=VozZz690lLnNS90BWIjn6DJ7usDAUt/gfyF2JZg2E7eqt4nuOdXmG/B/s45G55pjMs +VEjAESb7lTXETCY3kvR0MLpFgJixIDJAL/Fo1K3493NcU8X0UojJ8cKSCbvL/wYIzf4 dehSOvZu+KN/O5hh1pCnbe8aZWZwjE/oYEqB3PSSvk4n0RP2u6u673yaxOD26LXspyyf IzZqHk8FK20tY7Pgo5G0KuZBtEO9F7XCs+u1plhMSbNT+OLozs/5klEFWfDFxpDZwtu2 aQcYgj08aegxwETfzcT7OdKmSIhCHZBhlYuWi9jqU3ghi2PBFMJAExB8lRV9ZLwodgxw dDFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s3Oc5Fovoof+St9z2rSoNTT35y0Fxll9qvvgsIYRPS8=; b=x+oz4GmvZk3a2cduCMILHrij9l8AbhBrf3o3oZXdtEIOhMQ/Kxwo3fAi4hIjbWBHAu q37gD+SsPx7n5u+T6fo5aVzIfQYDFT7dSugrFS2hRy64Dv4VULgO6s98zca6+9H3uMHn OON3H7jUdLBn8amzT1ICxXNRsfk2g+1IrngPTFBd+oTdotlJayYwmYAgGcSmtGqn0QmB F6qjxoU576fiiV2s7kwTl4/jQHxxMTqgmp8EjtfI5W7V1a7oS16BJq97Sfo5QfUf/8v8 laYBTQB5fT+rX1Bt5Wv6P9XoNzZwHz4b3caCgk+SsR482xk0Sk8VwCynxQZcb5ZC8YQK XYnw== X-Gm-Message-State: AOAM533rlBDcrSeUVoEbI8klaBo6Qyj1LHUNIM1877UQf6Wrilluxfrr 3mYHQHUUJAC7j+wY6BKxy4XJ6xKrHMs= X-Google-Smtp-Source: ABdhPJwW2mVTTk9XGsAc/e2hpzrUhIfvoW72FiO2cw6Ts7OFeBQ24rFXg51SKVrsBdUdjTX1VUPW/Q== X-Received: by 2002:a5d:6548:: with SMTP id z8mr7251524wrv.297.1643027124204; Mon, 24 Jan 2022 04:25:24 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:23 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 10/12] drm/amdgpu: drop amdgpu_gtt_node Date: Mon, 24 Jan 2022 13:25:12 +0100 Message-Id: <20220124122514.1832-11-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We have the BO pointer in the base structure now as well. Signed-off-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 49 ++++++++------------- 1 file changed, 18 insertions(+), 31 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index ce5eeb3c1097..a55bbe1a154c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -26,23 +26,12 @@ #include "amdgpu.h" -struct amdgpu_gtt_node { - struct ttm_buffer_object *tbo; - struct ttm_range_mgr_node base; -}; - static inline struct amdgpu_gtt_mgr * to_gtt_mgr(struct ttm_resource_manager *man) { return container_of(man, struct amdgpu_gtt_mgr, manager); } -static inline struct amdgpu_gtt_node * -to_amdgpu_gtt_node(struct ttm_resource *res) -{ - return container_of(res, struct amdgpu_gtt_node, base.base); -} - /** * DOC: mem_info_gtt_total * @@ -107,9 +96,9 @@ const struct attribute_group amdgpu_gtt_mgr_attr_group = { */ bool amdgpu_gtt_mgr_has_gart_addr(struct ttm_resource *res) { - struct amdgpu_gtt_node *node = to_amdgpu_gtt_node(res); + struct ttm_range_mgr_node *node = to_ttm_range_mgr_node(res); - return drm_mm_node_allocated(&node->base.mm_nodes[0]); + return drm_mm_node_allocated(&node->mm_nodes[0]); } /** @@ -129,15 +118,14 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, { struct amdgpu_gtt_mgr *mgr = to_gtt_mgr(man); uint32_t num_pages = PFN_UP(tbo->base.size); - struct amdgpu_gtt_node *node; + struct ttm_range_mgr_node *node; int r; - node = kzalloc(struct_size(node, base.mm_nodes, 1), GFP_KERNEL); + node = kzalloc(struct_size(node, mm_nodes, 1), GFP_KERNEL); if (!node) return -ENOMEM; - node->tbo = tbo; - ttm_resource_init(tbo, place, &node->base.base); + ttm_resource_init(tbo, place, &node->base); if (!(place->flags & TTM_PL_FLAG_TEMPORARY) && ttm_resource_manager_usage(man) > man->size) { r = -ENOSPC; @@ -146,8 +134,7 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, if (place->lpfn) { spin_lock(&mgr->lock); - r = drm_mm_insert_node_in_range(&mgr->mm, - &node->base.mm_nodes[0], + r = drm_mm_insert_node_in_range(&mgr->mm, &node->mm_nodes[0], num_pages, tbo->page_alignment, 0, place->fpfn, place->lpfn, DRM_MM_INSERT_BEST); @@ -155,18 +142,18 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, if (unlikely(r)) goto err_free; - node->base.base.start = node->base.mm_nodes[0].start; + node->base.start = node->mm_nodes[0].start; } else { - node->base.mm_nodes[0].start = 0; - node->base.mm_nodes[0].size = node->base.base.num_pages; - node->base.base.start = AMDGPU_BO_INVALID_OFFSET; + node->mm_nodes[0].start = 0; + node->mm_nodes[0].size = node->base.num_pages; + node->base.start = AMDGPU_BO_INVALID_OFFSET; } - *res = &node->base.base; + *res = &node->base; return 0; err_free: - ttm_resource_fini(man, &node->base.base); + ttm_resource_fini(man, &node->base); kfree(node); return r; } @@ -182,12 +169,12 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, static void amdgpu_gtt_mgr_del(struct ttm_resource_manager *man, struct ttm_resource *res) { - struct amdgpu_gtt_node *node = to_amdgpu_gtt_node(res); + struct ttm_range_mgr_node *node = to_ttm_range_mgr_node(res); struct amdgpu_gtt_mgr *mgr = to_gtt_mgr(man); spin_lock(&mgr->lock); - if (drm_mm_node_allocated(&node->base.mm_nodes[0])) - drm_mm_remove_node(&node->base.mm_nodes[0]); + if (drm_mm_node_allocated(&node->mm_nodes[0])) + drm_mm_remove_node(&node->mm_nodes[0]); spin_unlock(&mgr->lock); ttm_resource_fini(man, res); @@ -204,16 +191,16 @@ static void amdgpu_gtt_mgr_del(struct ttm_resource_manager *man, int amdgpu_gtt_mgr_recover(struct ttm_resource_manager *man) { struct amdgpu_gtt_mgr *mgr = to_gtt_mgr(man); + struct ttm_range_mgr_node *node; struct amdgpu_device *adev; - struct amdgpu_gtt_node *node; struct drm_mm_node *mm_node; int r = 0; adev = container_of(mgr, typeof(*adev), mman.gtt_mgr); spin_lock(&mgr->lock); drm_mm_for_each_node(mm_node, &mgr->mm) { - node = container_of(mm_node, typeof(*node), base.mm_nodes[0]); - r = amdgpu_ttm_recover_gart(node->tbo); + node = container_of(mm_node, typeof(*node), mm_nodes[0]); + r = amdgpu_ttm_recover_gart(node->base.bo); if (r) break; } From patchwork Mon Jan 24 12:25:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722031 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 12512C433F5 for ; Mon, 24 Jan 2022 12:25:56 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2C4D010EAC6; Mon, 24 Jan 2022 12:25:55 +0000 (UTC) Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by gabe.freedesktop.org (Postfix) with ESMTPS id 40DD610EA87 for ; Mon, 24 Jan 2022 12:25:26 +0000 (UTC) Received: by mail-wr1-x434.google.com with SMTP id h29so10610318wrb.5 for ; Mon, 24 Jan 2022 04:25:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=reNHSkedLLRdsPCa9ikZR9nGjtzrqWMgZdGa7MJh10c=; b=VTpXD2rZBdLPNCpNxoBPpwW681oLuToi9wMSAZOWH8Jk23EQEDAI01y/ip87pc2Drb VKK9z6+yWgStPbljQ9U8yPjbZDuwpHgozrboXK38yDd6D87Slx8+HrbSeS4WamS49x5O O+4IaXROir4aYOCDaHeffEmiSCvtCiUU+aw5rHgrBrZ4V8H5ToJihrwg7XFNtXelv+Qq jBDyMBNlWBQ/+oFeN5HlO546jQBerMpoZI9eUSr+1k/zKRFYxLQCx6KFBrE0NC06plbL Qd6AwACvYNKVFOlUPMxCVJw1nzHaw6/kP0qPxu4ko32vqpnAosx2PQ1UahcOEwXFZb7s S3eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=reNHSkedLLRdsPCa9ikZR9nGjtzrqWMgZdGa7MJh10c=; b=I2F2FyABcxfaD4bbphaWbESd5dhB6Bxl5j+KoYlFfcaeWIT9tl59QdEpmlsi8U2g8I w6UGS3FnGji9RmBff2bY5SUQoW7Un3wm74nUAE5irPh25+Tru+qORnfenfIBIN77IoZh cwvCKi3gTaHH1+CbjdRo6U+Z0kElWcOlG6sR8a81HoB2ji0orbzz8UODCoS30P5qsDQD AzEZT/S90vo8YdUuCCw4gUMZBCashrFhfusRwYjm6MJiinCmWkokobDJIcD0jPDMZAZI DKPjpP/4FH4sy5hkEMYW2+nEnZEn4JYGPFW6wSPh3zSWOlKZhAlAl2Jc+qB0v+J+dVEF riMw== X-Gm-Message-State: AOAM530lyisMXMbtCzJxFyIJVQmkKpCrIdh58umKC1PPcG2U1Xvq7uUD xMz8xJiTwzvb/AUezJNCpU0= X-Google-Smtp-Source: ABdhPJwWEyJy0McgdI+OVgJtw9uurfAFuWwMr/84u3g4PL9SpkyhmaVPFV+HHqvqPdSXAvemJxMX5w== X-Received: by 2002:adf:e351:: with SMTP id n17mr13785982wrj.243.1643027124901; Mon, 24 Jan 2022 04:25:24 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:24 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 11/12] drm/ttm: allow bulk moves for all domains Date: Mon, 24 Jan 2022 13:25:13 +0100 Message-Id: <20220124122514.1832-12-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Not just TT and VRAM. Signed-off-by: Christian König Reviewed-by: Daniel Vetter --- drivers/gpu/drm/ttm/ttm_resource.c | 49 +++++++++--------------------- include/drm/ttm/ttm_device.h | 2 -- include/drm/ttm/ttm_resource.h | 4 +-- 3 files changed, 16 insertions(+), 39 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index 9e68d36a1546..2ea8fb83377e 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -51,36 +51,23 @@ EXPORT_SYMBOL(ttm_lru_bulk_move_init); */ void ttm_lru_bulk_move_tail(struct ttm_lru_bulk_move *bulk) { - unsigned i; - - for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { - struct ttm_lru_bulk_move_pos *pos = &bulk->tt[i]; - struct ttm_resource_manager *man; + unsigned i, j; - if (!pos->first) - continue; + for (i = 0; i < TTM_NUM_MEM_TYPES; ++i) { + for (j = 0; j < TTM_MAX_BO_PRIORITY; ++j) { + struct ttm_lru_bulk_move_pos *pos = &bulk->pos[i][j]; + struct ttm_resource_manager *man; - dma_resv_assert_held(pos->first->bo->base.resv); - dma_resv_assert_held(pos->last->bo->base.resv); + if (!pos->first) + continue; - man = ttm_manager_type(pos->first->bo->bdev, TTM_PL_TT); - list_bulk_move_tail(&man->lru[i], &pos->first->lru, - &pos->last->lru); - } - - for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { - struct ttm_lru_bulk_move_pos *pos = &bulk->vram[i]; - struct ttm_resource_manager *man; + dma_resv_assert_held(pos->first->bo->base.resv); + dma_resv_assert_held(pos->last->bo->base.resv); - if (!pos->first) - continue; - - dma_resv_assert_held(pos->first->bo->base.resv); - dma_resv_assert_held(pos->last->bo->base.resv); - - man = ttm_manager_type(pos->first->bo->bdev, TTM_PL_VRAM); - list_bulk_move_tail(&man->lru[i], &pos->first->lru, - &pos->last->lru); + man = ttm_manager_type(pos->first->bo->bdev, i); + list_bulk_move_tail(&man->lru[j], &pos->first->lru, + &pos->last->lru); + } } } EXPORT_SYMBOL(ttm_lru_bulk_move_tail); @@ -118,15 +105,7 @@ void ttm_resource_move_to_lru_tail(struct ttm_resource *res, if (!bulk) return; - switch (res->mem_type) { - case TTM_PL_TT: - ttm_lru_bulk_move_set_pos(&bulk->tt[bo->priority], res); - break; - - case TTM_PL_VRAM: - ttm_lru_bulk_move_set_pos(&bulk->vram[bo->priority], res); - break; - } + ttm_lru_bulk_move_set_pos(&bulk->pos[res->mem_type][bo->priority], res); } void ttm_resource_init(struct ttm_buffer_object *bo, diff --git a/include/drm/ttm/ttm_device.h b/include/drm/ttm/ttm_device.h index 0a4ddec78d8f..425150f35fbe 100644 --- a/include/drm/ttm/ttm_device.h +++ b/include/drm/ttm/ttm_device.h @@ -30,8 +30,6 @@ #include #include -#define TTM_NUM_MEM_TYPES 8 - struct ttm_device; struct ttm_placement; struct ttm_buffer_object; diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index 13da5e337350..1556d1f62251 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -37,6 +37,7 @@ #include #define TTM_MAX_BO_PRIORITY 4U +#define TTM_NUM_MEM_TYPES 8 struct ttm_device; struct ttm_resource_manager; @@ -216,8 +217,7 @@ struct ttm_lru_bulk_move_pos { * Helper structure for bulk moves on the LRU list. */ struct ttm_lru_bulk_move { - struct ttm_lru_bulk_move_pos tt[TTM_MAX_BO_PRIORITY]; - struct ttm_lru_bulk_move_pos vram[TTM_MAX_BO_PRIORITY]; + struct ttm_lru_bulk_move_pos pos[TTM_NUM_MEM_TYPES][TTM_MAX_BO_PRIORITY]; }; /** From patchwork Mon Jan 24 12:25:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12722029 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 89EAFC433F5 for ; Mon, 24 Jan 2022 12:25:49 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id EF20510EACE; Mon, 24 Jan 2022 12:25:47 +0000 (UTC) Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4291D10EAC6 for ; Mon, 24 Jan 2022 12:25:27 +0000 (UTC) Received: by mail-wr1-x42a.google.com with SMTP id v13so13239113wrv.10 for ; Mon, 24 Jan 2022 04:25:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ndW8uUyRFxJfWAN2bXLKUPMH6H/SKnJ2BF2hNlVGvs8=; b=BImV+18SDZ7ZfX/p76hSLb/0K9tx+a1gexske4/0Oiy8BsSHCZy1bJZUqAcDpizv4H 5WHRLjQunCya547zQAQqoNBE+WR0bjVcKmtQS6vCMQKVhx5YuTIL65zVmdyaUgYUs31D JQY8AsHDas1OE6ccxUP+CTyADhKgYQPmOMOEkwwd3fSMhlLGd9IpzY/5raMDxAyxBkKl hoG6Yyx8pXJNfJjlpTm8nZUf+KS1FyP9E/8An7dVRG6OlZP8pXpZ5JkbGDd75SMAn6Db ZScG7NJ6J5m/heoLRb7jbsokSUFXYNI9rGw/w8FtUIjPnK/sH2Wfe4J8P8xin8jfRqmN EkOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ndW8uUyRFxJfWAN2bXLKUPMH6H/SKnJ2BF2hNlVGvs8=; b=GbkXmoC24mxjmKRaPukpl9+oFmS4SS/Cl4GOqxZGCG36fUHbrv3eA53PR2GyCN01JG FYxT7oywEcUriyRf/7p8pPAu9IHm4vAzlNjnT+RnvV3/nsl2BJoBLGGmgTwDuKnrKTfq wy+MIciTux3sYNpsQYAde2i2htOsjU9V+UNn0xHMeMAeRqAz6iykAa1SdBGkvVqOSOhs q12D/59ocqiIsmNSpovBKPx1SX/WHY3mTvODBF5HNW8oUhN8akdea3fqqUcnwVskN1aL i0KVl2AkFQnHH3Cx31sPBsikGGLuyr09SkbAWuYuE1MNXWkaAnfS2NG+/P+w4zj87AZv N5Sg== X-Gm-Message-State: AOAM531ifzerWLjuLLv+2e26WxF+sToeWwxb+8EgtcLLs1GdusZOSB77 tDK6faBVZPilpibvsNNbEyU= X-Google-Smtp-Source: ABdhPJz8RIjZF00UI7t9LYoY6LoHuCpmHvRmHcUaNhepChUToVoqIaG4H7/gMFz1kpc0GnNex4+7aw== X-Received: by 2002:a05:6000:18a2:: with SMTP id b2mr4768576wri.282.1643027125763; Mon, 24 Jan 2022 04:25:25 -0800 (PST) Received: from abel.fritz.box (p57b0bff8.dip0.t-ipconnect.de. [87.176.191.248]) by smtp.gmail.com with ESMTPSA id 1sm12830516wmo.37.2022.01.24.04.25.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 04:25:25 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: ray.huang@amd.com, thomas.hellstrom@linux.intel.com, dri-devel@lists.freedesktop.org, bas@basnieuwenhuizen.nl Subject: [PATCH 12/12] drm/ttm: rework bulk move handling Date: Mon, 24 Jan 2022 13:25:14 +0100 Message-Id: <20220124122514.1832-13-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124122514.1832-1-christian.koenig@amd.com> References: <20220124122514.1832-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Instead of providing the bulk move structure for each LRU update set this as property of the BO. This should avoid costly bulk move rebuilds with some games under RADV. Signed-off-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 1 - drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 72 +++----------------- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 3 - drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 2 +- drivers/gpu/drm/ttm/ttm_bo.c | 42 ++++++++++-- drivers/gpu/drm/ttm/ttm_resource.c | 87 ++++++++++++++++++------- include/drm/ttm/ttm_bo_api.h | 18 ++--- include/drm/ttm/ttm_bo_driver.h | 2 +- include/drm/ttm/ttm_device.h | 9 --- include/drm/ttm/ttm_resource.h | 7 +- 10 files changed, 124 insertions(+), 119 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index f6b3695979a6..7424a4a8d35c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -1497,7 +1497,6 @@ static struct ttm_device_funcs amdgpu_bo_driver = { .io_mem_reserve = &amdgpu_ttm_io_mem_reserve, .io_mem_pfn = amdgpu_ttm_io_mem_pfn, .access_memory = &amdgpu_ttm_access_memory, - .del_from_lru_notify = &amdgpu_vm_del_from_lru_notify }; /* diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index cdb489cdf27b..35ed849cfe25 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -375,7 +375,7 @@ static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, if (bo->tbo.base.resv != vm->root.bo->tbo.base.resv) return; - vm->bulk_moveable = false; + ttm_bo_set_bulk_move(&bo->tbo, &vm->lru_bulk_move); if (bo->tbo.type == ttm_bo_type_kernel && bo->parent) amdgpu_vm_bo_relocated(base); else @@ -637,36 +637,6 @@ void amdgpu_vm_get_pd_bo(struct amdgpu_vm *vm, list_add(&entry->tv.head, validated); } -/** - * amdgpu_vm_del_from_lru_notify - update bulk_moveable flag - * - * @bo: BO which was removed from the LRU - * - * Make sure the bulk_moveable flag is updated when a BO is removed from the - * LRU. - */ -void amdgpu_vm_del_from_lru_notify(struct ttm_buffer_object *bo) -{ - struct amdgpu_bo *abo; - struct amdgpu_vm_bo_base *bo_base; - - if (!amdgpu_bo_is_amdgpu_bo(bo)) - return; - - if (bo->pin_count) - return; - - abo = ttm_to_amdgpu_bo(bo); - if (!abo->parent) - return; - for (bo_base = abo->vm_bo; bo_base; bo_base = bo_base->next) { - struct amdgpu_vm *vm = bo_base->vm; - - if (abo->tbo.base.resv == vm->root.bo->tbo.base.resv) - vm->bulk_moveable = false; - } - -} /** * amdgpu_vm_move_to_lru_tail - move all BOs to the end of LRU * @@ -679,33 +649,9 @@ void amdgpu_vm_del_from_lru_notify(struct ttm_buffer_object *bo) void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev, struct amdgpu_vm *vm) { - struct amdgpu_vm_bo_base *bo_base; - - if (vm->bulk_moveable) { - spin_lock(&adev->mman.bdev.lru_lock); - ttm_lru_bulk_move_tail(&vm->lru_bulk_move); - spin_unlock(&adev->mman.bdev.lru_lock); - return; - } - - ttm_lru_bulk_move_init(&vm->lru_bulk_move); - spin_lock(&adev->mman.bdev.lru_lock); - list_for_each_entry(bo_base, &vm->idle, vm_status) { - struct amdgpu_bo *bo = bo_base->bo; - struct amdgpu_bo *shadow = amdgpu_bo_shadowed(bo); - - if (!bo->parent) - continue; - - ttm_bo_move_to_lru_tail(&bo->tbo, &vm->lru_bulk_move); - if (shadow) - ttm_bo_move_to_lru_tail(&shadow->tbo, - &vm->lru_bulk_move); - } + ttm_lru_bulk_move_tail(&vm->lru_bulk_move); spin_unlock(&adev->mman.bdev.lru_lock); - - vm->bulk_moveable = true; } /** @@ -728,8 +674,6 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm, struct amdgpu_vm_bo_base *bo_base, *tmp; int r; - vm->bulk_moveable &= list_empty(&vm->evicted); - list_for_each_entry_safe(bo_base, tmp, &vm->evicted, vm_status) { struct amdgpu_bo *bo = bo_base->bo; struct amdgpu_bo *shadow = amdgpu_bo_shadowed(bo); @@ -1047,10 +991,16 @@ static void amdgpu_vm_free_table(struct amdgpu_vm_bo_base *entry) if (!entry->bo) return; + shadow = amdgpu_bo_shadowed(entry->bo); + if (shadow) { + ttm_bo_set_bulk_move(&shadow->tbo, NULL); + amdgpu_bo_unref(&shadow); + } + + ttm_bo_set_bulk_move(&entry->bo->tbo, NULL); entry->bo->vm_bo = NULL; list_del(&entry->vm_status); - amdgpu_bo_unref(&shadow); amdgpu_bo_unref(&entry->bo); } @@ -1070,8 +1020,6 @@ static void amdgpu_vm_free_pts(struct amdgpu_device *adev, struct amdgpu_vm_pt_cursor cursor; struct amdgpu_vm_bo_base *entry; - vm->bulk_moveable = false; - for_each_amdgpu_vm_pt_dfs_safe(adev, vm, start, cursor, entry) amdgpu_vm_free_table(entry); @@ -2651,7 +2599,7 @@ void amdgpu_vm_bo_rmv(struct amdgpu_device *adev, if (bo) { if (bo->tbo.base.resv == vm->root.bo->tbo.base.resv) - vm->bulk_moveable = false; + ttm_bo_set_bulk_move(&bo->tbo, NULL); for (base = &bo_va->base.bo->vm_bo; *base; base = &(*base)->next) { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index 85fcfb8c5efd..4d236682a118 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -317,8 +317,6 @@ struct amdgpu_vm { /* Store positions of group of BOs */ struct ttm_lru_bulk_move lru_bulk_move; - /* mark whether can do the bulk move */ - bool bulk_moveable; /* Flag to indicate if VM is used for compute */ bool is_compute_context; }; @@ -454,7 +452,6 @@ void amdgpu_vm_set_task_info(struct amdgpu_vm *vm); void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev, struct amdgpu_vm *vm); -void amdgpu_vm_del_from_lru_notify(struct ttm_buffer_object *bo); void amdgpu_vm_get_memory(struct amdgpu_vm *vm, uint64_t *vram_mem, uint64_t *gtt_mem, uint64_t *cpu_mem); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index cdf1ff87c88d..abd1bde7a2b8 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -835,7 +835,7 @@ void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj) bo->priority = I915_TTM_PRIO_NO_PAGES; } - ttm_bo_move_to_lru_tail(bo, NULL); + ttm_bo_move_to_lru_tail(bo); spin_unlock(&bo->bdev->lru_lock); } diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index 599be3dda8a9..7d1133bef33e 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -69,16 +69,48 @@ static void ttm_bo_mem_space_debug(struct ttm_buffer_object *bo, } } -void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, - struct ttm_lru_bulk_move *bulk) +/** + * ttm_bo_move_to_lru_tail + * + * @bo: The buffer object. + * + * Move this BO to the tail of all lru lists used to lookup and reserve an + * object. This function must be called with struct ttm_global::lru_lock + * held, and is used to make a BO less likely to be considered for eviction. + */ +void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo) { dma_resv_assert_held(bo->base.resv); if (bo->resource) - ttm_resource_move_to_lru_tail(bo->resource, bulk); + ttm_resource_move_to_lru_tail(bo->resource); } EXPORT_SYMBOL(ttm_bo_move_to_lru_tail); +/** + * ttm_bo_set_bulk_move - update BOs bulk move object + * + * @bo: The buffer object. + * + * Update the BOs bulk move object, making sure that resources are added/removed + * as well. + */ +void ttm_bo_set_bulk_move(struct ttm_buffer_object *bo, + struct ttm_lru_bulk_move *bulk) +{ + if (bo->bulk_move == bulk) + return; + + spin_lock(&bo->bdev->lru_lock); + if (bo->bulk_move && bo->resource) + ttm_lru_bulk_move_rmv(bo->bulk_move, bo->resource); + bo->bulk_move = bulk; + if (bo->bulk_move && bo->resource) + ttm_lru_bulk_move_add(bo->bulk_move, bo->resource); + spin_unlock(&bo->bdev->lru_lock); +} +EXPORT_SYMBOL(ttm_bo_set_bulk_move); + static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, struct ttm_resource *mem, bool evict, struct ttm_operation_ctx *ctx, @@ -316,6 +348,7 @@ static void ttm_bo_release(struct kref *kref) int ret; WARN_ON_ONCE(bo->pin_count); + WARN_ON_ONCE(bo->bulk_move); if (!bo->deleted) { ret = ttm_bo_individualize_resv(bo); @@ -352,7 +385,7 @@ static void ttm_bo_release(struct kref *kref) */ if (bo->pin_count) { bo->pin_count = 0; - ttm_resource_move_to_lru_tail(bo->resource, NULL); + ttm_resource_move_to_lru_tail(bo->resource); } kref_init(&bo->kref); @@ -874,6 +907,7 @@ int ttm_bo_init_reserved(struct ttm_device *bdev, bo->moving = NULL; bo->pin_count = 0; bo->sg = sg; + bo->bulk_move = NULL; if (resv) { bo->base.resv = resv; dma_resv_assert_held(bo->base.resv); diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index 2ea8fb83377e..b1068053434c 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -72,40 +72,75 @@ void ttm_lru_bulk_move_tail(struct ttm_lru_bulk_move *bulk) } EXPORT_SYMBOL(ttm_lru_bulk_move_tail); -/* Record a resource position in a bulk move structure */ -static void ttm_lru_bulk_move_set_pos(struct ttm_lru_bulk_move_pos *pos, - struct ttm_resource *res) +/* Return the bulk move pos object for this resource */ +static struct ttm_lru_bulk_move_pos * +ttm_lru_bulk_move_pos(struct ttm_lru_bulk_move *bulk, struct ttm_resource *res) { - if (!pos->first) + return &bulk->pos[res->mem_type][res->bo->priority]; +} + +/* Move the resource to the tail of the bulk move range */ +static void ttm_lru_bulk_move_pos_tail(struct ttm_lru_bulk_move_pos *pos, + struct ttm_resource *res) +{ + if (pos->last != res) { + list_move(&res->lru, &pos->last->lru); + pos->last = res; + } +} + +/* Add the resource to a bulk_move cursor */ +void ttm_lru_bulk_move_add(struct ttm_lru_bulk_move *bulk, + struct ttm_resource *res) +{ + struct ttm_lru_bulk_move_pos *pos = ttm_lru_bulk_move_pos(bulk, res); + + if (!pos->first) { pos->first = res; - pos->last = res; + pos->last = res; + } else { + ttm_lru_bulk_move_pos_tail(pos, res); + } +} + +/* Remove the resource from a bulk_move range */ +void ttm_lru_bulk_move_rmv(struct ttm_lru_bulk_move *bulk, + struct ttm_resource *res) +{ + struct ttm_lru_bulk_move_pos *pos = ttm_lru_bulk_move_pos(bulk, res); + + if (unlikely(pos->first == res && pos->last == res)) { + pos->first = NULL; + pos->last = NULL; + } else if (pos->first == res) { + pos->first = list_next_entry(res, lru); + } else if (pos->last == res) { + pos->last = list_prev_entry(res, lru); + } else { + list_move(&res->lru, &pos->last->lru); + } } -/* Move a resource to the LRU tail and track the bulk position */ -void ttm_resource_move_to_lru_tail(struct ttm_resource *res, - struct ttm_lru_bulk_move *bulk) +/* Move a resource to the LRU or bulk tail */ +void ttm_resource_move_to_lru_tail(struct ttm_resource *res) { struct ttm_buffer_object *bo = res->bo; struct ttm_device *bdev = bo->bdev; - struct ttm_resource_manager *man; if (bo->pin_count) { list_move_tail(&res->lru, &bdev->pinned); - if (bdev->funcs->del_from_lru_notify) - bdev->funcs->del_from_lru_notify(res->bo); - return; - } - man = ttm_manager_type(bdev, res->mem_type); - list_move_tail(&res->lru, &man->lru[bo->priority]); + } else if (bo->bulk_move) { + struct ttm_lru_bulk_move_pos *pos = + ttm_lru_bulk_move_pos(bo->bulk_move, res); - if (bdev->funcs->del_from_lru_notify) - bdev->funcs->del_from_lru_notify(bo); - - if (!bulk) - return; + ttm_lru_bulk_move_pos_tail(pos, res); + } else { + struct ttm_resource_manager *man; - ttm_lru_bulk_move_set_pos(&bulk->pos[res->mem_type][bo->priority], res); + man = ttm_manager_type(bdev, res->mem_type); + list_move_tail(&res->lru, &man->lru[bo->priority]); + } } void ttm_resource_init(struct ttm_buffer_object *bo, @@ -129,7 +164,7 @@ void ttm_resource_init(struct ttm_buffer_object *bo, atomic64_add(bo->base.size, &man->usage); spin_lock(&bo->bdev->lru_lock); - ttm_resource_move_to_lru_tail(res, NULL); + ttm_resource_move_to_lru_tail(res); spin_unlock(&bo->bdev->lru_lock); } EXPORT_SYMBOL(ttm_resource_init); @@ -148,8 +183,6 @@ void ttm_resource_fini(struct ttm_resource_manager *man, spin_lock(&bdev->lru_lock); list_del_init(&res->lru); - if (res->bo && bdev->funcs->del_from_lru_notify) - bdev->funcs->del_from_lru_notify(res->bo); spin_unlock(&bdev->lru_lock); atomic64_sub(res->num_pages << PAGE_SHIFT, &man->usage); @@ -173,6 +206,12 @@ void ttm_resource_free(struct ttm_buffer_object *bo, struct ttm_resource **res) if (!*res) return; + if (bo->bulk_move) { + spin_lock(&bo->bdev->lru_lock); + ttm_lru_bulk_move_rmv(bo->bulk_move, *res); + spin_unlock(&bo->bdev->lru_lock); + } + man = ttm_manager_type(bo->bdev, (*res)->mem_type); man->func->free(man, *res); *res = NULL; diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 3da77fc54552..b40c28e884a2 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -135,6 +135,7 @@ struct ttm_buffer_object { struct ttm_resource *resource; struct ttm_tt *ttm; bool deleted; + struct ttm_lru_bulk_move *bulk_move; /** * Members protected by the bdev::lru_lock. @@ -287,18 +288,11 @@ int ttm_bo_validate(struct ttm_buffer_object *bo, */ void ttm_bo_put(struct ttm_buffer_object *bo); -/** - * ttm_bo_move_to_lru_tail - * - * @bo: The buffer object. - * @bulk: optional bulk move structure to remember BO positions - * - * Move this BO to the tail of all lru lists used to lookup and reserve an - * object. This function must be called with struct ttm_global::lru_lock - * held, and is used to make a BO less likely to be considered for eviction. - */ -void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, - struct ttm_lru_bulk_move *bulk); +void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo); + + +void ttm_bo_set_bulk_move(struct ttm_buffer_object *bo, + struct ttm_lru_bulk_move *bulk); /** * ttm_bo_lock_delayed_workqueue diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h index 6c7352e13708..059a595e14e5 100644 --- a/include/drm/ttm/ttm_bo_driver.h +++ b/include/drm/ttm/ttm_bo_driver.h @@ -155,7 +155,7 @@ static inline void ttm_bo_move_to_lru_tail_unlocked(struct ttm_buffer_object *bo) { spin_lock(&bo->bdev->lru_lock); - ttm_bo_move_to_lru_tail(bo, NULL); + ttm_bo_move_to_lru_tail(bo); spin_unlock(&bo->bdev->lru_lock); } diff --git a/include/drm/ttm/ttm_device.h b/include/drm/ttm/ttm_device.h index 425150f35fbe..95b3c04b1ab9 100644 --- a/include/drm/ttm/ttm_device.h +++ b/include/drm/ttm/ttm_device.h @@ -198,15 +198,6 @@ struct ttm_device_funcs { int (*access_memory)(struct ttm_buffer_object *bo, unsigned long offset, void *buf, int len, int write); - /** - * struct ttm_bo_driver member del_from_lru_notify - * - * @bo: the buffer object deleted from lru - * - * notify driver that a BO was deleted from LRU. - */ - void (*del_from_lru_notify)(struct ttm_buffer_object *bo); - /** * Notify the driver that we're about to release a BO * diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index 1556d1f62251..d938ee78f536 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -320,10 +320,13 @@ ttm_resource_manager_usage(struct ttm_resource_manager *man) } void ttm_lru_bulk_move_init(struct ttm_lru_bulk_move *bulk); +void ttm_lru_bulk_move_add(struct ttm_lru_bulk_move *bulk, + struct ttm_resource *res); +void ttm_lru_bulk_move_rmv(struct ttm_lru_bulk_move *bulk, + struct ttm_resource *res); void ttm_lru_bulk_move_tail(struct ttm_lru_bulk_move *bulk); -void ttm_resource_move_to_lru_tail(struct ttm_resource *res, - struct ttm_lru_bulk_move *bulk); +void ttm_resource_move_to_lru_tail(struct ttm_resource *res); void ttm_resource_init(struct ttm_buffer_object *bo, const struct ttm_place *place,