From patchwork Tue Oct 31 13:40:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tatsuyuki Ishi X-Patchwork-Id: 13441539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DFB27C4167B for ; Tue, 31 Oct 2023 13:41:23 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C5B3C10E4CF; Tue, 31 Oct 2023 13:41:17 +0000 (UTC) Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by gabe.freedesktop.org (Postfix) with ESMTPS id EB8C210E4C9; Tue, 31 Oct 2023 13:41:08 +0000 (UTC) Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1ca85ff26afso10477705ad.1; Tue, 31 Oct 2023 06:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698759668; x=1699364468; darn=lists.freedesktop.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=o+E9JqwmhnjEhxbRIH2HvbjlloiERNKW1YLqGvHb+1g=; b=D4syMZn7N1W1PPQCByjy2pFqrbA40V8cx83GCamrQ9z9OtyL/7XmDUQOParJIOVl8c OSLy9YXCZIFse69UVRGuf9/+I/mx1+iThNiAlsjI2iG6hVYP3D2sMZSj6xE6mWmEnZjM gMODBWky9dQUur6jOjnBCrlBO9eXWkyrHR7QzK5Ook+BXf9IkUfV2aIJ0ZhomF7Hz1ji fLlKoJb9dmAz7aWHcDdnxhDhijyHnv40EIj9Ja2zY8gXA7mcuN6kj3Ej0W4Gz3/uRFDq 1eyncbPqa/LP2z8EgqVVS6ap399PujShZK3c0/dg3ACCUlpXZBuQS8OfbnCo2WIkS8mt /gKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698759668; x=1699364468; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o+E9JqwmhnjEhxbRIH2HvbjlloiERNKW1YLqGvHb+1g=; b=mSUESIM84hWgqXBjwfEyhQvQNrfdvfLpGUoHUWbs8oJaBEYl3qs1bdn1BVQCqcpEwm i/4dx0oUe/Qi9qcImxgPT0DbXKRupm/Nwr1/LxNFLBklnJomxLVpEKwVD63yFZNgQwcK XOsoqOJZZ5IWvGWc//xb5TNTYARx5ZfOvl7MMSgd5O2MgbK2Rj2O0Omd+6tCc+cTspS5 06QCj7qxWHOvcaGb4Po5JH5ykTh4OKCjQOFHKcsYFHyCwLUnwJjBc1iDn9i5qCGxG6lz dhEzCbFoeu98titAvBX7xGaXLhEJdx6zMA1qeSj3ii5nTTdZak2MmFvp0um1Tw+EswN3 ZBSQ== X-Gm-Message-State: AOJu0YynH971funEAA8BPc9VVJyolX6Emv/Nb5tveJ6BxwEVwGQR8IsX MaIino6p3uIf866R98r+4rOyQBkZq7CxhdWM X-Google-Smtp-Source: AGHT+IFxKiKSRFU/iDR2UL+tJOeZwpK6uNuPNA5/kg2HT+JKfty0Vxg3tkqkmwEjycGVVWXPErXG9g== X-Received: by 2002:a17:902:c649:b0:1cc:3202:dcca with SMTP id s9-20020a170902c64900b001cc3202dccamr8878869pls.2.1698759667880; Tue, 31 Oct 2023 06:41:07 -0700 (PDT) Received: from localhost (zz20184013906F627101.userreverse.dion.ne.jp. [111.98.113.1]) by smtp.gmail.com with ESMTPSA id jm23-20020a17090304d700b001bb892a7a67sm1341152plb.1.2023.10.31.06.41.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 Oct 2023 06:41:07 -0700 (PDT) From: Tatsuyuki Ishi To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 1/6] drm/amdgpu: Don't implicit sync PRT maps. Date: Tue, 31 Oct 2023 22:40:54 +0900 Message-ID: <20231031134059.171277-2-ishitatsuyuki@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231031134059.171277-1-ishitatsuyuki@gmail.com> References: <20231031134059.171277-1-ishitatsuyuki@gmail.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tatsuyuki Ishi , christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" These are considered map operations rather than unmap, and there is no point of doing implicit synchronization here. Signed-off-by: Tatsuyuki Ishi --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index f5daadcec865..7b9762f1cddd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -902,7 +902,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm, /* Implicitly sync to command submissions in the same VM before * unmapping. Sync to moving fences before mapping. */ - if (!(flags & AMDGPU_PTE_VALID)) + if (!(flags & (AMDGPU_PTE_VALID | AMDGPU_PTE_PRT))) sync_mode = AMDGPU_SYNC_EQ_OWNER; else sync_mode = AMDGPU_SYNC_EXPLICIT; From patchwork Tue Oct 31 13:40:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tatsuyuki Ishi X-Patchwork-Id: 13441538 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A96CBC4167B for ; Tue, 31 Oct 2023 13:41:18 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5193410E4CD; Tue, 31 Oct 2023 13:41:15 +0000 (UTC) Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by gabe.freedesktop.org (Postfix) with ESMTPS id E3F4210E4CD; Tue, 31 Oct 2023 13:41:11 +0000 (UTC) Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1cc28719cb0so6728615ad.0; Tue, 31 Oct 2023 06:41:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698759671; x=1699364471; darn=lists.freedesktop.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=P9RKdJJhcMSfPq+WZs/RP57f6lPkPicq5MCLVBn4mRI=; b=Jk5tmAdBNVTK5cJh1OqJvDRfkdR68OCSsLu/UXy6/CY1Se68aPqjo289Z0pQ0YHr+R GkM0n17SV8T09hyAUIGOIhfaaK8SvOKZIzejJPzut09hlY2OtpD0UzacBpP1aoQYZ0cA EaeSz+Jg9RwVkb4GxmiV9ljhyX7VXF6eTVkgjvJCHTxjVWic1j92ok9Cl3LFFbjLJexY PucNkUPGrZBZz74RMzj65zsdpK02fiJQNZ+QolKtTM5eFSH6ObfZzCUxeX26yfbMxeXJ pmwNrDySnRHS7c3RSCr0O7b3osfw1/lyd0ClFq0RcyIct9JYa7Hw7TZbkOFWIOuN5BWO Qkdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698759671; x=1699364471; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P9RKdJJhcMSfPq+WZs/RP57f6lPkPicq5MCLVBn4mRI=; b=vAYDPNrt7D3DGjEGl11AaRoIF85/KHey5w/CJfN1Js5hesqAANbC+IQo8X4TVM9xgY gp9Nlqe3oByUV2yYnYZi9VlvWDmQMrAoO4Jygr3dFovjXbCyXfn3MsJgyI6ZS3cDxVdd JBZw8ZAEmi+AI4IYNpUPQhjg3W7iE0Yib7pxuNR/Wv5nLKjeSsWAaaDuJ9x9La2/Xj/2 AVra5NXIKWGJJON4eLA0sq/hELz0mAL77nZSe0ToM3Ltgs6qpKMX6LQP/ynN3/KsbmVC R7B1h3RHshY5EEDc6EzKIkCJeK9Bes1f3PJq2uB0zOPgUMn7cT8a6eVu7doTS5i/Usq3 AHZg== X-Gm-Message-State: AOJu0YxtMCn8+plJh6xrhJFw39Es7m9UqpJUXBtpYn812IJzTlQWJBhO GB1uC9frYnC+UJ7xBGctV5mkOSdCs3BxhJs5 X-Google-Smtp-Source: AGHT+IFKdeSiv6gV1IPEr0A1QYWZHU8I+1Kda5cvazHkRrgW5MCu5gP/Hc13TADoBD0pMrq9a6X08Q== X-Received: by 2002:a17:902:d28b:b0:1c7:5581:f9c with SMTP id t11-20020a170902d28b00b001c755810f9cmr12378077plc.0.1698759670766; Tue, 31 Oct 2023 06:41:10 -0700 (PDT) Received: from localhost (zz20184013906F627101.userreverse.dion.ne.jp. [111.98.113.1]) by smtp.gmail.com with ESMTPSA id k17-20020a170902c41100b001c6187f2875sm1330890plk.225.2023.10.31.06.41.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 Oct 2023 06:41:10 -0700 (PDT) From: Tatsuyuki Ishi To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 2/6] drm/amdgpu: Separate eviction from VM status. Date: Tue, 31 Oct 2023 22:40:55 +0900 Message-ID: <20231031134059.171277-3-ishitatsuyuki@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231031134059.171277-1-ishitatsuyuki@gmail.com> References: <20231031134059.171277-1-ishitatsuyuki@gmail.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tatsuyuki Ishi , christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" In short, eviction never really belonged to the vm_status state machine. Even when evicted, the BO could belong to either the moved or done state. The "evicted" state needed to handle both cases, causing greater confusion. Additionally, there were inconsistencies in the definition of an evicted BO. Some places are based on the `evict` parameter passed from the TTM move callback, while the others were updated based on whether the BO got its optimal placement. The second is more accurate for our use case. With this refactor, the evicted state is solely determined by the second rule. Signed-off-by: Tatsuyuki Ishi --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 67 +++++++++-------------- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 1 + 3 files changed, 29 insertions(+), 40 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 7b9762f1cddd..dd6f72e2a1d6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -174,19 +174,23 @@ int amdgpu_vm_set_pasid(struct amdgpu_device *adev, struct amdgpu_vm *vm, * State for PDs/PTs and per VM BOs which are not at the location they should * be. */ -static void amdgpu_vm_bo_evicted(struct amdgpu_vm_bo_base *vm_bo) +static void amdgpu_vm_bo_set_evicted(struct amdgpu_vm_bo_base *vm_bo, bool evicted) { struct amdgpu_vm *vm = vm_bo->vm; struct amdgpu_bo *bo = vm_bo->bo; - vm_bo->moved = true; spin_lock(&vm_bo->vm->status_lock); - if (bo->tbo.type == ttm_bo_type_kernel) - list_move(&vm_bo->vm_status, &vm->evicted); - else - list_move_tail(&vm_bo->vm_status, &vm->evicted); + if (evicted && bo->tbo.base.resv == vm->root.bo->tbo.base.resv) { + if (bo->tbo.type == ttm_bo_type_kernel) + list_move(&vm_bo->eviction_status, &vm->evicted); + else + list_move_tail(&vm_bo->eviction_status, &vm->evicted); + } else { + list_del_init(&vm_bo->eviction_status); + } spin_unlock(&vm_bo->vm->status_lock); } + /** * amdgpu_vm_bo_moved - vm_bo is moved * @@ -310,6 +314,7 @@ void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, base->bo = bo; base->next = NULL; INIT_LIST_HEAD(&base->vm_status); + INIT_LIST_HEAD(&base->eviction_status); if (!bo) return; @@ -336,7 +341,7 @@ void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, * is currently evicted. add the bo to the evicted list to make sure it * is validated on next vm use to avoid fault. * */ - amdgpu_vm_bo_evicted(base); + amdgpu_vm_bo_set_evicted(base, true); } /** @@ -460,7 +465,7 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm, while (!list_empty(&vm->evicted)) { bo_base = list_first_entry(&vm->evicted, struct amdgpu_vm_bo_base, - vm_status); + eviction_status); spin_unlock(&vm->status_lock); bo = bo_base->bo; @@ -1034,7 +1039,7 @@ void amdgpu_vm_get_memory(struct amdgpu_vm *vm, list_for_each_entry_safe(bo_va, tmp, &vm->idle, base.vm_status) amdgpu_vm_bo_get_memory(bo_va, stats); - list_for_each_entry_safe(bo_va, tmp, &vm->evicted, base.vm_status) + list_for_each_entry_safe(bo_va, tmp, &vm->evicted, base.eviction_status) amdgpu_vm_bo_get_memory(bo_va, stats); list_for_each_entry_safe(bo_va, tmp, &vm->relocated, base.vm_status) @@ -1153,21 +1158,10 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va, return r; } - /* If the BO is not in its preferred location add it back to - * the evicted list so that it gets validated again on the - * next command submission. - */ - if (bo && bo->tbo.base.resv == vm->root.bo->tbo.base.resv) { - uint32_t mem_type = bo->tbo.resource->mem_type; - - if (!(bo->preferred_domains & - amdgpu_mem_type_to_domain(mem_type))) - amdgpu_vm_bo_evicted(&bo_va->base); - else - amdgpu_vm_bo_idle(&bo_va->base); - } else { + if (bo && bo->tbo.base.resv == vm->root.bo->tbo.base.resv) + amdgpu_vm_bo_idle(&bo_va->base); + else amdgpu_vm_bo_done(&bo_va->base); - } list_splice_init(&bo_va->invalids, &bo_va->valids); bo_va->cleared = clear; @@ -1883,6 +1877,7 @@ void amdgpu_vm_bo_del(struct amdgpu_device *adev, spin_lock(&vm->status_lock); list_del(&bo_va->base.vm_status); + list_del(&bo_va->base.eviction_status); spin_unlock(&vm->status_lock); list_for_each_entry_safe(mapping, next, &bo_va->valids, list) { @@ -1959,13 +1954,18 @@ void amdgpu_vm_bo_invalidate(struct amdgpu_device *adev, if (bo->parent && (amdgpu_bo_shadowed(bo->parent) == bo)) bo = bo->parent; + /* If the BO is not in its preferred location add it back to + * the evicted list so that it gets validated again on the + * next command submission. + */ + uint32_t mem_type = bo->tbo.resource->mem_type; + bool suboptimal = !(bo->preferred_domains & + amdgpu_mem_type_to_domain(mem_type)); + for (bo_base = bo->vm_bo; bo_base; bo_base = bo_base->next) { struct amdgpu_vm *vm = bo_base->vm; - if (evicted && bo->tbo.base.resv == vm->root.bo->tbo.base.resv) { - amdgpu_vm_bo_evicted(bo_base); - continue; - } + amdgpu_vm_bo_set_evicted(bo_base, suboptimal); if (bo_base->moved) continue; @@ -2648,13 +2648,11 @@ void amdgpu_debugfs_vm_bo_info(struct amdgpu_vm *vm, struct seq_file *m) { struct amdgpu_bo_va *bo_va, *tmp; u64 total_idle = 0; - u64 total_evicted = 0; u64 total_relocated = 0; u64 total_moved = 0; u64 total_invalidated = 0; u64 total_done = 0; unsigned int total_idle_objs = 0; - unsigned int total_evicted_objs = 0; unsigned int total_relocated_objs = 0; unsigned int total_moved_objs = 0; unsigned int total_invalidated_objs = 0; @@ -2671,15 +2669,6 @@ void amdgpu_debugfs_vm_bo_info(struct amdgpu_vm *vm, struct seq_file *m) total_idle_objs = id; id = 0; - seq_puts(m, "\tEvicted BOs:\n"); - list_for_each_entry_safe(bo_va, tmp, &vm->evicted, base.vm_status) { - if (!bo_va->base.bo) - continue; - total_evicted += amdgpu_bo_print_info(id++, bo_va->base.bo, m); - } - total_evicted_objs = id; - id = 0; - seq_puts(m, "\tRelocated BOs:\n"); list_for_each_entry_safe(bo_va, tmp, &vm->relocated, base.vm_status) { if (!bo_va->base.bo) @@ -2718,8 +2707,6 @@ void amdgpu_debugfs_vm_bo_info(struct amdgpu_vm *vm, struct seq_file *m) seq_printf(m, "\tTotal idle size: %12lld\tobjs:\t%d\n", total_idle, total_idle_objs); - seq_printf(m, "\tTotal evicted size: %12lld\tobjs:\t%d\n", total_evicted, - total_evicted_objs); seq_printf(m, "\tTotal relocated size: %12lld\tobjs:\t%d\n", total_relocated, total_relocated_objs); seq_printf(m, "\tTotal moved size: %12lld\tobjs:\t%d\n", total_moved, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index 204ab13184ed..d9ab97eabda9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -156,6 +156,7 @@ struct amdgpu_vm_bo_base { /* protected by spinlock */ struct list_head vm_status; + struct list_head eviction_status; /* protected by the BO being reserved */ bool moved; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c index 96d601e209b8..f78f4040f466 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c @@ -652,6 +652,7 @@ static void amdgpu_vm_pt_free(struct amdgpu_vm_bo_base *entry) spin_lock(&entry->vm->status_lock); list_del(&entry->vm_status); + list_del(&entry->eviction_status); spin_unlock(&entry->vm->status_lock); amdgpu_bo_unref(&entry->bo); } From patchwork Tue Oct 31 13:40:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tatsuyuki Ishi X-Patchwork-Id: 13441543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8865CC4332F for ; Tue, 31 Oct 2023 13:41:33 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BE69110E4D7; Tue, 31 Oct 2023 13:41:26 +0000 (UTC) Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7AE2710E4CE; Tue, 31 Oct 2023 13:41:14 +0000 (UTC) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-6bf20d466cdso1244405b3a.1; Tue, 31 Oct 2023 06:41:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698759673; x=1699364473; darn=lists.freedesktop.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PCeDzcfZzIt44IqUc4Bdw7umWr+sqAg8YCgkraBLUT4=; b=Bts0/Tsgnlmu9U4Zi7YTPxp+lPhvA2CKXDgW72guNGGN0PQF3dGLrM3wKKkxFEKJE0 5pRXswjTx4R9UO6C0qDXmuEJmjEgOkfXQpRhitdgg4O+IwFP3w2WNntekETVeaEawSp0 N6Q277pa3wzIBSdUd+QPkE7219jLbcCIMDY23NRdLojB/j9cAV2cah7sHq2ukkAclhld nQrdF66rex5jKYLeG7gEjsJYVfQWlh/dwALqCvsBYmJyi4C46QKNHjd9/XyaJEDw7iHa JSYKo6H81U66q7Nne0uwb7Sgkj3xNRlA1PDx0ChozvGseaNI6+jU+Wyj2cbA2ptoQyre UN5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698759673; x=1699364473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PCeDzcfZzIt44IqUc4Bdw7umWr+sqAg8YCgkraBLUT4=; b=hTqpHWXTLO3B0rQW9zA1s9XRmsv8rjrwvC+v8x1CzpFoW1q3gnFXb4ILxIwVfv8N0/ MwJ04qfqI5Jj2/Jl4leNSPArwx6vGrVvXD73fzV0wJcR9yQjP4St6t4/fFZyWbCmcc3F KheSIBABuGYfJLBmQ+gtSkHUQmsPAUBy3eDpeGndZNxsPn8D7hmVy9qO0AyG+aKpvoSU 6FGdss0EOLgVwN1cMnhTYGs78h/QpUnkvW0fmaCyXvPU50a1u7dFqtdPeNiLK9UxGM+h P5+mlmZTeSR8M+yMXzOzdO9/WmFWdERlh30BzotCbRiNcfF/mYVjMhUQRYXHsUwrX91O bNIw== X-Gm-Message-State: AOJu0YyMiFJQvgM73vNq5CBTA+UqIbqikuqM0dgsLTaaNgppgRpRh2iA PpedPmMIVL2DOzrM18jURa3ZLBMCdmTSa6mk X-Google-Smtp-Source: AGHT+IHMejDkVMrbHXLssnODixamj6R+hG8U8/0Y0rAeIZS6LJ8QxUn/J+TIm0Xm4qH+HaUXmqf0ng== X-Received: by 2002:a05:6a21:6d85:b0:163:57ba:2ad4 with SMTP id wl5-20020a056a216d8500b0016357ba2ad4mr16253721pzb.2.1698759673411; Tue, 31 Oct 2023 06:41:13 -0700 (PDT) Received: from localhost (zz20184013906F627101.userreverse.dion.ne.jp. [111.98.113.1]) by smtp.gmail.com with ESMTPSA id g18-20020aa78752000000b0068fe7e07190sm1267938pfo.3.2023.10.31.06.41.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 Oct 2023 06:41:12 -0700 (PDT) From: Tatsuyuki Ishi To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 3/6] drm/amdgpu: Flush VM updates for split bindings eagerly. Date: Tue, 31 Oct 2023 22:40:56 +0900 Message-ID: <20231031134059.171277-4-ishitatsuyuki@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231031134059.171277-1-ishitatsuyuki@gmail.com> References: <20231031134059.171277-1-ishitatsuyuki@gmail.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tatsuyuki Ishi , christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The current amdgpu_gem_va_update_vm only tries to perform updates for the BO specified in the GEM ioctl; however, when a binding is split, the adjacent bindings also need to be updated. Such updates currently ends up getting deferred until next submission which causes stalls. Introduce a new state "dirty", shared between per-VM BOs and traditional BOs, containing all BOs that have pending updates in `invalids`. amdgpu_gem_va_update_vm will now simply flush any pending updates for BOs in the dirty state. Signed-off-by: Tatsuyuki Ishi --- drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 18 ++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 66 ++++++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 3 ++ 3 files changed, 63 insertions(+), 24 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index a1b15d0d6c48..01d3a97248b0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -604,10 +604,9 @@ int amdgpu_gem_metadata_ioctl(struct drm_device *dev, void *data, * vital here, so they are not reported back to userspace. */ static void amdgpu_gem_va_update_vm(struct amdgpu_device *adev, - struct amdgpu_vm *vm, - struct amdgpu_bo_va *bo_va, - uint32_t operation) + struct amdgpu_vm *vm) { + struct amdgpu_bo_va *bo_va; int r; if (!amdgpu_vm_ready(vm)) @@ -617,12 +616,18 @@ static void amdgpu_gem_va_update_vm(struct amdgpu_device *adev, if (r) goto error; - if (operation == AMDGPU_VA_OP_MAP || - operation == AMDGPU_VA_OP_REPLACE) { + spin_lock(&vm->status_lock); + while (!list_empty(&vm->dirty)) { + bo_va = list_first_entry(&vm->dirty, struct amdgpu_bo_va, + base.vm_status); + spin_unlock(&vm->status_lock); + r = amdgpu_vm_bo_update(adev, bo_va, false); if (r) goto error; + spin_lock(&vm->status_lock); } + spin_unlock(&vm->status_lock); r = amdgpu_vm_update_pdes(adev, vm, false); @@ -792,8 +797,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, break; } if (!r && !(args->flags & AMDGPU_VM_DELAY_UPDATE) && !amdgpu_vm_debug) - amdgpu_gem_va_update_vm(adev, &fpriv->vm, bo_va, - args->operation); + amdgpu_gem_va_update_vm(adev, &fpriv->vm); error: drm_exec_fini(&exec); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index dd6f72e2a1d6..01d31891cd05 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -191,6 +191,21 @@ static void amdgpu_vm_bo_set_evicted(struct amdgpu_vm_bo_base *vm_bo, bool evict spin_unlock(&vm_bo->vm->status_lock); } +/** + * amdgpu_vm_bo_dirty - vm_bo is dirty + * + * @vm_bo: vm_bo which is dirty + * + * State for normal and per VM BOs that are not moved, but have new entries in + * bo_va->invalids. + */ +static void amdgpu_vm_bo_dirty(struct amdgpu_vm_bo_base *vm_bo) +{ + spin_lock(&vm_bo->vm->status_lock); + list_move(&vm_bo->vm_status, &vm_bo->vm->dirty); + spin_unlock(&vm_bo->vm->status_lock); +} + /** * amdgpu_vm_bo_moved - vm_bo is moved * @@ -1042,6 +1057,9 @@ void amdgpu_vm_get_memory(struct amdgpu_vm *vm, list_for_each_entry_safe(bo_va, tmp, &vm->evicted, base.eviction_status) amdgpu_vm_bo_get_memory(bo_va, stats); + list_for_each_entry_safe(bo_va, tmp, &vm->dirty, base.vm_status) + amdgpu_vm_bo_get_memory(bo_va, stats); + list_for_each_entry_safe(bo_va, tmp, &vm->relocated, base.vm_status) amdgpu_vm_bo_get_memory(bo_va, stats); @@ -1411,6 +1429,17 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, dma_resv_unlock(resv); spin_lock(&vm->status_lock); } + + while (!list_empty(&vm->dirty)) { + bo_va = list_first_entry(&vm->dirty, struct amdgpu_bo_va, + base.vm_status); + spin_unlock(&vm->status_lock); + + r = amdgpu_vm_bo_update(adev, bo_va, false); + if (r) + return r; + spin_lock(&vm->status_lock); + } spin_unlock(&vm->status_lock); return 0; @@ -1476,19 +1505,16 @@ static void amdgpu_vm_bo_insert_map(struct amdgpu_device *adev, struct amdgpu_bo_va_mapping *mapping) { struct amdgpu_vm *vm = bo_va->base.vm; - struct amdgpu_bo *bo = bo_va->base.bo; mapping->bo_va = bo_va; list_add(&mapping->list, &bo_va->invalids); amdgpu_vm_it_insert(mapping, &vm->va); + if (!bo_va->base.moved) + amdgpu_vm_bo_dirty(&bo_va->base); if (mapping->flags & AMDGPU_PTE_PRT) amdgpu_vm_prt_get(adev); - if (bo && bo->tbo.base.resv == vm->root.bo->tbo.base.resv && - !bo_va->base.moved) { - amdgpu_vm_bo_moved(&bo_va->base); - } trace_amdgpu_vm_bo_map(bo_va, mapping); } @@ -1725,6 +1751,8 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev, before->flags = tmp->flags; before->bo_va = tmp->bo_va; list_add(&before->list, &tmp->bo_va->invalids); + if (!tmp->bo_va->base.moved) + amdgpu_vm_bo_dirty(&tmp->bo_va->base); } /* Remember mapping split at the end */ @@ -1736,6 +1764,8 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev, after->flags = tmp->flags; after->bo_va = tmp->bo_va; list_add(&after->list, &tmp->bo_va->invalids); + if (!tmp->bo_va->base.moved) + amdgpu_vm_bo_dirty(&tmp->bo_va->base); } list_del(&tmp->list); @@ -1761,30 +1791,18 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev, /* Insert partial mapping before the range */ if (!list_empty(&before->list)) { - struct amdgpu_bo *bo = before->bo_va->base.bo; - amdgpu_vm_it_insert(before, &vm->va); if (before->flags & AMDGPU_PTE_PRT) amdgpu_vm_prt_get(adev); - - if (bo && bo->tbo.base.resv == vm->root.bo->tbo.base.resv && - !before->bo_va->base.moved) - amdgpu_vm_bo_moved(&before->bo_va->base); } else { kfree(before); } /* Insert partial mapping after the range */ if (!list_empty(&after->list)) { - struct amdgpu_bo *bo = after->bo_va->base.bo; - amdgpu_vm_it_insert(after, &vm->va); if (after->flags & AMDGPU_PTE_PRT) amdgpu_vm_prt_get(adev); - - if (bo && bo->tbo.base.resv == vm->root.bo->tbo.base.resv && - !after->bo_va->base.moved) - amdgpu_vm_bo_moved(&after->bo_va->base); } else { kfree(after); } @@ -2136,6 +2154,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm, int32_t xcp INIT_LIST_HEAD(&vm->evicted); INIT_LIST_HEAD(&vm->relocated); INIT_LIST_HEAD(&vm->moved); + INIT_LIST_HEAD(&vm->dirty); INIT_LIST_HEAD(&vm->idle); INIT_LIST_HEAD(&vm->invalidated); spin_lock_init(&vm->status_lock); @@ -2648,11 +2667,13 @@ void amdgpu_debugfs_vm_bo_info(struct amdgpu_vm *vm, struct seq_file *m) { struct amdgpu_bo_va *bo_va, *tmp; u64 total_idle = 0; + u64 total_dirty = 0; u64 total_relocated = 0; u64 total_moved = 0; u64 total_invalidated = 0; u64 total_done = 0; unsigned int total_idle_objs = 0; + unsigned int total_dirty_objs = 0; unsigned int total_relocated_objs = 0; unsigned int total_moved_objs = 0; unsigned int total_invalidated_objs = 0; @@ -2669,6 +2690,15 @@ void amdgpu_debugfs_vm_bo_info(struct amdgpu_vm *vm, struct seq_file *m) total_idle_objs = id; id = 0; + seq_puts(m, "\tDirty BOs:\n"); + list_for_each_entry_safe(bo_va, tmp, &vm->dirty, base.vm_status) { + if (!bo_va->base.bo) + continue; + total_dirty += amdgpu_bo_print_info(id++, bo_va->base.bo, m); + } + total_dirty_objs = id; + id = 0; + seq_puts(m, "\tRelocated BOs:\n"); list_for_each_entry_safe(bo_va, tmp, &vm->relocated, base.vm_status) { if (!bo_va->base.bo) @@ -2707,6 +2737,8 @@ void amdgpu_debugfs_vm_bo_info(struct amdgpu_vm *vm, struct seq_file *m) seq_printf(m, "\tTotal idle size: %12lld\tobjs:\t%d\n", total_idle, total_idle_objs); + seq_printf(m, "\tTotal dirty size: %12lld\tobjs:\t%d\n", total_dirty, + total_dirty_objs); seq_printf(m, "\tTotal relocated size: %12lld\tobjs:\t%d\n", total_relocated, total_relocated_objs); seq_printf(m, "\tTotal moved size: %12lld\tobjs:\t%d\n", total_moved, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index d9ab97eabda9..f91d4fcf80b8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -276,6 +276,9 @@ struct amdgpu_vm { /* per VM BOs moved, but not yet updated in the PT */ struct list_head moved; + /* normal and per VM BOs that are not moved, but have new PT entries */ + struct list_head dirty; + /* All BOs of this VM not currently in the state machine */ struct list_head idle; From patchwork Tue Oct 31 13:40:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tatsuyuki Ishi X-Patchwork-Id: 13441542 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DE512C4167B for ; Tue, 31 Oct 2023 13:41:31 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 252FE10E4D4; Tue, 31 Oct 2023 13:41:26 +0000 (UTC) Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6828B10E4CE; Tue, 31 Oct 2023 13:41:17 +0000 (UTC) Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1cc13149621so10577255ad.1; Tue, 31 Oct 2023 06:41:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698759676; x=1699364476; darn=lists.freedesktop.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WY87l0JQJecUS5eNh5/9T1RB/XaRsBA9P7FgGHx3yR8=; b=CZ0jd4T2ekGtb0cD0jCfL48CfrfHtw+4/WHtJArqERqoBp1j2fKcau50bTUbGSsHCA 8wDZVpgcNRDluJO3lwUDNjJI9krihZ1DGEXEvJ+mvbHN4KKzhKRANlsmAqgvFpQMKo6+ MTDf9Q8oi+SjFBB73xr6NaDh/UoEfpzsXeNRVtQWzkwEuQWvMCJOwuwOL2bAAYX+bi9k rBYI+WBXrodHxAqY+KSzEOz5BMdTV9+EuR06e/oy+HMXNaPaokS6LWzP+sUXHSGWcQsv mgVmeuQ/RCp9bkGtiA7K9t6G8Fmg6pRHKCw6FcQ07eaekeqoLHvEfCNRxZ6UzrSW8ODE //2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698759676; x=1699364476; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WY87l0JQJecUS5eNh5/9T1RB/XaRsBA9P7FgGHx3yR8=; b=ifwYqVwKTCh+upQ3QQVga+csR491BNOkGW04vVK8IqjoxMa4P+54ZIWWPwtaOUPB/S 8Gl0c2agRzG07ytLKRtzM1+Dp+dX5T1MylTGssOP3cPvpEYl40RIilYV5kzkem3NSAoc rDtMxXhk7rfnls/1MDwPm/Emp+N6HflukVaTIpx90Mh0ZYND6rHLwaaCFfCNqQFAkMez 7BL1i1vApPhPutuHi0Fd47SXQHLz8n2H26YD0Do4OkYDmZCWt33RBvpB7pcfkarApiyz tFRKGe+dKsRaBhDOOWmhCgP1uTQDQPAeq2XFFTR/Av69OkHzMQtwT6lPXC19lCWdSbmc aA1A== X-Gm-Message-State: AOJu0YzUz2AvRnjx2ZWcbBHEGnPhe54wq2esqPbO/F1zyn/1hl5ZIeVa PuXHUsmLzhJmfptb2CYUVX84e5DttL+ZBEut X-Google-Smtp-Source: AGHT+IFwtVzRkNcTy9skkE5aci9x4sNK6+drL8OUwSHr70IlPrdtWDnGxwVf6cUO7/3m9DEeG1d4xA== X-Received: by 2002:a17:902:d1c2:b0:1cc:277f:b4f6 with SMTP id g2-20020a170902d1c200b001cc277fb4f6mr10650649plb.6.1698759675898; Tue, 31 Oct 2023 06:41:15 -0700 (PDT) Received: from localhost (zz20184013906F627101.userreverse.dion.ne.jp. [111.98.113.1]) by smtp.gmail.com with ESMTPSA id j18-20020a170902c3d200b001cc54202429sm1328207plj.288.2023.10.31.06.41.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 Oct 2023 06:41:15 -0700 (PDT) From: Tatsuyuki Ishi To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 4/6] drm/amdgpu: Remove redundant state change after validation. Date: Tue, 31 Oct 2023 22:40:57 +0900 Message-ID: <20231031134059.171277-5-ishitatsuyuki@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231031134059.171277-1-ishitatsuyuki@gmail.com> References: <20231031134059.171277-1-ishitatsuyuki@gmail.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tatsuyuki Ishi , christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" All the state changes are handled in the TTM move callback; doing it again here just leads to more confusion. The table update remains here because it needs to be done exactly once, while doing it in the move callback will result it getting triggered twice, once by the actual BO and once by the shadow BO. Signed-off-by: Tatsuyuki Ishi --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 01d31891cd05..50f7cee639ac 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -495,12 +495,9 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm, return r; } - if (bo->tbo.type != ttm_bo_type_kernel) { - amdgpu_vm_bo_moved(bo_base); - } else { + if (bo->tbo.type == ttm_bo_type_kernel) vm->update_funcs->map_table(to_amdgpu_bo_vm(bo)); - amdgpu_vm_bo_relocated(bo_base); - } + spin_lock(&vm->status_lock); } spin_unlock(&vm->status_lock); From patchwork Tue Oct 31 13:40:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tatsuyuki Ishi X-Patchwork-Id: 13441541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 135DFC4332F for ; Tue, 31 Oct 2023 13:41:30 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6206710E4D5; Tue, 31 Oct 2023 13:41:26 +0000 (UTC) Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by gabe.freedesktop.org (Postfix) with ESMTPS id 52F4210E4D1; Tue, 31 Oct 2023 13:41:20 +0000 (UTC) Received: by mail-pj1-x102e.google.com with SMTP id 98e67ed59e1d1-28099d11c49so172298a91.1; Tue, 31 Oct 2023 06:41:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698759679; x=1699364479; darn=lists.freedesktop.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pfiD+aI7IJ8Ktty4Ji4GH3tqSYafiQEsDwIOPFCtAnA=; b=gmvgJtMgiPvZw4ekp1UG8s1zbdMWxUWO02HbJ13hXqMrAJvyeW15O99wbRCk41ycgd d2D79RwQnb5caXk8gaTRTtLyN2KEP5APwKBBD86DpoTeyk53kqBJbweMM4U+ys5dzFn4 kLAUZ6UQs5iPknlXrMpW0pCcgUoFl5W/d2/UctzP/fgtHfhEy8gOtjg6H3qeSePr9qsl zgGAw3uFjRbMzJe1Qo9mwC8QI+9u6XTrFRu9MDSz1mMQB1H5PtBzo9EcSjV0ieg8vbiY DYnTVDRmCvSziseD9nNiFps90l4RmlDcBb9LV1aNVmR6zOppsmcLyZoM11KGG01rMA0K hVdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698759679; x=1699364479; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pfiD+aI7IJ8Ktty4Ji4GH3tqSYafiQEsDwIOPFCtAnA=; b=AnGEMSgU0reRdSMXm0ZlFiR091FhdRNsoKJ512taXwu+uhl4c/npBGb2RIqJghDwfS xiYg9X9M6GRHbE7VCAM8eh7kqKwh7Y6Oz4I1GW6RIwz4zritg2Z0pVVLrQFrdKig/t2J xjTr4peQ0P2k0U6CAd3iRb4O9ESRTHiQhOHUZJewp+Re42ijmDvC228+Nal9+ymbT3PK h74Tgd41w/S+PVwAvwtMJAWwjT2+SHWOWsFzS/UKxJc/KumgYFarudDNaCpD9TylDlJ4 93tUnCHBugq3N3p1yYM66Y0BG8AIRvqjKHA/VkX8VfUxLb3Z0LzP1uwq/d0KTHKvZr9O oPzA== X-Gm-Message-State: AOJu0YwPsNEFUmT9ufmlJhcU91m0EcuVDTxvMitI5XQ9HVPfU5Iu3380 JOXcV1zSv+rQ0mTIQ5I7/JGi2J4C3uvu2GrK X-Google-Smtp-Source: AGHT+IGknz29UvDWeBkNo1FEVz+fQdn/jWyiKKyVi6/GbTvX9B01JkUBkJHMHw/d/2zetukxZgGgwA== X-Received: by 2002:a17:90b:3781:b0:27d:15e3:3aa9 with SMTP id mz1-20020a17090b378100b0027d15e33aa9mr12103342pjb.3.1698759679055; Tue, 31 Oct 2023 06:41:19 -0700 (PDT) Received: from localhost (zz20184013906F627101.userreverse.dion.ne.jp. [111.98.113.1]) by smtp.gmail.com with ESMTPSA id s8-20020a17090aad8800b002807c61ca2bsm1138306pjq.26.2023.10.31.06.41.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 Oct 2023 06:41:18 -0700 (PDT) From: Tatsuyuki Ishi To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 5/6] drm/amdgpu: Add flag to disable implicit sync for GEM operations. Date: Tue, 31 Oct 2023 22:40:58 +0900 Message-ID: <20231031134059.171277-6-ishitatsuyuki@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231031134059.171277-1-ishitatsuyuki@gmail.com> References: <20231031134059.171277-1-ishitatsuyuki@gmail.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tatsuyuki Ishi , christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" In Vulkan, it is the application's responsibility to perform adequate synchronization before a sparse unmap, replace or BO destroy operation. Until now, the kernel applied the same rule as implicitly-synchronized APIs like OpenGL, which with per-VM BOs made page table updates stall the queue completely. The newly added AMDGPU_VM_EXPLICIT_SYNC flag allows drivers to opt-out of this behavior, while still ensuring adequate implicit sync happens for kernel-initiated updates (e.g. BO moves). We record whether to use implicit sync or not for each freed mapping. To avoid increasing the mapping struct's size, this is union-ized with the interval tree field which is unused after the unmap. The reason this is done with a GEM ioctl flag, instead of being a VM / context global setting, is that the current libdrm implementation shares the DRM handle even between different kind of drivers (radeonsi vs radv). Signed-off-by: Tatsuyuki Ishi --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 14 +++-- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 7 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h | 6 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 55 +++++++++++-------- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 23 ++++---- drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 18 +++--- include/uapi/drm/amdgpu_drm.h | 2 + 9 files changed, 74 insertions(+), 55 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 7d6daf8d2bfa..10e129bff977 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -1196,7 +1196,7 @@ static void unmap_bo_from_gpuvm(struct kgd_mem *mem, struct amdgpu_device *adev = entry->adev; struct amdgpu_vm *vm = bo_va->base.vm; - amdgpu_vm_bo_unmap(adev, bo_va, entry->va); + amdgpu_vm_bo_unmap(adev, bo_va, entry->va, true); amdgpu_vm_clear_freed(adev, vm, &bo_va->last_pt_update); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c index 720011019741..612279e65bff 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c @@ -122,7 +122,7 @@ int amdgpu_unmap_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm, } } - r = amdgpu_vm_bo_unmap(adev, bo_va, csa_addr); + r = amdgpu_vm_bo_unmap(adev, bo_va, csa_addr, true); if (r) { DRM_ERROR("failed to do bo_unmap on static CSA, err=%d\n", r); goto error; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index 01d3a97248b0..0d9496a06947 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -672,9 +672,9 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, const uint32_t valid_flags = AMDGPU_VM_DELAY_UPDATE | AMDGPU_VM_PAGE_READABLE | AMDGPU_VM_PAGE_WRITEABLE | AMDGPU_VM_PAGE_EXECUTABLE | AMDGPU_VM_MTYPE_MASK | - AMDGPU_VM_PAGE_NOALLOC; + AMDGPU_VM_PAGE_NOALLOC | AMDGPU_VM_EXPLICIT_SYNC; const uint32_t prt_flags = AMDGPU_VM_DELAY_UPDATE | - AMDGPU_VM_PAGE_PRT; + AMDGPU_VM_PAGE_PRT | AMDGPU_VM_EXPLICIT_SYNC; struct drm_amdgpu_gem_va *args = data; struct drm_gem_object *gobj; @@ -685,6 +685,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, struct drm_exec exec; uint64_t va_flags; uint64_t vm_size; + bool sync_unmap; int r = 0; if (args->va_address < AMDGPU_VA_RESERVED_SIZE) { @@ -720,6 +721,8 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, return -EINVAL; } + sync_unmap = !(args->flags & AMDGPU_VM_EXPLICIT_SYNC); + switch (args->operation) { case AMDGPU_VA_OP_MAP: case AMDGPU_VA_OP_UNMAP: @@ -779,19 +782,20 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, va_flags); break; case AMDGPU_VA_OP_UNMAP: - r = amdgpu_vm_bo_unmap(adev, bo_va, args->va_address); + r = amdgpu_vm_bo_unmap(adev, bo_va, args->va_address, + sync_unmap); break; case AMDGPU_VA_OP_CLEAR: r = amdgpu_vm_bo_clear_mappings(adev, &fpriv->vm, args->va_address, - args->map_size); + args->map_size, sync_unmap); break; case AMDGPU_VA_OP_REPLACE: va_flags = amdgpu_gem_va_map_flags(adev, args->flags); r = amdgpu_vm_bo_replace_map(adev, bo_va, args->va_address, args->offset_in_bo, args->map_size, - va_flags); + va_flags, sync_unmap); break; default: break; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index f3ee83cdf97e..28be03f1bbcf 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -67,7 +67,12 @@ struct amdgpu_bo_va_mapping { struct rb_node rb; uint64_t start; uint64_t last; - uint64_t __subtree_last; + union { + /* BOs in interval tree only */ + uint64_t __subtree_last; + /* Freed BOs only */ + bool sync_unmap; + }; uint64_t offset; uint64_t flags; }; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h index 2fd1bfb35916..e71443c8c59b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h @@ -276,6 +276,7 @@ TRACE_EVENT(amdgpu_vm_bo_unmap, __field(long, last) __field(u64, offset) __field(u64, flags) + __field(bool, sync_unmap) ), TP_fast_assign( @@ -284,10 +285,11 @@ TRACE_EVENT(amdgpu_vm_bo_unmap, __entry->last = mapping->last; __entry->offset = mapping->offset; __entry->flags = mapping->flags; + __entry->sync_unmap = mapping->sync_unmap; ), - TP_printk("bo=%p, start=%lx, last=%lx, offset=%010llx, flags=%llx", + TP_printk("bo=%p, start=%lx, last=%lx, offset=%010llx, flags=%llx, sync_unmap=%d", __entry->bo, __entry->start, __entry->last, - __entry->offset, __entry->flags) + __entry->offset, __entry->flags, __entry->sync_unmap) ); DECLARE_EVENT_CLASS(amdgpu_vm_mapping, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 50f7cee639ac..a15463e0bbc5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -861,6 +861,7 @@ static void amdgpu_vm_tlb_seq_cb(struct dma_fence *fence, * @immediate: immediate submission in a page fault * @unlocked: unlocked invalidation during MM callback * @flush_tlb: trigger tlb invalidation after update completed + * @sync_unmap: wait for BO users before unmapping * @resv: fences we need to sync to * @start: start of mapped range * @last: last mapped entry @@ -878,8 +879,9 @@ static void amdgpu_vm_tlb_seq_cb(struct dma_fence *fence, */ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm, bool immediate, bool unlocked, bool flush_tlb, - struct dma_resv *resv, uint64_t start, uint64_t last, - uint64_t flags, uint64_t offset, uint64_t vram_base, + bool sync_unmap, struct dma_resv *resv, + uint64_t start, uint64_t last, uint64_t flags, + uint64_t offset, uint64_t vram_base, struct ttm_resource *res, dma_addr_t *pages_addr, struct dma_fence **fence) { @@ -919,7 +921,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm, /* Implicitly sync to command submissions in the same VM before * unmapping. Sync to moving fences before mapping. */ - if (!(flags & (AMDGPU_PTE_VALID | AMDGPU_PTE_PRT))) + if (!(flags & (AMDGPU_PTE_VALID | AMDGPU_PTE_PRT)) && sync_unmap) sync_mode = AMDGPU_SYNC_EQ_OWNER; else sync_mode = AMDGPU_SYNC_EXPLICIT; @@ -1165,10 +1167,10 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va, trace_amdgpu_vm_bo_update(mapping); r = amdgpu_vm_update_range(adev, vm, false, false, flush_tlb, - resv, mapping->start, mapping->last, - update_flags, mapping->offset, - vram_base, mem, pages_addr, - last_update); + true, resv, mapping->start, + mapping->last, update_flags, + mapping->offset, vram_base, mem, + pages_addr, last_update); if (r) return r; } @@ -1349,7 +1351,8 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, mapping->start < AMDGPU_GMC_HOLE_START) init_pte_value = AMDGPU_PTE_DEFAULT_ATC; - r = amdgpu_vm_update_range(adev, vm, false, false, true, resv, + r = amdgpu_vm_update_range(adev, vm, false, false, true, + mapping->sync_unmap, resv, mapping->start, mapping->last, init_pte_value, 0, 0, NULL, NULL, &f); @@ -1589,6 +1592,7 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev, * @offset: requested offset in the BO * @size: BO size in bytes * @flags: attributes of pages (read/write/valid/etc.) + * @sync_unmap: wait for BO users before replacing existing mapping * * Add a mapping of the BO at the specefied addr into the VM. Replace existing * mappings as we do so. @@ -1599,9 +1603,9 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev, * Object has to be reserved and unreserved outside! */ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev, - struct amdgpu_bo_va *bo_va, - uint64_t saddr, uint64_t offset, - uint64_t size, uint64_t flags) + struct amdgpu_bo_va *bo_va, uint64_t saddr, + uint64_t offset, uint64_t size, uint64_t flags, + bool sync_unmap) { struct amdgpu_bo_va_mapping *mapping; struct amdgpu_bo *bo = bo_va->base.bo; @@ -1625,7 +1629,7 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev, if (!mapping) return -ENOMEM; - r = amdgpu_vm_bo_clear_mappings(adev, bo_va->base.vm, saddr, size); + r = amdgpu_vm_bo_clear_mappings(adev, bo_va->base.vm, saddr, size, sync_unmap); if (r) { kfree(mapping); return r; @@ -1658,9 +1662,8 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev, * * Object has to be reserved and unreserved outside! */ -int amdgpu_vm_bo_unmap(struct amdgpu_device *adev, - struct amdgpu_bo_va *bo_va, - uint64_t saddr) +int amdgpu_vm_bo_unmap(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va, + uint64_t saddr, bool sync_unmap) { struct amdgpu_bo_va_mapping *mapping; struct amdgpu_vm *vm = bo_va->base.vm; @@ -1688,6 +1691,7 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev, list_del(&mapping->list); amdgpu_vm_it_remove(mapping, &vm->va); mapping->bo_va = NULL; + mapping->sync_unmap = sync_unmap; trace_amdgpu_vm_bo_unmap(bo_va, mapping); if (valid) @@ -1706,6 +1710,7 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev, * @vm: VM structure to use * @saddr: start of the range * @size: size of the range + * @sync_unmap: wait for BO users before unmapping * * Remove all mappings in a range, split them as appropriate. * @@ -1713,8 +1718,8 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev, * 0 for success, error for failure. */ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev, - struct amdgpu_vm *vm, - uint64_t saddr, uint64_t size) + struct amdgpu_vm *vm, uint64_t saddr, + uint64_t size, bool sync_unmap) { struct amdgpu_bo_va_mapping *before, *after, *tmp, *next; LIST_HEAD(removed); @@ -1782,6 +1787,7 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev, tmp->last = eaddr; tmp->bo_va = NULL; + tmp->sync_unmap = sync_unmap; list_add(&tmp->list, &vm->freed); trace_amdgpu_vm_bo_unmap(NULL, tmp); } @@ -1899,6 +1905,7 @@ void amdgpu_vm_bo_del(struct amdgpu_device *adev, list_del(&mapping->list); amdgpu_vm_it_remove(mapping, &vm->va); mapping->bo_va = NULL; + mapping->sync_unmap = true; trace_amdgpu_vm_bo_unmap(bo_va, mapping); list_add(&mapping->list, &vm->freed); } @@ -2481,20 +2488,19 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) struct amdgpu_device *adev = drm_to_adev(dev); struct amdgpu_fpriv *fpriv = filp->driver_priv; - /* No valid flags defined yet */ - if (args->in.flags) - return -EINVAL; - switch (args->in.op) { case AMDGPU_VM_OP_RESERVE_VMID: + if (args->in.flags) + return -EINVAL; /* We only have requirement to reserve vmid from gfxhub */ if (!fpriv->vm.reserved_vmid[AMDGPU_GFXHUB(0)]) { amdgpu_vmid_alloc_reserved(adev, AMDGPU_GFXHUB(0)); fpriv->vm.reserved_vmid[AMDGPU_GFXHUB(0)] = true; } - break; case AMDGPU_VM_OP_UNRESERVE_VMID: + if (args->in.flags) + return -EINVAL; if (fpriv->vm.reserved_vmid[AMDGPU_GFXHUB(0)]) { amdgpu_vmid_free_reserved(adev, AMDGPU_GFXHUB(0)); fpriv->vm.reserved_vmid[AMDGPU_GFXHUB(0)] = false; @@ -2633,8 +2639,9 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, goto error_unlock; } - r = amdgpu_vm_update_range(adev, vm, true, false, false, NULL, addr, - addr, flags, value, 0, NULL, NULL, NULL); + r = amdgpu_vm_update_range(adev, vm, true, false, false, true, NULL, + addr, addr, flags, value, 0, NULL, NULL, + NULL); if (r) goto error_unlock; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index f91d4fcf80b8..3574987595d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -427,12 +427,12 @@ void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, struct amdgpu_vm *vm, struct amdgpu_bo *bo); int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm, bool immediate, bool unlocked, bool flush_tlb, - struct dma_resv *resv, uint64_t start, uint64_t last, - uint64_t flags, uint64_t offset, uint64_t vram_base, + bool sync_unmap, struct dma_resv *resv, + uint64_t start, uint64_t last, uint64_t flags, + uint64_t offset, uint64_t vram_base, struct ttm_resource *res, dma_addr_t *pages_addr, struct dma_fence **fence); -int amdgpu_vm_bo_update(struct amdgpu_device *adev, - struct amdgpu_bo_va *bo_va, +int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va, bool clear); bool amdgpu_vm_evictable(struct amdgpu_bo *bo); void amdgpu_vm_bo_invalidate(struct amdgpu_device *adev, @@ -448,15 +448,14 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev, uint64_t addr, uint64_t offset, uint64_t size, uint64_t flags); int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev, - struct amdgpu_bo_va *bo_va, - uint64_t addr, uint64_t offset, - uint64_t size, uint64_t flags); -int amdgpu_vm_bo_unmap(struct amdgpu_device *adev, - struct amdgpu_bo_va *bo_va, - uint64_t addr); + struct amdgpu_bo_va *bo_va, uint64_t addr, + uint64_t offset, uint64_t size, uint64_t flags, + bool sync_unmap); +int amdgpu_vm_bo_unmap(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va, + uint64_t addr, bool sync_unmap); int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev, - struct amdgpu_vm *vm, - uint64_t saddr, uint64_t size); + struct amdgpu_vm *vm, uint64_t saddr, + uint64_t size, bool sync_unmap); struct amdgpu_bo_va_mapping *amdgpu_vm_bo_lookup_mapping(struct amdgpu_vm *vm, uint64_t addr); void amdgpu_vm_bo_trace_cs(struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c index bb16b795d1bc..6eb4a0a4bc84 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c @@ -1291,9 +1291,9 @@ svm_range_unmap_from_gpu(struct amdgpu_device *adev, struct amdgpu_vm *vm, pr_debug("[0x%llx 0x%llx]\n", start, last); - return amdgpu_vm_update_range(adev, vm, false, true, true, NULL, start, - last, init_pte_value, 0, 0, NULL, NULL, - fence); + return amdgpu_vm_update_range(adev, vm, false, true, true, true, NULL, + start, last, init_pte_value, 0, 0, NULL, + NULL, fence); } static int @@ -1398,12 +1398,12 @@ svm_range_map_to_gpu(struct kfd_process_device *pdd, struct svm_range *prange, * different memory partition based on fpfn/lpfn, we should use * same vm_manager.vram_base_offset regardless memory partition. */ - r = amdgpu_vm_update_range(adev, vm, false, false, flush_tlb, NULL, - last_start, prange->start + i, - pte_flags, - (last_start - prange->start) << PAGE_SHIFT, - bo_adev ? bo_adev->vm_manager.vram_base_offset : 0, - NULL, dma_addr, &vm->last_update); + r = amdgpu_vm_update_range( + adev, vm, false, false, flush_tlb, true, NULL, + last_start, prange->start + i, pte_flags, + (last_start - prange->start) << PAGE_SHIFT, + bo_adev ? bo_adev->vm_manager.vram_base_offset : 0, + NULL, dma_addr, &vm->last_update); for (j = last_start - prange->start; j <= i; j++) dma_addr[j] |= last_domain; diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h index f477eda6a2b8..3cdcc299956e 100644 --- a/include/uapi/drm/amdgpu_drm.h +++ b/include/uapi/drm/amdgpu_drm.h @@ -556,6 +556,8 @@ struct drm_amdgpu_gem_op { #define AMDGPU_VM_MTYPE_RW (5 << 5) /* don't allocate MALL */ #define AMDGPU_VM_PAGE_NOALLOC (1 << 9) +/* don't sync on unmap */ +#define AMDGPU_VM_EXPLICIT_SYNC (1 << 10) struct drm_amdgpu_gem_va { /** GEM object handle */ From patchwork Tue Oct 31 13:40:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tatsuyuki Ishi X-Patchwork-Id: 13441540 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D234AC4167B for ; Tue, 31 Oct 2023 13:41:27 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B53B410E4D3; Tue, 31 Oct 2023 13:41:25 +0000 (UTC) Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by gabe.freedesktop.org (Postfix) with ESMTPS id D694910E4D3; Tue, 31 Oct 2023 13:41:23 +0000 (UTC) Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-6bbfb8f7ac4so1736310b3a.0; Tue, 31 Oct 2023 06:41:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698759682; x=1699364482; darn=lists.freedesktop.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jLY3DfRw2y/RKAr/iJks119OQB+JfG13EdCTLNSU2Qg=; b=jr1KL2j+vZ796Vz4GPPm59Pud93Cj8ZPLd2mzbHTVv8uFQ6gN4+C/ducAEE4WpNL/M afJKC2znId6OQF4veGscxF7a6ukBz7gSUxa7IMdAn9jI9qZauzpkx8XtuQG6qU6tCl+V l/PEnyvCkHM7hFyAKebxK09vnoEgKCL5BTdVIka9XPobpJork3tzq/IxOvQOjDSgYom5 uwe//F5kPimnbmac+w/k+aKsFGa/vmqzJh6PaOndOSsmsTlk0laCsdjqE1gOrPZiNk1z G16U8FRYRps0JOfneMQwze2Ib2Dkoy+9R1mvPVMLJjKskcSj1nUhc/3Dl0CJrFXpuYwH Ot1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698759682; x=1699364482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jLY3DfRw2y/RKAr/iJks119OQB+JfG13EdCTLNSU2Qg=; b=MeJhDJvSi2HYsO/VltgAEWfmWAilQhQbElhAU5EvYIDZd/m95DPO1LdfZ4o3Z3n3GL m50PDwp2URQ47kRXESvWAGZdzdYzb3tQikOD9hkqsEkYrwOkqtNf8Zu+ZfGLVQjinWI1 R5NpzGQ+binpi7w4jkAOcAp6BJmD/9fbAg932/sA2aDoiEAWloluSyyZg1cDVbQMJTPf MD/4fJSGLXbrzdS//s3rTLSl/hHTAf/tW7foaZAvrgA1SWqUk8FDrvAnv8c8O39rCxXX 0mOud+41BaEsvD2yyzjmP7u3IXSFUyz33l55wj/WfYB1pkHBULRZv7+ET7X1M0qxRaqw D9Qg== X-Gm-Message-State: AOJu0YxdP5txCp0eu7xr/Bgio8eSY5N03QdiDcOTRQaR4NBUMzeCNUB5 8eBo7n84w3VwX9ql+0qS8uxC8H9CTQYx0t2L X-Google-Smtp-Source: AGHT+IEMSr4k+h6zJ4PjgVM9J89g/YeoNcd17WODfkF+nY7a/j81/1rt1K7p2g1HYg0/vl+bhX2C8g== X-Received: by 2002:a05:6a00:4a14:b0:68e:2fd4:288a with SMTP id do20-20020a056a004a1400b0068e2fd4288amr13471046pfb.3.1698759682302; Tue, 31 Oct 2023 06:41:22 -0700 (PDT) Received: from localhost (zz20184013906F627101.userreverse.dion.ne.jp. [111.98.113.1]) by smtp.gmail.com with ESMTPSA id d14-20020a056a00198e00b006c06804cd39sm1257366pfl.153.2023.10.31.06.41.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 Oct 2023 06:41:21 -0700 (PDT) From: Tatsuyuki Ishi To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 6/6] drm/amdgpu: Bump amdgpu driver version. Date: Tue, 31 Oct 2023 22:40:59 +0900 Message-ID: <20231031134059.171277-7-ishitatsuyuki@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231031134059.171277-1-ishitatsuyuki@gmail.com> References: <20231031134059.171277-1-ishitatsuyuki@gmail.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tatsuyuki Ishi , christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" For detection of the new explicit sync functionality without having to try the ioctl. Signed-off-by: Tatsuyuki Ishi --- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 81edf66dbea8..2aa406dee192 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -113,9 +113,10 @@ * gl1c_cache_size, gl2c_cache_size, mall_size, enabled_rb_pipes_mask_hi * 3.53.0 - Support for GFX11 CP GFX shadowing * 3.54.0 - Add AMDGPU_CTX_QUERY2_FLAGS_RESET_IN_PROGRESS support + * - 3.55.0 - Add AMDGPU_VM_EXPLICIT_SYNC flag for GEM operations. */ #define KMS_DRIVER_MAJOR 3 -#define KMS_DRIVER_MINOR 54 +#define KMS_DRIVER_MINOR 55 #define KMS_DRIVER_PATCHLEVEL 0 unsigned int amdgpu_vram_limit = UINT_MAX;