From patchwork Mon Oct 21 21:18:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844686 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C80FED17140 for ; Mon, 21 Oct 2024 21:18:13 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AF6B810E0B4; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="lgexrK55"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id D4B9410E0B4; Mon, 21 Oct 2024 21:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729545492; x=1761081492; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IF3szAzYLAmUMg1AdI9+r6nlyZGZlqpJGjOyeoigxZo=; b=lgexrK55lX33avPfH9h3OxGl+3KpN9PZeebGlHf1rre0gqUOvHRIhZ8O Vq1niTTzjpMXLTi9Tw57wvyT+3t2gk2MSzduj5qIugS1CTEtc1e2qcqdh ydw6FrZE8tI3tTcWiEWcc8VEfLqpHnJqVrskV4oGLQ/Jseg6KXX4L+1MC bbtnvGbpooWAdCm0F6onu9IwmIvf5jboGHHhUcR8bdomkpHZ4doDxFIom 0d5YQaIMITMJVDDmGV+X8eJ1c7DRf1RWFDC7u62ky3N87hL+BwiToxSDq Sg2qBjxd2Skf3BQ6BKYktne80V7Q893/Cl+wHFcjAS279S03RTfsvCF5A g==; X-CSE-ConnectionGUID: HMVNmVvhTjuI3Xj/hjNbGw== X-CSE-MsgGUID: oDW/cQCgQTiih5hh/iIvLA== X-IronPort-AV: E=McAfee;i="6700,10204,11232"; a="29257678" X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="29257678" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 X-CSE-ConnectionGUID: t8rhjofMSYauXbjBhDmysQ== X-CSE-MsgGUID: ULHYPp3fQi2KaO7nOSYnVw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="80469579" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v5 1/9] drm/xe: Add xe_bo_vm_access Date: Mon, 21 Oct 2024 14:18:27 -0700 Message-Id: <20241021211835.1675640-2-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021211835.1675640-1-matthew.brost@intel.com> References: <20241021211835.1675640-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add xe_bo_vm_access which is wrapper around ttm_bo_vm_access which takes rpm refs for device access. Suggested-by: Thomas Hellström Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/xe_bo.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 5b232f2951b1..0261a8b29351 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -1236,11 +1236,26 @@ static vm_fault_t xe_gem_fault(struct vm_fault *vmf) return ret; } +static int xe_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, + void *buf, int len, int write) +{ + struct ttm_buffer_object *ttm_bo = vma->vm_private_data; + struct xe_bo *bo = ttm_to_xe_bo(ttm_bo); + struct xe_device *xe = xe_bo_device(bo); + int ret; + + xe_pm_runtime_get(xe); + ret = ttm_bo_vm_access(vma, addr, buf, len, write); + xe_pm_runtime_put(xe); + + return ret; +} + static const struct vm_operations_struct xe_gem_vm_ops = { .fault = xe_gem_fault, .open = ttm_bo_vm_open, .close = ttm_bo_vm_close, - .access = ttm_bo_vm_access + .access = xe_bo_vm_access, }; static const struct drm_gem_object_funcs xe_gem_object_funcs = { From patchwork Mon Oct 21 21:18:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4F1EFD17141 for ; Mon, 21 Oct 2024 21:18:19 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CCBA410E119; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="IkVuteXx"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 068BF10E056; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729545492; x=1761081492; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Os2V+KRKEhyXzew/5Rz+DuBmMm0Uhq0rfkIAT6jkQrM=; b=IkVuteXxccEqbCeVSpIjFt+a/D6hNjvRsiUgbhmhm9FiiBEzOHMzit58 eatEZ9HARyY+3wXu5OzNgdc5xGX/yeOgu2C5zJKq+KLrrIZgg7Nwd6xWc WsTJr6qLXq0ZNX5WhWPeluaa4OIGQBgdFXDZ4xuHOnbO3RvWs2tPMFMjA g5lnBVaLm4KysF6b7DwoIFprWCCmMCMdhrxiIaevk9iEyWL2fQ/5F/DqH xneVpfGA3RkJ9EjRMPv+urVZsp+ptzzspGZfID114V2JGuEFxwNHLDVmx FSWaTEudJGd5PUx2hpHkStTkhQ089z2YrpoLyD9BzNcoRZueeWPxWhSON Q==; X-CSE-ConnectionGUID: vEa6/9FATbmZfUmbk4j5aQ== X-CSE-MsgGUID: KXFTZzFdR6KWzZeFpbBy6g== X-IronPort-AV: E=McAfee;i="6700,10204,11232"; a="29257679" X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="29257679" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 X-CSE-ConnectionGUID: 961B/XQ/Rf69D6N6NBcm2Q== X-CSE-MsgGUID: /7r4Oyh1RYO1uJbUXGR77A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="80469582" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v5 2/9] drm/ttm: Add ttm_bo_access Date: Mon, 21 Oct 2024 14:18:28 -0700 Message-Id: <20241021211835.1675640-3-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021211835.1675640-1-matthew.brost@intel.com> References: <20241021211835.1675640-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Non-contiguous VRAM cannot easily be mapped in TTM nor can non-visible VRAM easily be accessed. Add ttm_bo_access, which is similar to ttm_bo_vm_access, to access such memory. v4: - Fix checkpatch warnings (CI) v5: - Fix checkpatch warnings (CI) Reported-by: Christoph Manszewski Suggested-by: Thomas Hellström Signed-off-by: Matthew Brost Tested-by: Mika Kuoppala Reviewed-by: Matthew Auld --- drivers/gpu/drm/ttm/ttm_bo_util.c | 86 +++++++++++++++++++++++++++++++ drivers/gpu/drm/ttm/ttm_bo_vm.c | 65 +---------------------- include/drm/ttm/ttm_bo.h | 2 + 3 files changed, 89 insertions(+), 64 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c index d939925efa81..41bb5a7477d3 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_util.c +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c @@ -919,3 +919,89 @@ s64 ttm_lru_walk_for_evict(struct ttm_lru_walk *walk, struct ttm_device *bdev, return progress; } + +static int ttm_bo_access_kmap(struct ttm_buffer_object *bo, + unsigned long offset, + void *buf, int len, int write) +{ + unsigned long page = offset >> PAGE_SHIFT; + unsigned long bytes_left = len; + int ret; + + /* Copy a page at a time, that way no extra virtual address + * mapping is needed + */ + offset -= page << PAGE_SHIFT; + do { + unsigned long bytes = min(bytes_left, PAGE_SIZE - offset); + struct ttm_bo_kmap_obj map; + void *ptr; + bool is_iomem; + + ret = ttm_bo_kmap(bo, page, 1, &map); + if (ret) + return ret; + + ptr = (void *)ttm_kmap_obj_virtual(&map, &is_iomem) + offset; + WARN_ON_ONCE(is_iomem); + if (write) + memcpy(ptr, buf, bytes); + else + memcpy(buf, ptr, bytes); + ttm_bo_kunmap(&map); + + page++; + buf += bytes; + bytes_left -= bytes; + offset = 0; + } while (bytes_left); + + return len; +} + +/** + * ttm_bo_access - Helper to access a buffer object + * + * @bo: ttm buffer object + * @offset: access offset into buffer object + * @buf: pointer to caller memory to read into or write from + * @len: length of access + * @write: write access + * + * Utility function to access a buffer object. Useful when buffer object cannot + * be easily mapped (non-contiguous, non-visible, etc...). + * + * Returns: + * 0 if successful, negative error code on failure. + */ +int ttm_bo_access(struct ttm_buffer_object *bo, unsigned long offset, + void *buf, int len, int write) +{ + int ret; + + if (len < 1 || (offset + len) > bo->base.size) + return -EIO; + + ret = ttm_bo_reserve(bo, true, false, NULL); + if (ret) + return ret; + + switch (bo->resource->mem_type) { + case TTM_PL_SYSTEM: + fallthrough; + case TTM_PL_TT: + ret = ttm_bo_access_kmap(bo, offset, buf, len, write); + break; + default: + if (bo->bdev->funcs->access_memory) + ret = bo->bdev->funcs->access_memory + (bo, offset, buf, len, write); + else + ret = -EIO; + } + + ttm_bo_unreserve(bo); + + return ret; +} +EXPORT_SYMBOL(ttm_bo_access); diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 2c699ed1963a..20b1e5f78684 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -366,45 +366,6 @@ void ttm_bo_vm_close(struct vm_area_struct *vma) } EXPORT_SYMBOL(ttm_bo_vm_close); -static int ttm_bo_vm_access_kmap(struct ttm_buffer_object *bo, - unsigned long offset, - uint8_t *buf, int len, int write) -{ - unsigned long page = offset >> PAGE_SHIFT; - unsigned long bytes_left = len; - int ret; - - /* Copy a page at a time, that way no extra virtual address - * mapping is needed - */ - offset -= page << PAGE_SHIFT; - do { - unsigned long bytes = min(bytes_left, PAGE_SIZE - offset); - struct ttm_bo_kmap_obj map; - void *ptr; - bool is_iomem; - - ret = ttm_bo_kmap(bo, page, 1, &map); - if (ret) - return ret; - - ptr = (uint8_t *)ttm_kmap_obj_virtual(&map, &is_iomem) + offset; - WARN_ON_ONCE(is_iomem); - if (write) - memcpy(ptr, buf, bytes); - else - memcpy(buf, ptr, bytes); - ttm_bo_kunmap(&map); - - page++; - buf += bytes; - bytes_left -= bytes; - offset = 0; - } while (bytes_left); - - return len; -} - int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write) { @@ -412,32 +373,8 @@ int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, unsigned long offset = (addr) - vma->vm_start + ((vma->vm_pgoff - drm_vma_node_start(&bo->base.vma_node)) << PAGE_SHIFT); - int ret; - - if (len < 1 || (offset + len) > bo->base.size) - return -EIO; - ret = ttm_bo_reserve(bo, true, false, NULL); - if (ret) - return ret; - - switch (bo->resource->mem_type) { - case TTM_PL_SYSTEM: - fallthrough; - case TTM_PL_TT: - ret = ttm_bo_vm_access_kmap(bo, offset, buf, len, write); - break; - default: - if (bo->bdev->funcs->access_memory) - ret = bo->bdev->funcs->access_memory( - bo, offset, buf, len, write); - else - ret = -EIO; - } - - ttm_bo_unreserve(bo); - - return ret; + return ttm_bo_access(bo, offset, buf, len, write); } EXPORT_SYMBOL(ttm_bo_vm_access); diff --git a/include/drm/ttm/ttm_bo.h b/include/drm/ttm/ttm_bo.h index 5804408815be..8ea11cd8df39 100644 --- a/include/drm/ttm/ttm_bo.h +++ b/include/drm/ttm/ttm_bo.h @@ -421,6 +421,8 @@ void ttm_bo_unpin(struct ttm_buffer_object *bo); int ttm_bo_evict_first(struct ttm_device *bdev, struct ttm_resource_manager *man, struct ttm_operation_ctx *ctx); +int ttm_bo_access(struct ttm_buffer_object *bo, unsigned long offset, + void *buf, int len, int write); vm_fault_t ttm_bo_vm_reserve(struct ttm_buffer_object *bo, struct vm_fault *vmf); vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, From patchwork Mon Oct 21 21:18:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 39132D17140 for ; Mon, 21 Oct 2024 21:18:26 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 82D3E10E5B5; Mon, 21 Oct 2024 21:18:18 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="lr+OOAL1"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0DE0010E119; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729545492; x=1761081492; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a5HCOwVKHgAmUvo9H7JPdG4YV6V4zmZgKNtprKhXjJ4=; b=lr+OOAL1M8247ZRKOnBg4QR+O9wI+7ADG3rbcbRd6cp2F++prHb3ccDC erSFEb1mMIF2iiDC/wKVd/ds6R5YY+p0T0YPVqJgY9zCW7mmS7yYtDDz2 8Ttm5xvdCfk1EqiPDo5hXGmI+abzaWrKlB7D2CgMCTtjQ0qbSwZOc/qpI 7zDTOQ9ymRQ/jeRgU6vMJi3J2SY9ink5ScyIa95WGruhF9t2NBKCIxHXP /ABByLTrDmD/ZQeYLNLSO0Tf3tWzlaFaxt6YYhZGCATul7OA0YXY+AUyr m2850YMc7xYd2LfkWZt8JzI7me6+TTtwDe8Ie2t3J/MTvygAjAiLq0OU2 Q==; X-CSE-ConnectionGUID: wxfaNiRjQ0CVotnRGbpTdQ== X-CSE-MsgGUID: 4t2CwpfbTAq9Vpp4uQgPIQ== X-IronPort-AV: E=McAfee;i="6700,10204,11232"; a="29257681" X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="29257681" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 X-CSE-ConnectionGUID: MkrSM9ppRxW20rgKf3nrBg== X-CSE-MsgGUID: Jpj8EGAqTYqVaeAUZo1iow== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="80469584" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v5 3/9] drm/xe: Add xe_ttm_access_memory Date: Mon, 21 Oct 2024 14:18:29 -0700 Message-Id: <20241021211835.1675640-4-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021211835.1675640-1-matthew.brost@intel.com> References: <20241021211835.1675640-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Non-contiguous VRAM cannot easily be mapped in TTM nor can non-visible VRAM easily be accessed. Add xe_ttm_access_memory which hooks into ttm_bo_access to access such memory. v4: - Assert memory access rather than taking RPM ref (Thomas / Auld) - Fix warning on xe_res_cursor.h for non-zero offset (Mika) Reported-by: Christoph Manszewski Suggested-by: Thomas Hellström Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/xe_bo.c | 59 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 56 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 0261a8b29351..04bc042f3bc7 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -442,6 +442,14 @@ static void xe_ttm_tt_destroy(struct ttm_device *ttm_dev, struct ttm_tt *tt) kfree(tt); } +static bool xe_ttm_resource_visible(struct ttm_resource *mem) +{ + struct xe_ttm_vram_mgr_resource *vres = + to_xe_ttm_vram_mgr_resource(mem); + + return vres->used_visible_size == mem->size; +} + static int xe_ttm_io_mem_reserve(struct ttm_device *bdev, struct ttm_resource *mem) { @@ -453,11 +461,9 @@ static int xe_ttm_io_mem_reserve(struct ttm_device *bdev, return 0; case XE_PL_VRAM0: case XE_PL_VRAM1: { - struct xe_ttm_vram_mgr_resource *vres = - to_xe_ttm_vram_mgr_resource(mem); struct xe_mem_region *vram = res_to_mem_region(mem); - if (vres->used_visible_size < mem->size) + if (!xe_ttm_resource_visible(mem)) return -EINVAL; mem->bus.offset = mem->start << PAGE_SHIFT; @@ -1111,6 +1117,52 @@ static void xe_ttm_bo_swap_notify(struct ttm_buffer_object *ttm_bo) } } +static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, + unsigned long offset, void *buf, int len, + int write) +{ + struct xe_bo *bo = ttm_to_xe_bo(ttm_bo); + struct xe_device *xe = ttm_to_xe_device(ttm_bo->bdev); + struct iosys_map vmap; + struct xe_res_cursor cursor; + struct xe_mem_region *vram; + int bytes_left = len; + + xe_bo_assert_held(bo); + xe_device_assert_mem_access(xe); + + if (!mem_type_is_vram(ttm_bo->resource->mem_type)) + return -EIO; + + /* FIXME: Use GPU for non-visible VRAM */ + if (!xe_ttm_resource_visible(ttm_bo->resource)) + return -EIO; + + vram = res_to_mem_region(ttm_bo->resource); + xe_res_first(ttm_bo->resource, offset & PAGE_MASK, + bo->size - (offset & PAGE_MASK), &cursor); + + do { + unsigned long page_offset = (offset & ~PAGE_MASK); + int byte_count = min((int)(PAGE_SIZE - page_offset), bytes_left); + + iosys_map_set_vaddr_iomem(&vmap, (u8 __iomem *)vram->mapping + + cursor.start); + if (write) + xe_map_memcpy_to(xe, &vmap, page_offset, buf, byte_count); + else + xe_map_memcpy_from(xe, buf, &vmap, page_offset, byte_count); + + offset += byte_count; + buf += byte_count; + bytes_left -= byte_count; + if (bytes_left) + xe_res_next(&cursor, PAGE_SIZE); + } while (bytes_left); + + return len; +} + const struct ttm_device_funcs xe_ttm_funcs = { .ttm_tt_create = xe_ttm_tt_create, .ttm_tt_populate = xe_ttm_tt_populate, @@ -1120,6 +1172,7 @@ const struct ttm_device_funcs xe_ttm_funcs = { .move = xe_bo_move, .io_mem_reserve = xe_ttm_io_mem_reserve, .io_mem_pfn = xe_ttm_io_mem_pfn, + .access_memory = xe_ttm_access_memory, .release_notify = xe_ttm_bo_release_notify, .eviction_valuable = ttm_bo_eviction_valuable, .delete_mem_notify = xe_ttm_bo_delete_mem_notify, From patchwork Mon Oct 21 21:18:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EA321D1713C for ; Mon, 21 Oct 2024 21:18:23 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id ACB0D10E5A8; Mon, 21 Oct 2024 21:18:13 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="dEZGS9iH"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2B34B10E0B4; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729545492; x=1761081492; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VFeTNJiJ7lL3Wyx7JIq1pnfndPJQjcZDjNTMoynIz8U=; b=dEZGS9iH1OWzD9FjqO5RnqzZoAN7vSeyRTolNHz+8qGoeJQ+eB3zGadN KGNSmCVzbhzyDq+DiC/P4/UA/hkzS3trwmawBopCMwjCos7IxSOau4Res OyxqVlCPE122JbICllcF/VAnqC+Px6Poc3RHuFSokarIvIz4lpLkdawxh +RdTQbI3houYuIpeY5GjIigS6gAfrTedUryVbVMEXhyPQ2zt1YuXZv55u V4xtmRFgwkF0eyf4zirmzweLek0Q/yRC3XiVMAVxK0bRdBdSiox9rSrHk sBZJlx5vvLnYGCXnRUJvV7PC3pwMHzBS+S+1NEp6lQG3T2eOVkCFtTtCK A==; X-CSE-ConnectionGUID: PbmMNz+MR4STcFWi904n6Q== X-CSE-MsgGUID: P4ChFc25Qjycbf0M8tJdoA== X-IronPort-AV: E=McAfee;i="6700,10204,11232"; a="29257682" X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="29257682" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:11 -0700 X-CSE-ConnectionGUID: 8CeWgTrcQuGOdiQSYutr+g== X-CSE-MsgGUID: Ll1PLFmkSFe5czBWV7LOeg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="80469590" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v5 4/9] drm/xe: Take PM ref in delayed snapshot capture worker Date: Mon, 21 Oct 2024 14:18:30 -0700 Message-Id: <20241021211835.1675640-5-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021211835.1675640-1-matthew.brost@intel.com> References: <20241021211835.1675640-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The delayed snapshot capture worker can access the GPU or VRAM both of which require a PM reference. Take a reference in this worker. Cc: Rodrigo Vivi Cc: Maarten Lankhorst Fixes: 4f04d07c0a94 ("drm/xe: Faster devcoredump") Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/xe_devcoredump.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c index 8b0ea77661b2..956bf087d0db 100644 --- a/drivers/gpu/drm/xe/xe_devcoredump.c +++ b/drivers/gpu/drm/xe/xe_devcoredump.c @@ -23,6 +23,7 @@ #include "xe_guc_submit.h" #include "xe_hw_engine.h" #include "xe_module.h" +#include "xe_pm.h" #include "xe_sched_job.h" #include "xe_vm.h" @@ -158,8 +159,11 @@ static void xe_devcoredump_deferred_snap_work(struct work_struct *work) { struct xe_devcoredump_snapshot *ss = container_of(work, typeof(*ss), work); struct xe_devcoredump *coredump = container_of(ss, typeof(*coredump), snapshot); + struct xe_device *xe = coredump_to_xe(coredump); unsigned int fw_ref; + xe_pm_runtime_get(xe); + /* keep going if fw fails as we still want to save the memory and SW data */ fw_ref = xe_force_wake_get(gt_to_fw(ss->gt), XE_FORCEWAKE_ALL); if (!xe_force_wake_ref_has_domain(fw_ref, XE_FORCEWAKE_ALL)) @@ -168,6 +172,8 @@ static void xe_devcoredump_deferred_snap_work(struct work_struct *work) xe_guc_exec_queue_snapshot_capture_delayed(ss->ge); xe_force_wake_put(gt_to_fw(ss->gt), fw_ref); + xe_pm_runtime_put(xe); + /* Calculate devcoredump size */ ss->read.size = __xe_devcoredump_read(NULL, INT_MAX, coredump); From patchwork Mon Oct 21 21:18:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2F72D17143 for ; Mon, 21 Oct 2024 21:18:21 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5260110E5A4; Mon, 21 Oct 2024 21:18:13 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="evGmWcDq"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 320DF10E191; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729545492; x=1761081492; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6Rk/rfb6XrQZmrOtBxoemgoxDUAhf+3rm0ayKq9Zd2Q=; b=evGmWcDqk9pF3bzGIrPYfWNnzF8O2SJdwthuLt0Jh80G1lnn3uiF1y0Z jW01Xl3WGcqiz0hC3T21KlPEVXf6QMWnA0nO3waqfv011/gJC6mPklSZ0 1Voz3yB5l2xp6pImrdoaNwUR0TpdzI0ZMH6szPfgvjIP/K/EMLYtmcNd+ YGpxFgeDQV73vHrKe7VZuuEbAKoPWpqutSbeG7coP2uyM064LaQO8Nlxc 3yQiwrCGIiz9rXn9YQCIjdnsPpJch9EA6siihzRZDb9mbaSlGoXZZUeaW QHQhEF8lNaJbadc4Y0zvd93aPQmOOqWqOg9W2yTeyi2urjbBLKxamT8b7 w==; X-CSE-ConnectionGUID: 1uNlOJTZQ2mPt5IMS3/IuA== X-CSE-MsgGUID: aSkdMwhTQt+ZQ9pOKxyNLQ== X-IronPort-AV: E=McAfee;i="6700,10204,11232"; a="29257684" X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="29257684" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:11 -0700 X-CSE-ConnectionGUID: 6jv4GSHgQYafiXXxhnDsrw== X-CSE-MsgGUID: 2wO+npBCQ9yMbPMGvPRTMA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="80469593" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v5 5/9] drm/xe: Update xe_ttm_access_memory to use GPU for non-visible access Date: Mon, 21 Oct 2024 14:18:31 -0700 Message-Id: <20241021211835.1675640-6-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021211835.1675640-1-matthew.brost@intel.com> References: <20241021211835.1675640-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add migrate layer functions to access VRAM and update xe_ttm_access_memory to use for non-visible access and large (more tahn 16k) BO access. v4: - Fix non-page aligned accesses - Add support for small / unaligned access - Update commit message indicating migrate used for large accesses (Auld) - Fix warning in xe_res_cursor for non-zero offset v5: - Fix 32 bit build (CI) Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/xe_bo.c | 15 +- drivers/gpu/drm/xe/xe_migrate.c | 325 ++++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_migrate.h | 4 + 3 files changed, 340 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 04bc042f3bc7..4e1f54aad5bc 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -1127,6 +1127,7 @@ static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, struct xe_res_cursor cursor; struct xe_mem_region *vram; int bytes_left = len; + int err = 0; xe_bo_assert_held(bo); xe_device_assert_mem_access(xe); @@ -1134,9 +1135,14 @@ static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, if (!mem_type_is_vram(ttm_bo->resource->mem_type)) return -EIO; - /* FIXME: Use GPU for non-visible VRAM */ - if (!xe_ttm_resource_visible(ttm_bo->resource)) - return -EIO; + if (!xe_ttm_resource_visible(ttm_bo->resource) || len >= SZ_16K) { + struct xe_migrate *migrate = + mem_type_to_migrate(xe, ttm_bo->resource->mem_type); + + err = xe_migrate_access_memory(migrate, bo, offset, buf, len, + write); + goto out; + } vram = res_to_mem_region(ttm_bo->resource); xe_res_first(ttm_bo->resource, offset & PAGE_MASK, @@ -1160,7 +1166,8 @@ static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, xe_res_next(&cursor, PAGE_SIZE); } while (bytes_left); - return len; +out: + return err ?: len; } const struct ttm_device_funcs xe_ttm_funcs = { diff --git a/drivers/gpu/drm/xe/xe_migrate.c b/drivers/gpu/drm/xe/xe_migrate.c index cfd31ae49cc1..05ade498f559 100644 --- a/drivers/gpu/drm/xe/xe_migrate.c +++ b/drivers/gpu/drm/xe/xe_migrate.c @@ -669,6 +669,7 @@ static void emit_copy(struct xe_gt *gt, struct xe_bb *bb, u32 mocs = 0; u32 tile_y = 0; + xe_gt_assert(gt, !(pitch & 3)); xe_gt_assert(gt, size / pitch <= S16_MAX); xe_gt_assert(gt, pitch / 4 <= S16_MAX); xe_gt_assert(gt, pitch <= U16_MAX); @@ -1542,6 +1543,330 @@ void xe_migrate_wait(struct xe_migrate *m) dma_fence_wait(m->fence, false); } +static u32 pte_update_cmd_size(u64 size) +{ + u32 dword; + u64 entries = DIV_ROUND_UP(size, XE_PAGE_SIZE); + + XE_WARN_ON(size > MAX_PREEMPTDISABLE_TRANSFER); + /* + * MI_STORE_DATA_IMM command is used to update page table. Each + * instruction can update maximumly 0x1ff pte entries. To update + * n (n <= 0x1ff) pte entries, we need: + * 1 dword for the MI_STORE_DATA_IMM command header (opcode etc) + * 2 dword for the page table's physical location + * 2*n dword for value of pte to fill (each pte entry is 2 dwords) + */ + dword = (1 + 2) * DIV_ROUND_UP(entries, 0x1ff); + dword += entries * 2; + + return dword; +} + +static void build_pt_update_batch_sram(struct xe_migrate *m, + struct xe_bb *bb, u32 pt_offset, + dma_addr_t *sram_addr, u32 size) +{ + u16 pat_index = tile_to_xe(m->tile)->pat.idx[XE_CACHE_WB]; + u32 ptes; + int i = 0; + + ptes = DIV_ROUND_UP(size, XE_PAGE_SIZE); + while (ptes) { + u32 chunk = min(0x1ffU, ptes); + + bb->cs[bb->len++] = MI_STORE_DATA_IMM | MI_SDI_NUM_QW(chunk); + bb->cs[bb->len++] = pt_offset; + bb->cs[bb->len++] = 0; + + pt_offset += chunk * 8; + ptes -= chunk; + + while (chunk--) { + u64 addr = sram_addr[i++] & PAGE_MASK; + + xe_tile_assert(m->tile, addr); + addr = m->q->vm->pt_ops->pte_encode_addr(m->tile->xe, + addr, pat_index, + 0, false, 0); + bb->cs[bb->len++] = lower_32_bits(addr); + bb->cs[bb->len++] = upper_32_bits(addr); + } + } +} + +enum xe_migrate_copy_dir { + XE_MIGRATE_COPY_TO_VRAM, + XE_MIGRATE_COPY_TO_SRAM, +}; + +#define CACHELINE_BYTES 64ull +#define CACHELINE_MASK (CACHELINE_BYTES - 1) + +static struct dma_fence *xe_migrate_vram(struct xe_migrate *m, + unsigned long len, + unsigned long sram_offset, + dma_addr_t *sram_addr, u64 vram_addr, + const enum xe_migrate_copy_dir dir) +{ + struct xe_gt *gt = m->tile->primary_gt; + struct xe_device *xe = gt_to_xe(gt); + struct dma_fence *fence = NULL; + u32 batch_size = 2; + u64 src_L0_ofs, dst_L0_ofs; + struct xe_sched_job *job; + struct xe_bb *bb; + u32 update_idx, pt_slot = 0; + unsigned long npages = DIV_ROUND_UP(len + sram_offset, PAGE_SIZE); + unsigned int pitch = len >= PAGE_SIZE && !(len & ~PAGE_MASK) ? + PAGE_SIZE : 4; + int err; + + if (drm_WARN_ON(&xe->drm, (len & CACHELINE_MASK) || + (sram_offset | vram_addr) & CACHELINE_MASK)) + return ERR_PTR(-EOPNOTSUPP); + + xe_assert(xe, npages * PAGE_SIZE <= MAX_PREEMPTDISABLE_TRANSFER); + + batch_size += pte_update_cmd_size(len); + batch_size += EMIT_COPY_DW; + + bb = xe_bb_new(gt, batch_size, true); + if (IS_ERR(bb)) { + err = PTR_ERR(bb); + return ERR_PTR(err); + } + + build_pt_update_batch_sram(m, bb, pt_slot * XE_PAGE_SIZE, + sram_addr, len + sram_offset); + + if (dir == XE_MIGRATE_COPY_TO_VRAM) { + src_L0_ofs = xe_migrate_vm_addr(pt_slot, 0) + sram_offset; + dst_L0_ofs = xe_migrate_vram_ofs(xe, vram_addr, false); + + } else { + src_L0_ofs = xe_migrate_vram_ofs(xe, vram_addr, false); + dst_L0_ofs = xe_migrate_vm_addr(pt_slot, 0) + sram_offset; + } + + bb->cs[bb->len++] = MI_BATCH_BUFFER_END; + update_idx = bb->len; + + emit_copy(gt, bb, src_L0_ofs, dst_L0_ofs, len, pitch); + + job = xe_bb_create_migration_job(m->q, bb, + xe_migrate_batch_base(m, true), + update_idx); + if (IS_ERR(job)) { + err = PTR_ERR(job); + goto err; + } + + xe_sched_job_add_migrate_flush(job, 0); + + mutex_lock(&m->job_mutex); + xe_sched_job_arm(job); + fence = dma_fence_get(&job->drm.s_fence->finished); + xe_sched_job_push(job); + + dma_fence_put(m->fence); + m->fence = dma_fence_get(fence); + mutex_unlock(&m->job_mutex); + + xe_bb_free(bb, fence); + + return fence; + +err: + xe_bb_free(bb, NULL); + + return ERR_PTR(err); +} + +static void xe_migrate_dma_unmap(struct xe_device *xe, dma_addr_t *dma_addr, + int len, int write) +{ + unsigned long i, npages = DIV_ROUND_UP(len, PAGE_SIZE); + + for (i = 0; i < npages; ++i) { + if (!dma_addr[i]) + continue; + + dma_unmap_page(xe->drm.dev, dma_addr[i], PAGE_SIZE, + write ? DMA_TO_DEVICE : DMA_FROM_DEVICE); + } + kfree(dma_addr); +} + +static dma_addr_t *xe_migrate_dma_map(struct xe_device *xe, + void *buf, int len, int write) +{ + dma_addr_t *dma_addr; + unsigned long i, npages = DIV_ROUND_UP(len, PAGE_SIZE); + + dma_addr = kcalloc(npages, sizeof(*dma_addr), GFP_KERNEL); + if (!dma_addr) + return ERR_PTR(-ENOMEM); + + for (i = 0; i < npages; ++i) { + dma_addr_t addr; + struct page *page; + + if (is_vmalloc_addr(buf)) + page = vmalloc_to_page(buf); + else + page = virt_to_page(buf); + + addr = dma_map_page(xe->drm.dev, + page, 0, PAGE_SIZE, + write ? DMA_TO_DEVICE : + DMA_FROM_DEVICE); + if (dma_mapping_error(xe->drm.dev, addr)) + goto err_fault; + + dma_addr[i] = addr; + buf += PAGE_SIZE; + } + + return dma_addr; + +err_fault: + xe_migrate_dma_unmap(xe, dma_addr, len, write); + return ERR_PTR(-EFAULT); +} + +/** + * xe_migrate_access_memory - Access memory of a BO via GPU + * + * @m: The migration context. + * @bo: buffer object + * @offset: access offset into buffer object + * @buf: pointer to caller memory to read into or write from + * @len: length of access + * @write: write access + * + * Access memory of a BO via GPU either reading in or writing from a passed in + * pointer. Pointer is dma mapped for GPU access and GPU commands are issued to + * read to or write from pointer. + * + * Returns: + * 0 if successful, negative error code on failure. + */ +int xe_migrate_access_memory(struct xe_migrate *m, struct xe_bo *bo, + unsigned long offset, void *buf, int len, + int write) +{ + struct xe_tile *tile = m->tile; + struct xe_device *xe = tile_to_xe(tile); + struct xe_res_cursor cursor; + struct dma_fence *fence = NULL; + dma_addr_t *dma_addr; + unsigned long page_offset = (unsigned long)buf & ~PAGE_MASK; + int bytes_left = len, current_page = 0; + void *orig_buf = buf; + + xe_bo_assert_held(bo); + + /* Use bounce buffer for small access and unaligned access */ + if (len & CACHELINE_MASK || ((uintptr_t)buf | offset) & CACHELINE_MASK) { + int buf_offset = 0; + + /* + * Less than ideal for large unaligned access but this should be + * fairly rare, can fixup if this becomes common. + */ + do { + u8 bounce[CACHELINE_BYTES]; + void *ptr = (void *)bounce; + int err; + int copy_bytes = min_t(int, bytes_left, + CACHELINE_BYTES - + (offset & CACHELINE_MASK)); + int ptr_offset = offset & CACHELINE_MASK; + + err = xe_migrate_access_memory(m, bo, + offset & ~CACHELINE_MASK, + (void *)ptr, + sizeof(bounce), 0); + if (err) + return err; + + if (!write) { + memcpy(buf + buf_offset, ptr + ptr_offset, + copy_bytes); + goto next; + } + + memcpy(ptr + ptr_offset, buf + buf_offset, copy_bytes); + err = xe_migrate_access_memory(m, bo, + offset & ~CACHELINE_MASK, + (void *)ptr, + sizeof(bounce), 0); + if (err) + return err; + +next: + bytes_left -= copy_bytes; + buf_offset += copy_bytes; + offset += copy_bytes; + } while (bytes_left); + + return 0; + } + + dma_addr = xe_migrate_dma_map(xe, buf, len + page_offset, write); + if (IS_ERR(dma_addr)) + return PTR_ERR(dma_addr); + + xe_res_first(bo->ttm.resource, offset, bo->size - offset, &cursor); + + do { + struct dma_fence *__fence; + u64 vram_addr = vram_region_gpu_offset(bo->ttm.resource) + + cursor.start; + int current_bytes; + + if (cursor.size > MAX_PREEMPTDISABLE_TRANSFER) + current_bytes = min_t(int, bytes_left, + MAX_PREEMPTDISABLE_TRANSFER); + else + current_bytes = min_t(int, bytes_left, cursor.size); + + if (fence) + dma_fence_put(fence); + + __fence = xe_migrate_vram(m, current_bytes, + (unsigned long)buf & ~PAGE_MASK, + dma_addr + current_page, + vram_addr, write ? + XE_MIGRATE_COPY_TO_VRAM : + XE_MIGRATE_COPY_TO_SRAM); + if (IS_ERR(__fence)) { + if (fence) + dma_fence_wait(fence, false); + fence = __fence; + goto out_err; + } + fence = __fence; + + buf += current_bytes; + offset += current_bytes; + current_page = (int)(buf - orig_buf) / PAGE_SIZE; + bytes_left -= current_bytes; + if (bytes_left) + xe_res_next(&cursor, current_bytes); + } while (bytes_left); + + dma_fence_wait(fence, false); + xe_migrate_dma_unmap(xe, dma_addr, len + page_offset, write); + + return 0; + +out_err: + xe_migrate_dma_unmap(xe, dma_addr, len + page_offset, write); + return PTR_ERR(fence); +} + #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) #include "tests/xe_migrate.c" #endif diff --git a/drivers/gpu/drm/xe/xe_migrate.h b/drivers/gpu/drm/xe/xe_migrate.h index 0109866e398a..94197d262178 100644 --- a/drivers/gpu/drm/xe/xe_migrate.h +++ b/drivers/gpu/drm/xe/xe_migrate.h @@ -102,6 +102,10 @@ struct dma_fence *xe_migrate_copy(struct xe_migrate *m, struct ttm_resource *dst, bool copy_only_ccs); +int xe_migrate_access_memory(struct xe_migrate *m, struct xe_bo *bo, + unsigned long offset, void *buf, int len, + int write); + #define XE_MIGRATE_CLEAR_FLAG_BO_DATA BIT(0) #define XE_MIGRATE_CLEAR_FLAG_CCS_DATA BIT(1) #define XE_MIGRATE_CLEAR_FLAG_FULL (XE_MIGRATE_CLEAR_FLAG_BO_DATA | \ From patchwork Mon Oct 21 21:18:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6D7E9D1713C for ; Mon, 21 Oct 2024 21:18:17 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B6CAC10E216; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="ArrPIYPP"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5042710E056; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729545493; x=1761081493; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5uXjQe9E+NBXZdyGgi2OooU8JncrVZKopc7IxW16Bl0=; b=ArrPIYPPv+MskrphlIf48IeQsbkeqUod6dhYVjpejmQ23AhIqoRyddpm DHicyRWlsMJ31fJ+LKrg4UsIcyzlzR7bg+04uHto/CX7isEsdgwNQyksB giJs11BlsaQbBDEPS6u7VAvk3EGm+Tu7iB530bX/PkgqSAm0em0NynMy4 I848JwGDtuBGBJ3qE82EPpEp8HAqCqhTq0U0+DQwhxhPzHmQMSKL0dXvI gxkEhrpUmoVu4h0M4i4Jqswj3xeHwCFfHtWAXvyYFXaIzj7ypfPkgjJ8P HgVJjYpGueP9eBXBm2FqYwgGYAmKSnMdbVD8Gpjfj9qOmrhkwR074D4Et g==; X-CSE-ConnectionGUID: qddCHWnPQPmf8ze4JNkrjg== X-CSE-MsgGUID: JBJKS337S+aPYe2frAnJ7Q== X-IronPort-AV: E=McAfee;i="6700,10204,11232"; a="29257685" X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="29257685" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:11 -0700 X-CSE-ConnectionGUID: DD1yGEgXTtSS+SWgBvpxRg== X-CSE-MsgGUID: 8Cfu/+THTQqRo/bbvLNelw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="80469595" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v5 6/9] drm/xe/display: Update intel_bo_read_from_page to use ttm_bo_access Date: Mon, 21 Oct 2024 14:18:32 -0700 Message-Id: <20241021211835.1675640-7-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021211835.1675640-1-matthew.brost@intel.com> References: <20241021211835.1675640-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Don't open code vmap of a BO, use ttm_bo_access helper which is safe for non-contiguous BOs and non-visible BOs. Suggested-by: Matthew Auld Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/display/intel_bo.c | 25 +------------------------ 1 file changed, 1 insertion(+), 24 deletions(-) diff --git a/drivers/gpu/drm/xe/display/intel_bo.c b/drivers/gpu/drm/xe/display/intel_bo.c index 9f54fad0f1c0..43141964f6f2 100644 --- a/drivers/gpu/drm/xe/display/intel_bo.c +++ b/drivers/gpu/drm/xe/display/intel_bo.c @@ -40,31 +40,8 @@ int intel_bo_fb_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) int intel_bo_read_from_page(struct drm_gem_object *obj, u64 offset, void *dst, int size) { struct xe_bo *bo = gem_to_xe_bo(obj); - struct ttm_bo_kmap_obj map; - void *src; - bool is_iomem; - int ret; - ret = xe_bo_lock(bo, true); - if (ret) - return ret; - - ret = ttm_bo_kmap(&bo->ttm, offset >> PAGE_SHIFT, 1, &map); - if (ret) - goto out_unlock; - - offset &= ~PAGE_MASK; - src = ttm_kmap_obj_virtual(&map, &is_iomem); - src += offset; - if (is_iomem) - memcpy_fromio(dst, (void __iomem *)src, size); - else - memcpy(dst, src, size); - - ttm_bo_kunmap(&map); -out_unlock: - xe_bo_unlock(bo); - return ret; + return ttm_bo_access(&bo->ttm, offset, dst, size, 0); } struct intel_frontbuffer *intel_bo_get_frontbuffer(struct drm_gem_object *obj) From patchwork Mon Oct 21 21:18:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BA5ADD17138 for ; Mon, 21 Oct 2024 21:18:24 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E83CF10E5AA; Mon, 21 Oct 2024 21:18:13 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="n5DVA5dZ"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6603410E0B4; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729545493; x=1761081493; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rgdWjLCAsWDICAk8WcCYB+yo/wtl3ra7r8m7MBpFXLE=; b=n5DVA5dZbkVEWtfIRxulVnvXmNbxnLr2hrUu+PW/aT1V6qRx0SOBxd23 olyziKyNWTD5vkW7x2DbJ+1hrpzN39G0jRjyPHoG/RXrvdcr8pUxBGQdj eKGazCx6ld3cfEwGQsVdICPp99Bs3ud5KOPdY4xruSDgU0T4/ljhQbKOd X3AedaRyMOGj9iyBHMW3gUIPzpmF+29bqMZvsJ+h3n3OfJgRTbyQBGpvn 8oXc8jC6EgTk+Acq/v65ALIL8KkjMq58Vo76noO3+LwAzNNOLSWdCx3BR Iz73yn2mtskp0vtf+rNMIT027YEHHxvH2m0Rpz5W+fLTncpf77nCmYfgR g==; X-CSE-ConnectionGUID: /Nhn1Wr1TFCIuNlXXs6rBQ== X-CSE-MsgGUID: yd2hc8olSZeHa3Q0T5KfTw== X-IronPort-AV: E=McAfee;i="6700,10204,11232"; a="29257686" X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="29257686" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:11 -0700 X-CSE-ConnectionGUID: tqQFife7TDyk8EkrfPPUbw== X-CSE-MsgGUID: GZbL78PPS9uSjvmteDtI1g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="80469598" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v5 7/9] drm/xe: Use ttm_bo_access in xe_vm_snapshot_capture_delayed Date: Mon, 21 Oct 2024 14:18:33 -0700 Message-Id: <20241021211835.1675640-8-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021211835.1675640-1-matthew.brost@intel.com> References: <20241021211835.1675640-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Non-contiguous mapping of BO in VRAM doesn't work, use ttm_bo_access instead. v2: - Fix error handling Fixes: 0eb2a18a8fad ("drm/xe: Implement VM snapshot support for BO's and userptr") Suggested-by: Matthew Auld Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/xe_vm.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index c99380271de6..c8782da3a5c3 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -3303,7 +3303,6 @@ void xe_vm_snapshot_capture_delayed(struct xe_vm_snapshot *snap) for (int i = 0; i < snap->num_snaps; i++) { struct xe_bo *bo = snap->snap[i].bo; - struct iosys_map src; int err; if (IS_ERR(snap->snap[i].data)) @@ -3316,16 +3315,12 @@ void xe_vm_snapshot_capture_delayed(struct xe_vm_snapshot *snap) } if (bo) { - xe_bo_lock(bo, false); - err = ttm_bo_vmap(&bo->ttm, &src); - if (!err) { - xe_map_memcpy_from(xe_bo_device(bo), - snap->snap[i].data, - &src, snap->snap[i].bo_ofs, - snap->snap[i].len); - ttm_bo_vunmap(&bo->ttm, &src); - } - xe_bo_unlock(bo); + err = ttm_bo_access(&bo->ttm, snap->snap[i].bo_ofs, + snap->snap[i].data, snap->snap[i].len, 0); + if (!(err < 0) && err != snap->snap[i].len) + err = -EIO; + else if (!(err < 0)) + err = 0; } else { void __user *userptr = (void __user *)(size_t)snap->snap[i].bo_ofs; From patchwork Mon Oct 21 21:18:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 02FC0D1713F for ; Mon, 21 Oct 2024 21:18:23 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AABBC10E5A7; Mon, 21 Oct 2024 21:18:13 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="EJdIuy4B"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8148C10E119; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729545493; x=1761081493; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nZMJwEdjNoBvlGVgeI7A/wYeq0nHCWPF1Z/N58o2JT8=; b=EJdIuy4B3asCvTA2X6DUS00PFach7Mn9oQCg4PT1EvqozGEyF10BvRbN EwyWTIyFm6DQL69xkWmk5ZTUZCIH1/kkrCIIezcCbD/llBlPycMk4ogJF X/cs4wbbgwEZ1VtIdhs5fO94Pga/6LqQv0tCAUxD1ebLJv8oDDlVZLEuo LOQvQWE1y2/hdOfzJcDJgMGDOi2l//OM9PBv0KFL4SCtux0LU3twEipNU WJDrYLWKasTAURUjaDI00SLTLbdhb3yDcEYuIpYsXbQUyATLr7ReGGW5N BTtuhX9vizczF00syg2DQBSmm6Cz+s83XKnRN+N4AQkep8Iw5y8np82C+ w==; X-CSE-ConnectionGUID: Mxa6Yd0nQAi/LmTT3yEQ8g== X-CSE-MsgGUID: cPVPKNbAQ/WQecnGny+b5w== X-IronPort-AV: E=McAfee;i="6700,10204,11232"; a="29257688" X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="29257688" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:11 -0700 X-CSE-ConnectionGUID: zJ+P3zgNRsGfzTYeIR0NWQ== X-CSE-MsgGUID: LXJvVhDVSr6d30UATVHo2A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="80469603" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v5 8/9] drm/xe: Set XE_BO_FLAG_PINNED in migrate selftest BOs Date: Mon, 21 Oct 2024 14:18:34 -0700 Message-Id: <20241021211835.1675640-9-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021211835.1675640-1-matthew.brost@intel.com> References: <20241021211835.1675640-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We only allow continguous BOs to be vmapped, set XE_BO_FLAG_PINNED on BOs in migrate selftest as this forces continguous BOs and selftest uses vmaps. Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/tests/xe_migrate.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/xe/tests/xe_migrate.c b/drivers/gpu/drm/xe/tests/xe_migrate.c index 1a192a2a941b..4cef3b20bd17 100644 --- a/drivers/gpu/drm/xe/tests/xe_migrate.c +++ b/drivers/gpu/drm/xe/tests/xe_migrate.c @@ -83,7 +83,8 @@ static void test_copy(struct xe_migrate *m, struct xe_bo *bo, bo->size, ttm_bo_type_kernel, region | - XE_BO_FLAG_NEEDS_CPU_ACCESS); + XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(remote)) { KUNIT_FAIL(test, "Failed to allocate remote bo for %s: %pe\n", str, remote); @@ -642,7 +643,9 @@ static void validate_ccs_test_run_tile(struct xe_device *xe, struct xe_tile *til sys_bo = xe_bo_create_user(xe, NULL, NULL, SZ_4M, DRM_XE_GEM_CPU_CACHING_WC, - XE_BO_FLAG_SYSTEM | XE_BO_FLAG_NEEDS_CPU_ACCESS); + XE_BO_FLAG_SYSTEM | + XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(sys_bo)) { KUNIT_FAIL(test, "xe_bo_create() failed with err=%ld\n", @@ -666,7 +669,8 @@ static void validate_ccs_test_run_tile(struct xe_device *xe, struct xe_tile *til ccs_bo = xe_bo_create_user(xe, NULL, NULL, SZ_4M, DRM_XE_GEM_CPU_CACHING_WC, - bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS); + bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(ccs_bo)) { KUNIT_FAIL(test, "xe_bo_create() failed with err=%ld\n", @@ -690,7 +694,8 @@ static void validate_ccs_test_run_tile(struct xe_device *xe, struct xe_tile *til vram_bo = xe_bo_create_user(xe, NULL, NULL, SZ_4M, DRM_XE_GEM_CPU_CACHING_WC, - bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS); + bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(vram_bo)) { KUNIT_FAIL(test, "xe_bo_create() failed with err=%ld\n", PTR_ERR(vram_bo)); From patchwork Mon Oct 21 21:18:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844694 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6DE73D1713F for ; Mon, 21 Oct 2024 21:18:25 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E7F9B10E5B2; Mon, 21 Oct 2024 21:18:17 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="GgAhkswS"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8A3B410E191; Mon, 21 Oct 2024 21:18:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729545493; x=1761081493; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=eSG5vY32/ZdefK/ah9eMHe8q26+p/YDG8IpBok17Zzc=; b=GgAhkswSRA2tKUMVHR2yC+HkBSUvQQflKenli6RDdX70XnxbXV2eFSKi cAyEd28qpC7OgY1Do0jVQTBziZQ1lRmWvAU2aHVTYRZigF9xG+X8LjEOE Iwv9lo2aCTW8pZ1R8dLiKgsSMfI6JyYydP0xYS+FEJMq54Ww1qlQ4GoUE fedSuR+anqmznmjlbQCy7lJjHRwX5mZR3tx2IawlAQ7zTZrMLurRSFHxj ObfBvGTajEXNRyGMF5yCHMjTNkbltNCY9Bogo5lgJ1ahxpRKIK1h/CHH/ ybqbIvyKD/FUAB/J0Lf/w5zhZsXEopHVOm/a47Ym/TN40FmvqrvAKi1U2 Q==; X-CSE-ConnectionGUID: /qsRLJUvQ9mNeUp1GYMWaQ== X-CSE-MsgGUID: o9Vt3JgCSGq70L1YGAf4sQ== X-IronPort-AV: E=McAfee;i="6700,10204,11232"; a="29257689" X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="29257689" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:11 -0700 X-CSE-ConnectionGUID: 38RZgWlmTK+MGInACneSqQ== X-CSE-MsgGUID: 6rX2DUHbQb2/cO9/tmEUgQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="80469606" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 14:18:10 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v5 9/9] drm/xe: Only allow contiguous BOs to use xe_bo_vmap Date: Mon, 21 Oct 2024 14:18:35 -0700 Message-Id: <20241021211835.1675640-10-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021211835.1675640-1-matthew.brost@intel.com> References: <20241021211835.1675640-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" xe_bo_vmap only works on contiguous BOs, disallow xe_bo_vmap on BO unless we are certain the BO is contiguous. Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/xe_bo.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 4e1f54aad5bc..59766369a15b 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -162,6 +162,15 @@ static void try_add_system(struct xe_device *xe, struct xe_bo *bo, } } +static bool force_contiguous(u32 bo_flags) +{ + /* + * For eviction / restore on suspend / resume objects pinned in VRAM + * must be contiguous, also only contiguous BOs support xe_bo_vmap. + */ + return bo_flags & (XE_BO_FLAG_PINNED | XE_BO_FLAG_GGTT); +} + static void add_vram(struct xe_device *xe, struct xe_bo *bo, struct ttm_place *places, u32 bo_flags, u32 mem_type, u32 *c) { @@ -175,12 +184,7 @@ static void add_vram(struct xe_device *xe, struct xe_bo *bo, xe_assert(xe, vram && vram->usable_size); io_size = vram->io_size; - /* - * For eviction / restore on suspend / resume objects - * pinned in VRAM must be contiguous - */ - if (bo_flags & (XE_BO_FLAG_PINNED | - XE_BO_FLAG_GGTT)) + if (force_contiguous(bo_flags)) place.flags |= TTM_PL_FLAG_CONTIGUOUS; if (io_size < vram->usable_size) { @@ -212,8 +216,7 @@ static void try_add_stolen(struct xe_device *xe, struct xe_bo *bo, bo->placements[*c] = (struct ttm_place) { .mem_type = XE_PL_STOLEN, - .flags = bo_flags & (XE_BO_FLAG_PINNED | - XE_BO_FLAG_GGTT) ? + .flags = force_contiguous(bo_flags) ? TTM_PL_FLAG_CONTIGUOUS : 0, }; *c += 1; @@ -2033,13 +2036,15 @@ dma_addr_t xe_bo_addr(struct xe_bo *bo, u64 offset, size_t page_size) int xe_bo_vmap(struct xe_bo *bo) { + struct xe_device *xe = ttm_to_xe_device(bo->ttm.bdev); void *virtual; bool is_iomem; int ret; xe_bo_assert_held(bo); - if (!(bo->flags & XE_BO_FLAG_NEEDS_CPU_ACCESS)) + if (drm_WARN_ON(&xe->drm, !(bo->flags & XE_BO_FLAG_NEEDS_CPU_ACCESS) || + !force_contiguous(bo->flags))) return -EINVAL; if (!iosys_map_is_null(&bo->vmap))