From patchwork Fri Sep 6 08:52:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Zimmermann X-Patchwork-Id: 11134807 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D066A14ED for ; Fri, 6 Sep 2019 08:52:27 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BA56120842 for ; Fri, 6 Sep 2019 08:52:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA56120842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4BE2C6E1F5; Fri, 6 Sep 2019 08:52:24 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by gabe.freedesktop.org (Postfix) with ESMTPS id D7DBB6E1F7 for ; Fri, 6 Sep 2019 08:52:19 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id D0199AEE0; Fri, 6 Sep 2019 08:52:17 +0000 (UTC) From: Thomas Zimmermann To: daniel@ffwll.ch, noralf@tronnes.org, airlied@linux.ie, rong.a.chen@intel.com, feng.tang@intel.com, ying.huang@intel.com, sean@poorly.run, maxime.ripard@bootlin.com, maarten.lankhorst@linux.intel.com, dave@stgolabs.net, kraxel@redhat.com Subject: [PATCH v3 1/3] drm/vram: Add kmap ref-counting to GEM VRAM objects Date: Fri, 6 Sep 2019 10:52:12 +0200 Message-Id: <20190906085214.11677-2-tzimmermann@suse.de> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190906085214.11677-1-tzimmermann@suse.de> References: <20190906085214.11677-1-tzimmermann@suse.de> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Zimmermann , dri-devel@lists.freedesktop.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The kmap and kunmap operations of GEM VRAM buffers can now be called in interleaving pairs. The first call to drm_gem_vram_kmap() maps the buffer's memory to kernel address space and the final call to drm_gem_vram_kunmap() unmaps the memory. Intermediate calls to these functions increment or decrement a reference counter. This change allows for keeping buffer memory mapped for longer and minimizes the amount of changes to TLB, page tables, etc. Signed-off-by: Thomas Zimmermann Cc: Davidlohr Bueso Reviewed-by: Gerd Hoffmann --- drivers/gpu/drm/drm_gem_vram_helper.c | 74 ++++++++++++++++++++------- include/drm/drm_gem_vram_helper.h | 19 +++++++ 2 files changed, 75 insertions(+), 18 deletions(-) diff --git a/drivers/gpu/drm/drm_gem_vram_helper.c b/drivers/gpu/drm/drm_gem_vram_helper.c index fd751078bae1..6c7912092876 100644 --- a/drivers/gpu/drm/drm_gem_vram_helper.c +++ b/drivers/gpu/drm/drm_gem_vram_helper.c @@ -26,7 +26,11 @@ static void drm_gem_vram_cleanup(struct drm_gem_vram_object *gbo) * TTM buffer object in 'bo' has already been cleaned * up; only release the GEM object. */ + + WARN_ON(gbo->kmap_use_count); + drm_gem_object_release(&gbo->bo.base); + mutex_destroy(&gbo->kmap_lock); } static void drm_gem_vram_destroy(struct drm_gem_vram_object *gbo) @@ -100,6 +104,8 @@ static int drm_gem_vram_init(struct drm_device *dev, if (ret) goto err_drm_gem_object_release; + mutex_init(&gbo->kmap_lock); + return 0; err_drm_gem_object_release: @@ -283,6 +289,34 @@ int drm_gem_vram_unpin(struct drm_gem_vram_object *gbo) } EXPORT_SYMBOL(drm_gem_vram_unpin); +static void *drm_gem_vram_kmap_locked(struct drm_gem_vram_object *gbo, + bool map, bool *is_iomem) +{ + int ret; + struct ttm_bo_kmap_obj *kmap = &gbo->kmap; + + if (gbo->kmap_use_count > 0) + goto out; + + if (kmap->virtual || !map) + goto out; + + ret = ttm_bo_kmap(&gbo->bo, 0, gbo->bo.num_pages, kmap); + if (ret) + return ERR_PTR(ret); + +out: + if (!kmap->virtual) { + if (is_iomem) + *is_iomem = false; + return NULL; /* not mapped; don't increment ref */ + } + ++gbo->kmap_use_count; + if (is_iomem) + return ttm_kmap_obj_virtual(kmap, is_iomem); + return kmap->virtual; +} + /** * drm_gem_vram_kmap() - Maps a GEM VRAM object into kernel address space * @gbo: the GEM VRAM object @@ -304,40 +338,44 @@ void *drm_gem_vram_kmap(struct drm_gem_vram_object *gbo, bool map, bool *is_iomem) { int ret; - struct ttm_bo_kmap_obj *kmap = &gbo->kmap; - - if (kmap->virtual || !map) - goto out; + void *virtual; - ret = ttm_bo_kmap(&gbo->bo, 0, gbo->bo.num_pages, kmap); + ret = mutex_lock_interruptible(&gbo->kmap_lock); if (ret) return ERR_PTR(ret); + virtual = drm_gem_vram_kmap_locked(gbo, map, is_iomem); + mutex_unlock(&gbo->kmap_lock); -out: - if (!is_iomem) - return kmap->virtual; - if (!kmap->virtual) { - *is_iomem = false; - return NULL; - } - return ttm_kmap_obj_virtual(kmap, is_iomem); + return virtual; } EXPORT_SYMBOL(drm_gem_vram_kmap); -/** - * drm_gem_vram_kunmap() - Unmaps a GEM VRAM object - * @gbo: the GEM VRAM object - */ -void drm_gem_vram_kunmap(struct drm_gem_vram_object *gbo) +static void drm_gem_vram_kunmap_locked(struct drm_gem_vram_object *gbo) { struct ttm_bo_kmap_obj *kmap = &gbo->kmap; + if (WARN_ON_ONCE(!gbo->kmap_use_count)) + return; + if (--gbo->kmap_use_count > 0) + return; + if (!kmap->virtual) return; ttm_bo_kunmap(kmap); kmap->virtual = NULL; } + +/** + * drm_gem_vram_kunmap() - Unmaps a GEM VRAM object + * @gbo: the GEM VRAM object + */ +void drm_gem_vram_kunmap(struct drm_gem_vram_object *gbo) +{ + mutex_lock(&gbo->kmap_lock); + drm_gem_vram_kunmap_locked(gbo); + mutex_unlock(&gbo->kmap_lock); +} EXPORT_SYMBOL(drm_gem_vram_kunmap); /** diff --git a/include/drm/drm_gem_vram_helper.h b/include/drm/drm_gem_vram_helper.h index ac217d768456..8c08bc87b788 100644 --- a/include/drm/drm_gem_vram_helper.h +++ b/include/drm/drm_gem_vram_helper.h @@ -34,11 +34,30 @@ struct vm_area_struct; * backed by VRAM. It can be used for simple framebuffer devices with * dedicated memory. The buffer object can be evicted to system memory if * video memory becomes scarce. + * + * GEM VRAM objects perform reference counting for pin and mapping + * operations. So a buffer object that has been pinned N times with + * drm_gem_vram_pin() must be unpinned N times with + * drm_gem_vram_unpin(). The same applies to pairs of + * drm_gem_vram_kmap() and drm_gem_vram_kunmap(). */ struct drm_gem_vram_object { struct ttm_buffer_object bo; struct ttm_bo_kmap_obj kmap; + /** + * @kmap_lock: Protects the kmap address and use count + */ + struct mutex kmap_lock; + + /** + * @kmap_use_count: + * + * Reference count on the virtual address. + * The address are un-mapped when the count reaches zero. + */ + unsigned int kmap_use_count; + /* Supported placements are %TTM_PL_VRAM and %TTM_PL_SYSTEM */ struct ttm_placement placement; struct ttm_place placements[2]; From patchwork Fri Sep 6 08:52:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Zimmermann X-Patchwork-Id: 11134803 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF67614ED for ; Fri, 6 Sep 2019 08:52:23 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C847E20578 for ; Fri, 6 Sep 2019 08:52:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C847E20578 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 82D8C6E1B2; Fri, 6 Sep 2019 08:52:21 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9DD1A6E1B2 for ; Fri, 6 Sep 2019 08:52:19 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id D2CE4AF3F; Fri, 6 Sep 2019 08:52:17 +0000 (UTC) From: Thomas Zimmermann To: daniel@ffwll.ch, noralf@tronnes.org, airlied@linux.ie, rong.a.chen@intel.com, feng.tang@intel.com, ying.huang@intel.com, sean@poorly.run, maxime.ripard@bootlin.com, maarten.lankhorst@linux.intel.com, dave@stgolabs.net, kraxel@redhat.com Subject: [PATCH v3 2/3] drm/vram: Add infrastructure for move_notify() Date: Fri, 6 Sep 2019 10:52:13 +0200 Message-Id: <20190906085214.11677-3-tzimmermann@suse.de> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190906085214.11677-1-tzimmermann@suse.de> References: <20190906085214.11677-1-tzimmermann@suse.de> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Zimmermann , dri-devel@lists.freedesktop.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This patch prepares VRAM helpers for lazy unmapping of buffer objects. Signed-off-by: Thomas Zimmermann --- drivers/gpu/drm/drm_vram_mm_helper.c | 12 ++++++++++++ include/drm/drm_vram_mm_helper.h | 4 ++++ 2 files changed, 16 insertions(+) diff --git a/drivers/gpu/drm/drm_vram_mm_helper.c b/drivers/gpu/drm/drm_vram_mm_helper.c index c911781d6728..31984690d5f3 100644 --- a/drivers/gpu/drm/drm_vram_mm_helper.c +++ b/drivers/gpu/drm/drm_vram_mm_helper.c @@ -98,6 +98,17 @@ static int bo_driver_verify_access(struct ttm_buffer_object *bo, return vmm->funcs->verify_access(bo, filp); } +static void bo_driver_move_notify(struct ttm_buffer_object *bo, + bool evict, + struct ttm_mem_reg *new_mem) +{ + struct drm_vram_mm *vmm = drm_vram_mm_of_bdev(bo->bdev); + + if (!vmm->funcs || !vmm->funcs->move_notify) + return; + vmm->funcs->move_notify(bo, evict, new_mem); +} + static int bo_driver_io_mem_reserve(struct ttm_bo_device *bdev, struct ttm_mem_reg *mem) { @@ -140,6 +151,7 @@ static struct ttm_bo_driver bo_driver = { .eviction_valuable = ttm_bo_eviction_valuable, .evict_flags = bo_driver_evict_flags, .verify_access = bo_driver_verify_access, + .move_notify = bo_driver_move_notify, .io_mem_reserve = bo_driver_io_mem_reserve, .io_mem_free = bo_driver_io_mem_free, }; diff --git a/include/drm/drm_vram_mm_helper.h b/include/drm/drm_vram_mm_helper.h index 2aacfb1ccfae..7fb8700f45fe 100644 --- a/include/drm/drm_vram_mm_helper.h +++ b/include/drm/drm_vram_mm_helper.h @@ -15,6 +15,8 @@ struct drm_device; &ttm_bo_driver.evict_flags * @verify_access: Provides an implementation for \ struct &ttm_bo_driver.verify_access + * @move_notify: Provides an implementation for + * struct &ttm_bo_driver.move_notify * * These callback function integrate VRAM MM with TTM buffer objects. New * functions can be added if necessary. @@ -23,6 +25,8 @@ struct drm_vram_mm_funcs { void (*evict_flags)(struct ttm_buffer_object *bo, struct ttm_placement *placement); int (*verify_access)(struct ttm_buffer_object *bo, struct file *filp); + void (*move_notify)(struct ttm_buffer_object *bo, bool evict, + struct ttm_mem_reg *new_mem); }; /** From patchwork Fri Sep 6 08:52:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Zimmermann X-Patchwork-Id: 11134805 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 132991398 for ; Fri, 6 Sep 2019 08:52:26 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F147620578 for ; Fri, 6 Sep 2019 08:52:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F147620578 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B385C6E1F3; Fri, 6 Sep 2019 08:52:22 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by gabe.freedesktop.org (Postfix) with ESMTPS id ADD9F6E1F3 for ; Fri, 6 Sep 2019 08:52:19 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id D0FB5AF39; Fri, 6 Sep 2019 08:52:17 +0000 (UTC) From: Thomas Zimmermann To: daniel@ffwll.ch, noralf@tronnes.org, airlied@linux.ie, rong.a.chen@intel.com, feng.tang@intel.com, ying.huang@intel.com, sean@poorly.run, maxime.ripard@bootlin.com, maarten.lankhorst@linux.intel.com, dave@stgolabs.net, kraxel@redhat.com Subject: [PATCH v3 3/3] drm/vram: Implement lazy unmapping for GEM VRAM buffers Date: Fri, 6 Sep 2019 10:52:14 +0200 Message-Id: <20190906085214.11677-4-tzimmermann@suse.de> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190906085214.11677-1-tzimmermann@suse.de> References: <20190906085214.11677-1-tzimmermann@suse.de> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Zimmermann , dri-devel@lists.freedesktop.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Frequent mapping and unmapping a buffer object adds overhead for modifying the page table and creates debug output. Unmapping a buffer is only required when the memory manager evicts the buffer from its current location. Signed-off-by: Thomas Zimmermann --- drivers/gpu/drm/drm_gem_vram_helper.c | 48 ++++++++++++++++++++++----- include/drm/drm_gem_vram_helper.h | 4 +++ 2 files changed, 44 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/drm_gem_vram_helper.c b/drivers/gpu/drm/drm_gem_vram_helper.c index 6c7912092876..973c703534d4 100644 --- a/drivers/gpu/drm/drm_gem_vram_helper.c +++ b/drivers/gpu/drm/drm_gem_vram_helper.c @@ -352,18 +352,17 @@ EXPORT_SYMBOL(drm_gem_vram_kmap); static void drm_gem_vram_kunmap_locked(struct drm_gem_vram_object *gbo) { - struct ttm_bo_kmap_obj *kmap = &gbo->kmap; - if (WARN_ON_ONCE(!gbo->kmap_use_count)) return; if (--gbo->kmap_use_count > 0) return; - if (!kmap->virtual) - return; - - ttm_bo_kunmap(kmap); - kmap->virtual = NULL; + /* + * Permanently mapping and unmapping buffers adds overhead from + * updating the page tables and creates debugging output. Therefore, + * we delay the actual unmap operation until the BO gets evicted + * from memory. See drm_gem_vram_bo_driver_move_notify(). + */ } /** @@ -489,6 +488,38 @@ int drm_gem_vram_bo_driver_verify_access(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(drm_gem_vram_bo_driver_verify_access); +/** + * drm_gem_vram_bo_driver_move_notify() - + * Implements &struct ttm_bo_driver.move_notify + * @bo: TTM buffer object. Refers to &struct drm_gem_vram_object.bo + * @evict: True, if the BO is being evicted from graphics memory; + * false otherwise. + * @new_mem: New memory region, or NULL on destruction + */ +void drm_gem_vram_bo_driver_move_notify(struct ttm_buffer_object *bo, + bool evict, + struct ttm_mem_reg *new_mem) +{ + struct drm_gem_vram_object *gbo; + struct ttm_bo_kmap_obj *kmap; + + /* TTM may pass BOs that are not GEM VRAM BOs. */ + if (!drm_is_gem_vram(bo)) + return; + + gbo = drm_gem_vram_of_bo(bo); + kmap = &gbo->kmap; + + if (WARN_ON_ONCE(gbo->kmap_use_count)) + return; + + if (!kmap->virtual) + return; + ttm_bo_kunmap(kmap); + kmap->virtual = NULL; +} +EXPORT_SYMBOL(drm_gem_vram_bo_driver_move_notify); + /* * drm_gem_vram_mm_funcs - Functions for &struct drm_vram_mm * @@ -498,7 +529,8 @@ EXPORT_SYMBOL(drm_gem_vram_bo_driver_verify_access); */ const struct drm_vram_mm_funcs drm_gem_vram_mm_funcs = { .evict_flags = drm_gem_vram_bo_driver_evict_flags, - .verify_access = drm_gem_vram_bo_driver_verify_access + .verify_access = drm_gem_vram_bo_driver_verify_access, + .move_notify = drm_gem_vram_bo_driver_move_notify, }; EXPORT_SYMBOL(drm_gem_vram_mm_funcs); diff --git a/include/drm/drm_gem_vram_helper.h b/include/drm/drm_gem_vram_helper.h index 8c08bc87b788..e5ef0e4ab2e4 100644 --- a/include/drm/drm_gem_vram_helper.h +++ b/include/drm/drm_gem_vram_helper.h @@ -117,6 +117,10 @@ int drm_gem_vram_fill_create_dumb(struct drm_file *file, void drm_gem_vram_bo_driver_evict_flags(struct ttm_buffer_object *bo, struct ttm_placement *pl); +void drm_gem_vram_bo_driver_move_notify(struct ttm_buffer_object *bo, + bool evict, + struct ttm_mem_reg *new_mem); + int drm_gem_vram_bo_driver_verify_access(struct ttm_buffer_object *bo, struct file *filp);