From patchwork Thu Feb 11 13:29:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12083265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19B7EC433E0 for ; Thu, 11 Feb 2021 13:29:47 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C19F564DE0 for ; Thu, 11 Feb 2021 13:29:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C19F564DE0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8E9FF6E0EB; Thu, 11 Feb 2021 13:29:45 +0000 (UTC) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7C9E96E0EB for ; Thu, 11 Feb 2021 13:29:44 +0000 (UTC) Received: by mail-ej1-x62a.google.com with SMTP id i8so10045849ejc.7 for ; Thu, 11 Feb 2021 05:29:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=6kVXaL62eNbd5DKEqcS0gAZae+RIUEySQd7TZO396/4=; b=fp8Cww/poZLvPSoQrQb0Av793kYNMCzcd6sx8uNYrQeLvW5SNSAtk7VZht9IwRosWQ hKz4HhTbfilpVy4HkivFQGxjVopXTjMrTcvPMpqgrldLI5y1A1lrT2In5e48F5olm2wK AIQmAuND567T9xMaaKQO/iWPcSASPZyf2hbRrCsFiJK6Zbsy8IX7uKIRJlLul+BRiPmi 6zZmOSZqVeYC6NI7cI6JLoa9XOxrVpst2ga5fXzeNqgPQax/E6RfMFV3TdAT75TaLrgE jxMdh8/o9w5QNoQDn2f4tQ0ueNarYOR1sZukuqvAzeY+91I03uz/pekhpGog/JNi/DN9 Ir+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=6kVXaL62eNbd5DKEqcS0gAZae+RIUEySQd7TZO396/4=; b=HCobuhnQ14uiDmzRXcL1JSLViwn7lotTU2BOhZm9r8WQxNCVpQf4JxleFgAiWZWsXb 8Nlz7aBkWhsvfRqNWF/ND7Op1byJ0ZlV5mwFrnNpuHeXAWytBnLPpBSgjBS59sAr81PV hiF1M+MKblNdoWAeMpUpQ3nwnDk1mgq6yvTT0IPcbanewkfXECfoAtM6RK6/T3h5PY5B vGSeXr4gRaY1cybRvWNd8Bkxl+OzQRE4iZTsc9+qGWWxWrvElc1OHV43TBbHpejSlipM bTzRyK4Ut1Cc3oGThvPhJPiiLYoATRbVDqMTJmxpmwuWHdHcxsxcR4tHyFuByVhytc07 kPQg== X-Gm-Message-State: AOAM533RbsSiOk+cucPjhAjDs6fomWC79w4c8dV/YYe6i7lH1CO14SyS OjW8zFFnU5HmpzcUZqDYTDuLOfXTM9lLbw== X-Google-Smtp-Source: ABdhPJzYjErSh4w1eGF/cBMekDvjolEDcGNYJq2/8Ah0UtHujHr+6noGDVlsemx4tKoFUJZzMxOydw== X-Received: by 2002:a17:906:5390:: with SMTP id g16mr8592285ejo.19.1613050183175; Thu, 11 Feb 2021 05:29:43 -0800 (PST) Received: from abel.fritz.box ([2a02:908:1252:fb60:820:5af6:9146:2c9f]) by smtp.gmail.com with ESMTPSA id w26sm3862627edq.46.2021.02.11.05.29.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Feb 2021 05:29:42 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: dri-devel@lists.freedesktop.org Subject: [PATCH 1/3] drm/ttm: move swapout logic around Date: Thu, 11 Feb 2021 14:29:39 +0100 Message-Id: <20210211132941.290772-1-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: ray.huang@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Move the iteration of the global lru into the new function ttm_global_swapout() and use that instead in drivers. Signed-off-by: Christian König --- drivers/gpu/drm/ttm/ttm_bo.c | 57 ++++++++--------------------- drivers/gpu/drm/ttm/ttm_device.c | 29 +++++++++++++++ drivers/gpu/drm/ttm/ttm_tt.c | 2 +- drivers/gpu/drm/vmwgfx/ttm_memory.c | 3 +- drivers/gpu/drm/vmwgfx/vmwgfx_drv.c | 2 +- include/drm/ttm/ttm_bo_api.h | 3 +- include/drm/ttm/ttm_device.h | 2 + 7 files changed, 53 insertions(+), 45 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index e38102282fd5..d33578a112b4 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -1184,56 +1184,35 @@ int ttm_bo_wait(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_wait); -/* - * A buffer object shrink method that tries to swap out the first - * buffer object on the bo_global::swap_lru list. - */ -int ttm_bo_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags) +int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, + gfp_t gfp_flags) { struct ttm_global *glob = &ttm_glob; - struct ttm_buffer_object *bo; - int ret = -EBUSY; bool locked; - unsigned i; - - spin_lock(&glob->lru_lock); - for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { - list_for_each_entry(bo, &glob->swap_lru[i], swap) { - if (!ttm_bo_evict_swapout_allowable(bo, ctx, &locked, - NULL)) - continue; - - if (!ttm_bo_get_unless_zero(bo)) { - if (locked) - dma_resv_unlock(bo->base.resv); - continue; - } + int ret; - ret = 0; - break; - } - if (!ret) - break; - } + if (!ttm_bo_evict_swapout_allowable(bo, ctx, &locked, NULL)) + return -EBUSY; - if (ret) { - spin_unlock(&glob->lru_lock); - return ret; + if (!ttm_bo_get_unless_zero(bo)) { + if (locked) + dma_resv_unlock(bo->base.resv); + return -EBUSY; } if (bo->deleted) { - ret = ttm_bo_cleanup_refs(bo, false, false, locked); + ttm_bo_cleanup_refs(bo, false, false, locked); ttm_bo_put(bo); - return ret; + return 0; } ttm_bo_del_from_lru(bo); + /* TODO: Cleanup the locking */ spin_unlock(&glob->lru_lock); - /** + /* * Move to system cached */ - if (bo->mem.mem_type != TTM_PL_SYSTEM) { struct ttm_operation_ctx ctx = { false, false }; struct ttm_resource evict_mem; @@ -1253,29 +1232,26 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags) } } - /** + /* * Make sure BO is idle. */ - ret = ttm_bo_wait(bo, false, false); if (unlikely(ret != 0)) goto out; ttm_bo_unmap_virtual(bo); - /** + /* * Swap out. Buffer will be swapped in again as soon as * anyone tries to access a ttm page. */ - if (bo->bdev->funcs->swap_notify) bo->bdev->funcs->swap_notify(bo); ret = ttm_tt_swapout(bo->bdev, bo->ttm, gfp_flags); out: - /** - * + /* * Unreserve without putting on LRU to avoid swapping out an * already swapped buffer. */ @@ -1284,7 +1260,6 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags) ttm_bo_put(bo); return ret; } -EXPORT_SYMBOL(ttm_bo_swapout); void ttm_bo_tt_destroy(struct ttm_buffer_object *bo) { diff --git a/drivers/gpu/drm/ttm/ttm_device.c b/drivers/gpu/drm/ttm/ttm_device.c index 95e1b7b1f2e6..dfc2a7e4e490 100644 --- a/drivers/gpu/drm/ttm/ttm_device.c +++ b/drivers/gpu/drm/ttm/ttm_device.c @@ -102,6 +102,35 @@ static int ttm_global_init(void) return ret; } +/** + * A buffer object shrink method that tries to swap out the first + * buffer object on the global::swap_lru list. + */ +long ttm_global_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags) +{ + struct ttm_global *glob = &ttm_glob; + struct ttm_buffer_object *bo; + unsigned i; + int ret; + + spin_lock(&glob->lru_lock); + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + list_for_each_entry(bo, &glob->swap_lru[i], swap) { + uint32_t num_pages = bo->ttm->num_pages; + + ret = ttm_bo_swapout(bo, ctx, gfp_flags); + /* ttm_bo_swapout has dropped the lru_lock */ + if (!ret) + return num_pages; + if (ret != -EBUSY) + return ret; + } + } + spin_unlock(&glob->lru_lock); + return 0; +} +EXPORT_SYMBOL(ttm_global_swapout); + static void ttm_init_sysman(struct ttm_device *bdev) { struct ttm_resource_manager *man = &bdev->sysman; diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 2f0833c98d2c..95b5cff25f4c 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -369,7 +369,7 @@ static unsigned long ttm_tt_shrinker_scan(struct shrinker *shrink, }; int ret; - ret = ttm_bo_swapout(&ctx, GFP_NOFS); + ret = ttm_global_swapout(&ctx, GFP_NOFS); return ret < 0 ? SHRINK_EMPTY : ret; } diff --git a/drivers/gpu/drm/vmwgfx/ttm_memory.c b/drivers/gpu/drm/vmwgfx/ttm_memory.c index e972af07d029..104b95a8c7a2 100644 --- a/drivers/gpu/drm/vmwgfx/ttm_memory.c +++ b/drivers/gpu/drm/vmwgfx/ttm_memory.c @@ -38,6 +38,7 @@ #include #include +#include #include "ttm_memory.h" @@ -277,7 +278,7 @@ static void ttm_shrink(struct ttm_mem_global *glob, bool from_wq, while (ttm_zones_above_swap_target(glob, from_wq, extra)) { spin_unlock(&glob->lock); - ret = ttm_bo_swapout(ctx, GFP_KERNEL); + ret = ttm_global_swapout(ctx, GFP_KERNEL); spin_lock(&glob->lock); if (unlikely(ret < 0)) break; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c index 4efed3bf0ef9..01da355b86f3 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c @@ -1384,7 +1384,7 @@ static int vmw_pm_freeze(struct device *kdev) vmw_execbuf_release_pinned_bo(dev_priv); vmw_resource_evict_all(dev_priv); vmw_release_device_early(dev_priv); - while (ttm_bo_swapout(&ctx, GFP_KERNEL) > 0); + while (ttm_global_swapout(&ctx, GFP_KERNEL) > 0); if (dev_priv->enable_fb) vmw_fifo_resource_dec(dev_priv); if (atomic_read(&dev_priv->num_fifo_resources) != 0) { diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 4fb523dfab32..5044ac330858 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -560,7 +560,8 @@ ssize_t ttm_bo_io(struct ttm_device *bdev, struct file *filp, const char __user *wbuf, char __user *rbuf, size_t count, loff_t *f_pos, bool write); -int ttm_bo_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags); +int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, + gfp_t gfp_flags); /** * ttm_bo_uses_embedded_gem_object - check if the given bo uses the diff --git a/include/drm/ttm/ttm_device.h b/include/drm/ttm/ttm_device.h index 035bbc044a3b..6a0b267d4fe6 100644 --- a/include/drm/ttm/ttm_device.h +++ b/include/drm/ttm/ttm_device.h @@ -297,6 +297,8 @@ struct ttm_device { struct delayed_work wq; }; +long ttm_global_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags); + static inline struct ttm_resource_manager * ttm_manager_type(struct ttm_device *bdev, int mem_type) {