From patchwork Tue Nov 17 14:06:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 11912365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.3 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05348C64E7C for ; Tue, 17 Nov 2020 14:06:22 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8D8C12083E for ; Tue, 17 Nov 2020 14:06:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AF4TrxYN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D8C12083E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B130E6E223; Tue, 17 Nov 2020 14:06:20 +0000 (UTC) Received: from mail-ed1-x541.google.com (mail-ed1-x541.google.com [IPv6:2a00:1450:4864:20::541]) by gabe.freedesktop.org (Postfix) with ESMTPS id 017FA6E223 for ; Tue, 17 Nov 2020 14:06:18 +0000 (UTC) Received: by mail-ed1-x541.google.com with SMTP id t11so22512477edj.13 for ; Tue, 17 Nov 2020 06:06:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=reZZShoJvbxFnzqezOjKxAnSYmnXe+sIgfR7XEAfjpA=; b=AF4TrxYNKQeyCU6GkrTf+TkHnw8WXBsk7M03TbQ1bds/xq4EDDoOQlXbyG1PzlCuZg pVKMb9i+ABYVOPNk85Wj6plyvQdjij19WfpE/bvdtO1EvxynlDo2xyEV9mSCyubCoZ+U YhxltLXMKM7pY6veujgLmiCtoZ2pKWrMV1XqJJe00l9NNsGxpgmqIjW8SeZ8Xk2sABpr ci7TmlB95Gs5MuB3o+IlReLHAVsJ9TsSnTVw4zw12gzjCTk2WfCmXkvCmi3rsHMJfd37 PXswz92sTvEIfRuf8tJiwuIAcUl/X39Bik8VgpnHWhC5SafmbCpHOI/2OeKypol4ZO7h 4/Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=reZZShoJvbxFnzqezOjKxAnSYmnXe+sIgfR7XEAfjpA=; b=AfJDGqPYg9ZEqSl/SG/WIOqHKiCQd477iefkzeKWODxOrYdAJFvTzgjhAdFgxC+R8E I63QbYtIFT7rz+0lxUeS99B4RNVOHbTNl5RbQswp1m2fzv4K90mNtRy5TPAz5nIE+56n 4OhdWVJsz2/1lkYgAY6lJbLvETTLnr5V1Z4V+rymABJ0uKTLyRNYVJ2+WEX+lazN/+ij hAFf/dNIKPuUHZvx7diBAswgPDXrLSOErYO+97UaDvluYOWq+yU+x5CooVfUI6EuO6Tg o+9tFIxVrLhs9cADba6HUfM4pFONAtcFgXLXGtJnDAQ11Eudm08nFjJfbcJ1r9JLZ1TK Yhaw== X-Gm-Message-State: AOAM530+AvkwUVsGWceSBp+iFsVMVDoPzXkp9EPYzYD3+igv8lV+HPSs SuKd6IrQ+NBGBM8rOV5Q158= X-Google-Smtp-Source: ABdhPJzz4obCsuBWie5wj7xNOAe9TPlmyX58h9gh4gBnclQkrfUcwVnpVxQVi3jFstJmYJOul7s1ag== X-Received: by 2002:aa7:c4c2:: with SMTP id p2mr21729599edr.371.1605621977592; Tue, 17 Nov 2020 06:06:17 -0800 (PST) Received: from abel.fritz.box ([2a02:908:1252:fb60:4d18:8ca3:78f:2e63]) by smtp.gmail.com with ESMTPSA id h23sm11998900edv.69.2020.11.17.06.06.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Nov 2020 06:06:17 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: airlied@linux.ie, Felix.Kuehling@amd.com, daniel@ffwll.ch, dri-devel@lists.freedesktop.org, linux-graphics-maintainer@vmware.com, sroland@vmware.com Subject: [PATCH 1/3] drm/ttm: rework ttm_tt page limit Date: Tue, 17 Nov 2020 15:06:13 +0100 Message-Id: <20201117140615.255887-1-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" TTM implements a rather extensive accounting of allocated memory. There are two reasons for this: 1. It tries to block userspace allocating a huge number of very small BOs without accounting for the kmalloced memory. 2. Make sure we don't over allocate and run into an OOM situation during swapout while trying to handle the memory shortage. This is only partially a good idea. First of all it is perfectly valid for an application to use all of system memory, limiting it to 50% is not really acceptable. What we need to take care of is that the application is held accountable for the memory it allocated. This is what control mechanisms like memcg and the normal Linux page accounting already do. Making sure that we don't run into an OOM situation while trying to cope with a memory shortage is still a good idea, but this is also not very well implemented since it means another opportunity of recursion from the driver back into TTM. So start to rework all of this by accounting the memory before allocating the pages and making sure that we never go over a certain limit pages locked by TTM in the ttm_tt objects. Signed-off-by: Christian König Reported-by: kernel test robot Reported-by: kernel test robot --- drivers/gpu/drm/ttm/ttm_memory.c | 17 ++++++++- drivers/gpu/drm/ttm/ttm_tt.c | 64 +++++++++++++++++++++++++++++--- include/drm/ttm/ttm_tt.h | 2 + 3 files changed, 77 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_memory.c b/drivers/gpu/drm/ttm/ttm_memory.c index 5ed1fc8f2ace..c53136048630 100644 --- a/drivers/gpu/drm/ttm/ttm_memory.c +++ b/drivers/gpu/drm/ttm/ttm_memory.c @@ -38,6 +38,7 @@ #include #include #include +#include #define TTM_MEMORY_ALLOC_RETRIES 4 @@ -414,10 +415,11 @@ static int ttm_mem_init_dma32_zone(struct ttm_mem_global *glob, int ttm_mem_global_init(struct ttm_mem_global *glob) { + uint64_t num_pages, num_dma32_pages; + struct ttm_mem_zone *zone; struct sysinfo si; int ret; int i; - struct ttm_mem_zone *zone; spin_lock_init(&glob->lock); glob->swap_queue = create_singlethread_workqueue("ttm_swap"); @@ -452,6 +454,19 @@ int ttm_mem_global_init(struct ttm_mem_global *glob) zone->name, (unsigned long long)zone->max_mem >> 10); } ttm_pool_mgr_init(glob->zone_kernel->max_mem/(2*PAGE_SIZE)); + + /* We generally allow TTM to allocate a maximum of 50% of + * the available system memory. + */ + num_pages = (u64)si.totalram * si.mem_unit; + num_pages = (num_pages * 50 / 100) >> PAGE_SHIFT; + + /* But for DMA32 we limit ourself to only use 2GiB maximum. */ + num_dma32_pages = (u64)(si.totalram - si.totalhigh) * si.mem_unit; + num_dma32_pages = min(num_dma32_pages, 2ULL << 30) >> PAGE_SHIFT; + + ttm_tt_mgr_init(num_pages, num_dma32_pages); + return 0; out_no_zone: ttm_mem_global_release(glob); diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index cfd633c7e764..c7b71c9861a6 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -38,6 +38,19 @@ #include #include +static unsigned long ttm_pages_limit; + +MODULE_PARM_DESC(pages_limit, "Limit for the allocated pages"); +module_param_named(pages_limit, ttm_pages_limit, ulong, 0644); + +static unsigned long ttm_dma32_pages_limit; + +MODULE_PARM_DESC(dma32_pages_limit, "Limit for the allocated DMA32 pages"); +module_param_named(dma32_pages_limit, ttm_dma32_pages_limit, ulong, 0644); + +static atomic_long_t ttm_pages_allocated; +static atomic_long_t ttm_dma32_pages_allocated; + /** * Allocates a ttm structure for the given BO. */ @@ -295,8 +308,8 @@ static void ttm_tt_add_mapping(struct ttm_bo_device *bdev, struct ttm_tt *ttm) ttm->pages[i]->mapping = bdev->dev_mapping; } -int ttm_tt_populate(struct ttm_bo_device *bdev, - struct ttm_tt *ttm, struct ttm_operation_ctx *ctx) +int ttm_tt_populate(struct ttm_bo_device *bdev, struct ttm_tt *ttm, + struct ttm_operation_ctx *ctx) { int ret; @@ -306,12 +319,25 @@ int ttm_tt_populate(struct ttm_bo_device *bdev, if (ttm_tt_is_populated(ttm)) return 0; + atomic_long_add(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_add(ttm->num_pages, &ttm_dma32_pages_allocated); + + while (atomic_long_read(&ttm_pages_allocated) > ttm_pages_limit || + atomic_long_read(&ttm_dma32_pages_allocated) > + ttm_dma32_pages_limit) { + + ret = ttm_bo_swapout(ctx); + if (ret) + goto error; + } + if (bdev->driver->ttm_tt_populate) ret = bdev->driver->ttm_tt_populate(bdev, ttm, ctx); else ret = ttm_pool_alloc(&bdev->pool, ttm, ctx); if (ret) - return ret; + goto error; ttm_tt_add_mapping(bdev, ttm); ttm->page_flags |= TTM_PAGE_FLAG_PRIV_POPULATED; @@ -324,6 +350,12 @@ int ttm_tt_populate(struct ttm_bo_device *bdev, } return 0; + +error: + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); + return ret; } EXPORT_SYMBOL(ttm_tt_populate); @@ -341,8 +373,7 @@ static void ttm_tt_clear_mapping(struct ttm_tt *ttm) } } -void ttm_tt_unpopulate(struct ttm_bo_device *bdev, - struct ttm_tt *ttm) +void ttm_tt_unpopulate(struct ttm_bo_device *bdev, struct ttm_tt *ttm) { if (!ttm_tt_is_populated(ttm)) return; @@ -352,5 +383,28 @@ void ttm_tt_unpopulate(struct ttm_bo_device *bdev, bdev->driver->ttm_tt_unpopulate(bdev, ttm); else ttm_pool_free(&bdev->pool, ttm); + + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); + ttm->page_flags &= ~TTM_PAGE_FLAG_PRIV_POPULATED; } + +/** + * ttm_tt_mgr_init - set defaults for the number of pages + * + * @num_pages: global limit of system pages to use + * @num_dma32_pages: global limit of DMA32 pages to use + * + * Set reasonable defaults for the number of allocated pages. + */ +void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages) +{ + if (!ttm_pages_limit) + ttm_pages_limit = num_pages; + + if (!ttm_dma32_pages_limit) + ttm_dma32_pages_limit = num_dma32_pages; + +} diff --git a/include/drm/ttm/ttm_tt.h b/include/drm/ttm/ttm_tt.h index da27e9d8fa64..05ca43f13884 100644 --- a/include/drm/ttm/ttm_tt.h +++ b/include/drm/ttm/ttm_tt.h @@ -157,6 +157,8 @@ int ttm_tt_populate(struct ttm_bo_device *bdev, struct ttm_tt *ttm, struct ttm_o */ void ttm_tt_unpopulate(struct ttm_bo_device *bdev, struct ttm_tt *ttm); +void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages); + #if IS_ENABLED(CONFIG_AGP) #include From patchwork Tue Nov 17 14:06:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 11912363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.3 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78F43C64E69 for ; Tue, 17 Nov 2020 14:06:25 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1BAC120729 for ; Tue, 17 Nov 2020 14:06:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iMj3VJTF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1BAC120729 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 468766E233; Tue, 17 Nov 2020 14:06:24 +0000 (UTC) Received: from mail-ej1-x642.google.com (mail-ej1-x642.google.com [IPv6:2a00:1450:4864:20::642]) by gabe.freedesktop.org (Postfix) with ESMTPS id C51846E233 for ; Tue, 17 Nov 2020 14:06:20 +0000 (UTC) Received: by mail-ej1-x642.google.com with SMTP id y17so23929009ejh.11 for ; Tue, 17 Nov 2020 06:06:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=oFyHoiV2d00mo5V49sT/52n1Qfw4rgsyOigOvwy2QxY=; b=iMj3VJTFLfOrPxcBH52cWhxLXPMM0y1PB6j6dPHRHemhFSO26WeZPXerYVhC0p6fzA bJdzTool0Gy8QrHSxNrnFBy3X/DPonbsiGOP3o7Njjaf4r4nBPgcchyOBRIE701ASbwB bgxNDRbL48ONoopDjFSAWTpaj6bPGIyL3V9EMslZx7vfCXXNtJeJMvHu0jO4Q188WhYZ QpcBsnPknIoKE9EKkcPDgs0VVYmBqlZwqVeHd89QaI25Fm14rK5CdpKicMtLuRvkeQJR dqT1YxjmVrTXtBUiZTsCzEAK3k2nSCFHPDN6ej8s2aF3k/MMUjYAi4XSrPH2LxZx5qm2 dx5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oFyHoiV2d00mo5V49sT/52n1Qfw4rgsyOigOvwy2QxY=; b=KVkvU18thBRBIrlq7FUYtbxUcOg+K6RtL8EkKQ1V27Mb0zohm5eWlrfV95DhKYy+Gi CaFdl5xW1p/Ot9b3LwII+O0/6QM3UvMJ2a4xu+F/Mn7vWJZ/izNZuLvA1W4m/I3HMLko d+j9lY0BUbMUPDQJl5GyDUGf4iuHS9lm62GtwUuM2Ra2gSt4cZtSXg/UUHm4tLRrVy9c rLubQlFK7wKez0or5quuaSbyIlhCfEg5ktD1rHZTCz1MRx3RynID/9thuC2upknSdmKG a0rsVpEKXOg6KZWW4fSmuKjZsjCfzQfXM3yMRwCD8EuKntZFdoQVT+BCwdNnlx9It6Hg zomQ== X-Gm-Message-State: AOAM533dXpnt0kLminxMXT2coBEG/T7btIi6gcDeXYSADKX7zuFieRjU FvVKzyujYYPj7B33SFbvypw= X-Google-Smtp-Source: ABdhPJxT0aRt7dNz8VGn0baSCWvCmgUQdQ6JZuKgQlCHV7JFilZmBkSJuG3nrtbiHDgoGDpTzQg5Vw== X-Received: by 2002:a17:906:1497:: with SMTP id x23mr20603707ejc.457.1605621979465; Tue, 17 Nov 2020 06:06:19 -0800 (PST) Received: from abel.fritz.box ([2a02:908:1252:fb60:4d18:8ca3:78f:2e63]) by smtp.gmail.com with ESMTPSA id h23sm11998900edv.69.2020.11.17.06.06.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Nov 2020 06:06:18 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: airlied@linux.ie, Felix.Kuehling@amd.com, daniel@ffwll.ch, dri-devel@lists.freedesktop.org, linux-graphics-maintainer@vmware.com, sroland@vmware.com Subject: [PATCH 3/3] drm/ttm: make up to 90% of system memory available Date: Tue, 17 Nov 2020 15:06:15 +0100 Message-Id: <20201117140615.255887-3-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201117140615.255887-1-christian.koenig@amd.com> References: <20201117140615.255887-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Increase the ammount of system memory drivers can use to about 90% of the total available. Signed-off-by: Christian König --- drivers/gpu/drm/ttm/ttm_bo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index a958135cb3fe..0a93df93dba4 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -1267,7 +1267,7 @@ static int ttm_bo_global_init(void) * the available system memory. */ num_pages = (u64)si.totalram * si.mem_unit; - num_pages = (num_pages * 50 / 100) >> PAGE_SHIFT; + num_pages = (num_pages * 90 / 100) >> PAGE_SHIFT; /* But for DMA32 we limit ourself to only use 2GiB maximum. */ num_dma32_pages = (u64)(si.totalram - si.totalhigh) * si.mem_unit;