From patchwork Wed Jan 26 22:06:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paneer Selvam, Arunpravin" X-Patchwork-Id: 12725864 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1FEDCC433EF for ; Wed, 26 Jan 2022 22:07:41 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1290D10E560; Wed, 26 Jan 2022 22:07:38 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2072.outbound.protection.outlook.com [40.107.244.72]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7A31210E1BE; Wed, 26 Jan 2022 22:07:36 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XeT8Hma3KeAi95BdAwWmif8Vlmn3co3PtNDHbn0gb4SGex8w5WBk/fgsNB4Lt9/ry7ee3J8z6V5iew0ZWJxHoT+O0BXeJvq6lsmfoqZ8JvWgWbBEn2fTvma81QCV4YzCSHRV9IuzYurHkbNclllGjrtopXJbcW4zyno0YpwmLWFNWeOYKYxhYfq7PQUvtzzA2TFiBHM6n9VQ7J0oOS9Mz9z1jCUw4sXLD3nV3/ENq3rGknViawnFpohGvPq6S7g599i4rCfoyno6SMhOsQBf910C8tfx5xWxKajWEsPVLGl5+uIZLcT+05Z7xhOEG99jzZPBDmSQm9xWD1QCgAw19A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/dD8dM/Ts6xVU/NIObFR3mBbb1c8OJvMLBdGnmxUe9A=; b=lZKYzTprVVoeMars10Nv047GK7/UJ/eoitzd4O4NConibMIQ4Xjn2xwo48Rn4jrFu7ZOALVlLSvHca38ggN2rFiDj5y8Z8lfB1DE/B3YrGK219lb5ecaJzCx7HJNsWayQcCaY7WSwd8dyAHKg9c1Afgb1JAUEvWosAloaiEf4UVM26VCOZlfVk9TJC5FfJvSXLxWtW9Y0hKwQIejJ7/+KSn83Mt5zUG9bHGCWC2cHiVhEdDUGps4S3tqnTeBkJ6c0k7JHMy2Qf7z+/Mi1I/OQsR5VFknn5JYIa1Z9kh1TB/VBdhN0yS5vMNTrK8W/b1hS1o8vKM+mtZtagKQaXs8sg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/dD8dM/Ts6xVU/NIObFR3mBbb1c8OJvMLBdGnmxUe9A=; b=hLgcQpm3gcb+1DHRqDadGlpqvf+JY73p5hbIjX7BrZoGGzLkv6i+J5GAM5F+c7VvEX+qCVdr1FjMu3xvj7j2Mep7XMSwDynKdl3eEB/LOaNblPAmUMqZHrOrPM5T6dtZIk9d1XBSdb1er87/v4hLEUSYVV8mOTx4eslkKfvp9+0= Received: from DM3PR03CA0024.namprd03.prod.outlook.com (2603:10b6:0:50::34) by SN1PR12MB2350.namprd12.prod.outlook.com (2603:10b6:802:24::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Wed, 26 Jan 2022 22:07:33 +0000 Received: from DM6NAM11FT009.eop-nam11.prod.protection.outlook.com (2603:10b6:0:50:cafe::30) by DM3PR03CA0024.outlook.office365.com (2603:10b6:0:50::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15 via Frontend Transport; Wed, 26 Jan 2022 22:07:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT009.mail.protection.outlook.com (10.13.173.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4930.15 via Frontend Transport; Wed, 26 Jan 2022 22:07:33 +0000 Received: from rtg-amd.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Wed, 26 Jan 2022 16:07:29 -0600 From: Arunpravin To: , , Subject: [PATCH v10 1/5] drm: improve drm_buddy_alloc function Date: Thu, 27 Jan 2022 03:36:07 +0530 Message-ID: <20220126220611.3663-1-Arunpravin.PaneerSelvam@amd.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ae11adb8-f81a-4408-d198-08d9e11840df X-MS-TrafficTypeDiagnostic: SN1PR12MB2350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:116; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Z25w1fF2lnU80TBJ327iz70KJ9qJEyJdK98ktiLlJBvixoaXlvQBpSiu1GFhpIIOaNrAy8cJf7Mj82sfhzd5aYSVSkK+LBEfGJ1ntm3J2Dgk1pQN/vtbmtpPwWmx1wSqWNE5TdrIWZ/lAw9xKzzYW+xlQXjfk8SW7JcnbPC7ySB26A6pzlavDV7/Jhl2GlLmoswuCfRlIooYKc35RmOJuz4phTh46ZqLEGujklHp+QD8LxlWmU9C65X5I7HD5XWlKzk11HJOfbQA7GZ2Bl7V/JF5Pmpdj1waw/vz7uErFcen6W7rq4KNLOuXYNmRUKjslXUFskULs0zeC0WbZRf+uz1tNTfoEzKgI/TccrsPAqDPCXdpwTuJVSXeUH5dY2F4+sPCIXQDg/DIQZrfymsy2lFkB58w0o4ACnymHXEH5a4PG0L+4aaCaCVLXpyTRJhEEGnJYVmQpvDMGOLZIBriLQ6E2c1LJgT0AJWWCYosZ6uhBtoiPmCR3Bkp50SiNhwnmKNXUGdrqouI80mrUcr5caJla6xOROGk0abjTC3rUe6Dh8+DzzFT5R7ukbnJTM0aMCwcrGGuLOgGHTaqE9t6gFxZM8Pr1Y2EASNuowQqtDCvWQuU+ka0eP4erAONlho4+MEIXsP2B0WTUnwidd6A74xILCFVS2AECfmd889lh+EWUjgkwpuwtn1wxG7xTV5aXseEkBWOOzC0cBlJldvMSg== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(47076005)(1076003)(2616005)(508600001)(4326008)(82310400004)(6666004)(426003)(186003)(36860700001)(7696005)(16526019)(83380400001)(36756003)(336012)(86362001)(40460700003)(26005)(8676002)(356005)(5660300002)(54906003)(110136005)(316002)(30864003)(8936002)(70206006)(70586007)(2906002)(81166007)(36900700001)(20210929001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jan 2022 22:07:33.2573 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ae11adb8-f81a-4408-d198-08d9e11840df X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT009.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB2350 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arunpravin , matthew.auld@intel.com, tzimmermann@suse.de, alexander.deucher@amd.com, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel v4(Matthew Auld): - keep drm_buddy_alloc_range() function implementation for generic actual range allocations - keep alloc_range() implementation for end bias allocations v5(Matthew Auld): - modify drm_buddy_alloc() passing argument place->lpfn to lpfn as place->lpfn will currently always be zero for i915 v6(Matthew Auld): - fixup potential uaf - If we are unlucky and can't allocate enough memory when splitting blocks, where we temporarily end up with the given block and its buddy on the respective free list, then we need to ensure we delete both blocks, and no just the buddy, before potentially freeing them - fix warnings reported by kernel test robot v7(Matthew Auld): - revert fixup potential uaf - keep __alloc_range() add node to the list logic same as drm_buddy_alloc_blocks() by having a temporary list variable - at drm_buddy_alloc_blocks() keep i915 range_overflows macro and add a new check for end variable Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 315 +++++++++++++----- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 67 ++-- drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 2 + include/drm/drm_buddy.h | 13 +- 4 files changed, 280 insertions(+), 117 deletions(-) base-commit: 348109d01999f0feea85e8f336dc804b782ab870 diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index d60878bc9c20..cfc160a1ef1a 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -282,23 +282,97 @@ void drm_buddy_free_list(struct drm_buddy *mm, struct list_head *objects) } EXPORT_SYMBOL(drm_buddy_free_list); -/** - * drm_buddy_alloc_blocks - allocate power-of-two blocks - * - * @mm: DRM buddy manager to allocate from - * @order: size of the allocation - * - * The order value here translates to: - * - * 0 = 2^0 * mm->chunk_size - * 1 = 2^1 * mm->chunk_size - * 2 = 2^2 * mm->chunk_size - * - * Returns: - * allocated ptr to the &drm_buddy_block on success - */ -struct drm_buddy_block * -drm_buddy_alloc_blocks(struct drm_buddy *mm, unsigned int order) +static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) +{ + return s1 <= e2 && e1 >= s2; +} + +static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) +{ + return s1 <= s2 && e1 >= e2; +} + +static struct drm_buddy_block * +alloc_range_bias(struct drm_buddy *mm, + u64 start, u64 end, + unsigned int order) +{ + struct drm_buddy_block *block; + struct drm_buddy_block *buddy; + LIST_HEAD(dfs); + int err; + int i; + + end = end - 1; + + for (i = 0; i < mm->n_roots; ++i) + list_add_tail(&mm->roots[i]->tmp_link, &dfs); + + do { + u64 block_start; + u64 block_end; + + block = list_first_entry_or_null(&dfs, + struct drm_buddy_block, + tmp_link); + if (!block) + break; + + list_del(&block->tmp_link); + + if (drm_buddy_block_order(block) < order) + continue; + + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block) - 1; + + if (!overlaps(start, end, block_start, block_end)) + continue; + + if (drm_buddy_block_is_allocated(block)) + continue; + + if (contains(start, end, block_start, block_end) && + order == drm_buddy_block_order(block)) { + /* + * Find the free block within the range. + */ + if (drm_buddy_block_is_free(block)) + return block; + + continue; + } + + if (!drm_buddy_block_is_split(block)) { + err = split_block(mm, block); + if (unlikely(err)) + goto err_undo; + } + + list_add(&block->right->tmp_link, &dfs); + list_add(&block->left->tmp_link, &dfs); + } while (1); + + return ERR_PTR(-ENOSPC); + +err_undo: + /* + * We really don't want to leave around a bunch of split blocks, since + * bigger is better, so make sure we merge everything back before we + * free the allocated blocks. + */ + buddy = get_buddy(block); + if (buddy && + (drm_buddy_block_is_free(block) && + drm_buddy_block_is_free(buddy))) + __drm_buddy_free(mm, block); + return ERR_PTR(err); +} + +static struct drm_buddy_block * +alloc_from_freelist(struct drm_buddy *mm, + unsigned int order, + unsigned long flags) { struct drm_buddy_block *block = NULL; unsigned int i; @@ -320,78 +394,29 @@ drm_buddy_alloc_blocks(struct drm_buddy *mm, unsigned int order) while (i != order) { err = split_block(mm, block); if (unlikely(err)) - goto out_free; + goto err_undo; - /* Go low */ - block = block->left; + block = block->right; i--; } - - mark_allocated(block); - mm->avail -= drm_buddy_block_size(mm, block); - kmemleak_update_trace(block); return block; -out_free: +err_undo: if (i != order) __drm_buddy_free(mm, block); return ERR_PTR(err); } -EXPORT_SYMBOL(drm_buddy_alloc_blocks); - -static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) -{ - return s1 <= e2 && e1 >= s2; -} -static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) -{ - return s1 <= s2 && e1 >= e2; -} - -/** - * drm_buddy_alloc_range - allocate range - * - * @mm: DRM buddy manager to allocate from - * @blocks: output list head to add allocated blocks - * @start: start of the allowed range for this block - * @size: size of the allocation - * - * Intended for pre-allocating portions of the address space, for example to - * reserve a block for the initial framebuffer or similar, hence the expectation - * here is that drm_buddy_alloc_blocks() is still the main vehicle for - * allocations, so if that's not the case then the drm_mm range allocator is - * probably a much better fit, and so you should probably go use that instead. - * - * Note that it's safe to chain together multiple alloc_ranges - * with the same blocks list - * - * Returns: - * 0 on success, error code on failure. - */ -int drm_buddy_alloc_range(struct drm_buddy *mm, - struct list_head *blocks, - u64 start, u64 size) +static int __alloc_range(struct drm_buddy *mm, + struct list_head *dfs, + u64 start, u64 size, + struct list_head *blocks) { struct drm_buddy_block *block; struct drm_buddy_block *buddy; LIST_HEAD(allocated); - LIST_HEAD(dfs); u64 end; int err; - int i; - - if (size < mm->chunk_size) - return -EINVAL; - - if (!IS_ALIGNED(size | start, mm->chunk_size)) - return -EINVAL; - - if (range_overflows(start, size, mm->size)) - return -EINVAL; - - for (i = 0; i < mm->n_roots; ++i) - list_add_tail(&mm->roots[i]->tmp_link, &dfs); end = start + size - 1; @@ -399,7 +424,7 @@ int drm_buddy_alloc_range(struct drm_buddy *mm, u64 block_start; u64 block_end; - block = list_first_entry_or_null(&dfs, + block = list_first_entry_or_null(dfs, struct drm_buddy_block, tmp_link); if (!block) @@ -436,8 +461,8 @@ int drm_buddy_alloc_range(struct drm_buddy *mm, goto err_undo; } - list_add(&block->right->tmp_link, &dfs); - list_add(&block->left->tmp_link, &dfs); + list_add(&block->right->tmp_link, dfs); + list_add(&block->left->tmp_link, dfs); } while (1); list_splice_tail(&allocated, blocks); @@ -459,7 +484,143 @@ int drm_buddy_alloc_range(struct drm_buddy *mm, drm_buddy_free_list(mm, &allocated); return err; } -EXPORT_SYMBOL(drm_buddy_alloc_range); + +/** + * __drm_buddy_alloc_range - actual range allocation + * + * @mm: DRM buddy manager to allocate from + * @start: start of the allowed range for this block + * @size: size of the allocation + * @blocks: output list head to add allocated blocks + * + * Intended for pre-allocating portions of the address space, for example to + * reserve a block for the initial framebuffer or similar + * + * Note that it's safe to chain together multiple alloc_ranges + * with the same blocks list + * + * Returns: + * 0 on success, error code on failure. + */ +static int __drm_buddy_alloc_range(struct drm_buddy *mm, + u64 start, + u64 size, + struct list_head *blocks) +{ + LIST_HEAD(dfs); + int i; + + for (i = 0; i < mm->n_roots; ++i) + list_add_tail(&mm->roots[i]->tmp_link, &dfs); + + return __alloc_range(mm, &dfs, start, size, blocks); +} + +/** + * drm_buddy_alloc_blocks - allocate power-of-two blocks + * + * @mm: DRM buddy manager to allocate from + * @start: start of the allowed range for this block + * @end: end of the allowed range for this block + * @size: size of the allocation + * @min_page_size: alignment of the allocation + * @blocks: output list head to add allocated blocks + * @flags: DRM_BUDDY_*_ALLOCATION flags + * + * alloc_range_bias() called on range limitations, which traverses + * the tree and returns the desired block. + * + * alloc_from_freelist() called when *no* range restrictions + * are enforced, which picks the block from the freelist. + * + * blocks are allocated in order, the order value here translates to: + * + * 0 = 2^0 * mm->chunk_size + * 1 = 2^1 * mm->chunk_size + * 2 = 2^2 * mm->chunk_size + * + * Returns: + * 0 on success, error code on failure. + */ +int drm_buddy_alloc_blocks(struct drm_buddy *mm, + u64 start, u64 end, u64 size, + u64 min_page_size, + struct list_head *blocks, + unsigned long flags) +{ + struct drm_buddy_block *block = NULL; + unsigned int min_order, order; + unsigned long pages; + LIST_HEAD(allocated); + int err; + + if (size < mm->chunk_size) + return -EINVAL; + + if (min_page_size < mm->chunk_size) + return -EINVAL; + + if (!is_power_of_2(min_page_size)) + return -EINVAL; + + if (!IS_ALIGNED(start | end | size, mm->chunk_size)) + return -EINVAL; + + if (end > mm->size) + return -EINVAL; + + if (range_overflows(start, size, mm->size)) + return -EINVAL; + + /* Actual range allocation */ + if (start + size == end) + return __drm_buddy_alloc_range(mm, start, size, blocks); + + pages = size >> ilog2(mm->chunk_size); + order = fls(pages) - 1; + min_order = ilog2(min_page_size) - ilog2(mm->chunk_size); + + do { + order = min(order, (unsigned int)fls(pages) - 1); + BUG_ON(order > mm->max_order); + BUG_ON(order < min_order); + + do { + if (flags & DRM_BUDDY_RANGE_ALLOCATION) + /* Allocate traversing within the range */ + block = alloc_range_bias(mm, start, end, order); + else + /* Allocate from freelist */ + block = alloc_from_freelist(mm, order, flags); + + if (!IS_ERR(block)) + break; + + if (order-- == min_order) { + err = -ENOSPC; + goto err_free; + } + } while (1); + + mark_allocated(block); + mm->avail -= drm_buddy_block_size(mm, block); + kmemleak_update_trace(block); + list_add_tail(&block->link, &allocated); + + pages -= BIT(order); + + if (!pages) + break; + } while (1); + + list_splice_tail(&allocated, blocks); + return 0; + +err_free: + drm_buddy_free_list(mm, &allocated); + return err; +} +EXPORT_SYMBOL(drm_buddy_alloc_blocks); /** * drm_buddy_block_print - print block information diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index 247714bab044..b9b420cabc14 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -36,13 +36,14 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); struct i915_ttm_buddy_resource *bman_res; struct drm_buddy *mm = &bman->mm; - unsigned long n_pages; - unsigned int min_order; + unsigned long n_pages, lpfn; u64 min_page_size; u64 size; int err; - GEM_BUG_ON(place->fpfn || place->lpfn); + lpfn = place->lpfn; + if (!lpfn) + lpfn = man->size; bman_res = kzalloc(sizeof(*bman_res), GFP_KERNEL); if (!bman_res) @@ -52,6 +53,9 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, INIT_LIST_HEAD(&bman_res->blocks); bman_res->mm = mm; + if (place->fpfn || lpfn != man->size) + bman_res->flags |= DRM_BUDDY_RANGE_ALLOCATION; + GEM_BUG_ON(!bman_res->base.num_pages); size = bman_res->base.num_pages << PAGE_SHIFT; @@ -60,10 +64,16 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, min_page_size = bo->page_alignment << PAGE_SHIFT; GEM_BUG_ON(min_page_size < mm->chunk_size); - min_order = ilog2(min_page_size) - ilog2(mm->chunk_size); + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { + unsigned long pages; + size = roundup_pow_of_two(size); - min_order = ilog2(size) - ilog2(mm->chunk_size); + min_page_size = size; + + pages = size >> ilog2(mm->chunk_size); + if (pages > lpfn) + lpfn = pages; } if (size > mm->size) { @@ -73,34 +83,16 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, n_pages = size >> ilog2(mm->chunk_size); - do { - struct drm_buddy_block *block; - unsigned int order; - - order = fls(n_pages) - 1; - GEM_BUG_ON(order > mm->max_order); - GEM_BUG_ON(order < min_order); - - do { - mutex_lock(&bman->lock); - block = drm_buddy_alloc_blocks(mm, order); - mutex_unlock(&bman->lock); - if (!IS_ERR(block)) - break; - - if (order-- == min_order) { - err = -ENOSPC; - goto err_free_blocks; - } - } while (1); - - n_pages -= BIT(order); - - list_add_tail(&block->link, &bman_res->blocks); - - if (!n_pages) - break; - } while (1); + mutex_lock(&bman->lock); + err = drm_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT, + (u64)lpfn << PAGE_SHIFT, + (u64)n_pages << PAGE_SHIFT, + min_page_size, + &bman_res->blocks, + bman_res->flags); + mutex_unlock(&bman->lock); + if (unlikely(err)) + goto err_free_blocks; *res = &bman_res->base; return 0; @@ -268,10 +260,17 @@ int i915_ttm_buddy_man_reserve(struct ttm_resource_manager *man, { struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); struct drm_buddy *mm = &bman->mm; + unsigned long flags = 0; int ret; + flags |= DRM_BUDDY_RANGE_ALLOCATION; + mutex_lock(&bman->lock); - ret = drm_buddy_alloc_range(mm, &bman->reserved, start, size); + ret = drm_buddy_alloc_blocks(mm, start, + start + size, + size, mm->chunk_size, + &bman->reserved, + flags); mutex_unlock(&bman->lock); return ret; diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h index 312077941411..72c90b432e87 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h @@ -20,6 +20,7 @@ struct drm_buddy; * * @base: struct ttm_resource base class we extend * @blocks: the list of struct i915_buddy_block for this resource/allocation + * @flags: DRM_BUDDY_*_ALLOCATION flags * @mm: the struct i915_buddy_mm for this resource * * Extends the struct ttm_resource to manage an address space allocation with @@ -28,6 +29,7 @@ struct drm_buddy; struct i915_ttm_buddy_resource { struct ttm_resource base; struct list_head blocks; + unsigned long flags; struct drm_buddy *mm; }; diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index f524db152413..54f25a372f27 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -22,6 +22,8 @@ start__ >= max__ || size__ > max__ - start__; \ }) +#define DRM_BUDDY_RANGE_ALLOCATION (1 << 0) + struct drm_buddy_block { #define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12) #define DRM_BUDDY_HEADER_STATE GENMASK_ULL(11, 10) @@ -131,12 +133,11 @@ int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size); void drm_buddy_fini(struct drm_buddy *mm); -struct drm_buddy_block * -drm_buddy_alloc_blocks(struct drm_buddy *mm, unsigned int order); - -int drm_buddy_alloc_range(struct drm_buddy *mm, - struct list_head *blocks, - u64 start, u64 size); +int drm_buddy_alloc_blocks(struct drm_buddy *mm, + u64 start, u64 end, u64 size, + u64 min_page_size, + struct list_head *blocks, + unsigned long flags); void drm_buddy_free_block(struct drm_buddy *mm, struct drm_buddy_block *block); From patchwork Wed Jan 26 22:06:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paneer Selvam, Arunpravin" X-Patchwork-Id: 12725866 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3742DC433EF for ; Wed, 26 Jan 2022 22:07:53 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4A31610E97B; Wed, 26 Jan 2022 22:07:45 +0000 (UTC) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1anam02on2054.outbound.protection.outlook.com [40.107.96.54]) by gabe.freedesktop.org (Postfix) with ESMTPS id 85F3E10E974; Wed, 26 Jan 2022 22:07:39 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aVMJ62KaMMcrjfBOgXUgfJo1FJQ4LdeRbJTbL7f09PkasquvtJo5qDBNd43nmCIIDjZnSZJEgWN43dP840P6reGPCaD0tmjcAqMT1zutTtoCHxUcwdOmx2WVvHPUPMkXj1Ml01knsHVsW85zuF1KLsURppz/C0Fkklj+/ulMcSSv4n7k3EyVD7vH3RrkSmWkA4Yte5OI70ZS1CxMNEShOKOrLBl+EcumgrN5StjDYZJUgV6ZnWfLxjnCKMWUat8uaoUz6VFfiCUngaV+8mJTRciSIFXYJNG5lUrLryhxZYcozCfNWEa/s/M0petZAEdCeMW46u/b7uyK03Kd0w25gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jecuX40XGqXfqe13o0rlRRt/d8eeaBkxa3LzFd5jFRc=; b=UjuwYbqnBAvBUki2sfCiYGLdcDieG2zRPbDr90XDt1NO+oZfeEHfkpbuiEJ6ruP3/4po2TA4uzfw5GqdmMWeI59eorxSb46GN1YW6dC5ODi5yHpjD6wPVzGv2rmKhcEOKPnXOXHeEekdwHXx/yAT5PROiz06m90PyZMKmxV+DXAQSeqIpvYZzLseXrFmUju0GNNyZXSovNci4FMfguBtEaocgdu/9XztLfab0IVA6waA5243PDfATM1SAq4gSvxrPnD5m8efPtVcDFEAb9YKNBgGnCRSV5dmryDq92bWIlnmIGXEmpUU0oFx9RRBr76i7TiPLOAW/lga+82DLZpoOw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jecuX40XGqXfqe13o0rlRRt/d8eeaBkxa3LzFd5jFRc=; b=XbFqLbl4pNa6ETLZ05BoxuJfcFXVx3jA8hYb9EOgfeX7aw3GlKYKFiCGlDBu03YYbUyXOZBuYtUDf+zWRGyIXWonXs/Jeoef9hiSuQyVI9RWhnc9lP9EtcfOS7NYaTNZSQZbFeoCHLLuATKeytrNmdsJo8QoInn1rXxzCH6DZPI= Received: from DS7PR06CA0047.namprd06.prod.outlook.com (2603:10b6:8:54::27) by BYAPR12MB2936.namprd12.prod.outlook.com (2603:10b6:a03:12f::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Wed, 26 Jan 2022 22:07:37 +0000 Received: from DM6NAM11FT038.eop-nam11.prod.protection.outlook.com (2603:10b6:8:54:cafe::35) by DS7PR06CA0047.outlook.office365.com (2603:10b6:8:54::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.10 via Frontend Transport; Wed, 26 Jan 2022 22:07:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT038.mail.protection.outlook.com (10.13.173.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4930.15 via Frontend Transport; Wed, 26 Jan 2022 22:07:36 +0000 Received: from rtg-amd.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Wed, 26 Jan 2022 16:07:32 -0600 From: Arunpravin To: , , Subject: [PATCH v10 2/5] drm: implement top-down allocation method Date: Thu, 27 Jan 2022 03:36:08 +0530 Message-ID: <20220126220611.3663-2-Arunpravin.PaneerSelvam@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220126220611.3663-1-Arunpravin.PaneerSelvam@amd.com> References: <20220126220611.3663-1-Arunpravin.PaneerSelvam@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c266ca51-3fa4-45ed-c1ef-08d9e11842d7 X-MS-TrafficTypeDiagnostic: BYAPR12MB2936:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lXgj+Xe5ca2mucX92PGeCPq+WrbVndPUlLQmDXGzFW3b6pORq9x0HVj0twDxmUYOLpUoO6aiNvisqyjm6lcscuFNEfXph4wnffEffF0GWXssKn2HeJ3E54oT2aTudSqHy4iW1VmOyhHtTmS3LZ6jwqpPWpIY8X2l4UV0hW/xpKh2DozjHSLadXzJTt4LRyst6pZzKRcB6RO9gX0mdbYEMh+ksuNiJRxNiJj6otWFSWdvRrBxxsW4IO+3r3OfJ4MNAmS9Vpd1bvfT6FcpFsZ/7sjOpuHAg+TN27QQA0zca1u2nfM5YBSMO48TACR3+OrnDquzr85AJH19NmM1UUqovSIlld2kOEBS0rcyzXcUkgAlkUEBCj52hDyxczNuhrn1BEqhpLVaTMjN7w87pQ/byDCkqjam8tyrKRjK40AwSpRJDxk5pDa7Cxx9kcX7vAhO0I7LIVMsvTPUBux+BqqpccGRiP6sPa6YDkTFPVkILGd4Sq/J9UPjV7rCaSRQEISwFcUQAwOM0fHppzZpk8lZ1doMhsBUPir1hvCSUq6wlQsb8iuky0SdPaihzO3qiOJ4tMWbzgJ9Jz+HpOo5lxXidU9cOLif2UTtr8JAe66MpDVCu8+O4UyF25YvDvg6RmvvifRnva+IVzM7q7EmEYlDLNiRXR2mBc5xaGQ0WQaegydKFnTSBqp9hcQs+ckcfSgo0VLwrAYs/tnSgwQQPZWtqg== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(47076005)(1076003)(2616005)(508600001)(4326008)(82310400004)(6666004)(426003)(186003)(36860700001)(7696005)(16526019)(83380400001)(36756003)(336012)(86362001)(40460700003)(26005)(8676002)(356005)(5660300002)(54906003)(110136005)(316002)(8936002)(70206006)(70586007)(2906002)(81166007)(36900700001)(20210929001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jan 2022 22:07:36.5599 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c266ca51-3fa4-45ed-c1ef-08d9e11842d7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT038.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2936 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arunpravin , matthew.auld@intel.com, tzimmermann@suse.de, alexander.deucher@amd.com, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Implemented a function which walk through the order list, compares the offset and returns the maximum offset block, this method is unpredictable in obtaining the high range address blocks which depends on allocation and deallocation. for instance, if driver requests address at a low specific range, allocator traverses from the root block and splits the larger blocks until it reaches the specific block and in the process of splitting, lower orders in the freelist are occupied with low range address blocks and for the subsequent TOPDOWN memory request we may return the low range blocks.To overcome this issue, we may go with the below approach. The other approach, sorting each order list entries in ascending order and compares the last entry of each order list in the freelist and return the max block. This creates sorting overhead on every drm_buddy_free() request and split up of larger blocks for a single page request. v2: - Fix alignment issues(Matthew Auld) - Remove unnecessary list_empty check(Matthew Auld) - merged the below patch to see the feature in action - add top-down alloc support to i915 driver Signed-off-by: Arunpravin Reviewed-by: Matthew Auld --- drivers/gpu/drm/drm_buddy.c | 36 ++++++++++++++++--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 3 ++ include/drm/drm_buddy.h | 1 + 3 files changed, 35 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index cfc160a1ef1a..30cad939a112 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -369,6 +369,26 @@ alloc_range_bias(struct drm_buddy *mm, return ERR_PTR(err); } +static struct drm_buddy_block * +get_maxblock(struct list_head *head) +{ + struct drm_buddy_block *max_block = NULL, *node; + + max_block = list_first_entry_or_null(head, + struct drm_buddy_block, + link); + if (!max_block) + return NULL; + + list_for_each_entry(node, head, link) { + if (drm_buddy_block_offset(node) > + drm_buddy_block_offset(max_block)) + max_block = node; + } + + return max_block; +} + static struct drm_buddy_block * alloc_from_freelist(struct drm_buddy *mm, unsigned int order, @@ -379,11 +399,17 @@ alloc_from_freelist(struct drm_buddy *mm, int err; for (i = order; i <= mm->max_order; ++i) { - block = list_first_entry_or_null(&mm->free_list[i], - struct drm_buddy_block, - link); - if (block) - break; + if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) { + block = get_maxblock(&mm->free_list[i]); + if (block) + break; + } else { + block = list_first_entry_or_null(&mm->free_list[i], + struct drm_buddy_block, + link); + if (block) + break; + } } if (!block) diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index b9b420cabc14..45b091626278 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -53,6 +53,9 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, INIT_LIST_HEAD(&bman_res->blocks); bman_res->mm = mm; + if (place->flags & TTM_PL_FLAG_TOPDOWN) + bman_res->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION; + if (place->fpfn || lpfn != man->size) bman_res->flags |= DRM_BUDDY_RANGE_ALLOCATION; diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index 54f25a372f27..f0378fb48d06 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -23,6 +23,7 @@ }) #define DRM_BUDDY_RANGE_ALLOCATION (1 << 0) +#define DRM_BUDDY_TOPDOWN_ALLOCATION (1 << 1) struct drm_buddy_block { #define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12) From patchwork Wed Jan 26 22:06:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paneer Selvam, Arunpravin" X-Patchwork-Id: 12725865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 658FAC433FE for ; Wed, 26 Jan 2022 22:07:50 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 415F110E941; Wed, 26 Jan 2022 22:07:44 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2064.outbound.protection.outlook.com [40.107.92.64]) by gabe.freedesktop.org (Postfix) with ESMTPS id 921F110E941; Wed, 26 Jan 2022 22:07:42 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Jcj2swVh3fgTgGzAY4bTDwWo5KrcIrg+ZcXwWG0IqkX0eRjZv6WmfNnzqEYHashU+FE7G+HmnOSILQ+681JYDcRIFYbnecmSP3lUZOKPgEBSvLAYxjHxSoLAq6VZruvdUXVFfqem3XE09qHZI0SNzeY44jh2rEywqWVfsowm/9zYErEdQH2C3zjC3SgRzm6NOUCn1RJQoii4XHEUkH54c9YGhLh7I7gy/nd2qdRgB7tPEPjoXN69V65b88Ft5J2huYhHb9Jj9hhObCxJaoa69J8rQKhaWr4WFK9e1WQvK4sYg6l1kHm9FH2gjqgAQ14UlzYT51g4O88LqJ4NUnpnaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BzUSaWLbfFjYKZA+3YMkt2bzyq+BVqlpMYmAO8MKOqY=; b=DmAAmdV9havzY347VB27LW5CzVUuC6URuJellawN9ew01CkI2CO0GBl/ZYOhYwbdAzdWcRUL0jA7yL16p6VAi31dzbay/kCOXHxhbCd1Ju+CIqKOJu26MpMfrnOidn6GvRd2hE7Nk/CF7fZpXOv89dB9toN69A1oYxqWO9KmL+4IjNIjh9gTG4F8ZZi5NoGLCGTJ41GcAVbU1rSROYtUOVyP/88AYwrH3TcldyHMQ1vzMLmOndhZ3OvF+YIkgHakkeDMMn0DtyyXhf8k0s5faaPm/jAdiiRCWYQwLKMtwoSHhLV9aODhq2IHY9eS2P1CVZjhlxlnzkAcHoji4632rQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BzUSaWLbfFjYKZA+3YMkt2bzyq+BVqlpMYmAO8MKOqY=; b=g5q/1OF201N52Luu/2QGbfbC0nyC15FvzNWsnazq7mNFJr/hywOlTyG9s8hPdkywxtY7EjI45ZbGCQZbH1Xrqxa7Nzzb6413QRCSEbzTaQVpX29e2fpYiJLzQINOzsum70/WWLt8oxh48UoU1QfErhUbxyukoBAT9nXkDJ0xylA= Received: from DM3PR03CA0023.namprd03.prod.outlook.com (2603:10b6:0:50::33) by CH0PR12MB5042.namprd12.prod.outlook.com (2603:10b6:610:e1::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Wed, 26 Jan 2022 22:07:40 +0000 Received: from DM6NAM11FT009.eop-nam11.prod.protection.outlook.com (2603:10b6:0:50:cafe::2c) by DM3PR03CA0023.outlook.office365.com (2603:10b6:0:50::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.8 via Frontend Transport; Wed, 26 Jan 2022 22:07:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT009.mail.protection.outlook.com (10.13.173.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4930.15 via Frontend Transport; Wed, 26 Jan 2022 22:07:39 +0000 Received: from rtg-amd.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Wed, 26 Jan 2022 16:07:36 -0600 From: Arunpravin To: , , Subject: [PATCH v10 3/5] drm: implement a method to free unused pages Date: Thu, 27 Jan 2022 03:36:09 +0530 Message-ID: <20220126220611.3663-3-Arunpravin.PaneerSelvam@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220126220611.3663-1-Arunpravin.PaneerSelvam@amd.com> References: <20220126220611.3663-1-Arunpravin.PaneerSelvam@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 101b0771-d572-4c9f-5184-08d9e11844d0 X-MS-TrafficTypeDiagnostic: CH0PR12MB5042:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d1OMWD19UpfOi7pYoY6gGs/hRMNXCYUSpjAzGLhEtwUxJcojR993sIfmk2Q4brgbd7eA6MKv+n1CVHduLfBOnzhtht5jVG9ItT8zhcJaxjG+SSwktgJy4zwAy8CA9OJZS+yQsmUXKB9j8Ts53EcSLr6pfUkwx1fCYFt4ZxN0k6tQf8u+BftNuD7VEiqgobgjSjXgZYv5DYglALj9sDr7mrxWZ6QRF/CXUCgMNV9ozDvHWd/B1DdOQGqBNoD0DogPCiQpZSOkHOhOngblfW/4edkPsCRQh/PbBKyFNmDzrZ4cZhwlwHc8HWSye+i5CCyKXX1JHEn60svHUDuVj+M/bPoMHQNLXblnok60to5UQAP0wloPEAN1J8EyIwS8cpys1KlcRpCaeT4ejntli9f3v6660n+HnxY8F4kYaSRUDxgDWVcLd497ndU+F+iwoYvcYiKpVqkNixqR4+8kaR1vSehAzs8mW2mEE1GQopa/5CtJcZOxVGgx7at7xHlagdQnY/CCiWUsTu6uKzNLk+Oqw+NdQ9YUo8f1YwNZ4Ejq29JNy4l/C13LuremDRfMXUsF4TPql39VUonqrSPhXPyDuQeqjNDHfg2hzNV9MIxANjYX0ioo7Lxhairxubtde8AkXdWre6k2fFkyvZinNfSa/UWjAbrIwV0w3PBERAbwlC/B5rO8iEXEd2LreSy0ew80WRTOzjz+oH6KhKiltWVAoA== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(86362001)(316002)(70206006)(70586007)(6666004)(7696005)(508600001)(5660300002)(4326008)(47076005)(8676002)(8936002)(36860700001)(36756003)(40460700003)(81166007)(356005)(82310400004)(2906002)(54906003)(83380400001)(110136005)(16526019)(186003)(2616005)(1076003)(426003)(336012)(26005)(36900700001)(20210929001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jan 2022 22:07:39.8666 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 101b0771-d572-4c9f-5184-08d9e11844d0 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT009.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5042 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arunpravin , matthew.auld@intel.com, tzimmermann@suse.de, alexander.deucher@amd.com, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On contiguous allocation, we round up the size to the *next* power of 2, implement a function to free the unused pages after the newly allocate block. v2(Matthew Auld): - replace function name 'drm_buddy_free_unused_pages' with drm_buddy_block_trim - replace input argument name 'actual_size' with 'new_size' - add more validation checks for input arguments - add overlaps check to avoid needless searching and splitting - merged the below patch to see the feature in action - add free unused pages support to i915 driver - lock drm_buddy_block_trim() function as it calls mark_free/mark_split are all globally visible v3(Matthew Auld): - remove trim method error handling as we address the failure case at drm_buddy_block_trim() function v4: - in case of trim, at __alloc_range() split_block failure path marks the block as free and removes it from the original list, potentially also freeing it, to overcome this problem, we turn the drm_buddy_block_trim() input node into a temporary node to prevent recursively freeing itself, but still retain the un-splitting/freeing of the other nodes(Matthew Auld) - modify the drm_buddy_block_trim() function return type v5(Matthew Auld): - revert drm_buddy_block_trim() function return type changes in v4 - modify drm_buddy_block_trim() passing argument n_pages to original_size as n_pages has already been rounded up to the next power-of-two and passing n_pages results noop v6: - fix warnings reported by kernel test robot v7: - modify drm_buddy_block_trim() function doc description - at drm_buddy_block_trim() handle non-allocated block as a serious programmer error - fix a typo Signed-off-by: Arunpravin Reviewed-by: Matthew Auld --- drivers/gpu/drm/drm_buddy.c | 69 +++++++++++++++++++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 10 +++ include/drm/drm_buddy.h | 4 ++ 3 files changed, 83 insertions(+) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 30cad939a112..4845ef784b5e 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -542,6 +542,75 @@ static int __drm_buddy_alloc_range(struct drm_buddy *mm, return __alloc_range(mm, &dfs, start, size, blocks); } +/** + * drm_buddy_block_trim - free unused pages + * + * @mm: DRM buddy manager + * @new_size: original size requested + * @blocks: Input and output list of allocated blocks. + * MUST contain single block as input to be trimmed. + * On success will contain the newly allocated blocks + * making up the @new_size. Blocks always appear in + * ascending order + * + * For contiguous allocation, we round up the size to the nearest + * power of two value, drivers consume *actual* size, so remaining + * portions are unused and can be optionally freed with this function + * + * Returns: + * 0 on success, error code on failure. + */ +int drm_buddy_block_trim(struct drm_buddy *mm, + u64 new_size, + struct list_head *blocks) +{ + struct drm_buddy_block *parent; + struct drm_buddy_block *block; + LIST_HEAD(dfs); + u64 new_start; + int err; + + if (!list_is_singular(blocks)) + return -EINVAL; + + block = list_first_entry(blocks, + struct drm_buddy_block, + link); + + if (WARN_ON(!drm_buddy_block_is_allocated(block))) + return -EINVAL; + + if (new_size > drm_buddy_block_size(mm, block)) + return -EINVAL; + + if (!new_size || !IS_ALIGNED(new_size, mm->chunk_size)) + return -EINVAL; + + if (new_size == drm_buddy_block_size(mm, block)) + return 0; + + list_del(&block->link); + mark_free(mm, block); + mm->avail += drm_buddy_block_size(mm, block); + + /* Prevent recursively freeing this node */ + parent = block->parent; + block->parent = NULL; + + new_start = drm_buddy_block_offset(block); + list_add(&block->tmp_link, &dfs); + err = __alloc_range(mm, &dfs, new_start, new_size, blocks); + if (err) { + mark_allocated(block); + mm->avail -= drm_buddy_block_size(mm, block); + list_add(&block->link, blocks); + } + + block->parent = parent; + return err; +} +EXPORT_SYMBOL(drm_buddy_block_trim); + /** * drm_buddy_alloc_blocks - allocate power-of-two blocks * diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index 45b091626278..b52684552523 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -97,6 +97,16 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, if (unlikely(err)) goto err_free_blocks; + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { + u64 original_size = (u64)bman_res->base.num_pages << PAGE_SHIFT; + + mutex_lock(&bman->lock); + drm_buddy_block_trim(mm, + original_size, + &bman_res->blocks); + mutex_unlock(&bman->lock); + } + *res = &bman_res->base; return 0; diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index f0378fb48d06..bd21f9dfd15e 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -140,6 +140,10 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, struct list_head *blocks, unsigned long flags); +int drm_buddy_block_trim(struct drm_buddy *mm, + u64 new_size, + struct list_head *blocks); + void drm_buddy_free_block(struct drm_buddy *mm, struct drm_buddy_block *block); void drm_buddy_free_list(struct drm_buddy *mm, struct list_head *objects); From patchwork Wed Jan 26 22:06:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paneer Selvam, Arunpravin" X-Patchwork-Id: 12725868 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BF8E2C433EF for ; Wed, 26 Jan 2022 22:07:59 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 63FA910E986; Wed, 26 Jan 2022 22:07:52 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2083.outbound.protection.outlook.com [40.107.237.83]) by gabe.freedesktop.org (Postfix) with ESMTPS id AD99F10E97E; Wed, 26 Jan 2022 22:07:47 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vn1l8OVpLGAlxPQRlXop1boEIKha6ubZo6Gc51+tUBYVWhjeUq0c+EpvJSxBCO99abNiZQD8XjfN88pKeS9fWL2FbynGUBrVHvW/Nd0bQbKYMTn9qDh3s4W181GYk2CcG9HR44R5Zm3Ys6bs8eEjzPxdUhzubltdTOG+qpKpPzvX3A/IeGDQQZQuRSX/8gOGW15doqoE/2O6ybaCMmaFJK6lO7hAImLlwdmIgITbkgGqadhwbuaC/VjepcOCibUbF8+FSvrsySrg5sS4twYWleWMANGW+Tsl6IdWtG4BQ+9hp4XAjN5lnsKAA/hdtJEG4nin1dxptueh8nM32tpIdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2kIgAxTsZOmMt/R5KguyZmOmobVdLpO5qp1KF1znmrk=; b=lJ5kC1CLtOnsLOOeNnU2SNLRkzdv8mwy6SzYUqFdZSmBkCF+93l0sA56c8NADDzIqwBD+XTHKH7OHnnGjOuzQcLvfOnKnIYDSDpWChMF9jHJGtbqWelFbT3QMX22I6l/X4jHg2Ac/UmDQmRuZi/BFmSTJCqB1dZTRvV4D6Rq+XIKa72CmqGUgC9L6tp8LOa5hzHbOqmC0nIeZuNpdaGXmK+9ShOVZrsHrIj76B4IaA9vohj2QlllsbkZVwIkFawrlrDjv2COCoTCCoovqqhjZYO+SpwEozseT3tZbq12asL8EqRz9vV4xFpmojroW01sO80irGHGPCVr3Y5YuKWGBw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2kIgAxTsZOmMt/R5KguyZmOmobVdLpO5qp1KF1znmrk=; b=d84uxtIUWngPupg2VyTFxDj+syJ5paw3k7HzyxqVkiyizwm4lfSQQYonJpRxb5JuM131D9k9Gn9TgrWIs0BgrZ0CxtxWrIyS2jMHSvfcDwmDOsUCr7wPC0IOCOjGj0Vz0d7DJ/QZ4Tt5m12HGZHi5A1WbHHzPvcaNXNMvBMNgsY= Received: from DM5PR07CA0145.namprd07.prod.outlook.com (2603:10b6:3:ee::11) by BYAPR12MB3319.namprd12.prod.outlook.com (2603:10b6:a03:dc::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.13; Wed, 26 Jan 2022 22:07:43 +0000 Received: from DM6NAM11FT057.eop-nam11.prod.protection.outlook.com (2603:10b6:3:ee:cafe::2f) by DM5PR07CA0145.outlook.office365.com (2603:10b6:3:ee::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.8 via Frontend Transport; Wed, 26 Jan 2022 22:07:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT057.mail.protection.outlook.com (10.13.172.252) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4930.15 via Frontend Transport; Wed, 26 Jan 2022 22:07:43 +0000 Received: from rtg-amd.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Wed, 26 Jan 2022 16:07:39 -0600 From: Arunpravin To: , , Subject: [PATCH v10 4/5] drm/amdgpu: move vram inline functions into a header Date: Thu, 27 Jan 2022 03:36:10 +0530 Message-ID: <20220126220611.3663-4-Arunpravin.PaneerSelvam@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220126220611.3663-1-Arunpravin.PaneerSelvam@amd.com> References: <20220126220611.3663-1-Arunpravin.PaneerSelvam@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c4c4b6ce-7877-49e2-fda5-08d9e11846d9 X-MS-TrafficTypeDiagnostic: BYAPR12MB3319:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: vj1NsD04DDnyp0C7LAojctG8b31x9OGfWv3ze7+BKMtyPdxMpLVGipEwAhZ9B1aJo8J4umjsQAQz8OyhtxHYJiCWjlXRfjDdx3f2nS8RZUPC6Vx94cchFDgxVkDUWLKj360W54Fczvqe+SKCfi33K/VWffsf39aTLVFzkoePPpT5aoN1sDHQ8crS0h//lUG170USc1Z8jWZ4F4J9joAU9Ed41v5Z0OalcvSunXSbyte9legKHWlvrDhPVOYGGnpD+wom2EuBl1oyqlBYYkP53bVDKTJVym9TqcHohyqOg1fcrp/m05uQ5LEJWrmDxq/TO0hZOnpTBB5r05MSPA3pMrbzkjCFBA41jpc9LgKmzw1GzRsf1MlByxfBED3bkVXJV3tk02WlbLvIfywHFEtkjVO/oTaCreb5NULAq6pHgCqoAubXtTxRcxqWWnLR7/BZm7tuB5YlQ0BhDcrlcsukcr7qKAsBmlCIRFC/T7BF/EuU4yKMuIUEyJZvtAcVxOs5yHN8bWxIxiFAV5d1wzZtKKzeHgTmyjfuP948VmZNbo9SSkX5KL7EYTwVduQMMjZaZ8rSnGWdRcefqopM+i1VK06xu+6dWAxrXKPfO8oGBHZsuaHCpKgBRg5OVwU0hU/86JL3xQmn0gOsIjhdXOJgCaX64narlQMYCH5vEKFOTRcjp9i4Ouf3m3186fHkaJm/aAtbkH2tUkGi5Kf/XWp2G4fNRObAiM+288FvKpfL8AjnLOrHzMbmrH6uTUCnzGv+8IKb/q3DGRK0oE4VFIDgNyToRQAgTLiWquBKHPRz0TM= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(40470700004)(26005)(1076003)(2906002)(36756003)(8936002)(356005)(54906003)(81166007)(110136005)(2616005)(5660300002)(86362001)(316002)(82310400004)(40460700003)(4326008)(426003)(336012)(6666004)(47076005)(70206006)(70586007)(7696005)(186003)(83380400001)(16526019)(36860700001)(508600001)(8676002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jan 2022 22:07:43.2698 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c4c4b6ce-7877-49e2-fda5-08d9e11846d9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT057.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3319 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arunpravin , matthew.auld@intel.com, tzimmermann@suse.de, alexander.deucher@amd.com, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Move shared vram inline functions and structs into a header file Signed-off-by: Arunpravin --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h | 51 ++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h new file mode 100644 index 000000000000..59983464cce5 --- /dev/null +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: MIT + * Copyright 2021 Advanced Micro Devices, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + */ + +#ifndef __AMDGPU_VRAM_MGR_H__ +#define __AMDGPU_VRAM_MGR_H__ + +#include + +struct amdgpu_vram_mgr_node { + struct ttm_resource base; + struct list_head blocks; + unsigned long flags; +}; + +static inline u64 amdgpu_node_start(struct drm_buddy_block *block) +{ + return drm_buddy_block_offset(block); +} + +static inline u64 amdgpu_node_size(struct drm_buddy_block *block) +{ + return PAGE_SIZE << drm_buddy_block_order(block); +} + +static inline struct amdgpu_vram_mgr_node * +to_amdgpu_vram_mgr_node(struct ttm_resource *res) +{ + return container_of(res, struct amdgpu_vram_mgr_node, base); +} + +#endif From patchwork Wed Jan 26 22:06:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paneer Selvam, Arunpravin" X-Patchwork-Id: 12725867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 487A7C433FE for ; Wed, 26 Jan 2022 22:07:58 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 022BE10E980; Wed, 26 Jan 2022 22:07:52 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2042.outbound.protection.outlook.com [40.107.92.42]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6445610E984; Wed, 26 Jan 2022 22:07:50 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RocTu2Ti0gbIRMmMdN48sSdnEM+xxv29aTV3hrINlUu5DCBe7mnrUnoYi00cKK+Y0FQFI6u0YjKoS9H30vwLx+N+Q3FFyRe7CdhA+Z4xmtQbKbHEUn+eOyn1EjPk37xVOTby8aU0bwh6Hs11S2kjIJ/LSBqT0zDIQRGEUmgWg6gMMPIgKrKMlGOexp+QJ9AzZWQrKx7FDKsIU+9cUgsk0fRoxHFpqVP6Lqy9QraNWBb/xRVI95YKm9vYF/TAGVAKUuEzr+7W74McnKtD6zG5o7b/dNJo06+P6vylC8b2eFuILkyuoS7mqcB3bDIvWeAQo62kX3sAGFzAsTWHpnnhZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QyaAFspuztBX4ZbjkquvL7sj0Xcyt0822MjLqtsf/VI=; b=agDVmRmA02VjR5QQ7la4rXGfQhfyPX+Iu1/pB6Fh+0hyNbBYCAwkCIWBZfArTPsc2XmD8RKPD2QjWMeDigpYtLQBE84McO8ews3vptcaMTj8alsp70PYvXwHYotQGlaUiX5CDRDguVe0i5gvI9cMWTRwuHEALq2pgKpKcnuJEkSMaS7YPFDjXLb9Kb56Qczz2nqSTXz0amc/yMtE7NptSZKZXRpdXofuxqkzGgn7j30fpTApYsKLXrEYDK/+5y791wWIzX1VoAh7z9ouasAgEWbO9Gj8NjNTSvcWoMh9esA9oNDTP5fYX2YmOImJARgVOCB4/Ipu5QLUzVplUHgPQg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QyaAFspuztBX4ZbjkquvL7sj0Xcyt0822MjLqtsf/VI=; b=YYiylsPyVGj/ZJUDSHG5LVxcaXG3XwDpqU4rmWjsXnXA+5I44Kjt/TMYpzB/FeAzcJMLNg+Znedr1otKPV4kF3P/4Lu3+XSmkZUev/vVT4N7Cn2+WCdbaW54aXUBT0/ZVtzUrtQ+4O6cemgwHu0MSUeoTEF2j6VD6W1vc+s7LP8= Received: from DM5PR13CA0058.namprd13.prod.outlook.com (2603:10b6:3:117::20) by MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Wed, 26 Jan 2022 22:07:47 +0000 Received: from DM6NAM11FT068.eop-nam11.prod.protection.outlook.com (2603:10b6:3:117:cafe::67) by DM5PR13CA0058.outlook.office365.com (2603:10b6:3:117::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.5 via Frontend Transport; Wed, 26 Jan 2022 22:07:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT068.mail.protection.outlook.com (10.13.173.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4930.15 via Frontend Transport; Wed, 26 Jan 2022 22:07:46 +0000 Received: from rtg-amd.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Wed, 26 Jan 2022 16:07:42 -0600 From: Arunpravin To: , , Subject: [PATCH v10 5/5] drm/amdgpu: add drm buddy support to amdgpu Date: Thu, 27 Jan 2022 03:36:11 +0530 Message-ID: <20220126220611.3663-5-Arunpravin.PaneerSelvam@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220126220611.3663-1-Arunpravin.PaneerSelvam@amd.com> References: <20220126220611.3663-1-Arunpravin.PaneerSelvam@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c068526e-d091-424d-3e64-08d9e11848fb X-MS-TrafficTypeDiagnostic: MN2PR12MB4192:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2150; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gOWd57pwGZw9yQb+Zmd9NkcQrxDXDRxx48wQbeh8fGTxntF8b09C+Pgs6RInATl6JhMLOaipGVUj8Wq0CFsqtu/QU82NhfN7N+jB29T2QXx//GXMUGq9QLZU7ViLgoQxdkkA4jjSf5BP1yS/yqzPPfMRVrNhjA8bHjhmmoWjJZi8vi2cXAzRZq5+G7hUNpHu8cu3nnrPgetCeU0camDH9ZtD4f9Nkw/y1XWyF86LM7cHpt3DQY5QztWNFDRO/T42PlrnUm5MOf/yYPmYWcvbbiPJY40E0FcGy7S6OCFdh3lfezry5dHRwsTn6/AIGC1OGIXVbw2MZnvsudhJKEFePrN6yhlwIi17NW67rq2HQs+I4b9CHH0FrH9Si534NRokrYek9skC1eAv1iRTXvUX6ENNSayHM2Sr1oY0Yjbb8LSHWZlmiQwpNRM0qVThv6/7GTXn13sj2V1pfLtXpHF18OUXhiJvrnDHJsj6Ea7m56cnFv7UOesZhjxNubFcRsyZA7zSIl54PPG7bWNp9jVJ9FR62zzLWUFgUMyEa11Nc6f6EbzXvFhw5Ope6TRk7i0qokoM9MsoFVw5htoQAWx2mSrmlBreR/lZAHRxjsEXjUfb6ALUptQ/lFxF91omGWzbrTh+svhZsrT2XAT1Iha6VtP9EE2tEuQuc2tRI2soFZY6ki6j2DDINxnpAmw4a44MFqYLTVXB6LBYD8XlZIedcQ== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(36840700001)(46966006)(7696005)(36860700001)(356005)(36756003)(30864003)(5660300002)(2906002)(47076005)(2616005)(26005)(81166007)(186003)(1076003)(16526019)(86362001)(82310400004)(4326008)(54906003)(110136005)(508600001)(8676002)(6666004)(316002)(40460700003)(83380400001)(426003)(336012)(8936002)(70206006)(70586007)(36900700001)(20210929001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jan 2022 22:07:46.8431 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c068526e-d091-424d-3e64-08d9e11848fb X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT068.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4192 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arunpravin , matthew.auld@intel.com, tzimmermann@suse.de, alexander.deucher@amd.com, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" - Remove drm_mm references and replace with drm buddy functionalities - Add res cursor support for drm buddy v2(Matthew Auld): - replace spinlock with mutex as we call kmem_cache_zalloc (..., GFP_KERNEL) in drm_buddy_alloc() function - lock drm_buddy_block_trim() function as it calls mark_free/mark_split are all globally visible v3(Matthew Auld): - remove trim method error handling as we address the failure case at drm_buddy_block_trim() function v4: - fix warnings reported by kernel test robot v5: - fix merge conflict issue Signed-off-by: Arunpravin --- drivers/gpu/drm/Kconfig | 1 + .../gpu/drm/amd/amdgpu/amdgpu_res_cursor.h | 97 +++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 7 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 259 ++++++++++-------- 4 files changed, 231 insertions(+), 133 deletions(-) diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index dfdd3ec5f793..eb5a57ae3c5c 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -279,6 +279,7 @@ config DRM_AMDGPU select HWMON select BACKLIGHT_CLASS_DEVICE select INTERVAL_TREE + select DRM_BUDDY help Choose this option if you have a recent AMD Radeon graphics card. diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h index acfa207cf970..da12b4ff2e45 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h @@ -30,12 +30,15 @@ #include #include +#include "amdgpu_vram_mgr.h" + /* state back for walking over vram_mgr and gtt_mgr allocations */ struct amdgpu_res_cursor { uint64_t start; uint64_t size; uint64_t remaining; - struct drm_mm_node *node; + void *node; + uint32_t mem_type; }; /** @@ -52,27 +55,63 @@ static inline void amdgpu_res_first(struct ttm_resource *res, uint64_t start, uint64_t size, struct amdgpu_res_cursor *cur) { + struct drm_buddy_block *block; + struct list_head *head, *next; struct drm_mm_node *node; - if (!res || res->mem_type == TTM_PL_SYSTEM) { - cur->start = start; - cur->size = size; - cur->remaining = size; - cur->node = NULL; - WARN_ON(res && start + size > res->num_pages << PAGE_SHIFT); - return; - } + if (!res) + goto err_out; BUG_ON(start + size > res->num_pages << PAGE_SHIFT); - node = to_ttm_range_mgr_node(res)->mm_nodes; - while (start >= node->size << PAGE_SHIFT) - start -= node++->size << PAGE_SHIFT; + cur->mem_type = res->mem_type; + + switch (cur->mem_type) { + case TTM_PL_VRAM: + head = &to_amdgpu_vram_mgr_node(res)->blocks; + + block = list_first_entry_or_null(head, + struct drm_buddy_block, + link); + if (!block) + goto err_out; + + while (start >= amdgpu_node_size(block)) { + start -= amdgpu_node_size(block); + + next = block->link.next; + if (next != head) + block = list_entry(next, struct drm_buddy_block, link); + } + + cur->start = amdgpu_node_start(block) + start; + cur->size = min(amdgpu_node_size(block) - start, size); + cur->remaining = size; + cur->node = block; + break; + case TTM_PL_TT: + node = to_ttm_range_mgr_node(res)->mm_nodes; + while (start >= node->size << PAGE_SHIFT) + start -= node++->size << PAGE_SHIFT; + + cur->start = (node->start << PAGE_SHIFT) + start; + cur->size = min((node->size << PAGE_SHIFT) - start, size); + cur->remaining = size; + cur->node = node; + break; + default: + goto err_out; + } - cur->start = (node->start << PAGE_SHIFT) + start; - cur->size = min((node->size << PAGE_SHIFT) - start, size); + return; + +err_out: + cur->start = start; + cur->size = size; cur->remaining = size; - cur->node = node; + cur->node = NULL; + WARN_ON(res && start + size > res->num_pages << PAGE_SHIFT); + return; } /** @@ -85,7 +124,9 @@ static inline void amdgpu_res_first(struct ttm_resource *res, */ static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size) { - struct drm_mm_node *node = cur->node; + struct drm_buddy_block *block; + struct drm_mm_node *node; + struct list_head *next; BUG_ON(size > cur->remaining); @@ -99,9 +140,27 @@ static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size) return; } - cur->node = ++node; - cur->start = node->start << PAGE_SHIFT; - cur->size = min(node->size << PAGE_SHIFT, cur->remaining); + switch (cur->mem_type) { + case TTM_PL_VRAM: + block = cur->node; + + next = block->link.next; + block = list_entry(next, struct drm_buddy_block, link); + + cur->node = block; + cur->start = amdgpu_node_start(block); + cur->size = min(amdgpu_node_size(block), cur->remaining); + break; + case TTM_PL_TT: + node = cur->node; + + cur->node = ++node; + cur->start = node->start << PAGE_SHIFT; + cur->size = min(node->size << PAGE_SHIFT, cur->remaining); + break; + default: + return; + } } #endif diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h index f8f48be16d80..1a5cbd5ef738 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h @@ -26,6 +26,7 @@ #include #include +#include #include "amdgpu.h" #define AMDGPU_PL_GDS (TTM_PL_PRIV + 0) @@ -40,12 +41,14 @@ struct amdgpu_vram_mgr { struct ttm_resource_manager manager; - struct drm_mm mm; - spinlock_t lock; + struct drm_buddy mm; + /* protects access to buffer objects */ + struct mutex lock; struct list_head reservations_pending; struct list_head reserved_pages; atomic64_t usage; atomic64_t vis_usage; + u64 default_page_size; }; struct amdgpu_gtt_mgr { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 7442095f089c..0b4f0ab8e286 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -32,8 +32,11 @@ #include "atom.h" struct amdgpu_vram_reservation { + u64 start; + u64 size; + unsigned long flags; + struct list_head block; struct list_head node; - struct drm_mm_node mm_node; }; static inline struct amdgpu_vram_mgr * @@ -194,10 +197,10 @@ const struct attribute_group amdgpu_vram_mgr_attr_group = { * Calculate how many bytes of the MM node are inside visible VRAM */ static u64 amdgpu_vram_mgr_vis_size(struct amdgpu_device *adev, - struct drm_mm_node *node) + struct drm_buddy_block *block) { - uint64_t start = node->start << PAGE_SHIFT; - uint64_t end = (node->size + node->start) << PAGE_SHIFT; + u64 start = amdgpu_node_start(block); + u64 end = start + amdgpu_node_size(block); if (start >= adev->gmc.visible_vram_size) return 0; @@ -218,9 +221,9 @@ u64 amdgpu_vram_mgr_bo_visible_size(struct amdgpu_bo *bo) { struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); struct ttm_resource *res = bo->tbo.resource; - unsigned pages = res->num_pages; - struct drm_mm_node *mm; - u64 usage; + struct amdgpu_vram_mgr_node *node = to_amdgpu_vram_mgr_node(res); + struct drm_buddy_block *block; + u64 usage = 0; if (amdgpu_gmc_vram_full_visible(&adev->gmc)) return amdgpu_bo_size(bo); @@ -228,9 +231,8 @@ u64 amdgpu_vram_mgr_bo_visible_size(struct amdgpu_bo *bo) if (res->start >= adev->gmc.visible_vram_size >> PAGE_SHIFT) return 0; - mm = &container_of(res, struct ttm_range_mgr_node, base)->mm_nodes[0]; - for (usage = 0; pages; pages -= mm->size, mm++) - usage += amdgpu_vram_mgr_vis_size(adev, mm); + list_for_each_entry(block, &node->blocks, link) + usage += amdgpu_vram_mgr_vis_size(adev, block); return usage; } @@ -240,21 +242,29 @@ static void amdgpu_vram_mgr_do_reserve(struct ttm_resource_manager *man) { struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); struct amdgpu_device *adev = to_amdgpu_device(mgr); - struct drm_mm *mm = &mgr->mm; + struct drm_buddy *mm = &mgr->mm; struct amdgpu_vram_reservation *rsv, *temp; + struct drm_buddy_block *block; uint64_t vis_usage; list_for_each_entry_safe(rsv, temp, &mgr->reservations_pending, node) { - if (drm_mm_reserve_node(mm, &rsv->mm_node)) + if (drm_buddy_alloc_blocks(mm, rsv->start, rsv->start + rsv->size, + rsv->size, mm->chunk_size, &rsv->block, + rsv->flags)) continue; - dev_dbg(adev->dev, "Reservation 0x%llx - %lld, Succeeded\n", - rsv->mm_node.start, rsv->mm_node.size); - - vis_usage = amdgpu_vram_mgr_vis_size(adev, &rsv->mm_node); - atomic64_add(vis_usage, &mgr->vis_usage); - atomic64_add(rsv->mm_node.size << PAGE_SHIFT, &mgr->usage); - list_move(&rsv->node, &mgr->reserved_pages); + block = list_first_entry_or_null(&rsv->block, + struct drm_buddy_block, + link); + if (block) { + dev_dbg(adev->dev, "Reservation 0x%llx - %lld, Succeeded\n", + rsv->start, rsv->size); + + vis_usage = amdgpu_vram_mgr_vis_size(adev, block); + atomic64_add(vis_usage, &mgr->vis_usage); + atomic64_add(rsv->size, &mgr->usage); + list_move(&rsv->node, &mgr->reserved_pages); + } } } @@ -277,13 +287,16 @@ int amdgpu_vram_mgr_reserve_range(struct amdgpu_vram_mgr *mgr, return -ENOMEM; INIT_LIST_HEAD(&rsv->node); - rsv->mm_node.start = start >> PAGE_SHIFT; - rsv->mm_node.size = size >> PAGE_SHIFT; + INIT_LIST_HEAD(&rsv->block); + + rsv->start = start; + rsv->size = size; + rsv->flags |= DRM_BUDDY_RANGE_ALLOCATION; - spin_lock(&mgr->lock); - list_add_tail(&mgr->reservations_pending, &rsv->node); + mutex_lock(&mgr->lock); + list_add_tail(&rsv->node, &mgr->reservations_pending); amdgpu_vram_mgr_do_reserve(&mgr->manager); - spin_unlock(&mgr->lock); + mutex_unlock(&mgr->lock); return 0; } @@ -305,19 +318,19 @@ int amdgpu_vram_mgr_query_page_status(struct amdgpu_vram_mgr *mgr, struct amdgpu_vram_reservation *rsv; int ret; - spin_lock(&mgr->lock); + mutex_lock(&mgr->lock); list_for_each_entry(rsv, &mgr->reservations_pending, node) { - if ((rsv->mm_node.start <= start) && - (start < (rsv->mm_node.start + rsv->mm_node.size))) { + if (rsv->start <= start && + (start < (rsv->start + rsv->size))) { ret = -EBUSY; goto out; } } list_for_each_entry(rsv, &mgr->reserved_pages, node) { - if ((rsv->mm_node.start <= start) && - (start < (rsv->mm_node.start + rsv->mm_node.size))) { + if (rsv->start <= start && + (start < (rsv->start + rsv->size))) { ret = 0; goto out; } @@ -325,32 +338,10 @@ int amdgpu_vram_mgr_query_page_status(struct amdgpu_vram_mgr *mgr, ret = -ENOENT; out: - spin_unlock(&mgr->lock); + mutex_unlock(&mgr->lock); return ret; } -/** - * amdgpu_vram_mgr_virt_start - update virtual start address - * - * @mem: ttm_resource to update - * @node: just allocated node - * - * Calculate a virtual BO start address to easily check if everything is CPU - * accessible. - */ -static void amdgpu_vram_mgr_virt_start(struct ttm_resource *mem, - struct drm_mm_node *node) -{ - unsigned long start; - - start = node->start + node->size; - if (start > mem->num_pages) - start -= mem->num_pages; - else - start = 0; - mem->start = max(mem->start, start); -} - /** * amdgpu_vram_mgr_new - allocate new ranges * @@ -366,13 +357,13 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, const struct ttm_place *place, struct ttm_resource **res) { - unsigned long lpfn, num_nodes, pages_per_node, pages_left, pages; + unsigned long lpfn, pages_per_node, pages_left, pages, n_pages; + u64 vis_usage = 0, mem_bytes, max_bytes, min_page_size; struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); struct amdgpu_device *adev = to_amdgpu_device(mgr); - uint64_t vis_usage = 0, mem_bytes, max_bytes; - struct ttm_range_mgr_node *node; - struct drm_mm *mm = &mgr->mm; - enum drm_mm_insert_mode mode; + struct amdgpu_vram_mgr_node *node; + struct drm_buddy *mm = &mgr->mm; + struct drm_buddy_block *block; unsigned i; int r; @@ -391,10 +382,9 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, goto error_sub; } - if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) pages_per_node = ~0ul; - num_nodes = 1; - } else { + else { #ifdef CONFIG_TRANSPARENT_HUGEPAGE pages_per_node = HPAGE_PMD_NR; #else @@ -403,11 +393,9 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, #endif pages_per_node = max_t(uint32_t, pages_per_node, tbo->page_alignment); - num_nodes = DIV_ROUND_UP_ULL(PFN_UP(mem_bytes), pages_per_node); } - node = kvmalloc(struct_size(node, mm_nodes, num_nodes), - GFP_KERNEL | __GFP_ZERO); + node = kzalloc(sizeof(*node), GFP_KERNEL); if (!node) { r = -ENOMEM; goto error_sub; @@ -415,9 +403,17 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, ttm_resource_init(tbo, place, &node->base); - mode = DRM_MM_INSERT_BEST; + INIT_LIST_HEAD(&node->blocks); + if (place->flags & TTM_PL_FLAG_TOPDOWN) - mode = DRM_MM_INSERT_HIGH; + node->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION; + + if (place->fpfn || lpfn != man->size) + /* Allocate blocks in desired range */ + node->flags |= DRM_BUDDY_RANGE_ALLOCATION; + + min_page_size = mgr->default_page_size; + BUG_ON(min_page_size < mm->chunk_size); pages_left = node->base.num_pages; @@ -425,36 +421,61 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, pages = min(pages_left, 2UL << (30 - PAGE_SHIFT)); i = 0; - spin_lock(&mgr->lock); while (pages_left) { - uint32_t alignment = tbo->page_alignment; - if (pages >= pages_per_node) - alignment = pages_per_node; - - r = drm_mm_insert_node_in_range(mm, &node->mm_nodes[i], pages, - alignment, 0, place->fpfn, - lpfn, mode); - if (unlikely(r)) { - if (pages > pages_per_node) { - if (is_power_of_2(pages)) - pages = pages / 2; - else - pages = rounddown_pow_of_two(pages); - continue; - } - goto error_free; + pages = pages_per_node; + + n_pages = pages; + + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { + n_pages = roundup_pow_of_two(n_pages); + min_page_size = (u64)n_pages << PAGE_SHIFT; + + if (n_pages > lpfn) + lpfn = n_pages; } - vis_usage += amdgpu_vram_mgr_vis_size(adev, &node->mm_nodes[i]); - amdgpu_vram_mgr_virt_start(&node->base, &node->mm_nodes[i]); + mutex_lock(&mgr->lock); + r = drm_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT, + (u64)lpfn << PAGE_SHIFT, + (u64)n_pages << PAGE_SHIFT, + min_page_size, + &node->blocks, + node->flags); + mutex_unlock(&mgr->lock); + if (unlikely(r)) + goto error_free_blocks; + pages_left -= pages; ++i; if (pages > pages_left) pages = pages_left; } - spin_unlock(&mgr->lock); + + /* Free unused pages for contiguous allocation */ + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { + u64 actual_size = (u64)node->base.num_pages << PAGE_SHIFT; + + mutex_lock(&mgr->lock); + drm_buddy_block_trim(mm, + actual_size, + &node->blocks); + mutex_unlock(&mgr->lock); + } + + list_for_each_entry(block, &node->blocks, link) + vis_usage += amdgpu_vram_mgr_vis_size(adev, block); + + block = list_first_entry_or_null(&node->blocks, + struct drm_buddy_block, + link); + if (!block) { + r = -ENOENT; + goto error_free_res; + } + + node->base.start = amdgpu_node_start(block) >> PAGE_SHIFT; if (i == 1) node->base.placement |= TTM_PL_FLAG_CONTIGUOUS; @@ -468,13 +489,13 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, *res = &node->base; return 0; -error_free: - while (i--) - drm_mm_remove_node(&node->mm_nodes[i]); - spin_unlock(&mgr->lock); +error_free_blocks: + mutex_lock(&mgr->lock); + drm_buddy_free_list(mm, &node->blocks); + mutex_unlock(&mgr->lock); +error_free_res: ttm_resource_fini(man, &node->base); - kvfree(node); - + kfree(node); error_sub: atomic64_sub(mem_bytes, &mgr->usage); return r; @@ -491,29 +512,29 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, struct ttm_resource *res) { - struct ttm_range_mgr_node *node = to_ttm_range_mgr_node(res); + struct amdgpu_vram_mgr_node *node = to_amdgpu_vram_mgr_node(res); struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); struct amdgpu_device *adev = to_amdgpu_device(mgr); + struct drm_buddy *mm = &mgr->mm; + struct drm_buddy_block *block; uint64_t usage = 0, vis_usage = 0; - unsigned i, pages; - spin_lock(&mgr->lock); - for (i = 0, pages = res->num_pages; pages; - pages -= node->mm_nodes[i].size, ++i) { - struct drm_mm_node *mm = &node->mm_nodes[i]; - - drm_mm_remove_node(mm); - usage += mm->size << PAGE_SHIFT; - vis_usage += amdgpu_vram_mgr_vis_size(adev, mm); + mutex_lock(&mgr->lock); + list_for_each_entry(block, &node->blocks, link) { + usage += amdgpu_node_size(block); + vis_usage += amdgpu_vram_mgr_vis_size(adev, block); } + amdgpu_vram_mgr_do_reserve(man); - spin_unlock(&mgr->lock); + + drm_buddy_free_list(mm, &node->blocks); + mutex_unlock(&mgr->lock); atomic64_sub(usage, &mgr->usage); atomic64_sub(vis_usage, &mgr->vis_usage); ttm_resource_fini(man, res); - kvfree(node); + kfree(node); } /** @@ -663,10 +684,19 @@ static void amdgpu_vram_mgr_debug(struct ttm_resource_manager *man, struct drm_printer *printer) { struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); + struct drm_buddy *mm = &mgr->mm; + struct drm_buddy_block *block; + + mutex_lock(&mgr->lock); + drm_printf(printer, "default_page_size: %lluKiB\n", + mgr->default_page_size >> 10); - spin_lock(&mgr->lock); - drm_mm_print(&mgr->mm, printer); - spin_unlock(&mgr->lock); + drm_buddy_print(mm, printer); + + drm_printf(printer, "reserved:\n"); + list_for_each_entry(block, &mgr->reserved_pages, link) + drm_buddy_block_print(mm, block, printer); + mutex_unlock(&mgr->lock); drm_printf(printer, "man size:%llu pages, ram usage:%lluMB, vis usage:%lluMB\n", man->size, amdgpu_vram_mgr_usage(mgr) >> 20, @@ -690,16 +720,21 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) { struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr; struct ttm_resource_manager *man = &mgr->manager; + int err; ttm_resource_manager_init(man, &adev->mman.bdev, adev->gmc.real_vram_size >> PAGE_SHIFT); man->func = &amdgpu_vram_mgr_func; - drm_mm_init(&mgr->mm, 0, man->size); - spin_lock_init(&mgr->lock); + err = drm_buddy_init(&mgr->mm, man->size << PAGE_SHIFT, PAGE_SIZE); + if (err) + return err; + + mutex_init(&mgr->lock); INIT_LIST_HEAD(&mgr->reservations_pending); INIT_LIST_HEAD(&mgr->reserved_pages); + mgr->default_page_size = PAGE_SIZE; ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager); ttm_resource_manager_set_used(man, true); @@ -727,16 +762,16 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) if (ret) return; - spin_lock(&mgr->lock); + mutex_lock(&mgr->lock); list_for_each_entry_safe(rsv, temp, &mgr->reservations_pending, node) kfree(rsv); list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, node) { - drm_mm_remove_node(&rsv->mm_node); + drm_buddy_free_list(&mgr->mm, &rsv->block); kfree(rsv); } - drm_mm_takedown(&mgr->mm); - spin_unlock(&mgr->lock); + drm_buddy_fini(&mgr->mm); + mutex_unlock(&mgr->lock); ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL);