From patchwork Tue Nov 29 11:30:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pan, Xinhui" X-Patchwork-Id: 13058491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9DC18C46467 for ; Tue, 29 Nov 2022 11:30:41 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DF48710E3B2; Tue, 29 Nov 2022 11:30:39 +0000 (UTC) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on20625.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eb2::625]) by gabe.freedesktop.org (Postfix) with ESMTPS id BBBD310E3B1; Tue, 29 Nov 2022 11:30:33 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lmwmmcuKcaNv1BtT55V3lOgucftAaDNK35NkhXdqJbTwlf2DOPVUDpJlAnVhBbwL4kwoakGi7JXdhkNdlgz9S4ojNlWOTQCf3o0ckJ34ss/JAi1DpJRsizsWyscGnMOnV77VRcWpJlMj/zToj1Le3Z0mNTWAMbSrdY5uPKRPherBnmyROBOUrpr+rsBDJtMzYG5M1QEjWZnDI5ZevBFwRf7rV6SFS/oNKBG9Fakx+A0Q6SBEKMLT2j5XSGwgtowtoPEf+nrD627F2+ReJbuUju1wh3JA8VA9I6/Pn4jIy93xzfn4DNxhA0qgHO6PUs5REniD8eKTSSYWujM9TFCY/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ezmSncooZomdRmvkuyRj+e7licZUwsmTIVG2/JOMCZg=; b=c8GevYnTiULVEKc3QBuGET58LoljQB82yBA3L5c7CZs7MieXL7OqnTeGOLJUfA6V9YZ/9vJzERZ+TqSKW1YT9VK4Sx/aqbXweNi3MjiJ+LreW2Glggkb7ILxi1CYSmJTfwJYCiL80+eUBzjX1e+J1SZ3ToBWPKclmChuDFGRz/XrkuNTeKgo9APgg+VAbDN+S9IIdvA0fbdfqujOXzSzmgC4eb6lpluj4Q1VW59cEGFowiw0U3T8e6UCnJmcJsFTJI5XIBJik3oQRmgruMbFrCjDjGAkZQztiFtLLVCA13E4QcONj1cTMDbzi1JzkJ51GSrcYvR8feBeBzRiFayowQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ezmSncooZomdRmvkuyRj+e7licZUwsmTIVG2/JOMCZg=; b=CGS2lKx0yDBTnhvhdGk7bxkjKLV+mEMES8R9sRx5sCZNq9EgllDluWKsY7PujYBj95ncO1UDpKeAI+voMPblJBB5oXnxUCONWHcTo/MgZ33HclNcyAxfDawm0hkfTCy2siyfZ8icgrGPZNa50LKkm32J8WT2/ruzXXJ106m4nhM= Received: from DM4PR12MB5165.namprd12.prod.outlook.com (2603:10b6:5:394::9) by SN7PR12MB7132.namprd12.prod.outlook.com (2603:10b6:806:2a4::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Tue, 29 Nov 2022 11:30:29 +0000 Received: from DM4PR12MB5165.namprd12.prod.outlook.com ([fe80::af94:f2ad:209b:c565]) by DM4PR12MB5165.namprd12.prod.outlook.com ([fe80::af94:f2ad:209b:c565%4]) with mapi id 15.20.5857.023; Tue, 29 Nov 2022 11:30:29 +0000 From: "Pan, Xinhui" To: "amd-gfx@lists.freedesktop.org" Subject: =?eucgb2312_cn?b?u9i4tDogW1BBVENIIHY0XSBkcm06IE9wdGltaXNlIGZvciBjb250aW51?= =?eucgb2312_cn?b?b3VzIG1lbW9yeSBhbGxvY2F0aW9u?= Thread-Topic: [PATCH v4] drm: Optimise for continuous memory allocation Thread-Index: AQHZA+FYf6F+KXzzGkaj7rVsGUeL5q5VwyZh Date: Tue, 29 Nov 2022 11:30:29 +0000 Message-ID: References: <20221129105655.125571-1-xinhui.pan@amd.com> In-Reply-To: <20221129105655.125571-1-xinhui.pan@amd.com> Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_Enabled=True; MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_SiteId=3dd8961f-e488-4e60-8e11-a82d994e183d; MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_SetDate=2022-11-29T11:30:26.929Z; MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_Name=General; MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_ContentBits=0; MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_Method=Standard; authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM4PR12MB5165:EE_|SN7PR12MB7132:EE_ x-ms-office365-filtering-correlation-id: 91791c7e-604e-4e00-a55f-08dad1fd1e62 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: EY1GnQYO9k/CRVDTbnpFg9KwKlJnb52l8c3uj+NQFfeVZRnF7XKv7TuwHuEdYnWiIg02oAuozXYm6iYZCesfp9UwZGlN7RldtW2sYm5zMO7SXJKt7DFHqwjgsKGVCG6eNTJLKARdJ6e1Z4HN9JZ14TFQWYUxG0/9KZA1SNTuz1TdiX54BHw09Zn/LeKSxGWEovSf8B0nUT5sXHIDlScixtxIBhEjlotb10fU8+XfUlR3w2a3RhdoRfLP48GCOmtrDKJiMxZypXEgxbnw8HNpEiwiQt4XE8zBKnPZkSVXGKhukdPQD0A3bSYkxno+qkfCqrLSH14LrzHHuNgIrhYZ6scek7g7FZ0pjf30Fneub5MGmSUiXd4PJpEkqSY8baKFIUCw/EgkR73G78lG1GoJ1Hqzz0ZADqBkAz0S0USEuLDMQ9V7RNB/SXuykDcmJoHP5z4+Mo8qzR4o8GldmmgNgxi8Eau20t5hqyLKSp80eSsjJSsgJ5cASJwOxzn3WqGdg5dV6MVOn9hoikhiOgDNwJKR9D8W3eI4202GpU7xQg/vXKglPW3LpEfGj6g3WmKtg5/8w6g2vd7Ko/0s3Mw3gW3C+NL5KDdrHBEPVjyjsHTdyYkR5yRT3xLpNkiwpn/J/Tzqvks4LBN1Y3afbb8HOy9HCE0wGhTti6thCXQSrUzIQY85b0bIOye96sbvznwuDvtkscKzQ70/KwN6MPLFRg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR12MB5165.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(136003)(366004)(396003)(39860400002)(376002)(346002)(451199015)(54906003)(6916009)(478600001)(9686003)(66899015)(6506007)(7696005)(66946007)(316002)(91956017)(76116006)(71200400001)(224303003)(8936002)(52536014)(66476007)(64756008)(4326008)(66446008)(66556008)(186003)(41300700001)(5660300002)(2906002)(83380400001)(122000001)(38100700002)(55016003)(33656002)(38070700005)(86362001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?eucgb2312_cn?b?MjlaOGdHMnIzUi9KeE1WaDly?= =?eucgb2312_cn?b?VFJaSXhPa0RMMkZwWlpQdGJPNWI4d3hlNVl6dTN2enpIbEZxMHZsaFlaelkyQVdh?= =?eucgb2312_cn?b?aTFHT0ljRjE2RmdzQVJsM0RxakNOajNrNkNvVzZMS216WlJoQS9wb0dTS2pMdDRm?= =?eucgb2312_cn?b?Y2o5eGFzeFRDYXNucEh0QVppNFRXYmFkUDMyMFVRS1ZGN0NxT3JCb3RiTUhFRTV5?= =?eucgb2312_cn?b?a2w1VHZ2YWUwWXdMRE1TcE1SYWE4dTJEYTNjVFd5aGFEWGMrUkhlL1JzcjY4cXRD?= =?eucgb2312_cn?b?N1BWRVJHMVRUbC9yYUtZVWR2dXo0czFRUFRYY0tMck0yS3c5NTRFQTQ1M2FpMGZn?= =?eucgb2312_cn?b?a3ZDeGpaM3FxcVNScHFaS0hWNlh0UTNYQmN6UjJnVTd2RnhQemN3ZUlSdlFjYnJM?= =?eucgb2312_cn?b?MnVLbEs1ZkJpaWZLelVzL3ZhQ21QQ3o0ZUJjTkZLdkQ2bmpqYWEwQ0lBUUV6UUNX?= =?eucgb2312_cn?b?aUkyMnF1Q2xjRmtEZ1JxY05VbElld3hWTWJPQjdFazFlR3M5cjRERlZmUnc2aThS?= =?eucgb2312_cn?b?c0FYS05PVEdsbmRBOHJ0VXMrZDB5MngrRHRwSldJeWxOcVpyUmFkM2dHZWd6RDBX?= =?eucgb2312_cn?b?aXF5ZGxUR0t3cEtMK29aaUJXMVJrTW93NlhmenFabEEvT1hkbjJSeExxMVpGMVdX?= =?eucgb2312_cn?b?M1N5TzQ1d3RCZ3l5UFFhc1JKYU1SYWJLbjBXcGkvQWhPaHhMZHFPZm84ZTJyakMw?= =?eucgb2312_cn?b?cWxxMkoySGJ0YytWbW16d21vL1VQRnNnZVViajdDZC9HMVpHM2dzbDVCejQxd2NU?= =?eucgb2312_cn?b?Q1IvVUEyb25LQ3pHd2V4RE9iUWNwRHJoR1krZ2ZnaWo1RE9pdng5ZTZVL0MrVGoz?= =?eucgb2312_cn?b?UGVXRUlYZmtXdEZnSjN1OGJYajZGZ2tFVE9ZNW9Jbk10ZVVGZFcrV0xTcUdsT01E?= =?eucgb2312_cn?b?Ti9uRmNLcUdTaGdpcC8zL0wrUGd2MXZteTJHNStQWCtNNHpUWVVIYk9wTHQyelZV?= =?eucgb2312_cn?b?ei9MaVB4amcwQWNRS3BTYjFDeW0wNHFlV0N6NGFqc0pzQi82M3ZIcWpscGR6SGN3?= =?eucgb2312_cn?b?UjVRc0NFV2dnVGdYbks1djdSL3RqVW5ReE44YVJsMDVXclB0SlNhdEUveFBPZFFE?= =?eucgb2312_cn?b?ZGpWVGZ3RFRra1A5YXI2OThLaUw1QzNsT0J1NkxDNnZSVFIxbHBSbysyUWhqYldO?= =?eucgb2312_cn?b?SVFnWm5lVmYzYk5waGFoNVlRNXZYWXpmOXdNcTRweTRVcVRndUdNdGZxN08ySkZU?= =?eucgb2312_cn?b?YzNXaUZCd21aSUhOWnN5aE1OcitwY2NFMms3a05RamptbDNEVEVUbzNwY3NUSUsv?= =?eucgb2312_cn?b?NWJ4OTZyc0UrUmVQb2xtV3N5dXBuaFpIWC9kOXZLSzRlSXVIc0ExQ3FWNDhDSVE2?= =?eucgb2312_cn?b?Z2dhalVsLzdMV2d6YjFPQjdNUzU5c0RZV0UzeUczcVdPOEJWaDVaVThuUnpCWXhK?= =?eucgb2312_cn?b?QkFZUWRpZ1hIYWFRZ3FXa1g5eGVQcTZDcDNZdVRYWWpqRlNxVGgrQnZkbGhlbHky?= =?eucgb2312_cn?b?WVkxZkYxcS9LLzZWbFJncFJNWngvNTJKQjhLVXVEV0dENHpTMlN6K1V3cTBYVDBk?= =?eucgb2312_cn?b?cHhkUzladjlycGdXQVMrY2tBRjF6NHhFd0ZuMVFoTVd6UU1MdGZHbldFUjd3eFkz?= =?eucgb2312_cn?b?Smt0b0FSbFJOZnVsRUVabzJnSEkycGJOQzhHWFhOSTNRREpKeXZOWk5ROVIrSjdn?= =?eucgb2312_cn?b?NkdOU3pWSjVvZVdZZzlLWUFRdlo0ekZxWlVSTi83b0lXV2xiRTVia2ZWZ1psdFJr?= =?eucgb2312_cn?b?d0I2QWVPQ0I2SHFGMHl1cXdoeFpQYnRIV1grN01PemxpTlBud2txRXllUjc5OUVl?= =?eucgb2312_cn?b?ZGxVTlMzWVdvVTdzTDVXVjNUZ3RPc1ViZ0tBL2tpL0pMU2l0Z2lBaXU4dW55OFE0?= =?eucgb2312_cn?b?dUo0VWhmaThJVUlzV0NmUmVQSGlhLzkwNHdtVitmRGxBc3VzME1sMmk4OUQvKzl1?= =?eucgb2312_cn?b?QXROcld6TE54WkNxNHVFMDJEQWU0bm96ejBnYmJWVHRmV21sRFhhUXlYNnNXUnlO?= =?eucgb2312_cn?b?ZUI2L21rV3NzMmpLRytHdzVCNkdMckhFaG9TYWlKMk1xYVB1bFhWbXZFZlI3MXF0?= =?eucgb2312_cn?b?WExhZFFVMFVNQXF6Y3Q0NHhMTEM3MWJQUkFJWXB4bkUxSFlRSnc1WHE1RUkyaTZD?= =?eucgb2312_cn?b?eEwwZ0dqbHlGakErVlMzOHhlSzNGTFJkNXlLTExnM0QrRVgxWWkyVzA9?= MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5165.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 91791c7e-604e-4e00-a55f-08dad1fd1e62 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Nov 2022 11:30:29.2902 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 0J87ne0JapJ9ini61mmxcX6El5dVhrGG/2lla6mvLhbQonT73/bK76yC4rWX/YMo X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7132 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Paneer Selvam, Arunpravin" , "intel-gfx@lists.freedesktop.org" , "linux-kernel@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , "matthew.auld@intel.com" , "Koenig, Christian" Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" [AMD Official Use Only - General] In one ROCM + gdm restart test, find_continuous_blocks() succeed with ratio 35%. the cod coverage report is below. 772 3998 : if (order-- == min_order) { 773 352 : if (!(flags & DRM_BUDDY_RANGE_ALLOCATION) && 774 352 : min_order != 0 && pages == BIT(order + 1)) { 775 79 : block = find_continuous_blocks(mm, 776 : order, 777 : flags, 778 : &rblock); 779 79 : if (block) 780 : break; 781 : } 782 300 : err = -ENOSPC; 783 300 : goto err_free; thanks xinhui diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 11bb59399471..8edafb99b02c 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -80,6 +80,7 @@ int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size) { unsigned int i; u64 offset; + LIST_HEAD(leaf); if (size < chunk_size) return -EINVAL; @@ -136,6 +137,7 @@ int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size) goto out_free_roots; mark_free(mm, root); + list_add_tail(&root->leaf_link, &leaf); BUG_ON(i > mm->max_order); BUG_ON(drm_buddy_block_size(mm, root) < chunk_size); @@ -147,6 +149,7 @@ int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size) i++; } while (size); + list_del(&leaf); return 0; out_free_roots: @@ -205,6 +208,9 @@ static int split_block(struct drm_buddy *mm, mark_free(mm, block->left); mark_free(mm, block->right); + list_add(&block->right->leaf_link, &block->leaf_link); + list_add(&block->left->leaf_link, &block->leaf_link); + list_del(&block->leaf_link); mark_split(block); return 0; @@ -256,6 +262,9 @@ static void __drm_buddy_free(struct drm_buddy *mm, break; list_del(&buddy->link); + list_add(&parent->leaf_link, &block->leaf_link); + list_del(&buddy->leaf_link); + list_del(&block->leaf_link); drm_block_free(mm, block); drm_block_free(mm, buddy); @@ -386,6 +395,78 @@ alloc_range_bias(struct drm_buddy *mm, return ERR_PTR(err); } +static struct drm_buddy_block * +find_continuous_blocks(struct drm_buddy *mm, + int order, + unsigned long flags, + struct drm_buddy_block **rblock) +{ + struct list_head *head = &mm->free_list[order]; + struct drm_buddy_block *free_block, *max_block = NULL, *end, *begin; + u64 pages = BIT(order + 1); + u64 cur_pages; + + list_for_each_entry(free_block, head, link) { + if (max_block) { + if (!(flags & DRM_BUDDY_TOPDOWN_ALLOCATION)) + break; + + if (drm_buddy_block_offset(free_block) < + drm_buddy_block_offset(max_block)) + continue; + } + + cur_pages = BIT(order); + begin = end = free_block; + while (true) { + struct drm_buddy_block *prev, *next; + int prev_order, next_order; + + prev = list_prev_entry(begin, leaf_link); + if (!drm_buddy_block_is_free(prev) || + drm_buddy_block_offset(prev) > + drm_buddy_block_offset(begin)) { + prev = NULL; + } + next = list_next_entry(end, leaf_link); + if (!drm_buddy_block_is_free(next) || + drm_buddy_block_offset(next) < + drm_buddy_block_offset(end)) { + next = NULL; + } + if (!prev && !next) + break; + + prev_order = prev ? drm_buddy_block_order(prev) : -1; + next_order = next ? drm_buddy_block_order(next) : -1; + if (next_order >= prev_order) { + BUG_ON(drm_buddy_block_offset(end) + + drm_buddy_block_size(mm, end) != + drm_buddy_block_offset(next)); + end = next; + cur_pages += BIT(drm_buddy_block_order(next)); + } + if (prev_order >= next_order) { + BUG_ON(drm_buddy_block_offset(prev) + + drm_buddy_block_size(mm, prev) != + drm_buddy_block_offset(begin)); + begin = prev; + cur_pages += BIT(drm_buddy_block_order(prev)); + } + if (pages == cur_pages) + break; + BUG_ON(pages < cur_pages); + } + + if (pages > cur_pages) + continue; + + *rblock = end; + max_block = begin; + } + return max_block; +} + static struct drm_buddy_block * get_maxblock(struct list_head *head) { @@ -637,7 +718,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, struct list_head *blocks, unsigned long flags) { - struct drm_buddy_block *block = NULL; + struct drm_buddy_block *block = NULL, *rblock = NULL; unsigned int min_order, order; unsigned long pages; LIST_HEAD(allocated); @@ -689,17 +770,30 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, break; if (order-- == min_order) { + if (!(flags & DRM_BUDDY_RANGE_ALLOCATION) && + min_order != 0 && pages == BIT(order + 1)) { + block = find_continuous_blocks(mm, + order, + flags, + &rblock); + if (block) + break; + } err = -ENOSPC; goto err_free; } } while (1); - mark_allocated(block); - mm->avail -= drm_buddy_block_size(mm, block); - kmemleak_update_trace(block); - list_add_tail(&block->link, &allocated); - - pages -= BIT(order); + do { + mark_allocated(block); + mm->avail -= drm_buddy_block_size(mm, block); + kmemleak_update_trace(block); + list_add_tail(&block->link, &allocated); + pages -= BIT(drm_buddy_block_order(block)); + if (block == rblock || !rblock) + break; + block = list_next_entry(block, leaf_link); + } while (true); if (!pages) break; diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index 572077ff8ae7..c5437bd4f4f3 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -50,6 +50,7 @@ struct drm_buddy_block { */ struct list_head link; struct list_head tmp_link; + struct list_head leaf_link; }; /* Order-zero must be at least PAGE_SIZE */