From patchwork Tue Apr 30 07:01:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chunming Zhou X-Patchwork-Id: 10922863 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BAB191398 for ; Tue, 30 Apr 2019 07:02:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A9D3F28A03 for ; Tue, 30 Apr 2019 07:02:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9D00928A08; Tue, 30 Apr 2019 07:02:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C5D6728A03 for ; Tue, 30 Apr 2019 07:02:09 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8788F893CD; Tue, 30 Apr 2019 07:02:07 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-eopbgr760074.outbound.protection.outlook.com [40.107.76.74]) by gabe.freedesktop.org (Postfix) with ESMTPS id EA3E3893CD for ; Tue, 30 Apr 2019 07:02:05 +0000 (UTC) Received: from DM3PR12CA0071.namprd12.prod.outlook.com (2603:10b6:0:57::15) by SN1PR12MB0656.namprd12.prod.outlook.com (2a01:111:e400:c428::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1835.15; Tue, 30 Apr 2019 07:02:04 +0000 Received: from CO1NAM03FT029.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e48::205) by DM3PR12CA0071.outlook.office365.com (2603:10b6:0:57::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1835.12 via Frontend Transport; Tue, 30 Apr 2019 07:02:03 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV01.amd.com (165.204.84.17) by CO1NAM03FT029.mail.protection.outlook.com (10.152.80.168) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1835.14 via Frontend Transport; Tue, 30 Apr 2019 07:02:03 +0000 Received: from zhoucm1.amd.com (10.34.1.3) by SATLEXCHOV01.amd.com (10.181.40.71) with Microsoft SMTP Server id 14.3.389.1; Tue, 30 Apr 2019 02:02:01 -0500 From: Chunming Zhou To: , , Subject: [PATCH] drm/ttm: fix busy memory to fail other user v5 Date: Tue, 30 Apr 2019 15:01:51 +0800 Message-ID: <20190430070151.16674-1-david1.zhou@amd.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(39860400002)(136003)(376002)(396003)(346002)(2980300002)(428003)(199004)(189003)(1076003)(8676002)(7696005)(6666004)(126002)(476003)(356004)(486006)(70206006)(5660300002)(70586007)(47776003)(14444005)(53416004)(2201001)(36756003)(478600001)(2616005)(426003)(2906002)(4326008)(72206003)(53936002)(86362001)(68736007)(8936002)(51416003)(50226002)(81156014)(81166006)(50466002)(316002)(186003)(305945005)(26005)(110136005)(77096007)(336012)(16586007)(48376002)(97736004); DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR12MB0656; H:SATLEXCHOV01.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; A:1; MX:1; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 58b6541c-0f8a-4a66-df85-08d6cd39bfd9 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(2017052603328); SRVR:SN1PR12MB0656; X-MS-TrafficTypeDiagnostic: SN1PR12MB0656: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6430; X-Forefront-PRVS: 00235A1EEF X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info: utZSRAcGaZEvXqavMsDJo28KQN2CNlHlQZrPsmctXuTN/Slntkp+2ldIRq/0IMTvBNiXDNVHaj712dXzl+ZFx4dqIfRZXnzIS123X0dE+AV9SXLO87tx+pjGiSDPwLV5hu2MyWJ0gk1Hola3ZVAf7rHFYTXzf0L3TGLyhios+0GtJf+nwgWQ6g9shL7l4qaF62dvfkcTHi3X4KCynyS3QC7j2WOx+q3vT8DwYj+fdWZasp9yI2atLjzDgz6epVfe0knvkniolzkekyyl15ijSNAtS/rfcdq6DO2pjsEhU2KRREQAr5oLHSvt8gKwDY5gKvEpuBnd6yfGryKGzZ0BZe9zlPDVl2weREEP7/dZ950+TNNk/A9V7sYHknOAvrTmm2qH+AMsC/Jp1IX4gO16rlNmgeUD2yw7pkPlPmCWpUE= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Apr 2019 07:02:03.1217 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 58b6541c-0f8a-4a66-df85-08d6cd39bfd9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0656 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xdr4LDXiwLQzRfSpnCAUisLRQjRjcLW+OM1ioz+W7C0=; b=w8LV/J7opQ9TkT4hd6PKRgL6Rl1wIDVImpNZnKZ20R7SN0ndr3o4TOlOeOSOSKDlCSyyPzu8+zpbQnggfIq0rPurru799GQ8ffFV4oNegfkjD7Jw6TVYAkWOlFPUe+nr2UzzD4PZPGA5kv57o0nalDywcs+qh14d0Zy9+XqzGJU= X-Mailman-Original-Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=permerror action=none header.from=amd.com; X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP heavy gpu job could occupy memory long time, which lead other user fail to get memory. basically pick up Christian idea: 1. Reserve the BO in DC using a ww_mutex ticket (trivial). 2. If we then run into this EBUSY condition in TTM check if the BO we need memory for (or rather the ww_mutex of its reservation object) has a ticket assigned. 3. If we have a ticket we grab a reference to the first BO on the LRU, drop the LRU lock and try to grab the reservation lock with the ticket. 4. If getting the reservation lock with the ticket succeeded we check if the BO is still the first one on the LRU in question (the BO could have moved). 5. If the BO is still the first one on the LRU in question we try to evict it as we would evict any other BO. 6. If any of the "If's" above fail we just back off and return -EBUSY. v2: fix some minor check v3: address Christian v2 comments. v4: fix some missing v5: handle first_bo unlock and bo_get/put Change-Id: I21423fb922f885465f13833c41df1e134364a8e7 Signed-off-by: Chunming Zhou --- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 7 +- .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 22 +++-- drivers/gpu/drm/ttm/ttm_bo.c | 81 +++++++++++++++++-- 3 files changed, 99 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index affde72b44db..523773e85284 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -811,7 +811,12 @@ int amdgpu_bo_pin_restricted(struct amdgpu_bo *bo, u32 domain, u64 min_offset, u64 max_offset) { struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); - struct ttm_operation_ctx ctx = { false, false }; + struct ttm_operation_ctx ctx = { + .interruptible = false, + .no_wait_gpu = false, + .resv = bo->tbo.resv, + .flags = 0 + }; int r, i; if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm)) diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index a5cacf846e1b..cc3677c4a4c2 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -4101,6 +4101,9 @@ static int dm_plane_helper_prepare_fb(struct drm_plane *plane, struct amdgpu_device *adev; struct amdgpu_bo *rbo; struct dm_plane_state *dm_plane_state_new, *dm_plane_state_old; + struct list_head list, duplicates; + struct ttm_validate_buffer tv; + struct ww_acquire_ctx ticket; uint64_t tiling_flags; uint32_t domain; int r; @@ -4117,9 +4120,18 @@ static int dm_plane_helper_prepare_fb(struct drm_plane *plane, obj = new_state->fb->obj[0]; rbo = gem_to_amdgpu_bo(obj); adev = amdgpu_ttm_adev(rbo->tbo.bdev); - r = amdgpu_bo_reserve(rbo, false); - if (unlikely(r != 0)) + INIT_LIST_HEAD(&list); + INIT_LIST_HEAD(&duplicates); + + tv.bo = &rbo->tbo; + tv.num_shared = 1; + list_add(&tv.head, &list); + + r = ttm_eu_reserve_buffers(&ticket, &list, false, &duplicates); + if (r) { + dev_err(adev->dev, "fail to reserve bo (%d)\n", r); return r; + } if (plane->type != DRM_PLANE_TYPE_CURSOR) domain = amdgpu_display_supported_domains(adev); @@ -4130,21 +4142,21 @@ static int dm_plane_helper_prepare_fb(struct drm_plane *plane, if (unlikely(r != 0)) { if (r != -ERESTARTSYS) DRM_ERROR("Failed to pin framebuffer with error %d\n", r); - amdgpu_bo_unreserve(rbo); + ttm_eu_backoff_reservation(&ticket, &list); return r; } r = amdgpu_ttm_alloc_gart(&rbo->tbo); if (unlikely(r != 0)) { amdgpu_bo_unpin(rbo); - amdgpu_bo_unreserve(rbo); + ttm_eu_backoff_reservation(&ticket, &list); DRM_ERROR("%p bind failed\n", rbo); return r; } amdgpu_bo_get_tiling_flags(rbo, &tiling_flags); - amdgpu_bo_unreserve(rbo); + ttm_eu_backoff_reservation(&ticket, &list); afb->address = amdgpu_bo_gpu_offset(rbo); diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index 8502b3ed2d88..2c4963e105d9 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -766,11 +766,13 @@ EXPORT_SYMBOL(ttm_bo_eviction_valuable); * b. Otherwise, trylock it. */ static bool ttm_bo_evict_swapout_allowable(struct ttm_buffer_object *bo, - struct ttm_operation_ctx *ctx, bool *locked) + struct ttm_operation_ctx *ctx, bool *locked, bool *busy) { bool ret = false; *locked = false; + if (busy) + *busy = false; if (bo->resv == ctx->resv) { reservation_object_assert_held(bo->resv); if (ctx->flags & TTM_OPT_FLAG_ALLOW_RES_EVICT @@ -779,6 +781,8 @@ static bool ttm_bo_evict_swapout_allowable(struct ttm_buffer_object *bo, } else { *locked = reservation_object_trylock(bo->resv); ret = *locked; + if (!ret && busy) + *busy = true; } return ret; @@ -791,7 +795,7 @@ static int ttm_mem_evict_first(struct ttm_bo_device *bdev, { struct ttm_bo_global *glob = bdev->glob; struct ttm_mem_type_manager *man = &bdev->man[mem_type]; - struct ttm_buffer_object *bo = NULL; + struct ttm_buffer_object *bo = NULL, *first_bo = NULL; bool locked = false; unsigned i; int ret; @@ -799,8 +803,15 @@ static int ttm_mem_evict_first(struct ttm_bo_device *bdev, spin_lock(&glob->lru_lock); for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { list_for_each_entry(bo, &man->lru[i], lru) { - if (!ttm_bo_evict_swapout_allowable(bo, ctx, &locked)) + bool busy = false; + if (!ttm_bo_evict_swapout_allowable(bo, ctx, &locked, + &busy)) { + if (!first_bo && busy) { + ttm_bo_get(bo); + first_bo = bo; + } continue; + } if (place && !bdev->driver->eviction_valuable(bo, place)) { @@ -808,6 +819,7 @@ static int ttm_mem_evict_first(struct ttm_bo_device *bdev, reservation_object_unlock(bo->resv); continue; } + break; } @@ -820,7 +832,65 @@ static int ttm_mem_evict_first(struct ttm_bo_device *bdev, if (!bo) { spin_unlock(&glob->lru_lock); - return -EBUSY; + /* check if other user occupy memory too long time */ + if (!first_bo || !ctx || !ctx->resv || !ctx->resv->lock.ctx) { + if (first_bo) + ttm_bo_put(first_bo); + return -EBUSY; + } + if (ctx->interruptible) + ret = ww_mutex_lock_interruptible(&first_bo->resv->lock, + ctx->resv->lock.ctx); + else + ret = ww_mutex_lock(&first_bo->resv->lock, ctx->resv->lock.ctx); + if (ret) { + ttm_bo_put(first_bo); + return ret; + } + spin_lock(&glob->lru_lock); + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + /* previous busy resv lock is held by above, idle now, + * so let them evictable. + */ + struct ttm_operation_ctx busy_ctx = { + .interruptible = ctx->interruptible, + .no_wait_gpu = ctx->no_wait_gpu, + .resv = first_bo->resv, + .flags = TTM_OPT_FLAG_ALLOW_RES_EVICT + }; + list_for_each_entry(bo, &man->lru[i], lru) { + if (!ttm_bo_evict_swapout_allowable(bo, + &busy_ctx, + &locked, + NULL)) + continue; + + if (place && !bdev->driver->eviction_valuable(bo, + place)) { + if (locked) + reservation_object_unlock(bo->resv); + continue; + } + break; + } + /* If the inner loop terminated early, we have our candidate */ + if (&bo->lru != &man->lru[i]) + break; + bo = NULL; + } + if (bo && (bo->resv == first_bo->resv)) + locked = true; + else if (bo) + ww_mutex_unlock(&first_bo->resv->lock); + ttm_bo_put(first_bo); + first_bo = NULL; + if (!bo) { + spin_unlock(&glob->lru_lock); + return -EBUSY; + } + } else { + if (first_bo) + ttm_bo_put(first_bo); } kref_get(&bo->list_kref); @@ -1784,7 +1854,8 @@ int ttm_bo_swapout(struct ttm_bo_global *glob, struct ttm_operation_ctx *ctx) spin_lock(&glob->lru_lock); for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { list_for_each_entry(bo, &glob->swap_lru[i], swap) { - if (ttm_bo_evict_swapout_allowable(bo, ctx, &locked)) { + if (ttm_bo_evict_swapout_allowable(bo, ctx, &locked, + NULL)) { ret = 0; break; }